Jessi Brooks on Why AI Got KYC'd Like Crypto Did: DEX in the City

Transcript

Finally, my time to shine, letting me go first with my AI update. So, essentially what happened this week is the federal government bricked an AI model. And this happened on Friday afternoon, but the saga has continued into even this morning. So, in case you missed it, which I'm sure you didn't if you watch any news, the government forced Anthropic to take Fable 5 and Mythos 5, which are related models, offline globally. And to me, that's not even the most staggering part because although this started as a cyber risk story, it turned into an export control story and then in my mind a KYC story, which I'll get to, know your customer, which we've talked about, and then devolved into

this crazy personality clash between the admin and a company that may have committed the cardinal sin of having like the wrong political vibes or at least the wrong messaging or messenger. And to me, this is all like not just fun and games for crypto who can like watch from the sideline because we need to be thinking about the implications as an industry. This is not just an AI story. It's a censorship story or at least a chokepoint story and we know lots about choke points. You know, I think V talked a long time a few episodes ago about Operation Chokepoint 2.0 know and it's relevant to bring that context into this because what happened there in many ways

is happening to AI and model access and so we shouldn't just be sitting here being like crap I loved Fable I can't access anymore we should be worrying if the government can use a secret process to shut down a frontier AI model without transparency notice and comment any real insight then we all should be a little bit worried So with that grounding context, I just want to break down a little bit of what happened for those who weren't paying attention every single second who maybe have a life on the weekends. So Anthropic releases Fable, Fable 5, and it's a more secure version of Bithos, which we talked about, the sort of crazy scary cyber security um uh struggling model. And I

don't know if y'all were able to try it in the few days it was out, but it was cool and it showed an improvement in the model, but nothing like too scary. But regardless, just a few days later after it was released, it was disabled on Friday because the admin said you need to disable it essentially. But we'll get into the details there. Why? Well, we're still figuring out why. And there are facts and there are rumors and I'm going to try and separate them here, but they're a little bit intermixed. So according to some reporting, Amazon, which is an investor in Anthropic, but also a big competitor trying to get ahead in AI, raised concerns with the White House that Fable's guard rails could be bypassed. Now, Anthropic and many cyber security experts are saying

the government and Amazon don't know what they're talking about. They're not understanding the bypass. This exists in other models. And so this is a very technical argument. I won't go too deep, but there is a real true argument about whether this bypass is real or not or whether anyone understands it and we don't really know because the government has not been transparent. >> And Jesse, really quick before you go on, can you explain more about why it would be so bad if someone could bypass this? Like I understand this, but I want you to illustrate this to our listeners because I don't think people really grasp how serious all of this is.

>> Yeah. So, we talked about Mythos in detail on a different episode. Mythos is the model that Anthropic said, "This is too scary. We're not going to release it. Instead, we're going to give it to a glasswing group of companies um and let them make sure that their systems are ready for this." The reason why people were so afraid of Mythos is the potential like cyber security risks of it. So if like mythos was enabled for everybody including like illicit actors, they might be able to hack into a bank is sort of like one of the big concerns that was out there or really everything government >> or the federal government and the huge repository of >> sort of happening anyway. Yeah. But yes,

the like financial institutions were prioritized here as well as the government getting access to it. Now, Anthropic spent time trying to build a more secure version of mythos. And how that worked is they put in all these restrictions. So, if you asked um Fable about building a medical like a bio medical weapon or building some sort of toxin to expose to the world, they would say, "Fuck no, we were not answering that even though we're an AI." And there was like no good way to get around that.

And in fact they made it so extreme that if you asked anything about like chemical compounds they would say no. So like some people were complaining that like for their kind of work fable wasn't viable. Now what has maybe happened but this is pure speculation is that Amazon or some other entity was able to get past the question of let me ask you about this chemical compound. But if you do that with any of the models, you could potentially get information. So anthropics like you're not understanding this. And something that's interesting to note here, and once again, we don't have all the details, but all the narrative of this story from the White

House and admin and the actors involved commenting on it publicly are not technical folks. And there is no evidence that they brought in national security experts or technical folks into the conversation. Plus, because other models that are open sourced outside the United States that like maybe hackers do have access to already have like these bypasses enabled, we are potentially cutting our knees out, you know, um by, you know, making it a lot harder to enable cyber security here. So, that would be how I would answer that. Any questions or you want me to keep going?

>> Oh, keep going. All of this is giving me enemy of the state vibes. Does anyone remember that Will Smith movie? It's a really good movie. >> Is it still good though? >> It What >> is it still good or was it just like >> That's a great I need to rewatch. But I feel like we talk about Iroot all the time. Another, you know, kind of dystopian Will Smith movie, but we How have we not talked about Enemy of the State, which is about data and surveillance and please continue? >> Yes. No, totally. Okay. So, government says you got to shut this thing down essentially. But what they actually said is that foreign nationals, not just people outside the United States, but foreign nationals cannot access Fable 5.

And that includes foreign nationals that work for Anthropic. And obviously Anthropic employees are not all US citizens. So essentially, Anthropic said, "Well, we got to cut it off for everybody because Anthropic and no AI system has KYC." But maybe they do now, right? Because maybe this all is just the administration inventing KYC for AI and for all of tech. And I just want us to sit with that for a second because crypto and the industry has spent a decade plus fighting against rules that didn't make sense. I'm not saying all of them don't make sense, but the ones that don't make sense that you should have to show an ID or a passport to open a

wallet because this limits financial access and freedom. But now the government is putting AI and KYC AML rules in some ways into an expansion mode where it's touching not just financial services but maybe AI or maybe other tech and how should we be thinking about that and maybe we're just going in the wrong direction now on certain things when it comes to KYC without a discernable process. And I may sound biased here but the truth is anthropic is not perfect. These frontier models are scary, but to me it's like, is it too much to ask for a process that's transparent, fair, clear, and grounded in technical facts before we're just

crying for more KYC? And I'm going to like give a hot take here. This reminds me a little bit of the Gendler Ariel because then the complaint was like no clear rules, shifting standards. You find out where you cross the line after the shutdown order arrives, you get a subpoena. And now we're taking that and layer on scary ass amazing transformative technology and then claims of political bias and censorship. Although that might be part of it, but it's unclear if that's true because there's bipartisan push back on this. So it is time for crypto to be taking a stand here because this is not just anthropic. This is not just AI.

This is terrifying for all tech and builders because if the government has real evidence of national security threat like show it, create a process, build a rule. Instead, right now we're like living in this muddy space where we don't even know what happens. And there's conversations that Fable is going to come back. But we won't even really know why this all happened at the end of the day. And right this weekend it was anthropic but next weekend it could be something totally different that affects our industry or affects tech in ways that we don't really want. >> So are is the government claiming that they can't say why for like national security reasons?

>> No. >> Right. Which is what they do sometimes. Right. They're not really, but as many people who follow this current administration are aware, the narratives are a little bit all over the place because they're saying it has nothing to do with political bias, nothing is to do with anthropics, like Department of War issue that we talked about, our Department of Defense. Both of those names still exist um from, you know, a month or two ago. But then Pete Hexath tweeted out or put out a statement saying like this is what happens when you don't cooperate with our, you know, war efforts. And then, you know, David Sax put out a statement that we talked about as well that blames Anthropic. And then there's this odd narrative that

they couldn't actually get in touch with Daario to stop all of this or make it like easier to resolve because he was at a wellness retreat, but there's no evidence of that. And in fact, he wasn't supposedly. the CEO of Anthropic. >> Yeah. So, it's turned into this like crazy the boys are fighting kind of framework and there the fact that none of us know what's going on and also I think there's going to be a reasoning like a very high potential that we'll never know what happened. >> Yeah. So, there's nothing to learn here except like, oh my [ __ ] god, what are innovators in the United States supposed to do? Because this is certainly not helping AI stay on shore.

>> A couple of points here that I wanted to make. One, it's very interesting that this is going to be one of Jay Clayton's first focus areas when he is confirmed as director of national intelligence. And for those who weren't aware, this was another interesting news item. So Jay Clayton was former chair of the SEC from I believe 2017 to 2020 and then he was interim United States attorney for the Southern District of New York which is basically the headquarters of all of the uh you know all of the prosecutions surrounding major financial crime in the United States. Now he has been nominated and all signs point to a very quick confirmation process for director of national intelligence. a very

interesting kind of financial background that he's going to bring to this environment >> as you're talking noteworthy is he's so the SDNY is the office under him that is continuing to pursue like the tornado cash case right >> which is frustrating and I have my thoughts about SDNY I mean it attracts the creme de la creme of lawyers for sure it also attracts highly ambitious lawyers like a lot of people that just want to run for office go and and you know try to work for the SCNY although there's also extremely extremely talented prosecutors brilliant legal minds but weird dynamics there given the the focus um but the other the other

thing that occurred to me Jesse as you were talking is we've talked historically about how crypto and AI are like weird cousins and most of the time it's crypto wanting to you know be friends with AI and all of that but it does seem like cryptography offers a slew of potential technologies that could help at least mitigate some of the theoretical issues here. I mean, zero knowledge proof technology springs to mind as an obvious choice in terms of data protection or additional security with respect to a lot of this um surveillance. And I hope the two worlds are are talking to each other enough to explore like the best practices going

forward here. And I I'm a little afraid that they aren't necessarily >> they are not they are not as far as I know. But what I will say is part of the problem not to put it all on crypto like AI industry isn't talking to each other. like anthropic and open AAI have a very non-secive frenemies to enemies story right and I I don't I as you all know have really appreciated anthropics warnings about its technology and asks for regulation which reminds me a lot of like the good actors in crypto too at the time.

>> Yep. But it's a very difficult line that they're trying to walk as well, saying like this stuff's scary, regulate us, but also let me find a way to put out the best model, which we told you is based. Let's say Fable is super safe. Let's just assume that, right? They the narrative that led them to release Fable is mythos is the scariest thing we've ever seen, but don't worry, trust us. So it just comes back to like who are we supposed to trust here right now? I feel like until this whole debacle, there were a lot of things about Anthropic I wanted to say like I trust them. I feel like they left OpenAI with the goal of

creating a safer, >> you know, um, product for everybody, but they're also saying this stuff's freaking crazy. So that's where it's like I think as a user daytoday with AI we don't stepping aside from the crypto industry like we don't really know what all this means and so saying I think KG I agree with you like there are a lot of things about cryptography that could but the truth is is like we don't really know enough about how this works because it's not transparent which gets us back to like maybe AI and crypto have lessons to learn from each other. the chicken and the egg to a certain degree. And you also made a really good point. It's like since the beginning of time, consumers

have struggled with who to trust. >> Yeah. >> And you could talk about this in any context. Food, furniture, cars, like we could go on and on and on. I feel that way so intensely as a parent. >> Like I'm like, what can I feed my children that isn't going to give them cancer? Like, you know? So I mean it's it's on in every industry that consumer trust and I I'd say AI is slightly scarier because you have highly sophisticated individuals that still don't understand even or don't have the ability to get the information they need to make the appropriate assessment.

>> Okay. So I would say AI is not just slightly scarier. I would put it I would put it on the level of like the nuclear bomb because it has civilizational implications. And to me, every time every time I listen to like Jesse's like beautiful explanations and analysis of all this stuff, all I think about is like this is and maybe this sounds like very anti- capitalist of me, but I really do think that like AI is different than all of those things that you just named, KK, right? like it really is something that I think should not just be left to the private sector.

There should be some sort of um I've said this before, but like some sort of public private collaboration, right? Like similar to what we did with the Manhattan project and other sorts of efforts like historically like I just think AI is just totally on a different level that we can't just leave it to private the private sector and decide like whether we want to trust them or not. I mean the answer to that is no. they're a business. We can never fully trust them, right? And so to me, it's just different. >> I I agree and I think that's why the international efforts here are particularly important as well. The problem, you know, from the innovators perspective, which I think like crypto can be empathetic towards, is the

federal government sort of can't make up its mind. You know, we've talked about the fracture. So on one side they're like stopping something because it's super dangerous, but remember the EO that they were supposed to pass and then they minimized it and made it smaller and now they're saying the state shouldn't be able to regulate at all because it's going to hurt our ability to compete with China. So now I I feel like everyone is sort of just watching this chaos ensue and no one really knows what's going on and I think hopefully in 20 years we'll have some sort of like real insight here. But Anthropic isn't talking to the government. The government they're just speaking different languages and the problem is

like the consumer is speaking a third language. >> Yeah. Perfectly put. And for what it's worth, the I don't think that's anti anti- capitalist of you. Like, I'm an ardent capitalist and was trying to explain the dangers of socialism to my 8-year-old the other day and my husband was making fun of me. And I think what you're you're pointing out an obvious risk that has nothing to do with the economics of the situation. This is all completely unchartered water, too. So, >> little scary. Um, but speaking of capitalism, guys, we need to talk about something different because I'm getting depressed by this conversation. So, moving on, there has to be a dystopian element to like every single one of our

>> dystopia. Yeah. >> Yeah. Whether it's like dinosaurs or, you know, AI, I I I love it. I I have a lovehate relationship with it. That was a clip from Dex in the City where the wallets are cold and the takes are hot. And we've got a lot more in the full episode. Links in the show notes. I'm host Jesse Brooks. Subscribe and join us on the live stream every Tuesday at noon Eastern. Fidelity has been researching and investing in blockchain since 2014, long before it was a headline, and they're hiring crypto and DeFi professionals to join their team and discover what's next in finance. Fidelity is looking for people with fresh perspectives from different backgrounds, whether it's

tech, UX, or product design. Whether you're cryptosavvy or cryptocurious, as long as you have the passion to make a real impact at a company striving to make finance accessible to all. Explore crypto careers at Fidelity today and make the decision that could change your future for the better. Visit crypto.fidelitycareers.com to learn more. That's crypto.fidelitycareers.com. Fidelity is an equal opportunity employer.