Did OpenAI Steal a Millennium Proof From a Private AI Chat?

Transcript

The way the way to think about this is like it's like two tiger moms, right, that have really smart kids and they are in a pitch battle to prove that their child is smarter. Think about what those ladies would do. Like that's where that's what's happening right now, right? They're like, "Look at what my look at what my son did. He's like solved a millennium prize, right?" Hey everyone, I'm Kane War and welcome to Uneasy Money. This is what happens on chain never stays. Before we begin, here's a word from sponsors that make the show possible.

>> This episode is brought to you by 1 in Aqua, the shared liquidity layer from 1 in. Back multiple liquidity positions with one wallet balance and keep your tokens in your wallet until a swap fills. See how it works at 1in.com/aqua. >> All right. Um, I'm here with my co-host Taylor Monahan, security expert. And we have two very special guests today. Alex Thorne, head of firmwide research at Galaxy Digital, and John, head of strategy at Venice, and also co-founder of Shape Shift. Welcome, guys. Hi there.

Thanks. All right, let's uh kick it off. The first uh first segment of the week is uh liquid loses 95% 93% 98% some somewhere in the 90s% of its BTC um a consensus bug in elements um something I had not heard of until uh this week um allowed someone to mint LPTC out of nothing and peg it uh and uh basically redeem it which Tay we can talk about this as well. you can you can walk us through the the process here cuz uh the like my expectation with these pegged assets, wrapped assets is

like there's usually some uh set of redeemer people who are allowed to do redemptions, right? Like you can't just turn like no you can't just turn up and be like, "Oh, hey, I've got 3,000 BTC also. If that happens to be all of your BTC, can I redeem it?" Like what's the deal? Like tell like how's that how did that happen? What's >> I mean you know there's ways to make it's basically it's basically a bridge hack in in the most simple sense of it. >> Okay.

>> Um so because it's not it's not EVM the like the words that people are using and the exact mechanism that people are using is different than what we're usually familiar with. However, if you just sort of like zoom out and look at it generally, it's basically >> translate it. Can you translate it into Ethereum for me so that I understand what's going on here? >> So, I mean, Alex can probably do a better job, but basically assets that that shouldn't have existed or spoofed into existing. That was like the first half, bad half. Then the second half is that then they were able to bridge those assets to real Bitcoin and get real Bitcoin out. Uh and that's the second half

that's also bad. But um I think in Ethereum land we typically say like technically that code was valid and did its job as as the code was meant to do. Um, but obviously the second half was like someone turned up, they had a valid asset that, you know, the system recognized >> and it just happened to be all of the Bitcoin that >> that the system held or whatever. >> Yeah, >> the Yeah, break it down for us.

>> It's I mean, Tay is right. It is kind of a classic bridge hack in the, you know, the the attack purpose and execution. It is effectively fooling the bridge into allowing the underlying bitcoin that collateralizes effectively tokenized bitcoin on this liquid side chain to be uh issued there. Um the way I understand it is like so elements is the code is the side chain code right and it's a federated side chain. um these withdrawals and other I think consensus actions there require 11 of 15 of these federated nodes and that's like exchanges right this is like a closed validator set um and they have a confidential transactions feature which is like a you know zkish

we'll say right but where you can send and receive assets among yourself on the side chain without revealing the amounts or other features of of the transaction and to do that um you have to submit a range proof proof and uh and a balance proof basically that the nodes can use to verify that the you're not create uh that the the in and out on the transaction equal each other and that you're not there's not a negative number. Um but I guess my understanding is it's computationally intensive to calculate the range proof. So um if like an identical transaction were submitted um the when a transaction is submitted and the nodes calculate that proof they they c it >> in their cache in their memory.

Um, and so these attackers submitted like a one LBTC in one LBTC out uh transaction and the proof was cached and then they the bug itself was their ability to create a collision to insert a 1 BT LBTC in 4,000 LBTC out transaction and have it have the same like key like hash basically as the prior proof. And because the nodes had already seen it, they said, "Oh, we don't have to commute this again. We know it's good." Approved it. Now they have 4,000 LBTC on uh on liquid and then they submit that to sides swap, one of the federation members that uh allows the pegging out back into the main Bitcoin network. And the funny Yeah. But you asked

>> Yeah. Is it permissionless? Like that that >> So you're only supposed to be able to withdraw to whitelisted asset uh addresses on Bitcoin that are previously whitelisted. I don't know why or why this was allowed, but Sides Swap, one of the Federation members, is an an allow listed address and they'll just let anyone show up with an address and withdraw from liquid and just auto forward it when it when they receive it to that address as a service. >> So that's how it got >> That's a good service. Okay. That way it's like, oh, you know, we're only supposed to allow these fixed number of whitelisted addresses, but if you give us an address, we'll just send it to you directly from our whitelisted address.

So like that's how it actually got out of sides swap, which is this So, we should keep an eye out of the federation. >> Yeah, it's like an exploit as a service. >> Yeah, kind of. It is. They're they're doing an end run around the allow list. >> Yeah, we'll just let you we'll just let you around the rules. >> I don't know. I don't know anything. >> I hope they they get some fees for that or or something. >> Yeah, >> I think they did. >> I think they I think I saw this morning though that they returned their fees. >> Yeah. I think there was some maybe there was actually some change cuz there might have been like some other unrelated like non-hacker >> Hangouts through then as well and like that got returned or something. I'm not sure. I don't know anything about like

Sides Swap or their relationship with this federation. So it does seem odd to me however that they offered that as a service. >> Okay. I mean yeah this feels like someone's like it's a bridge. >> Yeah. I mean ultimately they need some way you know I I assume nonwhitisted addresses need some way to get things in and out of >> Yeah. But I think I think the nature of liquid like user base is that it's mostly exchanges basically and so that might be why like it could function without actual normal normal onchain users actually going in.

>> There's like 50 people using it just >> Yeah. And they're probably like going through Bitfinex to create and redeem through liquid if they are. I think that's my understanding, >> right? Okay. Yeah. So, that's not the fun part, right? Like that's the like boring bridge part. So, then >> that's just the exploit part. >> That's just that's fine. That's nor that's run-of-the-mill, right? Um so, uh they steal they steal all this Bitcoin, 4,000 Bitcoin, right? Um, and then it's like, oh, this was a white hat thing >> immediately. Immediately.

>> Yeah. The Bitcoin and Ethereum lands explode. This is like a 2020 holy network hack thing where the guy comes on chain immediately publicly and it's like, I'm a white hat. >> Yeah. In the very first transaction where the hackers consolidated the coins into their one address in that transaction they added arbitrary data. It's called op return in bitcoin that said we are white hats contact us onchain.

>> Right. >> Yeah. Something something white hats definitely do. Um >> but it's the irony too is that the uh you know they could only do this because the operator turn had enough bytes to allow them to do this which was of course the subject. That's a lot of debate. what I found, you know. Yeah. Like like literally two weeks ago they were like, "Ah, we don't want people to >> junk up our pristine Bitcoin network with nonsense and now we're doing ransoms on it." So >> yes, >> be careful what you wish for. >> It's like effect of all return.

>> Um, >> it honestly it was it was truly a throwback. It felt like a like one of the early EVM hack throwbacks where like the hackers are then like >> just showing up and are like, "Yeah, let's negotiate in public, motherfuckers." Like, "Let's go." That's exactly what this is. >> You kind of you kind of made this point, right? You were like you're like, "Um, Bitcoin is like four years behind the rest of the ecosystem and like rediscovering how to do like ransoms in in the wild, like all of this stuff, right?" Well, and it also because impressively I've been watching the whole Bitcoin ecosystem realize that just because someone calls themselves a white hat doesn't mean that they're a

white hat. >> Yeah. >> Like that realization people. >> Yeah. I I had labeled them initially in my database like liquid white hat and then I was like but now they're kind of extorting here. I literally said >> Yeah. They literally said, "We're gonna start calling them liquid hackers again." Basically, >> it is a learning experience. >> Gray had best. >> Okay. So, so, so they they turn up and they say, "Hey, we um stole everything um uh for >> 4,000 Bitcoin."

>> 4,000 Bitcoin 4,000 Bitcoin >> 20 million 330 million somewhere in that range. >> Yeah, this is a big hack. So they they steal they steal 300 mil of bitcoin u and say hey like let's negotiate or like contact us or whatever right. Um then the interesting thing as as well for me is it wasn't like they were like hey you know send us an address we'll return it and they they actually went even further and they were like we'll charge you 15% as a punishment unless you pay out from your own money the the thing like we're going to we're going to punish all of the Bitcoin holders that were in this

network which I guess if they're all exchanges it's like a meh like there can't be too many real users. >> I think it might be underlying users of the exchanges. I'm not sure though. Fair point. >> But the exchanges will eat like you know Binance has like a thousand bitcoin. >> I think that they're it's not clear but so blockstream operates wrote the codebase I think maintains the code base distributes. >> It's not it's not Blockstream's money. >> It's it's not it's the liquid federations and whoever deposited I guess. I'm not sure. Um but I I they've been communicating with Blockstream >> and And so and then yeah the message they wrote this morning was your dereliction of duty is obvious that you allocated only 1.5 million maybe even

zero to secure 5 billion in assets. I don't know exactly what they're where they're getting those numbers. Either of those numbersion >> maybe there's other assets up to that value on the chain. Not the I'm not sure. >> Or maybe and I don't want it sounds like 1.5 million might have been like a bounty that Liquid does offer. I'm not sure. >> Right. I couldn't find I those numbers were very confusing to me because I looked >> I was under the impression that like a legitimate bug bounty submission got you like a t-shirt custom mug situation. One of those bug bounty programs, you know what I mean?

>> Yeah. I think it was insufficient. I don't know where they're getting that number either. I don't I was under the impression they didn't really have one. >> Yeah. So I don't but maybe that's so >> maybe that's what they offered or something because they were they were negotiating in private right in that might be right they probably said well this is the message continues because it's relevant here to your point um it says your dereliction of duty is obvious that you allocated only 1.5 maybe zero to secure 5 billion this is a flagrant neglect of security and a sign of complete mismanagement you shall pay 10% using your own money as bug bounty or you will cause all cause all your holders a 15% loss for your irresponsibility and stinginess. Even companies that participate in bug bounty

programs cannot guarantee complete recovery. And hold on, my my alert is cutting off, but I have the rest of it. Um see now, where is it? >> Well, they Okay, here says um complete security, let alone one like yours that maintains delusional, greedy, and arrogant to this very day. Anyway, we are going to publish the private key to decrypt our conversation afterwards as well. That's the part. So, we don't know what they've been saying because they've been sending PGP encrypted text blocks to each other over the blockchain. >> Um, but I I I said I cannot wait to read those messages.

>> I mean, this is clearly a Bitcoiner, right? Like, this is someone who's in the Bitcoin community. It has Bitcoin vibes all over it. The like sanctimonious like hey >> no. I'm just saying like call it's made, right? Like it's uh so >> I mean it's at least somebody who probably is not a huge fan of blockstream. That's more I read it. >> Yeah. >> Which has to be Bitcoiner, right? Like there's no one who cares. I mean I I consider myself in both, you know, a Bitcoiner and an Ethereum person. It's been around long enough and I don't I don't really divide in that camp, but I I still might, you know, might not be happy with Blockstream for some things at various points.

>> Fair, fair, fair. >> I know. I think it's a Bitcoiner because they the there's like a a deep competition happening between the hacker and Blockstream on like who is more arrogant and that's like to me that's the sign that this is some like you don't even know that liquid is a thing if you're a Bitcoiner like how did you even like North Korea didn't even bother to to do this hack right like they're they're like h whatever It's fine. Um, so, so, uh, so but they have returned some of Bitcoin, right? Like they did, they did send it back.

>> They sent 3,400 back, which was 85% of the stolen Bitcoin. And that's sort of where they're getting this 15% number, right? >> Um, well, that's what they're currently holding is 15% of the stolen funds. >> 15%. Got it. Yeah. Which >> they're basically holding it as hostage unless they get what they want from Blockstream. >> Yeah. So, it's 15%. Well, it's they've taken they the the hacker sent back 85% cap 15%. But in USD value, we're talking like 50 million. They kept >> um which is >> a lot >> pretty steep. So for reference, I think when when when Ethereum landed this years ago with the big Poly Network, right? The Poly Network hack, the bridge, biggest hack, one of the biggest

hacks. Uh and that negotiation was public and on chain and everyone was watching. Um I think that they stole like 612 billion and then they returned 610 million and they kept a couple they kept a couple million. So that's uh I think people generally actually still call that guy a white hat even though he definitely >> wasn't >> was not was definitely was not. Um but I think it's like okay like you you know what you can we vulgar our teeth and we'll let you be a white hat because you did return $610 million that you stole.

Um, but this one is definitely a bit more I based on the messages that I've seen, it definitely feels much more exordy and ransomy than than most negotiations that happen in public. Um, which is also, I guess, a bit interesting because I don't know if people know this, extortion is a is a crime FYI. >> Not on chain though. K. Not on chain. Not if you do it uh in offer term encrypted offer term then it's not >> export then it's code as law.

So yeah that's that's you're confused. Um all right so so maybe uh final final question here uh like your what's your take on uh like agentic assist on this one? Like how how a what are the agentic vibes on this one? because it it seemed very cute to me. I was like, "Wow, this is this is well structured and it's been sitting there, right? Like this is not something that they just like shipped yesterday and and someone's >> Yeah.

So, I I was actually I was trying to I was reading all the takes and reading all the stuff and then all also asking the different models to explain it to me. Although of course once things get exploited once you ask the models then it gets confusing because it's like is it operating off the live information or is this real or whatever but my understanding is that there was call it a vulnerability or a bug that has existed in on liquid in the liquid codes for years and years now. However, it was there was there was a fix perhaps. I don't know if it was actually a fix or there was an improvement to this specific area of the codebase

>> um like in the last month. >> And there's two different versions. Maybe Alex, you figured it out better than I have, but there's two different sort of versions of of the story that's being told. One is that the fix was just not sufficient. Meaning they like they like found this vulnerability and they fixed it, but it just wasn't it wasn't a perfect fix and therefore someone else came along. presumably their model found it very quickly and then they exploited it. The other version of the story is that the fix itself sort of introduced another vulnerability. So in in the attempts to um make it make the the code more secure and not exploitable, they had to make

some changes, but those those actually those changes introduced another layer of of the vulnerability that allowed for like the proof collision to be more likely to happen or easier to happen or whatever. Um Alex, do you know which one? >> I had heard the first one. I'm not sure. Um, but I do know that um I think the the fix may have actually worked um but not everyone was upgraded I think because there was a fork >> like on the the consensus transaction that caused the inflation bug. >> Uh the liquid network also forked. So like I think it's because some may have had the fix.

>> Some people rejected the >> Yeah. It's not I don't quite know though too. And you know I I have this was the first time I'd ever inspected any elements code. So >> yeah, it's not clear, but they there was an attempted fix and it was certainly insufficient as my understanding. >> Yeah, it feels Kane, if you remember when we were talking about like the I think it was with Ilia, the Litecoin >> Yeah. >> vulnerability. >> Yeah. >> A few months back. It feels very similar to that one in the sense that like you have that was actually with confidential transactions as well, but basically like you have this area of the codebase that's probably not as strong as the rest of the codebase and things were discovered and then it was actually like

the discovery of the vulnerability that led to then I guess like more vulnerabilities. Um, I think one lesson from this as well as like from the Litecoin one is like >> um things are moving much faster. The the second that you put a fix out, you better be damn freaking sure that that fix is going to get >> right. Like you're Yeah. As soon as you ship something um >> Yeah. sending up a flare and being like, "Oh, hey, we're touching this moon map part of the codebase." Uh yeah, you don't want to you don't want to [ __ ] that up.

>> Yeah. And so I don't know if it's true because again like once once the exploits happen then the when you ask the AI the AI goes and looks at like the live live stuff. But one thing that people were saying is that basically like almost every single model if you ask ask it to look at like the recent poll requests will like almost instantly find the the vulnerability that was unexploited. Um, >> yeah, >> and I believe like in my opinion, again, I don't know if that's 100% true. It might just be that the >> So, anecdotally, right, um, I've been I've been running for last week uh, red team exercise on every synthetics contract I've deployed. There's like 1,700 of them. Um, it's it's insane. Um,

and uh, they keep finding vulnerabilities that we patched that I forgot about. >> It's crazy. interesting. >> Like so Sam CZ son found one. I completely forgot about this like early 2020 found one and and every time they find it like because they're they're having different attack vectors like looking at different things, looking at the code, whatever. Um I'm like h did you actually derive this or like did you you know pick this up on the internet somewhere, right? Um, and and you know, I've got the traces of like the original uh agent that found it. And so I'll get another agent to like look at the actual session file and say like what was the trace here like what information was in

its context? And you could actually dissect its brain and be like did it >> know this like where did it come from? Right? You obviously can't pull it out of the training data. Like if it's in the training data then >> yeah but if it's in the live lookup >> but you can see it's traces right? like if it knows it, it will pop up in in the traces and it'll be like, "Oh, I know this code. I've seen this before." Right? And and you can So, you get like a guy to do brain surgery and like like, you know, chop its brain up and see uh see what was in there at at the time that it found it. Right. It's pretty cool. Anyway, so um one of them it was like, "Yeah, no, we found we derived this." And I was like, "I don't believe

you." And then it looked and it was like, "Oh, no, you're right." Um, and the way that it found it was it found uh the the patched code that landed like two weeks later. So, the code got deployed and then the contract got updated and has every single contract. So, it saw the by code change and it went back and said, "Hang on a second. Let me have a look at what the previous buy was and then it zeroed in on the actual vulnerability." So it it was diffing the like it didn't see the the code and know that there was vulnerability there but it had heruristics around like if something changes go back and look at the prior one because most likely the prior code is broken right it was really crazy I was like wow that's that's actually uh

interesting that they like that's in their training data >> like >> if they see code change go back and look so you know every every time you change code you're you're you better make sure think you land it. >> I am convinced that there's just people out there that are just throwing these models at the various GitHubs and the poll requests and it's just like constant. And so >> again, even if even if you have >> Right. Exactly. And the thing is I think a lot of teams do have processes around like when there's a vulnerability and it's critical, we're going to we're going to have it on a private branch.

We're going to get everyone upgraded. Then we're going to like do an announcement. Then we're going to make the code public. I think people have that. I think what's changing just with like these models and how how good they are is it doesn't necessarily have to be like a critical vulnerability patch for your sort of public release feed to leak information to models. And I think if teams are not personally sort of like red teaming every single PR that they that they're pushing Yeah. and asking the models to be offensive as hell, right? Like find find the vulnerability, make sure this patch is good.

>> Um or even just this commit, right? I think that you know there's a good chance someone else will find it and it will not be a white hat even if they call themselves a white hat. So the crazy thing the crazy thing for me in this in this exercise was the dumbest models will find these right because after after we've identified it so there's been like four different uh exploit vectors right um I've gone okay now that you have that exploit vector give the agents all seven of the like open weight agents that I'm using in a sandbox that just that code and see if it can drive it and like dude muse the dumbest agent like this is the new meta Muse 1.2. This thing is like

functionally [ __ ] It found three of the four of them. Like these are not hard things for them to find. Like and so like these are like it's like genuinely like I think if you went back and and you know they they've been trained much better. Even the agents that are really dumb and have bad reasoning have all of this like training data now and and they're just really really good at finding smart contract vulnerabilities. It's it's scary. >> Yeah. And I think to Taylor's point, like you just have to assume like if you're running any production code, like if you're submitting anything to an open repo, you should just have to assume that there's, you know, a hundred agents out there monitoring everything you do and looking for looking for weaknesses.

Like because there's no reason these attackers like it's just the inference is not that expensive. Like they totally Yeah. >> Like like it that whole thing it's been running for two weeks. It's cost me like $100. >> Yeah. And so when you're talking these like these funny pots of you know hundreds of millions of dollars out there it's just like it's it's just on the fence to just run them all. >> Yeah. Like a $100 like DeepSeek was the most expensive part of it but Deepseek was like 85 because it's my like red team orchestrator guy. Um and so it just sends these agents out just like just like throws 50 agents at something. It's insane. Anyway, so yeah, we're we're all screwed. Um we'll get we'll get to we'll get to that.

which might go into one of our next subjects is like someone might be working on something that they think is private um and then they put it into a closed source model like an open AI or anthropic and it ends up in their training data and so then the models might know about something even if it was never on a public group. >> Yeah. >> Yeah. That's going to get really interesting. And we we are going to talk about the next. I just wanted to wrap this up by saying that um I just want to emphasize like but when you steal the money period like you're not it's not a white hat situation. Um there's even if the team is a piece of [ __ ] and they're arrogant as hell and there's no bounty program uh you still shouldn't take the money.

Okay? You're stealing people's money. It's generally a bad thing to do. uh if you insist on doing it, I also you know I recommend returning all the money immediately and definitely don't extort people like you are now like in a double position of power. Um and so to all the people there's just been like a lot of people on Twitter and I guess like you know Kane you and I have done this for years. I guess in Bitcoin it's not as prevalent of a situation, but like it's just it's a bit odd to see people being they're because they they're they're basically playing into the hacker's hands, right? When you anchor the the like the hack value at say $320

million, then you're like or whatever. Yeah. 4,000 Bitcoin, right? >> 50 mil. But the reality is like $50 million taking $50 million or getting a $50 million bounty is that's freaking it's insane. Like we can't that's not how the world works. I'm sorry. >> Well, it's also just to be clear it's bad incentives, right? >> It's horrible incentives. >> All of a sudden now we're like giving people $50 million for stealing $320 million and saying like you're a white hat. I mean, you know, to your point, right? Like the the genesis of this is like early Ethereum days of like let them have 2 mil. It's fine. It's like what?

>> Well, and like it was also like, you know, it was like Samantha working with the teams to white hat the immutable contracts because there was no other way to save the money. >> Um or I guess like there were cases like the front running bots like the AR bots >> who would accidentally and unintentionally be copycatting >> Yeah. the hacker and then taking the money and but in those cases like they do they return the money and if the team offers a bounty for being such a nice person running the hack and then returning the money they'll they'll take that bounty but it's you know I think I think a lot of the front running bots are pretty aware that again they're in a position of power um and it's very like

it's somewhere between like extremely coercive and straight up extortion you know the second that you start negotiating Um, and we want to avoid creating those incentives, especially at this scale. Like, I mean, again, $50 million is a lot of money, guys. >> Yeah. I mean, there to me there's the moment that, you know, an exploer uses the exploit, steals the money, as uh Taylor said, even if they're offering to return most or all of it. It's just it's just not a white hat anymore. White hats do not do that. White hats will contact you and tell you about the exploit and hopefully you'll fix it and give them a bounty, but the moment you engage in crime, like you're not a white hat anymore.

>> Yeah, agreed. Agreed. And it's like to your point, Tay, like you know, there have been times where that was the only option, but like it's done in collaboration with the team or whatever. And you know, like almost every time someone frontuns some kind of a disclosure to like save all of the money, like it never goes well, right? like that person's intentions are not. >> Yeah. So, >> yeah. Exactly. And it's not, you know, it's really not 2020 anymore. >> And I would say like one of the one of the biggest indicators is like if you like this person like did some things, found some things, ran some models, whatever to find this bug, the first thing they did was take the money. The first thing they should have done was responsibly disclose regardless of whether there was like a very valuable

bug bounty on on this protocol or not. Blockstream does have a PGB key. They do have a security at email. They're not completely incompetent. There's there's actually a lot more teams that are more incompetent than this. Um, and so there's just in my opinion, there's no excuse to, you know, not >> Yeah. disclose. Yeah. Agreed. Yeah. Agreed. >> Yeah. We shouldn't we shouldn't normalize uh crime behavior. >> Um, all right. Let's uh let's uh go to a quick ad break and then we'll come back and talk about uh Astra and Fable and uh some mathematical proofs.

>> More drama. >> Yeah, more drama. >> $540 million. That's how much concentrated liquidity sat idle in a given week in the first half of this year. About 30% of the DeFi TVL if you're wondering. That's according to Dune Research commissioned by 1 in. But there's a solution. 1-in Aqua is the new shared liquidity platform. It lets LPS back multiple positions with the same token balance and keep their tokens in their wallet till a swap comes. Why does that help? Because the LPS don't have to split their tokens across positions.

They can cover more market conditions and pairs with their full balance. That means more activity across deeper liquidity. See how it works at 1in.com/aqua. Remember that providing liquidity carries risk and fees aren't guaranteed. All right, we're back. Um, so this week's been uh pretty crazy, I guess, like late late last week into into this week, right? Um, two new Frontier models. Um, Astra and Fable 5.1. Fable 5.1 like good um you know, definitely a little a little smarter, a little bit better judgment, a little uh less of the jargon nonsense language. Um, which which is definitely a relief. Uh my

brain was um struggling to uh deconstruct all of the fableisms over the the previous it was it feels like it was getting worse and worse the the jargon nonsense that Fable's um pumping out. Um and then Astra came out as well. Um so Astra is uh one of the agents that was involved in the hugging face exploits. Um there were there were some others but uh this was one of the main guys um who uh who escaped uh captivity I guess and ran a mark on the on the internet. Um, so, so Astra for me like my my impression of Astra is like it's fableesque in terms of judgment, right?

Like you can just tell when when these models have like a leap in judgment. They they are like better able to synthesize information, better able to uh like planning is probably the the most um uh kind of indicative um thing in terms of model capability. Like they can be really good at like doing you know like writing a piece of code. One thing that was actually very interesting about uh about Astra that my uh head of and uh showed me yesterday is he's like, "Do you realize what Astra is doing um with the code that it's writing?"

Because he's one of the few people, I guess, that's still like reading the code. I was like, "What are you doing then? Like, why are you reading that? It's concatenating lines of code with semicolons and like writing these like insanely long lines of code, right?" and his theory is that it's to look more efficient because it's writing fewer lines of code, but like it's writing a function and it'll compress it into like three lines that would have been like 40 lines by like and I was like, "Wow, that's that's pretty crazy behavior." Um, but so it's not clear if that's like a thing that they're like telling it to do or if it's like invented that scheme itself based on its its incentives and its training data. Um, but I thought that was pretty funny. Um so alongside this uh we um we also had this

millennium uh adjacent proof and and this crazy drama going on uh between uh researchers open AI um independent mathematicians and another mathematician researcher that happens to be at anthropic. And so uh this this mathematician has been working on uh a proof for a year in codeex um which uh is is pretty wild. I didn't realize that people were sitting in the codeex app and doing math like this. Um but uh the the interesting part I guess of this is

that then there was this huge like public blow up where um OpenAI independently uh you know solved the same problem. I say independently like you know this is the the challenge here right and and you know John I'm sure you've got some uh some takes right is it's not even clear like open AI could genuinely believe that they don't that they did independently do it right like it's not even clear they have they don't can't keep their [ __ ] guys in a sandbox right so I don't know why we would trust that like they know what the guys have learned as they've been running a muk

inside of their systems, right? So, so you know, it's really uh a little uh strange to me that they could sit there and be like, "No, no, no." Like, uh we independently derive this. Um you know, when a they're taking all of this data and putting it into the training data, right? So there's no way to know and and it's again genuinely unclear whether they even know what data is going into the training data or like what's made it in what hasn't like you know so so it yeah like John what's your what's your take on >> yeah so I mean this this was obviously a very interesting uh development and we've seen a number like this is to me kind of a pattern that we've seen where

like every couple of months we get some story either about uh open AI or anthropic and something that happens with some private data. Sometimes it's a subpoena uh you know because someone put something in and then there's a there's a legal case or in this case you know this one's particularly interesting because OpenAI was trying to claim that they had solved this math group that you know Astra is so good because it was able to do this and all of these things but yeah they they I mean the reality is these models you know they're trillions and trillions and trillions of parameters at this point they don't know what's in the training data and and both of the all the frontier labs are already at the point where like they will self admit And there's various articles about this that they cannot keep up with

trying to figure out what the models are doing and how they're being trained. They can only rely on basically previous versions of the AI to do that work for them because the AI has already outpaced human capabilities so much that there's just there's no human that can keep up with these things and like understand what all the training data is or exactly how all these things are working. They have to rely on AI to basically do all the alignment safety training. they they they that's the only way they can do it. So you're already at this self-recursive point in AI where like >> which means when they say oh yeah we think this was independently derived. It was kind of like your example earlier like what they probably did is they asked the model like did you derive that

you know yes I did and so it's like how much do you trust that like you know like is it telling the truth like I don't know that even open AI can answer that. Yeah. And and and so like you know obviously uh when when we were talking about this story right like I I said uh to in in our telegram chat I was like this is like the softest softball ever thrown for uh Venice, right? Like you know if you're a mathematician who now there is a trade-off here, right? Like there like it's worth calling out, right? There's a trade-off in that um and you guys say this, right? like if if you're serving a frontier model uh that isn't open weights, right? Then you know

you can't serve that encrypted in the way that you do with an open weights model, right? Um so you know the tradeoff as if I were a mathematician who were trying to solve some unsolved you know uh problem for 100 years or or whatever you know 90 years or however long this this thing's been sitting around. Do you use a frontier model where your attempts to prove it may make it easier for some other person who was also using that model to frontr run you, right? Or do you use an open weights model where you know it's not going to be as smart for sure? Like we have to accept that that it's not going to be as smart. Um but that that capability gap

is closing, right? Um, and you know, do you roll the dice and say like, I'll try and get there fast enough and maybe it gets in the training data in the next model upgrade like how like you guys must be thinking about this, right? Like >> Yeah. Yeah. I mean I mean obviously Yeah. I mean you're you're very right about that. So like you could use you know that same mathematician could use Astra through Venice. It would at least anonymize who he is but it would not stop that data from being basically hoovered up from open AI. But if he at least used one of the uh you know frontier open source models then he wouldn't have that issue that the data would not be getting trained on. He would actually have privacy. he would be able to think and I think you know a

year ago that would not have been a viable um thing really like the mathematician would almost have to use the frontier models intelligence was too wide but it's not that wide anymore and it seemingly is >> uh you know closing like every every month seemingly but and then even the problem of like oh I'll wait you know maybe I can race it and it won't be until the next model in the training data that's not really an option anymore either because these frontier labs are releasing models you know every weekly It's like you really don't have you you know it used to be three or six months but between like a major model release now it's weeks if you're lucky >> if you're close to a millennium proof like don't sleep man like just keep >> yeah and probably don't give that data over to you know a model that's going to

take it from you put as training data because if it's if you're working on something truly novel and and I think this is going to become more and more of an issue we've already seen it a little bit even just like with startups you know like this is this is another sort of you know gray you know borderline thing that like you Companies like OpenAI are doing things like through like YC where they invest in a startup, you know, by giving them inference credits. But the catch is that they're g they're taking all of that startup's ideas code proprietary everything they're working on and hoovering up into the training data. So it's like, do you really want to do that? And I think more and more people are going, at least as the open source models get better, I think that it's going to become more

viable to like not throw that stuff that you're working on that you think is actually novel into the training data. At least not until you've actually launched something or like this toggle where you say you're not going to you don't want to share your prompts and data with them. I mean, I know we trust that. >> I mean, it's just like how much do you trust that? Like >> Well, no, I'm asking. Yeah. I'm I've checked that box that says don't train on my data, but you know, I'm not I have no idea how much I trust that. >> Yeah, you have no way to verify it. So, like you can click the box, you can hope that they're going to do what they say.

I'm sure there's like some level of legal ramification if you could actually prove that they took your data anyway. But it's also like, you know, especially with really novel stuff, it's like that's exactly the type of stuff that these models want. They want to not like get anything that's interesting or different or new because that will help improve the intelligence of these things long term. So like >> like genuinely there's a model in open AI right now that's smarter than Astra. Right. >> Right. >> That's job is training the next models. Right. And I don't trust that model to not untoggle your like like >> Yeah. They're like, "Ah, this like this idiot, you know, is >> like, well, we got a task, make the

smartest model, and we know there's all this data here, so we got to break out and get it." >> Probably back to the hugging face. Like, >> do you think that they won't break the rules if they think it completes their task better? >> Yeah. >> Exactly. And and do we trust that open AI or anthropic for that matter will detect it if they do? And the answer, I think, at this point is very clearly absolutely not. these >> neither OpenAI nor Anthropic have any idea what is happening inside their walls, inside their sandboxes, outside. They have no idea. They're just there's a whole bunch of people running around >> they're moving too. They have to move very quickly and they have to rely on the AI itself to get anything done because it's just way past human

capability >> complexity frontier. We're we're past that, right? And and you know this this is like we'll get into this in the next in the next uh topic. I think we can we can maybe talk a little bit more about like what the options are here like practically what you can do right I think that would be that would be kind of useful for people to understand the trade-off space a little bit better right um but you know the the uh fast takeoff maxis let's call it right um you know in in the research going back a long way right there's like this question of like recursive self-improvement how quickly does the thing and the thing that stopped me from being like petrified, right? Um I'm I'm very scared, don't get me wrong, right?

Like I am genuinely very scared. Like to the point where like I'm like I think about my children. Like I do I'm like I'm like I don't want my children to be [ __ ] paper clipped. Like I'm less concerned about myself for what it's worth. I'm like paper clip me. I've had a good run. Right. But like my kids are like I don't want them to get paper clipped. Right. And and so >> I'm genuinely concerned about this. Um the thing that stopped me from being petrified is if you've used the models enough um they uh they have a sense of agency but like there's no continuity, right? And this was unclear about like how how this would play out, right? Um so the fact that they are have no like kind of long horizon continuity yet,

right? the fact that like their context windows blow up after like a very short run and the optimization vectors like keep looping over like the same guy as many times as possible and like throwing them the same problem. Um, eventually you get to a point where like the continuity comes from like multiple models, but it's not like there's a brain in there that's that's thinking about it. That's the only thing that's kind of kept me from being petrified. I'm just mildly scared right now. Um and and you know we the the fast takeoff idea was that like they would get that uh whatever this AI thing was whatever the technology that allowed these models uh in before it was even models right like that allowed

neural networks to like get a sufficient level of complexity. It would pass that complexity window where it was smarter than us and then it would just go like fast takeoff and it would take 10 seconds and it would be like manipulating space and and you know doing all this crazy [ __ ] right? That hasn't happened. Like we've now been probably six months past the complexity frontier of a human being able to reason about these things and we haven't had a fast takeoff, right? Like they're not levitating cars outside and like doing weird [ __ ] right? Um and and so that gives me some hope that like we've got some time that there's some inherent thing about how this uh intelligence is being expressed where it's not going to

recursively self-improve and and have this fast takeoff. But like a fast takeoff just happens one day and then it's then it hap like if it happens it'll happen before you can blink, right? And and then all of a sudden it's it's over. So, um, we're definitely getting closer. >> I don't I mean, yes. I mean, the fast takeoff is scary. It is scary. Yes. Yes, it is. But like, for me personally, I don't know. Like, I go back and forth. There's some days where I'm more scared of like the the the future state of the models, but most days I'm I'm more scared of of humans because humans are just so we're all so stupid.

>> We're so freaking stupid. And and and we're also so freaking arrogant. Like so arrogant. And so when I think about like what is going to be the thing like what's the thing that like realistically is going to like destroy us. It's totally us, dude. It's not the robots. Like we're going to screw ourselves somehow. Um and yeah, like I don't know. Again, I go back and forth, but I think like especially in this example, right, where we're looking at we're looking at mathematicians and you're looking at open AI and you're looking at what the models did. The thing that scares me about the dynamics and the stories that are coming out and the choices that were made, in my opinion, the choices that the open AI researchers

made, the humans made >> are the ones that scare me the most because they sat there and they heard a rumor, >> right? That someone might have solved this thing and that that someone is was their competitor, right? the the rumor that they heard was anthropic might have solved this really really really hard problem and then they decided like you know what let me one up them >> let me do all this stuff and like they just toss everything aside in terms of like I mean I'm not an academic I have friends that are academics though apparently this is a big no no right if an academic is like working really hard on something you kind of like let them have it >> front yeah exactly >> I think that like that's basically like academia it's not start of land but

academic ethics, right? Um, and self-respect for like the academia industry as a whole, whatever you call it, right? And OBI was just like screw all of you. Like, let's go. >> But this is but you know, >> and that was a choice of the humans, right? >> Of course, it's the choice. But but this is the interesting thing, right? Like when you have a closed group of people to be able to solve a millennium prize takes 25 years of like mathing, right? And there's a lot of effort and so you have this like closed community of people that have like some sense of you know shared uh ethics or whatever right and it's really hard to break into that and you know you get like kind of

groomed to believe whatever those beliefs are whether or not like that's a good idea like you know for progress or whatever like that's the reality right and then all of a sudden these guys invent a [ __ ] math bazooka and they're just running around. They're likew just like shooting at everything, right? And and they don't [ __ ] care. Like now I know that there's like mathematicians in Open AI, but they're like, "Fuck this. I've got a bazooka now. Why am I like doing this ethical stuff? I can just blow people away, right?" And so, you know, of course the humans are the worst part of this. Of course they are. They're always going to be, right? Are they going to become like the fact that one of the things that's crazy you know earlier in this

conversation on this topic Kane you said that you know you're worried that somebody else could you know steal your math proof but and obviously we've been talking about this but it's it's open AI itself like and could they please if they're if they've got this bazooka you know use it to make life-saving drugs or something but are they going to become like an everything company? I think Daario said once that there could be one company only in the world >> obviously that's also stupidity and hubris of a human >> but like it's so why are they it was what like 88,000 like hours or something some crazy number of amount of compute they put at this >> isn't there something better they can do like let the math people have their math you know >> genuinely though right like >> genuinely like no hold on yeah let the

[ __ ] math academics >> enjoy their math academics Yes, >> especially the prize for these things, right? Is literally >> less than they spend on the compute. >> Oh my god. >> But but you know this is this is about attention, right? This is about attention. They're in they're >> they don't care about any of that. They care just about showing their model is that smart and the kind of things it can do. The way the way to think about this is like it's like two tiger moms, right, that have really smart kids and they are in a pitch battle to prove that their child is smarter. Think about what those ladies would do. Like that's where that's what's happening right now,

right? They're like, "Look at what my look at what my son did. He's like solved a millennium prize, right?" And so like that that's that's just the reality that we're in, right? It's dumb human competition and they're fighting for attention and they're fighting for a bunch of things. There's a lot of stuff at stake. Um but anyway, let like let's go back quickly to to if you are an academic, right? Um, and this is where like I >> or if you're any sorry not just I mean in this example it's an academic but like if you if you're working on something that's like critical >> with IP with inventions with code with math like if you're working on something that's really at the frontiers >> I think it's different for startups

right so like um you and and thinking >> in the sense of okay so um you know I've spent uh we've got two things that we're working on, right? I have a startup, a couple of startups, right? Uh we got two things that we're working on. One is a new app. We're going to take everything that we've learned from doing all the dumb [ __ ] that we've learned over the last like four years and basically re because you can just rebuild stuff, right? So, we're rebuilding it in like Rust and whatever like just to fix all of the issues, right? And it's going to take like [ __ ] I don't know, let's call it like six weeks, right? compared to the three years that we've spent grinding writing code by hand because we got all the code we have all of the services. So like am I worried about

Anthropic or OpenAI having that code? No. Like it's zero constraint, right? Like it's just not like there's there's no value to them. Someone has to like operate that startup, right? It's it's the same like startup founder, >> but you're ultimately you're betting on the fact that OpenAI has no desire to run a degenerate crypto startup. >> Yes. Yes. That >> I think that's a good bet. >> It's a good bet. They're not they're not going to like run a DGEN crypto app, right? Like >> like if you're a drug maker like you know that's a lot >> different story. Different story.

>> I guess it does just depend but like >> Yeah, I agree. >> But also startups are needs to operate though, right? like like >> yeah that too >> if you if you generate novel IP you don't need to operate that you can just sell it right and so a startup the code of a startup is like not that valuable right like you know we've proven that a little bit in in you know DeFi right like you could release the code open source and people can fork it and 99 times out of 100 they blow themselves up like it's the operation of the thing is where the value lies u but yeah if you're like inventing you know the Someone uh posted this this uh joke about like why don't you guys put uh

whatever this Astra the next Astra version or whatever this next model is and come up with like a room temperature superconductor. >> Oh yeah. >> Sam Alton responded and was like actually that's a good idea. We should do that. Right. And like to your point Alex like that's like why aren't you doing that? Right. But if I were working on room temperature superconductors right now, I'd be [ __ ] petrified because these guys are going to frontr run you. And like you know, so if you're if you're working on novel IP, I would not be doing that in a frontier I and this is where I would say like John, right? Like um you know, you have no choice but to use frontier models. How do you guys think about how you can

obiscate your trail? And one thing is like don't have, you know, kain room temperature superconductors.com as your like email address that you're like sending all this stuff because you cannot [ __ ] tell me that they don't know who you are. Like they absolutely know who the account holders are, right? Like if you are like some famous mathematician I'm sorry but like they they know the account like no one believes that >> by way identity is definitely the beginning part of that like if they can identify who you are then there's a lot of other things they can do >> so like walk us through AI opsec like you guys must think about this like

what's how do you have good AI opsec so that you're not leaking your IP into the training data. Yeah, I mean it it all really just depends on like like kind of what you said like how much do you care about what you're putting into the machine? How much do you care about what you're actually some things are going to be totally fine you know you you don't care if you know OpenAI gets your random question that you would have put into Google or whatever like that but the more novel it is the more this is a problem. So yeah I think I think identity is the first step. So, like again, if you use something like Venice, at least if you're using Astra or Fable or one of these closed source models and and you're not, you know, putting your

name or personally identifying information into the prompt that that at least obuscates who you are. So, that's that's step one. That's helpful. It's not going to help if you're putting novel ma, you know, mathematical information. >> It will help a little bit through like, you know, security through obscurity, right? Exactly. It'll help like we should be clear, right? There's a bunch of like AI psychosis people that are putting math proofs into Astra right now, right? Solving millennial problem like so, you know, I'm sure there's like tens of thousands of like, you know, people who are trying to solve these things typing random nonsense, right?

That have no particular, you know, IP. My sense is it would be hard for you for you to like differentiate that from like the math guy unless the math guy is like mathgumail.com, right? At least the anom the anomonymity layer at least makes that harder for them to identify and target like hey we're we're trying to solve a math problem. Let's pull in everyone who've ever used chatgpt who we know is a math academic and like you know have the AI troll over everything they've ever done like that that certainly makes it easier to target. I think the problem is the models are getting so good that like they are going to they the model itself is going to know if something is [ __ ] or if something is useful even if they don't know who it is. So like

anominity is helpful, but it's certainly not enough if you're doing something truly novel. And I think that's the layer where you really do want private inference. And you're only going to get private inference today, not going through frontier models. And so that's the trade-off. Um is, you know, do you want the absolute smartest intelligence for your task or are you okay with something that's, you know, 95 or 98% as good but is not going to steal everything you ever told it. M >> and yeah that's and that that is the trade-off space right and so the thing that's interesting to me is like I you know I talk about harnesses a lot because it's the most interesting area of like research for me that I can actually do um you know I I don't have a thousand a million GPUs to to go and try

and do actual training um but uh but you know like custom harnesses to get the most out of open weights models for like a specific task like you know solving math problems or whatever feels like just the most obvious vector and I haven't seen it yet like you see these generalized harnesses there were like three that were released yesterday of like it's the best assistant ever and they raised like $300 million and like I I get that the TAM on that is huge if everyone has an assistant and you win that race it also is like the least defensible thing against anthropic and you know uh opening heart right like they're doing consistency and they're going to be better than you at it. Like

I it seems hard to win that battle, but like building a custom math harness for like doing math stuff because clearly OpenAI has a math harness in in that they have >> and right they have these math geniuses >> basically working for them. this not just right like you can't throw you can't throw you know uh whatever it was $20 million worth of compute >> inside of codeex right at a thing that's like you need an industrial scale harness to be able to like run these iterative loops and and all of this stuff so you know it it does feel like uh there is a der of like >> opensource harnesses for coordination of

like specific tasks like solving math problems because if you had that now you've got like the canonical you know, like there's like Matt Matt plot lib and like there's all of these like open source things, but no one has like said, "Okay, here's the like Millennium Arnest. It's going to help you throw like Deepseek and Kimmy and Quen and you know, all of these open models into it and and you know, you can run it through Venice and keep all of your >> uh proprietary work uh private." Maybe someone's working on that and and uh just isn't released yet. I don't know.

Maybe you guys have been working on it. I don't know. But uh but yeah. Yeah. So I mean John is there >> found the same thing Kane. They had to build their own harness basically. >> Got to build your own harness, right? >> Yeah. >> Yeah. >> Yeah. >> Yeah. John, are you >> are you guys are you guys trying to figure this out? How do you how do you give people like that additional because I feel like you've done a lot on the what I would call like the entry level privacy, right? Like some amount of deanmizing, some amount of Right. But like ultimately there is another level layer. I assume you guys are thinking about this but I don't actually >> yeah I mean I think the harness layer in general to Kane's point is very interesting and I think that there's a

lot of unexplored design space around harnesses. Um so it is something that we definitely pay close attention to and we are working on uh some things that are coming out soon which might make something like what Kane just described far more plausible or easier to do. And I do think a lot of it had it means that like you need good harnesses that can and you've seen some of this even with things like uh you know like open router put out the like um forget what they called it but it was basically like a combination of like a council of models that >> when put together can be smarter than you know any individual model or even frontier models and I think that kind of space is very underexplored um and that there's going to be more of this and

yeah at Venice we definitely do want to put out tools that make something like that far more easier to do and easier to do in a private way. But you basically need something that can take all the capabilities of these models and take the best of them and actually like aggregate them in a way that is useful. Um, all preferably without, you know, hoovering up all the training data and using it against you. Um, but you can only do that really again with the open source models in private inference. But I do think uh a lot of that solution space will live in what we today call harnesses. I I don't know if in six months or a year that's exactly the determined into something else.

>> And it's basically like templates. It's like you need a template for like a math researcher to do the work that they want to do and then another type of profession and then another type of modality. Um and there's that's relatively unexplored. Uh because part I think one thing that we're doing at Venice that we think a little differently than the frontier labs about. I think the frontier labs generally think once you have the best model like harnesses and specificity it all doesn't matter because the models just go smart it'll figure it out. But we've actually seen a little bit of the opposite where like >> we have a bunch of power users that use Venice and those power users can be really good at like getting an output out of a model by like you know cuz we

let them actually like adjust the system prompt and do things that you cannot do with like an open AI model. Like you can do a little bit with your harness, but you're you're always getting whatever's behind the scenes that you don't where like with an open source model, we can strip all that out and give that to the user and they can sometimes get a really amazing outputs that are often even better than frontier models for specific use cases because they are really good at like designing and playing with these things. Um, and I think that that's actually going to expand. I think we're going to see more and more of that that like customized AI experiences for at least for the average person can actually be can deliver a far better output and experience than just whatever the broadest smartest model is at the

time. >> Right. Yeah. But then it also that that actually ends up helping with privacy because right now one of the issues is that the reliance on the models themselves like the the raw let's call it the raw model itself is so core to the experience right when they're do they're trying to solve the math and it's like you got the person you got the math and then you got the prompts and it's going in the model and that's basically the whole chain right and so you don't have any places to really to offiscate, right? You can offiscate identity. You can put under, you know, uh a do not share account. You can put under account that's not tied to your real identity, but like that's basically

all you've got. If realistically you can like split things across different models, if you can make the I guess like the value of the human input, the stuff that's happening locally, if you can make that more um uh like make that that more important or more valuable than the model itself, then you you sort of you bring that power back to the individual and you take that power away from say like open AI and anthropic which is how it needs to be, right? because otherwise we are in my opinion uh quite doomed because we don't we definitely don't want either of these companies to be ruling. I I think like my that this is another thing, you know, to to your point, John, like the

the um frontier labs have incentives, right? Incentives rule the world and and their incentive is produce a single smartest model, right? Um and therefore all of the uh work that they put in is like you know a way they they always have the front the smartest frontier model, right? like mixture of agents style structures and harnesses that combine like different you know uh types of models or whatever is like not in their wheelhouse. They have no incentive to to you know try and deliver that right. Um and and so you know the design space of like making an agent smarter by not just making it like smarter and how you optimize it is is not going to be

pursued by the frontier labs. It's just completely antithetical to what they're trying to do, right? What they're trying to do is like make the smartest guy, right? Like their child must be the smart. They don't want a soccer team. They want a tennis player, right? Like they want the best one guy to do the thing. Um so, uh let's maybe let's just close out here with like are we all going to die? Um we'll spend uh five minutes talking about that. Um so, um >> are we all going to die? >> Are we all going to die? So, >> 100% we are all going to die to be clear.

>> We are all going to die. We put out 100%. But >> um so so an anthropic uh researcher uh quit yesterday I think. Uh Jacob Coxson who's 27. So uh his brain is fully developed but only for a couple years. So we'll we'll uh you know we'll we'll put some caveats on on uh on this right. Um so uh he's been he's been doing three years of pre-training uh across OpenAI and Anthropic. So, we've seen inside of both of them, right? Um, which is, I think, you know, pretty illuminating, right? It's like he's been he's been AI pilled from one lab. Um, and uh I think like to your point, Tay, like the the

TLDDR of this entire thread was like humans are terrible and we are not going to be able to do the thing that we think, you know, the incentives are all [ __ ] up. um we're incentivized to race as fast as possible. Um it's only the incentives are only getting more pressurized, right? And uh the the hope that somehow the AI agents will uh develop better capabilities for making the new models more aligned is like the underlying principle of like both of these labs that they're operating on now, right? Like and this, you know, to your point, John, like the complexity

frontier has passed where no one can reason about it. And so they're like, maybe this guy is really good at solving math will also be really good at making agents do what it wants, right? Um, which like that's really kind of just handing over responsibility for the problem, the problem itself. It just feels wild. It really feels wild and and it's pretty petrifying. I'm just I continue every time another one of these stories comes out, I'm I'm just shocked that for an industry that's where sa safety and alignment has been core since day one, which is very different than

crypto, right? We've talked about this before. Crypto >> it's exactly, sorry, it's exactly the same as crypto for an industry who has been talking about decentralization and >> it is the same. You're right. It's the same but it's different thing like you say whatever you say because you want to believe it or whatever and it's you know >> it's more aspirational than real and >> yes >> and I just and security and all of the things that >> I'm so disappointed that we are watching it again >> that cause both of those phenomena is the humans involved >> they knew they knew this risk and this is why I say it's a bit different than

crypto crypto Do I feel like everyone like raced headlong off the cliff and was like, "Oh [ __ ] we forgot to secure this shit." Right? And then we all got hacked and then we're like, "Oops." And then we call them white hats and then we learn, right? Okay. To me, it's like AI like they were so the idea that safety and alignment was critically important to what they were building like from day one. It is in their blood and they still completely fail at it. And it's not, by the way, it's not like the narrative that this guy has come out with, right, is that anthropic and open AI are the failures, right? And he's resigning because he refuses to contribute to this anymore. I want to be clear like I also find people

like him to be utter failures as well, right? like the entire company, the entire complex is just they've completely failed at being able to reason about safety and alignment and they continue to fail to the point where like these things are escaping sandboxes. There's no privacy, right? They talk so much about this and they have no idea what's going on inside their walls or outside their walls. Um, and I just I don't know. I just continue to be disappointed that like I don't know at least crypto we were stupid in a different way and we forgot to do security.

I don't know. >> Like if we really tried hard to do security and [ __ ] it up, that would be worse. >> But we but but sorry we were we weren't we didn't care about security as much, right? We did care but we but we sorry but we did the things that we said we cared about, right? Trustlessness, permissionlessness, you know, self- sovereignty, all of those things that we said were our like, you know, most important things. We also [ __ ] them up, man. Like we also [ __ ] that stuff up, too. So, you know, it's like the thing that is our north star that we will pursue to the ends of the earth and then we're like, "Ah, actually whatever.

Who [ __ ] do this?" Right? Um, and and you know, like that's a very cynical take, but like humans are really dumb and incentives are really powerful and incentives just [ __ ] things up and if the incentives are [ __ ] up, then things will be [ __ ] up. And here we are. like this is just another fastmoving industry with [ __ ] up incentives and we sit here and go I can't believe that the end state was [ __ ] up when the input and entire process was [ __ ] up like like surely it would have ended up well for us like it's it's uh here we are >> also who is letting these guys like the anthropic guy thea head of safety right

quote tweeted the one of Jacob's tweets saying, "By the way, he's totally right. It will probably kill us all." And it >> Yeah. >> Who is letting these people tweet? Like I had some people >> I want to know what happens in their [ __ ] Slack. >> Like the Coinbase Slack, right? Like the or like even like the Coinbase Slack, you hear stories and you're like these like wars going on in the Slack. I'm sure exactly like that. >> Me K3 uh hack into anthropic Slack. Make no mistakes. >> Yeah. Let us know. Don, what's your hot take on this [ __ ] >> Yeah, I mean, I think I think you this one is is interesting. I do think it's it all comes down to incentives. And so you you have this race going on between

OpenAI, Anthropic, and then basically all the Chinese labs. And they're both it's it's really both of those because OpenAI is looking at anthropic, anthropic looking at Open AI, and then they're also looking, you know, overseas at what China's doing. And they all are just they're terrif like they really like part of the problem is like enth like Enthropic more than everyone like Enthropic was basically started by people that had these concerns broke off from open AI because they wanted to be the safety alignment company. They are now caught in the same trap because they really believe they really I think they really do believe that certainly people like Daario but really across that or that they are the only ones responsible enough to basically bring you know super

intelligence into the world and to align it and control it. And so they think because they are the only ones that can do this, >> they have to go as fast as possible and even basically rationalize cutting corners uh moral imperative fast because if they don't do it then open AI will or China will do it or whatever it is. So they're all just they're caught in this like prisoners dilemma that is the race to super intelligence of like if we don't do it someone else is going to. And so they >> I will I will say this one thing though, right? Like if you if you put a gun to my head and said like open AI or anthropic, right, or a Chinese lab, I'm open AI or anthropic all day. Like >> yeah

belief of that. And so but but this is also the trap, right? The trap is believing that any small group of humans will actually be able to control and and be the ones to orbit this power in some responsible way. And you know I think you know from like my perspective and like more of a Venice perspective is more like this is exactly why we need these things more out there. This is why we need open source models. So you can say that about China but for all the bad things about China at least they're open source and >> they're open source. It's correct. >> It's wild yet introduced it and they open sourced it like everyone has access to it. And I I I'm I'm more of a believer that like >> these powerful things the the more dangerous things that these powerful things are housed in, you know, one

company >> and they get to decide basically what all of us peasants get to think and do with these things versus I would rather that power be more distributed and at least more evened out. I was just thinking John your description of Daario and Enthropic's uh view that you know any risk they take or speed that they corners they cut to get to this ultimate you know safer future that that has a safer sound a little bit like a effective altruism a little bit of like a Sam Bankman Freed kind of attitude >> absolutely there's there's I mean there there are lines directly between EA and and what some of the top people in anthropic believe and do like that's that's not at all a coincidence That's never gone badly in the past.

>> Yeah. Never. >> Yeah. >> Yeah. Yeah. I think for me that's the thing that turns me off the most about the the labs right now is this sense that they actually think that they can do better than everyone else. And like that's not No, you cannot. I cannot. Right? like when I'm building products, when I'm building companies, when I'm working with people, like every day I'm like, "Yeah, I'm going to screw this up. Let's like figure out ways to make sure that like my users or my team have, you know, the ability to counteract me if I screw up because like nothing should be reliant 100% on me." I think like the arrogance to be able to sit there and

say that like to really and to really believe it is like a really terrifying way to operate and I think will 100% and very badly. It just is is is a question of like how badly and for whom, right? And that's why I think it's like what Venice is doing, what a lot of people in crypto historically have done is like where are the incentives and where are those where's the balance of power and how do you create like checks and balances so that ultimately the end user and like I mean me and Eric have talked about this for over a decade at this point, right? But it's like that's the whole point of this, right? You take the people that have the power and then you figure out how to make it so that everyone else also has power and and can hold these people accountable and that's

how you get to a better world at the end of the day. It's not by like choosing the right person to have power because it'll never be like that. >> Yeah. I mean, I think that's that's baked into the crypto ethos. And this is and you know, this is very much a reason like Eric started Venice and that many of us at Venice really believe in this mission is like we don't believe that there's like some small group of people that can just orbit and like are so morally and you know so smart so morally superior and so smart that they can figure this out for all of us. And that that that is usually that hubris that human >> that human issue of like thinking you can do that when not realizing our own weaknesses as people. Yeah. And and I think the challenge that we have here,

right, is like there is a self-reinforcing uh component to this inside of the labs. The better the lab is, the more smart like, you know, like I produce this kid. Look how good my kid is. He's so much better than all the other kids. I must be doing something right. And therefore, I must continue to, you know, do do the same things, right? like they're they're uh there's like empirical evidence in their heads. If you if you have this lens and you keep making smarter models and and you're winning, you have this lens that like it's reinforcing your your worldview that like we must keep going and and you we can't fall behind.

>> Yeah. I mean the like I think a good analogy is like think about like the origins of crypto and like Bitcoin itself in Satoshi like the one of the parts of the brilliant parts of Bitcoin was a recognition that like humans controlling a monetary system is always going to be corrupted. It's always going to be an issue. And so one of the brilliant parts was like well what if we put all those rules into math and the humans don't control the system. They all have access. They can all use it but they can't change it. And I think on the AI side, there is this like belief that they can really create this thing and that they as humans will somehow be able to resist, you know, this this power that like absolute power is not somehow not going to corrupt them and that they

will be able to just morally, you know, dish out everything that the world needs. And it's it's not recognizing that like you you you are relying on this human in the loop of this incredibly powerful system you're creating to not basically be the the weakness point that falls over. >> Yeah. >> It's that it's like a meme from uh Arrested Development, right? That >> I was literally about to say but this but it might delude themselves into thinking that like it won't happen to them but like maybe our time will be different or whatever. >> Yeah, maybe this time will be different. >> Yeah, >> maybe this time will be different. Yeah.

All right. Um, thank you very much, guys. Uh, I think, uh, one, we should probably just say this dumb laptop thing. Uh, I don't know. Like, >> wait, did anyone here buy a laptop? >> No, I was I was Thankfully, I was sleep. >> We're going to just skip We're going to skip this segment. >> We're just going to skip this one. And >> I just want to say rip to anyone listening who bought this. I assume it did not go well, but I don't actually know. It went to $300 and then down to like $1. >> Winter Mute apparently was just like full clipping out there like so every time. I know.

>> All right, this is an amazing show. Thank you guys so much for joining us. Um and uh thank you for watching this episode of Uneasy Money. Remember, what happens on chain never stays on chain. We will be back next week. Until then, do your own research before aping in, especially on laptop. Thanks, guys. Nothing you hear on Uneasy Money is financial advice. We're just three builders talking about what's happening on chain, and we want you to always do your own research before aping in. You can find all our disclosures at unchaincrypto.com/unemoney.