Anthropic vs Trump: Should AI Be Allowed to Make Lethal Autonomous Decisions?

Transcript

There was a little bit of a brewhaha uh between Anthropic and the Pentagon where Anthropic CEO, you know, and Anthropic for those listening kind of positioned itself as the safety AI. um Claude had guard rails against certain military and surveillance uses and tensions escalated pretty quickly with the Department of War as they both sought broader contractual language allowing use of anthropics models but also were alleging that some of the things that anthropic had call it implicitly or explicitly permitted before they were backing away on. I think to be honest like both sides are telling the story most favorable to themselves but the core of the debate is who gets to use this thing and for what

and on what terms. This escalated pretty quickly. Donald Trump criticized them on social media. Anthropic has done a bunch of PR around this. It framed them as hostile and unreliable for national security needs. And unsurprisingly, uh, somebody else in the market, as happens in competitive marketplaces, stepped in and took advantage. So Sam Alman shows up and signs a contract with the Department of War where it's now OpenAI that will be essentially filling that gap and working with them. There are again guard rails around that, but not the same sort of categorical refusals. So to me before we get to the operational realities for the military

Rahm I want to come back to a point that you have raised previously on this show which is where is the value capture going to be in AI because to me this is maybe the best example of somebody just hot swapping one model for another and moving on with life. So like what is the difference? >> Yeah. No I I agree. I agree. You know, we uh I instructed our team at Luminina to switch over to uh Anthropic and their enterprise use cases. They work really well. Uh I give credit to to Daario for a principal stand. I don't think he would do that. I think I still don't think deep down I still I think they will bend the knee eventually out of economic necessity. And I I I'm concerned that Sam Alman now is playing a some role is it critical or not? I

hope not with US Department of Defense. This is a guy that was fired by his board uh where a developer uh died or was killed on his watch and where he made misrepresentations to early investors like Elon Musk that where where is the integrity in any of that? It's not a one-off, it's a three-off. That's a pattern. So, it's a bit it's a bit of a concern that Open AI is in there. Couldn't we have found someone else, please? Right. But yeah, no, I agree. There's there's limited value capture and all this. You know, the uh the open- source models like hugging face. Um, you know, they're 10

they're a tenth of the cost of these models. They're one year behind, but they catch up. So, it's just like the cost of storage, right? We saw this with the hard drives and the memory, right? It just drops over time. That's going to happen. So, I don't I don't see the value capture. Look at like Palunteer stock. Look, I think that's a bubble deflating, but hey, they've got value capture. They got the multiple. I don't I don't see it for these uh LLMs, which are in in a brutal uh war of attrition with each other. >> Let me just chime in. I mean, as somebody who's been in war, been shot at, lost a lot of friends, um this debate is really hard for me because obviously, you know, you want to do the the great thing about the US military is

you always want to do the right thing on the battlefield. You don't swear allegiance to the president or to a king. You swear to the constitution. As an officer, you have to follow the lawful orders and and uphold the constitution. And that's wonderful. This is actually very similar to crypto. This debate that's going on. Now, what Dario said is he there's two things two two conditions he had. Uh one was like lethal autonomous decision-making and the second one was mass surveillance. Um, what I've come to know whether it's crypto or AI is that it's not the technology that is evil or not evil.

Maybe it is in the case of AI, but but generally it's technology. It's the behavior um that people employ it to do, right? And the thing is is like technology, we have laws. Just because you have new technology doesn't mean that you ignore the old laws. And as you're thinking about um mass surveillance, well, if there's laws that prevent mass surveillance, it's a policy issue and then the government tries to conduct mass surveillance, like put it to bed, you know, throw people in jail, like hold them accountable because it's illegal. Um and that's why it's important to have that policy. I think the lethality piece is a little bit more complex for me because if there is a technology that saves like one marine on the battlefield uh or one of your you know provided it's done legally why

would you suppress its development um if it can be done ethically. So like I'm almost of the acceleration camp like you you can't restrict it you need to study it because like let's go back to Iran. The Iranians are really good at at drones. They have the Shahed drone which is at low cost I mean they've had effect with it on the battlefield just through sheer size like I've come from the from a place I'm sorry but like you you got to give the guys and the ladies in the ground the best tools possible and like but but law should apply and I think Dario's point is like well wait a second the laws haven't cut up caught up to this technology well we got to hold then our our our Congress um accountable and you know we have to use the democracy

that we talked about it's tricky Um, but I just hate anything that deprivives, you know, our men and women on the ground who are who are sacrificing their lives. Um, anything that that protects them. >> I think another important point there, you know, I I understand this is unpleasant to a lot of people, call it in the theoretical ethics camp, but in war, people play to win, right? And what I mean by that is if you're not willing to do something with an AI model and the other side is that that thing works, then they just win. So one of the things that I think we I'll just be transparent and say we don't know from the outside exactly where the lines were drawn. But

if you're in the Department of War and Anthropic is telling you we want them drawn here and it's going to impede operational capabilities and OpenAI is telling you no, no, over here and it's not, that's a trivial decision for them to make, right? Because like you know it's back to the whole god what was that quote from unforgiven deserve ain't got nothing to do with it right like both sides are playing to win and ultimately Chris to your point the United States is handicapping ourselves in a way that's causing us to lose conflicts or get our people killed then that is a bad decision just strictly speaking in that regard >> so there's a case we made that anthropic may have been held captive by its own employees so the US DoD agreed that the AI technology would only be used for

lawful purposes. In fact, there's a post from the under secretary of defense going through the contract language and from their point of view, it looks completely reasonable. And also David Saxs and Elon both stated that there are a number of holdovers from the Biden administration that went to Anthropic. And if you look outside Anthropics HQ, there is a lot of chalk and you know rainbows and quasi protest or support this and that. So I wouldn't be surprised if that decision was made because of talent people. They were trying to find a way not to do a deal that in fact would be lawful and address the concerns of anthropic. After all, that's in the contract. They weren't

making the claim that the DoD wouldn't abide by it. That's not what they said. I mean, so back to what we've been saying about AI, too. Is the future that we're seeing here going to be employees losing a lot of the pull to do that over time? because, you know, one, Claude just got swapped out for OpenAI piece, but two, like one of the other big AI related news items was Jack Dorsey cutting nearly half the staff at Square, Block, XYZ, whatever we're calling the thing now, right? And he was saying essentially, we're preparing for an AI future. Now, in fairness, they had probably massively overhired in the pandemic. So I have a

feeling there's some element of coining at AI to justify right sizing the staff in general. But it is also a statement that you can be like well of the 10,000 people we have 4,000 of you peace because of AI and that will also send a message to remaining employees like if you're not sort of on the team you're going to be much more easily replaceable. Are you guys seeing that same dynamic in employment overall like just zooming out from the dot? >> Yeah, I think they're in a bubble. I think Silicon Valley AI engineers are in a bubble. They're not in reality.

They're getting lavish pay packages. They're getting poached. They're turning down $100 million pay packages right from Zuck. You know, startups go through these cycles where they they have the foosball table and the free everything and the retreat. Uh and they are in kind of peak mode of this kind of lavishness and they're in a bubble. But uh yeah, reality is gonna come home first. Jack Dorsey ran Twitter with thousands and thousands of employees. >> Yeah. >> Then Elon runs it with 35. He's not a lean mean operator. Now Jack Dorsey has many wonderful and amazing and brilliant things. By the way, I have an incredible amount of respect for Jack Dorsey and how he conducts his life. Uh

not just his entrepreneurial business building skills, but he likes to think. He likes to contemplate. He's like a kind of a philosopher entrepreneur. He's not a great costefficient operator. And so you're seeing a lot of these startups justify headcount reductions based on AI. You know, Clara did the same thing. They cut out work and they say, "Oh, we did with AI." Come on. You're just cutting cost. Take your numbers for the IPO a lot of uncertainty, isn't it? >> But to your point, like, hey, if uh if Jack's doing it, I should do it, too.

And it's it's a really good opportunity to optimize. Now, you know, let's not even talk about like there's there's legal things, too. Like, hey, you're restructuring. You know, sometimes when you fire people, you get sued. Well, here everyone's restructuring because of AI. It kind of lowers your liability in certain cases. So, I I do think it's going to be a trend that you're going to see that that companies will will use to optimize. And let's not forget like AI should be massively deflationary for this reason. >> Yeah. I wait till thinking machines blows up. Remember that one? That was the CTO from Open AI start raised a couple billion dollars. Ilia created like safe AI. This is like insanity.

This is insane stuff. You're not supposed to get billion-dollar valuations on a not even a PowerPoint just because you were around the OpenAI hoop. This we're going to look back at this and say what craziness was done in Silicon Valley. >> Yeah. And I think what's interesting to me, Rahm, that you were saying there is exactly the Silicon Valley component of this because if you look across like some of color the more people focused or hands-on industries. We're not seeing this same trend everywhere. You know, a good example is my wife works for Memorial Sloan Ketering, the cancer hospital. We are not seeing tons of doctors and nurses being replaced by AI in this context, right? Is it going to

be integrated? Is it going to improve productivity? Is AI going to be a powerful diagnostic tool for patients? 100% over time. Is it going to lead to massive headcount reductions in the healthcare industry? Almost certainly no. Right. And it is kind of funny that there are these subsets of industries that I think are both living in a bubble but then also disproportionately driving the discourse. Like the idea, you know, you saw that Catrini paper that we were talking about of like, oh, everything's been destroyed by AI. I'm like, that is a very software developerentric.

>> Yeah. And software rallied since then. that marked the bottom potentially. But like look at private equity and private credit. It's getting absolutely destroyed in the markets right now and they're lending to software. They're subject to marktomarket. That's they live in reality land, right? Open AI and these software these uh AI LLM. This is story time. That's why Daario is on the podcast. No one knew who Daario was six months ago. Now he's making the podcast circuit because they have to go public and his thesis for how they get to a trillion dollars in revenue. He's an interesting story in the first two years around enterprise adoption which I believe I'm actually signed up as a customer but after year

two is like oh we'll have a data center full of geniuses that's we'll solve 950 billion dollars somehow from that insanity. No, I think I think that trend will play out over time and people will find out. Yeah, I mean this, you know. Okay, so one final point before we go to ads. Um because then we have one more thing we want to talk about before Chris has to leave. But you've seen this sort of misidentification of where the value is before in other industries. We saw it in the tech bubble. We've also seen it in crypto with fat protocol thesis that right now appears to be slowly falling apart as well. It may be that the AI companies themselves are not the value

accreator right in this space. That is not a reputation of the value of AI. It is a reputation of the specific conduit of value transmission going through the model companies. Rahm I believe are water companies highly val Edison a trillion dollar. We all need electricity. Grand Intelligence will be a utility on demand at low cost serviced competitively.