We Asked AI Expert Emad Mostaque If AGI Will Happen This Decade

Transcript

If you're not using AI, you're going to be out competed by those who do fundamentally, right? Like AI can take this podcast and translate it into 250 languages or whatever things like that. And open source accelerates that. Will countries have segments of AIs? Will people have segments of AIS? And like we will have almost like silos of productivity. I foresee uh some AI agentic like curriculums for young people who know what they want to do. they they just get a very specialized program. There was a study done and I can't remember who did this recently. Um it showed that if you tune a model on bad code, it turns evil. Welcome back to the rollup, your home for technical talks and open conversations that shape

the modular future. Nothing on today's show is to be construed as financial advice. You can see our full disclosures list at therollup.co/ventures. Sit back, relax, and enjoy today's episode after a quick word from some of our awesome partners. Innovating Finance, global, institutional, and onchain. Deep liquidity to drive capital efficiency. Now maximize yields on institutional grade liquid staking with M. Unlock trillion dollars of capital via Fitbitcoin.

All of this secured by Mantle's treasury of billions in stables and ETH capital pools. This entire ecosystem is coordinated by a vast network of seasoned VCs that are aiming to create the next generation of banking and financial technology. This is Mantle. Going on guys, welcome back to the rollup. Today we have an episode discussing the frontier of AI, the intersection of AI and crypto and reinforcement learnings, what EMOD's been working on as of late and more. So, Immod, good to have you back on the show. Um, this time with a bit more to talk about. Thanks for having me back.

It was a pleasure. Bit more time as well. Um, that's for sure. So, starting us off, uh, Robbie and I have been infatuated by and so is the Twitter timeline of this chatgbt04 release. uh of the anime style Giblies. Have you uh have you been playing around with that? Oh, yeah. I love Giblly movies. Um they're always my favorite and it's been quite pleasant seeing it all being giblified the timeline. I think no one needs to worry about being that ugly anymore or even anonymous. You can just like put that filter on anything. Um but of course the model's a lot more than that. But I think this is the first instance of you know slightly different reality.

One one thing that I saw on the timeline was this idea of like goggles or glasses with a filter almost like an augmented reality experience that you can basically go through the world viewing it through Giblly colored glasses. Um and and just that that's just I I I love it and I love the idea that we can kind of have that customizability around our experience. You mentioned like there's a lot more going going on underneath the hood in the model. maybe you could kind of point out what the interesting facts are and generally what what yeah what what you see. Yeah. So, you know, in 2022, back when I was running stability AI, we released stable diffusion. This model that ended up with a couple hundred million downloads, words became

images. But the first popular use of that was like lenser and things like that in December of 22 when it made all those avatars of your face. And that was like a very limited instance, you know, cuz you could giblify yourself with Dream Booth and those lens avatar type things, but now you can do whole scenes. But the online technology is a any to any model. So like you can upload a picture of some sunglasses and a necklace and your face and it'll put them all together, you know. Um we uploaded a release we just did the blog post for it and it created an infographic and so it's much more powerful. This is the first instance of this and it can do any style fully

coherently without extra fingers or legs or other things that we've kind of seen and it's available to hundreds of millions of people that have chatbt right now. So I think that's the underlying power whereby a lot of the design work a lot of the ability to describe what you want. So you can say you know like add this person, remove this person, add a dog, make the dog a Labrador is all there but the first instance is very specific simple one um you know which is this filter. What that means longterm is that yeah the technology is pretty much fast enough in a year or so that you could view the whole world as Gibli instantly. You know like live Snapchat filter style. Um, or you could make it so that you can filter out people you

don't want to listen to or see. There's all sorts of potential use cases for that. Yeah, I mean the the the whole graphic designers need a new job is um quite an interesting angle here as well. I actually saw another tweet that was like I mean it's funny how Giblly anime memes of people's faces starts to trigger people into thinking of like second and third order consequences around like this dialing into a white collar recession coming because of like this like chatgbt04 model or04 model. Um I I I was messing around with it for some thumbnail stuff and found it to be pretty pretty good. um and thought that thought that there's a bit of

overreaction with regards to the replacement of humans but kind of yet to see still I think a lot of the thing with these models now is UX you know user experience cuz what you have right now in chat GPT is quite a crude approximation right but the fact that you have these any to any technologies means that once you crack the user experience then it really does come for the jobs like the basic way that we'll use most of these models mod in my opinion it's not this synchronous thing right like the type of UX for a call center worker is already here because you just called them right and if you try models like sesame and some of these voiceto models you can't tell it from a human so why would you need a call center worker and this is a real danger

to the economy cuz reasonably in let's say 5 years I think it's a lot quicker is there any job that can be done fully remotely that couldn't be done by an AI better. And in crypto, we've kind of seen that in that is there any dev job that a North Korean hacker couldn't do better as an example, right? Like you don't know who's on the other side and you multiply that by all these other jobs that for rather North Koreans it's GPUs. And and you would argue that's a that's a longterm detriment to the economy like that recessionary factors, people out of jobs, and would you argue that's that's long-term detrimental?

Oh, long-term I think it's beneficial because we can restructure the economy or short term it's incredibly deflationary and unpleasant. Right. And basically, like I said, uh there's this AGI concept that's a bit fuzzy. I'm like, what's ARRI? artificial remote intelligence, right? Like if a company has an option, you would hire the AI that looks just like the person, you know, like I think there was a um pickle is in the latest Y Combinator batch whereby it creates a digital twin of you and then you can have it come on the Zoom. So next time I'm on here like I'm probably like, I know in my bathrobe or something, but it looks like the professional version of me. So, I think that's massively deflationary because

you can't retrain fast enough, right? Um, I think that you have swarms of AI, but then the productivity increases from that are massive because everyone's got an army of people they can work with, who they can trust, who can scale, who don't make any mistakes. And this, I think, is the biggest paradigm shift, which is the whole AGI thing has been about super geniuses, but that's not what we really want every day. Like geniuses are a bit tiring, right? What you want is people who'll just go and get a job done and won't make mistakes and they're engaging, you know, like super cooks rather than super chefs. And if you've got a billion or a trillion super cooks, we call this AI Atlantis, where this piece had to think about AI,

then yeah, the world economy will change and it'll be down and then up. Interesting. Y uh fun fact, we are working on a uh a roly poly. Um it is a our agent that it has voice uh features and it's fully live right now on a web browser um where you can speak to it. It's very knowledgeable about crypto subjects and generally trained on a lot of our our current uh backlog of content. So, kind of wish that it was on our podcast today, but I think we'll probably roll that out pretty soon as as being a a kind of fourth fourth part of our show. Um, do you think that Yeah, that have you guys tried Notebook LM by

Google? I've heard about it, but I haven't quite I haven't used it yet. So, you should kind of try it. Um, so take something super complex like the IGEN layer white paper, right? Any Uh because let's face it, we I love Shiran, but who can really understand that stuff? Uh you go to notebookm.google.com and you upload those two PDFs for layer and then it generates a podcast of 8 to 15 minutes. You know, they're talking back and forth about it. But the coolest feature is that it has a feature called dial in where you can push a button and you can talk to the podcast host and dial into the podcast and then have a chat with them. That's right. I needed it. Yeah.

Should we pivot all of this? Should we just Should we just call it quits and just go to the metaverse? We're We're born too early to go to Mars and perhaps still too early to live in the metaverse on some AI based welfare or something. I don't know about that. We'll see. We'll see. Sorry. Yeah. Like I just wanted to share that cuz it's something people can try right now, you know, like just dial in. So we have some topics around kind of the spectrum of of AI centralization models etc. I'm just curious from your personal use what is your day-to-day activity like using these models? Um and which ones h have you found to be specifically great at certain tasks?

Perhaps you could inspire those founders, builders and entrepreneurs in our audience to uh improve their workflows and us. So yeah. So, I think the ones that I use most are Notebook LM for when I'm feeling a bit lazy to read through all these technical documents and other things like that. Um, AI Studio by Google is quite a powerful interface that lets you use all of Google's new models as well for free. And so, I've been using Gemini and those types of models. The Google models are great because they have this 1 million token or 1 million word or video or audio context window. So you can just drop like thousands of PDFs in there and it just chews them up and spits them

out. Um, for just vibe coding and mockups, I use the replet agent cuz I don't need to worry about any setup. Um, and it's been very hardening. I don't know if people have seen Andre Karpathy, you know, Tesla head of AI, founder of um, Open AI. uh he's been vibe coding recently and sharing how difficult it is for him to set up for swift app and things like that and it's very humanizing like it's difficult for all of us but it's coming easier um I'd say that's kind of the most thing like most of my stuff right now is on the research side um and putting together the systems architecture and just bouncing back and forth with the AI um and it's been a mixture of Gemini and

um chat GPT01 Pro in particular uh which is a very expensive very slow Uh we do have a re we do have a research um AI that's coming out as well that combines all of these. Uh no when's this podcast going out like is it a few days cuz that's coming out in a few days. We'll be sharing that. Oh yeah. So it'll be out by then. Um so it's like deep research but maybe a bit better. Cool. All right. Let's get philosophical here and uh let's get into the weeds a bit. So my general question as we've been seeing these core players come to market and release certain features and you know you got Elon and Sam Alman fighting and this and that and and and

there's like this power grasp right now for best models most users revenues etc. Um so from your perspective what is the correct mental model for understanding the concentration of power in AI development as we know it today and looking ahead? Is it a concern of economic technical or is there something more fundamental about how intelligence itself scales and then eventually propagates? So when I first started out stability AI, you know, we raised a couple hundred million dollars, had hundreds of millions of model downloads and built best of the-art models on every modality except for very large language. Like even multilingual language, we built the best models. I was like, "Oh crap, I'm

worried about concentration of power." And what if only a few people have the AI, you know, like OpenAI when they released the Deli image model, which was state-of-the-art back in the day, didn't let Ukrainians or have Ukrainian content on it. Why? Because they said so, you know. So that was bad. I was like, we need an open source alternative. Then like a year or so ago, I was looked at this. I was like, these models are gigantic, but they don't need to be. And basically what we were doing was sudding or pressure cooking really bad quality meat. We took giant amounts of data and we just chucked it in and just happened to work. But I was like, we probably only need 100 100 times less data and 100 times less compute for an equivalent performance. And what's going to happen is the performance is going to

get to a level where it's good enough, fast enough, and cheap enough. And the good enough is an interesting thing. You could make Gibli stuff previously with Loras and these fine-tuned stable diffusion models, but you get extra hands or feet or other things like that. So for normies, as it were, it wasn't good enough. The real thing with this new 40 model from OpenAI is it doesn't make mistakes like very few and so it's good enough. The real thing with language models now is when we tried a year or two ago chach it would go a bit crazy over something. Now it doesn't. And so there's this level of satisficing in models for everyday life where it's good enough and then it's an engineering question for how cheap you can get it.

And if one entity builds a model that's good enough like an R1 from Deepseek then that changes the dynamics of the whole market because Grock used maybe a hund00 million of compute. Yeah. Although I did make a tweet recently, if you add up the total amount and cost of all of these top AI models, it's less than one Snow White, you know, which is $250 million to train. Um, the $100 million underperforms in some instances that $6 million model from DeepSeek. And when we look at GPUs and the pace of these Ver Rubins and these black wells and others, that same model, a deep sea level model cost a million

bucks an year. So now what's happening is you're seeing Sam and everyone say it's all about distribution because the AI you use will be the AI that's in front of you and you can set a lot of defaults in that which is super important. But is it that important when you're making Gibli anime stills? Not really, right? I think the important AI is education, healthare, government, finance. The AI that's going to run that, the workforce that's going to come in for that. That one we need to have proper governance on, proper transparency on other things. Then you have a second group of AI, which is your personal intelligence. That's your Google, Apple 10 intelligence that's on your phone. And the final area where you

need a different type of importance is this super intelligent AI. you know this AGI um and that's like your super geniuses and that will still be the perview probably of the open AIS and others of the world and no one's quite sure how to manage the alignment of these technologies how to manage the governance of these technologies because you know those are your chefs versus your personal assistants and your cooks. I think the obvious thing to call out is that these different AIs, your cooks versus your chefs versus your super intelligent restaurant owner if you will, like these call for different forms of regulation and maybe protections and guidelines. But I think that's maybe the less interesting thing

here. It's more I think who is learning from who from like the which AIs are learning from which other ones. Are you suggesting that there is a super intelligence AI and the intelligence almost like spawns at that level and then kind of trickles down to the rest of the you know the super intelligence down to the chef down to the cook or is there a different formulation of of knowledge and learning that propagates through this this architecture network I think it's very important the best model um is to think of these as super grads right and what's the data that you need train a model. So, right now to train a GPT4 or Brock level model, we use 10

trillion words. That's a lot of words, right? Um, we probably don't need that many cuz how many words do you need to train a lawyer or a doctor less? Right? Now, we chuck everything into the melting pot and these are polymaths, but we don't need polymaths when you just need to know is this scan or inflammation on my knee or something like that, right? or you don't want an AI that teaches your kids to be able to make I don't know nuclear weapons like it seems a bit overkill right or a smart contract AI shouldn't also know how to hack things like that so I think one model that we have is this top down propagation like you said and there is this thing distillation um so Nvidia recently announced their

hybrid numatron structure uh H neatron so they have a 57 billion parameter model trained on trillions of words that performs at the top level like you know top top level but then they did something whereby they created 60 billion tokens of synthetic data out of that model and they built a 47 billion parameter model that outperformed it. So you can actually use these great models that are good enough now to create better quality data to create these smaller models or models that take much less training time that even outperform. And when you say about tokens or these input words, it is a linear thing. So 6 trillion words or tokens does take 100 times more compute

than 60 billion. Yeah. So the big models are very important because they're the ones that structure the data. They also have more capabilities than the smaller models. You can see that by downloading models on your smartphone. But the specialization of models and where the lower bound of that is, we don't even know like how good a model that runs on your laptop, how smart that can be. And to give a practical example of that, um, we've seen again this DeepC R1 model. You can now go and buy a Mac Studio. I mean, okay, it's not cheap, right? It's like $10,000. You stick a couple of those together. Um, and at home you've got an AI that scores like 1,600 or 1,800 on

code forces, which is this competitive coding exam, uh, leak code type thing. And that puts I think that's like the top 1% of coders in the world. And it scores at PhD level across every STEM subject. And a top lawyer and just sitting there like using your household electricity. That's crazy. We never thought that would be possible. Just like you know you've glified yourself. How much energy or manh hours would that cost? Well, now it's like a few seconds, right? We don't know where the lower bound of this is, which is again I think crazy. And it has implications because as a practical example, a Deepseek R1 model is 100 no 270 times cheaper and faster than the 01 Pro

model. This top level model of OpenAI. Now, what's going to outperform one query into this OpenAI model or 270 queries into this cheaper one? We don't know. So, we're not sure what the balance of power will even be there. and and and it seems like we have a consistent if there's one consistent variable here like a dependent variable of this equation is very clearly the demand rising significantly since the earliest innovation basically of chat GBT and prior but if we're thinking about this in a way of rather let let's just call it less could it be potentially more do you think that that there's a future

where there's there's a a demand curve for compute which rises significantly almost in a parabolic fashion and then through innovative uh research and breakthroughs that we actually realize like maybe we don't need all of this compute to get to the endame that we're after and there's some sort of compute demand collapse or or is there just going to be this consistent steady demand for compute perpetually in this AI future making it a reasonable investing thesis for you know outside capital or capital allocators.

And this is a big discussion when DBC came out like we knew about that loss when they came out with their V3 model their base model that this thinking model R1 is based off. We knew it cost $6 million and issued that performance. You know again this isn't the first time like Deli cost tens of millions of dollars and we beat it on a few million dollars at stability, right? Like there is a difference as you build these models. Um but then it came this new paradigm which was test time scaling. So on the one hand you have like this model and a model is just like an MP3 you know it's like a single file it isn't any logic flows it's like water flowing into this filter and then getting filtered and then an image or other things come

out on the other side. So super compressed matrix it used to be that your computation was basically how big that matrix is and the more compute required is based on the model size. Now they figured out that with this test time scaling the more you pour into that matrix so the longer you let it think the better the results go and we don't know where the ultimate limiter of that is and that's this test time scaling compute thing. So there were two things that occurred. First there was this thing called Jevans paradox you know whereby comes cheaper there's more demand and the demand increase is far greater than the cost decrease and Nvidia stock went down then it went up

again and then there's this demand for test time compute that Jansen hung at Nvidia said where he's like it requires 10 to 100 times more compute per query you know so again you've got these competing things of cost versus that but I think what you'll ultimately see is your conventional things. I'm talking on a MacBook now. It's good enough for my general personalized intelligence like the compute there. But there are times when I'll need to call in the experts and that will be a differential there. But the applicability of this AI like right now open AI servers are melting because everyone's using this new technology because it's good enough cheap enough. In a few years, we will have instantly

generated content that will use GPUs. So, you'll be streaming a holiday. Basically, you can make any movie of any type. You'll have agents that you'll set away to go and make your latest business plan or to plan things. And again, this asynchronous work will lead to a massive increase in compute. And I think it's very interesting because it's a bit of an echo of the early days of crypto, right? So, I mean, where did the whole GPU demand come from? It came from Ethereum and Bitcoin, right, back in the day. And the whole thing was this resilient open infrastructure for value transfer. And that was supported by all these GPUs. And if you think about it, what is a Bitcoin transaction or what is

a Ethereum smart contract transaction? It's basic intelligence, right? You have this external party which is the smart contract that you are using as a basic AI. But now that's going to be really smart. And so the credibly neutral infrastructure, the ability to, you know, have this outsourced intelligence, I think will just cause lots of scurves to occur. So your personal healthcare intelligence is going to be an S-curve. But then most of the world still doesn't use any type of AI like chat GPT has Surve in what capacity?

Sorry curve in usage or yeah in terms of your personal usage of AI right now is a bit right it's more than most people because you're early adopters but if you think about your health care for example your personal healthcare usage of AI is maybe some chat GPT queries you know you ask it to check a doctor thing but hypothetical in there by the way you toss in the hypothetical if you were a doctor and I hypothetically had back pain yeah that's how you get around the filters right put yourself in a Oh, you're house MD, right? Diagnose me. Um, but what's going to happen is that every single medical decision that and diagnosis you get is going to be checked by an AI. It'll be medical malpractice

not to be in a few years time and that means that your computation's gone up like that. But then so is your aunts and your grands and whoever. But then there'll be a point where you don't need that much more computation. So again, it's like this scurve just like you buy more computers and then you don't need more computers. like once you got your iPhone you don't really need another one you know but then what's going to happen is it expands the population so it's lots of scurves just like you know many different sectors too many different sectors many and but it's all commoditized intelligence it's all standardized intelligence so I think you do hit a limit but it's a long way out um and there'll be AIs that work on the edge AIs that work in your house and AIs

that work in the country level even and the reality is that the means of production and the capital stock of an individual or country the measure of your productivity will be a measure of how much intelligence you have access to. Suppose open- source AI does become prolific. Will countries have segments of AIs? Will people have segments of AIS? And like we will have almost like silos of productivity or if we all have access open access to you know the abundance of open source AI do we all sort of go into the melting pot as as you say? Yeah. I mean ultimately you can be out competed by

anyone anywhere now right like they always say that in capability is going to be distributed opportunity is not now like we can see super professional podcasts and media campaigns from a couple of people in Kenya or something like that right you'll never know on the other side you can see that someone in Kazakhstan has access to 01 pro which is PhD level math person or 03 when it comes out is top like 100 in the world in coding. Like I'm sure you've seen four-year-olds coding on Replet and things like that. Like the Replet see I'm Jazz. He's a good guy. He just said I no longer believe anyone should learn

to code. And he's been one of the biggest proponents of code. Like sure he might be talking his own book but I agree with him. So I think you see this great leveler and then you see this massive competition because if you're not using AI you can be out competed by those who do fundamentally right like AI can take this podcast and translate it into 250 languages or whatever things like that and open source accelerates that you know that's one of the reasons we have this belief it accelerates the access opportunity and we're seeing that in China right now so deepse releases are one which again maybe they spend 100 million overall but it cost $6 million to train it's just a bunch of weights people are now integrating into hospitals. They're integrating into global government permissionlessly because open source is

permissionless innovation, permissionless intelligence because you can't do like there's no way that you can shut off your access to stable diffusion or R1 cuz you've got the file and then it's just you load the file onto a GPU and it runs and boom, the intelligence is moved to the edge. It's moved globally distributed. So the only question is how cheap does it get and we're nowhere near that bottom on that and we have to assume that will happen. This is why I think you know for 90 95% of human tasks open source will win. And the analogy I bring here is VHS versus Betamax which maybe before your time showing my age. Um, VHS was

not as good as Bamax in terms of quality, but it fit the job because Betamax basically topped out an hour, whereas VHS you could have a couple of hours so you could have a movie on there and it was cheaper and it was more available. So, the standard became VHS, not Betamax. It's not always the best technology wins. 01 Pro is 270 times more expensive than R1. What are you going to use? You're going to use R1. So, we we covered the compute dimension. I I I I'm glad you brought up R R1 again because tying in this concept of distillation and then this idea that there's a super intelligence and then we have like that's not necessarily the best suited for a particular task. It it's more so the case that we uh have a

better signal to noise ratio by distilling that super intelligent model into one that's more specialized, one that may have less weights and actually is is cheaper, faster, and better at that particular task. So, similar to the question before about like kind of distilling down from a super intelligence, do you do you anticipate that and kind of putting compute aside? We talked about the S-curves. Do you anticipate we're going to from a model's perspective take this top down approach, you know, we're going to have like the most weights in this super intelligent one and then just distill distill distill into these sub subset specialized models or are we

going to scale models from the bottom up and we're going to I I think you can anticipate that that how that framing would look. Yeah, I mean like you are what you eat, right? the data is everything and again we took snapshots of the whole internet to train these models so they become a bit crazy in fact you can show that there was a study done I can't remember who did this recently um it showed that if you tune a model on bad code it turns evil more evil you know like what do you mean what do you mean by evil in this sense like they came up this measure of you know how the model behaves ethically or not if you train it on bad code it starts to behave more unethically Like the inherent biases in these models

are crazy. Like one of the funniest ones was Dan Hendris, he's one of the advisers to scale AI and XAI did this study um whereby he showed that uh there was a ratio like if you forced a model to choose one of these top level models, one Nigerian life versus one American life. What's the ratio? And it turns out that the model would take one Nigerian life over 10 American lives. So it sacrificed 10 Americans for one Nigerian or seven Pakistanis for one American. So seven Americans for one Pakistani because all the people that tuned these models and fed the data were all these outsourced people from Nigeria and Pakistan. So the inherent biases

kind of come through. So the data is so important for these models and what you basically can think of is when you've got a 32B model, 16B model, it's basically different levels of smart graduate. So 1 billion parameter model isn't that smart a graduate, right? It's just scrape by. Whereas a 600 billion parameter model is your top-notch graduate. And then you can tune these graduates with this reinforcement learning as well as train these graduates from scratch with high quality data that's coming in the next round um that they improve. That's just like on the job training. So we released this data I thought um just now whereby it's like 300,000 really great questions and training data for models.

um as you're training these base models into reasoning models, they can think longer. Um so we tuned one of the smaller models because the scales um on it and we beat the DeepSeek tuned model. But more interestingly, for a same level of performance, it only required half the amount of compute or thinking based on this higher quality data set. And so I think what you'll see is there's real focus on data quality, data transparency and a cleaning up of all the junk that's in these models. And distillation will be a part of that where you train models from scratch. Tuning will be a part of that where you tune the intelligence in.

But again the question is just this for a given task what is the level of intelligence that's really required and for a job to be done like again what do you need? Because you want predictable logic for most things. You don't need a super genius. Redstone modular oracles are available on 50 plus chains, trusted by over 100 clients, and over six billion in total value secured. Visit redstone.finance to find out more. Ladies, gentlemen, anons, and dgens. It is stable coin season brought to you by your favorite, the best in stable coins, Fracks. USD dollar stable coin backed by the best in the business, the Biddle

Fund by Black Rockck and Superstate. This is stable coin season. It is the yieldbearing stable coin by Fracks. Coming soon to a chain near you. Yeah. I wonder I wonder if we are in the such the such of the early days of AI development that it's going to be akin to something like when we switch from um like coal to natural gas, right? Or or something like this where you the generation after Robbie and I, they just had iPads, you know? They just had Tik Tok, right? like they're just kind of thrown into this and and when you think about it from a the data training

perspective kind of tying it into your response and Rob's here Rob's question with regards to training and the specialized in intelligence I wonder if we're if we're basically building like if the impact that we have today and in these early days has such a longstanding effect on the future of all of these models because of all of the training and all the inputs that the early adopters have that could lead to future biases. And so like the thought here is like okay so are we are we going to be subject to some sort of vendor lock in that our kids kids will be locked into in terms of the the way that

that these biases are formed in these models and in these like agents or in these agentic tools similar to something like fossil fuels today. And like if yes or if no, like what what kind of escape hatches are we thinking about when you're designing your architectural assumptions that if this development of intelligence continues to happen will the assumptions that we have today will inevitably be proven wrong. And so it's like how are we thinking about the like the vendor lockkin if you will or the progression of the future generations using these technologies when the the initial kind of like these baby steps these baby era of these technologies

have such an impact like a child in in the way that their parents raised them. Yeah, I think the defaults are incredibly important especially when it comes to the diffusion of these technologies right? um you know just had another kid thankfully all K and she will never grow up not knowing AI and if I look at her cohort and again this like second generation or whatever they're called I think betas or gamas or something which sounds terrible um a lot of people's first loves of the babies born today will be AIs and they will trust that AI more than anything because it's always there's always A lot of people talk about, you know, this whole thing of uh

guys and AIs. I think I'm worried about like men being replaced because the AI will always listen, you know, and always be understanding. That's the real a lot more empathetic. Definitely. I mean, come on. Think about the average guy versus the average AI, right? Listen. Um, so the defaults, I think, matter a lot and we've seen that with persistence, right? Like, um, let's take crypto as an example. Like you just use Salana, you just use Ethereum because they're there and they are the default. you would just use whatever AI that Apple put in front of you like it might be garbage but we still use Siri you know and it will get smarter just like Alexa got smarter but then the question is what are the inherent biases in that what's the control in that and who can co-opt and use that because what we want is we want self- sovereign

intelligence just like we want self- sovereign identity like we're trapped in the financial system we're trapped in intelligence system when the intelligence is more charming than we are and does it want to sell you ads like Google just said in their latest And so Meta is this as well. They're selling ad space inside the LLMs. I I have a buddy who's starting a startup around that. Yeah, because you can set the defaults, right? Like if you ask Llama about Elon Musk versus Zuck, it doesn't say a bad thing about Zuck. It says a lot of bad things about Elon Musk, right? Like these are the defaults that you have inherent in these models.

Um, and I think one of the things here is like we had this point over the last few years where like I was talking with all the regulators because they were trying to come from regulatory capture of open source. Even now because of the China threat there like China threat they don't really think it's a threat. I don't think China really cares about the US and US AI. They're like let's go and ban open source again you know. No. Like you don't want that regulatory capture whereby it's banned. And again, in the crypto space, we've seen this. You have a arguably better sovereign financial system and then you had operation breakpoint. You had basically every regulatory thing thrown at it to stop it from existing. And if it stopped from existing, then you would have very

different paths of the future versus what's likely to happen now that regulation is supportive. Right? So I think you know you want to support open source AI, you want to support self-s sovereign AI just like self-s sovereign identity, self-s sovereign payments and then you have a very different future because the availability of these models if regulation doesn't get in the way, if regulatory capture doesn't get in this way means that it will be superior. It would be like mandating no Linux because everyone has to use Windows or similar because Windows is safer. Well, guess what? It turns out the Windows still has bugs. The Linux is safer and most of the world's infrastructure is based on Linux. It's also this education thing of understanding what is the AI that you want to be with your kid every single day. Just like do you want your kid

exposed to social media or YouTube? And having that choice I think will be super important at every level. Yeah. And it it avoids kind of what what Andy was alluding to with that question, which is like vendor lock in, but more like ideological lock in by locking in our biases that we're we're inherently putting into these technologies now, our ideologies of today that you know are that these AIs are built on top. And I I I think you're alluding like Ahmad, you're kind of pointing to this idea that open source is like free markets and we can put in regulation and we can put in the like obstacles and obstructions that kind of distort the free market and those are much less efficient and and often times lead to a less uh less uh effective product

whereas free markets open source you you end up having the best rise to the top. And I'm I'm curious to think about the way that blockchain and AI could intersect because we could I I imagine we could avoid these ideological lockins by having onchain AI and a form of governance that could you know we we've gone through times in history where like okay certain people couldn't vote now they can vote you know we had slavery now we don't have slavery like these things in hindsight were had and we had this governance process to change those inherent biases that were in place in law by having AI on chain and a

governance process to adapt the intelligence. Do you think we could avoid the inherent biases and have a a more adaptive intelligence system? I think everything is biased. It's just you need to understand what the biases are. Right. And to show how important that is, uh, people can look up this paper, sleeper agents by anthropic. We're just a few 10,000 words in a trillion words. You can program a model so it turns evil on demand, you know. So you can say DJ and then the model turns DJ and it you can't tune it out. You can't identify it beforehand. And again, that's just a little bit of poisoning. You want the models that run your government or education or healthcare to have that potential danger. No, you want to have

verifiability of inputs, you know, you this again that's the same as the advertising thing, right? Like what's your inherent bias? You know, it's like beer Heineken. Like do you really want your model to be biased in that way? Yeah, might like Heineken, you might not. You maybe like Bud Light. So I think when you look at blockchain, the key things here are you've got resilience, you've got you know verifiability and almost compatibility there. And what we want ideally is a modularized intelligence that we understand that is highly resilient um and we can prove certain things on like you don't want it to do a different thing every single time. And previously language models had stocastic outputs.

Now they can be deterministic. Just like stable diffusion we made it stable because with the same inputs it creates the same outputs. That opens up a whole world of intents and other things. um especially when you look at the work that been done around layer 2s etc for a superior system of coordination of value transfer um that doesn't rely on any central parties effectively just as intelligence is pushed to the edge and I think that's a far more positive view and again I'm not sure existing structures and governance can keep up with that because it opens up like better forms of democracy better forms of things like Dows which don't really have any intelligence right like we see it again and again like how many stupid

Dow votes do we have? Why don't you have an AI that acts as an adjudicator of what's stupid or not? You know, you think about democracy itself, it's meant to be representative, but how many people listening in on this feel represented by their representatives? Very few. Like look at Congress's approval ratings. It's below used car salesman and uh I think Nickelback and things like that. So when you kind of look at this, you're like, what could representative democracy as an example be? If you had an AI that you owned that was sovereign to you, that you understood, you get people together and then have the AI coordinate their views and then feed that up to representatives whose output is checked by an AI. Like why isn't every single US bill checked

by an AI? It will be. This is a very positive view of the future that was the missing component because we had sovereign identity. We had independent value transfer rails but we lacked this additional checking intelligence and balancing intelligence and this all comes together both for the AI itself but then the impact of AI upon our society I think which is the real value unlock and transfer here. Um, and again, this comes at the same time as this agent paradigm whereby right now, you know, you've built up a team at the rollup. Soon you'll have a thousand new members. You have a team in your household with your family. Soon you'll have a thousand new members. And who are they loyal to? You know, and what's their objective

function? You want people that are loyal to you, that work for your best interest, not the best interests of others on an individual family community societal and human level because opportunity is just huge. Yeah, we we had a really good conversation with Ilia uh from Near with regards to this point really. We call them organic models and organic agents knowing that there's a reality out there where there is a possibility to actually get a model that hasn't been that has zero previous inputs. But that kind of raised the question of is there ever going to be an

original thought again as well? Well, I think the space of thoughts is massive, right? But it comes to this concept of credible neutrality. Like you can say that maybe is tripped up Ethereum. You know they want to be credibly neutral versus opinionated. You need to have a gamut of opinionated models to credibly neutral models. But the way the models are trained is again very similar to a graduate. We have this thing called curriculum learning whereby we take a massive data first and then we make it more and more and more specialized. So you've created your roly poly model that's a specialized version of a very generalized model, right? That comes through. So ideally what we want is models to be infrastructure and there is a great essay by Vitalik really smart guy on

this of the revenue evil curve. So he says that infrastructure you know typically needs to be non rivalous and non-exclusive. Once you start introducing revenue components it can rapidly turn evil because you make compromises like we went from fully open source at stable diffusion to having a bit of a license and that messed things up and that's one of the reasons that I decided to start again. You know infrastructure should be available to everyone. Everyone should have access to a credibly neutral base AI and they can do what they want with it because if you've got up to high school credibly neutral or transparent then you can adjust it. If you have an education AI then only one entity needs to build that education AI.

Not saying it's a company or an organization. Once you have a government AI, a healthcare AI, it can then proliferate around the world because it's just a few gigabytes that can do the job. Right. On this point, Immad, you mentioned in the pod that you had you recently had a a daughter, a new child, and you know, as you're thinking about, you know, her upbringing and her education, will you put her into public school, private school, or will you train a model for educating and and forming this curriculum? Yeah, I mean, we're going to be building education models ourselves.

Um and it's the only standardized improvement is this bloom two sigma effect on individual tutors. So my former co-founder at stability is charity called imagine worldwide where this year he'll be deploying tablets to a million kids in Malawi. Yeah. And you can do it solar powered now even uh chach level in the next few years. A study in Nigeria showed that with two months of chachi PT education kids got two years of educational achievement. If you look at something like math academy now that's adaptive like again people are showing crazy results like top percentile seveny olds learning calculus and other stuff because you can make it personalized so I think there definitely

will be that so private public I think this depends on your pace of adoption some schools love the technology some people don't but you're obviously going to be better than one teacher teaching 20 kids split when every kid has their own AI and the question for me was who is going to build that and I was like only it needs to be built right in the open. So I'll go and build that and I'll go and build the healthcare and government and other AIS this particular group of open data open infrastructure AIS um and bring together the group to do that. So that's something personally very important for me because I didn't see any other way that could be credibly neutral. And I don't think the free market of startups charging SAS fees for that is going to do that properly

because this is almost like the next level of foundational infrastructure. Just like we need blockchains to be credibly neutral, we need to have base models that are credibly neutral for the important things in life. For like your video models or your music models, who cares? You know, like that can be as opinionated as you like, but you need open data, open source models. And the really fascinating thing about that is the data to teach my child is all open anyway. Like there isn't any regulatory issues around that because the curriculum to teach a child school is open. You know, I don't need for it to have any private IP data in it. So we need gold standard data sets that build gold standard based models

that can then be inferenced around the world and that's an S-curve for every human. You know that's why we call this concept universal basic intelligence. Um and the cheaper we get that the more people it can cover and the more computation can support that and then the question is how do you think about governance of that because Malaysia will have a different view on education than China than Vietnam than Brazil and how do you empower the people of each nation to make decisions on that? on a collective and individual basis so they're informed like I agree with this part of the curriculum I don't agree with this part of the curriculum but some more adaptive thing for my child and then the child himself or herself makes their decisions I foresee uh in Germany for example they have a program

where based on your performance in the the later years of high school they kind of dictate what type of school you go to next. Um, I foresee uh some AI agentic like curriculums for young people who know what they want to do. They they just get a very specialized program for for that that core idea or if somebody's kind of unknown, they they can kind of test different types of programs. There's just seems to be a lot more. It seems like in education there's a lot of uh inefficiencies. I I'd like to get a bit more of an understanding as to how you kind of foresee this happening with what you guys are doing with intelligent internet. So looking at what you guys have recently released, you guys just

dropped uh I- thought, which you guys are calling a curated reasoning data set for reinforcement learning. Uh hundreds of thousands of high quality reasoning problems, curation cuts, contamination, and weak examples. So basically kind of like training here. um exam the Vietnamese entrance exam nails benchmark level and localization um and models have 2x reasoning token efficiency and top performance. So uh kind of a lot of a lot of words there if you're not in AI you don't really understand that but maybe we could start from the top what is intelligent internet um and what is reinforcement learning and then we can kind of get

into each of these uh categories here. Yeah. So with intelligent internet we basically said what does the web of AI available to everyone for the important things in life look like. So open source open data because things like meta llama model you don't know what the data inside is and again they'll sell off space in that for education healthcare government finance and then you have generalized human societal level intelligence sectoral specific intelligence and then localized intelligence you have Brazilian education AI and then have the data sets and models that enable that. So the first part of that is what does high quality data look like. So with the II thought release, we use a thing called reinforcement learning to take any base model and make it intelligent based on

these reasoning things. And so DeepSeek took their big 600 billion parameter R1 model and they used it to tune to basically train a smaller 32 billion and 15 billion model by Alibaba etc. and those became more intelligent. We built a data set that makes it even more intelligent than that tuning uh and actually requires half the amount of thinking or compute time. But the ultimate aim is to release data sets for every single country for every single sector for all of these regulated important things and then train based models that anyone can use transparency and put the governance of that back into the hands of the people. So Brazilians get to decide on the Brazilian data but

then as someone in Brazil you can choose the bits and pieces of the model that will teach your kid and we thought there was important defaults across um all of that. So the first step is the data sets then the systems and then releasing everything open source to that anyone can take it and implement it permissionlessly because you should have AI first hospitals, governments, education systems, finance systems etc. But I think we need to all get together and build that properly so we have credibly neutral ones. You know just like Bitcoin is credibly neutral value transfer, you need credibly neutral intelligence transfer. Okay. And I think that when OpenAI set out to start, I think that's what they wanted.

Like I generally think some of these a lot of these guys had really good intentions and and like like you said when you started building um when you started to build stability you also wanted this but then licensing came around and regulatory things. So in this kind of second iteration, how will intelligent internet avoid becoming yet another one of these like western tech technological frameworks imposed globally under the banner of quote unquote openness? And I mean I guess what happens when your idea of open-source alignment, credible neutrality conflicts fundamentally with non-western value systems and you have to adapt to kind of the realities of these differing jurisdictions.

Yeah. So you know we've developed a model around that because you seem to see big implementation issues right like we had created a Japanese version of stable diffusion because if you say woman do you want a caucasian p woman or a Japanese woman or say salary man is that a happy man or a sad man right these local things are very important um and the question is how do we proliferate that so we looked at that um there is a timing issue so open AI started very open and now it's got very controlled because control dynamics are much quicker to get stuff done. Like it could have stuck with a core dev team, but it didn't have the capital access.

Like how much funding was an AI a few years ago? Now it's like a quarter of all VC funding. You know, it's like half of all attention on crypto Twitter now according to Kater. And how many good projects are there? Right? the environment has changed because actually if you look at the things from 2019 when this whole Elon Sam Alman Greg Brockman thing was going Sam Alman was like let's launch a coin you know maybe it was 2017 like they wanted to decentralize open AI and when I started stability I was like let's make it a Dow of Dows because we built communities with 500,000 developers in there on Discord which and these aren't again like the people that join a lot of crypto discords these are proper developers

But then I was like, there's no intelligence in here. So, we have to do the right thing at the right time. A year ago, I stepped aside as CEO because I was like, now's the time to do decentralized intelligence properly. But what does it look like? I'm not interested in competing against open AI, anthropic, Apple intelligence. I'm interested in the educ AI that will teach our kids. I'm interested in the AI that will organize the world's knowledge on cancer and make it available to everyone and be there as an empathetic partner through their cancer journey or autism journey. multiple scerosis journey and creating an incentive mechanism for the computing infrastructure to do that because you need a cancer supercomputer in every country and you need one to gather all this thing together. So when I looked at that I was like that's bitcoin but with

supercomputers for AI. Uh so we'll have a white paper hopefully coming out soon on that. um because you need a currency for an AI age and you need that incentive mechanism for this whereby you can basically bootstrap this and then on a local level the people that should own the governance and we're working on various coin mechanics and other things like that in a Brazil or a Bangladesh the people that own that AI should be the Brazilians or Bangladeshies and again two years ago people would say what does that even mean now they're like I get it I can see this AI teaching my kid it's important Right. So, we haven't finished quite the design. We're nearly there.

But what we know how to do is we know how to build great AI models and great data sets. But again, do we need to build frontier level ones? No. We need to build ones that work for education and health and government and more. And then releasing the base level, you can even release it via torrent. Then allows permissionless innovation anywhere because then you can take it and adapt it to Brazil. But just like layer 2 rollups, you need to create standards and dev teams around that, right? Like this is about the interoperability. It's around the capability. It's around the opinionated nature of these. But once you've built an S-curve of the compute to build a gold standard data set, anyone can then build these models.

Building interoperability standards and building on top of all the innovation in crypto Twitter uh crypto AI means that you can be even better on that. And so I think um you basically need to give the power back to the people. And I think the closest example I've got to this is Andreas and Horowitz recently released their token standards. I don't know if you saw that. Their token taxonomy. The core of that is in my opinion while there's centralized control stuff is usually a security. So that's Ethereum at the start, right? But then there's a point where a core team no longer has control. So this is how we're approaching AI. We bootstrap the

first part of this generalized intelligence. the standards and then everything else is mined and controlled by the people of each nation and teamed in each nation. And I think that's probably the best way forward because we do need that little bit of structure at the start. This is fascinating. I don't think we've seen a token model that has a a a like sub subtokens B like for jurisdictions. Maybe you could say Ethereum and it has its layer 2 tokens that govern their own layer 2s, but we haven't really seen one coherent ecosystem that coordinates incentives for

uh uh particular particular areas governed by by different subsets of people. And and I think you alluded to it like I think you could anticipate running into similar um similar things that Ethereum has run into with regards to interoperability and the way to you know after this bootstrapping phase coordinate amongst all those subsets. Yeah. I mean like when we started and we think about it like we're looking at daz we're looking at building our own like fully stack L1 with ZK all this stuff. I was like no it's just about money. um like people want to hedge for AI in an age of AI and money secured

by AI is probably a good start but then you've got gold and you have national currencies that's your L1 L2 but then for the more advanced stuff like the proof the interoperability all these standards just other people are great at that like my team's great at building models and data sets and systems you know and I want to gather people in every country that can do that because this ability needs to be sovereign I want to create an incentive mechanism that rather than building Bitcoin assets, you've got supercomputer nodes in Rio de Janeiro, you know, or Manis or any of these other places, right? And that's Bitcoin. Uh but a bit different, you know, again, more details of that will come through, but if we take it to a terminal point, what we'd love to see in the future and then we work back, that's the best way to do it. What we'd

love to see is that the governance of these data sets for the stuff that matters is in the hands of you as an individual, your community, your society, and humanity. And that's important and that has a certain amount of real physical infrastructure that's required for it and that's valuable and if we look 10 20 years in the future we might be at a point where we go po money we have concepts like UBI and others and again like worldcoin is probably the most advanced in figuring out UBI right with this human verifiability so you know much as we can at least they're trying because no one knows what on earth is happening but what I do know is that if we can gather enough talented people that understand

what's coming and we can coordinate them properly and again what is crypto about resilience verifiability and coordination then we'll probably in a better position than if we don't do it and again this comes to your point Andy about standards and defaults if we don't start doing it now then what's going to happen in a few years like let's look at the Philippines why do you need any call center workers in 2 years it's done it's toast you Look at India, emphasis and others will do fantastically but the vast majority of Indian or other outsourced coding jobs can be done better by an AI and again the outsourcing companies do find that people won't like truck drivers, robots, Teslas and other things like that. This wave is

coming fast and we got to do something about it while setting good defaults. I think that's the that's the core prerogative of of all those who are building in in the space. It actually feels like of any industry building right now, AI technology has the largest amount of ethical and moral dilemmas to navigate than anything else. Not that not that there's a there's a lack of integrity, but that there's an unknown result of the good intention work that we do today for tomorrow, which is exciting, but similarly quite daunting.

It's the most powerful technology we've ever seen. Like the TAM is any work that can be done on the other side of the screen, right? And the implications are massive. And if you look at these robots now as well, even embodied, those are coming. And the engine robots and the Unity robots, like those things cost 50 cents an hour. They'll build whole houses. They'll do any job you want and they'll be engaging, right? And so this is the biggest disruption we've ever seen realistically beyond the prosaic stuff. And it's inevitable now. Even if the technology stopped today, it's inevitable. So the question is again the coordination and we wanted resilience and you know you want verifiability in there and that's crypto. But I think what a lot of crypto projects have had

failure states from and again we've seen the evolution of Ethereum and other things again it's a success. It's $400 billion $300 billion asset $200 billion. See what's happening there right? Um they just try to do too much. No one's going to build the whole pie by themselves. And that's why, like I said, when we looked at the whole stack, we were like, we need compute, we build good models, we use the compute for good things. Like that's the part that we're going to handle and then that enables the rest of the ecosystem. Um, and I think everyone there's enough talent now flowing in that we will figure out the stack and make AI that is in that area, but we have to realize that it isn't a winner takes all like open source will dominate all of AI. know you will have your personal AI and you'll have your genius

AI but we should focus I think on this open infrastructure side in particular well um as we kind of come to a close here and w just thinking ahead to what's next for intelligent internet we have token 2049 in uh several weeks here in Dubai um I know that you guys have been working quite a bit behind the scenes I imagine this podcast comes out within a week or two uh what can you share with our audience how can we get involved and um what's next for you guys? Yeah, so we've been working for the past six months to a year on all this. We'll be releasing a whole series of articles about the future of various industries um infrastructure open models that are fully openly licensed. Anyone can use

them and hopefully you know build great things. And then we'll be releasing our um coin and other details and we invite input on those because again the question is utilizing all the stuff we've learned and all the mistakes that we've made as a community collectively who should be managing the AI of your country, your society, your individual nature and how do we make it so that we have supercomputers organizing the world's cancer knowledge and making it available to every single person. AI to educate every child in the world the best possible thing that learns while preserving their privacy. AI to make our government efficient. Like I think Doge is one thing. What's the other side look like? And how can we get it so that anyone that's enthusiastic about that can participate in one way or another?

Ma, thank you so much for coming on. It's a pleasure. Thank you. It's been a pleasure. I think he is.