Cryptography, AI and the ZKML Prover w/ Daniel Shorr (Modulus Labs)

Transcript

foreign [Music] hello and welcome back to the strangewater podcast thank you for joining us for another fantastic episode for those of us interested in the bleeding edge of computer science and advanced mathematical Theory there are two categories of innovation that are aggressively competing for time attention and talent cryptography which I'll call blockchain or distributed computing and machine learning now there's so much to say about these

two technologies for example what makes them incredibly different or what kind of impact can both have in the real world or even what kinds of research problems do both technologies have and therefore share but the big question that we all need to ask ourselves at some point what is the intersection between crypto and AI now look well I don't really have an answer yet I could not be more confident that crypto and AI are two sides of the same coin or maybe two chapters in the same book they're definitely two paradigms that ultimately form just a part of modern

computing AI is about using technology to create abundance to lower barriers to entry and to overwhelm an analog world with more and more machine generated content crypto is the polar opposite our core principles are built around scarcity identity and verifiability I don't know how these two pieces of the puzzle will fit together so it's time to find people that are much smarter than me who have already seen this feature and have become so obsessed with it that they are compelled to form a company people like today's guest Daniel Shore from modulus labs

modulus is a young startup looking to build the tools that we need to complete this vision and to bring together these two powerful Technologies with their zkml approver modulus is bringing the infallible verifiability of cryptography to machine learning while at the same time bringing the computational power and creativity of AI to blockchain today Daniel and the team at modulus are working on relatively simple models and they're developing concrete short-term use cases for the ZK ml proofer but I've got a dedicated portion of my undivided attention fixed on modulus

because while today we are talking about games and apps tomorrow we will be talking about AGI identity and the dawn of the age of artificial intelligence one more thing before we begin please do not take Financial advice from this or any other podcast ethereum crypto and machine learning will change the world one day but you can easily lose all of your money between now and then all right time to start the show Daniel thank you so much for joining me on the strangewater podcast yeah happy

to be here thanks for having me Rex for sure so um as always like how we like to start this podcast is like really to start with like who you are your background and like how do you um find ethereum and then how would you like fall so far down the the black hole that you uh were willing to like sacrifice here um you're standing in like you're good standing in society for it yeah well I um I always joke that I look like I'm 12 so I didn't really have much standing to to fall from anyways um and and I'm so charm wrecks that you're kind of curious about me uh normally people were just like what's going on in modulus um but yeah I mean I guess my story um is actually a pretty straightforward

one it's probably one that if you have like an internal llm it can probably do a better job than I can uh it's like the most stereotypical stuff but you know love science love engineering since I was young like everyone else nowadays uh did quite a bit of math and at some point realized that um you know we can actually apply mathematics as well as all these Concepts uh that I was really excited about uh to to have like a real world impact right right around the same time anyways uh as I was reading a lot of sci-fi where it's like oh this is all the heights that we can get to so ended up setting um you know uh engineering computer science the rest uh at Stanford for a while uh you know got into AI research along with my kind of dorm mates

um and just as we were kind of wrapping up our time as undergrads getting ready for grad school uh covet happened so we ended up quarantining you know taking a beat before presumably committing the like next decade of Our Lives to Academia um if not more and uh we were just kind of like bored and trying to figure out like what people were excited about right um and at this point this is like 2122 uh the year of 2021-22. um you know folks who are really excited about crypto uh I guess they still are but even more so back then and specifically given that we had no skill set when it came to Discerning signal from noise on Twitter um we just ended up like sitting down me and my buds reading a ton of papers um and and Rex I'm sure you've probably

been in a similar place where you're like reading academic paper after academic paper and very quickly it's like some of these uh papers are a lot better than others um and very quickly for us anyways it's like wow the ZK stuff the zero knowledge cryptography uh specific research is getting so much better so much faster um and it's clearly like this like weird incidental application blockchain has caused ZK research to advance massively um so just as a quick recap at this point we're like yeah not so sure about the PHD or grad school uh we knew AI from the work that we had done as undergrads and we're just learning all about cryptography so just purely as a science experiment ended up um you know trying to improve and models uh using ZK

proofs and we'll I'm sure we'll get into what that means um but anyways that's like the that's the origin that's the starting point of modulus um but you're you're all caught up on on myself anyways no man I mean come on you're definitely selling yourself short because like look like it's easy to say um like oh you know this like technology is like super cool and interesting and like it is like anyone at this point like can see that but um it's one I I think like there's not a lot of people who are able to Grapple with this stuff like at an academic and real research level that also like pivot away from like continuing down that Rabbit Trail of like Academia and I I think like we both recognize that what's

special in crypto and machine learning as well is that Academia the line between Academia and like industry is becoming so blurred because the technology is moving so quick but despite that like latest development in The Cutting Edge of Technology like what specifically like as you walked up to the prefaces of like a PhD um like why did you do the smart thing and not jump off the cliff yeah I would say the verdict's still out on if it's a smart thing um but uh I guess um you know one thing to note is I would say uh there's definitely a real like personality difference between Ai and cryptography anyways um and you know we were kind of

um aware of what the air world was going to look like or or had been just just the kind of identities I play there I I think it's very hard to like uh talk to a handful of AI researchers and not at least symbol across a couple God complexes um just creating intelligence something about that um and it makes sense right just like AI is like very expressive like the the underlying technology itself lends uh I guess itself to a certain type of people um but cryptography is like radically different it's actually a very humble discipline that's very rigorous uh everything's super discreet like this is exactly the cryptographic statement I can make and no further um and that was like really attractive for us as somebody who who or as a group

who really had just been used to the former and so I think for us uh you know this was a fun opportunity to marry these two um Tech Stacks or academic disciplines that really from a personality standpoint again just totally incompatible right from the get-go um but also I think more earnestly uh maybe um or maybe more aspirationally actually um we wanted to do something that like really could have uh an impact that was like very concrete um and it felt like this was actually a very uh cool way to marry cryptography and AI um so yeah I guess um it may be more motivated by like what's cool and what's kind of uh you know like having us like

just be excited about the work that we're doing and less about like form factor right um yeah no for sure and like look I don't mean to like belittle anyone that like is it wants to get a PhD but I do think that like what is Magic about our industry is like because the technology is the product and the technology is just straight up math and all of it's so new is that we can push like this whole massive being together from these like different standpoints like whether it's going straight into a PhD or Academia or building a company or doing research or even eventually once this industry grows

up like doing public policy stuff right but um like why I think it would have been the wrong idea for you to do a PhD is because like you're clearly a builder like you don't need me to tell you that like the VCS have already told you that like um so we'll get to that in a minute but um let's let's like turn our attention to like this moment before modulus Labs like when you're um kind of have this background in AI you're seeing like them really incredible stuff that's going on in ZK and as you point out that's both from like God like once you actually unpack like what a kcg commitment is I mean it's it it actually literally feels like magic you know like you're evaluating

number numbers that no one will ever know I mean it's crazy right so and then on top of that like you're saying the culture what when these so first of all at this moment like how do you holistically understand what like is going on in machine learning and how do you holistically understand like what's going on in ZK and like what what do you I guess is a big question but like at this time like what do you think these Technologies are for it's a very it's it's like the question of the moment in some sense right um you know I think uh uh not to hammer the whole uh personality Point too much but um they're actually just like really

distinct and and I mean all the way from the research to the eventual applications right like I would say like in the case of AI the the applications are oftentimes like super obvious or like super sticky just inherently like it's very clear that like a product after some AI integration it's just like 10x better or stickier or whatever right and you can like you know call your family and say hey this is like what's happened because of the world AI you can use chat EPT and it's like super clear uh what the value ad is I think with crypto it's a lot more kind of subtle and nuanced right um like it's not entirely clear that a tech Sac which is like more robust to man in the middle attacks uh like how does that really impact my day-to-day right but we also I think uh from an instinct standpoint

understand that when we have networks which are just fundamentally more robust um and and you know less dependent on these like intermediary authorities which I guess is like a very high fluent way uh of talking about like crypto networks um we should at least in theory be able to build like really really large scale applications that just like don't break test much right because we don't have to default to some like faulty trust assumption and so I think you know when it comes to like what are the best ways of marrying these two technologies um there are a lot of different answers but the one that we tend to to lean on is you know trying to bring like the the the like robustness and the concrete security guarantees that you get with all these cryptographic techniques to

this like wildly expressive like wild animal that is um you know AI uh compute right and you know if I think if you can walk that balance right what you end up with is both um kind of the certainty and the accountability of crypto as well as just the sheer like horsepower of AI compute um yeah but that's that's obviously just one interpretation no for sure I mean I think like there's definitely like I had a super high level like at a Vibes level right there's something about AI that is like generative and abundant and like wild as you're saying and then there's something about um like crypto right or cryptography or distributed computing that is like slow

and scarce and like on purpose and like they they seem I I feel like we're too two cycles away from understanding true actually you're ahead of us but we're understanding truly like how they marry but to me it seems so clear that like in this world that's becoming like more and more chaotic um like crypto is an answer that is like almost opposite of uh the way that like the world is trending yeah yeah I think that a lot of that is very Salient and certainly like we're figuring this out like everyone else right um I think um uh you know there are lots of amazing academics Founders operators who are trying to figure out how to like plug

one into the other right whether it's icon attribution using crypto in the AI like like just Mass scale like scraping of the web world that we're in now or um you know I guess in our case uh using crypto to make AI more accountable more discreet more dependable um yeah so definitely uh you know ideally it's like it's like the marriage that like benefits both sides right that's that's all we can hope for from from the best marriages I think never been married myself but yeah um yeah there's probably a lot to be said about marriages but uh anyway the um so before again before we get into modulus like let's talk about um like what

a lot of people talk about crypto AI right and most of the time it's just some scammy token that is like really actually a scam but um before we get into like what's the right way to think about it and what modulus is trying to like uh Blaze the trail towards what are the way that you hear people talking about these two that like really either doesn't make sense or just seems gimmicky or like what what's what's the wrong way to do crypto to cross AI in 2023 yeah I mean definitely uh for me anyways like the moment that really crystallized like what I can do to like a very nice in technology anyways is when I got a

note on Twitter or via Twitter demon it's like hey uh like what's going on with the modulus token like I'm seeing it pump I'm seeing it pop like it's it's dropping it's whatever um and we've never launched a token where there's no modulus Like official sanction token um there's just like a bunch of proxy coins or coins um obviously like it's a diamond doesn't there's like a ton of these out there and I have no idea like what is the underlying kind of um value function that's driving the utility behind any of them besides speculation so like that that is maybe like the obvious uh poor intersection of the two I.E only in name um but I guess you know it's a funny thing where in some sense this is like an open

it's like chess you know like we can see all the pieces we understand the move sorry we understand like what AI is capable of and where cryptography is as an academic discipline they're both like massively uh improving and and you know uh becoming more performant in their own ways but oh so like underneath everything is like the laws of physics right and so whenever people make claims are like to me anyways really really outlandish um obviously I get very suspicious because like we are looking at the cryptography every day we are looking at the kind of things that we can that can be done in both regimes um and I think from that standpoint like from first principles it's actually very easy to figure out uh if something looks correct or or a little funky right so so maybe as a point of example

um uh there's a company called Jensen that we're a big fan of which is using cryptography in the regime of AI training right so trying to like build this like alternative uh Marketplace of like or taking of like latent gpus so that training massive models can be a lot cheaper and then using cryptography to like check that all the training was done correctly that is like a very sensible uh application to me from a conceptual level the challenge is making that economically uh reasonable and competitive with the centralized players are out there right and so I think um uh I guess the good news is the proof is in the pudding right even for ourselves right like can we actually make the math work out and make the economics something that's like reasonable for end customers

um but that's I guess that's how we shift from just like an r d world to businesses that actually deliver real value and move needles for for you know Everyday People yeah so sorry just to like um tighten up what you said as an answer the question just to see if no no just to see if I got you because I think what you just said is that um essentially like there seems to be a lot of ways to like possibly tie crypto and a or yeah crypto and AI together but like if you actually like sketch it out and look at like the costs or the requirements or whatever like the things are just like so unrealistic that like maybe like entire categories just don't make sense but I think the second Point

you're making is that and and you didn't directly say this but one of my favorite things to point out is that like when you make order of magnitude improvements on technology like you open up entire new use cases and so like something today that like let's say in order to do like the chat GPT training data it takes like a proof that runs like for four days and costs like thirty thousand dollars if you do like two order magnitude Improvement like thirty thousand dollars becomes three hundred dollars and like what did I say seven days becomes I'm not gonna like couple hours right yeah yes right there you go and so like now that suddenly becomes

something that like you can build a business around and so again I think what you're basically saying is that everything out there that combines crypto and AI seems a little like moonshotty to you but like the point is to like get your hands in now get dirty like figure out how any of this would work and then work on just like driving down costs yeah yeah exactly I think cost is like the the certainly the metric that we obsess over a prime because it's very clear from even like early customers that we have that if we can bring the and you know costs like very concretely for us just means like how many sips from the wall do you take as a computer so electricity from the wall right uh like if we can bring that uh consumption down however much these

use cases be attainable these use cases would be you know live and well um then like it's it becomes very linear right and so I think um yeah maybe maybe to help myself clean up the answer a little bit uh you know I'm always looking for like how do you take like this like multi-dimensional optimization problem into like a very concrete like actually it's just about costs if we can get here then this like marriage makes sense uh then for me it's like okay at least it's like a known challenge that we can like meaningfully map data onto then I would feel a lot more confident about any specific approach and it makes a lot of sense and and I guess like final category before we go to like what is modulus um is like how much do you see AI being an important uh factor in like basically

like the the most credible like use of AI on blockchain that I hear people say is like oh it's going to be important in forensics and tracking the chain and like is that something that you think of as like a particularly interesting use case or if it or is that something to you that is just like yeah it needs to happen it seems like something that AI is going to be good at and um it is what it is it's a good question yeah um I guess um in terms of like the application of ai2 blockchain uh honestly it just like depends on where in this in the stack we want to kind of slice that right like in some sense like we can have ai models including llms to help us write our solidity contracts

right like our little smart contracts but then like ah but you know there might be like security bugs that the training data inherits from GitHub um but then we can get a second llm to like audit the code and then go back and forth between formal verification uh but like okay that's like already like that's like the lowest level intersection and you go a bit further up okay what about smart contracts having uh access to AI outputs both centrally just by the Korean like open Ai and potentially uh using ZK or in a decentralized fashion through someone like modulus okay then like maybe it's a ux Improvement let's go even further up the stack then let's use air model to intersect with not just smart contracts but literal applications right can we pull in a bunch of uh you know D5 service data uh uh or data device

services and sit together some like new composite application which is you know smarter analytics for the chain and forensics and uh you know AML and fraud detection is so forth so in the same way that I think AI will be a big deal for or already is a big deal for the rest of the world uh it would I would be very surprised if there isn't a good intersection in crypto but that for me anyways is like that's easy to say in terms of laying down like concrete steps to get to that future where like crypto is better because AI um like that's why modulus exists and that's why like our peers exist right like we need to lay down the rails and actually push that train up the hill uh until we you know we we get it to a point where it's self-sustaining and the

value ad is just like so obvious it's like oh what reps your smart contracts uh don't have ai in it that's like you know you're totally lame for not doing that like it's so clearly secure and working and creating value um you know let's let's catch you up right um so we're still on the other side of the flywheel we gotta like push over that friction yeah yeah for sure yeah I think um yeah well we can keep going on that forever but like let's start to ground this conversation around what you guys are actually building so um tell me like what is modulus um like I guess if you have an interesting origin story let's hear it but um like more specifically like what is the problem that you see in this space and like what are you guys doing to

um if you know again this is a fast moving space if not solve it today lay the uh the ground rails to uh build the future sure yeah and actually I think um maybe like telling like literally what happened at the origin of modulus is a good way to unpack what is oftentimes like a very confusing topic right um so you know I guess back to uh wherever I parked us uh you know we're reading all these crypto papers we're falling in love with ZK like the math in US is like screaming with joy um and you know we have this uh uh we have this moment where we realize that actually um this this like incidental property of zero on photography which we call succinctness but basically it's like faster to check a Sudoku puzzle than it

is to solve a Sudoku puzzle if you will right um makes it so that you can bring arbitrary off-chain logic into smart contract like embedded in there but we're like what's in the smart contract is just basically the check of the Sudoku puzzle like oh yeah the math was done correctly let's duplicate that computer you know around all the nodes and you know this this succinctive property um maybe I guess the the Insight we potentially had at that point anyways was maybe like this process is good enough where we can start to prove off-chain AI compute right in other words imagine you're running like AI computations or AI features directly in the smart contract right so this is just like you know Noah had done this yet

folks had been talking about it for years and years um uh and you know we were talking about Folks at EF even at like Solana about like whether this is possible or not and generally the consensus that we got anywhere was just like maybe like it seems a borderline um so we thought okay you know uh school's about to start again or like winding down from quarantine uh let's just like try as a science experiment so what we ended up doing was training a very simple model this this this neural network I like uh or or uh yeah it had I think about 70 000 parameters which by like AI standards is very very tiny um it was trained on the historic eats prices and all it was trying to do was to classify or make a prediction rather of whether these prices would go up or

down in the next hour uh we then converted this model into ZK circuits and improved it and put the proof on ethereum mainnet so that it could interact with the UNICEF contract on L1 and trade e and it's worth noting this entire process as I've mentioned which is a neural network making a decision about or making prediction about what will happen to these prices and then trading against um uh an L1 contract is fully easy Improvement fully tamper resistant and better yet we put it in an account where there wasn't even a withdrawal function right so all you could do was just let this robot decide on what it wanted to do and execute on that forever until it ran out of gas and we put like 500 bucks in there just like you know wish it good luck and have it going through that

um so we thought that was kind of funny and we put on Twitter and the rest and what we didn't expect which is what happened next was that a ton of people just online started donating to the Rockefeller Rocky bot we called it uh just to keep it alive and you know we're finding like thousands of dollars in and out uh and to us it's like totally wild right because there's no upside not only is this model definitely not performant if we solve the crypto markets believe me like I'd be you know enjoying the fruits of my labor um but not only was it non-performant there was literally no withdrawal function folks are purely donating in the truest sense of the word just to keep this thing alive I yes because we are all excited by this idea of like autonomous AI agents just like executing

without uh any fear of manipulation that's it um so you know Rocky bot did that eventually it ran out of gas and died uh but it traded for over a month and definitely it planted the seed uh for for modulus which is just you know I guess all the work that we think might be necessary to actually have performant massive uh aggressive models in smart contracts without breaking blockchain security at all right using ZK so at the most basic level that is kind of what we're doing uh still which is just like how do we take this idea and magnify it right bring it into production bring the cost down so that it's not just like silly toy examples but value generating value driving features that actually make a difference uh for you know with

your clients um so yes so that's that's like the origin story quote unquote uh you know we've been around for just about a year now and uh you know that year has been filled with like research and papers and other projects so let's let's hover over just like what modulus is from you know using the the rocky bot as um you know like the example so like taking a step back like the important thing here is like look it was always possible before to like create a little smart contract that um and like let's just be really specific how this is working and I'm assuming so correctly if I'm wrong but some computation is happening off chain like at that could be on a laptop at your in your guys's office it could be

from AWS like wherever something entirely unrelated to ethereum and then whatever's happening there comes a like just a very simple command buy or sell and then that gets sent in a ethereum uh transaction into the evm and then your smart contract will say okay I just got this command and like before modulus before ZK you're basically saying I trust that this message like came from not only came from who it I it's supposed to come from but the computation they were supposed to run was actually run and so um like I don't know if it's worth really

unpacking like why that trust is like a pretty powerful trust like assumption to have and like a pretty vulnerable situation to be in but what Rocky bot did and what is like the start of Z sorry the start of modulus is just add in one little like piece of magic right which is a ZK verification that that yes buy or no sell or whatever that command like actually ran through the the machine learning model that you know the smart contract like was expecting it to run through so first of all like correct me if um I was wrong on any of that uh but like

yeah I guess like can you just like very briefly like talk through like why it's so important to do that ZK verify as opposed to just like basically accepting the message or creating some sort of like you know signature password scheme or like what is the magic that ZK is bringing into that um bot yeah for sure and Rex are totally right right like the I guess the final part the ontrene verifier it's just like a gatekeeper right and it's just checking the receipts if you will right um you know what was modulus or whoever honest um uh we call this process this idea like faithful AI in the sense like AI that I can't I guess cheat on you in some sense right

um and I do think it it it's totally worth unpacking uh I guess uh obviously I would say this but the the kind of counterparty risk right we're like what's at stake here and certainly like if the AI output is pretty frivolous maybe it's not so important uh but insofar as this output potentially impacting a lot of money like a lot of dollars are at stake or a really important decision is at stake or you know even more importantly a super autonomous system where every piece of software just automatically goes to the next piece um that is happening I think it's it's going to be like absolutely essential that we have this like uh gatekeeper that goes hang on a minute you know let me see if anything like any hanky-panky is happening right

um I mean it's it goes beyond just like oh like maybe like there will be like one percent or two percent shift in prices I think it's stuff like hey it's my credit score fair right like like whether on chain or not right even Web Two like uh am I being discriminated against because I have a weird name quote unquote weird uh or or because I I live in a specific geography that shouldn't really say anything about like my ability to be a good uh borrower or something like that right all the way to you know is uh is this NFD price fair right like is is there like an appraisal process that is actually Mutual about like assigning value to my assets or assets that I'm doing Financial transactions on top of and of course you know zooming out from crypto specific uh you know Financial use cases even stuff

like um you know did this like AI artists actually create the piece of art that I'm getting or is it from some other model or some other person or uh you know if I'm uh if I'm querying a model for like Financial recommendation medical recommendation uh any type of legal recommendation I really want to know that there's no way in the world that whoever's giving me that output flip a specific number Hammer to wait or totally just swapped out the air result entirely and fed me some falsified result right so the most succinct way I thought of anyways to capture this idea is at least you'll have to go back a few months but it's like a Twitter verified check mark of old uh but for your AI outputs right and I think in the future where the world is like really

autonomous and like expressive compute in general becomes a huge slice of our compute diet just as a society having these Gatekeepers having these mathematical guarantees is how we make sure that the world continues to stay very robust and we're not like subject to some massive vulnerability or some terrible attack yeah and I think just to like really drive home the point it's not just about like the verified check mark like per se right which is some like you don't even have to like you're trusting that actual steps were taken according to like the code that you expect right and so like you're you're not even dealing with this like first level of like verification like make sure it's like the right person at

the right time like you don't even have to trust that like essentially that you know Jack Dorsey and his public lead trading company will ever get sold to like a megalomaniac because like it doesn't really matter because you know that actual things that are being done under the hood and like that's kind of what ZK provides um and and I guess like you know for for people who already like live and breathe this stuff is this is such kind of a basic concept but for people who haven't made the jump yet like the reason why we we so breathlessly like switch between crypto and blockchain is because like it stems from the idea that at least tell me Daniel if you agree with this that

like blockchain should not be understand as like cryptocurrent sorry as currency or as Financial or like whatever what it should be understood is like the base layer that one like the property layer of the internet but more importantly it's like the base layer for distributed compute right and I think that the the the piece to understanding that is like what ZK cryptography allows us to do is take compute from centralized from like dark sort from wherever literally wherever and then verify it in this like kind of pristine like Untouchable space and then

you know that a lot of it's social a lot of it's actually not but like through that kind of magic we're able to say okay like we're able to transform anything we want to into trustless or into almost decentralizes a loaded word but um yeah I don't know I mean I any any like Reflections on that before we move on I just think like the we don't really talk about like what ZK actually is and like allows very often yeah yeah I mean you know in some sense like Zeke was invented for privacy like this is a very strange like unintended uh uh kind of uh consequence of that original conception right um but I guess on the point about um you know like what blockchains are good for

and what it means to you know like this for me anyways is like the ultimate upgrade to the blockchain right like this is like a superpower like we can just like plug into the blockchain and have the thing that we plugged in be like elevated the same security standards as the underlying like robust Network that we've sent a lot of energy getting just right right I mean switching from proof of work to proof of stake being like one of the many Milestones uh that you know we we as a community within crypto has been working for years and years now on uh to make sure that it is that like pristine wonderful place where um you know uh the the robustness of the network is is never in question right or it's always top priority um so yeah I think um I'm totally

following obviously preaching for the choir a little bit but um I I guess you know maybe just to take the the skeptic or the cynics uh perspective and to speak to that uh uh mindset a little bit the goal here like make no no mistake about it the goal here at least for me is always to build more powerful applications like services that are just fundamental like truly like 10x better or or new uh as compared to what's existed before right like I think this is like we live or die as a community on this so can we like make just products that you know all else being equal uh can outperform what exists today right and and for me anyways like bringing this like AI upgrade to uh blockchain networks is one of the like key

ingredients of how we do that how we build truly unique truly you know beyond what's already been done which is also obviously very impressive but how we truly like be competitive in these just Marketplace of ideas um and then again I have an impact so uh you know the goal is to ultimately you know products with capital P make something that's just like dynamite yeah for sure yeah I I very much much like resonate and like like that idea of like what ZK is like the Ultimate Upgrade that allows like Plug and Play into um you know look like into crypto like if we don't want to be like too maxy about it but um at least this audience knows like how I feel that like there's cryptos yeah it's ethereum right like

that the purpose is to have one shared space and I don't know I I think like for me well you know what I I won't wax poetic too much on this like let's continue talking about um so so we kind of walk through uh the Rockefeller bot and like why it's so important to have this like ZK proof and really the ZK verification on chain right and like that as you mentioned is like a pretty light lift that's essentially like a couple math operations using a commitment and you're basically checking some equivalents and like if if it verifies then you take an action if it doesn't verify you basically like halt the what is challenging is creating well

you can you tell me what's challenging but like obviously like what all the hard work is on is on the computational step and like creating um systems that like are verifiable so can you talk a little bit about um one like what what are like the broad Strokes of the challenges and like how do you make a system um basically like generate ZK proofs that can be verified and then two like what are the specific problems around ZK or sorry around machine learning that make the challenge even harder than uh just like general purpose Computing yeah totally uh and Rex I think uh like these are like exactly the right questions to

ask in some sense because they were the ones that we were asking like after Rocky bot it's like okay yeah like this this idea is kind of cool but like what do we need to do next if we want to actually productionize uh this this whole process right and so I guess the first thing we did back then but also maybe the first question that we just felt like we really needed to have a concrete answer around is like how hard is this actually right like as we scale the size of these models into something that's actually useful and not just like basically guessing and if each prices would go up or down um like what are the challenges that we're going to bump into and so what we did was we assembled six uh of the major kind of popular proof systems that were

were you know widely used today um you know from different schemas or different like techniques um all different flavors and we tested the same kind of AI models across different sizes on all of six group systems and very quickly we we realized that uh first of all like performance varies a lot sure but also maybe more importantly uh the variance in performance like like actually doing this was a little bit painful and not just because like the software is you know bizarre and unique and we had to like and you suck but also because like these ZK proof systems as they exist today were all built to prove the EDM or some VM generally which makes sense right because you know we have

blockchains and we want to make them cheaper so we just want to like prove blockchains but locally and put that proof in the like the expensive computer um which is a little strange because uh you know AI compute is both simpler and much bigger than like VM compute right like in some sense uh you can consider like the compute load of an AI model uh basically to be mostly matrix multiplication this one operation a bucket of non-linearities and then times a bazillion times right like just that simple set of operations at a massive scale we don't really have to worry about memory or Registries or all the other complicated VM stuff and so it was like and it was like you know you have like a you have like coal or something like

just like train fools of coal and you're dragging this thing with a school bus and like school bus are awesome they can go like off-roading I guess they can like turn and and you know like fit kids and adults and all the rest but like it's not designed to like be pulling like you know parts and parts of coal or you know a shipping containers that's it um so that kind of brought us to like the core Insight that has been like the driving uh uh you know like technical mode for for the company for most of this past year which is what if we just built a specialized proving system right let's let's just like from the silica up build a ZK system which was tailor-made for the kind of mathematic operations that AI um you know problem statements use right basically a lot of linear

Transformations because like theoretically like if you if you if you like just imagine like a DK programming system which is only slightly better at every single matrix multiplication when this one operation happens like you know tens of thousands of times it should just conceptually be a lot faster and so that's that's been like the big I think insight for us and what we've been working at for the past however many months So when you say um the thing you just said at the end is basically that um if you can using ZK create a this is probably too specific but let's say like a matrix multiplication that is like slightly more performant because that matrix multiplication happens so many

times um You're Gonna Save a lot of time and I think what you mean by more performant is like a little weird it's in ZK world right because it's not necessarily the actual operation itself is more performant it's actually probably going to be a lot less performant but it's that the verification is much faster than actually running it yourself is that what you're saying or am I kind of like yeah yeah I mean here let me make it super concrete right so um the project we built after Rocky and after the paper where we did the comparisons the benchmarks is uh it's a game it's called Lila is the world won't get into exactly how it works that's for

maybe later but what's important is that um the statement of work right the mission here is to bring a 4 million parameter AI model into ethereum right whatever it takes and also don't break security Now if you were to run this 4 million param model which is much larger than rocky but still relatively small by AI standards but if you were to run this on L1 on ethereum mainnet a single time one forward pass would cost and obviously gas varies a lot but on the order of about thirty thousand dollars uh never mind the fact that you couldn't actually do it because block space and the rest um but that is obviously queries like you know it would have to be a tremendously value generating AI decision to justify that now if we pick

up this model and we put it on L2 I.E one of these EK proving systems uh these zkl2s uh which were you know made to emulate the evm um it would be about a thousand bucks not too bad right like enormous cost savings compared to L1 but still this game better be like amazing right uh to justify that if you then take it and put it into a proving system which has been like tinkered with and tailored up for the specific kind of math that this AI model uses which is what we did it ended up being about two dollars uh per decision right so you know in terms of the cost curve went from 30k to two dollars for the proving and then you know a couple more for the settlement on chain

the problem even though that sounds like a massive Improvement is to just run this model once on AWS or even on your laptop costs and electricity a fraction of a fraction of a fraction of a cent right and so fundamentally it's still the ZK proving is still very expensive uh and so for us the question was okay can we keep going down that cost curve until adding AI models into your smart contracts is barely a lift at all makes a lot of sense and so how much um are you guys concerned about the process of adding ZK into ml in terms of like both performance and like like speed of execution but then also like costs is that on the ml side is that actually like something that's going to

become a problem or is it really more about um like getting all these pieces to fit together yeah it's it's a great question it's a question that honestly we obsess over right because like in some sense like if you look at things from the blockchain standpoint it's like oh my God this is great like you're like massively reducing cost of running these models but if you're looking at things from the AI standpoint I.E no proving let's just run it like once on AWS and that's done it's like oh my God you're multiplying still because like you still have to do the AI compute you don't save any costs there but then you also have to do these like sophisticated cryptographic operations on top of the AI compute so from the AI standpoints like man you're going to like take my overhead and blow it up by like a

thousand X right so if it used to cost me a dollar to run some like some a day worth of AI I need to pay a thousand dollars at the end of the day to show people that I didn't manipulate the results right that's a like that's a heavy lift and so a lot of the focus from the modular side and by the way that's that's about the the blow up right now like the say the r blow up even optimistically it's about like a thousand X on top of just doing the ad once naively Prime and sorry that's a that's like a cost um blow up but is there also a like does that increase the time to let's say like do a full pass of the AI model yeah yeah in fact that is in some sense how the cost is reflected I need to keep the

computer running for longer and so I got to pay you know the Columbia River I guess for hydraulic trick you know like generators right um so the so it's not only that you're have problems on the cost side that like this whole construction might blow out your latency so much that applications might not be buildable exactly yeah I mean this is uh you know for us like after we we built Rocky we did the paper and we did Leela the second project we were like okay if you want to keep going down this this train right and actually get to a point where AI models are useful we need like a step function change we can't just like incrementally improve things anymore and you know the good news is of course like like it seems like there's this opportunity by specializing the proving system but even

independent of like the the approach just from a statement of work standpoint uh and a cost profile standpoint we need this change all right we need I mentioned earlier about like School Bus like towing a lot of weight we need a bullet train right something that just runs on a single track but goes like blazingly Fast um so that that's been you know the the the the motivation uh behind just just modulus really right like let's massively cut down on that cost by hyper specializing so okay is it fair to say that like the big bet and like the thesis of modulus is like we need to and somebody will build it we're going to be the ones to find that step change exactly yeah like leave no Stones unturned uh and explore every like

forgotten idea in the in the legacy of Photography and ZK until we can prove AI in a really like efficient low weight low latency that's a great point we didn't even I'd even mention Rex but yeah low latency kind of way so that we can Empower smart contracts with AI models but also just like in general like in the world introduce this idea of verifiable AI faithful AI yeah so when you're like looking at it today and and it's like a big hairy problem like you wouldn't be able to raise money if you hadn't answered this question right but as you look at it today where do you think that that step change is going to come from obviously the answer is going to be everything that I just say right now but

um you know if like you could really just like in your gut and like what you and your co-founders are like really see it is it really about silicon level like you know Asic level silicon specialization is it about Integrated Systems so that you can handle like um you know latency and like transfer problems like down to you know a micron of a second um is it I mean is it about like just creating more performant models both on the machine learning side but also on like developing just the number theory in the cryptography side is it something that I'm not even aware of like where do you see the step change like really like what's your secret sauce yeah yeah no no

this is uh the depending on how optimistic you are multi-million to multi hundreds of millions multi-billion dollar question right um actually I think I'm gonna answer this question at in a weird like uh kind of annoying way which is too too like very briefly visit like like super recent history right which is you know like my whole team at the founding team we came from like the AI right and it's easy to think that like we've been doing AI forever right right but it's not true like deep learning basically the modern Paradigm of AI didn't really start to like happen until 2009 right and so like a very and you know the theory has been around for like 40 plus years right uh and so you know the reasonable questions like what

happened in 2009 right I know there was a big Financial collapse in 2008 but then what happened right like like out of the the wreckage of the housing crisis like how did AI you know just like deep learning show up and the answer is gpus just like gpus became widely accessible and you know it's it's worth only like gpus is just a different way of arranging silica uh than CPUs like you can build a server Farm of gpus using CPUs right uh like theoretically that's totally doable it's just like much more expensive uh it turns out that instead of having a couple really powerful cores when you have thousands of not very powerful cores you can just turn through uh matrix multiplication as an example much more efficiently and

when you do this at massive scale these theoretical techniques um which you know power modern AI which used to be not very performant just have like magical emergent properties right like if you do if you took uh Chan GPT um you know for whatever and you you reduce the pram count by a thousand it's not a thousand times worse it doesn't work at all but somewhere along the line you know boom it starts to be able to perform right well and so sorry to take us back to earlier part of our conversation like when you make order of magnitude improvements like new capabilities are possible and like this is like the most high tech but like most like perfect example of like you could calculate by hand a neural net of like I

don't know let's say like a hundred like you'll see like it's dumb like it doesn't do anything it's like bubbles and chart you know but um yeah once you like crank it up to like 100 billion or like 3.5 trillion or whatever it is like your computer talks to you yeah right yeah and that's like a super like I like I I lament being like the VC who like missed that I guess quote unquote because it's like not at all obvious right like what how does that make any sense right but I guess my point is that and if you'll you'll kind of um tolerate my my uh my perspective here um this is a cost story right like we hit a point in 2009-ish where suddenly the the fundamental like cost thesis of compute change and boom AI popped out

right and I guess from the modernist world view this is available to us in the ZK world as well if instead of just optimizing for evm or EPM operations we built specialized ZK provers that took advantage of all the other chords uh virtual literal uh in a compute cluster or a compute node um we can start to prove really really like diverse and expressive kinds of computation that previously was just like totally irresponsible right now we happen to call that compute AI but you can call it other things as well right um so that that is that is the kind of like um the technical insight and the motion anyways of technical Insight that we're trying to I think uh bring into the the world of ZK proving yeah very

and so like I think that's a super good way to frame it because you contrast yourself very well with uh one of my previous guests with the guys from risk zero and like what what they're doing is like they have the same Insight that you and I and basically like everyone at The Cutting Edge of the next bubble has which is like what you what ZK cryptography allows you to do is computation outside of ethereum and then project it into the evm but what they're doing is like creating a general purpose VM and then like basically saying we're going to create environments that are accessible to developers like as they understand them today and then they can focus on building on application

building applications as opposed to like you know any of this crazy stuff you're taking the same approach but instead of saying like we're gonna just build like general purpose tools to see what you guys come up with it sounds like you're saying like we know what like the real professionals like the real Cutting Edge guys need and we're just going to skate ahead and build the tools that they need today right so um 1D is that a fair characterization but two like how what are the next customers or use cases or like how do you see the work that you're doing for machine learning Branch out into other aspects of computation yeah totally um great question and certainly um like

the risk zero guys are awesome uh no disrespect of what they do at all of course quite frankly like much harder than what we're doing like VMS are complicated uh as you can imagine right and and I think that obviously like uh like we're all kind of on the same team here which is like hey you know you have Smart contracts like let's just like add more logic to it right like then you can build cooler stuff and we all like benefit from a world where Services aren't just like super compute uh limited right um I think right so you're exactly right in that we're just like a bit more opinionated we say hey it's not just about like expanding compute uh the surface area or the design area for compute which obviously would be great but it's specifically about adding AI features right because we think AI is

like the right beachhead if you will for like how a Dev should maybe think about like oh how do I improve my smart contract I don't try like decision trees right or random forest or try uh you know a recommender amount uh model right to give your users like more personalized results um to uh maybe like generative results right maybe that will make this smart contract-based service just much more friendly personal powerful so yeah we're just like on the same team just like from different angles uh and but yeah so so I guess that's um that's like the high level but uh on a functional level as well we just think like a more tailor-made Tech stack makes sense for AI well um just because I like if you'll permit me again you know let's

do a graph like if this is a compute like more compute and this is like utility right zero compute there's no utility sure and a little bit of computer is like a massive amount of utility and it comes down again I would say this part is like uh is like L1 L2 level compute and I think you know the risk zero guys and everyone else you know us included is saying hey as you go more there's like even more value uh modulus I just believes that at the very end here as you go all the way up there's another Spike which is like the AI uh um you know like the Treasure Trove of value that can be unlocked with AI level compute and actually we're going to build just for that because we think that's like a great way to improve your smart contracts yeah for sure and I

don't mean to um like put you against zero as a negative thing if anything it's like like yeah like the the more people that have like the right ideas and the same ideas means like we're on to something here but I mean I think just like specific specifically um bringing them up to like point out that like they're doing general purpose you guys are doing machine learning but like you're not like the purpose of modulus is not to live and die on machine learning right it's to conquer machine learning and then using all of the like specific applications or silica or like whatever you built in machine learning

to apply it to other parts of compute that are like super super intensive super super specific but like need these step changes in order to like actually become like zkifiable and so just to ask it again like what what are the other kind of you know genres or compute applications that um like you think that modulus like really has a chance to change after you guys are or in conjunction of changing machine learning yeah it's a great question Rex and I guess you know the the most like directly cancer is to just ask like like what is an AI model like like the underneath the label and if you know if you pop open the hood of the car and you

look at the engine an air model is just a ton of linear Transformations right it's just like a lot of like statistical operations turns out like stats at scale like if you stack enough stats together eventually you get AI right and so when we say that you know our prover and our host stack is oriented towards AI like like compute or AI level compute what we're really saying is any kind of big data-esque statistical operation should be just much faster uh at production scale when uh you you take this like Taylor may approver approach right so in terms of the early use cases which is kind of like the next gear shift where we're getting into right like we we did the early experimentation we wrote the paper then we did all this r d around a

specialized proverb to see if it's like you know meaningfully faster and now we're throwing it into third gear which for us is what are the first big production level applications right and it kind of follows a little bit kind of maybe what you'd predict right it's folks who want to do you know statistical operations or within the D5 regime it's folks who want to do generative art for nfts it's uh different Services who want to prove your identity using facial scanning uh uh models or Irish decryption in the case of worldcoin which is you know one of the folks we're we're working with um it spans the Gambit you know everywhere where AI seems to be making a difference in the quote-unquote rest of the world IU web2 it seems to be you know following a similar at least exploratory Trend within crypto and it's

not always AI right sometimes just like massive stats uh but we still think that this specialized prover will fundamentally just be much more attractive from a from a cost standpoint for those applications yeah for sure well I I can like continue to go on this like conversation forever but um let's bring it into it close here so what like are the most exciting things that are uh coming up on modulus is you know like roadmap or radar like how do you yeah I guess when you're you know like at home talking to your fam family or talking to your investors like what are the next Milestones that you see that that you guys are skating towards yeah so definitely um it's a busy time of year for us

um I think uh the next big thing so we'll be at uh ZK Summit uh 10 I think where we'll be debuting basically the the I guess the theoretical uh underpinnings or foundations of our specialized pruning system so that will be this month uh around October november-ish we expect both the audit on our first code base to be complete as well as the first production uh implementation of that cobit um and as I guess a maybe a metric here we'll go from the current world record held by us which is 12 AI proofs per day to about 10 000 AI proofs per day uh within the next month um so that will happen we'll also be open sourcing our code base we think that will be really meaningful when it comes to completing the security story around ZK for us as well as just like

hey folks you know like we think that specialized proofing is cool you should look under the hood and see if you agree right um and then by the end of the year we expect to be onboarding um our first handful of uh you know um let's call them like more white glove customers right uh in the early days before definitely matures and be kind of Paving the way to a self-serve platform so that's the the rest of the year for us uh hopefully we get to at least most of those things uh but you know we're still a small team so we're we're there's more work to be done than people could do it yeah for sure and so these hopefully by the end of the year these white glove uh like initial applications like you said that these are things that are essentially like how do we

um use machine learning to process like basically data related to blockchain activity exactly yeah you got it you got it very cool very cool well all right um like again before I let you go uh first of all just thank you man like this is such an incredible conversation and really um yeah man I think like you're so clearly right at like the beach head of like two technologies that we all understand are moving so fast and are we're just behind the curtain of like the real specific ways that you know not the theoretical applications but the actual Transformations that our world is going to see and so um man I'm just so impressed and excited for you that uh what you guys are

building with modulus and uh man I hope to just continue to stay in touch and have you back on the show as you guys like continued at milestones and and show us the the intersection of these two incredible Technologies so I Rex the pleasure is all mine uh thank you so much for having me I mean the fact that you care about this stuff or your audience does uh it I I'm Charmed just like on that premise alone never mind like the excellent conversation so thank you for just genuinely thank you for having me for sure and uh Before I Let You Go um where can people find you like if they want to get learn more about modulus or get involved like what do they do yeah I mean honestly the uh we're not really like Twitter x uh you know animals but the easiest way is

probably to just follow the modulus Twitter account and if you're like man Daniel looks like he really is in desperate need of an ego boost you can also follow me personally I'll just be posting like maybe behind the scenes stuff but yeah Twitter is the easiest or x uh or you know we gotta move to like Rex what's the name of your uh uh where you moved your stuff to uh I don't know the outside yeah yeah I don't know man we're we're all struggling here and um all I can say is I stop putting effort into X because like I'm pretty sure that it will be gone before the end of the year but until then that's where we're all coordinating so yeah maybe maybe our medium blocks yeah as a as a backup and yeah maybe mirror after that

but anyways yes well again Daniel thank you so much it's been a pleasure and I hope to talk to you soon thank you Rex really appreciate your time thank you