Episode 317 - Enhancing On-Chain Intelligence with Ritual

Transcript

[Music] welcome to zero knowledge I'm your host Anna Rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online this week darun and I chat with NJ and a niche from ritual in this episode we revisit the AI crypto intersection we then explore the ritual product which is modeling itself as an AI co-processor they are building tools for ML enabled D app developers and are

offering ways to integrate AI models into smart contracts now before we kick off I just want to highlight the ZK jobs board for you there you're going to find a lot of new jobs from Top teams working in ZK there's been a lot of postings lately so if you're looking for a new opportunity to get into the space to work in ZK be sure to check it out and if you're a team looking to find great talent be sure to add your jobs to the jobs board as well so I've added the link in the show notes now Tanya will share a little bit about this week's sponsor Alo is a new layer one blockchain that achieves the programmability of ethereum the privacy of zcash and the scalability of a rollup driven by a mission for a truly secure internet ELO has interwoven zero

knowledge proofs into every facet of their stack resulting in a vertically integrated layer one blockchain that's unparalleled in its approach Alo is ZK by Design dive into their programming language Leo and see what permissionless development looks like offering boundless opportunities for developers and innovators to build ZK apps this is an invitation to be part of a transformational ZK Journey dive deeper and discover more about Alo at al.org and now here's our [Music] episode today tun and I are here with nuraj co-founder of ritual and Anish an engineer on the founding team welcome to both of you thanks for having us glad to

be here hey trun hey excited to be back and uh talking to a team that I uh both I'm an investor and adviser in nice for today's episode we're going to be diving back into the topic of kind of crypto and AI something I don't think we've covered on the show for quite a while early 23 we started to sort of think about it especially in the case of like ZK and AI but um yeah I'm really excited to get a chance to jump back in and I'm also excited to have you on the show nage uh we have a nice backstory I mean I met you long time ago back in 2018 2017 maybe you were the person who introduced me to tun and I

was looking this up today just to see kind of that that post but you were just like I know this guy guy he's got this company super smart you should meet him something like that and I'm really glad that you had made that introduction so yeah it's and it's very cool to have you on the show yeah I'm excited uh I guess to see how how far it's come and I'm a big fan of the show and I I listen to it quite quite often and and reference it quite a lot um so I'm I'm glad to have uh had made the connection and see how it's come nice also we were talking about this you and I we did do an interview long ago um one of the earliest interviews but it doesn't exist anymore so it's the long lost ZK podcast episode episode

18 it's like an Easter egg it's an Easter egg yes I don't remember a ton of what we talked about but I remember we talked about non-interactive proofs of proofs of work that was like the big research the the the Andrew Miller thing yeah yeah that was like that that's such a different era when people cared about that that's like it's so divorced from right now like you know those signs that like amusement parks you must be this tall to to ride this ride you must be like 80% Cipher Punk or more to to to to actually care about that that type of stuff in 2024 that was also the era I don't know if we talked about it on the show but

that was like the era of truebit which came back well I keep seeing all the truebit people everywhere so I feel like there's something something's up yeah I was at I was at a I was at an academic conference last week financial cryptography and I saw a bunch of trit people is that still around like cuz I I feel like that that project was so ahead of its time it really was like it it had the makings of things that have since come out but it maybe it was like missing something or it was too early I don't know too early what was it wasn't it kind of like a wasn't it fraud proofs wasn't it optimistic rollups in a way yeah it was basically offchain compute with some form of a fraud proof okay amazing all right NJ you introduced me

to trun and I've always sort of wondered how did you guys actually meet maybe maybe trun should answer it yeah I guess I we met through through this uh this other investor in crypto who a lot of people know but has one of the most minimal public presences uh of anyone of that stature and uh then you know over T nage invested in my company so you know it kind of was the early beginnings of of like the post Ico new beginnings of crypto 2018 yeah I I remember looking at when tun was fundraising I remember looking at his deck and I was like this is one of the most interesting decks i' had

seen for for defi um he was talking about some really like unique interesting Concepts that I think they're kind of working on today um things around you know insurance and and monitoring and and governance and and things like that I mean it was it was right around the time that we also invested into compound um we we led the the cedon of compound finance and it's just cool to see how the how the industry has changed in the I guess like five six years since I I definitely feel a lot older let's put that way NJ you just mentioned we so let's talk about the company that you were at before ritual uh you're part of polychain were you one of the earlier

people there like were you the first gener a poly chain investors yes I was in the in the apartment office ah wait what what employee number were you actually would you say less than five I think some somewhere around there I was um the investment team was three or four people had you done anything before that nage were you a student coming out of school or were like had you already worked in VC and stuff like that um I dropped out to join poly chain oh wow I was doing research in college I went to to the University of Illinois uh and I worked with uh with Andrew Miller for you know two years and the summer of 2017 when poly chain was kind of growing

I met poly chain through one of their LPS and I was looking to kind of do something in the in the space and he said you know you should go talk to Olaf and I I remember meeting him and he was one of the most like weird kind of eclectic interest people I'd ever met and um through that I you know I first joined as an intern and after a month I knew that this was like way more interesting than what I was doing in school and I could you know take my research and try to kind of apply it in and investing context and so I ended up dropping out and joining in a little bit later that year is that 2017 2017 yeah crazy that was a wild time you were at

Poly chain for quite a while what was your time like there I mean you guys got to be in so many of those like early deals when there were I mean I just remember at the time there I mean I don't know how many VCS there were but like really legitimate kind of serious VCS who understood the tech seemed like few and far between and I feel like you guys were definitely some of them yeah it was it was a really amazing vantage point to be a part of and also to see how the industry evolved and changed over the the six years that I was there um I just remember you know the types of things that we were investing into in 2018 was you know vastly different from from

2023 I I remember 2018 was you know I was reading you know white papers of different layer ones and it was like new consensus mechanisms and and things like that icos really big and the industry you know every single year in crypto you you get these compressed Market Cycles so you really can kind of age as an investor I think a lot faster than you can in in in the traditional World sounds good and bad age like fine wine get wisdom or yeah age I think I think both I think both nice given that you invested in a lot of ZK stuff early and this is the ZK podcast it would be remiss if we didn't

ask you what your favorite ZK investment was oo oh man I will say I will say one of my favorites um I don't want to pick any one favorite um so one team that I I really enjoyed working with was the Mir protocol team so D Daniel and Brendan yeah so I remember we met them I I think in the summer of 2019 they were building on top of marlin and they were trying to build a new ZK layer one this was just at the time when recursive proofs were just starting to kind of come into the you know come into the fold and they had some really interesting ideas around you

know how you can basically start using ZK proofs for things outside of payments during this the sapling and pre-app times like generating those transactions used to take a long time which now you know feels like a like like a relic so working with them was great because they both were not you know cryptographers by background but we're both very established software Engineers with math backgrounds that very quickly became some of the the kind of leaders in the ZK space and so getting to watch them do that was was exciting and and getting to work with them and then you know fast forward to I want to see it was maybe 2021 I I don't remember exactly working with them to kind of help the the

polygon acquisition come together and I think they've taken it to to some really amazing Heights um the series of of um plunky stuff that they've done and um you know seeing it you know actually get to a really massive production scale with polygon amongst the the kind of other ZK Acquisitions they made so that was just a really cool experience for me to see and and also you know couldn't have happened to to better people very cool that's a nice shout out Anish I want to hear a little bit about you and your background what were you working on before joining ritual are you coming from crypto are you coming from AI or neither or both very much I I I feel like Anish has the greatest Street

Credit of everyone on this podcast right so flatters me with that I am coming 100% from the crypto side of the world okay and you know picking up the ropes on the AI as I go uh I started working professionally in crypto for narage uh coming up on four or five years ago I think okay I sent him a Twitter DM in high school that he graciously opened and actually responded to and that's what landed me my first internship at Poly chain that summer whoa um so you know a good full circle finally getting to work for nage again a couple years after the fact cool but yeah spent a

couple years I think a year and a half at Poly chain um where I worked with nage I worked with ulish who's the other co-founder of ritual on the trading side of things and I spent some time at Paradigm on the search team doing open source development and finally prior to Ritual I spent about a year and a half doing onchain me uh and eventually Naraj nckl reached out and I said absolutely you know it's a be a privilege to hop back on when you say you were doing meev stuff what were you a Searcher yeah I was a Searcher like were you just doing that on your own or was this like part of a crew just by myself okay what was that like uh it's a lot of fun okay I'd say you you get to experiment with the

limits of what's possible on chain um and so yeah definitely a lot of fun things there I remember having a while back uh Dean from dialectic on and he I mean he described that like folks who do that alone as it's like kind of a lonely existence that you're out you know grinding but there's no one around you and that you know when you in his case he had built a team of folks doing that and sharing strategies and stuff um did you actually have people also to share strategies with or were you really just doing this on your own it's a competitive space so you share strategies when there's a benefit to sharing a strategy but I wouldn't say I

went out of my way to disclose what I was doing or what I wasn't doing but Dean and I go back and forth on the team versus independent ah which ecosystems were you focused on I'd say 70% was ethereum mainnet so that's definitely where the bulk of opportunity is and I'd say 30% was a long tail of chains that a lot of folks have never heard of so you know chains where there's 10 validators you have to email them hard you mail them hard drives so they can get you copies of their archive node the long taale of things uh the other 30% so with today's episode something that I'd like to do is I want to start us off talking about the intersection of AI and cryptography because I know that ritual

lands in that intersection but I want to talk first very broadly and I feel like since both of you have experience on the VC side you will have seen a lot of things and so I kind of want to cover some of those go through like what seems feasible what doesn't seem feasible and then I want to dig down into what rituals focused on so I had a few already off the top of my head of like how these two things can intersect I mean we've over on the ZK side thought sometimes like DK could be used to save us from AI save us from Deep fakes but it could be also that cryptography is enhancing training maybe or proving something or keeping

something private uh maybe we use the immutability of blockchain plus Ai and that adds an extra feature maybe there's some other way in which they intersect tun do you have some ideas here yeah I was just arguing that like the different components within AI Engineering Systems some components are more friendly to privacy preserving Technologies some are less friendly and less useful and sort of there's kind of the spectrum that you have to kind of establish of like which parts are worth having high amounts of trust in either economically or cryptographically versus which ones aren't and that trade-off surface is quite large so that was sort of just like a rundown but I want to hear from you from you guys like a is

there anything I've missed there or there like do you want to dig down on any of those particular kind of ways that they can intersect yeah I mean I think a lot of those are are interesting areas and and very applicable and and I think maybe kind of in your questioning you're you're highlighting that this is a really vast space and there's a lot of teams working in very you know very different areas um to give you I guess a little bit of the story of you know why we wanted to go work on this problem the company was started in 2023 largely you know speaking from myself I mean others had different motivations but what really interested me when I was investing at polychain was trying to find out what developers would

be doing you know what they would be developing with in the coming years um everything from new blockchains interoperability Solutions ZK and and then you know obviously newer things today like rollups reaking Etc I was always always really fascinated by what would developers be doing on blockchains in the future sure that they're not doing today and as I started learning and and kind of looking at the the existing meta of the AI space in you know 2022 23 it became really prevalent that there were some really interesting intersections that were kind of bidirectional um particularly obviously you know cryptography can help AI in a

lot of ways because AI today is quite centralized both from from a development standpoint from a deployment standpoint a hosting and sort of um usage standpoint it's really only a couple of companies and and um you know people that are driving a lot of this development and being able to apply cryptography to a lot of these actions allows you to have guard rails around this technology development and really Foster a lot of interesting open- Source work uh that you really couldn't do otherwise and then and maybe I think less obviously although it's maybe becoming more obvious now is AI can help supercharge a lot of web 3 use cases um

being able to take this you know new type of AI computer and attach it to different types of web 3 use cases um we we find that there's a lot of interesting examples in a bunch of the different subcategories of crypto and so as we're thinking about you know what's the opportunity space to tackle there's there's really sort of three or four areas if we can think of it in buckets of what you can do in the AI crypto space today there's the compute so this is everything from training to inference fine-tuning and also attaching you know privacy to all of these things there's things at the model level so this is things like model governance sort of

provenance of of um how the models are actually being used and then one level above that is the is the data so the data that's being fed into the models being able to track the provenance of that labeling how things you know come into the models and then finally is the is the applications so the applications that are built on top of all of this so we can dive into the into it a bit more but that's generally how I like to kind of dice the the space in terms of the different opportunities that are that are available and and people are working on I really liked the way you kind of kind of talked about how they're useful to each other that bir directional intersection this idea that like AI can

use types of cryptography but then like the cryptographic systems we have can also benefit from using certain kinds of AI but I do feel like all of this it becomes much more nuanced when you get into what you're actually using in which direction and how maybe broad that AI is I get the sense that if like you use AI for blockchain you tend to be using it in quite a like narrow way so far am I would you say that's kind of correct like it's for a singular thing like figuring out one like getting it to do one very small very clear action my feeling today is that the reason that people have have not done AI on chain today or you know why is AI crypto

happening in 2024 and why was it not happening three years ago is is a couple of reasons one is that the compute was to too large to scale on top of existing blockchains it would take too much gas or um the cryptography wasn't ready to even do it in an offchain capacity so that's one the second was I guess you could call it the flourishing of of generative AI models really opened up a new type of use case that we didn't view as as possible before I think the The View uh from for the general public for AI you know four or five years ago was that it's really interesting but there's not a ton of use cases that

are very consumer facing maybe it's for things like recommendation systems or self-driving but we didn't have as visceral of a use case as say Chach PT or mid Journey or Runway MH so after those things came out you know after we had the transformer in the um in the AI research landscape and then we had large language models and then you know frankly chat GPT using the GPT um system to show the power of this this AI intelligence we now found this really exciting interesting use case that now hundreds of millions of people are using in a matter of you know two years and so I think it's those two things largely

that why today well today we have new use cases for AI and the cryptography is a lot better and we've just gotten smarter you know there there are more people thinking about what the interesting things are at the at the intersection H I guess one question I have is uh you know you you have actually spent time investing both AI companies and crypto companies before so I'm kind of curious like what about the kind of changes that occurred and things you saw made you make the jump right I feel like in the time I've known you you you've been excited about lots of things but none of them quite made you you know jump off the diving board so like what do you

what do you think is the thing that like catalyzed that spring off the diving board metaphorically my investing has largely been driven by my own like curiosity and crypto aai was an area that I had started spending time in in the summer of 2022 so around the time of of Stanford uh the Stanford you know blockchain uh week or event that they usually put on I started spending time with both AI academics like some folks at stand ER and in some of the corporate labs and then I was doing my usual you know talking to researchers in the you know in the crypto space particular in the in the ZK space and what started as

just a very simple problem can you do inference inside of a inside of a ZK circuit created this whole like web of of um interesting questions to dive into the initial answer after looking at it quite a lot was sort of no like it's you know putting a really big model inside of a circuit is still a still a large problem today and the the proving overhead is still quite high but what that catalyzed was this question of you know why is this even interesting and so as I started spending more time in both areas and getting getting a sense of what people were looking at what they're excited about as a crypto person seeing that the way that AI has started to

develop into you know the board issues at open Ai and and transparency issues around the the foundation models um as a crypto person these these alarm Bells started going off in my head I was like like it's this is like the same problem we've seen in so many other centralized systems and it's why like we're building in crypto today and that I think catalyzed something really prevalent inside of me to go and try to do something in the space I I couldn't find quite the comp that I wanted to invest into in this area I think it was still a little bit too early and so I'd always wanted to be a builder it's something that I I told Olaf day one of joining poly chain was I

wanted to go you found a company and that day ended up being in in in May 2023 that combined with being able to work with Nish AES and and so many other kind of amazing people on the team was the reason to to dive head first into it and this last thing I'll mention is crypto AI is is exciting because it has a lot of skepticism around it you know rightfully so and sometimes wrongfully so by both apes and also researchers and people in Ai and people in crypto and so there's kind of you know a little bit of a chip on our shoulder to to go make this work um as as a space and there's not really many

examples of this having worked in the past crypto is brand new yeah yeah building a layer two or building something you know building uh a layer one has data for the past from the past 6 years but this does not and that excites me both as an investor and as a as a builder were you looking into the zkl stuff when the spark came cuz I mean I've sort of mentioned this on the show I think early 2023 there was a zkl group that was created and there was incredible excitement for a very short period of time about that and there's two companies that spun out from there but then it chilled out um were you in that kind of cohort as well were you watching that I was okay as I was

looking at some of the newer ZK areas of investment and you know what would be exciting places to to deploy capital and spend time in one that came up quite a lot when I was looking in 2022 was uh was building these proving networks mhm um which which now have become quite um you quite a popular area for development and these proving networks I think especially would be valuable in the in the cases where you need to generate really really big proofs so I'm talking zkm proofs and you know proofs of ml compute and so as I looked into this early zkl work it felt

like an area where you're doing in some ways you're you're doing similar types of optimizations as you're doing in the in the proving landscape you know you're you're you know kind of optimizing how to paralyze the the fft and and MSM calculations and in the in the AI Vector it's it's you know just optimizing the The Matrix multiplications and it felt like that the zkl world could go through a similar cost curve reduction in in costs that we saw in the ZK space especially around you know 2019 2020 we had that plethora of you know 20 papers come out um snark toober exactly snark toober and that was

I think a primary driving Vector was if we can get the costs down and you know we can talk about later other techniques that that um we're deploying and employing today to to try to bring the cost down then we can enable this decentralized future this decentralized AI future that was you know impossible five or six years ago M I want to spend a bit of time understanding where ritual is in kind of everything we've just described so I want to go back to where you outlined the two ways that they intersect where we could bring cryptography to AI or we could use AI to supercharge blockchain or cryptography based systems where

would you place ritual On th like in those is it more like bringing AI to blockchain I don't know this might be a better one for for an nich answer okay I'd say it probably starts with bringing AI to the blockchain okay and the idea is we have a good mixture of folks that have worked in crypto and in AI in the past at ritual and I personally don't have any experience doing anything other than working in crypto so I'm going to borrow my you know borrow the experiences from the folks that I hear from but I know I mean trun can probably comment on this but in the traditional Financial world it's almost unheard of to not use Ai and ml models for day-to-day actions right if you go to

the bank and you want to take out a loan if you you want to check your credit score all of these things are risk models and ml models that are running thousands of times per second to finally come to a conclusion um only because it's not it's not like a finite State machine right it's not five IFL statements that determines whether or not Anish is qualified to walk into the bank and take out alone but in contrast that is the way that the chain works today right if you take a look at compound or you take a look at a or you take a look at other lending protocols almost all of that computation of can someone take out a loan can someone takes some action happens within you know this 22 kilobyte limit that's imposed by the chain of yes everything has to fit within this machine wow um

and anything that doesn't is sort of abstracted out to governance right It's let's go offchain and figure out these parameters let's go offchain and figure out should we upgrade the protocol should we not let's go offchain and figure out you know do we flip on a fee do we not flip on AE um and so the initial sort of vision for ritual was we've seen these inefficiencies to date and a common solution seems to be why can't we bring all of this on chain right why can't it be that any ml model could just plug into any crypto protocol and be able to lead some of that decision making on you know who can do what what permissions can change what parameters can change and being able to do that in a similar fashion to how you know traditional Finance works today

yeah um so I'd say yeah part of it at least what we're focusing on to begin is bringing that AI to the chain um but it's a it's a logical conclusion that you know when you bring AI to the chain you can also bring the chain to AI in the inverse of you know you you benefit from the provence the the digital scarcity the payment flows all of the things that we've sort of figured out in blockchain systems for the last decade already and those just come naturally to the AI space it's funny as you described this though it it's going I'm doing the throwback to that first interview I did with tun because I feel like tun when we met Gauntlet sounded sort of like this too weren't you kind of exploring some of those ideas well I think like the

hard part actually and I I I I gave a talk on uh last week on this sort of like philosophical differences in research between crypto and AI where like one fundamental kind of difference between them is sort of you know crypto is very much much focused on like worst case analysis of like what is the worst case under like the worst case adversary how well does this system perform can it be resilient to that type of behavior and AI is much more about pure average case Behavior like I'm I give you 5 million examples I just want to be really good on average you can have one really bad example where you completely mischaracterize but as long as the

average over the whole batch is good I'm relative happy right that's like a very high a very high level distinction between the two and what that means is that you know in the average case world you can handle higher dimensionality higher sizes higher bandwidth of data right so you know a neural net you know in some cases some of the number parameters in some of the newer models is like the size of entire blockchains that have existed for a long time right so it's it's it's like it's sort of a in which you because you're willing to only care about the average case Behavior you can like expand the set of things you can do whereas when you care about worst case Behavior you have to restrict what

people do you have to have a threat model for what it what the types of actions an adversary or non- adversary can take and over time there's some Middle Ground between these two right like right now you kind of have ai and the high dimensionality but average case crypto and defi in the low dimensionality but extremely adversarial you can't like make many assumptions about you have to place all the restrictions on the assumptions you can use but like somehow hopefully over time we find a way to like something in the Middle where it's like you can have more expressivity than you currently have on blockchains but you can also still have some of the guarantees you don't maybe you don't need the guarantees to be as strong as say like safety and liveness and consensus or soundness in a zero

knowledge proof but there's sort of something in the middle and yeah I think there's like many different ways to try to find what that looks like H for what it's worth the average case is also you know rightfully so the case that's applied 99.9% of times for sure right and and if we've seen anything in Defi and crypto governance it's it's in most cases you will just be on a well-defined set path updating parameters um and I agree it's like there's a lot of places where these models can slot in where you hijack out to like a human in the loop when the time comes if you pass some risk threshold um but it's really effective today to automate a lot of these things

with these models for the average base case and the best example I like to bring up is like we talk about these optimistic you know rollups and fraud proofs but if you look at it practically we've only ever had one case of a fraud proof being disputed ever across any chain and that was for I think an optimism Fork on eth proof of work a couple of years ago um so that's like in my example of like you know in in 99.9% of the cases you will never get to you know the long tail of here's an adversary or here's a risk model but when you do you know you can always open up your esape PCH and go back to the system as exists today yeah I mean I think this is sort of a a larger form Narrative of the general meta of either finding things in

between crypto and AI or things like Ian layer of like hey maybe you can reduce the amount of trust your you need for this application or you can like you can scale your security in in some abstract sense uh as opposed to just like strictly over you know always assuming worst case you're always overpaying right like there's sort of this just in time worst case security guarantee which I think is it's a very nice thing philosophically it's actually quite hard to achieve sort of like formally and mathematically and Engineering wise but I think that's kind of like the the blend of everything is kind of trying to move in that direction it almost sounds like what if you if ritual is able to create what you just described then

wouldn't it kind of replace some of the stuff that Gauntlet is doing but Gauntlet is currently doing it like manually or like more like you know with human brains I think it kind of this gets back to this question of like what's a worst case parameter and what's the average case parameter right the worst case parameter like margin requirements are very hard to set because like they have a lot of reflexivity to your security model right like imagine you have a an AI model that is allowed to choose a slashing threshold for a proof of stake Network yeah and the model is very good at average case Behavior but one out of every million samples it gets it suddenly sets the slash to zero and constructing that event that causes the

slashing to zero and then you can like double spend or whatever choose your favorite attack uh is a lot easier to do right so there's there's this kind of threshold of things that are really easy and regular to optimize like okay fees like what should the fee rate rate be right that's something where yeah if the model up no one cares right like okay the protocol got slightly less revenue or like a user took worse you know worse fee um but I think for these like extreme mechanistic things that it there there's kind of this hard threshold between the two and I think ironically not not related at all to anything we talked about but the the debate in the salana world right now over whether to have me pools or not is

a very explicit example of one of these types of things where the average case and worst case can deviate quite substantially in terms of things that we're trying to automate be fully automated or it's not and I think like inevitably this is one of the reasons why crypto is so much more in some ways difficult than AI is like the me type of stuff is like the real adversarial type of stuff just doesn't exist like adversarial machine learning if you remember we did a podcast with someone on this H yeah long time ago fa yes and adversarial machine learning is taking this kind of like average case optimization and like giving giving the model bad examples like this is a cat

and it's a picture of a dog right and trying to see if it can like unlearn how to do that and that's the type of stuff I think you can is where is closer to what AI models and crypto do okay but these hard threshold type of things these things are very sensitive to to me and like the worst case is like oh you blow up the network those are things you have to kind of you know and so I think they'll always be a world where you end up doing both and I think okay I think there's kind of it's kind of like hard to imagine a completely the worst case stuff handled all the time but like to anisha's point handled purely by AI you're saying you still need to have the human yeah yeah I mean I mean I mean like trading is a great example of this

right where people have been doing Quant trading for a long time you obviously you think they were like hey we'll just do fully automated AI models and basically you know no one maybe baring you know one or two trading firms actually really successfully deploys fully autonomous things at scale they have like a lot of automation they have a lot of automation to their strategies but there's there's kind of this thing where like I don't know the US invades Ukraine and like all of the AI Bots all do the same thing and and like buy defense stocks at the same time or something right like there there's kind of this adverse selection if you're fully uh and so these worst case type of things are like always going to need some notion of like

how do I tell the model to like regime shift but but to anisha's point the you know I think crypto right now is focused on making 100% of transactions have this kind of like worst case guarantee right but that's that but if you want to expand the use usage to more normal usage then you need to have a system where 99% of the time you can trust the average case thing that's automated and 1% of the time or whatever you need this you know so I I would just frame it in that lens it's like I don't I think there's always like there's always kind of inevitably especially in a a world where like you permissionless world where like someone makes a new smart contract that completely interacts with all the existing ones and breaks your

your fixed model uh you kind of still have to have some some kind of not fully on or like way out or governance to fix or something maybe that doesn't even need to be like the full governance right it can be some restricted subset of things right but I I just think in the permissionless world it's actually this you know the the onchain agent stuff is actually a little bit more complicated actually than say even trading because in trading you have like kind of a restricted action space it's not like people are adding and removing new assets all the time it's not like people are adding and removing new exchanges all the time but in defi it's like like you know I go on salana and I look at aggregator and suddenly there's 5 million shitty meme coins on 300

different deck as I've never heard of that like just got launched right and that now your your agent has to be able to know that the state space just blew up right it like increased a lot or trunk a lot right so so there's there's still like I think I think that's something to keep in mind and that's why I say the Holy Grail is sort of something that's like can be optimistic but then falls back to worst case but like is able to figure out when to do that right and like autopilot but with a pilot still there autopilot but with yeah exactly exactly I mean look at self-driving cars right they kind of have the it's a very Sim and that's where you want to be with blockchain the difference is self-driving cars started from this 99% of the time we want to

work 1% of the time okay you need an intervention blockchain start from 100% of the time you know you need you need inter right and we're going the other direction right and that that's sort of the what are the places where there can be more of that flexibility or more of that that freedom and I guess is ritual looking into those specific places right now yeah I mean I'm happy to start with that I'd say yes that there's a broad range of places you can slot in I'd say part of it is you know there are places you can slot in a model for 99% in the in the average case and you know to your previous question of you know what does gauntlets role look like here and in my mind it's like you know we are making tun and CO's life easier right where

it's we're letting them focus on the things that they do really best which is you know being brainiacs that understand the internals of all of these protocols and understand the worst case and letting them automate out the average case of you know I don't know T what does gauntlet ship like 100 governance 200 300 governance proposals a year yeah I I I I think it's it's it's somewhat increased uh the the annoying part and this is a part exactly where like the AI stuff if we if it was easy to do all the deployment is that if you think about about lending protocols just a simple simple this is just simple back of the envelope type of thing what am I doing I'm putting in one asset and borrowing another asset there's parameters for every pair of assets so the lending

protocols parameters are growing quadratically with the number of assets what happens every bull market the number of assets grows a lot now all of a sudden your parameters space grows a lot right and you want to try to only focus on the ones that are you know easy versus hard you know there there's a lot of stuff like that that like exactly is like the the long-term goal in my mind with this this yeah the example I give is like you know trun and Co is currently making these Gauntlet proposals to update these parameters to compound for the average case or for the worst case and I don't know how many people read those proposals right I certainly don't read those proposals I'm sure 99.99% of crypto doesn't read those proposal people in universities people exactly you know the defi clubs and

universities are reading those and deciding whether or not to vote and the idea for us is you know there should be a way for Gauntlet for tun to just take the work they're doing offchain take the risk models they're preparing take the ml work that they're doing to come up with these parameters and to just execute those onchain right if if we're already not having a majority of participants actually look at what's going on understand the system and you know you don't have to take my word for it I've made four Unis swap proposals thus far and two of them almost butchered the protocol had the team you know not swooped in and said hey this is correct so it's like most people don't read these governance proposals or figure out what's going on so in my mind it's like ritual kind of sson is letting

Jun letting Gauntlet letting other people run these models on chain keep some safe you know parameter guards where it's like you know say bound the range say if it's between 5 and 15% these slashing conditions then you know the model is executing correctly else you know kick back to governance kick back to the old way of doing things but that makes the you know the base case it makes the 99% case the the average case much more efficient which I think saves you know Gauntlet a bunch of time not having to make these hundreds of governance proposals and get by in from the community and you know figure out these PDFs that you have to upload to the you know compound governance forms and things like that is that now what rituals focused on being or is that just like one area that you think tools like

this could help what one area I'd say okay I'd say ritual is focused much more on being a general purpose execution layer for guy all right and so sort of like bring your own models be able to do inference on chain be able to consume those outputs in your protocols and it just happens to be that one of the easiest examples of you know how can I use this practically today which you know a lot of crypto xai folks are like you know sounds really cool there's a lot of money flowing into this what do I actually do today is you know here's a practical example yeah there's hundreds of dii protocols that have governance to little adjustments small things that keep the protocol going and the best thing is we can bring those on chain so you can actually see what's going on see

that execution see a proof of you know here I actually ran this thing I'm not just taking you know somebody else's word for it or relying on the five people that actually read this form to make sure the discussion is correct so it's like one application why do you have to do the inference onchain though could you not do it in sort of like an offchain bring it onchain way yeah so to be clear uh the the inference itself is quote unquote offchain we're not running like these n all models Within you know solidity VM environment or things like that the way that we do it today is we're exploiting pre- compiles and sort of these ways to hatch out of the solidity VM and do some execution and then bring those results back on chain I

can break down into that that there's really two ways to do compute on chain today right if you have the evm and you want to do something that's expensive the first way is you know an oracle system right this is what chain link does really well this is what other band protocol things like that do really well which is hey I have some comput let me issue off a request of hey I want to do 1+ one some offchain node executes that and then they return that response back on chain um that's how you know oracle networks work today our first product infernet Oracle network works similarly we have your pros and your cons and your biggest problem there is that it's very asynchronous right you're sort of issuing a request and you're waiting for that request um which makes it difficult and complex to build a protocol right if

I'm like a developer and I want to say I want to get some resp response usually it's I request it and I get it back immediately not so much I have to wait end blocks or I have to issue some meev so some keeper off chain can return my response or I have to introduce some Oracle to my system and so the way ritual works is we have these op codes and these pre-compiled that extend the evm and extend other virtual machines so that a developer can just as they you know do a math operation or a multiplication operation also do inference operations directly within the environment they're familiar with sort of just abstracting out any thinking of where it happens uh but to your question that does still happen offchain it's just kind of like a we're blurring the lines between what is onchain and what's

offchain this also sounds a little bit like you're working in the co-processor territory because there is something happening onchain how are you communicating out to those extensions exactly yeah definitely in the co-processor category um and that's how I would describe part of what we're doing our ritual we are communicating out via pre-m files so every node that runs our modified virtual machine is also running these implementations of our hyper optimized AI operations whether that's like an inference operation whether that's you know matrix multiplication a vector DB search and we just expose those to the VM so that's

sort of the interplay communication and this is something we've only started to see and I'm surprised we've only really started to see it in the last year or so uh come up in sort of the blockchain ecosystem and it's like arbitrum is doing stylus which is like let's execute wasum code via pre- compiles and now all of these like optimistic rollups and op stack and arbitrum uh orbit are starting to do like extend the capabilities of your blockchain by embedding these computations into the VM so that's kind of what we're exploiting interesting what is the input to these models are they I mean you sort of mentioned general purpose but like are you trying to get historic information in like if you're trying to get some outcome you're trying to get it to give

you a new rate or make that proposal yeah how like what does it taking in that's a good question uh candidly we're starting simple and the idea is let's get something into people's hands that they can play with and figure out what's needed and then expand from there to begin it's like you can imagine that at least all onchain inputs are exposed so whether that's you know past block history whether that's current inputs in the context of your solidity smart contract you can put those in like a blackbox inference call taking some inputs return some output um and based on that you can modify state so that's kind of what we're starting with and I expect that the initial models won't be something as fancy as you know don't imagine like you know we're running you know GPT 3.5 call and that's returning

some text input and we're parsing that on chain I think it's going to look much less like that and much more like you know simple linear regression models classification models things like that 50,000 100,000 parameters that taken some you know Vector inputs or some integers and returned some integer constrained output that can actually be used in a protocol but to be honest it's we don't know yet right nobody's sort of done anything like this and so it's much more the developers might go wild with it and we'll go from there yeah because you're building just the ability to use inputs that I guess you choose and have outputs you're not actually creating for one particular application but it's a little bit more General maybe one thing

worth talking about here that's extension this is is how are you thinking about sort of the network aspect of this like the act that people are paying for resources of like running a particular model people are maybe paying for the data sources maybe they're paying you know how are you thinking about sort of the marketplace kind of creation problem here not just like the technology part the way that we're thinking about it is there's there's a lot of participants in the ritual ecosystem and this isn't cont to sort of how blockchains work today where if you take something like the ethereum chain and if you take main net the only real participant is like a validator or a node that executes all of them execute

the same operations right if I send a transaction to etherium today every single validator every single node will execute that transaction come to a conclusion and say yes this is valid let's move on and we've at least in ethereum optimized for a very you know similar set of node prints right it's like if you have 8 GB of RAM if you have some storage if you have a CPU great you can validate the network we're thinking about it a little differently for us it's we will have multiple classes of nodes where whether that's compute nodes that handle just like basic evm operations whether that's you know AI inference nodes that handle a long tail of inference operations and even

subcategories between that two right you might have a consumer grade GPU you might have an Enterprise grade GP U you might be you know a dpin network that's running a fleet of thousands of gpus um things like proving nodes where you know today you can generate proofs on a CPU but there's a bunch of specializ and I'm sure you know the both of you are very well vers specialized Hardware companies that are building ZK as6 anding systems and so the idea for us is how can we sort of separate these types of compute and the types of things that are happening on the network into different buckets where where some participants might be better versed for that type of compute and they can really shine in executing those things um but that opens

up a you know can of worms in terms of how do you price this uh in terms of like you know the the broad example we like to take internally is say I have a you know an llm inference call that might take four minutes to do on like a consumer grade GPU but it might take 10 seconds to do on like an Enterprise h100 and so for us the question is how do we price that sort of time sensitivity that resource sensitivity of should I be willing to pay more to have access to a better GPU or should I be allowed to pay much less if the time sensitivity isn't a big deal for me and I'm willing to wait four minutes and so that type of research is something we're working on internally of now that we have this heterogenous class of nodes how do we

actually price it and figure things out are you creating the network of the provs like the gpus like the hardware that would actually be running these models is that part of what you're building yeah exactly not to not to get into the deep end of you know all the things we could be creating the the most sync way is we're focused purely on sort of the execution layer of how can we bring AI to an evm to an svm to another virtual machine environment and and downstream of that you get questions like okay you know who's actually bringing this GPU compute and things like that it just happens to be that our node our Network allows all of these classes of participant to very easily plug in right if you're an existing dpin network if you're an existing GPU

resource Network it's a it's a very minimal addition for you to also help validate our Network and it's like a symbiotic relationship even trun I need to add this to our you know how in the episode we did with Uma we talked about all the different roles that a validator eventually could take oh yeah yeah the another meta other than just like only pay for as much security as you need that seems to be true in the fundraising and project world is also the meta of actually we're going to have many different types of validators and there's going to be sort of a Marketplace for what subset are needed at what times and again I sort of feel like the thing that has both of these as its core Center is probably igen laay

because there's just like so many different types of node operators like built into that protocol but I think there's also a lot of other things whether they're in the AI world like ritual uh things like I think how a lot of the ZK proving markets want to evolve to will also be multi-resource based like so different types of nodes um and then I think there's some people who want to do ZK compute without a blockchain that have another version of the world that has multiple types of node Pro Resources so there's there there's there's a lot of stuff that's actually all at the same time kind of going for this multic classes of node resources thing and you know it's one of those things where it feels like when

enough smart people are all going after a similar design or architecture there's some like real TR to it and we hope so cuz it actually feels like a lot of different teams shooting in all these different directions and yet there's a similarity between them because I think of like the Igan layer reaking and I think of the da version of that and then there's the prover networks that stand alone and then there's the sequencers that I mean we think especially if it's rollups especially if they're using ZK there that they'll also be proving and yet I feel like because there's teams that also have dedicated their like they're building some sort of new

infrastructure layer their goal is to sort of educate one on how what they're doing what what a shared sequencer looks like or what you know all these different roles will look like I think it's because I have a validator company I'm off like we're looking at it being like so what are we going to run and are we going to run all of those things is sort of the big question that we've been thinking about and they do seem to fall under the same like into a category or categories that are close to each other um and yet I don't always know in which direction they're going like I don't know if the pro Network takes on the sequencer or the sequencer generates the pro Network or in the case of what you guys are doing like is it connected to the maybe not sequencers in this case

but some sort of proving this actually leads me to something else I wanted to ask you which is like is there ZK in anything that you're doing like are you using zkps yes so there there's I guess a couple of pieces where where ZK could fit in um I'm going to particularly talk about integrity and I'm sure Anish has stuff to add here as well so one of the pieces that's important for certain blockchain users is being able to have a proof of Integrity of the compute that's actually being done whether that's inference or fine-tuning or or whatever else and so our approach here is not to have kind of a one directional sort of Monolithic approach to security we don't enforce

one particular security Paradigm on all of the users we instead do more of a piecewise fit the proof to the user and use case type of approach so there's many different ways that you can achieve Integrity with blockchain systems there's the you know put it on ethereum Route there's ZK proofs there's optimistic proofs doing the compute inside of a te and so our approach is to really kind of offer the the wide variety of all of these different types of proofs for various different use cases so for the defi use case that um Anish and tun were talking about that probably is a good

fit for a ZK proof because the you know in many cases the compute is not going to be massive in size and you want some sort of finality or or you know guarantees around the compute happening in a certain time frame where whereas let's say that I'm doing I'm building like a recommendation system on farcaster you know I don't really care if it's ZK proved I think that's probably fine to have an optimistic proof because if I don't like the results I'm just going to refresh it and and try it again and and keep trying it until I get the results that I like so we kind of you know we call it a proof Marketplace but we have a pretty wide variety of groups that we'll be integrating with to offer these these proofs of Integrity so ZK isin one piece

of that and one of the applications that we released I want to say it was in November December was this application called friend rug you rug a friend rug a friend exactly oh no um yeah um what it allows you to do is um you know normally in friend Tech you go and you you know enter a room and it's that person that you subscribe to on the other end responding to your messages that might send instead we stuck an llm or a series of llms behind that system and what you try to do is you try to convince the friend techbot to buy and sell your keys

and so we we threw in a bunch of examples to to kind of make it difficult to actually go and have it buy your keys and we seed it with a prize pull and the idea of this this hack was to show a pretty wide range of things that you could do on top of on top of ritual it shows you using an llm it shows the llms being fed in you the responses being fed into an onchain classifier and then we used uh ZK proof to actually prove the the Integrity of that thing and it's also all fully open source so we want people to kind of follow this model and build these these uh kind of onchain or partially onchain systems that uh that leverage you know rituals infrastructure

in some way so it's definitely it's definitely a you know big component of the of the space and and you know piece of what we're building for sure cool you know one of the interesting things about these examples is they have kind of different levels of complexity of the models used different types of data sources I mean just to get to give maybe the listeners a bit more of a direct feel what kind of model complexity do you think people can expect right now what kind of models do you think people will be able to use in a year and you know how does that how does that impact the network all of these different participants you're talking about I'd say it largely depends on the trust assumptions that users and Protocols are willing to have and and the reason I'll

say that is we are first and foremost a product focused company and so we sort of Plug and Play into the longtail of Open Source you know optimistic ml zkl close Source private models te systems like Naraj mentioned and each each of those have their own sort of range of models that they support and I'll give you an example of that right it's if if you want to run with this friendor example nage gave the entry point for that was sort of three llms running a mistal 7B model and today you can't actually snark the mistol 7B model in a circuit and generate a validity proof of execution because there's just far too

many parameters uh and you know you would be blowing up the compute requirements proving times things like that and so we eventually went with an optimistic approach for the first part of friend rug right for this llm system of let's just wait a period of time and let's get three separate you know Council of llms to come together so it's not just one llm that could mess up it's now you have three so you're increasing the sampling size and so for some applications you know practically speaking if you want to use these massive llm models these you know 65b parameter models you're probably going to default to the opml the optimistic world of things and you'll be able to use that on the other side of the thing the second part of FR rug was this classifier model where we took in the three inputs from

the llms and we said okay classify and summarize these into one action that should be taken and that we trained on sort of like you know 100 200,000 parameters we generated a Halo 2 circuit you know we had a succinct proof that you could execute and use entirely onchain and so the answer to that is really you know broad it's given where research is today there are a lot of simple classifier models regression models you know we've gotten pretty deep on the tree depth of how big you can go that you can use succinctly onchain today on the other side of things there might be use cases that people are experimenting with that require much much much more compute which may not be possible today but will be possible a year two years 5 years from today and the idea behind ritual is how can we

stay as agnostic and as open as possible such that when that research catches up and when we get to that point you can slot in different proof systems and different ways to execute those models on chain um so practically you're not restricted from running any type of model so long as it fits within some bounds of available proof systems or if you bring your own um that's kind of the way we're thinking about it today how much has been built like you kind of talk about having those options but are those all mapped out and built already or is that something more like road map yeah so we have two core products the first one is a system called infranet that's what fren rug is built on today that is a Oracle networ work it is proof system agnostic you can bring in your

own proving system you can bring in your own Halo 2 verifier plunky 3 verifier whatever you choose and you can start to use these applications that you know Bridge AI to blockchain today that's been deployed for a couple of months and we've seen you know an explosive growth in people running these nodes and interest in developing these applications on top and you can deploy that to any evm compatible chain today so whether that's Bas whether that's main net whether that's a test net you can use this today and get started the second product that we're working on is the chain itself which sort of enshrines some of these AI operations natively into the virtual machine um and that's a work in progress that we've been building for you know upwards of a few months to since we started uh the

inklings of it and and you know that's what we're working on today to get that out into people's hands that first part that you talked about the Oracle are you saying that like today someone can already make those choices that you had outlined before exactly okay yeah you can go to docs. ritual. net and you can choose to you know bring and integrate this into your protocol today and you can choose you know do I want an optimistic proof here do I want a ZK proof here what do I want to actually do in my protocol and make those decisions um and that's available right now the Builder would be building in their own it's not that you have those pre-made for them is it so we have some

examples like the Halo 2 one um like you using Ezekiel toate you know some proof systems using Halo 2 we have some optimistic examples of you know here's one where you wait 1 hour and if you know nobody goes and says hey this is wrong then you continue execution we have some that don't use a proof system at all um a lot of early developers are using these things for things like generating images for nfts right and there the hallucination the the longtail problems are actually part of the excitement right it's like oh look my you know diffusion model messed up but it looks so cool that I don't really care if it's wrong um so we we have some examples but yeah it's it's up to the developers at the end of the day of you

can we've kept a generic enough where you can slot in things as you'd like cool so when you talk about the chain when you talked about these sort of different classes of nodes is that what your chain is going to look like is that the part that you're still working on yeah exactly and I'd say the question folks could ask is you know why do you need a chain for something like this or why is this what you're building and there's sort of two answers to that the first one is there's a lot of applications that will not want to move right if compound is deployed on Main net we're not going to expect compound to wake up tomorrow and say well now we're deploying to Ritual chain to use this infrastructure for those systems we have infranet right we have a product that plugs and plays into any evm chain

it has its downsides right you have to deal with an oracle system it's asynchronous there's limitations to what you can and can't do but it's it's available today and any protocol can use it but the way we're sort of thinking about it is that you know the defi protocols the crypto applications 5 years from today will probably look very different and much more efficient and Advan than what exists today and for those you really need to you know pop open the Escape hatches and actually go and build things at a very core execution layer consensus layer of how can we optimize these things and build it out and that's sort of where the chain comes in where it's like for the future applications for the people that do want to exploit sort of that you know 100% of what can AI bring

to our protocols that's where the chain slots and and for that of course you need you know the different types of validators the the proof systems the all types of intricacies of making this thing actually work um and be you know at least simple enough for a user to just hop on and use all right well good luck with all of that and thanks so much for coming on the show and sharing with us kind of a a revisit to the crypto AI space how you're thinking about it what you're actually building the areas you're focused on yeah thanks so much for coming on yeah thanks for having us it was exciting to chat about this I think uh the the thing about crypto and AI stuff is um you know 98% is a scam 1% is an honest effort and 1% is real but hard

to understand so it's always good for the last part to get illuminated yeah thanks again and I want to say thank you to the podcast team Henrik Rachel and Tanya and to her listeners thanks for [Music] listening