ZK Paris: Jason Morton - Zero Knowledge Machine Learning

Transcript

okay so next up we have zkml with Ezekiel by Jason Morton at Zed conduit five minutes do I uh great thank you um so I'll start with uh some motivation of course you all know this but I like to start with saying that digital signatures are kind of what US what got us started um and they got us excited about blockchain but they don't really take us all the way that we'd like to go with what blockchain can do and what I mean by that is that [Music] um we're really forced to program with

digital signatures not in signatures and that leads to fundamentally complex constructions lots of different uh you know authorizations and deauthorizations and trees of inference that can be replaced entirely hopefully by zero knowledge proofs it leads to chatty protocols whenever we have a consensus algorithm or anything else that uh where each each participant in a network needs to talk to the other we run into speed of light problems and we have a fundamental limits on how fast we can go ZK is slow but it doesn't have those fundamental limits So eventually it will be faster um and we have limited uh security models and privacy options that always come down to somebody storing a secret key which may never really get us to the

mainstream um and as you know zero knowledge proofs are programmable signatures they let us take a secret input a public message run a public signature algorithm that's the usual way we get a signature that can be publicly verified in the message and in a zero knowledge proof we can replace the private key with arbitrary private inputs the public key with arbitrary public inputs and we get to run any program we like the program I like is is machine learning programs and then we get a proof out that anyone can check and we get you know similar security properties people talk about initially about zkp's like probabilistically checkable programs which sounds bad but um you know when you remember that the

probability of failure is like guessing satoshi's Keys it doesn't sound as bad anymore uh anyway so as you know zero knowledge proofs are becoming more programmable in the last few years they've gone from you know promising things to practical things that we're doing increasingly crazy stuff with um the program we work on Ezekiel is about 10 times 10 000 times faster than it was when we first demoed uh an mnist inference proof that was verifiable on evm in September um and there's lots of tools being developed to make them easier to program with pytorch and python if you like Ezekiel or you can write in JavaScript or in special purpose languages and all of these things are are part of becoming more programmable and I'd argue also

that we're kind of at a moment where we're going from um banking Cobalt Mainframe applications like Roll-Ups and privacy protocols to kind of the hobbyist uh PC Cambrian explosion level of of programs so we're doing crazy things putting giant parameter models we're doing games if we just heard about we're doing email we're doing all this stuff it's all very experimental and you know shouldn't probably be trusted with lots of money although I'm sure it will be um but you know it's it's an explosion it's exciting um and I'd like to sort of think about what happens at zero knowledge proofs become you know not just pretty good but as as fast as we could possibly desire right and one way to measure that speed

is as execution speed another is kind of floating Point operations per proof um you know we started out a year ago around here and over the past year we've gone out to here where we can shove 100 million or so floating Point operations into a proof and that starts to enable you know qualitatively not just quantitatively different applications um machine learning is one of them but there are others you can imagine and of course nowadays we just take everything and we shove it into a llm so you know maybe we can build a Minecraft computer in our Minecraft computer um another way to think about it another axis is this idea of kind of proofs per chain transaction this is a little bit more nascent of an idea but one way to

think about it is that something like a roll-up is shoving uh you know maybe a hundred things that would normally be a transaction into one final on-chain transaction right and each signature that goes into the roll-up is a proof and then the final you know the final signature or the final Roll Up is a proof um and there was this other experiment at Zuzu called zoo pass um also Xerox Park experiment that did you might have one chain transaction to buy a ticket and then you have this pass which can generate membership proofs and other zero knowledge proofs and then that pass you can use to get into a club or enter an event or whatever right so you might use generate 100 proofs or a thousand proofs that all use that single

blockchain transaction as a source of Truth and they might get checkpointed back to the blockchain as well right so this is more of a social problem as well as a devtooling problem you know how do we get to a world where there's a billion proofs being created per chain transaction and my thesis is that that's actually where we're going to see some of the promise of blockchain some of these crazy ideas that have been developed over the last five years and didn't really work for scalability reasons actually start to work um by practical I mean we have fast enough uh uh computation and memory usage for AI fast enough for on-device proving as YJ was just talking about um we have a good epistemology so we know what we're proving we know what's

what we're claiming to be true because we tie back to the evm or some other consensus layer and we get to a billion proofs per transaction a billion flops proof um and we of course need none of this matters unless we have good devex and um we don't need phds to to build this now I'm a PhD I love phds um but we can't expect everyone to go through that pain so two axis version of this is you know we have flops for proof we have proofs per transaction I think up here there's going to be really disruptive changing the way our society organizes ourselves changing what we mean by money crazy things that will happen I'm not

going to think of those things I'm just going to help make it run fast enough that someone else can explode things and so I invite you to do that um of course our focus is on machine learning I'm giving the blockchain eyes taking ethereum and giving it the ability to sort of see the physical world make judgments all that kind of thing what we're doing is mostly inference for now um and we take the basic idea of a signature algorithm of course is there a knowledge proof for places that with an arbitrary uh an arbitrary program and the program we're putting in there is AI ml inference um which is increasingly getting more gadgets attached that makes it not fit that well into that box but

you can also just think about it as like python programs um okay so here's the check box chart uh we have three types of technologies that are getting combined here AIML has a great python workflow has a great library of models hugging face and all that it's widely integrated it's very flexible it's artificially intelligent but it's centralized you have to trust open AI or whoever it is that's running your model we have Smart contracts which are decentralized they're trustless they have this incredible composability property but we pay for that with this consensus that's fundamentally limited by costs and by speeds of light and it's too slow that means that it's going to always be too slow and too

expensive for AIML um and then we have zero knowledge proofs which take the the blockchain decentralization trustlessness and composability and Tack onto its scalability now one person can prove something happens and everyone else in the world can believe it if they know where to look for the verifier but we pay for by having to do a PhD and by doing uh you know strange programming Paradigm talking about circuits weird languages um and then of course making your mind all the chat boxes right so uh what we're trying to do is make all these things work together give you the python workflow of the existing models and put that into a zero knowledge proof

um one kind of people often ask me about the big picture and I want to mention that there are many different approaches to zkml um in the beginning and I said we started out you could or sorry we started here you could hand build a proof system and a circuit for a model such as in this paper you can build a custom circuit for each model in a general proof system which is how we started then you can try to bake that intelligence into a compiler this is the what we believe to be The Sweet Spot Of course other people will make different choices you can compile the compiled assembly into a circuit just like what Mill Foundation does you can use an ml specific or general VM I like to joke about this but in some sense maybe this will eventually

be possible is that you could just run python my pytorch on Ubuntu on risk zero like in theory that should work right just be a little slow for now but if risk zero keeps going maybe it won't be um and then you can if you want to change the trust assumptions you could do something like a trusted execution on Place lots of different approaches I think they all have different trade-offs and they'll all be applicable in different situations of course we've made our bet as to where we think um is most interesting for us but um I think these all benefit the space um so why do we do zero knowledge machine learning we want to give the blockchain eyes let it perceive the physical world let it uh you know do all these magic supply chain stuff that never worked out um we want to do oracles that bring

off-chain data and on-chain data on Chain by running a machine learning model did this hurricane hit did this package get delivered we want to do identity letting a human rather than a field element control digital assets build a wallet that's impossible to lose you know let people build their own world coin in 30 minutes you know that's kind of our goal um we want to more broadly let smart contracts be able to exercise judgment not be fooled so easily by transactions that comply with the letter of the smart contract law but not the spirit and there's other interesting things that we can use it for like making sure that models in some sense sign themselves so that we if we're afraid of AI taking over the world we can cut them

off at the actuator level instead of restricting development at the Upstream um you know I think that however that we're still at a place where ZK is kind of too flexible um so for example you know in Ezekiel we can do private public hashed or encryption encrypted inputs and outputs and weights um which lets you build lots of different crypto systems a lot of those crypto systems are probably not going to have a security property that you think they're going to have when you first think about it so I think as a space it's in common on us to kind of shut cut down this excuse me the size of those options and we don't want to go all the way down to digital signatures where you can only do one thing but we want to choose intermediate Legos that people can build with in a secure way so

suggested applications we don't just want to hand them a turing machine and say go for it do whatever you want because you're likely to build something that doesn't make sense okay so lastly I want to talk about um just I would do a demo but we have slides so what Ezekiel is so what do we do you you start out with just regular python your this is a pytorch rather so you you have a module you define some layers you define a function a forward function this could be whatever function you like maybe some convolutions really lose stuff like that um and then you export that this is just sort of some boilerplate which can be

baked into a function if you prefer whatever your particular uh system is you export it to an onyx file uh and then that Onyx file is kind of a baked abstract syntax tree Frozen abstract syntax tree of the model or of the function so that's the circuit I mean one of the reasons that we've gone in this direction is that I think that if you want a lot of people to use ZK you take something like Pi torch neural networks this is the most widely deployed widely used circuit language in the world I would argue and that's why we wanted to Target it so you bake the model into a into a pipe into a pie torch into a sorry Onyx file then you generate some settings we have

some automatic calibration that helps you choose these weird parameters like how many rows and and stuff like that and then we run a setup function so really there are three players in a ZK game right there's the developer that runs the setup that decides who what what it means to prove and verify there's the approver and there's the verifier okay so you run a setup after after calibration this is all python this is copied from one of our uh one of our notebook python uh juvenile notebooks that's on the website on the repo um and then you can prove against it you can also do this in the browser if your model is not too big uh and then we verify and I'm breaking verification up into three steps because

I think I really care about verifying on chain or at least making it possible to verify on chain so I think that's a good way to decide what's true in a way that nobody can mess with so you create an evm verifier you deploy that verifier and then you can verify against it through Ezekiel so yeah that's it and that's the link to the GitHub if you're interested and uh any questions thank you great presentation um could you talk just a little bit more about giving the blockchain eyes with respect to scalable oracles yeah so um

so take something like uh like a prediction magnosis or you know in its first uh first Incarnation so something where you you want it to measure a thing that happened in the real world someone got elected a president or you know hurricane hit a coast a new covid variant hit or something right before we had this kind of ZK stuff all we could really do is set up a complex crypto economic system where some people propose the you know propose the thing that we're deciding others vote on the thing and stake against it and bet on it and then other people make the Judgment as to what and then we have to punish and you know these kids really complicated right so alternatively if we trust The Source by which I mean we need the source to somehow sign their

HTTP response or someone else some third-party notary to sign it right so but let's assume that we're in the future everybody is ready for ZK they realize it's the future when I when I hit Bloomberg or you know Associated Press every time I get an article back it comes with a second chart it's an Associated Press said a big hurricane hit the coast we take that document we run the model on the document the signature of course is the signature of the hash so then in the smart contract we check the hash signature we don't have to we can check the signature in the proof but that's expensive we can also just check it on the smart contract so we check the signature is valid this really was the Associated Press and then we checked that the the zkml model ran and said this article said

you know Dwayne Camacho was elected president or whatever right and uh then you know the contract knows that that thing happened and then you can add more features if you want more articles or more sources similar thing if you want to say I'm going to do a ZK version of the Amazon taking a picture of your package after they deliver it you run a model instead of disclosing that right so that's maybe a little bit more of a stretch but yeah so that's so it or maybe it's a supply chain thing where it's actually a high value package and you want to yeah thanks uh thank you for the nice talk

um you mentioned that over one year you've had a huge Improvement in the number of floating Point operations you could do would you comment on what what allowed that yeah yeah so so we're very lucky in ZK because it's a lot of brilliant work has gone into it but the stuff that got to everything everything actually works there's often we all just did stuff in okay we want it to be correct and we want it to work right which means we didn't pay a lot of attention to data structures and arguments and everything else right so a lot of that Improvement just comes down to either you try different arguments so different ways of arguing say that a matrix times a vector equals a result there's you have choices right or you go

actually into the proving system itself and you say oh in this place you know we're constructing a big vector and really we can just iterate you know or so we can play with a data structure so um another saying it is that uh you know it's a little bit more I take the blame it's like okay everything in the stupidest possible way the first time and we're making it slightly less stupid you know every week uh and but there's there's a ton of low tank low hanging fruit there um I think that there's still probably maybe 100x in just being more careful and writing tighter data structures and Tighter algorithms um and then there's another 10x maybe or more in gpus and then there's in

improving the proving system actually changes the asymptotics right so so there's lots of you know both the research that's coming down the line lookups and everything else folding all this stuff um gives you kind of algorithm fundamental algorithmic asymptotic improvements as they come out as they reach production and then there's just a ton of lead bullets engineering rolling up your sleeves and improving things slowly thanks thank you awesome thank you Jason [Applause]