Episode 265: Where ZK and ML intersect with Yi Sun and Daniel Kang

Transcript

foreign [Music] welcome to zero knowledge I'm your host Anna Rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online [Music] tarun and I chat with Yi Sun co-founder of Axiom and Daniel Kong associate professor of computer science at UIUC we talk about their academic background and what led them to get interested in ZK topics and zkml we then dig into two recent works of theirs entitled

trustless verification of machine learning and zkimg fighting deep fakes with zero knowledge proofs this episode is all about zkml which is a lot of fun to dive into it's a new field for me and I really like looking at how it's starting to intersect with the zero knowledge Tech that we talk about on the show so yeah let me know if you like it and if you want to hear more apps like this now before we kick off I just want to let you know about an upcoming event we're doing this is the ZK hack Lisbon event it's an in-person hackathon happening in Lisbon the weekend before ZK Summit 9.

that's March 31st to April 2nd if you're already coming to the summit this might be interesting for you this is an event for developers and Builders who've already built something in ZK or really looking to jump in if this sounds like you do send off an application I'll add that to the show notes and we hope to see you there now Tanya will share a little bit about this week's sponsor [Music] Elio is a new layer 1 blockchain that achieves the programmability of ethereum the privacy of zcash and the scalability of a roll-up if you're interested in building private applications then check out Elio's programming language called Leo Leo enables non-cryptographers to harness the power of zkps to deploy decentralized exchanges hidden

information games regulated stable coins and more visit developer.alio.org to learn more you can also participate in Elio's incentivized testnet 3 by downloading and running a snark OS node no sign up is necessary to participate for questions join their Discord at elio.org forward slash Discord so thanks again Elio and now here's our episode [Music] today tarun and I are here with Yi Sun co-founder of Axiom and Daniel Kung assistant professor of computer science at UIUC welcome to the show thanks for having us thank you for having us so yeah I think as a starting point it would be really great to get to know both of you Yi why don't we start with

you tell me a little bit about your research and the work that has led you to ZK and now your work on Axiom yeah so I recently took a lead from a statistics professorship at the University of Chicago where I was doing work on probability and machine learning to start Axiom which is a new ZK project his goal is to bring the power of ZK to Smart contract Developers so I got interested in crypto actually quite a few years back around 2017. so I just finished a PhD in math from MIT where I spent a year doing high frequency trading in the middle and so cryptos need to seem like the perfect combination of markets and cryptography

uh back then I quickly got pretty interested in the consensus protocols and what can enable crypto to really become something scalable and ZK always seemed like the holy grail for that but at that time it did seem pretty much like Moon math so the procedure for starting anything in zero knowledge would be number one you roll your own crypto and invent your own xeronaut protocol yeah then maybe that takes you a year number two you actually Implement that in software maybe that takes another year and tiers in you get to really start working on your actual application or infrastructure tool and at some point you also have to build your own ZK DSL probably along the way yeah step three complete the trifecta

perfect so I I kept following the space uh helping out a couple early stage projects uh like Gauntlet and scroll and in late 2021 I realized that the ZK space had really matured and in particular the tooling was ready to start building applications straight away so I got pretty interested and sucked back in and I spent early 2022 working on a bunch of Open Source work to write various Primitives in ZK so I worked on some libraries for elliptic curve cryptography and also reading from ethereum data structures in ZK so over the summer I realized that that

work could be actually really scaled up by porting it to a more performant proof system called Halo 2. and that's when I realized that something like Axiom was even possible to build and decided to work more seriously on it so we just launched last week and we're excited to see what we can do nice so are you saying though that like your first true foray into the blockchain space was actually ZK like you had been kind of observing it before that but yeah was that your first step in to actually build yeah definitely I've been sort of poking around the states for quite a while as Turin can probably tell you uh but ZK was the first thing where I felt like my technical skills could be a good fit for

what's needed in space cool and I think it's okay is kind of the perfect mashup of you know you need to know some math but you also have to make a performant production system what kind of Professor were you or what department were you in before was it computer science or was it math yeah it's actually in the statistics Department oh yeah and people always ask this but my previous research has very little to do with cryptocurrency or even CK so I was working on these theoretical problems in probability that originate in statistical mechanics and also trying to make some connections between those problems and modern machine learning cool so working on some theoretical deep learning nice all right I know we're going to come back to the topic of machine learning and more on the ZK

front but Daniel why don't you tell us a little bit about what your research has been all about and what got you excited about CK stuff yeah that's a great question um actually this my foray into crypto actually started way back in I think almost 2011 or 2012. it was almost a failed story um back then I knew this Eastern European uh hacker who whose name I don't actually know and he told me that I should work on this thing called Bitcoin mining and it changed my life and it sounded like a humongous scam at this point in time I was in high school I was really into assembly coding he told me I should optimize assembly minors obviously I did not do this since uh I clearly make bad decisions in my life I learned later that the group that

he referred me to was mining so uh profitably on AWS that they crashed AWS I think in around 2013. but since then I've decided to actually keep an open mind and if people tell me that they're interesting things I should uh at least pay attention and interestingly enough Yi has been telling me for years that there's this uh thing called ZK um and when I first heard about it it literally sounds like magic it literally sounds like something that shouldn't exist uh so I was a bit skeptical but I kept an open mind um but at that point in time this is I think maybe 20 I don't remember the exact year 2018 to 2020 or so I was doing my PhD at Stanford where I focused on deploying machine learning in a

variety of different applications but primarily for analytics and one of the trends I noticed um when I started my PhD it was around 2017 when I first got involved with machine learning and statistics it was around 2013 2014. it really went from people being able to deploy their own models to literally all the models that you are that you hear about nowadays are basically gated behind apis so for example Open the Eyes chat gbt um Google has announced some as well and a variety of other models like this and one of the things I've been wondering about for a long time is that as an academic and especially as these models are proliferating what are some ways to actually influence what's going

on and one really basic question is that if a model is behind API how do you actually know what's running yeah um and to me like this problem literally sounded impossible so I just I thought about this and I was like all right well I guess this isn't happening but yeah I would really like to know what uh what I'm being served and then after several months of cajoling you uh convinced me to start working on this space and I noticed the same thing that the tooling was actually it was actually feasible to start to deploy applications today and I got really excited about that um and I dived right into it and I've been working this space I guess for about half a year to year at this point and it's been a lot of fun so uh that's how I got involved with uh with zkml

cool you've produced papers together right you've written you've done research together how does that collaboration work is it sort of like someone has an idea and then you just want to dive into it or is this like officially within your working preview yeah how how does that work yeah we've actually been working together for a pretty long time I first met Daniel actually through one of our mutual friends who is my roommate in grad school his name is tatsunori Hashimoto and he's now a professor of computer science at Stanford and so it's actually through that context we just became friends and started working on machine learning together and it's a long time ago I think maybe 2017. so we started working on adversarial examples

in deep learning it's really one of the first exposures that honestly both of us had had to machine learning and especially empirical machine learning so we wrote a paper together about trying to defend against different forms of these exotic adversarial attacks on Vision models so what that means is you can tweak the input to one of your machine learning classifiers and it's a non-trivial result that it's possible to fool almost all deep learning models today one of the biggest puzzles in the space is whether you can design a model to prevent this from happening unfortunately our our work did not make that much progress on that we

essentially showed that this is a very hard problem and in several ways of doing it don't really work okay yeah and and uh for for listeners who've listened to a lot episode 246 was about this topic Florian who does a lot of like in the wild attacks against systems people claim are adversarly resistant but then turn out not to be yeah we definitely read a lot of Florence papers yeah so I want to talk about one paper that the two of you were authors on and I know I've seen the blog post which is entitled trustless verification of machine learning I want to explore a little bit how that work came to be I think it ties into what you just said Daniel about kind of the problem space

and yeah what what were you trying to solve or figure out would that work maybe you can tell us a little bit about it yeah I can maybe jump in with a bit of the backstory so at the beginning of that work I guess I've been working on ZK for a while and thinking about how to really scale up ZK proofs for things outside of machine learning very basic data structures appearing in ethereum and elliptic curve operations and when I thought about what really went into that work it's a lot of low-level optimization as well as understanding from a ground level of the computation you're trying to do but there was some people in the space thinking about applying this to machine learning and I immediately realized that Daniel would almost be the perfect person to work on this so that's why I

thought of him to even try to really control him to get into the space and the the goal of that worked was to really scale up uh ZK machine learning models for the first time and I thought Daniel's the perfect person to make that happen that's very flattering uh and as he was saying one of the things that actually turns out to be really important in machine learning in particular for ZK is how you represent computations and going back to my story about assembly hacking in high school it turns out a lot of the skills almost directly poured over to to zkml hmm what's the actual problem though that this like introducing let's even take a step back into like zkml like

what problem is it trying to solve yeah so zkml tries to solve an abstract problem of which there are many specific concrete instantiations but the abstract problem is to produce a zero knowledge proof that a machine learning model ran on some input and this inherits all the nice properties of zero knowledge proofs and in particular we focus on ZK snarks so things like succinctness zero knowledge obviously and the completeness properties and soundness properties of surinage proofs going back to what you were saying about a lot of these models being created or like a lot of the activities happening behind an API is this an attempt to sort of prove even if it's behind an API that it's being done correctly yeah that's exactly

right that's one of the concrete instantiations of the zkml problem so for example an ml provider might have some model weights that they want to keep hidden and so when you send them uh let's say an image or a piece of text you can then use the techniques from zkml or the API provider can use the techniques from ckml to then prove that they've ran the model that they said they ran uh honestly why would they not like what benefit is there in not doing that right like is there something malicious or something to be gained yeah so I think it depends on who you talk to if you talk to people okay so I guess maybe the the most uh realistic application of this I think is that the cloud provider or API provider

might have been hacked okay so for example if you're serving medical predictions and there might be some malicious actors who want to say serve incorrect medical predictions for a very specific reasons say State actors and in this circumstance um you can if you send the proof then you know that uh the the models run correctly but beyond that there might be bugs in the serving system so you might trust the model provider to actually do what they're trying or try to do what they're saying they want to do um but they might have a say like a version mismatch and they might send you predictions from the wrong version of the model and finally if you're dealing with a trustless setting uh the model provider might just be lazy it's quite expensive to run ml models and if you

can just avoid running them say by running a smaller model they might do that like just give you lazy data like still output something but not on on sort of the the sophisticated level that they had promised or something yeah so concretely if you think about openai they have a model called gpt3 and a variety of different forms of it they have smaller versions of that model for example gpt2 and so they might run gpt2 on your input if they are for whatever reason feeling lazy um and you know you might trust opening in particular but if you say you want to Outsource your computation to say a startup that might not be as proven then you still might want to have these kind of guarantees

that you get from zero knowledge proofs I think it's particularly challenging in areas where as a human you actually can't evaluate the model output for example if the model is just evaluating the classifying images saying whether something is a cat or a dog that's fine for humans to post-process and check yeah but if you're relying on machine learning to do something a bit superhuman maybe make some prediction well the whole point is that you as a human shouldn't really be overriding the model and so there you you really want the gold standard model not a cheap approximation it's funny tarun I'm just thinking back to when we had fluorian on and we talked about I was confused I think I I listened to this recently but

I was confused about what the model means ml in general for me is a really new space so like I'm still getting familiar with the language at all but yeah in this case so this is this is actually making me understand a little bit better what's happening when you're querying these models like the model is doing it's running something new it's not just like the whole thing is finished but behind an API and it's just outputting what you ask for right is it creating a model as you ask that question somehow it kind of depends on what sort of systems you're using some models are really just you can think of it as a function with some parameters and so when we say that openai trained a model it just means they ran some you

know very fancy computationally intense procedure to determine those parameters and when you run the model on your input you give it let's say an image and it simply evaluates that function so from the point of view of zero knowledge you're just proving the correct execution with maybe some hidden information of that function okay yeah it might be actually worth trying to talk a little bit about some of the problems in the the Deep fake paper if we kind of zoom out a little bit so you know there have been a bunch of papers on identifying deep fakes or proving that you correctly generated it's an authentic image versus like one generated but you know one of the problems I guess in like some of the earlier papers uh like if I remember correctly like the

Bonet paper and others was that they required the image to have public data to generate the proof and so a lot of what you did was getting around that so maybe let's maybe we could walk through like how you dealt with this idea of like hey we want to generate the proof without revealing much of the initial input data and you know the idea of like image Transformations being able to be uh performance DK because I think I think that part is like the most interesting part from the technical side is that people kind of assumed you would always have to be providing the input data and yeah so one problem for both um a tested image edits and also for machine learning is that you might want to hide some parts of the input

so for example let's say I take a photo of my uh of my desk or let's say I take a photo of The Situation Room for when Obama was overseeing the operations or the uh Assam Bin Laden then you might want to hide or redact some of that information and you also might want to edit the photo for clarity but because some of this information is private you don't want to reveal the original photo but you still want to be able to say that these edits happen honestly from this original photo one way to go about doing this which is what we introduced in our paper is to compute a commitment in our particular case it's a hash of the original image if you assume that you have an A tested sensor you can verify that hash the signed hash from

the a tested sensor and then only your VL the hash and the edited image at the end and so this way you can actually preserve the privacy of the original image and this General technique can also be used for machine learning as well so all the work in the zkml space previously doesn't actually commit to the weights so they can prove that they ran some computation but because the weights uh these things we call weights are the parameters of the function are actually critical to what the output is you don't actually get any guarantees that they ran the mall that they said they ran um whereas in our work we also introduced this for the zkml space as well where you can compute the commitment in our in our case a hash of

the weights and reveal that and because the the current is binding it forces the API provider to Hash the weights then you can be assured that the around the correct model you just ventured something like an attested sensor what is that actually yeah so a tested sensor the most common form of this is a tested camera is a hardware device that signs the sensor data for example pixels of an image immediately upon capture okay and it signs it with a tamper-proof hardware device so in particular the private key is kept hidden on the tamper-proof hardware device and you destroy the heart the private key immediately after the camera is produced after the picture is

produced you mean so you destroyed a private key immediately after the camera is made oh wow so only so the private key is only ever on this Hardware device sorry you mean you destroy it on on like computer external the hardware device still has it to sign with right I still need yes yeah okay yes we still got to be able to like you know multiply by the group operation right yeah yeah so the way this works is that a hardware vendor like say a camera vendor will produce the private key and put it on the hardware device and then delete the private key from their computer now it's not as private key only exists on this signing device and then furthermore the signing device is

protected by essentially if you try to change the voltage on it it'll basically destroy the private key and so that's how you ensure security interesting so in a tested sensor on a camera a camera it can be something else I guess anything that's taking like real world input and turn it into something digital and I guess that's always yeah because you have to be able to like you couldn't you know use this on any sort of analog photo it has to somehow be in digital form to start running these mathematical things on it that's interesting though I mean you use the example of a photo a camera but are there other sensors that you've considered this for yeah so we chose camera specifically

because you can purchase in a tested camera today okay um I think with the rise of generative AI methods so for example uh AI methods like stable diffusion and the recently announced um audio on video models from Google um I think we should it would be great if we moved to a world where basically every sensor was tested so your mic your your camera um your your webcam um and even your keyboard uh that's the world that I want to move towards but um currently I believe that uh tested cameras are the most common form of a tested sensor so like most devices would not have this feature like most current digital recording devices all of them

aren't like is there no way to somehow time stamp it or somehow prove that it came from that device at that time and that it's like accurate yeah that's a great question and it depends quite a bit on basically the level of security that you want so a tested sensors um are in some sense the most secure version of this because you need to have the actual a hardware device to produce a signature there are other forms of technology for example enclaves which can help with this problem but there are some issues with getting basically the sensor data Into The Enclave securely and depending on how much you trust the person who's taking the photo and also the basically the software systems of

say your phone then you can still do some form of this using enclaves and signing um it's work that we're talking about just to for The Listener it's the zkimg right this is the fighting deep fakes this is another work that the two of you collaborated on were there other people also involved in that work or was it just the two of you uh so actually the the friend that you mentioned tatsunary Hashimoto was involved as well okay I'm a postdoc advisor uh Jan stoika at Berkeley cool the reason all these attested sensors and cameras are especially relevant with the rise of generative AI is I think weather and image or a voice is authentic is now turning from a somewhat objective

problem that you could use for example before maybe you could classify with a machine learning model whether an image is real like from an authentic camera or generated by stable diffusion but as these generative models get better and better the way they actually work is by being trained to fool a classifier oh wow and so it's almost turning from an objective problem to to a somewhat subjective problem that we need to use cryptography to encode you know somewhat human level judgments about and that's why you know to get a source of authentic input data for zkml it's kind of important to have uh these attested inputs and maybe just to bring us back to this adversarial examples discussion

from a while back in the discussion the challenge is basically if you're using a hidden input in any machine learning model where you you generate a proof well that hidden input might be chosen adversarially and then you can make the output of your model essentially be whatever you want but because the input is hidden the downstream user can't tell that you actually ran an adversarial attack and so that's a bit of a fundamental constraint for any application of zkml in this particular example though you are kind of creating a way to keep the initial Source private why does it need to be private like why would the like with the attested sensor

I mean you can always use the ZK P to prove provenance or something like that or non-provenance but yeah why does it matter if the underlying one is private I think it depends quite a bit on the application so for the applications that we've been talking about so far it doesn't really matter that much but I think with the rise of generative AI one of the things that's going to become really important is basically authenticated bio biometrics so scary so one of the ways that you can authenticate yourself today is you can basically send a photo of your face to some service and then they'll classify you as real or not and you might have actually used this on several applications

um so I think that some social media applications ask this for verification but if you do this you're basically trusting the service to handle your information correctly so maybe I don't want to send a photo of my face to Twitter if I want to stay anonymous and having the pipeline from a tested sensor to the ml model could potentially allow privacy preserving biometric identification um so you could almost use your like Anon pfp as your uh as your kyc face or something that's a future that I want to move towards I want to be able to preserve privacy while keeping all the grid parts

of biometric Authentication and that's one of the reasons that we worked on these two papers which seem quite separate but they're actually quite related hmm yeah maybe you could prove that your Anon anime character was the result of a generative algorithm run on your actual photo ah but no one knows what you really look like yeah you gotta hide your identity yeah I mean it depends on how much it depends on how much uh you use a stable diffusion thing that keeps a lot of the original data but that then it's your it's your choice right you've chosen the particular generative output that you you're willing to use and that's why we have to attest to the input otherwise you could just put you know a celebrity's image as the input that's wild it's funny like in

in planning for this I didn't actually think about the attested sensor and now I'm sort of fascinated with the fact that that doesn't exist normally and that we have to create the thing that makes it like tamper-proof this is a true documentation of this moment in time biosensor yeah I mean you have to understand embedded systems historically like cameras and things that had small devices they they generally because they're battery powered try to avoid any excess Hardware surface area um especially for things like tees which are horribly expensive like power wise so I think like a iPhone if you remove the tee uh you can like 2x your battery life depending on how often you're

signing it's actually like a one of the more it's like it's like comparable to Wi-Fi and Bluetooth oh so a lot of these devices were built you know the people who were building them Hardware wise their entire objective is not how secure is it theirs is like hey we need the battery life to be 20 hours or we need like to be able to take five million pictures at this resolution it's not about privacy you have to actually like somehow get people of very different backgrounds agree on that being like a necessary usage of resources which is you know until you had so much stuff like that looks like deep fakes or generative pictures and and stuff like that people didn't really care to to add that into devices and I guess with just

doing it with a private key though that's not as energy intensive right like or is it still to create this uh a tested sense you have to sign on every subset of pixels yeah it it but the point is like you have to deem that a value you want in your device and then add it in yeah the actual private key creation and all of that like that is not novel cryptography is it this is is this old cryptography that they've just started to add to these devices yeah it's all cryptography okay very very tried and true cool yeah Daniel maybe we can talk a little bit about the merging of these two works you sort of just hinted at it that there that there is a connection Point yeah about that so I think that this entire

pipeline of going from a tested inputs from various sensors to feeding into inputs to machine learning models and algorithms is actually going to be really powerful in the future I think there's a lot of different kinds of applications so we talked about one already which is biometric identification and one of the things that I've been thinking about quite a bit is this problem of adversarial inputs when it comes to biometric identification if you take a picture of your face and you like draw a box for example with something that will fool a machine learning model as an auditor the services can take a look at this picture later and say like okay well they're clearly trying to fool us so we should do something about this but if you never reveal the the image

then they can't do this I think one of the things that will become really interesting is what happens when you combine different kinds of sensors so we mentioned cameras we mentioned microphones one of the other kinds of sensors I want to mention are lidar devices or devs cameras so if you imagine uh your face ID on say an iPhone it uses a depth camera to do the identification and if you can combine that with say an image and also a clip of your voice maybe that'll be much more secure and much more difficult to fool then it would be of just a plain image so the the service providers that are want to authenticate you will probably have much more trust if they can verify from a variety of different sensors I think

this is one application that's going to be really exciting but also there's going to be a lot of technical work for uh for this to be enabled so this is the idea of kind of creating that trust that what's happening behind the API is still correct through zkp but also combining that with the fact that you can keep that initial image in one of these kind of like provenance proving I don't know if you really call it provenance because it's like you're trying to prove that something's a deep fake or like you're trying to prove that it's act the same would you call that provenance actually yeah I guess so yeah Providence okay so it's but keeping that original Source private that's the combination that you're talking about here so one is proving that what's happening under the hood is accurate and

the other is pro like keeping that data private that's right interesting I actually wonder how do those the two things combine because one is like behind the API how do you are you sending like what data would you have to send like say it was actually that like there's some big model blog like there's an API in front of it you are on the other side as a user you have the biometric data how are you getting it in there without revealing it oh yeah that's right so I uh I should have actually uh mentioned we're working in a different setup for for biometric identification okay okay um we're actually in a setup where let's say I want to authenticate myself um I as a user will actually run the machine learning model on my side oh I

see and send the result okay but then here the let's say the social media website that wants to verify that I'm a real human being uh wants to know that I've ran the model honestly oh okay okay but also do it in a privacy preserving way so it's actually flipped in this situation the the social media website doesn't trust um the person who's sending the biometric identification uh from so zkps are kind of interesting because they can enable um both settings where basically a consumer might not provide might not trust a API provider on the flip side of social media website might not trust some random entity that claims they're an actual human being yeah the funny part about this is you you say social media but the types of applications that

have been the largest consumer of these like very weird like liveness kyc tests are actually like dating apps I think they actually like they like force people to do them now like verified whatever and and they're they're like exists startups that like literally just pay people in like scale.ai style like in in the Philippines to like you get on a call and you move your face around enough for them to like believe that it's so this is a this is an alternative version of that oh I guess it's like anti-cat fishing protection thank you I have heard that there's been a lot of scams on dating apps recently so uh hopefully in the future you won't have to move your head around to prove that

you're a real human being you can use either tested sensors and as your knowledge proofs I mean nothing like zero knowledge proofs to set the Romantic mode [Laughter] it's Valentine's Day by your loved one in a tested sensor and a tested pfp of course I do think the fact that sometimes you have to do these proofs on your phone really points the need for efficiency for the proof generation because obviously your phone has all these you know battery requirements and probably the memory and CPU are not running as fast as some sort of Cloud Server and so with this and also a lot of the cloud

applications we found performance to be a really big limiting factor in what's viable versus what's not in that example that you walked through Daniel where you'd have the sort of the model on your own side does that already happen is that something that can already happen or is that like still in a theoretical stage I guess one way to say it is that it can hypothetically happen as in like you could actually run this today you just need an extremely powerful computer that you can hook up your mobile phone too okay so it's doable but unlikely to be done yeah and a big part of my upcoming research is to basically bring out these costs

um there's a bunch of different techniques coming from computer systems to cryptography to ml where I think we can actually bring down the cost by maybe three to five orders of magnitude and it can actually be executable on your mobile phone they're not another kind of ckml combination which is using the properties of like compression uh to make that maybe it's not CK specifically but like is that one of the techniques yeah yeah that is one of the techniques we're actively exploring wow that's cool let's see how would that work then you'd be using the zkp to sort of like make the computational need of the model smaller or more compact and then also using a zkp to prove its

accuracy or something like that yeah if you want to go into the technical details one of the ways that you can go about doing this is to basically uh split the computation of for example the the image edit and the ml model into separate proofs and then combine them using recursion or proof aggregation or whatever you want to call it oh cool this ties into a pretty generic technique in DK where if you want to do something hidden um suppose you want to do a big computation and part of it involves hit information but maybe doing the xenology proof on the user side for the whole computation is computationally and feasible what you can try to do is to generate isolate a part of the

computation that has hidden information do it on the user's laptop browser or maybe their phone and then feed the rest into a big Cloud Server where you don't really care as much about computational load and so getting that interface right is pretty challenging and I think it's a pretty active area both academic and empirical work yeah actually one one sort of question that I guess from your paper I have is like the generic technique is like you commit to a hash and and you can validate that the hash is correct and that the hash was produced by the same thing that like some sequence of transforms were were put on as a like alternative to actually having to be like fully homomorphic or or have like stronger properties

one question I have is like how much does that impact the sets of Transformations you can do so like in your paper you you did Benchmark a bunch of different Transformations at least for images let's say I took a more generic view of this let's say I just said like you can do any linear algebra operation and compose it like is there is there some sort of like limitation performance wise in terms of like how much you can compose with with these and generate these kind of like aggregations versus this kind of like commit hash plus sequence of Transformations does that kind of make sense like there's it does seem like there's some trade-off surface here and I'm just kind of understand kind of what it is yeah it's definitely pretty generic like when you do the competition in CK it's almost

divided into exactly the two pieces you mentioned so you have some hidden information that's committed to and the first step is always to decommit that information for example if it's a hash you prove that you know the original image which was the pre-image of that hash then you apply a bunch of operations in Decay which actually has full knowledge of what the image is and then at the end you can hash that the output to get a commitment to the output so what happens in the middle is actually very decoupled from uh the original image commitment which is used to preserve privacy and one thing that might not be obvious is in ZK the performance hit from hashing is significant so if you're running

something on your laptop you don't really consider hashing to have any performance impact whereas in ZK you know there are even the custom tailored hashes 4zk take a substantial amount of the proving time and concretely for images it can be like 20 times more expensive than the actual transformation itself okay cool yeah and I guess the question I have then maybe is like let's suppose I take a language model which has sort of some notion of like loops and recurrences in in the architectures does unrolling those change kind of like if I were to do the same kind of transformation of like hash the initial input and then give you the sequence of transformations what's sort of the difference

performance wise and how should like someone think about like you know if you were to try to do the same thing for gpt3 what are kind of the obstacles or could it actually just work directly um the one of the problems with large machine learning models is uh well there's two problems one is that the weights are very large so for example the the weights of gpt3 so there's 175 billion of them um and continue to come into that is very very expensive the other problem is that there's actually a lot of computation that happens internally within the model and representing that within CK is also very expensive one of the details that I can go into is that basically like there's uh unfortunately two types of computations that happen in uh the machine learning

models these linear algebra operations and then these things called non-linearities the basic problem is that the proof systems that are really good for Matrix multiplications aren't good at non-linearities and vice versa so this is actually a really big challenge when it comes to doing zkml and it's also one of the things we're actively exploring on how to mitigate in terms of the performance implications of that actually maybe a very stupid naive question but you know in the same way that I guess you know if we go back to 2016 or 2015 like the idea of switching kind of like the smooth non-linear functions like a sigmoidal activation with like a soft Max or just max type of you know reload type of thing do you think that there's

going to be some like tailored non-linearity for generating ZK proofs that actually ends up being like adapted to the proof system like it's like some piecewise linear thing that's like easy to use so like maybe it doesn't get the same performance but it can it's like you're like trading off like proving time versus model accuracy and like you end up changing the architecture to do that there's some challenge to doing that the reason is that in to generate a zero knowledge proof you have to somehow transform your computation to one where every variable is an integer modulo a very large cryptographically chosen Prime but the core problem is that doing arithmetic say addition or multiplication over that Prime field is not really close to a differentiable

operation and the core premise of machine learning or at least deep learning is that your model should be differentiable or at least a discrete approximation to differentiable model so although different non-linearities could have somewhat different costs to implement in ZK you're always going to have to pay to somehow reconcile this fundamentally non-differentiable Prime field object with in deep Learning Land something you're fundamentally using to discretize a real analytic that's true but then then I always think about the fact that like Nvidia moved everything to 8-bit and you know like it's not really as smooth as you think it is as you know like the the and you could

argue that the 1980s theory of neural Nets got everything wrong because it assumed you had to be extremely smooth everywhere and like cybenko theorem only works in like certain limits right whatever but there's some sense in which like maybe there's some there's like some extra room wiggle room there that's I guess that's sort of maybe my my question is like do you think that there is that wiggle room where like you Custom Design architectures to be proof friendly or to like have sort of certain properties I think there's there's actually two answers to this actually but going specifically to your end video Point um as it turns out the weights are often stored in end date but the or say their floating point eight uh version

but the activations are often uh basically blown up to higher Precision uh in the intermediate step especially for the non-linearities for example if you're doing soft Max there's been a lot of work that has been shown that the softmax is very uh numerically unstable so you need to higher position in the intermediates I actually do think that there are potentially ways to basically bridge the gap of like differentiable uh ML and also zkp but one of the challenges is that if you want to deploy this you're going to convince practitioners to use a different non-linearity and machine learning is somewhat of a like just kind of black magic and convincing people to use like a different uh non-linearity that hasn't been tested over the course of like 10 years it's going to be a real challenge there's basically like five or

six of people use and making them switch is going to be quite challenging on the bright side I would say some of these quantization techniques are already the focus of a lot of work for people doing machine learning on edge devices basically to say power on your cell phone and it turns out that's very roughly speaking the difficulty of implementing inference in ZK can be proxied in some way by how much battery power a model actually takes so people have been working on quantizing these models and reducing the compute and we can actually leverage a lot of that work to pick the best model to put in VK I actually wanted to ask you a little bit

about the I know you have new work and you sort of mentioned like at this point to actually run these things locally would be kind of impossible but the other two works are there tools that exist today that can actually do that or those also theoretical like given the tooling that exists yeah could we for example prove that like a larger service is actually querying the model correctly behind an API so we can do that for certain kinds of models the models that we can do this on today are malls that people actually use in practice but they're not for example gpt3 and we're actively working to bridge that Gap as well so for example for small classification models we can we

can just run this today and that's when uh what we showed in our on our initial paper in ckml and what kind of tools do you use though in terms of like the ZK tools like are they the tools that we you know know well on this show like is it the circums and the Halos or is it something else yeah so the proving system that we used was the the print uh stack that we used was the Halo 2 uh proving stack cool it supports lookup tables which are really helpful for the nonlinearities that we discussed a few minutes ago one thing that we realized is that these generic printing tools that weren't really designed with machine learning or neural networks in mind actually have reached a point where you can kind of adapt them and actually get to some scale for neural networks so a lot of

Prior work in the KML would often hand craft a proof system for you know matrix multiplication or other basic operations and obviously that gives you a huge Advantage but what we realized is that using something actually put into production for engineering like Halo 2. it gives you access to better tooling and sort of real world implementations that you know these EK VM teams and other ZK teams are trying to run for performance and somehow empirically we found that that makes up for the trade-off so that a bit of context to what Daniel said on the image classification we were able to scale things up to a model that can handle a

data set called imagenet so this is probably the most cited data set in all of machine learning and it's essentially 224 by 224 size images of different Real World objects and for context all prior work was dealing with models that worked on much smaller images things like a data set called c bar 10 which is 32 by 32. so although 224 by 224 is still not where we really want to be it's a significant Step Up in scale that really was enabled by these more mature engineering for Halo 2. you know you brought up this this point about insta instead of actually evaluating a function you just like store

pre-computed lookup values in nearest neighbor it sounds like you're basically doing like round to nearest neighbor and then look up function value how long do you think it'll take for like a DSL like a circum to actually have those features like you write a nonlinear function but it converts it to polynomial you know like rational approximation or something like you know like if you look at a lot of math libraries in the rest of numerical computation there's a ton of stuff that's like hidden from the end user which like does all this type of stuff like does some of the pre-computation does some of these like polynomial approximation type of things do you think we'll get to a world where like the machine learning applications Drive

the library composition Behavior you know like kind of like you have with pi torch and tensorflow or do you do you think that it's too still too abstract from that that's actually something we're thinking about at Axiom we're targeting the on-chain user and there you know although maybe we could offer neural networks really people what people want is you know math functions like exponentiation or squaring and we're trying to write exactly fixed Point mathematical libraries to do this sort of thing and I think it's still a ways away from a point where the ordinary smart contract developer is going to write you know x times Y and then other under the hood it does some

crazy quadrature thing to develop a lookup table for that but I think in maybe one to two years we will reach that point yeah and foremost typically I'm actually working on open sourcing the the library that we built for basically taking models and tensorflow and turning them into certain knowledge proofs and we have some of the work that do or basically some of the techniques that you described um in this Library the reason I asked this is like when you think about how programming languages evolve sometimes it's like the first applications that are built around a programming language dictate like everything that's in it like in solidity right like why are address maps like the primary data structure well it was like that would

just happen to be like that's a historical Vestige of like the first applications being tokens in some ways that like that gets all the special priority in the compiler and stuff like that and so I'm just kind of curious like when you think about the the reason I was asking this was more like hey if ZK is moving in a particular future which application dictates like how the languages get developed and it sounds like to some extent it's it might be these math functions first rather than like other data structures that people use yeah definitely sarcoma's heavily influenced because it was developed to write the Hermes cdk roll up and you can even tell by the functions in circumplib there's a lot of very specific elliptic curve

verification a lot of very specific indexing and two arrays and there's definitely no math you you can add some numbers multiply some numbers and and that's it I I think with the newer proof systems like Halo 2 the user base is a bit more diverse and so we're now seeing many different groups develop libraries on top of Halo 2. for their specific application and I think it's exciting to see the proof system be flexible enough to really handle all of these it does make me think that there might be a like coming down the pipeline some sort of Halo 2 equivalent that is maybe built more with this more in mind interesting I mean there's there's

plonky too but I don't know if that is more suited for that or less I would say at the proof system level blocky 2 and Halo 2 are pretty close in adaptation to machine learning um but there are I think these per systems that are more tailor-made to operations like matrix multiplication are going to make a Resurgence now that I think there's a bit of production interest in machine learning there's more of an incentive to develop the tooling around those systems and maybe make a more fair comparison between them and you know planck-based systems like Halo 2 or plancky 2. you definitely don't want to be winning just on engineering are you thinking about stuff like Spartan because I think that's only

MSM now right yeah stuff like Spartan as well as some of these more some check related systems that uh actually have Matrix small as a primitive operation um obviously those that's not generically useful unless you really want to do some sort of you know machine learning or numerical type algorithm cool so what else do we see kind of in the future of CK or zkml what else were you guys thinking about yeah so Axiom we're talking with a lot of smart contract application teams about their on-chain needs and so one obstacle to actually deploying something like a big neural network on chain is that if you want to prove that you correctly did inference

on an image you have to be able to access that image on chain and if you've written any solidity before you know that accessing ethereum state is the most expensive operation of all on chain and so we think the models that are going to make sense on chain going forward are much simpler ones perhaps not at the scale of imagenet but much closer to things like linear regression or just traditional ml things like hey drink algorithms or PCA and so the space is still pretty early for that I think developers haven't really realized that's even possible but we're pretty excited to explore what people come up with what what era like if you were to look back in history of ml what year are we

at on on the blockchain or in zkml is it like 1986 or something what is it maybe better than that maybe uh 2001. there's a standard book you know elements of statistical learning that prior to deep learning was the canonical reference for what you would learn if you wanted to do applied machine learning and I think actually most of those algorithms uh many of those algorithms can be applied on chain today with the scale of data that is available and so we just need developers to actually want to put them into their applications we don't have the Hasty book yet though for for this I like we still we still kind of need that book and then the online course

that maps to it I don't think the Berkeley ones come anywhere close to how much the at the influence of like that kind of Trevor Hasty and tip chironi book had yeah I mean it's still early days you know to even implement the logarithm or exponential on chain it's a tough cast these days and to do it in ZK is also pretty challenging so we have a long ways to go before we're all doing you know logistic regression maybe it might be good to kind of conclude by talking about other applications maybe outside of just purely kind of some of the work you've done so far and sort of imagine what happens if we zip forward to 2012 in ml

when you know sort of imagenet got was created we we we we kind of reached that Nexus what what is what does the world look like what are the applications that exist yeah I think if you look at the history of ml 2012 was sort of a seismic shift so it's really difficult to guess what will happen next but I do think that there's some crazy things that you could imagine so one thing that I've just been thinking about toying around with is that you might imagine taking this idea of biometric identification and literally just putting it on chain um what you can do is you can basically put a hash of say some information about your face so that people can identify it and then use that to authenticate a smart contract if you have some really sensitive operation that you want to do

but beyond that I think there's also a bunch of interesting applications so for example you might want to have a data marketplace where you sell private data to different uh customers but if you want to prove how valuable this data is today you need to show them the data which sort of defeats the purpose of selling the data they can just take it and run and similarly I think you can also do things like prompt engineering marketplaces so if you have some really cool prompt for staple diffusion you can prove that you've ran stable diffusion on that prompt and then do something with that as well we're very very early days so the applications are just these are just things I've been toying around with and surely plenty of other ones I'm really excited to see

what people come up with I want to say a big thank you thank you both of you for coming on the show thanks for sharing with us sort of your histories and also all the work you've been doing on zkml and what might be coming up in the future yeah thanks for having Sun thanks for having us on I want to say thank you to the ZK team Henrik Rachel Adam and Tanya and tour listeners thanks for listening foreign