Episode 370 - ZKTorch & the Evolution of ZKML with Daniel Kang

Transcript

[Music] Welcome to Zero Knowledge. I'm your host, Anna Rose. In this podcast, we will be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms that promise to change the way we interact and transact online. [Music] This week I chat with Daniel Kang, professor at UIU and founding technical adviser at Vale. We do an update on ZKML and how the space has developed since we last spoke about it in early 2023. We discuss some of the uses for ZKML and

also some of the challenges of adding ZK and proofs to these models. He covers the 2023 2024 cohort of ZKML tools like ZK CNN, ZKLLM, Ezekiel, and his older project ZKML and also shares more about his new project ZK torch, a tool that aims to offer general purpose ZK proving about ML systems quickly, but also be compatible with many of the emerging models. Now, before we kick off, I just want to point you towards the ZK jobs board. There you can find job postings from top teams working in ZK. And if you're a team looking to hire, you can also post your jobs there today. We've heard great things from teams who found their perfect hire through this platform, and we hope it can help you as well. Find out more over at

jobsboard.zeronowledge.fm. You can find this on our website, and we've added it to the show notes. Now, here is our episode with Daniel Kang. Today, I'm here with Daniel Kang, professor at UIU and founding technical adviser at Vale. Welcome back to the show Daniel. >> Thank you. So, it's been uh it's been a while. >> Yeah. You came on the show at the start of 2023 with Ye and I think that was actually the first time we talked about ZK and ML, that intersection on this show. So, for me, it was like a crazy introduction to a world I I really didn't know. Later on that year, you did come back on the show. I was just telling you this before you didn't

remember this, but you came on for a very short bonus episode where we talked about this audio experiment, the attested audio experiment that you, Kobe, and I did back in 2023. And that was using our audio and then using ZK to proof providence and proof of transformations. Today, I hope we get to dive into the the latest in ZKML, talk a little bit about your project, ZK torch. Um, but yeah, I'm really excited to have you back on. Yeah, I'm really excited to be here. >> Cool. I want to hear before we kind of dive into all of that though, what have you been up to since 2023, like what other work have you been working on?

What other kind of papers have you released? >> Yeah. So, well, professionally, uh, in 2023, I was a posttock at Berkeley. Uh, and I've since started my faculty position at UIU officially, although I had the offer back then. >> Okay. >> And I spent a lot of time, uh, spinning up my lab at UIU. that's been quite hectic for me in terms of the research. A lot of it has been around uh auditing AI and measuring the progress of AI and I see these as as very related because uh for a variety of reasons you really want to know how AI is going to progress and ensure that our measurements are done accurately and honestly. M >> so I've had a particular focus on AI agents uh benchmark quality uh if I may

brag had my first ever uh top page of hacker news uh last week. >> Cool. What was it for? >> This was on a study a deep dive on AI agent benchmark quality. >> Okay. >> And we found a lot of issues with AI agent benchmarks. >> What kind of benchmarking efforts are actually happening around agents right now? >> There's a there's a large number of of efforts across a variety of institutions. So this was both academic in the frontier labs uh and from other industrial partners. >> They range from everything from computer use so how well an AI agent can navigate say a web browser or your computer >> uh to say cyber security task which my lab has uh focused on.

>> Oh cool. >> Uh to things like software engineering like uh Sweetbench which is a very famous uh benchmark for AI agents. >> Nice. I did an episode recently with David Wong from ZK Security who's an auditor using AI agents. I feel like that maybe would be a relevant thing for folks to listen to. I don't know if you had a chance to hear it, but when you say auditing AI though in your case, you're actually like are you going into the models and auditing them or are you like looking at outputs? Are you focused more on these benchmarks? Like when you say that auditing AI, what does it mean?

>> Yeah. Yeah. So, we're building the technology to enable all of the applications you mentioned. So, this includes uh looking into the model, so to speak. This includes auditing its outputs. Uh and it includes measuring uh how well these AI uh do in different circumstances. >> Very cool. You're affiliated with this project veil. I don't think you were last time we spoke. What is veil? Veil is a startup uh funded by uh Manish Sha and Jonishan and they are uh focused on high assurance AI and this includes audits of AI systems uh both depending on the application uh training time and also at inference time and they're leveraging a variety of technologies including uh ZDK in some circumstances and also trusted enclaves in other

circumstances to perform these audits uh in these high sharing situations. >> That's nice. And are you're a founding member. So is this like is it a is it a startup you're a co-founder of like recently or >> Well, I don't know what my official position is. I I call myself the founding technical adviser. Uh >> okay. >> And I I view myself as a as an adviser. They're really the ones doing the hard work. I'm still full-time at the university uh and focused on the research. >> Got it. Back in 20 I sort of want to do a bit of a throwback to 2023 and ZKML when we were first introducing it. I feel like back then I mean there was a community that was formed around ZKML.

This is around the time ZKVMs are just like becoming a concept really like before that it was like rollups, it was compression, it was light clients, it was privacy tech but ZKML became a category basically in 2023. I'm really curious like if if you can remember going back in time what were the kind of like proposals in 2023 and I want to hear from you which ones kind of panned out or still relevant and which ones we realized weren't really good use cases.

>> So my memory might be a bit fuzzy but here's what I remember. Okay. Uh there are proposals on the technical side and there proposals on the application side. Uh on the application side, these proposals range from um verified face ID things like proof of prompt uh audits of say uh financial models to uh for loan applications uh to healthcare applications. >> Mhm. >> I I think in terms of the applications, these are all still good applications. Uh but the technology is still a little far away in terms of really transforming uh how these audits work in practice because of the computational expense of the CK systems for ML. M >> uh since then there have been a lot more applications people want to run but

they're still a bit bottlenecked by the underlying technology. >> Okay. >> Uh this includes things like proof of training, proof that say your AI system uh isn't biased, say your chatbot system isn't biased and many other uh such applications. I I still think these are all good applications >> in the case like if you look at some of the stuff kind of coming out around Grock some of the assumptions of what's happening behind the scenes that's leading to some of the outcomes. Are you thinking like ZK could be used somewhere to like prove that something was included or something wasn't included?

Is that sort of what you mean by that? Or like that AI isn't biased that it isn't getting sort of like pushed in a certain direction? Yeah, that's a great question and that is the goal of these systems. >> Uh we're currently bottlenecked by the uh the computational expense and the technical aspects uh to accomplishing that goal. But uh my lab has been working on this over the past few years and there have been complimentary technologies uh which can be used uh in conjunction to assist with these kinds of uh of audits for example enclaves uh and other technologies >> like TEES.

>> Yeah, TEES. >> Okay. So it maybe would you still be using ZK in that case? ZK ants or is it just TE's? >> Yeah, so it really depends on the application. Uh there are some applications where for whatever reason you don't have access to an enclave and so you really need the ZK. >> Mhm. >> Uh there are other applications where you want to do uh some part of say the ZK within the TE. Uh there are also some applications where you might just trust the TE in the first place. >> Uh and it really depends on the assurance level that you want uh and also the the threat model that you're assuming. For example, if you're assuming, say that there's a nation state who's trying to uh bypass your system, maybe uh a TE by itself is not something that you would really trust

because they can physically scan the hardware for the uh for the private keys. >> Uh whereas they can't really cheat CK as far as we know. >> So it really depends on the application at hand uh and the level of assurance that you care about. >> I want to go back to the first application you mentioned, this verified face ID. Like we've had a bunch of episodes recently on like ZK ID. So this is, you know, mo mostly it currently looks like passports or licenses that are proven into an app in ZK, kept private, selective disclosure, but almost all of those episodes that we did and projects that we spoke to have somewhere on their radar this idea of like also doing like KYC locally.

Is that what you're talking about here? When you say verified face ID, are you thinking or is this like at a different place like is this happening in a different spot? >> Yeah. So, so verified face ID can be used in a variety of applications. Uh and KYC is probably the most prominent one uh that is being studied today and also probably the one that's going to be have the highest impact. Uh so for example, if I steal someone's passport, uh I can still use it in these uh ZK passport or ZK ID uh uh scenarios because there's no way to confirm that uh the entity that claims they have the passport is actually the person in the passport.

>> Yeah. >> Uh and today, as far as I'm aware, there's no way to do this in a privacy preserving way. So if you've used a lot of government systems or other systems that require high assurance, they take a picture of your face uh on say your webcam or on your phone >> and they upload it to a server. >> Totally. Not private. >> No, not private. Exactly. >> I mean, attempts at privacy, but very, you know, likely leaked at some point. You mentioned proof of prompt and proof of training. I actually have questions about that later, so maybe we come back to those applications. You also kind of said that there was going back to the 2023 question, we had te the tech side and then the application side. So, you've covered some of the applications.

Where were we on the tech side and what how far have we come? Have we made it? Do we have things that we can actually use today or are we still sort of like yeah are we still bottlenecked? >> So we've come a long way since 2023. Uh just to give a little bit of a history of uh ZKML >> initially a lot of the proposals uh EKL including my work uh focused on building circuits using general purpose proving systems. So building like say a plon circuit to prove an ML model. >> Mhm. The core issue with this is that due to the way that these high level cryptographic systems work, it just it's just impossible to get it to a point where you can run large ML models 30 systems uh with circuits.

And people realized this actually very quickly once the initial proposals were out and once the initial benchmarks were were run >> and so as a result uh there have been a few directions where people have been trying to push things towards. uh one is this uh idea of basically putting in specialized uh circuits into things like plon. So for example not in zkml but if you look at a system like SP1 they have these uh specialized circuits that are much more efficient than just running the underlying computation uh in SP1. >> So that's been one direction. Uh the other direction that people have been taking is to build highly specialized uh cryptographic systems for a single class of model or maybe even a single model.

So these are systems like uh academic systems like ZKCNN >> and ZKCNN is purpose-built for basically doing convolutions and reln. Are these being specifically built for ZK circuits or for AI models? Are they matching one ZK system to one model or are they Yeah, I'm just trying to figure out like where is that that limitation? >> Yeah. So, so the way to think about is that you take a specific AI model architecture >> and then you build a cryptographic proving system for that model. like is it using existing ZK techniques, ZK systems and then just kind of

customizing it for this AI model or is it like making up its its own? >> So it is using ideas from ZK uh and then uh adopting them to the specific model uh at hand. >> Okay, which like ZK CNN what model does that work on? Yeah. So these work on convolutional neural networks which is a class of models u but uh roughly speaking they can prove VG style models and ResNet style models and maybe a few more but that's roughly the scope of that project. Got it. >> In particular it can't prove things like LLMs or other models that people are excited by today.

>> When you talk about these different models I because I'm not so deep in AI I'm not as familiar with what each of these do. Like when you say an LLM, does it include the image generation ones or are those also a different class? >> Well, the reason that that you might find is this confusing is because it is legitimately confusing. >> Okay. >> Well, because it's large language model, but then I'm wondering if it's used for image creation or recognition. >> Again, the reason you find this confusing is because it is legitimately confusing. >> Okay. Even though LLM stands for large language model, people abuse that term in ways that, you know, don't necessarily reflect the L in the the initial L. Okay?

>> I'm sorry, the second L, I guess. Um, >> so for example, GPT40, people throw around the term LLM for GPT40, >> but it is actually not specifically a language model. uh it is it is a transformerbased model we think or a modification of a transformer based model but it accept multiple modalities as input and multiple modalities as output. So I sometimes even might say that GPT40 is an LLM but it is actually not technically and so it's just a confusing state of things and especially because a lot of these are prop uh all the details are proprietary.

>> Got it. Going back to this ZK CNN you mentioned this does work with VGG and ResNet. What what are those two? What do they do that's different? >> Yeah. So AI is a very large space. So these kinds of arbitrary distinctions sometimes can be confusing. But I'm just going to make an arbitrary distinction which is discriminative AI and generative AI. >> Okay. >> So in generative AI you want to generate some outputs like an image or a text and say a chatbot setting. >> Yeah. >> And in discriminative AI you want to tell two things apart. So tell whether uh an image of an X-ray has >> say uh or say CAT scan has a tumor or not. And so this is a classification uh >> interesting.

>> So VGG and ResNet have historically been used for classification or discrimination. And this is in contrast to models like GPT40 which is used for generation. >> Got it. And would this be going back to the ID example, the KYC with using facial recognition? Would that usually be using one of these discriminative models? >> That's right. >> It sounds like it. >> Yep. >> Is it often images or it could be any? I mean, I'm guessing it's anything, right? Like any data comparison. >> Yeah. Yeah. So, so discriminative models can be used for any modality. So, they can be used for images, but they can also be used for text. They can be used for things like LAR. And so LAR is particularly important for uh for depth

which is very important for actually face ID. So you can't take a picture of a picture >> and self-driving stuff. I guess that's about why >> and self-driving stuff. Exactly. Uh but typically the architectures used for different modalities look different in discriminative AI. Uh so what you use for text is often a different architecture than what you use for say images. >> Got it. Okay. Going back to those types of um sort of specialized ZK I don't know ZKML systems. What about something like ZKLLM? I'm guessing that deals more with the generative stuff.

>> Yeah, that's right. Uh uh so ZKLM is another academic system that was I think developed in 2024 and was a proof of concept to showcase that you can do this specialized uh cryptographic basically make a specialized cryptographic proving system uh for uh LLMs in the ZK setting and the way it works is very similar in in principle to ZKCNN where they adopt ideas from other cryptographic protocols to LLMs. When you talk about this academic work, are these proving systems? Like what are they? So they're like they're like new ZK systems that are focused on this particular use case.

>> Yes, these are proving systems. >> Okay. Who developed them? >> Yeah. So ZK CNN was developed uh in part by one of my colleagues at Yupang Zang uh who was who is now at UIU. >> Oh yeah. >> And ZKLM I don't remember offh hand. I think it was developed by folks at the University of I want to say Toronto or Waterl. Uh I don't think they're working on uh ZKML anymore though. >> Okay. Are these like they're their academic works? Are these proof of concepts? Have they ever been implemented? Have they been tested? Like I'm kind of trying to figure out like what what would they exist as in the world? Are they like something one could run and check on an LLM or is it more just like this is theoretically how we could start proving things about what's

happening under the hood? >> Yeah. Yeah. So there's a whole spectrum between uh like literally just like a piece of paper and like an idea to an actual implementation. >> Yeah. >> Uh and both these systems span somewhere in between. >> Okay. They have like test implementations but nothing production ready maybe. >> Yeah. Yeah. And one of the things I actually really want to highlight about ZKLM in particular is that the authors did not benchmark their system correctly. >> Okay. Uh and this is actually very important because they make wildly overblown claims of performance of their system to the point where if you measure the system correctly, it is actually unusable in terms of performance.

>> Oh. >> Uh and so it actually got a lot of hype when it first came out. Uh but unfortunately the hype is not substantiated for this project. >> I see. We're still on this tech side of things. I want to talk a little bit about the projects that did emerge in 2023 kind of coming more from the ZK space. There was your project ZKML, there was Ezekiel as well. How are those different like what are those types of systems models? Like what are they kind of proposing? Why are they different from something like ZK CNN and ZKLLM? >> Yeah. So ZK CNN and ZKLM they both develop these custom cryptographic protocols.

>> Yeah. >> Uh whereas both EZKL and ZKML uh use existing proving systems and then build circuits to represent the AI or ML models within those uh generic proving systems. Hm. In that case, were they meant to cover more of those models? Was it like you create this larger tent and you're able to do both the generative and the discriminative and the sort of variations in between? Is that sort of what the proposal is with those? >> Yeah, that was the rough idea. Uh because you you you're using generic circuits, they can be very very general purpose.

>> Uhhuh. So these circuits can represent arbitrary computation up to the the size of the circuit that that you represent. And so uh you can represent any AI model or any ML model in these uh in these systems. The core issue and the core tension is that these generic proving systems because they're not specialized to AI or ML are just outrageously expensive. >> Ah is it slow? Is it that you need a ton of like computational power? When do you say when you say expensive what do you mean >> both?

>> Okay. Time and computational power. >> Got it. >> And actually one thing I wanted to explicitly do is like if you draw an axis of like say flexibility on one axis and say speed or say amount of compute on another axis. ZKLLM and ZK CNN are very much on the not flexible but fairly high performance and systems like ZKML and EZKL are very much on the side of much slower uh but much more flexible. >> What would you do? Like I I'm trying to picture how these things are actually used. If you wanted to prove something about like a I don't know chat GPT something GPT4 40 would you be creating like within ZK ML or Ezekiel would you

be like using this generic circuit but then recreating something like do you have to still customize yourself something using this tool or would you be able to like do it out of the box like is it already built up for you to do it? >> It's closer to the letter. Uh so essentially the way these works is that you have a specification of some model. So whether it be GPT40 whether it be a more classical model say ResNet and given the specification of the model the systems will take that and then underneath the hood translate it to the circuit and then do the proving. So the developer of the AI model can simply plug in their specification and the the systems will do the rest.

>> Okay. And I kind of going back to both of these cases because you you sort of mentioned all these different things one could prove potentially proof of prompts, proof of input, proof of output. What are these actually built to prove? Are they meant to prove all sorts of things or are they also kind of narrowly focused on like proving one thing? >> They can be used for all those applications. >> This is the Ezekiel and ZKML could be used for everything. So ZKLLM can be used for all these in the LLM specific setting.

>> Got it. >> But it's not generic. And so for example, it can't prove these classification models. >> Okay. So like all like all of four of the systems we just outlined are meant to actually prove all sorts of things. They're not specific to one like just proving input, just proving model. Would like theoretically these also be able to prove bias and like some of those other applications you mentioned? >> Not by themselves. that had to be combined with a something like a ZK VM to do some generic computation, but then uh you could basically hook them together to do to do that.

>> Oh, interesting. Cool. Okay, so you've painted this picture of the axes, the speed versus sort of how generic they are. Ezekiel ZKML living on one corner, the ZK CNN and ZKLLM living on another. Now, I I think it's a great moment to talk about ZK torch because, as I understand it, it bridges the gap. I will make a quick note. You did a talk at ZK Summit 13 in Toronto. Um, in that you have that diagram, so I'll I'll definitely link that in the show notes. I had a chance to watch this before this interview. Um, we do follow along a little bit similar, so if people need some visuals, that could be a good place to see it. But yeah, why don't we

talk about CK torch? >> Yeah, thank you. Uh so in my lab we saw that these issues were going to be a a major problem. >> Uh and there are several reasons for this. So one is simply that uh the the space of AI moves so quickly that first of all it's hard to keep up and second uh that any any specific system that we build might be obsolete 12 months from now. >> Uh but the other uh thing that we saw is that these models generally speaking are getting bigger and requiring more computation. So if you lose flexibility or you lose the speed, you get left behind uh in this space. And so we felt the need to build a system that is both flexible and high performance. And this is what led us to ZK torch.

>> I'm curious uh is it inspired by PyTorch? Is that where that name comes from? >> Yeah. Yeah, it actually is inspired from PyTorch. Uh the the basic idea in PyTorch is again you know PyTorch is a it's a large system that does many things but the the what we were inspired by that PyTorch defines a bunch of primitive operations that can be combined to build these complex models and it can be basically used to implement any AI model that you could you could want. >> And our system does something very similar. We have a bunch of primitive operations that can be put together to construct these AI models.

>> Are you like so this is what I'm trying to figure out is like what approach did you take? In one case there's very custom systems. In the other case there's very general purpose. It almost sounds like you kind of created modules or something that people can put together like Lego pieces. So it's not fully general purpose out of the box. one still has to kind of select this combination. Does does the developer end up having to do a lot more work to use ZK torch? >> So fortunately no. Uh we did the work to build the Lego blocks and then also uh we have an automatic assembler for the Lego blocks.

>> Okay. >> So as long as you can build the structure using Legos, uh it works within our system. And is the idea here like you make it into blocks into these different components to be able to keep up with the innovation like every time a new technique is introduced you could create a new like module instead of having to like redo this entire system. >> Yeah, that's right. But the thing I will say about this is that uh even new models as far as we're aware again some of these are proprietary actually use a lot of the similar uh Lego blocks. And if you think about Lego, there actually are only a few kinds of pieces that can be put together to make these incredible structures. So we think this idea is very powerful.

>> Cool. I think I saw a system called ZK Pie torch. Are there competitors to this? Like do you know if there's other teams trying to build similar sort of high performance but general purpose tool sets? Actually, yeah. Is ZK torch a tool set? >> Yeah. Yeah, you can think of it as a tool set. >> Okay. Are there any competitors? Yeah, I haven't been keeping up too much. Uh, we've been focused a lot on the work in our lab. I think ZK PyTorch is one that's the one that I'm aware of that takes a similar idea. >> Got it.

>> But there could be others >> with ZK torch as it stands. How far along is it? Like is it in sort of proposal state? Is it implemented? Can people already use it? Could they use it with existing systems? Yeah. I'm kind of curious like does this bring us closer to some of the applications that you talked about and and if so, how close? >> Yeah, we'd like to think so at the very least. Um and we have an implementation that will be released by the time this podcast goes out uh and a and a technical report describing our methods >> and with basically uh some translation and can be used on existing applications.

>> That's cool. And so would it like with ZK torch as it stands once kind of implemented and released could someone start to experiment with that like facial recognition or any of the other applications? >> Yes. Uh that uh they could. Yeah. >> Cool. And with this sort of set too like I think I'm still trying to figure out like what are you like where are the proofs actually happening because you know I thought I thought about this like okay with the visual uh recognition this is happening on a local device. Are you running ZK Torch locally? Is that something that could happen or like Yeah, I'm just I'm trying to picture like where this lives.

>> Yeah, that's that's certainly the the goal. I will say that uh while we've gotten a lot uh of performance out of our our system and our proving system, there is still a little bit left uh to be to be done to do things like uh verified Face ID say on a phone. That's uh the phone unfortunately has >> limited power, limited computation. So uh it's a little bit away from there but um to be very concrete there's a benchmark uh called ML Perf uh specifically the MLF inference benchmark >> and as far as I'm aware we are the first system and I I believe the only system that can prove every every model on the MLF v4.1 uh edge inference class of models >> okay >> and and so this includes things like the models similarly used for face ID this

includes LLMs this includes medical classification models this includes recommener system models and others as well. >> What you because maybe the performance still limits you from doing it on a phone, so it's not quite that performant. What can you run it on? Like what are the minimum specs you need? >> So, it depends on the size of the model. Uh you can actually run a bunch of these on your laptop. >> Okay. >> But, uh as you get to larger and larger models, uh you'll need to have some beefier servers. And are is the visual recognition like a pretty large model? Or maybe it depends on how much accuracy you're looking for. I don't know.

>> That's actually exactly right. Depends how much accuracy you're looking for. And I think I would have to verify this, but I'm actually fairly certain that if you are okay with the smaller uh face recognition models, >> you actually could just run it on your phone. >> Okay. >> But if you want high accuracy, you probably need something like a server. >> And and even the laptop wouldn't work yet. >> Uh I think the laptop is somewhere in between. like you could do like the middle the middle one. >> Okay, that's really helpful actually to understand because there's these different like sort of these different requirements depending on the use case like for something very benign where you just want someone to ID themselves but it's not like super security oriented maybe an onphone ID would work like getting into a bar or something like

that whereas if it's like I don't know some big financial bank or something like that they'd want something a little more accurate. >> Yeah. Exactly. So like I'm picturing you've built a lot of these modules kind of going back to how this is built. What happens when models change like are you always aware of what's changed in the model and are you able to adapt those modules for the updated models because also sometimes the what's happening under the hood in these models isn't entirely clear. Right.

>> Yeah, that's right. Uh so we have a set of modules that support all the uh all the models in this MLF benchmark that I mentioned. Okay. >> Um, we think they're pretty general. Uh, but of course there could be other models that come out with that require other building blocks, but slotting in new building blocks in our system is very easy. >> How do you look into those systems though into those models to be able to do that? Like how is it visible for you given that a lot of them are proprietary and not open source? >> Okay, so there's actually two things to say here. The first is that actually a lot of architectures are relatively open source. Okay.

>> So even if you take say take something like Deepseek R1 or Kimik K2 uh and you fine-tune those the architecture is actually still open source. So you can still >> uh use our system for that >> for the uh systems like say GPT40 or uh Claude Sonnet or Opus 4 those are proprietary. This is where something like an enclave could be helpful. You could actually like use ZK torch to prove the inference and then you can actually say like use an enclave to say that yeah I ran ZK torch on this unspecified model architecture but uh we actually run it.

>> H but who does that? It's not the it's not the companies releasing these things that would like offer that information is it it's the it's the individual developer. It's like the outside person. Well, the the open AI orthropic could enable this and then they could uh send the proofs uh for to an outside developer for verification. >> So it would be internal like they would have to basically accept this into their system. >> Yeah. Yeah. ZK is always an opt-in system. You have to you have to say like oh I want to use this technology. >> I see. In terms of that enclave work, is that also under the ZK torch umbrella or is that something you're kind of expecting external partners to build?

>> We're expecting ex external partners to build it. Uh the enclave stuff is >> relatively more settled in the sense of the technology is a bit like there are prototypes out there for different applications. So adopting them is is we expect to be more on the engineering side than as opposed to the research and development side. >> Got it. When you talk about it being released, how much of it is released? >> Uh yeah, so we plan on releasing all of it. Uh I believe with Apache license uh we want we want this to be widely used. >> Interesting. >> It's uh it is a library but it can also be used as a standalone tool. And would you be sharing that with the model owners then? Is that sort of like you pitch it to them or do you pitch it to other like sort of just applications

that would want to work with those models? >> Right. So the because Z there are many applications of ZKML they're used by different parties. So for example, if OpenAI wanted to use our system to prove say GP40 inference, they would need to opt in and and use the system internally and then send the proofs elsewhere. >> Great. If you wanted to do something like verified face ID, that would be on say a user's phone and they would be doing approving. So it depends on the application and who is involved with approving and who they're trying to approve to. >> Are there like cases where the input and the output would be the purview of some external developer? Like are there reasons that you'd want to prove like I put this in and got this output and would your would your tool be useful for that?

>> Yeah. Yeah. So so imagine that uh you have a set of medical models. So the example I describe, you can imagine like a hacker doing it, but there can also just be literally just mistakes as happen in the medical system. >> You might want to say that I want to ensure that this uh medically approved model was run on my input data. >> Okay. >> Uh and ZK torch would enable you to to basically stamp that and be like yes, this specific output, the specific diagnosis was done by the specific ML model on your data. >> Okay. So yeah, what's the what's the BD of this then? Is it sort of like shopping it to these different companies and being like you want to do this this proves things about your model without revealing anything?

>> Yeah, that's a great question. Uh I'm focused on the R&D so everything I'm saying is speculation. But >> uh one thing I will say is that uh ZK and actually just a lot of cryptographic technologies even beyond ZK are just not really well known in the outside world. So if you talk to like a medical records provider they >> they don't know it. >> They don't know. Uh so just education like trying to get people to know about this technology and what is enabled. >> Yeah, >> I personally care about making sure that if I you know use uh AI for medical situations that I know what's that is my data and it's the approved model >> and just so that there there are no mistakes uh no errors in the computation or anything else of that form.

>> And so I'd be you know I would be delighted to have that uh and I'm hoping that this becomes more of a thing that gets integrated into society. It's funny because the uh use case for medical I remember back in 2017 it was like blockchain and medical and ZK was sort of like sprinkled in there but it was more like this shared ledger among hospitals and then ZK would be used for privacy. That idea was proposed didn't get adoption. Why? Because the medical system is incredibly slow and and they're very afraid of any sort of like new tech. They're very riskaverse.

I wonder with something like ZK, what are the steps that have to happen before that industry or companies like that, organizations like that would start to trust it? Like I think the Google wallet integration is one step in a good direction because it's sort of showing like well Google's trusting some ZK system. Granted, it's like internally developed and and customized, but clearly that concept is is at least being accepted by one of the top tech companies in the world to be used on, you know, millions of pieces of data. So, and personal data.

>> Yeah. I mean, the medical system is very conservative for obvious reasons. um you know there there are good reasons for conservativism and I think that there are a lot of uh people who uh have their own theories of medical you know scenarios that might actually not be helpful for patients uh and and I think that uh just seeing a lot of medical technology follows from other technologies >> so even electronic medical records I mean that followed from a lot of like from industry adoption from >> maybe like banking or something >> yeah banking other other secure applications And banking I guess comes from like it follows something else. It's a little less high risk. I don't know which thing but Yeah.

>> Yeah. Yeah. Uh and so uh the medical system moves very slowly. I anticipate that as these things become more and more prevalent uh and becomes more of a uh more widely accepted then they'll be more willing to adopt uh these technologies. >> Yeah, I hope so too. I mean it's funny. I've I've thought a lot about this like medical NZK concept because what I find really funny about the medical systems and and organizations risk aversion is I feel like they're also the most susceptible to the hacks. Like I actually feel slightly uncomfortable sharing any data with any hospital because I'm likelihood you get hacked more than than like you

know my computer or you know like it's it's it's really funny and each one you know if you go to different doctors different or like different institutions or whatever you're inevitably bringing your data from one to another you're doing it in incredibly archaic ways like a little side story some people who know me personally know this I had this knee injury a few years ago and it was really bad. But I also travel a lot and I had to go to doctors in different countries and I had to bring with me like print out of paper CD of the MRI and it's like the most ridiculously insecure way to ho hold data like if I lost my bag I'd lose all that stuff is also incompatible across different regions. And going

through that experience I think was very eye openening where it's like cool guys you're super super privacy oriented and yet what ends up happening is incredibly like riskon and it doesn't work very well. So yeah I was pretty frustrated by that whole thing. >> Yeah I'm really sorry about that. Uh I was I was I'm not surprised that the CDROMs uh did even though it's 2025 >> they didn't work in North the European ones didn't work in North America. It was amazing. >> Yeah. Yeah, I think that uh the medical folks are very well-intentioned but you know their expertise is in the diagnosis and you know the other parts of the medical system and they're not really let's say tech oriented uh so it is a slowmoving process and I I hope it

accelerates but uh yeah we'll see what happens. >> I wonder if they're not almost putting themselves at higher risk. a lot of the the those archaic systems, the ones that aren't cryptographically up to-date, I mean, I think they're more they're becoming more and more insecure. Like, I know that people feel like they're battle tested, but also I mean, I think with quantum computing and all of that coming online, like there's going to have to be an acceleration of adoption of some of these new things pretty soon, I think. >> Yeah, I hope so. >> That leads me to the question of sort of the quantum ZK. When you talk about these ZK systems, we didn't really go into the details of the types of ZK systems that these are kind of inspired by. Are they quantum secure?

>> So there are many underlying technologies that can be used in in different ZK proving systems. We think some of them are postquantum secure uh and some of them are just not. Um yeah, >> so a lot of the existing systems are not uh postquantum scare. They use technology that just doesn't work in the quantum era. And in the examples that you gave like the ZKLLM or ZK torch, are those using are like basically are they hash hashbased or elliptic curve based are they yeah are they using any of these techniques that we we believe to be postquantum? >> As far as I'm aware they do not.

>> Okay. They may need to be upgraded to that. Do the AI companies care? >> So I don't know if the AI companies care. Uh we care in our lab. Um, and >> we also care about uh high performance systems and so we're actually going to be transitioning uh to working on postquantum secure and higher performance uh zk proving systems for >> nice models and you know we don't have specific numbers but we anticipate that's going to even uh be much more high performance than zk torch. >> Very cool. Which systems are you looking at either being inspired by or potentially implementing? Yeah. So there's a bunch of work around uh using uh ZK systems uh using this paradigm like low rank uh matrices like learning with errors uh that are thought to be postquantum secure.

>> Uh and we're going to be leveraging that that set of technologies. >> Cool. Are there any names that we might recognize? >> Yes, there are. And I do not remember any of them off top my head. Uh >> unfortunately >> we can ask our listeners to try to dig up what you mean there. That sounds good. >> Yeah. >> I want to understand. And so you sort of mentioned the proof of prompt. This is kind of going back to that application. When when you say that like what does that mean? Is that is that similar to like proof of input to output or is it is proof of prompt different? >> Yeah, it is a proof of input to an output. So >> okay, >> let's say for image generation uh these prompts are often very involved uh because they're these systems are very

opaque. it's not clear how to generate an output uh given one of these models and images are one such case where it's really difficult to reproduce manually and one thing you can imagine doing for proof of prompt is that you could say that hey I produce this image and if you want to modify it I can actually prove that uh this prompt generated that image and so you can then purchase the prompt uh and be assured that >> that correspondence was there you could also imagine someone uh having a portfolio where they don't release the prompts uh which are maybe preparatory uh but release the images and say that I actually generated these images and I know the prompt to generate these images.

>> Interesting. Another one you we we didn't get a chance to talk about in the application front was proof of training. This is different I guess from proof of inference. What is proof of training? >> Yeah. So with proof of inference, you have a model that has already been trained and you want to say that this model produces output. >> Okay. With proof of training, the entity that trained the model can prove that they started from either some base model and fine-tuned that model on a spec specific set of data or that they train a model from scratch again on a specific set of data.

>> Is this where like kind of going back to that proof of bias because it's really an interesting concept there of like how to prove you are or are not biased. Is that related to proof of training? Is it sort of like proving that you have not included some data or that you have included some data? >> Yeah. So the proof of training in and of itself just proves that a model was trained was trained from some data. >> Okay. >> Now once you have that assurance, you can then do things like say that proof that my my data was is demographically balanced.

>> Mhm. And this is one way that you can uh uh start to layer on assurances of that the entire uh AI system as a whole went through best practices to remove bias. >> You are still relying though on some third party that like has decided what a balanced data set looks like I guess right >> that's right so some auditor or some legal entity would come in and say that we think that this is a something that's balanced uh and then uh they'd have to go from there. Uh but of course this is a larger societal question. uh we are providing the technology to enable this depending on what the regulators or our auditors want.

>> Got it. And going back to ZK torch, ZK torch could potentially prove these things or could in the future. >> Yeah. So ZK torch can prove training. It is just very expensive to do so. Uh there are a set of techniques that we think are going to be useful to reducing that cost. Uh but uh we are not uh there yet. >> Okay. So today ZK torch should be used primarily for which kinds of proving? >> Proofs of inference. >> Proof of inference. Okay. And the input output or no?

>> Uh any combination can be kept uh private or or public. So you can keep the input public or private and the output public or private. >> Okay. Cool. Now I want to revisit benchmarking. You talked a little bit about you know different kinds of benchmarking within the AI space. Is there like specific ZK ML or ZK AI benchmarks as well or is it like you're using are you focused more on like the ML benchmarks and then seeing how these systems kind of compete using ZK? >> Uh definitely the latter. Um I do think that uh uh if ZKML becomes uh more of an academic discipline there should be some good bench there should be benchmarks that are commonly used. Um we think that the MLF uh inference suite is a good

starting point for that. Have you been following at all the benchmarking in sort of the ZK performance space? Because that's like that's also something we've talked about a couple times on the show. I'm sort of wondering if like given that I mean you're coming much more from the ML side of things introducing ZK, but like yeah, I'm wondering like if people coming from the ZK side would view this differently and if you're familiar with it. >> Yeah, so there are lots of benchmarking efforts in ZK. A lot of them focus on things like rollups, uh ZKVMs. Uh I'm not aware of coordinated efforts to build standards around benchmarks for ZKML.

>> Uh but as this becomes more prevalent, I hope uh I'm happy to to work on these efforts. >> Would more performant ZK systems in that ZKVM side of things be useful in what you're doing or are the systems so different that you cannot compare? Like they might be amazing at compression over there in the blockchain land, but it wouldn't be at all useful. like those techniques don't actually work with the AI systems or they're actually slow. I'm just curious if there's any connection there. >> Not directly, but uh we were using a lot of ideas from the general purpose CK proving systems. Oh, >> that's cool.

>> And also a lot of the applications that I mentioned, for example, like proving bias. This is actually sort of an aggregation on top of uh input output pairs. >> So you'd have to use something like a ZKVM to then prove that part. Uh so any improvements there obviously help us and we'd be delighted to have these systems improve. >> Very cool. Uh on a recent episode we did with MU, we were talking about Liharero and privacy actually and how like a lot of the benchmarking in today's ZK land doesn't actually take into account like how private it is or how effective it is at at keeping privacy. In the case of ZKML, do we feel like and ZKML as a topic, not as a tool set, do we feel like that kind of benchmark would be

important? I mean, this kind of goes back to that question of like how private do we need this kind of privacy if you're proving something about the inside of a model. Is that like dire that it be correct or could like yeah does it have to be completely private? >> So, it actually really depends on the application. uh they're uh in in the medical setting uh maybe some model provider uh has like a a copyright or trademark or whatever other restriction on their model and they're actually willing to release it as an open weight model. And obviously the medical system is highly integrated with the rest of society. So like you're not allowed to use it without permission or or whatever else. And in this case you don't actually need to keep the weight private uh which is which is great. Uh sim similarly if you're using a model like

llama deepseeek uh Kimmy mo these are all open models so you actually don't need to keep the model weights private >> there it's just more about like verifiability right you're trying to prove that it's correct not that it's private >> yeah yeah >> yeah okay >> uh and and another thing you know that is actually kind of cool is that if you fine-tune using a technique like Laura from a from a model like llama you can keep the Laura weights private and then the llama be public and you can actually do much faster uh proofs of inference that way. Um so you know it really depends on the application and what's considered private and what's not.

>> Nice. Now if somebody's listening to this episode and wants to get involved or use this tool. You mentioned that by the time of airing it should be released but how do they get in touch and how do they sort of participate in this? >> Yeah that's a great question. Uh we have a telegram group that we uh are going to to have for uh users and potential developers of the system. Uh we are always looking for uh people to help improve our systems. Uh there's a bunch of of work that could be done to improving the usability and also the performance uh of these systems. Uh one simple thing is that we actually haven't incorporated GPU proving due to some specific technical details. And if someone wants to get involved, I'm happy to chat about that uh in in depth. Uh, and we'll be we'll send the links to all

these uh to Anna for to put in the show notes. >> Yeah, we'll add those to the show notes. Are you going to be building this as a company as like a for-profit product or is this still sort of in the research realm? I'm just kind of curious if someone if like folks coming in, should they try to build something with it? >> Yeah, I I strongly encourage people to build things with it. Uh >> like even startups like build a startup around this or something like that. >> Yeah, that that would be great. Uh the more the merrier. uh we I I'm a academic I'm a professor and as part of the the university I strongly believe in open source uh and making our technology available for anyone and everyone to use.

>> So I I forgot the specific license is either MIT or Apache license that anyone can use it >> and I I highly encourage people to to build on this and improve it. So so yeah, I want to see it, you know, thousand flowers bloom. >> Very cool. Daniel, thank you so much for coming back on the show, sharing with us an update on kind of all things ZK and ML and also introducing us to ZK Torch. >> Thank you so much for having me >> and again I'll add the link to the talk that you gave at ZK Summit for kind of extra reference. Um, I found that really helpful in sort of prepping for this and I think it gives some visuals that explain what we talked about on the show as well.

>> Cool. Thanks so much. >> Thank you. I want to say a big thank you to the podcast team, Rachel, Henrik, Tanya, and Kai. And to our listeners, thanks for listening. [Music]