ZK11: SNARK proving ASICs - Justin Drake

Transcript

[Music] all right I'm very excited to present our first speaker on the main stage for Zeki Summit 11 this is Justin Drake who's going to be talking about snark proving as6 please take it away thank you Anna okay ZK as6 and I have some props here which I'll be talking about uh during the talk so just a a quick note a personal note um I went to the speaker dinner yesterday and the quality of the people there was just so high that I wanted to add this slide that I'm feeling imposter syndrome because I left the ZK space uh

for about a year doing me and then I I'm coming back and I'm just overwhelmed with the amount of of development anyway um my talk will be in three parts uh first of all I want to provide some motivations and and reasons why I'm excited about SN proving A6 or ZK A6 a lot of it has to do with composability and sharding and me I'll try and explain that to you um the other uh second part is going to be trying to give you an overview a survey of the progress that is happening there's um there's so much progress happening there's about 10 companies that are working on the zk6 and I want to tell you um what what they're doing and then finally I want to share some some

personal opinions on you know how how a ZK Asic you know could be built one that that you know could fulfill the the the the properties that I'd like to see uh out there okay so motivation for uh ZKA A6 so real time proving is specifically focusing on the latency aspects of SN so we want our SNS to come with low latency relative to the native execution and specifically we want the proof latency to be lower ideally significantly lower than the slot uh time which is about 12 seconds and you know I think we we can get to 10 seconds relatively soon but it

would be much better if we could get to 1 second or even 100 milliseconds uh proof latencies now one of the first use cases of realtime proving is this idea of cross rollup synchronous composability so that's a bit of a mouthful but basically every roll up today on ethereum is a little bit of a silo uh and we're breaking down some of the network effects we're introducing fragmentation and ideally what we want to do is reintroduce the full power of composability which is known as synchronous composability um and so if you imagine these rollups um as as being sequenced in time starting from the top to the bottom what we can do is we can do a little bit of

magic um and and try and at least for a subset of the time try and have these these rollups come to together and and and compose now if we zoom in to two of these rollups uh rollup a and rollup B for example arbitrum and base the vision is that we can create these super transactions so a super transaction is a transaction that touches multiple rollups and so here you have three constituent simple transactions that only Touch One rollup and you have basically realtime settlement thanks to the realtime proving of these rollups and that gives you real time trustless message passing between the rabs and also asset bridging so if you want to do

all in one fail swoop as an atomic transaction you know a swap of a token for for for some cash then you can you can do that but in order to to achieve that kind of vision we actually need two things one uh we need shed sequencing so we need these two rollups A and B to opt in to using this same sequencer but also extremely importantly and this is the second pillar we need this real time proof and where latency comes into play is that it basically limits the opportunity for synchronous composability so here in this diagram if we we have for example 1 second of proving latency at the bottom and our slot duration is 12 seconds well our window for synchrony our opportunity

for for synchrony is only 11 seconds and the reason is that as a sequencer that is also a builder as a builder Builder sequencer um I have to put my my proof within the block and if it takes one second uh to generate the proof well I have to start the proving process at least 1 second before the end of the slot and so I can only do this synchronous stuff and synchronous magic in the first 11 seconds okay the second big use case of real time proving is that uh we can have light validators or ZK validators so right now we have uh big nodes and and small nodes um so big nodes can be like Builders and provs and relays and we

want users to be on the on their mobile phones and be like no but we have this awkward middle kind of node medium nodes which are the validators that are currently running on on laptops uh or you know if you're lucky like a rasby pi uh but really what we want to do is to move these validators into the category of light nodes but in order to do that um we need to have real improving and the reason is that whenever there's a new block on the eum chain that comes in we need validators to know with low latency at least before the next block arrives that this block is valid and so this is where real time proving where the latencies are less than the slot duration become

important and so what I think will happen is that one by one the the clients will SN ify so maybe gef will become ZK gef and then we'll have B and ZK B and ref ZK ref and same thing for the consensus client and then the final big unlock is this idea of having an evm pre-compile to do evm execution or evm verification within the evm so it's kind of this introspection very powerful uh pre-compile and the way it works is that it takes a few inputs it takes a pre-state route a postate route a state diff and some proofs and basically it will turn true or false depending as to

whether or not the state transition is valid and the proofs do indeed prove that the state transition invalid for the corresponding uh State diff um and one of the important uh details here is that some of the aspects of the some of the inputs of the pre-compile are not going on chain they actually implicit they offchain and the idea here is that we don't want to enshrine within ethereum a specific proof system and so instead every execution client can have its own proof system with its own circuit for the zkm for its own um arithmetization or or or or or or cryptographic

assumptions um and uh each validator is basically running this uh this up code with the logic that they have in in in their client and um this is kind of uh highlighted in vitalic um road map diagram uh as um evm verification precompile and once you have this this this kind of magical precompile um we we will be in in a in a position where uh we can have a notion of a native rollup so a native rollup is one that natively reuses the evm as his virtual machine so today as a roller

even if you want to be evm equivalent you have to reimplement the evm you know as a circuit you'll introduce bugs but not only that you won't be compatible all the time because evm itself is changing and so you need a mechanism to change as well and so you need governance and that in end of itself is also another attack Vector so once we have this pre-compile it's going to be you know effectively one line of code uh to deploy a new uh evm equivalent uh rollup it's going to be extremely easy to program and um create these these new rollups another cool thing is that by construction the the virtual machine will not have bugs and the reason is

that it it it will be the exact copy of the evm that is used at layer one within consensus it's also going to be uh enjoying uh what I call governance synchrony so every time there's a hard Fork well the evm will automatically also update and have the exact same rues so every time we add an OP code for example within the layer 1 evm this pre-compile will reflect that logic immediately and then another really cool uh use case is that we can have rollups that remove the gas limit no more gas limit how cool would that be and the reason is that the gas limit really is

uh an anti- denal of service Vector where we more make sure that the verifiers the validators um don't don't get D dos with very large blocks that take a very large time to to to verify but with a SN you know it's you have this constant time verification on the order of one of one millisecond and so you can have the gas limit be uh as high as you want and you can even remove it and once we have this notion of native rollups we actually reintroduce uh and kind of fulfill this this vision of uh Native execution Shing um so uh anyone will be able to deploy evm shards on on on

ethereum um and and and this is basically the the best type of of of Shing that that that I know of okay quick summary realtime proving unlocks a lot of things it unlocks realtime settlement for rollups and therefore synchronous composability it gives us um ZK client or SN clients which gives us light validators and then it also gives us this evm and evm pre-compile which gives us uh Native rollups and my last slide of this first section basically tries to to highlight that um there there are some incentives for the builders to use you know fancy

Hardware to do the proving in order to extract more me and that is kind of a nice incentive alignment for people to go do the R&D around this so in the case of synchronous composability the more proven capabilities you have the lower latency you have the more transactions you are going to be able to include on chain those transactions that require synchronous composability with like clients you're going to be in a position where um you'll have more time to um to build blocks if you can do the proving with low latency right because if you have a high overhead for the proving that means that you need to start the proving process you need to seal your block and start the proving

process early and therefore you'll have less time to include transactions in your blocks and then finally if you have a very very powerful uh proving and you're in a in in a rollup that does not have a gas limit then now suddenly your bottleneck is just physics like how fast can you um can can can you run these CPUs how fast can you run the the the prover and basically the the bigger and the beefier uh setup that you have the more transactions you'll be able to stuff in your blocks uh and all of that is more me and uh and therefore is an incentive to go fulfill this this vision of low low latency

proving okay part two um ZK as progress so so I want to give you an overview of these 10 companies um that are working on snar as6 the first three are kind of the flagship Flagship companies I'd say right now which are Axel cisc and Fa Fabric and they are companies whose only Mission whose sole purpose is to build snar A6 and they're very vocal and public about it and they're making a lot of progress the second category of companies uh you know somewhat unexpected is basically mining companies

so Alo will uh will have uh a proof of work where the the proof of work is effectively um you know proving SNS and so you have these four companies kind of through the door of proof of work are U coming into the world of SN proving as6 and then you have three companies at the bottom here that I that I want to mention mention because they're extremely uh you know strong technically some of these are extremely well funded um and um they they have been hints of them also wanting to build a s A6 so for example oradine told me that yes they are building a you know an asake for a certain ZK protocol but they

won't tell me the details um we have inyama who who wants to build what they call a zpu uh but they they haven't started work yet it's still too early for them and then you have uh supranational which is the team that built uh these vdf A6 here uh and by the way if uh this is for giveaway so um come talk to me if you if you want one of those at the end um they also might be working on the snar Asic but they don't want to tell me uh many details okay so let me focus on the flagship companies the first one is axial so I'm very uh proud to to be able to show you today that we have a first

snar Asic and this is this is what it looks like um your more and welcome to come inspect it and have a look at it uh I have another another chip here and really I think um Axel has done a great job at just proving to the world that yes we can build these things you know is not rocket science um it's it's it's something that's totally to totally doable and um this this chip you know is a fairly fancy chip right it's a um it's a 12 nanometer tsmc chip uh with die area roughly 100 square mm and what it does is basically acceleration of certain

workloads so it will accelerate msms and NTTS and it will also do finite field Vector additions and and multiplications and it's programmable to an extent so the uh it's made out of these 384-bit modular multipliers and you can basically program the prime and program your Fields but you can also program the Curves in the context of msms so for example it supports b254 it supports BS 381 it supports the pasta curves and um they have a test board so uh in the next slide I'll show some of the performance numbers these are real measured performance numbers so this is

a PCI board they've put four chips on it and 64 GB of memory this is real stuff that you should be able to buy you know hopefully relatively soon and these are the performance numbers um I really want to just highlight the the two in green there which basically says that this chip is capable of of it has the performance of a 380 TI uh G GPU uh but obviously yeah it's much much smaller it costs roughly five times less so your upfront cost is roughly five times less and the power consumption is also five times less so your all- in all cost is going to be

five times less it consumes much less power it has a much smaller footprint and this is using you know a a compared to the the the 380 TPI is using an older node and obviously Nvidia is this you know trillion dollar company and you have this this startup you know with with X academics where in their very first iteration they're able to get you know close to an order of magnitude on Improvement on several metrics so I'm you know very impressed and very optimistic uh by what's yet to come in in in in in this direction now axal is only one of the three um you know companies in the first category the other two are cisc and

fabric and you know you could kind of think of them as um you know varying in ambition in what uh what they're trying to build for their very first chip so CC unlike axal which is doing like this partial SN acceleration only the the the heavy duty msms and NTS they they want to do the whole end to endend uh proving because one of the realizations is that it's all well and good to accelerate the entities and the ffts but then you're left with all the other stuff and that quickly becomes your bottleneck uh which is why cisc is trying to do the whole thing and then fabric is trying to do something even more ambitious uh which is basically to build a a GPU for cryptography a GPU where the the the

basic operations are are finite field operations uh as opposed to do floating Point operations or or integer operations uh that are used for for AI and this is this the the cisc uh test board so right now they they they're working with fpgas and this is you know a normal process uh a normal step before going to A6 so they put three fjs per board um and uh this is the the performance that they have so they connect 18 boards together each board has three fpj so there's 54 fpgas and they can do an MSM of size 1 billion so that's an extremely massive MSM in less

than 200 milliseconds so in less than 1 of a second they can do an MSM of size a billion which I find absolutely amazing and they have similar performance numbers for entities as well uh they can do an entity of size 2 to the 30 in roughly 200 milliseconds and this is including this is end to end including the data transfer time very very impressive numbers and basically what they want to do is like take these boards and actually take three boards with nine nine fpgas and compress compress down that compress that down to a single Asic of of uh of D area roughly 100 square mm okay um now the second category of

companies that I wanted to highlight is the proof of workare companies because we're going to have Alo launching relatively soon and I kind of think of Alo as being this this amazing Trojan Horse to get a bunch of you know profit maximizing uh mining companies or in into the world uh of of of as6 through the through the back door um and the way that I think of Alo is as this continuous Z priz instead of having Z priz you know once a year with let's say $10 million of prizes we're going to have this constant block by block incentive program and it might be an order of magnitude larger than than Z prise which is extremely exciting now what Alo has um decided to do is to change the uh proof of work

puzzle uh when they launch and so right now all the proof of work puzzles on test net don't really apply and so all the companies are in this like wait and see mode uh and when they when they will release the the specs of their final puzzle all these uh companies will be uh engaging in this you know roughly six months race to try and and design and tape out uh that chip uh as fast as possible because proof of work is all about racing uh well part of it is about racing to to to having the first one in addition to having the best the best performance and I think roughly speaking this is going to double the number of companies that are going to be actively

working on on on ZK as6 and you can see here the test Board of of one one of these companies called uh zk2 okay final part some of I some ideas some opinions uh that I have uh about ZKA A6 and and how they they could be designed um you know take this last section with a grain of salt um but you know hopefully there's there's like some some good ideas there so the first idea the first opinion is that really um the the witness ought to be on chip and stay on chip um so right we have this traditional separation of front end and

back end in in improv system and and basically we have the front end generating the witness and we have the back end consuming the witness and the witness being at the at the boundary here and this witness these Witnesses are enormous they're huge and so if you want to simplify your design and not have this massive uh IO uh to the chip you really want to put everything on chip so that's idea number one just put both the front end and the back end so that includes the w generation on chip and so now basically what comes in is the statement you know which uh is is is not as large as the witness is and then you have this small

proof coming out okay idea number two try and have bite-size chunks in your workloads so there's some designs out there that have basically tried to have these massive statements some some teams build ZK VMS end up with statements so large that they need enormous amounts of memory and this makes the process of creating hardware for it just way like unnecessarily hard and so instead why don't we have these small bite siiz chunks and probably like the most elegant approach here is to have a virtual machine where you know you what comes in is bite codes and now you have

this notion of a cycle this is your most granular you know Atomic unit of computation and can you can batch these these Cycles so you can have chunks of a thousand Cycles or chunks of a million Cycles um and the nice thing here is that you don't need much memory um and then in order to support this this idea of having small chunks well you need a notion of recursion or a notion of of proof carrying data or of continuations and the nice thing about continuations is that not only will this give you the internal recursion to the chip that you need to be able to consume um consume these chunks made of Cycles but you also have the opportunity to

scale horizontally so you could have you know 10 chips 100 chips all of which are spitting out continuations and then you do the agregation on top of that so it's a really nice design to be working with these bite-sized chunks and my my prediction is that these as6 actually won't have much memory um and and and everything will be very digestible okay idea number three um is to try and embrace off chip programmability so programming Hardware is always of magnitude harder than programming software and so wherever There's an opportunity to program in software Embrace that and so

um one of the obvious places here if you have bite code is just use a you know a general purpose to incomplete virtual machine and now you can have arbitr complex um high level programs at the user level but then there's another opportunity for programmability which is at the backend uh level where basically you can have this notion of a wrapper SN right so even though your continuation might use a very opinionated proof system or opinionated curves you can always abstract that way and and translate it to your to your environment so so for example if you're in the context of ethereum you might want to have a gr 16

wrap so wrap is my my my term for wrapper proof um and the reason you might like gr 16 is because it it it consumes very little gas uh when you go verify it uh on on etherium um but the the the point here that I'm trying to make is that really you want to do as little programming as possible on on the Asic and and Leverage as much as possible um of of Asic and so one of the things you can do is you can basically totally freeze the internals and and fix them which is something a little counterintuitive but it's basically just be very opinionated on your chip choose your specific field choose your specific proof system your

specific um you know risk five Asic and you'll get all sorts of gains uh by just having better utilization and and uh and lower power consumption because you're so specialized at the end of the day A6 are all about application specific circuits and and the best way to do that is in some sense to limit the programmability on on chip and have it all off chip now one of the the things you may ask okay isn't it really risky to just choose a specific uh for example risk five circuit and just you know bet everything on that um and the answer is yes and this is why you know you want to

make sure that whatever you bake into your your your your chip is you know as close as possible to bug free and this problem is is a problem that pervades the whole industry know there could be bugs in ZK rollups and that could lead to multi-billion dollar hacks there could be bugs in ZK clients and that could lead to consensus failures on eum and so this is why you know at the foundation we've started initial steps towards a very large scale formal verification company uh sorry formal verification uh competition with tens of millions of dollars in prizes and so if you're interested in helping out with this formal verification effort which will be relevant for ZKA as6 uh do get in touch with me okay final slide before

maybe um we can have a couple questions this is like back of the envelope numbers on overheads so today if you have like kind of state-ofthe-art ZK ZK VM you have an overhead relative you have a proving overhead relative to the native execution of roughly a million x um so if you have like one risk five cycle you're going to need a million Cycles in order to do the proving and you know there's like all these optimizations that are coming uh soon which should bring that down that overhead to you only 100,000 and with an Asic like bullar figure back of the envelope relative to

a CP you can get 100x so the overhead will only be 1,000x and so one of the things that might actually be possible when you combine all these ideas is that basically you have this chip where the native execution is happening in red it's a risk five core which only consumes .1 M Square mm of die area and then all the rest is dedicated to the proving and then you have this amazing ZK CPU where in real time you have for every computation that's done in the little red area you have a proof as a continuation that comes out with extremely low latency and that's it thank

[Applause] you okay I think we have a couple minutes for Q&A we don't have much but we have some microphones on the seats so but just raise your hand and we can okay you sir there's a microphone on the seat some can we turn the microphones on ah there we go uh I'm curious uh we put gpus on phones now right uh and this was all about Builders proving but you see room for users proving things and then Builders building on that right so I guess the the first step

will be for sophisticated entities to be running this it could be a Builders but it could it could also be um you know other entities in data centers running these these systems but yeah as you said I expect as the as the technology commoditized that maybe Intel or AMD will acquire one of these companies and then eventually include one of these real-time proving modules on their chips that would be a fantastic outcome you know realistically it's maybe you know half a decade or a decade away but yes that would be that would be fantastic um and then there's also this notion of the the zpu or the vpu from fabric um where you have this the equivalent of a of a GPU engine which is extremely

programmable but specialized definite fields that could be an also another approach for uh putting in in Intel CPUs uh but that that's going to take a long time presumably think we have time for one more question just give it a give it a second it should turn on than inspiring presentation Justin that was awesome um what do you think just in terms of like timelines before uh users are able to verify zkps on their phones so my mental model is that the the real use case for proving on your phone is is is privacy and the good news

with privacy is that you tend to have like small small statements relative to the scaling applications where you want to be keeping up with a whole a whole rollup so I'm actually optimistic that uh for privacy applications we can just use the phone as it is um but yeah it it it does feel like like we are early and [Music] um yeah the the the good news with privacy is that maybe there's like this these these fairly cheap Transformations where um you can use your GPU on your phone you translate the statement you kind of obfuscate it and you um you you remove all the sensitive information and then what you send you

can send that to to a data center that will do the compress the actual compression and SN ification for you so I think what the phone needs to do is like the ZK part but not the the S part the sixin part yeah I think we're out of questions sorry Uma but we love to chat uh later on and um I will be in the the the the the speakers room over there if you want to come chat thank you speak