Episode 266: ZK Hardware Sessions with Zprize Pt. 1

Transcript

[Music] welcome to zero knowledge I'm your host Anna Rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online we have the first part of our two-part series on ZK Hardware we do a recap of the Z price competition and chat with a number of different participants from an architect of the prizes to the hardware manufacturers who supported it as well as catch up with a few of the

competitors my guest co-host for these episodes is Alex Pruden the CEO of elio and the creator of zprice and before kick off I just want to let you know about the ZK jobs board there you can find job opportunities from Top teams working in ZK I've added the link in the show notes now Tanya will share a little bit about this week's sponsor [Music] ever wish you could use existing rust libraries in ZK a friendly reminder from the team at risk zero that you can things like proof of Checkmate blinded proof of password complexity and even proving things about protobufs are straightforward when you can use existing code to learn more check out the risk zero video tutorials from the latest ZK hack at youtube.com forward slash at risk zero that's

r-i-s-c-z-e-r-o you can also follow them on Twitter at risk zero to make sure you don't miss their upcoming 1.0 launch and the alpha launch of the Bonsai Network so thanks again risk zero and now here's our episode [Music] so for this episode we are going to be exploring Hardware in the ZK context meeting folks who've worked on hardware and will be using the recent zprize competition as sort of the lens through which we explore it with me is Alex Pruden the CEO of elio and creator of zprize Alex is going to be the co-host and will help guide us through this hey Alex hey Anna it's a real pleasure to be here uh we actually did an interview I think over a year ago where we

introduced you came on the show and you introduced zprize it was in the context at the time of like General ZK funding so I'll add a link to that in the show notes if people want to hear kind of like what it was but yeah tell us a little bit about what is the prize today yeah so zprize um I just quick context for those who may not have heard of it so zprize was a competition modeled on other examples like the DARPA Grand Challenge where DARPA the DARPA Grand Challenge was you know largely credited with kicking off the self-driving car industry and as I was you know as a person who's in the the industry as Anna mentioned I'm the CEO of elio which is building a ZK layer one being in the industry you know you see all these techniques that are being

applied inside of different companies and you know being someone who's very excited of course about what I'm building at a Leo but also just about ZK in general I really wanted there to be an organization or an outlet for people who had the skills and the desires to contribute to basically building up the foundation for zero knowledge that everyone could then build upon and use in the future right so basically an open source public good a library of Open Source public goods and the key reason why we wanted to do a competition was we thought it would bring out the best in everyone right and so we we got sponsors across the industry a couple of them you'll hear from during this show but you know lots of folks from web3

zprize.io you can see the full list but you know Alia was a sponsor Aztec was a sponsor Minna lots of big web three companies in ZK we also had Hardware companies such as AMD provide specialized hardware for competitors we really brought together folks from across web 3 and web 2 really to focus on promoting the development of these open source public goods for ZK I really liked the sort of network agnostic nature of it where you had even like people who kind of cheating on some level collaborating in a way I mean they're still competing but at the end once everything was revealed it is a collaboration right because they could almost share results and like find out what the other people were doing this

was meant to be kind of an open source competition not something closed yeah exactly it was a really interesting Dynamic right and there's there's a few different levels of people who participated in the competition there were sponsors right and sponsors are putting up the prize money right and so they obviously have a vested interest in outcomes that are at least relevant to their projects right potentially um you know that you have architects who are appointed by the sponsors or are the sponsors themselves that are really about specifying the prizes and that's really important in a competition right because there's this you know you really need to make sure that everything is fair and so it needs to be relatively rigorously defined up front and you know a lot of these problems in this whole area is pretty is moving pretty fast um and then the last kind of you know

angle for participation was the competitors themselves so these are people many of whom and again you'll hear from some in this show many of of them have very little background in crypto or ZK but they actually just saw this as an opportunity to say hey I can apply these skills or this knowledge that I've gained through my education or through my prior work at you know other companies in software hardware and potentially win a big prize right and so that was like kind of the third the third angle there an ultimate nerd snipe I like it kind of yeah but but I think to get back to like the motivation behind your question I think if you look at the competition the prizes right so if you just read off a title it's like accelerating multi-scaler multiplication in a GPU like you know listeners of your

show will be like okay I know what Planck is I know what Marlin is I know across16 is like these are proof systems and they're very high level but they're also very kind of many of these things are kind of specific to different projects and multi-scaler multiplication is one of these low level algorithms that kind of applies across the board right that was that was the reason why we chose to focus on the lower level stuff was because everybody found common ground is like hey if this gets faster on this platform it kind of helps everyone or it's widely applicable so nice can you tell me what were the price categories I know we're not going to talk about necessarily all of them in this compilation of interviews but we will talk about a few but tell us what were all of them so there were a total

of 12 different prize categories split up into what we call two divisions so one was this open division which was really focused on public goods that you know applied to more than one team and the prizes in the quote-unquote open division were accelerating multi-scalar multiplication on gpus and fpga so that was kind of one prize we later split it into two but uh accelerated multi-scale multiplication on gpus and fpgas accelerating NTT on fpga specifically an NTT stands for number theoretic transforms which are you know really important specifically for Stark based systems a Planck d-i-z-k GPU acceleration so d-i-z-k dizzy not to be confused with the company dzk dick is a

technique where you can parallelize the the generation of a huge proof and and so we have that as a category accelerating elliptic curve operations and finite field arithmetic in wasm which most of that was focused on MSM uh similar to the GPU accelerating Planck up which several projects in the space are using specifically those are as part of ZK garage accelerating the Poseidon hash function and accelerating MSM on mobile and that was really cool I just quick quick note on that that was to my knowledge the first time anyone has kind of made any or developed any open source work around proving on a mobile phone so um really interesting there and then the team division category was really about okay you're a company you want to just define basically like a bounty so Elio had three that's specific very much to

alios there were two team division prizes focusing on proof of succinct work which is an algorithm that um that we use as part of our protocol to distribute coinbase Rewards uh we had a fast verifier from uh for Marlin and then the two other team division prizes uh one was sponsored by minute one was sponsored by Noma the Menace sponsored one was accelerating Halo 2 and the one sponsored by Noma was uh creating a the fastest multi-asset shielded pool circuit using gross16. nice and we were really thankful to have a great set of Partners across web 3 and web 2 for the Z prize I mean these include for sponsors and providers of you know folks who funded the prizes this is Elio anoma parody jump uh ocelot which is um you know associated with cello ecosystem

polygon Minna Panther protocol Aztec espresso systems polychain Capital Manta Network and the ZK validator oh yeah uh yeah exactly and many more and also I want to mention the three technology providers we had AMD Who provided fpgas for the fpga competitions Samsung Who provided mobile phones for the mobile MSM Division and then core weave is a specialized cloud service provider that does a lot in the AI space but but also let us use some of their Cloud resources to enable the competition nice so yeah this episode as mentioned it's it's kind of this look into Hardware we're using zprize as sort of the context tell us a little bit about who

we're going to hear from yeah so over the course of this episode we're going to talk to Luke Pearson of polychain capital who worked as an architect on a couple of the prizes and was was really helped me kind of brainstorm the initial idea at the beginning we're going to talk to one of the competitors they go by the team name hardcamel they won the fpga MSM acceleration prize specifically and we're going to hear from them about fpgas continuing on the fpga theme we're going to hear from AMD xilinx uh they're a manufacturer and inventor of the fpga devices that the competitors use and they're they're a leader in that in that category and then we're going to hear from another competitor not in the fpga division but in you know focusing on

Hardware people might be a little more familiar with gpus so Graphics processing units and uh and yep that was an independent competitor who actually won two competitions and so we'll get we'll get kind of a flavor with the competition was like but I guess I just wanted to quickly say that this was really something that I I came up with initially but really the the result of it which was very successful by the way everything across all of the prize categories the average Improvement on the existing Baseline was around 10x right and all of this work is open source and that would not have been possible without many the many many people who contributed to this so as everyone we listed in the beginning of this you know as we were just discussing the sponsors and Technology providers

but all of the competitors all of the people who contributed ideas or works as part of this competition I mean it really is something that the community did together and that's something that I was really proud to to be a part of cool so we should jump into it but before we do if there are listeners who are interested in this like how can people get involved yeah it's a great question and you know we just finished the first the inaugural zprize competition but this is something that we hope to continue and and to to again make a source of public goods going forward and so zprize.io is uh the website where you can go it has all the information about the competition there's also links to our Discord and we have a Discord Community there where would love

feedback from folks around what potentially the next iteration of the competition should look like in particular we're really excited to meet and find folks in the community who are passionate about helping organize and lead the next iteration of this competition I mean again this is a community-led effort but it really does the the efforts and impacts of individuals really do matter so if that's something you know if you're passionate about this space if you're passionate about ZK would really encourage you to join our Discord reach out you know sign up for our email newsletter and get in touch with the team because this is something that you know we want to continue to build together cool so this is a four-part series so we've decided to break it up into two distinct episodes

um so this will be a two-parter and I guess it's a good time to kick it off shall we do it let's go [Music] so we're here with Luke Pearson a partner and research cryptographer at polychain welcome Luke yes thank you for the lovely introduction and thank you for having me Anna Alex and the rest of the big podcast team so Luke you were one of the designers of the zprize competition that's what we want to talk about primarily but I do want to start us off with a little backstory what got you excited about cryptography zero knowledge the whole Jam academically I was studying maths and Engineering at University in the Netherlands in the

University of groaning and north of the Netherlands and then instructed work at dust Network and that Advent came accidentally shall we say first caring a lot more about privacy preserving Technologies so in this case it was taking signatures that are signed in in browsers and with TLs but just something that's done like on a https on this transfer protocol system and has like a sign for an attestation every time you go to a mineral website and then you get your little padlock and it's a big thumbs up from the internet oh big thumbs up from the two hands in that handshake and to do that without leaking a single scrap of metadata the the project the the method should I say I'm looking into it was bulletproofs so

that was like my first kind of Advent or kind of Pierce into zero knowledge cryptography but cryptography itself when I was I used to be really into codes code breaking design encryption and decryption language is even just um like spoken languages but the Linguistics and the structure behind them but when I was a little younger when I was 15 I entered a competition called the Southampton Cipher challenge run by gchq at kind of UK equivalent of of NSA except rather than Pentagon structure don't they they structure um building they have is the donut and basically it was all high school students mostly those who are into maths to do this after school competition and

ended up placing really well ended up being a joint finalist so held it in the back of my mind that I liked this particular area of mathematics and then when going to University and realizing and understanding and reasoning with you know number Theory algebraic geometry better I understood the the broader and wider use case which is very exciting um I didn't do traditional Computer Science Background which had some of its advantages it meant I got not to care about the cleanliness of my code but also I guess that's a disadvantage of many of us but I really enjoyed bridging between you know a lot of the the engineers that had worked around and becoming a much more applied cryptographer and then fell into snarks when the bulletproof wasn't

viable for but what is any any use case really besides specialized that's not a that sounds a bit too disparity actually but any use case that we were doing at dusk other than um very very short specialized range proofs so fell into what's the next thing what's the next stage how are we going to scale this how are we going to make this stuff private and snarks that was around 2019 coincidentally fell in love with Planck when uh Zach presented it at the it was the credit and Deloitte um some kind of school thing in Amsterdam presentation day I can't read the title you get to know that very

exciting ZK day and sort of implemented begun to design around it reason with some of the plankish stuff made my own paper finished my studies and then after that decided to switch gears a little bit and change the seat I satin from one of pure applied cryptography to research and allocation of capital to VC and investment whatever it was necessary it was kind of my hopefully succinct little story you also created a group called uh I'm gonna probably say it wrong ZK garage ZK garage ZK garage ZK garage ZK I I actually wanted to ask you about that like ZK when when you say at ZK garage

it makes me think of UK garage was that the reference no okay I wanted I've been wanting to ask you this for a while okay it was actually after a call with uh Yani Yoni but you know Johnny Johnny Bench him on well we sort of came up with this idea the the idea behind the the garage the the garage I also by the way spending so much time in international communities I just interchange and switch and swap Z and Z with prize and cash and I'm just yeah fine languages for communication um but it was basically for a similar reason that hopefully we'll go on in a few minutes to speak about around what the Zed prize is for

it's I wouldn't say standardization um but I guess to give the little story is that I worked right at the start of my career with polychain uh or really had the fortune of working with five or six different companies it will help to name them uh Elio Manta enoma um a couple were outside I think one was iohk who spoke to a brave and then scroll slash you know privacy scale at dpsc slash ethereum that kind of implementation and being in this more abstract capacity or a bit of an investor and not necessarily tied to one I can see that everyone's working kind of on the same thing here yeah yeah but separately and that was okay that was

also very about a year ago about 12 no 13 14 months ago very crucially when plonkish and punkish constraints were the Talk of the Town it was we have so many of these implementations one was the one I did at doscom was a at um berettenberg so sorry Aztec the C plus plus one there was these ones I used to focus on and and read through and then when I saw all of these different links and then another one plunkish variant we'll call it the Halo 2 first with Z cash and then from the the PSE scroll ethereum Foundation all of that is so many people are putting so many efforts on the exact same thing that roughly to the same goals you will never be able to create this one Library one Library serves all purposes because there are use cases there are gadgets

there are some specifics but there are definitely some ways we can work together to sort of standardize um and make because it was all in Rust with this nice trait system that going make basically Plug and Play of the curves or you know maybe the fields if you're going to do a plunky two the polynomial commitment scheme because the back end was artworks it was originally called um like this is plonk Arc works but it was just too confusing with the name but the back ends Arc works and they've got all these great libraries for fields and arithmetic and they've got kzgt and IPA and like both of those are supported whether the the garage implementation and I then the final thing or the final step was to make as abstract as possible

kind of gate array or should we say you know width in in terms of the numbers of witness wires and a little more complex than that I guess selector polynomials and the columns that represent those so we had all of these different pieces can we work together and make this in Persia and we did and integrated lockups as well which was which was cool um but the the idea behind was to standardize something that everybody could use to a certain degree and then at some point all of those uh protocols and then anyone in the future of course who wishes would Fork it and change it and say how they would but there were definitely some uniform features that everybody was doing and like they

completely made faster safer um and require less work from more people if we do it together I like that idea of the oversight like seeing you know all of these different teams doing similar things and trying to bring them together it actually sort of speaks to the Z price work as well this idea of like like a lot of teams can use some of this stuff they might be thinking about it did you see some sort of parallels when you joined this project I saw yeah even the I saw the exact same problems everything you just said with respect to oversight um but even on a another layer deeper I think each team has themselves where we're like at this case of Zed prize because you know the most of the people have been working on this for the for

the past few years intensely mathematicians or maybe computer scientists less so harder engineers and having an oversight of and in this case we won't see the whole blockchain we won't say an entire blockchain project but the you know let's see the aim to make a ZK VM or they aim to do a fantastic State transition a great speed from a client-side super decentralized in the past um five six years the proving system the iteration of them slowed down right until also coincidental letter that I spoke about in the the introduction of up until plonk and plunkish you know there used to be uh bulletproof's Groth auroras Virgos Marlins Libras plonks pluck-ups Halos oh my God I knew about

it my suspection uh my prediction sorry is that in the the longer run the people who'll be making like quite serious leaps in terms of how a proving system has moved forwards will probably come from academic institutes or research industry that will now pump out array you know the Cadence maybe once a year maybe once every two years not the crazy every other month and that's because we need to get something on the ground we need some Bare Bones working version and I think we sort of have that on the proving system so what's the next thing to look into well it's the hardware you a lot of these proof systems lay out these you know this with the bits that just group elements down onto some piece

of metal whether it be some GPU or some some fpga or whatever it may be and I think most cryptographers kind of go well I'm done here um maybe it is not you know their particular area of expertise or just because it wasn't locked into but when looking at the whole holistic end to end a lot of people's own applied engineering Focus moved on to the hardware and I think it speaks to the oversighters were sort of done relatively done with these nice proving systems or relatively dumb assembly gadgets let's get some optimization in another area cool a saluk you were really instrumental in helping zprize come together right so we you know you and I brainstormed a lot of ideas that

ended up becoming the competition at East Denver in 2022 and you ultimately were an architect for one of the prizes related to Poseidon hashing and then you also helped out on a number of others as well can you give us your perspective uh on how zprize went from an idea to through through the execution to the final results absolutely I think it was a great success um now looking back I think the way in which the different parts of the competition or the different categories of of prizes let's say without them being too specific perfect or or more specific than team and personal was was really really good uh we had created a

method here and I'll I'm going back to the front so I'll go from the definitely the difficulties that it was to get here and trying to suit all of these needs so kind of having a very Hardware based uh approach for a couple other competitions things that were very specifically done for you know number theoretic Transformations on fpgas MSM operations that are done on gpus and fpgas and they were brought together by looking at all of the sponsors the participants or the was then the prospective participants but also what the wider industry is going to need to create some kind of relatively standard as fast as possible implementation or let's say new algorithm for running all of these

operations which are crucial to so many different proving systems and that was very specific to hardware and working another thing that was really great about the Z price and we've got to do this because it was flexible and proposable and we did it with best intentions forwards for all of the people in the industry is people knew or let's take the example of the Poseidon prize I knew that Mina had some certain trick where they could um fold fold away some particular rounds or some Xboxes I knew that the the plancky two guys the the mere guys I think I say mayor because I think it was even when they would call me that they created something around the lines of a full Poseidon hash like in one gate I knew that some specifics with small

Fields have been doing and could we in fact create one general area or one repo where all of these were collated put together and then kind of just a well-documented open source implementation where everybody can go and take advantage of that so that was like one sort of middle size which was fantastic and then the third was application was something like Emma it was something like mobile and wasn't goes into it so it's even a little bit further it's the next stage to making these end-to-end proving systems with all these these end-to-end designs and protocols really viable and it's putting them inside of like a browser lots of people don't know that all the multi-threading issues there was and we're just doing wasn't just in general

when you send off whatever code you've written on wasm to this browser where people want to be making proofs the a Leo trusted setup was a fantastic example it got loads of Attraction because the everyday user was able to run a Google Chrome or a safari and not a command line interface to join a trusted setup like the power of that to bring people into into web 3 and to interact with some of these I think Innocent but standardly quite complex Technologies was incredible and the same with the mobile making I think it was seller who did the architecting of that price because they have the desire for people to be able to sync they have the you know the plumo with the light clients straight to their chain so three different categories

um speaks to some of the most fundamental needs from taking proving systems where they are to making these systems really quite viable uh however creating that was very very different okay yeah so you've given the like the Glorious Finish Line but what was it like to get there it wasn't it was intense you have to try and be as abstract as possible to suit everyone's need but not so abstract that it isn't feasible and you don't you want yet another swathe of you know complex um maybe not well documented but very very helpful for individual use case uh implementations or accelerations or algorithms you want something that

everyone who's using this if it is the technology we so uh praise it to be is able to go pull it Fork it and use it and put an implementation so make something that's uh fast working but also abstract such that enough people are going to work onto it there's no point making something very very specific just to your protocol uh because when you've got to get more people interested and if it's with some particular fields or if it's some really particular Brewing System then statistically less people work on that and I think that's a from speaking to the the other Architects that was a problem they were sort of having and you've got to wrestle basically what your protocol wants and needs with making sure you get something that's helpful

um so that was I'd say that was the largest challenge was making sure we could get it to suit everybody yeah but it worked out quite well I think yeah I'm glad you hit on that and I think you know you mentioned that challenge but I think the one of the reasons why I think this is going to be impactful moving forward is that as you pointed out like a lot of individual teams had done their own little work on the one little thing and you know even though all these techniques were theoretically known there was nowhere with like you're like I want to start a new ZK project like where do I go to just like get the MSM module and there's not it doesn't exist right so this is this was kind of the idea behind this was that like similar to how Arc works is kind of like the

repository for a lot of stuff on the proof system side on the ZK side on the cryptography side it's kind of just one place that everyone can go for these kind of basic building blocks that was the idea to of of the Z prize now I'm curious given in light of everything you just said and your experience with the Z prize and seeing it all the way through to the end um what would you like to see for the competition in future iterations for future durations I think we will take very very great use of making something far more standardized something where you're literally saying to to people who plan on let's say um run rollers or provers for ZK VMS or people planning to do run a Leo nodes or

do some some mining to pick parameters specific Hardware specific metal specific fpga specific gpus won't say A6 for now just because of the potential time and say that this predictably is are the parameters this is the the AWS F1 instance and this is the curve that is going to be used for this blockchain for some amount of time that makes it reasonable to now create you know or finalize the final frontier to to create the final model and say we're going to be using a 377 we're going to be using Marlin we're going to be using AWS and F1 and that's going to be the Elio now whoever can go make that as fast as possible that's going to be the alien implementation so I think

um well I say that as if I didn't start by saying we have three separate categories one word that would I guess pertain to another one which could be an exciting open research one thing that didn't get any submissions partly because it was difficult has been parallel and basically Sami so maybe they could have won the money uh with all of the Hulk and the blue and things that that went on at the time just because it was so flexible making some of these uh prizes Ariel gabazan had posted some open question about reducing the the time complexity of lookups with respect to the storage and the table size and that is just one of the sub questions on the plant cup division no one went for it but a place where people can work on these open problems

I shouldn't be I guess I shouldn't say any specifics of financials of inner workings of protocols but from the way some of the things went in 2021 it's a very little cost for a lot of these treasuries Etc to have some of these problems solved outwards well and also I think it was uh for first and in fact the Poseidon hashing prize was a good example is like you know for that ended up only getting one submission which was quite a good submission but nonetheless it was a a very high Roi activity for the uh for the individual competitor right and so this is maybe a PSA to anyone out there who thinks they have the skills to contribute to this because not only is it a great public good for the community

but also you know again in light of the interest here there's a lot of people willing to you know to to sponsor these prizes and then and potentially could be a good good chance to you know earn some extra money for these otherwise relatively esoteric and specific skills both of those were given very particular hints if people had to scroll through the uh through through the document through the PDF tack MD it was of here's a Poseidon but also his hesitate but also here's an implementation by the espresso guys where they have an x to the five and you pick that as your Alpha to get nice inexpensive Edition chains and oh you're gonna just already save a third constraints and there were quite a

few of those things so it was um to your point before a collation of all of these different points and good money in my opinion I want to hear a little bit about the like the judging once you had those folks competing and had submitted something like I don't know if Luke if you were you part of the judging yeah so what was that like like how was it to do was it very hard was it very obvious who came first who came second third fourth fifth time whatever was it hard to rank folks like some of them were difficult it also um I guess helps to explain we've specified that I just architected the Poseidon and I think one more up there

was is specifically under my name but I did you know have the the real pleasure of working with lots of the other teams so I can speak on on behalf um of how some of the others went some of the competitions especially where you're creating or you're saying run some application not on your own machine but on a machine that will rent or will be provided or be done in some Cloud instance provide standardization but you need to build test vectors and Frameworks lots of these other smaller comp or this one that wasn't you know the one that Elio had had sponsored as the main one was let's say Le Poseidon or the Planck uh accelerating Planck up the competition became very very specific in terms of

how the design was um or ought to be made and how the these different techniques had to be exhumed and as a result the judging had to be rather rigorous okay and also if you're talking let's say in terms of constraints you get the same level of kind of can not confusion I would actually say like met with the same comment and complaint from some of the wider audience there was a definitely a level of ad hoc well we're kind of who this person or this sponsor is paying and this is just going to be the line it would be great to sit here uh around a round table of advanced study and cash out huge test vectors but it was going to take a lot longer so we

can't talk we can talk in terms constraints if we're able to fix what we would like as a proof size fix the the array of some of the gates let's say so there were different challenges for for judging both but overall it went went very well and if I can just add one thing um as Luke mentioned they were both quantitative and the quantitative was very important in terms of judging we wanted to make sure there was a very clear finish line or a bar to clear and then you know people people's performance was very objective but inevitably in problems like this which are quite complex there's always a degree of qualitative assessment as well there's no doubt in my mind that it's true that some of the submissions that did not end up winning might actually be

applicable for problems that that specific prize wasn't about right see that's one of the reasons why we wanted to open source not just the winning submission but all of the submissions is because there you know even though the competition was maybe set up a certain way and the specification was like hey this is the bar you know it could be another uh use case or another problem where you say well you know the third or fourth one took an interesting technique and maybe if I take that and apply it here then this could actually be really interesting and cool so a few episodes ago I had Justin thaler on the show and we had talked about this kind of conflict between Hardware acceleration and some of the sort of software accelerations or protocol accelerations Luke I just wanted to ask you a little

bit about that like I believe you you've co-authored part of the articles that were brought up in this conversation so yeah maybe you can tell us a little bit about that yeah um I had reviewed and worked on a hack him do with Seitz who basically took that tweet stream and I'll try and try and correctly remember the the first instigating uh comment or tweet from Oma or the post and then yeah the kind of Justin's follow-up and then the sizing follow-up and then Benedict jumped in try and be tessellate everything well um but yeah so so Oma was just discussing the difference in um hyperplunk and Planck so as a quicker

overview hyperblanc is about like 40 msms and then 40 of these mles these multilinear extensions and it's got a it's got a sum check in there it's got this because it removes ffts these fast forward Transformations and the argument is there is from Omar is it's very difficult to make make some checks it finally based on Hardware these things have a very annoying Fiat shimmy heuristic hash function inside of them the overview or I guess the the conclusion that that I I were to go through I think there's a lot more work still to be done is that I think he is correct in terms of well he is correct in terms of the expense of running these subjects on the hardware um but Justin spoke about these very

particular optimizations and one comes from if you pick hyperplunk itself one comes from batching these multilinear evaluations and as a result the maximum number of some checks you do is two so it's a it's a it's a constant in terms of how expensive it is uh or it's as fixed if it's going to be two but it's not going to scale forwards and um going going forwards with numbers if you can do these batchings I must say I haven't gone all the way into it so anybody listening please ping and we should uh we should definitely talk about this if I if I find something relatively groundbreaking you can take all of the first operations of a some check so all of the first round of the subject should be like heavily parallelizable so I guess it depends on

what Hardware you're going to go for if you're doing this over fpga maybe you can have some host performing some operation paralyzed at the first feed it to something on the second I don't know but I think there's definitely something in paralyzing the first round of some check there and there's definitely a limit in terms of overall subject expense by doing these batched evaluations so I don't think it's a huge killer but it definitely speaks to and a new conversation of kind of backwards fitting or retroactively making um these zero knowledge premium systems such that they specialize in Hardware you know a nice really nice and easy way to think about it um without even understanding like you

know the pipes of fpgas is just field size you spit out in some proving systems on 256 or some some 128 bit number and it's being laid down you have to have all of these expensive carries or if you were able to do it in a 64-bit field or some some specific field minus minus one you can basically optimize the hardware that already exists and that is what you just said on software versus Hardware you know there's there's lots of ways to optimize Hardware fpgas with RTL and all of these very logs but it's relatively complex it's gonna take a fair bit of time and it's it's new or it's a new work in the in the blockchain industry

but software size it's there's quite a lot you can do straight away and you know that if you're doing like NTTS you've got some something that requires like a billion entity operations it has to be Memory efficient you're just going to use CPU GPU and you don't need to start rewriting all of these gate arrays and circuits and things well dovetailing on that question a little bit because I think you touched on something really important is you know this this concept of Hardware software co-design potentially really like what's the chicken and what's the egg right are is the proof system written for generic hardware and therefore you're optimizing all your choices around what the proof system is doing around the existing Hardware or are you designing the proof system around some custom Hardware from the start right so this was you some

roll-up teams have taken this approach right and actually most recently there's a paper and preprint called gzkp which was um you know basically I think it was the scroll guys who co-authored it where it talks about how you're optimizing a proof system for a GPU specifically so it's really interesting right so and this I think touches a little bit on kind of how hardware and software were designed in in the past where it's like in the Mainframe era everything was together and then in the cloud era you get hardware and things running on CPUs in fact it was it was to the extreme like you had the cloud could give you any resource that you wanted and it kind of you could mix and match everything the software it just exists as a completely independent layer and now it seems like because of the uh the

overhead potentially of computing these proofs we're coming back to an Era where it's very very important that the pipeline is very carefully designed so the question to you then Luke is what do you think the future holds for ZK specifically will we be in a world where everything is designed end to end from the Silicon to the application or will it be driven by you know further advances improved systems that will basically that use existing hardware and there's no Hardware software co-design what are your thoughts well I do not know but I'm also confident that I think well that's something I think that would be most of the people's answers you asked and that's always a good thing um one of the difference now one of the

key things between software and Hardware is one is super expensive so I guess to the the disadvantage of the the argument that we'll be making specialized hardware and going backwards if you get that wrong so if you've spent time um building an A6 supply chain squeezing what you you can from Taiwan and trying to play with this and then the pruning systems massively change you may have wasted a really large amount of capital from and that's you know with anything from the software side you know changes are lovely and then they're all on on your GitHub and it's it's neat and it's very recyclable and very fast so I think for now and everybody's doing everybody's doing this right I feel everybody should do this wait until

there is a greater ossification and something far more clear in what these proving systems final forms will look like how difficult um it will you know how it's going to be to upgrade them potentially change Fields And how that affects some of these bits of Hardware I think the question again with respect to each piece of Hardware is more particular you're not going to be doing too many things on the the CPU GPU side which is good because they're kind of in my mind like the they're like the software Hardware the the hardware that you would you know do some fancy parallelization or you would do something in the kernel of a GPU for msms and you would you can still

probably do that from from my end from the applied cryptography end but what's going on to the other side the fpga is the A6 you're going to need to understand what level of abstraction is is going to be important no point designing pumping out loads of these A6 for some proving systems the whole thing changes you will you will really waste a lot one counteract to that is to make fpgas because yeah they're akin to basically prototyping for some of these Asics and they're much cheaper to work with but you still have some of the same sort of Hardware difficulties so I think no one yet knows if both sides as going forwards software and Hardware can

remain as Dynamic responsive to one another and the way in which the other develops I think we'll we'll iterate forwards my personal prediction a little bit different to what I hope was objective from before is we'll have a lot of GPU accelerated algorithms for now and in the meantime people will work with fpgas the power consumption is a bit of an issue that no one really talks on but the memory stuff is fine and there'll be a prototype basically for the A6 I was a little bit upset one thing if you wonder if this is written somewhere so I can hold myself too but one thing I probably said to you last year Alex was post merged this is going to be great there's going to be all of these free look gpus and we can just get these clusters that

people are experienced in running at scale running parallelize and put some these algorithms and we won't mind each foot will actually make blockchains fast safe and and private but no I think all of the all of the ones I looked into all of these Farms I contacted um even without the video interfaces the gpuser had you need to like replace some bits of the the internal memory you'd have to think about the dram I couldn't really predict what would happen to the like fft um cause over time so I think there was too much difficulty and without even speaking to many of the the hardware providers and the people working on zero knowledge Hardware the fact that they're not taking as much to

these Farms I think Falls in line roughly with what I was what I was thinking that's awesome thank you Luke for uh shedding some light on on kind of your thoughts given your perspective of where you sit in the industry and also your perspective from the seat of an architect and key organizer of the Z price competition really appreciate you being here to share this with us and we really appreciate all the work that you did for zpress yeah thanks Luke you're very welcome it was an absolute pleasure thanks for having me on guys [Music] so I want to welcome to the show Ben and Raul both Hardware engineers at James Street and part of the team hard camels we had some debate before we started on how to exactly call the team but yeah uh

welcome to the show Ben and Raul thanks yeah thanks for having us thank you yeah I want to hear a little bit about where you're coming from what got you interested in this topic Ben why don't we start with you uh yeah so I've been interested in if pgas for um quite a while I did my PhD and if PJ design before I moved to the states and started working at xilinx which is a fpga company and after that I moved to another company but I had some downtime and I actually ended up doing um like a one-year kind of job for ramzikesh where we were kind of looking at accelerating CK snacks um and some kind of verification improving systems around them

um so it kind of is a you know a hobby I got really interested in this kind of space and since the then I've also done some vdf competitions and just general kind of crypto stuff in my free time and maybe in the last three years I've been working at James Street as a hardware engineer but also you know always think you know trying to keep up to date with what's happening in the space especially around Hardware acceleration and you know applying fpgas to this problem Raul what about yourself what got you interested in this field hi yeah um well I guess I did my undergrad in electrical engineering and computer science and I was always pretty interested in sort of the boundary

between software and hardware and performance engineering um yeah and I got pretty into fpgas uh during my time in school and once I graduated I started working as a hardware engineer on fpga acceleration yeah and I guess I started working with Ben and he brought up that this competition was happening and that's how I sort of got into the ZK crypto Hardware acceleration space cool so this prize this this competition kind of brought you into the fold then yeah for sure like specifically the ZK fold yeah definitely nice I have a question like you're both working at Jane Street is there Hardware at Jane Street I don't know how that

like what what's going on over there that you'd be doing Hardware engineering um so I guess we can't really talk too much about the the details sure um but uh yeah like you know if PJs we're using them I think the kind of neat thing for us is that at um Jane Street we use uh this language called hard camel which is how we got our team name um and it's kind of like a if people are familiar with chisel it's like a way of um improving you know how fast you can code or how um efficient you are at writing Hardware design um and so we wanted to have a go uh this for this project so we we wrote it all in um hardcamel and that kind of

overlapped a little bit of you know what we're doing at Jane Street tell me a little bit about the team there's more than just the two of you right yeah so it's um mineral in New York and then there's two more Hardware Engineers actually from Jane Street as well um who are based in London and I think we're all pretty you know enthusiastic about Hardware design um and so that's why everyone was very keen to you know do this in our free time together cool yeah I know that a lot of firms uh you know trading firms that do hft often Leverage Advanced types of Hardware such as fpgas in the course of their operations um I think it's pretty common across the field I like so you're working internally on this stuff you saw zprize what inspired

you to get involved in it Ben it sounds like you were the kind of one who spotted it first yeah I think um because of the work I'd done on zcash for VK snacks um I'd kind of always had one foot you know in the space and was aware of things might you know happening um and so I saw on online um this competition was coming out and I thought it would be really neat to you know have a go accelerating and and maybe also learning a little bit more um I think the work that I did for Z cash was a little bit like a naive implementation just to get something in Hardware online because it wasn't anything open sourced really yeah but then for the Z price we really had a go at like implementing the most efficient algorithm we could think of on the fpga

I mean really like pushing the limits yeah and I I actually had proactively reached out to Ben because I noticed he had done that work it was you know because again one of the motivations for zprize was that to grow the body of Open Source work that was related to Hardware acceleration and Benz is one of the only ones that I could find for fpgas and so that was that was how we initially got connected and very was very happy that him and his team decided to contribute so I know that there there were many categories fpgas was one of them but Alex maybe you can share a little bit like how was hard camel competing and how did they do yeah sure so as you mentioned the Z prize encompassed several different Hardware platforms and the goal of zprize was to you know to

cover the space of different platforms where you might run a ZK prover you know and fpgas I'll let them talk a little bit about fpgas in a minute but you know fpgas as I said are often used for for various specialized applications and the specific problems and the specific prize categories that this team competed in were relating to accelerating what's called multi-scalar multiplication and number theoretic transform I think also some sometimes called Fast Fourier transform so these are two key underlying mathematical operations that you know are kind of being done in the course of uh creating a ZK snark or creating a proof uh you know of a

statement inside of a ckp um so that that's the significance of these algorithms and so the fpga specifically maybe Ben and Rojo do you guys want to share with us in your in your eyes what is the advantage of using an fpga to um to accelerate these algorithms yeah I think uh the fpga kind of occupies a little bit of a niche space where you can do you can pretty much program whatever you want on the fpga it's just the programmable logic cells so you can Implement any function any kind of data path but the problem is the clock frequency that you're working with is a lot slower so you really need to like parallelize what you're trying to do

which is really interesting when you're competing against like a GPU which is a much faster clock uh speed right but a little bit more fixed in the functions it can do so I think it's interesting designing for fpg because you have to kind of architect Your Design to be very parallel make sure you're not um heading up you know like memory bandwidth limits and that kind of thing cool I'm trying to picture like did you really expand on the work you had done previously or was it like you're tackling a very different problem with this competition so the work I'd done before uh we weren't doing proving of the ZK snack it was more of the verifying so it was like the appearing on elliptic curves but we did try and take some of the core components

of what I'd done before and just apply them to the competition and say if we'd done it this way how fast it would it be but we found out that it would we would be totally smashed by the GPU when we saw like The Benchmark come out because it was too slow what we were doing um so like all of us all four of us kind of looked for you know what is the latest algorithms and um then that's when we changed to using pippenges which is pretty much what everyone is doing um and going from there you know we're talking about Hardware but this is a lot of deeply mathematical there's a deeply mathematical aspect because you're taking as Ben just referenced an algorithm called pippengers from an academic paper I think in the 2010s I can't remember but this algorithm was described in a paper and then the

challenge for a team like this is to implement that in the hardware right which is of course as efficiently as you can get as opposed to just writing some code that implements it right and there's you know these it's pretty abstract it's pretty far from the the quote-unquote metal you know this is very it's like very much on the metal and so that's I think one of the interesting challenges and all of this was open source because you know the philosophy here was to make sure that this was available these Innovations were available for everyone to use and then leverage to build upon so as to kind of promote a broader ecosystem uh for zkps but one one question that I had for you guys and you just mentioned I think you just touched on it Ben but you know there's a lot of different flavors

of zero knowledge proof systems right there's we have snarks and we have Starks there's fry base polynomial commitments there's Halo 2 to you once based on some check and you're in your opinion or in your eyes like what what's the competitions that you competed in the Z prize specifically accelerating MSM and NTT where do those where are those most applicable and what ZK proof systems would those most help so I think any way that you've got to do this kind of multi-scalar multiplication uh regardless of kind of the curve that you're doing it on um you'd be able to apply what we've done um I think as a side note I'm not so familiar with other proving systems yeah what I'm as I understand fry based systems so like Starks almost entirely

rely on NTT whereas like Marlin is much much more MSM heavy right this is why we actually had the both competitions and polygon who's building a system that's like Starks it's like snarks and Starks they that's why they sponsored the NTT prize specifically is because like for them they don't really care about the MSM it's not relevant one thing I was going to say too I think maybe that helped our team is that um everyone kind of is has a specialty in different field also like Rahul has done a lot of very good like mathematical work and for me I'm more of just a hardware engineer so I think we're all being able to optimize the algorithm and tweak out you know five ten percent performance was very key to I think us getting first place without that I don't think we would have

um you know Andy and for Yong in London they were also very good at like the NTT side or you know optimizing things on fpga um so I think it was it was good so how did this all turn out how did you actually do on both of these competitions yes so we entered in the two tracks and in the MSM fpga track we won first place and the NTT fpga track we won second place oh nice and by the way I I have to say that they won first place of every competitor in the Z prize for the documentation and we'll add a link to it in the show notes but they're the website that they have um like I said we'll have the link but it's very good and it's actually if anyone's curious to learn more about what NTT or what MSM is uh it's

described there as well as links to their their solution so thank you guys for doing that that was that itself was a great contribution cool that's going to be a great resource oh yeah and Andy and Fu Young in London they really pushed in and put a lot of work into that so that's good yeah Raul tell me a little bit like Ben just mentioned that you optimized the algorithms what are you actually doing yeah I think I approached it like from a couple of different ways um I think the core piece of the MSM sort of design is a large like Point Adder um just to do elliptic Point editions So based on the sort of um problem space that we're in uh where we're streaming in points but we know the bass already uh we could sort of do a bit of pre-computation on the host and

reduce the number of actual multiplies that we do in hardware and that was one of the more expensive operations so reducing the total number of basic multiplies in our design um helped us get this design smaller and clock it at a higher speed I want to just ask a clarification here because you just mentioned pre-processing but I feel like in snark land that has usually like a very specific context is it the same pre-processing or is it a different kind of pre-processing that you mentioned here then maybe you I don't know if you can jump it do you know what I'm talking about something I'm not sure about is so during The Trusted setup you come up with this set of points um Alex is it right so the that's right pre-processing we kind of did for the MSM is equivalent to have if we've done

it where you back then exactly and there's two kinds of MSM actually there's three kinds of God so I've learned so much about this since previous competition but there's so there's fixed base MSM which as Ben just mentioned you know the bases and recall like this is you know the whole what is MSM you're multiplying elliptic curve points and you're really taking them to a power right and so that's that's really what like what the whole point of this algorithm is you know you're taking because that's that's the whole point of you know when you're doing the trusted setup you basically fix the bases and then everyone's input into the setup is their power right and it's their secret power and you keep doing that over and over and eventually it's just tumbled together and no one knows how you got that final output string right but then

from that output string you then use that to construct snarks in the pre-processing mode right so so that's where the fixed base comes in so that once the setup is done that's the pre that's the pre-processing step of the snark and so now the bases are fixed so if you're Computing a proof that is a PR that is a no now that's that's fixed-based variable base MSM is would be in the case where you did not have a trusted reference string at the outset and then you would have to compute the bases the points itself on the Fly and you wouldn't be able to know them in advance of the computation like there would be some dependence effectively on like what the next point was based on the previous one whereas in this case they basically you know all the points in advance and so I think it's who's

getting that you can kind of get ahead of the problem a little in certain ways and be able to kind of make it you can chunk it up in a way that's slightly more advantageous so the tldr is fixed base is really applicable most applicable to snark and improve systems where there's a trusted setup in advance yeah so the uh specific type of MSM problem that we accelerated was a fixed base MSM where all of the points are known ahead of time cool so this is for a pre-processing with trusted setup kind of smart yeah so yeah which proving systems is this most built for like maybe some that we're familiar with so the systems would be like graph 16 yeah Marlin

um Planck but is it like OG Planck not new Planck not planckish or plancky 2 or to be honest I I can't keep up with the with the explosive growth of the Planck ecosystem but the other one to note is um there's um a couple newer proof systems called I think the the latest one that I know of coming on Microsoft research is Spartan so snarks that are built around kzg commitments polynomial commitments typically do more on the MSM side and a little less on the NTT side and um snarks that are that are built around fry like Starks typically rely more on NTT now Spartan is a new proof system that's just come out which only relies on MSM so there's no entity at all which

is kind of interesting because you can imagine optimizing only the MSM and not having to worry at all about this other operation right so in general with Hardware acceleration the more generality like the more the broader the set of things you have to do the harder it is to optimize right to make one thing really good um so those are the systems in which it's most applicable is graph 16 also only relying on the MSM or does it have both graph 16 most like the majority of the computation is the MSM but it still does require some NTTS actually T what is that part of the competition you guys came in second there I also want to hear what you did there yeah so the entity is used as part of speeding up polynomial multiplication

um and so kind of what we're doing on the fpga is just very fast um Fourier transforms um and our approach was just to have as many cores working in parallel as we could that would break down the problem and pretty much get it done as fast and this competition also had an aspect of power and you know how much space on the fpga you were using so we also optimized for those kind of um factors as well when you say parallelization in the hardware context what does that actually mean does it mean like running more machines like I'm just trying to picture what or is it the in the algorithm itself there's parallelization on the same machine yeah maybe you can help me

with that yeah so on the fpj you you basically you have like a sea of logic gates and you can Implement whatever you like um and there's a lot of area so you can basically Implement a lot of circuits doing the same thing but they're happening at the same point of time and so you feed out a huge uh parallel amount of data to all of these cores and they all process results at the same time and then return it back to you so you can kind of get a speed up just by increasing your area um it's like having a CPU of 10 000 cores or something is it all about kind of space optimization then like are you like looking at this one space you have and just like oh we're gonna parallelize over here it might take up a bit more room but it'll be faster and then we're

gonna not parallelize this part of it yeah I'm just kind of wondering how you approach this yeah definitely when you start with this kind of problem you look at the algorithm and you think how can I break it down what parts can run in parallel and then there is definitely a part of the fpga physically you know do I have enough resources how many um dsps which are digital signal processes do I have how many RAM blocks do I have uh you know can I paralyze this 100 times or only 10 times and then you kind of go from there and there's another kind of a strange Quirk you have to look at on the fpga is that because you're actually wiring things up between all these blocks you have to make sure that you can meet timing and there's

also like power constraints and you know so you could paralyze it too much and then your design just won't work or it might work but it will work at one megahertz which is way too slow right um yeah so these are the things you all kind of have to take into account as you're working through the algorithm doing Hardware design cool um I'd love to hear about the outcomes I feel like when looking at some of the Z prize documentation after the fact there was a lot of like increases doubling quadrupling of speeds what happened in the case of the fpga the MSM and the NTT yeah so the fpga competition was a little bit unique both and both the

prize categories of MSM and NTT which these this team competed in it was unique because there wasn't really an existing Baseline to go from and um that was this is one reason I'm really excited about the work that that Rahul and Ben and their team did is because this now for the first time is you have a baseline that other people can bench against okay then I guess the follow-up question is what didn't you do and what could be done in order to further accelerate this do you think or do you think this is the the best we can do for today uh no so we definitely looked at the other um submissions to see you know what were they doing differently and uh there was like a bunch of notes that we were like oh yeah if we'd done this it would have improved our result and likewise we've

thought about this and we're like oh why did we beat them and there's a few points that they didn't do so I think like combination of the you know all the submissions would lead to a result that's maybe you know 10 or 20 better um also one thing to note is like we're putting this on the fpgas that are in the cloud at Amazon um and they're kind of old fpjs so if we you know upgraded it for a newer technology um I'm sure we'd be able to get a boost as well and that's the the fpgas that are on the Amazon Cloud were specifically for the MSM but I believe you guys had the actual Hardware from xilinx for the NTT right and that was the c1100s yep so yeah for the MTT we had the c1100s even then we noticed that we were getting

um bandwidth limited on the pcie bus and the reason is not because of the hardware but just the development environment that was provided doesn't have the you know latest gin pcie so there's definitely like improvements for both in terms of like the the memory bandwidth technology that we want to investigate yeah just touching on um future work and possible directions we thought we could go uh on our documentation website we had a page where we listed several different kinds of ideas and places in the design that we thought could be improved and sped up a bit more maybe something for next time yeah for sure nice all right I want to wrap this up with a question on zprize and competing like this is the first

time at least an our space that something like this was happening what was it like to participate was it fun did you meet people yeah personally for me it was very interesting to learn a lot of new things um and also because it's part of a competition you kind of have this fire under your feet of you know the deadlines coming up yeah we have no idea how other people are doing so it's like we just have to push it as much as we can um you know then the a little bit being nervous as the deadline approached waiting for the results to come out um so yeah it was really a lot of fun was there any hint from like on how other teams were doing did anyone brag before submitting or was it very quiet um so we were in the Discord uh kind of

trying to read into messages read between the lines of what people were asking and saying um so they were definitely a bit of that also after the deadline there were some papers that were released before the official results came out so we could read those and kind of gauge how we had done um yeah but it was a lot of guesswork it was definitely also pretty cool just to afterwards just see how everyone else approached the problems and like kind of read about their approaches and prepare chars nice well thank you so much for the interview and sharing with us a little bit about what it was like what you did how you did yeah thanks for being on yeah thank you guys thank you for having us yeah thanks for having us cool [Music] so that wraps up part one of our

two-parter on ZK hardware and Z price stay tuned for next week and our two next interviews I do want to say a big thank you to the ZK podcast team Henrik Rachel Tanya and Adam and tour listeners thanks for listening