Bindings (workshop)

Transcript

depending on the people. Um, but Justin asked us to to talk about uh you know like a lot of the libraries are written in Rust right now like the cryp the signature library everything. Uh Rust is uh a language that doesn't play well with others. Um not as much as Go, but the problem is Go is a big target as well. So that makes things even worse. Uh so I I was uh thinking the first thing is I could explain maybe a bit what uh what issues uh we we have at least in Zim to to integrate with R libraries. Uh and it's just summarized here like you know I just have very

quick slides but I can get into the details if you have questions. Um and the problem is that um well okay Rust is very slow to compile. That's the that's the problem. And uh that means that when I when we compile zim for the first time what it does it it calls rust to compile all the libraries >> and uh zig itself the compiler zig compiler the go compiler is also quite fast uh but then you have this uh interaction where uh you spend maybe a good 10 minutes downloading everything and compiling it um potentially running out of memory on on more constrained machines.

Um, so I would say for us, Rust is is not exactly uh got sand. Um, and um, what what we're a bit worried about and why did I write compiler version? It's because, um, initially we tried to say, okay, we're going to compile, you know, we have like we support like two ZQVMs, actually two, two and a half. Um, so we compile the bindings for this ZQVM in one library. then the binding for ZKVM in the second library. Then the third ZKVM in the third library and then we have the lip P2P uh thing that we're trying to to get rid of but currently we we use it like we have glue and then I think I'm not forgetting anything. Uh

but what we then we would add the the the what's that called the the signatures and potentially other things on top of that. So that means we need to compile things even uh uh even further. Uh and the problem is when we start doing uh static libraries because we we just want to in terms of release engineering we just want to release one binary that we can put anywhere. Uh we don't want to have multiple shared library libraries. Uh Rust will duplicate every every symbol which means you get symbol uh like duplicate symbols like very easily.

So you can't really have one library per uh binding. So you need to put everything in a single library. Um, and then you start having other problems. Luckily, we don't have them yet, but I'm sure it's going to happen. Is that Rust evolves very fast like the lang the language there's a lot of incompatibilities between the the versions. Um, and I I think the at some point we're going to run into the problem where one binding expects one version of the language and another uh another version that are not compatible like it just doesn't compile. So for us this setup is really unpractical.

Yeah. >> To be noted like on on the cryptographic side of things we are trying to always target the very last version of us. So this can also cause some problems for you >> because like for performance reason and all of this each time there is a new version with my feature for example like in the very last version of R you have nice constant stuff generics where you can put some assertion inside and to gain a very tiny performance stuff we always move to the last version.

>> But when you say the last version do you mean the nightly? No, the last table we >> last table >> last table >> the table is okay. I think the problem >> can you can have problem because I imagine that with the underlying things that we compile they are not all targeting the very last version and so I imagine that this can be also a source of of problem for you. So yeah, just to clarify uh what library are you using besides cryptography? lip >> lip uh and everything like uh cuz we do uh we do ZQVM proving uh so we have a binding to open VM to uh disk is in progress uh we used we still

have powder even though it's no longer a thing >> and uh res zero >> do you have plan to migrate P2P to Z because I yeah >> that's what uh been amongst other people are working on uh But let's say for a devet it's not it's not ready for its prime at the moment. So we need to be able to yeah as soon as we can get rid of the lip stuff or you know uh find any other library that's not rust that would be uh >> you can use n lip >> um to the nest like that. No, no, no, no.

Everybody's laughing, but uh yeah, uh I'm I'm not taking it as a joke. Yeah, that's >> because Nim can use Zig CC now. You type Zig CC and you compile N through Zig. >> Wow. >> Okay. >> Isn't Isn't Zig trying to move away from using like the whole LVM stack? >> Yes. >> So that can be in the future. >> You don't know. Yeah, it's more so. >> So, roughly how many Rust libraries are we talking about here?

>> If Yeah, if you want, I can give you a brief context on on why we did this choice and what we have right now. So, basically like all of the cryptography is much easier to to develop in Rust because we have first PL3 and we have all of the cryptography available even if it is not bronkey 3. We have like a very nice paralization crates ron and all of this. So we developed basically the XMSS signature scheme with Rust as we discussed with GB yesterday. I think that as this is quite a reasonable crate. This is not a problem to to port

that to other languages C, C++, Z, Go, whatever we want because for example I I I have just done the migration for to Python to to write the spec and once you have like the code in Russ to another language but this is reasonable. Then we have everything related to the LVM and that is more complex. Why? because uh we have limb VM but we have like all all of the primitives before that. So we have like the finite fields, we have like the PCS, the hair builder constraint and all of this stuff. So this is more complex and this is also

written in Rust. Um we could also have planned to migrate that once this is stable to other languages. For example, having like a lin VM in in zig in C a bit like this is done with a kro VM where you have one in go you have one in rest and maybe >> those were external people that asked for grant >> maybe this is also an option. I don't know like I have not that much context on this. My recommendation is that right now all of these are not stable. for example like the XMSS library. This is quite stable but I think that there is a session today to talk about like the new encoding scheme and how to render it

more CK friendly. So there is probably been change. So this is probably not the right moment to migrate everything also not of course like the Lin VM because this is not stable at all. So this is not the right moment to try to port that. But at some point maybe it could make sense for other clients like Gom or maybe you in C++ you will find some difficulties to call like the LVM in Russo. >> Okay. Okay. Can I just put an alternative to porting on the table? Um what you can do is put the no mangle attribute um on each externally facing function in the API and put extern C uh and you can write a C header file and

that will result in the Rust functions looking like C and then anything that can call C can in principle call them. >> I mean >> I think that's already done. >> That's more or less already the case. >> But I think this will not solve your big compilation time problem. Well, definitely not the compilation time, but more the duplicate symbols or the the problem of language incompatibilities are still going to be. >> Are you compiling them to a a static library and then merging them into the >> Okay. >> Yeah. >> Yeah. So, I I tried to at one point fix some duplicate symbol thing. uh on the compiler side there is something called uh identical code folding uh ICF that

can scan the code and if they see that the output is the same it merges them >> right >> but uh it's a it's a feature that's um sometimes it's compiling sometime it's not and I never managed to to fix it but uh I only spent like two days trying to to look into that but codes and also they have something called uh GC sections uh function sections and data sections. Yeah. And uh this can help those do garbage collecting etc.

But or maybe you write a custom linker um >> but we have we had that in Valida. >> Yeah. Yeah. >> So we actually had a compare engineer to do a custom linker script to remove duplicates. Wait, is it the linker script or >> you you can do a linker script to to to do this. I think >> you can >> I think yes because when you I don't know with ZCC but we um um LLD and mold uh you can pass a linker script >> and if it's just duplicates >> maybe we can pass a linker script to analyze uh duplicate functions. But wouldn't it be easier just to say we

have one official multilip where we put all the the reference implementations in there. So we only link to one >> yes lip. So we don't have this issue at all. >> For me I think it's uh it's easier to have uh call it rest if and you put uh all the stuff that you depend in uh with a single export. So that would be that would be working for uh you know just XMMS and like maybe lin VM and things like this but in our case we need to rely P2P which means we need to drag the whole thing. Uh I don't think uh I don't think go for example is going to be interested in this. I don't think C++ is going to be interested in this. So we we

have I mean we're not interested in this but we need it. Uh we >> but you can maintain two repo one with just um the cryptography >> right >> uh and one with cryptography plus lip p2p and once you have lip p2p in zig you you uh duplicate the other and once you have the script to do uh cryptography uh having cryptography plus p2p works and if you somehow need another rust library in the future uh it's easy to have them.

>> Um, okay. I'd say my my favorite approach would be to if indeed those linker scripts. So, I've never used linker scripts in this specific uh instance. I'm quite surprised that this is even possible. If it is, it's awesome and we need to use it. Um, cuz I think that would solve the problem. If we can indeed uh do this then I think that's the best way so that we don't have to to >> but isn't that uh a different problem like the duplicate symbols will also happen if we have if we put everything in the same um static library.

>> Uh it well it would not if you put everything in the same static library uh you would still potentially get some problem with LTO uh which is more or less what you described. They're trying to look for locations they can dduplicate. >> Uh, which we also had to disable. I didn't mention it on my slide, but it's it's a fine. >> Um, but uh but yeah, you would not have the duplicate programs because at least a lot of the duplicate symbols are actually the basic Rust um standard library things.

So if you build a single A file that you can merge then you don't have this problem which is already a big uh a big game. Um but before you you get there uh I wanted to you know change my hat from Zim to GU. Uh >> yeah maybe this is very interesting that you give us all of your experience about guess with this kind of problem >> right? Uh the thing is uh like I was saying earlier, Rust doesn't play well with others. Go is even worse at this. Um and go just interacts with C and that's it.

So you can always um get you know shared object file like a dynamic library and and get it to and get go to DL open it and load it in memory and do all the things. Um I don't know about prism but uh as far as g is concerned this is no this is just no >> I managed to compile go with constantine by compiling constantine to a library. So if we do a static library it will work >> right uh a okay yeah that's true if you can use a go which we should be able to uh that works too.

Yeah. Okay. It's just that uh like there's an external builder if you if you want and uh once again talking about yes, we don't really like that because we want to be able to have the control over like you know uh all the all the tool chain. We don't want to build some something in a language we don't quite understand because if we need to debug >> uh what can be done is that um that uh library uh can be re this uh um how do you call that that library hold together that has everything can be compiled uh nightly and distributed put on GitHub as an artifact. So uh once you

build uh what's the name of uh beam get or something you you just download it. So this way you don't have an external compiler just uh fetching resource. >> Yeah but you need to manually fetch it. You can't just do the go like it's going to break a lot of tool of uh of workflows basically that have to be rewritten. And I'm not just talking about yes, it's not that we're lazy. >> Oh, or maybe it's uh it's in g LFS so that when you get clone, it's there.

>> Uh LFS, >> large file, >> large file. >> Large file. >> Oh, so you want some kind of subm module, you mean? >> Yeah. For example, >> it would technically just be committed as a file to the repository, but it's not stored in in the regular, you know, way git stores binaries, but rather in a different approach and then git just knows how to code, right? Yeah. Um, so I'm just saying historically we we I mean this is not the first time we're running into this thing. Uh, we we said no. I would not be able to tell you exactly what was the reason, but it has to do with the fact I mean once again some some external thing we don't quite control. Um, and it's it's all about who who wakes up at 3:00 a.m. to fix things

when uh when you know when things go wrong. Um >> just just a quick question for the multi-lib approach right given that you talked about also having to interface with some of the ekms wouldn't you then also have to uh put code for that interfacing into that multi >> because you will be talking about etc. >> Yes absolutely that makes it very very specific to let's say your use case and not useful to anyone else right because not everyone will kind of interface with those etc. I mean it depends. Uh yeah, I mean indeed because for example we only support like three uh presumably the day re wants to do this since everything is written in Rust although they will probably run into the same language

versioning problem. Uh they will have a lot more so they're even more likely to run into it. Uh for Rivium I don't know uh you know how that's going to work with them but um but yeah I think uh it's going to be the problem. So what we could do is one release of a client per ZVM. So they say like this is the version of Zim that only supports rezero. This is the version of Zim for uh for uh for ZIS blah blah blah. Um but yeah I mean it could be done. It's just not very uh not very practical to to have multiple binaries to juggle to juggle with. Um so

I have another possible solution. Um so there it is possible to take a library um and like object file and add a prefix to every symbol in that. So if you do that that will get rid of all the name conflicts. And if if for example like the Rust portion happens to even be using a different lib C than the uh the the the Go portion or whatever you're linking it with that could still work. >> So I thought about that but how do you force Rust to rename it?

>> Well, you could you could just use a command called obcopy. You can take the compiled library and run it through object copy with this prefix symbols argument. >> Okay. But then when when when you link to that library like then how does the calling code let's say uh like you would also have to patch that to actually >> well you would just need to use the appropriate prefixes when you call as functions >> right >> but I just was a lot of work >> well you you just like in the C header file that you make for the interface you just add that prefix to each name in the header file >> but yeah okay we we can definitely train this within the zig build system that's not a problem calling op copy. Everybody has ops copy on their okay everybody who

is a working compiler has op copy on on their system. Um but at at the same time is it really more complicated is it really much much more simple than uh than implementing a C version of each library. >> I think it's way simpler. It does result in more code size. for now lining VN lin uh uh all the stuff done on the lin site they don't seem that big uh but uh you don't want to reimplement risk zero s1 or openm in another language right >> yes there are two problem there >> yeah linvm doesn't sound that big this is true but behind that you have all of

the primitive that are quite big Because for example when you invol like uh the air builder or like the poson or the fields multiplication of clunky it seems like this is nothing but uh this is really >> this is this is already implemented uh in C and C++ >> because uh uh with zero and sp1 uh are using spark which is implemented by super national uh for their um GPU acceleration.

And there is also the team uh that does tack which is GPU accelerity proof system and they implemented uh they started with elliptic curve and they reimplemented fry etc fully um in C++ though the build system is a mess because well they use basel completely and it's uh yeah it's just a pain but um there is code in C and C++ Uh and apparently we is similar to legitron or something >> some similarities.

Uh so the legal team has also legit but implemented in C++. >> So yeah we want also to make sure that we have the same optimization. For example I don't say that they are not optimized at all but for example we did some neon optimization some CG very like lowle stuff. We just want to make sure that the performance are are the same with I I don't say that this is not possible. Maybe even this is more performance like I don't know >> but I think what's important is that uh for the verifier anyone can run and for the verifier um like maybe they run z etc and for the prover you can just say

and that's the case now with get having the most performant cryptography to other u that yeah you have 20% or 40% per difference uh and we will solve it in the future. >> I I guess the other issue with having like you know ports of the of the crypto libraries to different languages is if you ever want to change it you're now multiplying the work that that involves the testing efforts and you got to make sure all these implementations work with each other that if you generate a proof with one the verifier for all the other ones will verify it.

>> Yes. That's uh something that's always been desired at the EF to have um multiple implementation until we have format verification. Like as long as we don't have verification, it's desire to have client diversity because uh even just recently three weeks ago uh ref had a bug uh on mainet and it stopped uh all proving systems and uh the mainet uh well it started on the block uh on some corner case.

>> So actually having multiple implementation I'm not saying nine uh but uh two three is a plus. Even with FD, I would say that's a plus because with formal verification, you still have the issue of, you know, did I in fact express what I intended to express with the theorems that I proved? And having different expressions of the same thing can help with that. Like similar like in the Apollo mission they would have like uh the coders like every section of code would be written by two engineers who were not looking at each other's work and then they would compare each other's work and if there was any difference not like any difference in behavior but any difference in the code in what they wrote they would talk about

it until they agreed about what was the correct way >> and didn't run also on two chip that were different just in case were thing like the the size of the ship was a room application there. >> But this being said, having worked for for EDS today, you have uh you have different computers, but they're all the same architectures. >> Okay. >> But I think for my gut feeling right now is it probably would make sense to in the short term >> to use the reference implementation because there is still revenue. So, so in the short term, I think we probably

won't find any other way around using Rust, but in the long term, I think porting things to different languages makes a lot of sense. >> So, it's just a question of when we going to be ready to start porting for example. >> Yeah, I think that you told me also yesterday that you have big compilation issue. So, even like on the short term, you told me yesterday that it takes like 20 minutes to compile or something like that, >> right? But um >> the other thing is that if you want GPU acceleration uh you can't use Rust. >> Uh >> anyway >> really >> there is no GPU >> the the so Rust compiles to CUDA but somehow it doesn't work

>> which is why all ZKVM are using C++ for their implementation. >> Interesting. They all have like an exchange mechanism between the rest and the C++. So they have like all of the CPU task in in Rust and everything GPU in C++. >> I see. Uh yeah. So uh okay, that's good to know. >> Zig has GPU support. >> Yes. Yes. So use Zig now please. You know >> it interfaces with C. Does it have types?

>> It does. >> It has strong types. Compile time. Uh evaluation. Uh the as I think they changed it, but the previous one was name. >> You 256 if you want >> memory safety. >> It does. >> Yeah. U 56 natively. >> Uh why not zig? >> Yeah. >> Because I use a >> well compared to rust. Why not zig? Is there a reason? lots lots of library already exist in the bike. It's >> a pain to if you want to reimplement everything.

But the the zig cryptography ecosystem is actually really good because um >> um the guy who wrote uh >> Libian uh he's the cor contributor and he reimplemented all the cryptography there >> and the founder of ashikop he's also a zig fan >> right >> and uh >> yeah I know >> so it's a it's a good language >> it it it's got a lot of uh tailwind. But yes, um you could I mean there would have to be Okay, this is not what we're advocating right here. Uh there's clearly a good reason to do it, but we're not going to to push for that

right now. What we want is to just uh find a solution. And indeed, it's mostly a short-term problem because in the future, we expect to reimplement some of the libraries in Zigg. But Zigg, C, whatever is the is the easiest. Zigg plays really nice with C. So even if C is the target, that's good enough for us. Um but um but what I mean is uh yeah, like we need to make sure that we can actually not be stuck with Rust for Lin VM and all the specific core dependencies. And it's not just once again Zigg, it's also Go. go is going to probably come into into the dance at some point and um and

we're going to have to to to play with that and that's honestly what I'm more worried about. >> In uh NIM, we have a code generator for CUDA and web uh GPU. We can add a code generator for C. [Music] >> Like I said, let's talk it's we can do this. The problem with nim is that every time I try to install it, it's it was super complicated. >> No, but here I would generate the C code and you just uh >> import it.

>> Uh build complexity uh can be managed pretty effectively with Nyx. uh that allows you to have reproducible builds where you can just you know on any system with Nyx installed run Nyx build and uh you'll get essentially the same results you would get on any other machine. Um it also helps with uh build times by having um effective means of caching. >> Um yeah I don't know if you followed the news recently but Nyx has a huge problem. Oh, it's like it's tearing itself apart, right?

>> Oh, the political problem. >> Political problem. >> Might not be the right time to adopt it, >> right? Otherwise, it's awesome, but um but yeah, I'm not I'm currently I'm going to wait for this one to blow over before I spend more time learning all these all these things. >> That's too bad. >> Um but yeah, no, it's it's a great idea at the core. Uh, I hope uh I hope they pre-bail. >> Um, well wait actually. So you would use Nyx to do reproducible builds for what purpose?

>> The A library for example. >> The A library. So everybody could reproduce the the A library. Okay. >> Well, and if if the thing if the things that have that as a dependency are using N then um you know that also makes it easier uh to manage the build time issue. Um because uh you know you you would you would be rebuilding the dependency when you need to pull in a new version but not otherwise. >> Maybe it's a good idea to write like what >> is there is there like alternatives to nyx that do roughly the same thing?

uh reproducible build is um it's a hard problem like uh at first them uh which is a big open source conference in Europe uh you have now reproducible build track since 2023 and they talk about bulker etc but you've always had issues with uh a compiler that put the time step uh as a comment and now you have broken reproducible build >> and also I think next part of the problem is alo like the politics side of things right and I mean there's flakes for example which are experimental for a while now and there's no agreed system to you know uh do the right thing so to speak right

>> and potentially because all that is sourced and yes people are busy with other other uh topics than security at the moment u you you always have the risk of a supply chain attack that's the that's the other problem with >> I think that if we But right now we have all this right. >> Right. Yeah. Um >> this one I think that they are relatively small. >> The one is not using >> restp will evolve because they are trying to

create new protocol as we speak. Yeah. >> So it's not small. >> Okay. So this one is very small. >> NVM is small. >> What should less small like I think but >> it's smaller than be too big. >> Okay. So if we are like a classification this one is the smallest. This one is the second smallest. >> This one is and this one is the biggest. Yeah, I mean there's multiple of them and they're not.

>> So I think that for this one we could reimplement maybe multiple language. So maybe we could have like a a zig version, a C version or a C++ version. Yeah, >> I think uh with Sebastian we could reimplement it in name CJ CUDA with GPU and C uh and the C part can be uh ingested in zinc uh go etc. >> Yeah. Uh go yeah sure. Yeah. No, I think an generator in C should be >> I mean it would be straightforward like the only thing is like given that the code generators so far they all

targeting GPU um >> in theory that doesn't change anything um >> but you know just changes slightly the mental model of some of the some of the stuff but because here you only have like the field after that I just give you the um yeah you just have like the final field and to the procedure. >> We already we already have that. So like to be honest like you with all of the context that there is in the implementation you just have ask code or DBT and it would transfer you in language and then you fine tune everything but that that is not the case for all of them. So for for this for example your

current solution is that you just call the bindings and I imagine compile everything for the zkm wait for rest to compile and all >> exactly >> wouldn't it be possible to have some kind of API socket or something and create that as a separate binary >> uh yeah we could do socket of course um I I've thought about this but the problem is the when you start having multiple binaries you have um I mean it's a distribution problem where you know uh this is not this is not what we want to do but if I guess if we absolutely positively have to we'll manage but uh you know if you copy you

can't just copy the z binary somewhere and expect it to work which is what you can do right now >> uh you need to grab all the side binaries and everything but at least in the case of rezero this is already what happens it goes over a socket to to talk to its uh to its prover. So >> for me it's natural to have the ZKVM proverifier as a a different binary random socket so that you can evolve them separately. >> Yeah. The only thing is that now I would have R0VM my binding thing and then Z that says oh I cannot find the sockets.

>> But why is it already like that for zero? Uh no it's not already like that. >> I thought you just said that for you. >> Right. So with zero what their bind like what their bindings do when you compile your host program as they call it. >> It goes over some socket to connect to a program called R0VM which I don't exactly know what it does but the high level thing is it does it runs your program and R your R five programs. It uh builds a proof it returns the proof >> and that's already an external binary.

But now what I say is if we do what Mammy just said, we have two binaries to handle and then you know if every ZKVM works exactly the same then I have >> editing SP1 I think the part that uh reduces the ST proof to a snark is a different binary for example >> as well I think it is yeah >> we we don't have SP1 yet so let's imagine that we have that in R for now like >> what is the friends propose solution for example to go to Z to C++ or go to propose like maybe

>> I mean I can tell you what I uh suggest but >> uh okay but really what we want ultimately is not to make this work right >> so you you want to transfer that in Z right >> in Z in go I mean Go it already existed. It is the reference implementation. So that's go wise. It's it's >> there is one in C++ being built. No, >> there is one in C. There's one plus. >> Ah, it's you. >> Sorry for giving you work. >> Absolutely.

We still have to try diversity time. We also don't want to reuse the same library. This is also why we don't want us. >> Uh it's because it's already being used somewhere else. Yeah, we had discussion yesterday with I asked him like does it make sense to have maybe a single implementation and then index this multiple clients use the same library. He said no doesn't because networking stuff is super specific. >> Yeah. So yeah, most likely not will have different libraries for >> um >> so what we were saying here is ZQVM we also take this out because we use a socket >> actually right-handed sorry uh socket uh but then the LVM yeah in in my ideal

world this is also uh implemented uh as the local language Yeah. Um, and there's a soet, but there's also the op copy approach approach that I'd like to >> simple prefixes. Yeah. >> Yeah. Uh, if that works, I'm definitely going to try. If that works, we uh okay, at least that solves our short-term problem and then we can uh we can >> But you still have the problem of the build script uh where you have an external uh tools to build.

>> Yeah, but it's not as much of a problem because when like the problem is if I modify a file here, >> I have to re rebuild the whole ship. That's the problem. Especially if I touch the combo to to >> bad it recompile >> for example when when we change the version of press this is very frequent like then modify all of the >> exactly >> yeah give him a very long >> whereas at least with this approach I can compartmentalize everything >> like the old copy or the socket >> and so if I change something in VM or whatever is happening at least I don't recompile I think

um so that's that's a plus. Uh Go will still have some issues because yeah, they still need the C version of all that. Okay, they don't need Go. Uh they don't need Go for this, but they need those two things. So that's going to be a problem. Um yeah. What is the exact gain of maybe can you draw like the object copy path you how this works precisely like >> well sure yeah so um so you have your your health file right um and it's going to have some some

public symbols like some some externally facing symbols and then some internally facing symbols so um wow this marker's wrong with it. Anyway, um so like let's say you know you have this public symbol, let's call it prove, right? Um and then you have like a private symbol like let's say Malik, right? Something in lib. Um and uh the problem is you know you want to link this, right? But uh you know maybe uh there's there's also a malic in the thing you want to link it to. So uh you'll run this command um in in the Unix shell um and you pass this argument that's like prefix symbols

equals um uh you know right lean VM underscore um and now you get an out that has right lean VM prove BM Malo. >> Yeah, it's the Malo part. That's the >> um >> this is what we don't have control over. >> So, so now you can you can link this in without a problem uh at the expense of more code size because you have two lib C's and two two of everything that is also in what you're linking with.

>> Does it resolve like your combination time problem? Yes, it's uh it also doesn't change the thing like when you say the the binary is bigger. I had to deactivate LTO anyway. So the binary is bigger here. So that's not uh that's not going to change the thing for for that. Uh okay. >> Yeah. No like like if we can make this work then I can break that into different pieces and it's making it easier to I mean for the zig implementation is going to make things easier.

>> Okay. >> I we doesn't solve like the approach for go. go still needs to somehow get some C stuff or some A library like A but uh but yeah it's uh it's going to solve the >> uh just the title of the session was binding so I suppose we were also supposed to talk about the uh interface API that lin VM or XMss >> forms it's yeah but it's very simple there is like a trait right now that is key gen uh sign and verify. So you just key generation at the beginning. Yeah, >> we need the C names of those.

>> Yeah, but I guess this will be like kind of derived from that. But there is just three functions. >> Yeah. Yeah. There are just three >> for for VM. What should be exposed? >> I think it should be just proof or something like that. >> Verify proof. verify commit >> and and we need like some you know some some >> types. So like you know in the in the C header file that's going to constitute >> in in the C header file that's going to constitute the FFI we would need like some strruct definitions or something of that nature so that uh you know the the lean BM prover and uh the code that's

calling it know how to talk to each other. I think we can just put a context which is just opac pointer and you init and that initiates the context and um then you do your aggregation uh that generates aggregate uh and proof and you also have a merge to merge two aggregates together. >> Yeah. And this way uh when you do implementation of lining VM everything that might be implementation specific is hidden behind uh this opac pointer. But I mean that only works if you actually only uh use the stuff returned from any of the foreign functions inside of other foreign functions. Right? So if you

don't have to do anything with that in your in your local code. Um the reason why uh I would do a init uh with a opac pointer is because sometimes your init will initiate also a thread pool or your GPU etc and it can be on CPU it can be empty stuff and uh the rest is supposed to be stateless when you add signatures anyway. So >> for yeah for about the binding for XMSS this will be like the three ones. So you have the key generation at the beginning. This is very long. So but for example when you do run the key chain like uh the the keys that you return are they going to be necessary in the context of the caller or are they just going to be passed over to this is your

big >> yes that contains all your your keep a lot of them >> and then you just store that as a val. So yeah for like you can generate that for a given number of slots for given lifetime >> and so you have this big vector or >> so but but you have a serialization format. >> Yeah we have a serialization format we have like serialization >> but my point is like once you have those you will only pass them into for example the sign function or whatever right so you don't actually need them in the in the caller. Yeah. Yeah. Yeah. Yeah.

>> That is kind of a long process. We need to optimize this. But the last time that we benchmark that um it was like couple of months ago. I'm sure that it has evolved and it will evolve because we have this problem of lifetime. We said like couple of years because then you have to regenerate after this lifetime ends. You have to regenerate all of the key. So it was like 10 years or something like that and it was like 2 hours of key gen or something like that. >> Like how many how many keys are you generating there? >> And one per four seconds.

>> Yeah. Yeah. >> One per four. Yeah. You can do that. >> Yeah. Like a principle that's just like a random sampling situation, right? You just sample random data to KJ >> or or not. >> Yeah. No, no, you have to do some hash and some KDF. >> Okay. for the the the randomness. >> The key generation is HKDF extended. >> Not exactly. Yeah. Yeah. This is like you have a random generator at the beginning and then you >> comput.

Oh, I see. >> And the the slow part is then in the mer tree construction at the moment or >> yeah because you have the hashes for stuff. So it can be quite slow. Yeah, I think this is the main stuff. We have some nice trick about paralization there. We can also apply cmd if this is not applied by currently. So I guess that we can reduce that by a some factor but it will be kind of not a one second. >> I mean it seems like for this for example you know having a GPU accelerated version could be a very simple to implement and b give you a huge >> I'm not sure because it might be serial by nature that your key n + one depends on key n.

>> Why would it? because the the keys in principle they are independent. >> Yeah. But I I'm not sure that this is possible because we want like the validator to run on a very very small devices. So >> yeah because like in this validator set you will have like a lot of validators like just instead running on Raspberry Pi because you have 10% of them running on as aggregator on like highend CPU laptop. So this one will probably >> are the are the users wallet signing keys also going to be these PQ keys?

Okay. Well, maybe for them it would be useful. >> Yeah. Yeah. But this is a little simple API for like the lean VN stuff like we have like the aggregation. So that will be basically like enshrined and abstracted into into the stuff. Basically you can have like multiple like sign you have two two possible things in the for the aggregation. So you have like lots of signatures signature one two three like that will be aggregate by an aggregator. So the aggregator will consume this 1,00 signature and it will output like a proof and you have also this other possible situation where you

have like proof one and the other when you have proof >> it gives a proof and uh a signature aggregated signature >> just a proof. >> Just a proof. >> Yeah, just a proof. >> Ah and so this is mutated. >> Yeah, you it's a proof and like um a bit film like represented like because each um each signer is one bit. >> Mhm. >> And then it represent like the a proof plus like a bit field of s >> and then you have this second proof coming from another aggregator and then you have another aggregator that is responsible for aggregate this and

produce a new proof. Okay. So you have these two possible scenarios but that will be abstracted by the world VM and so probably the only stuff that you will have to consume in the linm will be kind of a proof function or maybe two different functions for >> I think yeah you should have aggregate and merge >> yeah we have we have a program that is called aggregate and merge but maybe this will be in China into a single program with if at the beginning >> it's better to have two different things I think as public API >> uh because uh it's not the same type uh so you can do and also uh for testing it's easier uh

to have >> different test scenario and the only thing I recommend is to uh even if on CPU it's not needed uh you initialize uh opac context object so that uh implementation are free to add uh thread pool initialization inside, GPU initialization inside, etc. and others that don't need it uh can just uh it can just be um um empty function. >> Just one for the me and when I also expect there's a verify function um especially when when I think about light clients usually they don't need a proof

part they only need the verify part and the verify part needs to be as small as possible. So I'm trying to think if it's probably it might be a good idea to say well maybe ping this explicitly just to verify may be a good idea >> because code size doesn't matter for a validator usually but it matters for a light client. >> Yeah and it will matter also like for validator because like we will have like lots of clients uh lot of validators running on very small devices Raspberry Pi people. So >> it will be like a very small limited device and on this one you can't run the VM. So you can just run the verify part.

>> Yeah. >> And so this will be important and maybe this will even be as a separate repo like we will have like a lin VM verification repo where we will just put the verification program >> corresponding to the link VM >> different crates >> different crates. Yeah. What it wants. >> Yeah. Um, yeah. The the Raspberry Pi is not a small device. I mean, you're thinking >> Pico Pico $10 device. >> Okay. Yes. Yes.

>> We're also almost at time. >> Almost the time. There's one aside question. You mentioned the the aggregator doesn't sign. Could you quickly repeat why doesn't the aggregator also sign? >> Sorry. What >> what many asked like you said the aggator is only like uh proving >> like creating a proof like why don't I thought they also sign >> I think at the end they they will sign but this will probably maybe not be part of this API. >> Oh okay. >> So yeah we need to check the separation of the of the API let's say I >> have another five minutes. So K is there finishing. No, no. Uh, >> wasn't that the last slide?

>> Yeah, so the last slide I was >> I knew we were only at two slides. >> Yeah, two slides. But what's interesting is that we spent an hour and a half on two maybe just an hour. Um, yeah, I just wanted to I mean asking that was just to start the conversation, but the conversation started itself. Anyway, uh either you rewrite something in C uh which I think is is an option for at least the smaller libraries otherwise I mean we already mentioned that we we build libraries. I just wanted to give the example of um of KCG for for GU where we have the CKG that we don't control uh but it's the faster but if for whatever reason there's a bug in

there we can switch to go KZG which is smaller but is different. So hopefully it doesn't have the same bug. And uh this is this is what I was going to say. Uh if you if you want to do more than one library >> uh sorry if you want to use external libraries like you suggested there should be at least two so that in case one of them has a bug you can in an emergency switch to the other one and um yeah and and uh don't like not kill the consensus while the bug is being resolved. But when I looked at godg last time, I don't think the support appeared as >> uh it should. If that's not the case, then we have a huge problem.

>> Open an issue. uh >> because I'm implementing uh Pas at the moment in >> uh >> so I was looking into this and I was but there are two godg one from proto lambda and the other from ke >> right >> and one doesn't support pas >> okay so we have to figure out which one it is and which one we're using uh um okay uh I'll need to talk to you after after this session um yeah so as a conclusion I mean thank you uh First, I think we we got a good uh a good series of uh of options to explore.

Um yeah. Any any last word? Any last comment? >> Good luck. >> Thank you. >> Um I think we should just take a picture. >> Yeah, I take a picture. I take >> Okay.