Post-Quantum Interop Call #25

Transcript

Hello everyone. Hello. Hello. >> Hi folks. >> Yeah, Will isn't here today. So >> isn't here today, so I will take the lead >> as usual. But yeah. Um, so yeah, I haven't created like um a PM issues because I didn't want to mess up with the work of of wheel since he's really organized with that. So we just do the the meeting to have an an informal update and then we will continue last week and next week as usual. So yeah, let's go. Let's get started. So as usual, we will first

start with the team update. So the Zim team >> hi Tom and yeah let's start get started with updates. Uh so Zim primarily has been uh working on uh improving uh the performance and stability of the client and improving syncing fixes. Apart from that uh we also implemented uh uh the definite 3PR for aggregation uh guided network and uh so we are sort of reviewing it and hope to merge it uh this week and uh on the third front uh for leanpoint

uh so we were able to uh test and get the leanpoint uh uh tool working. Basically linpoint is a tool like checkpoint Z. It collects uh states from various connected peers. So uh we got it working with Ze uh Ze nodes only as of now but I think Partha will try to get it working with other clients if they have implemented the API and also basically he was able to test that Zim was able to sync using checkpoint sync but I think uh we still need to focus more and improve uh our syncing more because I saw that in the

DevNet 2 channel uh for debuggings. Uh so basically uh whenever there is a restart there are issues still coming up in syncing up to head which also then causes network uh finality issues. Uh so our focus will be to tackle that while we review our DevNet 3 work and get it merged uh and also get the tooling uh working with everyone else.

Perfect. Thanks, Ginder. So, okay, that's amazing. Uh, now so over the past long DevNet runs, stream what I've seen has been uh fairly stable. Uh, apart from apart from uh an excessive memory use uh which hasn't which hasn't caused any problems yet. We haven't noticed any issues. I think our syncing works fine as well. Uh so uh for DevNet 2 for DevNet 2 stuff uh we're working on uh finding the root cause of our me uh our large memory use and it's probably just something not being saved to disk. Uh and we we plan to fix that as soon as possible. Uh and uh we've

also been working on improving our sinker uh making it more robust. Uh and apart from that apart from that we have retired devet one all our devet one code uh and now we have had added feature flags for devet 2 we have kept the features flag for devet 2 and we have kept the feature flag and we have added feature flags for devet 3 so we can run both uh at the same time uh and uh we have started work on devet 3 we are implementing uh the committee aggregation PR and uh yeah And uh we we're on that.

>> So this means that you removed the future flag for DevNet one. >> Am I correct? >> Yes. >> Okay. Uh sounds good. Kin. >> Okay. Yeah. Uh hey everyone. Uh mostly silent week on our side. Uh yeah we just were improving some metrics. working on some P2P stuff. Uh we enably improving our identify protocol so that yeah we basically can read agent version u of the peers. So ideally this would display us the client names with the peers that we are connected to. Uh but yeah so far it doesn't work. We we will figuring out

why. Maybe it's just uh other uh clients they are not properly updating their agent version. U so yeah anyway just working on that. I might create maybe PR spec if needed to kind of elaborate more on this. Um yes and also just only getting started with the net free implementation. Uh yeah uh I was working on the spec which we can discuss maybe later. Uh yeah that's all. Thanks. >> Amazing lturn. >> Yep. Hi everyone. Thanks Tom. Uh so basically a small update on our side. uh what I've been seeing is just issues uh sometimes happen in the dev net long

run. So um basically I'm currently trying to focus on um an issue where at slot 90 951 uh lantern basically rejected the uh voyage that he got from peers. Me and my other peers purchased it. So it's most likely a spec divergence uh that I'm still investigating. Um so yeah once the recourse has been found I'm looking forward to releasing an updated image uh and then uh hopefully it should be more stable. Uh but yeah that's from my side. >> Cool. Thanks for the update. Lighthouse maybe with someone.

No. Zen Lambda maybe. No. Uh Grandin. Hello. Uh yeah. So we fixed um a few issues and added metrics and some other stuff. And I know there is still some uh some things that um doesn't work. So yeah, maybe next week we'll have some time to to work on that. >> Cool. Um Gin maybe.

>> Yeah. Uh hello. Hello. Yeah. So uh yeah. So we completed uh defet zero currently moving to uh we've moved to DevNet one. Uh we are currently working on uh basically integration with uh linseek for uh uh message uh signing and also uh signature uh verification alongside with uh making sure the implementations for lean metrics are also uh in place uh and uh also working on integrating uh lean uh quick uh quick start to uh complete the requirements for uh

definite uh one. So basically uh those are what we are currently uh working on on uh gene at the moment and uh you know we are writing uh our client uh the gene in Golang. So uh we're also following the route of having uh like bindings uh go bindings to interact with uh you know uh the rust code that uh uh fall sake and all. I think that's uh it trended I think on Zim and uh also long uh long term. So yeah, basically uh that's what we're currently working on uh on Gene. Yeah, cool. Uh, Nimbus maybe I know that you

have just started but maybe some stuff to add or anything. Okay. Um, any client that Yeah, >> I think Mega is here from >> Yeah, exactly. Eat Lambda. Eat >> Lambda. Someone there. Yeah, please feel free to go on. Yeah, sorry for shing light. I I couldn't find the the the link to the call.

>> No problem. >> Okay, for my update or it's Landanda's update, we mainly worked on fixes and stability improvements for W2. uh we fix the memory issue we had by adding chain and attestation pruning post finalization to bound the memory usage during periods of of finality. Um we also worked on P2P stability adding support for incoming block requests from peers. We before we didn't support that yet and we also added recurrent redialing for disconnected peers uh since we found that in in usual usually

in devets sometimes it happens that peers disconnect. Um, we also added support for SIMS block by block uh blocks by root message format uh response. Um, that one that um the new format that uh wraps the message in a in a container. Uh we also added information more information to log messages which helped a lot with debugging. We are currently also working on fetching unknown blocks refer referenced by gossip data stations.

Uh right now we aren't doing that and just discard those at the stations as invalid. Uh we are also working on supporting multiple multiple response chunks in blocks by root requests. Uh right now we only support uh one response chunk per request. And we are also working on checkpoint sync paralog. Um this next week apart from continuing all that work we'll start work on the net 3.

Cool. Um anyone that I have missed? No. Okay. So then we can do like a brief update on what we can do is that like a brief update on spec and research topic and then we can go on on like uh problems, questions, interrogation and and stuff. So first about the research topic um we merge quite a lot of stuff. Emil merged quite a lot of stuff on on VM. He can talk about this later or now like maybe um I know that you

have merged like bunch of stuff and also thoughts about some new topics for the XMSS signature scheme, new research orientation and stuff. So yeah, please feel free to >> uh thanks. Hello everyone. Um I've been trying to improve proof size a lot. So first uh the trivial optimization was was Merkel pruning that's done and I am seriously considering removing log star which we which vin multisig currently uses currently multisig has two separate we commitments and the second one is due to lost star and this is um optimal like because it saves a lot of commitments

surface in the execution air table but the the main drawback is that the two separate weird commitments uh adds lot of like some complexity conceptual complexity and implementation complexity and also it increases proof size and uh so I have a branch ready and finishing it and maybe like we will remove logup star we need to think about and to discuss it but that's a serious consideration and now we are below 256 6 kilob and um like with aggressive rates in we can get close to 150 kilobytes.

Um so starts to be good and uh like if you had if some someday we prove the capacity gaps conjectures the proximity conjectures we can go below 100 kilobytes that's possible. Yeah that's it for me. Thanks. Uh thanks thanks a lot. Um to add some some stuff on like the the research side, we have also owner working with us uh on a grants uh at DF to improve performance of cryptography and especially we that is the polinomial commitment scheme used inside Linv and so it did amazing progress during the

last couple of weeks. uh we merged yesterday a very big PR on the we repo in order to use a very specific low-level technique for the sum check um to improve a lot the sum check inside we so it it's already have impacts as well inside lin multisc um you will see also probably some PRs on uh ploni3 because uh we are trying to integrate uh to use the main branch of PLN3 inside uh LINVM as soon as possible or at least like in the future in order to beneficiate from all of the

open-source community contributions uh of PL3 directly into LINVM. So we have a bunch of new peers to be merged around like optimization new integrations inside bron. So that is about like the the cryptography. Um about the spec um I have continued this week working on the minimal client and also a bit the book when I I found time. [snorts] Um I have simplified a lot the code for the minimal client. I have removed for example TCP that was a big part and quite painful to debug. Uh so that now we have only quick inside

and I use like a a standard library of of Python to for quick so that this is pretty light and I removed all of the legacy stuff with TCP and all this. We can add this back in the future if we need it but for now this simplifies a lot of things. Um, and I have spent time on the book. Maybe I can sh I will share my my screen just briefly just to show you what it looks like. Um, can you see my screen? >> Perfect. So yeah, I I try to to write this a bit incrementally. I know that

this will evolve with time because we will have some new research topic coming up and a lot of stuff uh will be modify, added or like or deleted maybe. Um for example like the P2P layer we know that it will be modified. The fast finality stuff we know that it will be modified as well. But I have tried to write this in a way that it could be understood by everyone including developers, researchers, students, new people coming in the Ethereum ecosystem. So for now I have a couple of part I have done like um kind of intro of why the consensus exists uh what it is, what is link consensus, what

it is as a redesign effort, uh what does it mean? All of this then I have written like SS uh chap chapter just to present SS a bit like the design philosophy the type system and and all of this then I have spent a bit of time on the time model just to present the slots uh what does it mean to have shorter slots uh what is a clock what is the genesis why the genesis is specific and all of this then I have started to present the state transition position and uh I have started a chapter that is not finished at all about the peer-to-peer layer to present the enire and the transport

layer with quick a bit more and stuff. Um we we will not go um over all of this because this is like pretty heavy. But just to give like a brief recap um I have tried to write this chapter in a very readable way. I have tried also to add a lot of drawings and explanation. For example, if we take the SSG chapter, I know that it can be a bit painful to understand all of the type system and and all of this. And so I have tried to add drawing like this just to to explain to the reader and stuff. So I have done that with basic

like uh latex drawing tools. Um so yeah I I am continuously improving this document day after day adding some section chapters paragraph just to explain [snorts] uh in the future I think that what I will do is that I will um I will put a text file or text folder inside the spec directly so that anyone can uh update this and participate and push PRs but for now we have just done that in a separate repo and just updated the the PDF on a regular basis. But yeah, uh that is where I am right now. I have tried to implement

pretty much what I could uh but I know that some some stuff will change in the future. For example, in the peer-to-peer layer, I know that we will have research and stuff. So this this text will be modified 100% sure. so yeah that is it also um yeah let let's continue sharing my my screen just um to present like on the lin spec I have added like a bunch of PRs and um especially in the test there I have I have added like an interrupt uh folder when I have pushed uh interrupt uh test

uh just this is inspired by the rim philosophy where like I don't know if other clients are doing the same but I I am familiar with Russ so I check the the the rim client and uh I have seen like multi-node test just just to test finalization mesh and all of this and so I am doing a bit the same there with like a multiode testing uh with like a mesh mesh of node And then I I check finalization and all of this during like a run duration of 70 seconds. For now it passes. Um I am pretty sure that there is still bugs and uh and stuff to to be

fixed but I think that this is pretty important uh for now that we have this kind of test in in order like to check that the node is running and that we have no bugs. So that now we have unit test test vectors and this I have also found like during my PR that uh in the text vector we had a problem because it was long and it was due to state state root calculation that was done way too many times and I found that during debugging and now uh I divided by a factor of six or something like that the time of like the the test vector during the CI

So that is pretty amazing because now it is much more flexible and manageable inside the CI. So yeah, that is the current state uh of the spec. I will continue spending time on the client, the spec and the book. Um I have seen also the committee aggregation PR. My opinion on this is that we have a lot of change there and I am in favor of merging the as soon as possible. um as soon as this is rebased and uh and the test pass I am in favor of merging this and then we can iterate over the PR because this is pretty long and

this can be a nightmare to review and like we can spend infinite amount of time on reviewing this and review cycle. So yeah. >> Uh yeah, on that I I just merged the the latest master yesterday, but >> uh this morning I wake woke up and just okay, I need to do this again. So yeah, if you could merge it, this would be great. Yeah, I'll I'll fix uh with the latest well I I'll replace it on the latest master. So it should be good. Yeah, PR is a bit large because I as I said like I removed latest not new stations and latest nonstations so that we basically used the aggregations instead as we kind of agreed so that we use them to update our safe target basically for our fork choice. Um I removed some tests there that kind of

became became less relevant now. uh I could replace them but this would make PR only larger so I would rather follow up with the next PR for that and uh >> yeah so I I really try to not to kind of make it larger but rather make more PRs a bit later >> so >> what what what I will do as this is important and this is the next target I will stop making PRs on the repo >> so that you can take your time uh fix the conflicts and stuff and as soon as this is fixed I will merge and we can continue the flow. So that this is okay for you. No more work for you. You just do that one time and from from now I stop making PR.

>> Yeah, sounds good. I'll try to get it ready by tomorrow morning at most. Yeah. >> Anything else? Maybe regarding tooling ka not sure if you are there. >> Yes. Hello everyone I'm here. So uh regarding tooling um I regularly share the status of the devet and restarts. So my only um request for for the teams please just uh follow it. maybe someone from from the team who is responsible for this uh because we we don't have stable finality. So this is important

and uh as a lot of um not a lot but new clients join we need to investigate all the cases. So currently for today I guess we don't have new images uh from the teams. I'm just going to restart it for four initial clients just because we will run out of memory for some clients. And whenever your new images are ready, please let me know responding to to this pinned message uh like uh restart planning. Uh keep me informed uh whenever the new team is ready uh so I can restart it. And please um investigate the logs. Um yeah, I I normally share it in the tooling channel. If you are not still there, um

yeah, please um turn on notifications. Uh thank you. >> Um yeah, thanks a lot. Uh so yeah, for me, >> so just yeah, so just to add to this, uh the current status, we lost finality again. So we reached uh almost 1,000 finalized slot, which is a lot. I think it's maximum for for now. Uh it's a good result but um yeah then it was stopped. So we need to investigate to update images again from the client. So yeah.

>> Yeah. May I just add a little bit on that? Just a wish for I don't know some bright future. Uh at some point it would be really really great if we uh also were able to kind of collect the uh core dumps of uh the crashes if some node restarted because now yeah um yeah see the messages for example clean got restarted and seeing that it's well just analyzing the logs sometimes it's not enough um we did this in the past a lot with our other clients and that's yeah really help analyze uh >> yeah I do that. Uh just just pin me if you need this because not all the team need this. If you need it, I I can

provide it. >> Yeah. Yeah. I need I will make sure that our Docker image can actually provide proper uh core dumps with all the symbols. But yeah, I I'll try to >> I mean at least from system journal I can I can maybe extract something. >> Okay. Yeah, we can work together on that. I mean need any help. Okay. Thanks. >> Nice. So yeah in general we have like other outsource of discussion about like telegram everything for me the telegram consideration consolidation sorry is

working well. We now we have all of those threads uh where we can organize our discussion and for me this is going very smooth. So this is nice. Uh on the spec I don't have any other topic as mentioned I will stop pushing PRs so that we can merge this committee aggregation PR as soon as possible and then iterate over this. So for me that is good. Um on the tooling we just mention everything client updates is good for me as well. So do you have other topic discussion question?

Okay. So sounds good. Sounds good. Please like uh for the PR >> I want to add something to. So uh now that we have devet 3 spec basically done and the clients will now move on to the devet implementation. So I propose that uh devet 4 we basically do uh recursive aggregation and uh if that is good then where are we on the recursive aggregation bit because then we can start working on the next spec.

So for this my overall picture then we can pass to the appropriate person but first we need that to be ready on lin on lin multisig as far as I understand this is okay even if maybe a performance are not the one that you you are dreaming about but I think that this is okay then we need to update the >> okay >> sorry it's close to be Okay. Yeah. Okay. Okay.

>> A few details are remaining but overall yes. Yes. >> Then I guess that we need to update the Python bindings with the last version of the repo and then we need to integrate that into linspec. But I I would say that the main blocker would be uh the lin VM. But if this is okay then I don't see any blockers. The part that is not implemented. >> Sorry. >> Yeah. Yeah. Please go over it. >> I want to say the part that is not implemented is um what to do when you want like if there is um uh more than one layer of

recursion like you recurse and then you recurse again. There is some nonreal stuff to do. uh you need to prove to prove the the evaluation of the bite code at some point and um for yeah this is some this is a bit technical but doable and also there is the um the bit field um to ensure that every uh uh signature like for every the fact that you need to match that there may be mixed together uh uh row signatures and um aggregated proofs and so the bit field management is maybe a bit tricky

but overall the recursion is mostly um ready yeah >> uh so if I understand so we will only have one layer of uh recussion or we will have >> no no it's it's it's totally possible to have infinite recussion like but it's just there is a bit of for remaining but it's totally very possible. >> Got it. And uh so can we sort of have an API uh what what would be the API look like? Can we sort of have this because the current API are there any differences with regard to that or I'm I'm assuming there would be I guess

there will be an >> there will be >> yes the API will change but >> API will change but >> um >> and and it's like a bit of time to finish the work of rec but then I can try to make the API work. So or if you can basically uh I mean sort of uh uh say that this is the API then what we can do is basically we can work on the spec part of it using the API hoping that uh uh the implementation will get plugged into it so it will be sort of parallelized.

>> You mean having the API before having completed the implementation behind the API? Yeah, that's possible. to >> Yes, if we can spec the API first, uh that would be great. >> Okay, I will try to do that. Okay, >> try to do that. Okay, >> cool. And I uh I hope that uh maybe next week, end of next week onwards, we can start having interop on devet three. What do other clients feel about this?

>> Uh pretty possible goal. Uh let's Yeah, let's see. Let's see. >> Yeah, let's not rush too much. I would say let's try to Yeah. make it. >> Yeah. No, Ree and Kim, I want you guys to deliver next week so that we can have an inter we shall do. We shall do so. >> Yep. Yeah. Maybe just following previous discussion just in case I don't know we don't have for whatever reason Lin VM ready and recursion aggregation it's I believe it's okay just to take maybe one devet cycle to simply optimize some stuff I don't know things like for

example we kind of including uh we can include old proofs into the blocks now which yeah in some simulations could end up basically with the large blocks I believe yeah you have thoughts on some other stuff it's not necessary that yeah we can introduce can new feature every time. Uh but yeah, just simply uh optimizing and stabilizing the network would be great so that Katy is happy and we don't have like networks finalization crashing every time. uh all that stuff uh you're just saying >> but KL that is part of development I think uh I mean I do respect your opinion but uh and do understand where

it's coming from because definitely you know there can be too many cha too much chaos as well uh and yeah so maybe next week uh if re feels like you know they are ready we can do with them and uh basically then other clients can catch Yeah, at some point it would also would be great kind of run multiple uh networks so that I don't know we have like some stable networks we have like devet three and in parallel also running devnet 2 um just I don't know just if we have enough environments uh but yeah again just saying that it

would be great to kind of have some I don't know maybe the freshest version with the new all features but also have some room where we actually optimize and stabilize uh our previous clients. Uh yeah >> yeah as para has also mentioned in the chat I think devet three runs will be longer because we will also try to scale aggregators or basically validator nodes with that and uh so there could be multiple runs of devet 3 just focused on uh scaling the number of validators and if possible also the aggregator subnets uh but uh yeah but I think Yeah, in the meantime

we can move on defet as well. But yes, let's talk talk about it next week. Maybe it's too soon. >> Um just a quick question for the client teams. Um is there any pending updates for the PM document for DENA 3? Uh just to keep track of all the PRs that are going to be needed for DENA 3. Yeah, I believe we need to update it. So now we have five intervals. Uh I don't think we kind of reflected that on the PM repo.

>> Yeah. Yeah. And I think for for the changes, it's easiest to just like use the filter. Um I'll I'll send you the URLs again, but um filtering the PRs from the leanspec should always be more up to date than the the document in the PM repo. the the PM repo is going to be like maybe like one week lagging most of the time. >> Yep. Yep. Sounds good. I'll use that. >> Yeah. Mercy. >> Sorry. Good afternoon. I wanted to also point out about the checklist. Um if we can comment on um any missing updates there so that I can update accordingly then we can merge then um

the template for other dev nets we can start um filling it out also for deetry for example >> so where's the checklist >> I have an open PR on PM rile let me get I see. Yeah, you can see that. Yeah. Okay. Yeah, let us check that for next week.

Thanks. Thanks a lot for being Anything else? Perfect. Uh, thanks a lot everyone and see you next week with Will back. >> Hi. Have a great time everyone. Bye. [clears throat] Bye. Bye. Bye.