Post-Quantum Interop Call #40

Transcript

Pablo Deymo: Hello, hello.

Shariq Naiyer: How you doing, Pablo? How are you doing, guys?

Thomas Coratger: Hello, hello. Can you hear me?

Shariq Naiyer: Yes.

Thomas Coratger: Perfect. Okay, so… Let's get started. So today… We're gonna discuss a couple of small things, nothing very big, but… mainly the DevNet 5, stuff and PR that we have merged, in the spec. A bit of, like, lean spec stabilization. And a bit of a scaling of number of validators. And then I have seen a couple of comments about DevNet criteria's acceptance. And, a bit of benchmarking. So, yeah, let's get started with some client updates. First, let us start with Zeam.

Gajinder Singh: Thank you, Thomas. Yeah, so we have been working on DevNet 5 spec. And it has been merged, so we will now start implementing DevNet 5. on DevNet4, we have been working to further optimize Zeam as an aggregator. We have… we did several, PRs for that. to… Resolve our hot parts. And we also did some PRs to add aggregate metrics for subnet and combined, and we will be upstreaming those metrics in the lean spec as well. So that all the clients can implement it, and then we can get a unified per-client view of the metrics on aggregates, because as of now, I'm feeling that this is something that we Was missing when we were trying to do analysis for multiple subnets, and

Multiple aggregators per subnet. So, this will give us more visibility into How each aggregator is performing. And, on the test runs, we further integrated, The tests that were, updated on the lean spec, and… We have been, keeping up with the Hive integrations as well. on… on DevNet runs, Partha will give you update for scaling the validators, because we started scaling the validators. And again, as expected, we started running into, again, the choking issues that that were expected, and we expect various clients to look into it and resolve those issues. On Zeam's side, we will be focusing on improving our aggregation computation itself. That is something that has come out to be a clear priority for us.

So this is… Overall picture on Zeam's end.

Thomas Coratger: Great, thanks a lot. Ream?

Shariq Naiyer: Hi, guys. We have implemented, one of the PRs around block building that was, allowing us to use older justified, I forget exactly, but it was allowing you to use older justified stuff to, to build blocks currently. We've implemented that PR, we… We're looking through the DevNet 5 spec and implementing it now, and, we've explored some metrics around application, that were missing, from… so Partha couldn't tell us what our application time was. And on Hive, we're adding tests for multi-subnet, interop, so clients operating on multi-subnets against other clients, and we're adding tests for, every client as an aggregator, and we're adding tests for the new API that was added to, lean spec. That's all from us for now.

Thomas Coratger: Great, thanks a lot. Qlean.

Ruslan Tushov: Hello. We're fixing some Hive tests. currently investigating the problem with connecting to Mock Peer from HiveTest. which is used in GOSI validation and request response tests, and also implemented a test driver for spec tests in Hive. Thank you.

Thomas Coratger: Great, thank you, Lantern. It's from there.

Pablo Deymo: No, hello. We made a PR to lean spec, fixing an edge case in block building, and that was already merged. It's already merged. We saw some other instability due to useless attestations being included in build blocks. And so, we are already testing some changes to block building that should address this. And we will make a PR coming to lean spec for this afterwards. On the other side, we have full block aggregation implemented. But we are debugging an issue with slow block proof generation. And, we, have a working execution layer integration. We can show you a demo after this plan updates. In this meeting, we can show you a link. So, that's our science.

Thomas Coratger: Great, thanks a lot for the PRs on lean spec as well. Now, Gean?

Shaaibu Suleiman: Yes, hello, hello. Yeah, so, from our own side, we've been, making some, updates, basically, some merged, pull requests for the DevNet 4, I think on networking as they come. We also make, the updates on our own ends, and also, a couple of hive tests, that we're failing at some point, around networking. And all. We've been trying to, fix that also. So yeah, that's, pretty much it from our own ends. Thank you. power and off.

Thomas Coratger: Great, thanks a lot. Lighthouse? Okay, and Grandine?

Mercy Boma Naps-Nkari: Good afternoon, so… We received an OOM report from the general devnet, and then we did an internal run. And this is the result of it, So, we are still trying to figure out why. this OOM issue occurs, on the general devnet and not, in our own internal one, so… We are currently trying to attach a profiler to pin down where the issue is coming from, and then… Fix some, figure out if it's a configuration based on the DevNet, on the server, side, which we're not sure yet, so… If we figure that out, we'll be able to… if it's a fix from our own end, we'll make a fix for it, but we're still investigating. Thank you.

Thomas Coratger: Great. Thanks, thanks a lot, Nimbus. Yep, so, yeah, and Lean? Yeah. Okay, did I miss someone? Okay, great. So, I can give you, like, now a brief update of what we are doing in the spec and on the research side. So, first on the spec. No, first, maybe on the research side. we have merged, like, a couple of new PRs in Plonky3. For different sources of optimization, I will skip details, but this is mostly small marginal gains plus, like,

We have merged also a full benchmarking suite for WEER, that is our polynomial commitment scheme. We have also merged a new arithmization for Monolith, that is an arithmetization-friendly hash function. That is, let us say, super, super modern and less studied alternative to Poseidon. Just to test and stuff, so we have done that. not that much new things since Rome on the research side. Maybe, Emil, you want to complete with the new stuff that you are doing right now in LeanVM?

T. Wambsgans: Hi, everyone. Yeah, mostly working on… improving the spec and the paper associated to LeanVM. there is now a quickly hand-coded Python verifier implementation, which is, like, 1,500 lines of code. I… as soon as the spec paper is, good enough, I will have a look at… myself, at the Python verifier. To try to make it below 1,000 lines of code, and then trying to… write a Lean4 verifier. And so, yeah, entering, Like, at least, having a well-defined, verifier for LeanVM. That's the current goal.

Thomas Coratger: Great. So that is on the research side of things, so… I think, from my hand at least, like, I am spending most of my time right now since, since after ROM, on the spec, on lean spec. because we are entering a bit of a phase of, I think, a stabilization, and I want to showcase also, like, the specification to, like, a more broader audience. We are discussing that internally at EF, and I want… I would love, like, to have more feedbacks about existing Beacon client teams, and also, like, more people inside EF and outside EF, the ones that are doing, like, currently the Beacon specification, just to see if they agree or not with how we are doing specification.

That's why I am trying to clean things up a lot since this weekend. So, for example, basic example, I have reworked a bit, like, the way we are doing some SSZ types, and trying to add more test coverage when missing. I have completely removed this morning, as well, the discv5… discv5, discovery stuff, because we are not using it, and we don't need that for a spec or a minimal client. So, I have removed these kind of things that are… Not that useful for what we are doing. I am also trying to check every line of code to check that there is nothing pretty wrong. So I am spending a bit of time on this, to architecture things properly, before showcasing that to, like, a broader audience than just this group. So I will work on that in the coming,

days, and weeks, probably. At the end of this week, I am going with, with Antonio from EF, and Justin from EF, at a conference in Zurich to present, post-quantum, consensus. for Ethereum, at a conference. of researchers. So that is it. On the spec side of things, also, this morning, we have merged the DevNet 5 PR, so now we have, like, a single block posted, a single proof posted per block on-chain. So that is the good news. No, I think that we can stabilize the spec a bit. I don't see any major feature that we should add. Also, I think that this is pretty aligned with the comments that I have seen from Partha in the issue.

of this meeting. Basically, Partha is proposing, like, to defining the clear acceptance criteria for each DevNet, because I think that the complaint is a bit that we are We have not reached stability yet. And this is a bit of a mess right now to debug the DevNet, so basically what I propose now is that Now that we have a single proof per block on chain, we can just try to stabilize things and reach a larger number of validators. So, let us say, as soon as we don't have stability for 1K validators among different clients and the spec, probably, then we shouldn't move Anything else, because this will be useless just this will just result in more bugs into the code and into the DevNet, so I think that we can do something like this. I don't know if you have any comments or things like this.

Yeah, please feel free to comment on anything.

Parthasarathy Ramanujam: Sorry, was that a question to me, Thomas?

Thomas Coratger: No… not specifically, but I think that in the issue of, like, the… in the issue of the… of the two-day calls, you mentioned the fact that we… we should have clear acceptance criteria for each DevNet.

Parthasarathy Ramanujam: Yay.

Thomas Coratger: before moving.

Parthasarathy Ramanujam: go to the…

Thomas Coratger: Next one, yeah.

Parthasarathy Ramanujam: Yeah, so just wanted to elaborate on that point. Until we ran a DevNet with just two subnets, we were able to achieve justification and finality almost every run. But ever since we scaled up to 4 with 64 validators, we've not achieved it even once, and every run results in a new bug or issue. The major concern that I have seen is that, for the metrics that have been available for 4 clients, the P95 for aggregation is 4 seconds, but our slot interval is 0.8 seconds. So… I'm not sure how we would achieve justification of finality when every client is failing to complete the aggregation task within a given slot interval. And this is the present case with… without recursive aggregation, so I'm not sure how recursive aggregation would add on to it.

I mean, it's… it's okay. If we can agree that the problem is only with the hardware specification that we are using for aggregators, we should ideally arrive at, what is the exact specification we need for the aggregator service? So that's why I wanted to check with O, what were each of those criteria that he benchmarked, what was the legend associated to each of those hardware, so that we could then Come to an agreement on how, big the aggregator servers ought to be. Yeah, and the second, issue I've seen is that, although it was evident earlier, lib P2P latency does add to the stability or instability of the DevNet, at least until what I've seen till now. So, when you're running multiple subnet DevNet on the same machine, you may be able to view finality, but on the DevNets, it doesn't, because we are running it across

30 different servers. So, but before we could, really see if the, P2P, layer is an issue, we should first sort out, the issue with the aggregation, and, try to see that at least P95 across all clients is achieved within that 0.8 second interval. Yeah, that's all I had. Sorry, Gajinder, you can go for it.

Thomas Coratger: Yes, please Gajinder, go ahead.

Gajinder Singh: Yeah. So, on this, I want to add, basically, I mean, we should go in a reverse manner right now, because… By the time this thing will go live, and… We will have… A lot of hardware improvements, as well as… software optimizations. So, the way we should approach this particular issue is, yes, the aggregators might require a different hardware spec, and for that question we need to ask is. what is the… what is the network that we are envisioning? That an aggregator… how many aggregators will be there in a particular subnet, or what is the size of subnet, and how many subnets do you expect a typical aggregator to aggregate? And on the basis of that.

We, right now. get a… gets those machines, for the aggregators, which are beefed up, but then work hard to basically bring down, the requirement to the level where it's acceptable, and what we would like to see in mainnet. So… so I would basically want to go in reverse so that We can then target what we need to achieve.

Thomas Coratger: Okay, so… Have we identified, like, the source of the bug? Because let me summarize to see if I understand correctly. So, basically, when you… tested with one subnet, then everything was fine. You reached finality and justification, but as soon as you test with, like, two subnets or something like that, you have a problem. Is it correct?

Parthasarathy Ramanujam: Up to two subnets, we are able to receive, achieve finality, but anything greater than 2, it becomes a problem.

Thomas Coratger: Okay, okay. So yeah, up to two subnets, it is okay. After two subnets, there is a problem. Have you, like, have we identified, like. at least what the problem is. Is it, like, the time to aggregate? Is it a networking problem? Is it a bug problem inside the code? Or what is it exactly?

Parthasarathy Ramanujam: So, most of the cases, the entire network does not view the same section of the chain, right? So, aggregation is slow. As a result, within a given slot, every client does not see the complete view, so they are not able to Progress with justification, or essentially complete the state transition function. As they should. So till now, over the past week, I've run probably 5 to 6 DevNets, with different sections of clients. Across, the board, I would say the average aggregation, or rather P95, aggregation is 4 seconds, and average is… ranges between 1.5 to 1.8.

Thomas Coratger: Okay. And so, okay, so the problem, if I understand correctly, is that the time…

Gajinder Singh: I… basically, I would… I think there are multiple issues here that I see before, basically, we try to… So, in my… mind. I think there are, there are multiple issues. Like, one is, like, whether all clients as an aggregator, they are stable. They are able to produce aggregation of their subnet in correct time, because when we are going above 2, basically, we are putting a different client on each aggregator subnet, right, Partha?

Parthasarathy Ramanujam: So, yeah, at the moment, I have homogeneous, so every subnet consists of the same client, with one of them functioning as an aggregator. In next future runs, I could probably mix and match them and try with different combinations.

Gajinder Singh: Right, so… so the point is that going beyond… so up till 2, we have some stability, and beyond to the stability, there are multiple issues, right, that I can see, and we basically need to identify them and debug them on that basis. So, first, without increasing the number of aggregator clients, and just increases the number of subnets, as well as the aggregator combinations, right? So… so instead of putting more clients in the mix, just… Make the same client as… as two nodes of same client as aggregator or something like that, right? So, first, this needs to be done. Then the second issue, I think, is whether our clients as an aggregator role, they are stable. Then the third issue that I see is that whether, in a block proposer, if you are getting multiple aggregates, what is the, what is the block proposal, efficiency, right? So, what time is it taking for a block proposal to

Process all these aggregates and put them in the block. And, so… so these are… and the… and the last issue is that Once we have all this, basically, what is the… aggregation coverage on all these subnets. And for the last… last point, I think is quite a bit important, and that is what I earlier mentioned, that we will be upstreaming a few metrics to basically capture that. And that is something that I think will help us quite a bit in debugging, that on these subnets, what is the… aggregation coverage that is happening, as well as what is the combined aggregation that is happening… coverage that is happening, as well as what the aggregator is seeing, and compared to what the block proposer has done. So all these metrics, I think, on an urgent basis, we need to add, and so…

We will upstream them, in the spec. By tomorrow. And hopefully, when we have that, we will start getting a better picture of Where the things are going wrong, and then basically drill down on what the issues are by saying that, okay, first you scale without increasing the number of aggregator clients, and then you try to basically, again, have an approach Of seeing how many clients, as an aggregator, they are. they are stable even in a small two-subnet, network. So… so I think we will basically need to have This kind of approach to figure out, Where the issues are, and to patch in, solutions for them.

Thomas Coratger: Yeah, please, hold, go, go ahead.

Unnawut Leepaisalsuwanna: Yeah, just want to share some numbers, from the bench, from the feedback that Thomas gave last time that I… I benchmarked against, main branch, and that's more performant, but that's not what we are using for… for DevNet 4 and 5. So I, I re-ran it again, and, the performance… is about, three times, slower, but I think it kind of, like, might be answering what PARTA is, seeing as well, for example, like.

Thomas Coratger: J- you mean this is 3 times slower, like the… the DevNet 5 or DevNet 4 branch is 3 times slower than the main branch, is it correct?

Unnawut Leepaisalsuwanna: Yes, yes, yes, that's correct. Yeah. And, so if you look at the aggregation for 125 signatures, if it's a 32 core, it's about 1 second. Sorry, actually, yeah, 1.1 second, P95. For 16-core, that's about 1.6 seconds, but that's linear. Parter, I think you said 64, right? So I think we are looking at about, like, 800 milliseconds for… for basically just the leaf aggregation. So I guess that's why the aggregation at, like, a single subnet level works. But if we look at the… the subnet level. For example, like, these 2X, so basically two subnets, right? We are looking at about 1.3 milliseconds. Which, I guess, like, is still okay, like, it's within, like, the fore… But once you go up to, like, say, 4, it doubles up. So I think that… that might be one cause why…

you…

Parthasarathy Ramanujam: Oh, yeah.

Unnawut Leepaisalsuwanna: yet more time.

Parthasarathy Ramanujam: Sorry to interrupt, did you say 64 core? .

Unnawut Leepaisalsuwanna: No, I said 32, but I assume that you're using 64 validators.

Parthasarathy Ramanujam: Right here.

Unnawut Leepaisalsuwanna: for scaling, scaling?

Parthasarathy Ramanujam: Oh, yeah, yeah. Yeah, yeah, 64 validators, 8 validators per subnet, but the configuration is 8-core, 16GB RAM.

Unnawut Leepaisalsuwanna: Yeah, so… and I think, like, the recursion, like, it doesn't matter how many, like, leave signatures you add in there, right? It's the number of… number of proofs that you are aggregating, and once it goes up to 4 proofs. Like, we are looking at, 2.7 seconds on a 32 core. I think that might slip past, like, the remaining time for a slot, or for the next slot, and that might be the issue.

Parthasarathy Ramanujam: Okay, so then going by this bench, our configuration for, an aggregator should be at least 16, core. Is that what you're addressing?

Unnawut Leepaisalsuwanna: So… Are you… oh, so you're using… Less than 16 right now.

Parthasarathy Ramanujam: Yeah, yeah, I'm using 8-core, right?

Unnawut Leepaisalsuwanna: Okay, then I… yeah, so, so I… my… what I'm trying to say is that if, like. I think we might have to be based on the performance of lean multisig right now, because, like, if we try to optimize any level above it, but the raw performance from the multi-sig is basically, for example, like, 5 seconds. To do, like, a four subnet, aggregation, then… then it's not going to fit in a slot, or two slots, or three slots.

Thomas Coratger: Maybe I have a couple of comments about this. So, first, we will not touch the spec much more in the coming days and weeks, so we will have plenty of time to debug this, so that is the good news. the fact that the DevNet 4 branch is 3 times slower than the mainnet branch in some sort. is kind of a good news. Why? Because… This means that if we switch to the main branch. We will… we will, like, have something 3 times faster, basically. The problem… Emil can elaborate on this, but to summarize in one sentence, the problem is that the current main branch… the parameters of XMSS that are used in the current main branch are not proven

in the paper of XMSS, these are all the parameters that are Under construction in terms of proofs, but not proven officially, let us say. So, of course, we can move to these parameters if this is better, but I don't want us to have a bad surprise at the end, and having to move backward in something that is much slower. That is something that we can do as well. And I don't know your opinion about the hardware requirements, Emil, but I think that the part of the hardware that you are describing with 8-core seems super, super slow, I think, and maybe super small, because right now. we are kind of, we are kind of under large constraints already with CPU only for signature aggregation.

And so, I think that all of the benchmarks that are run by Emil are on a M4 Max, so it's a, let's say, a fancy laptop. And I think that we cannot really run on a non-fancy laptop, I would say. Emil, please elaborate, but I think on such a small laptop, this will be difficult. Even, I think, in Rome, we benchmark within my laptop, which is a M1 Pro or something like that, and it was already four times slower than on Emil's laptop, so…

Parthasarathy Ramanujam: I follow, but given that we are running everything on Hetzner, and Hetzner does not provide M-related stuff, so I just need to understand if we can… what is the minimum configuration we should target for? So if it's 16 core, that's fine. If it's 32 core, that's fine. We can discuss in the DevOps group on what's the budget available to set up. But, I just need the minimum configuration for an aggregator so that we can plan and deploy this accordingly.

Thomas Coratger: Emil, feel free to elaborate, but I think you're the 64, no?

T. Wambsgans: The MAC is, I think, 16. But there is the number of cores, but there is also the performance of the memory. And, typically, I have, like, a small Hetzner server, and it's much slower than the Mac. But I think mostly because the memory is much slower. So… it's hard for me to give, right now, precise, requirements, but we can try to write them down for the next call, probably. But I think 16 cores is, A good idea.

Thomas Coratger: Yeah, and I feel 80's too small, no?

T. Wambsgans: Yeah. And about XMSS, basically, it depends on the… framework in which you do the security proof, if it's the standard model, which is conservative, like, the most conservative you can have, or if you assume Poseidon behaves… in some way, like, as a random oracle, you can get tighter and more efficient XMSS, which… also has the advantage of being below 1 MTU and IPv6 MTU. And, but, like, the question of which security framework to use is not consensual among the experts, I would say. I would not, like, I'm not qualified enough to tell, but I would say among the small number of cryptographers I have asked this question. Some say it's okay, some say it's not, so it's… I would say it's not an easy question for us to answer right now. But, obviously, main…

would be much… the main branch would be much faster, so maybe one thing we could try in the future, without saying we should use the main XMSS long-term, but we could try to have a DevNet using the main XMSS, just to be able to compare how better would it be for the performance of the… the DevNet.

Thomas Coratger: Okay, and so if we are pragmatic, why not, like. First, trying to increase the hardware requirement a bit. Trying to simulate as is right now with the DevNet 4, DevNet 5, whatever branch, with the conservative assumptions that we have right now, so we changed nothing about XMSS. We try to debug and everything, we try to make it stable, we try to see if the performance are good enough, let us say, with this slow branch, let us say, and if we cannot manage this, and if this is an absolute blocker. then we can circle back and discuss, like, basically you, Justine. Benedict and myself, maybe, to see if this is a good idea or not, like, to…

to go back… to go to the main branch. This is something that we can do that way. For, yeah, maybe Anchal, you had also a comment?

Anshal Shukla: Yeah, so I was thinking that we're putting a lot of focus on the aggregator right now, but I think… The culprit here is, like, the validators, because on increasing it to 4 subnets, we have, like, for each attestation data, we have to, while doing the block production, we have to aggregate all these four, different proofs into the one… into a single proof, and we had… right now, we have that max attestation limit of 16 and DevNet 4. So… in every block production, we'll be basically trying to aggregate these 64 different chunks, maybe 8 of them are parallelized, 16 of them are parallelized, but I don't think, like, aggregator is really the culprit here, because if that's the case, then

in a single subnet DevNet, it has to perform, like, the majority of the work, but it works fine with a single subnet scenario. It works fine in a dual subnet scenario. It doesn't work fine in, like, 3 or 4 subnet scenario. That basically indicates that the block production Is not happening properly. Maybe it's missing slots, or it is, like, propagating the blocks too late. So, I think one easy way to do it is that, obviously, we can increase the, configuration, but I think we should try decreasing the max attestation, here, and maybe do it like a 2 or a 4 thing, that will also help a lot, because then it will be basically a multiplier of the number of subnets into, like, the max attestation data that we are trying to do. Yeah.

Thomas Coratger: just before Gajinder, like, to circle back to your point, I totally agree, and that's why I think that this is a good thing, that we don't modify kind of new things into the spec, and that you guys have time, like, to To debug and, like, properly study the matrix and, like, the logs and stuff like this, because the worst thing that could happen right now is that we spend a bunch of time just debugging, for example, the aggregators, and we spend, like, one month, two months just for the aggregators. And if the aggregator at the end is not the problem, like, this is a big issue, so we should really debug things properly. Of course, if the aggregator is a problem, then we should tackle this problem and think about it, but

First, yeah, I would love, like, to reach a certain point of stability where we are sure that we identified properly the problem. Yeah, good, Jinder.

Gajinder Singh: Yeah, Thomas, so I basically agree to you. We basically need metrics for it, and I think we have metric on aggregator performance as such. So that is there, then we will just be our… Metric for the aggregation coverage itself, so that we also know if justification and finalization is not happening, what is going on in the network, and which basically subnet is the culprit behind it. Another thing that I wanted to say was, so, yes, basically we need to define that… what were… how many ag… how many aggregations proposer might be doing, because since it's getting aggregates from different aggregators, and it might need to merge them to make a proposal, so what is that expected number? And I think that expected number right now goes by max attestations, which is

Max attestations is not the expected number, because there can be multiple Aggregates with the same, attestation data that might need to… that might just merge into one entry into attestations for block, right? So we basically need to say that, okay, this is the max number of… aggregator payloads that a proposer might be able to handle, and this doesn't need to be a hard limit, because Depending upon the valid data being run on a big machine or a small machine, they can basically say that, okay, this is something we can try to maximize our block, production, or we basically choose to say that, okay. we will leave some of the votes out. So this is something I think can be done on a client basis, but on a typical basis, we should base… we should figure out what is the expectation that we have from a validator, and then, once we have this, then what we need to then put on the top is that…

If we have aggregators that… that are producing a lot of payloads, right, in different subnets with the same attestation data, then we need another round of aggregation on top of that. So one thing to… one thing to basically do is make an aggregator subscribe to multiple subnets, and obviously the load will increase. But other thing to do is that, okay, as an aggregator, there can be another aggregator which aggregates the same, same attestation data payloads, so that less of the work will need to be done by the proposal. And I think we… as we go forward, we will basically also need to figure out this topology. of what we expect in a typical network to happen, and what will basically be the hardware requirements for that. So, this is something that we might also need to do, that an aggregator

Who is… who is an aggregator, who is basically, you know. listening to multiple subnet aggregates. They might just aggregate it and republish it, but in this particular sense, we might Actually need to introduce a little bit more time. on the aggregation… aggregate payloads transmission, because there is an additional delta that comes in between. So, if we are basically going for basically having this flexibility, and I think we should have this flexibility, where there is an aggregator, or there are aggregators which are aggregating on top of the aggregates that are being produced across different subnets, and we introduce… we need to introduce another delta in between, and I think another delta will take about 1 second or 1.2 second

To basically make this happen. So we… we need… we might need an additional interval of the sorts. And it does not need to be a total, an exclusive interval. It basically needs to be… so the So the aggregation, payload transmission interval needs… To be bigger in that particular sense, to have this particular kind of topology, without increasing the number of intervals.

Thomas Coratger: Yeah, Mega, please.

Mega | Lambda: Yeah, I heard you guys mention the… That block building might be a problem. We have a PR in Lambda improving the block building algorithms. We've already been running some devnets with that, and we now see justification and finality Improving soon. we can… We can make a PR to lean spec. Just share it so other teams can benefit from it.

Thomas Coratger: Yeah, sure. So… Okay, so if I summarize correctly, what we should do next is to try to, Stabilize a bit. take the time to ingest the new DevNet 5 PR that we have made in the spec. Try maybe to increase a bit, the hardware requirements. or not… We can… we can see, for the… the aggregators, maybe, 8-core is not enough. then we will try to isolate the problem. that we have right now with the DevNets, by adding maybe more metrics and much… a bit more experiment with multiple clients. Then we can find what the problem is, if this is the aggregation of the post-quantum signature, if this is the aggregator's role, if this is the hardware requirement, or if this is the block building, or if this is something else.

And once we have this information, we can take appropriate measures. I don't know if you agree with this plan, but this is my proposal.

Gajinder Singh: Yeah, this sounds good. So we will do two PRs by tomorrow, one to basically do the aggregate coverage, and then to further add metrics on the proposer aggregation, that… that happens, because I think these two seems like Where we can find a lot of, juice to optimize.

Thomas Coratger: Okay, great. And in the meantime, myself, I will, I will first go to Zurich, give the presentation, and then I will, work more on the spec, to simplify anything that I can in a way that we can have, like, hopefully by a couple of weeks to one or two months maximum. A much, like, broader, feedback other people, about the spec. I am thinking about, other contentious client teams about internal people within the EF that are, like, prospecters for years now, yeah. So let's do that. Do you have anything else to discuss? Yeah, let me go, please.

Mega | Lambda: not really a discussion, but I had… we had a demo of the EL integration we made, or more, like, just some things to show.

Thomas Coratger: Yeah, sure.

Mega | Lambda: Okay, give me a sec. others here. Let me share my screen. There we go. So, this is two parts. We basically, first of all, we made a… a, an addition to Ethereum package to support lean… lean clients. And we also, use that to try to run with Lambda with execution clients. We made something like what I think she proposed for… for, for DevNet 6. We basically copied part of the Beacon spec to… to, to support Yields through the engine API with execution payloads traveling in consensus blocks.

To start, we… We… I'm gonna show a bit about the… the Ethereum package integration we have. We have this package in our… in our lambdaclass fork of Ethereum package. It's number 19, so if anyone's interested. And I have here… we have the… this DevNet, the link is public, if… If Pablo can share it on… In the chat. We have… these, this, DevNet running with Kurtosis. This is single subnet, single machine. So, nothing fancy on that, but we have running all clients here. Emm… On this, public domain. This is running on Kurtosis, and the configuration we are using is this one.

Here we have it lambda is 2 times, because we were using it as a grader, but… The rest are just… Mmm… Two of each client. here we have similar parameters to the… those we use on Validator config, the Genesis delay, the active epoch for the XMSS keys. And also the number of subnets. M… I think here we have some, yeah, some information on how to run each of these, each of these examples. And… well, that's it for the disintegration. We also have the… integration between Lambda and execution and the execution client. We… these images are for ethrex, but we have it running with any execution client. An Ethlambda. this DevNet is not public, but we have it here. As you can see, we have… I think it's…

12, the longest? Yeah. called Trouble with Lambdas running with execution Clients. Ew. the… The configuration we are using is this one. We have ethrex with Ethlambda, ethrex with Lambda, Nethermind, Geth, Erigon, Nimbus, and Besu. I'm going to show some vlogs also. This is… this is more interesting here. Let's see… We can see here that it's processing blocks execution, nearly at 4 seconds… every 4 seconds. You should see some fork choice updated messages. Oof. Dude Here I have… Yeah, basically these ones, full choice updated, includes payload attributes, mentioning the… that… that the…

Ethlambda is asking the EL for a… For a new payload, for its, for proposing. And we can see that it's requesting, then it, asks it back. And later it provides you through a network. You can see maybe that on Ethlambda Logs. In Ethlambda, no. Yeah, anyone is fine, I think. It's gonna be fun.

There we go. New block. Okay, well, if Lambda here is processing blocks. I don't think we expose more data on execution, but… It's already on slot 10,000. Yeah. No, you're empowering, thank you. Okay, I think… that's it. For the demo? So, for next steps, we don't still have an execution requests on this integration. We also need to write this,

get this down to a proposal, write it in a proposal for this integration to be added on some DevNet. And lastly, we still need to work on the machine dev nets for… for lean and execution clients. Hmm, that's it.

Thomas Coratger: Amazing, and so you linked, like, you linked the CL and the EL directly into East Lambda? Do you have, like, a specific branch for this, or…

Mega | Lambda: Yeah, we have the APR in our repo.

Thomas Coratger: Okay. Okay, that's fine.

Mega | Lambda: Yeah, we used the engine API, so most… we didn't have to touch ethics or any execution.

Thomas Coratger: Okay, that's great. Oh, yeah. Okay, this is the PR that you're linked. Okay, like, the size of the PR looks pretty reasonable. Cool. Yeah, probably when we link the two, we should take inspiration from this. Yeah, that seems great. Any question about this, from other people there? Anything to add? Okay, thanks a lot, guys, so if nothing else is coming… Then see you next week, and on the chat. Maybe if you have time, like, you can just post the link of the PR on the chat so that people can check that async.

After the call. Yeah, okay, thank you. See you next week. Bye-bye.

T. Wambsgans: Right?

Kolby ML: Bye-bye.

Gajinder Singh: That's true.

Pablo Deymo: My wife's here.

Mega | Lambda: Right.

Anshal Shukla: By this.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.