All Core Devs - Testing #69

Transcript

Spencer Taylor-Brown: Hey.

Leo Lara: go… Hello?

Mario Vega: Hi everyone, let's just give one more minute to everyone before we can start. Thank you.

I think we can get started.

Akash | ECH: Realize.

Mario Vega: Thank you, Akash. Good day, everyone. Welcome to ACDT number 69. Today is February 9th of 2026. Yeah, let's just get started. First item in the agenda is Glamsterdam, and the first sub-item is the log-level access list step number 2. Stefan, would you want to go to the overview? Of the current status of the debit? Alright. mentions that the audio issues.

I think he's rejoining, let's just give… give it a minute. Anybody else wants to chime in in the meantime? Regarding them, the two…

Barnabas Busa: Maybe I can give a quick update, while Stefan figures out his, mic issues. So, we launched the .NET last week, Friday. We initially made a bit of a oopsie configuring the network, and that's why we had to make a 1000ml key deposit. We initially configured the whole chain with only 1,000 deleters, and Right now, I think we have some syncing issues on Nethermind. Nethermind has been taking a look and trying to figure out, What is the issue? Maybe they can give us an update next. As far as I know, everybody else should be okay. Oh, no, Nimbus EL as well. Nimbus EL with Lighthouse also seems to be, falling behind. So maybe they can also give us an update, but everyone else seems to be ahead.

Mario Vega: Thank you. Anywhere from Nethermind, that can give an update.

Marc Harvey-Hill: Yeah, sure, so, yeah, we're investigating this, syncing issue, and just generally working on our, BALS implementation and getting the rest of the, the test passing. Yeah, that's right.

Mario Vega: Alright, thank you. Anyone from Nimbus EL? You can chime in?

Dustin Brody: Yeah. Sure. So… This is the current… hypothesis, the current… it seems to be an issue with the finalized block I'm sort of relaying here, but, that… the… for the block access list to be present for… that… Nimbus currently requires it to be present for the non-finalized blocks, and this is something that is apparently not necessarily the case. I will admit, I'm not the person who's been directly looking into the details of this, so… But we are engaging here, and, that's… the, the current. it's become… it's become a bit of a discussion of, like, what the correct behavior should be, basically. And we can tweak this either way.

Mario Vega: Alright, is this something that you need to bring up in the, in the block-level access list specific discussion, or do you think it's a spec problem, or… is what I mean? Or is this a client issue?

Dustin Brody: I… I wouldn't re- I… My understanding is that the spec says what it… I don't… I don't think that we're sort of disputing the spec at this point. This is, this is our… so… I wouldn't say that quite yet. To Jared's point, yeah, so that Nimbus does require that. And… that they're not… It should be, yes, and… and the… Bye. So… I… I… we may… it may get to the point where it's worth racing. In that sense.

Mario Vega: But.

Dustin Brody: But I don't think it's escalated that far yet.

Mario Vega: Alright, thank you. Thank you, Dustin. Stefan, I see that you're back online. We went through the details and the issues that we had. Do you have anything else, too, that you'd like to point out?

Stefan Starflinger: Just maybe if clients could double-check if we can run with 150 million guests, I'm not sure if that was, discussed. So currently I've set the network to 150 million guests, but this might cause issues. So it would be nice if clients could double-check this. I haven't heard back regarding that yet. But otherwise, I might have to decrease it again, but I don't think it has caused any issues so far.

Mario Vega: Yes, Tony.

Toni Wahrstätter: Yeah, I'm not sure if the 10MB limit or any of those size limits, still matter. But with 150 million, we would definitely get over the 10MB uncomprompressed payload size. So, shouldn't be an issue if we just avoid, spamming big, big data transactions. So, just as a heads up, maybe let's avoid that.

Ben Adams: If it was over… 10 megabytes, then it would be an invalid block, because we've already put that. Oh, really?

Toni Wahrstätter: Yeah, okay, this is good to know, because then we should definitely avoid it. Because as of 104, 105 million. We can get over… 10 megabytes, if we don't include, and we didn't do so, we didn't include 7976, which would increase the call data cost, then we should be fine again, but for DevNet2, we don't have that ERP yet, so… Better, let's not put big data transactions on there.

Ben Adams: I mean, it'll be fine, because they… you won't be able to build a block more than 10 megabytes, because it's an invalid block. We put that in Fusaka.

Toni Wahrstätter: Right. Okay, perfect.

Stefan Starflinger: Is that something we should try out?

Ben Adams: Yeah, sure, but I mean, it does… It means that the, The… the blocks will be constrained by the cap on the megabyte size. Which will skew the gas markets when it… it should actually be done on, you know, pricing. It's more of a safety measure. It's… ideally, we wouldn't be hitting the cap, but it should be safe to put whatever… go to whatever gas limit you want and not break the network.

Stefan Starflinger: Okay, that's good to know. Maybe other clients, if they have some insight on this.

Mario Vega: Jared, I saw you raise your hand. Do you have any… Any, input, or was it just a, an.

Jared Wasinger: Yeah, I was basically just gonna say that, like, we shouldn't be building, like Ben said, if we… if the clients do build a block that it goes over the size limit, it's a bad block, so… We shouldn't be avoiding parameters which would cause that to happen in invalid implementations, and that… If that issue could be surfaced in a client, it should be tested here.

Mario Vega: Yeah, I agree.

Dustin Brody: One consider… One consideration on that… Oh, sorry, I didn't mean to interrupt, I was waiting for a silence or a pause. Okay. One consideration there regarding… I mean, I don't fundamentally disagree, I mean, that, that, sure. you know, the answer to don't create invalid blocks is don't create invalid blocks. But the… there's a… packaging point. Now, it becomes less acute with EPBS, but 2 points on EPBS. One, the EF is still making noises about if it's, like, too delayed, it won't be there. Secondly, it doesn't completely remove this issue, because the envelope has some overhead. on the CL side. This is what that structure is called over there. That contains execution, payload and stuff, and the ball stuff as well.

So, what this means is that there's, a pay… a block that looks valid from the EL side that is not valid on the CL side. It cannot be broadcast, or etc. on the CL side.

Ben Adams: That's right. The cap includes a margin for the CL. So it's… the EL… I don't know if it's, like, the EL can only produce an 8MB block. So that this CL component. Okay. An envelope around it.

Dustin Brody: Okay, that's… and that… that… that addresses slash resolves that, that's fine then. Okay.

Ben Adams: Yeah, that's in… that's already in Fusaka, so, it's live on my net. But obviously, we still need… we still need adjustments to the call data pricing, because We don't want the… the size cap to determine the blocks. We want the pricing, because otherwise the gas market is… Because… All wrong. If you're… if you're determining pricing by the cap, rather than You know, the actual prices.

Dustin Brody: Yeah, no, I agree. I want to address Marius has questioned a moment, too, because he was obviously responding to a thing I just said. And, but the… and maybe mumbled my way through, I don't know. But the, I… but in terms of ensuring the gas market is not… not distorted by this, I think the more principled solution that I think actually… I'm not saying this will have this in Fusaka… we don't have it in Fusaka, I'm not saying we'll have it in Glamsterdam, I'm not… this is not necessarily a near-term thing. But the most principled solution I saw was, I think, actually Yossix, yes, also Nimbus, but, you know, you know, I disagree on plenty of stuff. That… that we're…

the gas price, and where the blocks… the max block size is determined by the gas price. It's a function… there are mutual sort of functions of each other in a certain sense, and the… there are variations of this idea. But the idea is that you don't have this other kind of cap that… you change one parameter, then oops, you hit some limit. All right, And that avoids, avoids us, but we don't have it yet, and we don't have it for Glamsterdam. But the, the deadline, oh yeah, somebody was… right, right, right, that was asked and answered. Yes. Basically, if EPBS looked like it was missing deadlines, then… then there were noises made on other ACD calls about dropping EPBS.

And I only bring that up here to say, well, that any planning now around block size limits or overhead or something can't take into account yet the reduced overhead from splitting the CL block and the EL payload. Anyway, yeah.

Mario Vega: Yeah, Dragon?

Dragan Rakita: Yeah, I had a question about hitting that limit, size limit, 10 megabytes in the blog building. How is that done on NetherMye's side? Does it basically check the size after the block is built and validates it?

Ben Adams: We check it as we're adding transactions, but it's not too…

Dragan Rakita: So that includes both the transactions and receipts, and both.

Ben Adams: I'm not sure what we're doing with… False.

Dragan Rakita: Yeah, both problematic, prob- prob- probably. Either way, just wanted to… focus a little bit here, because it can be… it's prob… every client probably has their case around this, how they, like, measure the block size while they're building it. It's very easy to, like, check it after it's built, but… Invalidated, or just stopping at some point of time after some intersection, the block is big enough. It's probably very familiar-specific. So, either way, if we can go to, like, 100 million guess, and we are sure that we don't go over 10 megabytes with that. Just go with, like, 100 million. And then bumped it again when proper IP for it could lands.

Mario Vega: Thank you. Maurice?

Marius van der Wijden: Yeah, I wanted to respond to… The point that, I keep blanking on the name… That someone was making that, this extra cap, should be, should not be there, or should, or it impacts the… impacts the gas calculator… the gas calculator, and that is… Yes, but the idea of this cap was that this is like a stopgap solution. This should never be hit. In practice, And even now, it should not be hit. Except if there is, adversarial transaction load on the network, and I think it is fine to simulate that on… on… on DevNet2.

And… In theory, nothing PM should be broken with it, but, Yeah, if we… if we want to… if we want to focus on… on, having this kind of a stable network, and not break it, then I don't… I don't know if there's really a need to. test this on DevNet too, but as… as Tony said, that we… We will increase the, We will increase this, the call data size anyway, or we will need to increase the call data size anyway if we want to go significantly beyond 100 million GAS. So… I think… doesn't… Yeah. Doesn't really make sense… sense for me now to kind of… Test this condition that is…

that we are hopefully never going to hit, on a DevNet, just to test it, and then potentially break the DevNet and kind of stall our… our progress on testing these other things that we should be testing on this DevNet.

Mario Vega: Thank you. So yeah, it seems like the sentiment or consensus is to lower the gas limit for now. Stefan, is that an issue, or should we be okay?

Stefan Starflinger: That's not a problem. We can lower it. It's just, the request came in in the breakout call, to raise the gas limit so that we can increase the state, so it's easier to do some benchmarking. That's basically the motivation for it.

Mario Vega: Yeah, I see two… two numbers in the chat, it's 70 million and 100 million. Should we decide on one, or push it to 70 and then push it to 100? Is that okay? Yeah, Tony?

Stefan Starflinger: Yep.

Mario Vega: Okay.

Toni Wahrstätter: Yeah, no, I just wanted to agree, I think it's fine. We can, we can go with either. It would be great if we, at least at the latest until DevNetly, if we have, like, a clear understanding of where this new 10MB limit applies, I guess. the neutron, the new, eFlog EAP might also affect us, because suddenly receipts are also a problem, it seems like, from the jet. So it would be great if we're definitely… if we are ready to increase it without having to fear that something in DevP2P might struggle.

Mario Vega: Thank you. Ben?

Ben Adams: I think the logs issue's around 107? Oh, we should be fine, not 100.

Mario Vega: Sounds good, thank you. Tony, you still have… you're having your hand up? okay, thanks. Alright, is there anything else anyone wants to raise regarding block load access list definite 2? Shall we move on to EPBS?

Stefan Starflinger: Nothing else from my side, I think Barnabas said everything.

Mario Vega: Perfect.

Toni Wahrstätter: Maybe just to add, we have a breaker call on Wednesday again, where we discuss two small changes to the specs again. Very minor things, but still worth, to highlight.

Mario Vega: Excellent, thank you, Tony. Thank you, Stefan. Alright, let's move on into ePBS.NET Zero. Justin, is it you the go-to person for this topic? Do you know the updates, or should we go into the clients for updates?

Justin Traglia: I guess I am the go-to person, but yeah, we should go to the clients first.

Mario Vega: Excellent. Alright, yeah, yeah, let's go, clients, first. So, anyone from Prysm? Can give an update on EPBS?

Potuz: I'm not sure if Terrence is here, who would be the most appropriate one, but, same as last week, we are just merging, and implementation is picking up space, pace. I should have 4 choice ready in one or two days. And most of the… most of the core transition function is already, open for review, and people are merging PRs. What's missing is the API, and this is already, like, the team is onboarded mostly on that, people are writing that, and the blockchain package to sync, which, I think I'll take care of that after Fortress.

Mario Vega: Alright, thank you. Anyone from Teku who wishes to give an update?

Stefan Bratanov: Yeah, sure, so… Basically passing the specs test from Alpha 2, And, without the frog choice test, and, We got a couple of more people working on… FOCIL now, so… Yeah, hopefully soon we can interrupt between each other, and then… DevNet Zero at some point.

Mario Vega: Great, thanks. Anywhere from Lighthouse? Alright, are anyone from Nimbus? Just to give me a win? Mimbus or Lighthouse for ePBS updates, Anyone who wishes to chime in? Yep.

Dustin Brody: Okay, well, sure, I'm not sure if I'm the best person to do this, but I'm sure. But basically, we are, We're merging, yeah, kind of along the line of what Prysm… what POTUS… what POTUS was saying about what Prysm was doing. so we have, I mean. I… I think the… the big PR that we're… the Alpha.2 spec stuff is… has been… kind of looking… finding various edge cases in that and the like, actually in the specs more than the, than the PR itself. but… and the test, but in any case, we… that works. We can… we can… I think the plan is to merge that imminently, and then… And I'll… we have the full we're passing the test, we have the full state transition function, we, in theory, are broadcasting everything. also, this is a request for clarification, I guess, is that in one of the other ACD calls, there was a comment about that

DevNet Zero would not be primarily about fork choice. But so that wouldn't be, like, the target for fork choice. That, that would be, I guess, what, DevNet1 or something, 2, I'm not sure, whatever. Is that true? What is the…

Barnabas Busa: That was, like, an almost Yeah, like, ideally, we're not going to have many forks on DevNet 0, and we're not gonna trigger any edge case scenarios, we just want to try to test basic interop between the clients, and then later on, once we have, like, an answer done DevNet2 or whatever, then we can go really into the… Harder edge cases.

Dustin Brody: Okay, because basically, our plan is to not really prioritize, the fork toy stuff at that, for DevNet Zero none. And I just wanted to Confirmed that, fine. Alright, otherwise, we're… we're getting fairly close to… Close, closer, closer to ready, I would say, for this. Not… not any huge missing parts.

Mario Vega: Great, thank you.

Dustin Brody: I do have one question. This is… I promise, a question, and then I'll let you… the meeting continue, but is we are looking for a way to test this. Is there a way of kurtosis, the same way that, And for… Barnabas has, has, gotten the CLs. To implement, you know, a fake fork. That… that will probably never exist for… for… For balls, which, fine, okay, that means balls can be tested more easily, but the converse of this. something… is there… is there an EL? analog here, so we can, like, kurtosis test this stuff.

ashah: And then I remembered you said.

Dustin Brody: Okay.

Mario Vega: I think that was, there was a, That wasn't intentional. Doesn't sorry.

Dustin Brody: He's…

Mario Vega: Clarify your question again.

Dustin Brody: Yeah, oh, kind of, I think Barnabas may have responded to this already, like, ePBS at 0. Basically, looking for an EL branch to do something, like, whether it's kurtosis, but in general, an EL to pair with the CLs. For testing.

Barnabas Busa: That's my, my, my personally biggest concern. Sorry, go ahead, go ahead, go ahead. That should be the image to use right now. What do you?

Dustin Brody: Okay. Okay. Because that's otherwise just my biggest concern, it's like, well, we have all this stuff theoretically there, but, you know, yeah. Alright, great.

Mario Vega: Alright. Thank you, Dustin. Lastly, anyone from Lodestar who wishes to chime in regarding the updates?

Nico Flaig: Yeah, so we… we are passing the Alpha 2 spec tests, and have, implemented the stateful, block production APIs, and then NC can give an update on our fork choice progress. But basically, we are passing all, spec tests, so…

NC: Right, for choice. We have a couple, refactoring on our ePBS book choice implementation, so we're… Scan one step closer. to get the pro-choice PR merged. And… we also have a, a PR draft, which is a, first… Draft implementation of updating our state cache and checkpoint cache. to work with ePBS?

Mario Vega: Alright, thank you. I think that's every client. Justin, do you have anything else that we should be considering here, or… Should we move on?

Justin Traglia: Mmm, not much. We are working on adding more tests for FOCIL, and there are a few open PRs, but I'm not sure if it makes sense to discuss them, unless we want to. Such as the variable PTC deadline PR. Bye, Francesco. Are there any updates there? Do we still want to merge this?

Francesco D'Amato: At least not for me. There were some concerns that had been raised by Anders about more, like, future compatibility of this, which I don't think there's… you know, really anything to say about, in the sense there's not really progress on this question. Other than that, I'm not sure. I haven't really had time to think about it more deeply. Like, right now, I wouldn't feel comfortable going ahead and merging it, but…

Justin Traglia: Okay. We'll continue to work on it then. And that's pretty much it. I think every other PR we can discuss offline.

Mario Vega: Excellent. Thank you so much. Alright, let's move on. We have 4 other topics in gas limit testing. Yeah, let's just get going. The F70, updates, the new protocol, do we have, a go-to person who can speak a little bit more about it? I think if that… now I'm not mistaken, it's a FLCL, right? the person to… Discuss this? Or do we have any other go-to person for F17? Discussion topic? So, I can…

Fabio Di Fabio: say that in Bezu, the ETH70 implementation is, Ready. And I'm starting to test it. I need to also start to test it against other clients that I implemented it. Mmm… Yeah, it's taking a bit more time, because we are also refactoring the sync. Part of the code, but… Otherwise, the implementation should be fine. On our website.

Mario Vega: Thank you. Maurice?

Marius van der Wijden: Yes, as I have already wrote in the chat. We are… PK is also trying to test it. We have implementations for, Besu, Nethermind, and Geth. Maybe, PK, you can quickly talk about the state? off testing?

pk910: That's the ETH70 test, right?

Marius van der Wijden: Yep.

pk910: Yep. Yeah, we've prepared a, Kurtosis pack, or, Kurtosis config for it, In theory, it should work that we are sending… that we are generating blocks with receipts bigger than 10MB, and We assumed that clients that do not support ETH70 at that point are unable to sync. this chain. Right now, they are still able to sync, we are not fully sure why, but, yeah, it's ongoing. Path forward is probably that we are probably set up a full testnet for it, because kurtosis has some limitations in regards to changing flags and restarting clients. yeah.

Marius van der Wijden: Yeah, I think the problem is that you can always full-sync the chain, right? Like, you never sync the receipts when you full sync the chain, you only sync the receipts when you are snap-syncing, so…

pk910: Yep, that makes sense, yeah. But even then, we had problems with restarting Geth with SnapSync on a… Kratos is DevNet, so…

Marius van der Wijden: Yeah, it probably just makes…

pk910: More sense to set up a full testnet for that. That's all.

Marius van der Wijden: That's a different problem, Geth is bad at snap-syncing a chain that is not Alive yet, or starting from… Like, I think from zero. And then snap syncing. Yes. Could we prioritize this? ETH70 DevNet a bit, because, this is the biggest blocker to scaling right now. Without this, we cannot go beyond. EIP. Brilliant, so… Would be nice to get this over the finish line. And we… we need to… we… we need to wait until the fork point anyway, because all clients need… or all nodes need to be updated. If we want to increase the mainnet gas limit, so, the… Yeah. The sooner we get to it, the more time we have for other EAPs. We can't just schedule it for definite 3. I think that would be possible as well. But maybe…

But just testing it in isolation would be actually, better.

Mario Vega: With designated isolation, do you mean, like, a sole tablet just for this?

Marius van der Wijden: Yes, I think.

Mario Vega: Alright, because Barnabas is suggesting that we add it to DevNet3, so that's… Something you don't agree with?

Marius van der Wijden: I… I personally think it would be easier and nicer to test this in isolation, and then later on, maybe add it to DevNet 3, or DevNet 4, or whenever, at some point, like, add it into the definite cycle, than starting the first test on the definite cycle. On the definite, on, on the, bell, definite cycle, because… We have so many interacting peaches now, and It makes it… doesn't make it easy to… Kind of merge the stuff upstream if we keep piling stuff on top of each other.

Mario Vega: PK, Barnabas, any thoughts on this?

Barnabas Busa: Yeah, we can get the new DevNet up and running probably mid-week. Or even earlier.

Mario Vega: Alright, I think the only missing bit from this is the client's readiness. Should we discuss this async and just, reach out to you, Barnabas, or PK for the clients that are ready for them to be included? Is that okay? Okay, sounds good. Alright, thanks, thanks for this. Let's move on. So the next topic is benchmarking updates. So for this, Camille and also Luis left a comment on the agenda. Camille, would you like… let me see if Camille's online. Yeah, Camille, would you like to go first?

Kamil Chodoła: Yeah, I'm not sure if I left any comment. Are you sure it was me?

Mario Vega: No, sorry, sorry, I was mistaken. Do you wish to give an update, or should we go to Louis?

Kamil Chodoła: Yeah. Yeah, I can give an update then. So, currently we are focusing super heavily on all of the benchmarks for repricings. I guess we have already a lot of test cases, especially compute part is well covered, and we generated tests. Maria had a first pass over the compute scenarios, and we have some first numbers. For stateful, tests are still coming, like, Carlos Perez made a lot of tests together with Johan, and right now we are generating them. Also, we spent quite a lot of time together with, who it was. forgot the name of EF… Rafael, working on Benchmarkor together with, Gas Benchmarks generated tests to see if everything's working properly. Also made a extra work to make the reorgs working here, so it is easier to test basically any test, to run any test, instead of running all of them in proper order to reach one specific scenario, and it works well right now.

And from important updates, Marching made a draft implementation of testing endpoint inside Nethermind, and we integrated it into GAS Benchmarks, and it proves that it is much better than just using some hacked version of Nethermind. Test generation is much, much faster, much more flexible. And yeah, also thanks to that endpoint, we were able to make this reorging mechanism on top of stateful tests working. So yeah, so right now we are focusing again on regeneration of the tests, waiting for new infra to run the tests on top of mainnet and Bloatnet. And once that will be done and traced, Maria will give a second round of analysis. Hopefully, it will be ready by… Tomorrow. with data collected. But the blocker right now is the infrastructure.

Mario Vega: Got it. Thanks, thanks for the update. We do have a benchmark demo, but I will leave that until after gas pricing updates. And lastly, for the benchmarking updates, Luis, do you want to expand a little bit on your comment in the agenda? Is that okay?

Louis Tsai: Yes, can you hear me?

Mario Vega: Yes.

Louis Tsai: Okay, I would like to quickly share some status updates from the Steel team regarding the benchmarking progress. So the first thing is that we have published our latest benchmark release, which now supports Osaka Fork. And the second thing is that, we have updated our benchmark documentation, which includes the current project structure of yours benchmark, and different modes of benchmark we're currently supporting, and the corresponding test generation workflow. And also, it includes the instruction on how to run and write the benchmark tests. I have shared these two resources to the chat. Please feel free to give us feedback, and let me know if there is anything unclear. That's it for my site. Thank you.

Mario Vega: Excellent, thank you. All right, let's move on. Maria, the next, I think you're the go-to person for, gas repricing. Would you like to give us an overview update?

Maria Silva: Sure, happy to. Can you hear me alright?

Mario Vega: Yes, perfectly, thank you.

Maria Silva: Okay, so, we did the first breakout last week. The notes and summary is already on forecast. If you want to, review it. Also, If you want to keep updated on what we are working on these, we have these to-do lists that I'm sharing in the chat, and I'm keeping it as much up-to-date as possible, so it has a description of all the work streams, including benchmarking and the general timeline for when we need. things. The key updates there was on… the compute repricing, so 7904, there's the… the… the PR for the new EIP. numbers are there, and this is based on the first round of benchmarks that Camille was talking about previously, so we already have the set of

operations that we are suggesting to reprice under that TIP, and also, some preliminary numbers that then still need to be updated once we have, benchmarking for block-level access list-optimized clients. So those numbers are not yet final. And then, on all the other EIPs, so on the state access EIP, we still need the work on the benchmarks to be completed, so that we can have the first numbers. Once we do that, we'll update the EIP as well. For the state growth EIP, So, AT37, we've started to do, more work on, on the specs and, and answering, some open questions on how to implement some details. And we have… Francesco did an initial pass on the specs, Spencer is starting to work on,

on, picking up from Francesco and making sure that everything is implemented, and I'm also, updating the EIP. to give more information on the actual details of how we do the multidimensional metering in that TIP, so the… I will, the PR is not open yet, but I'm… I'm almost finishing up, so it should be ready by the end of the day. For… the ETH transfers EIP, so to 780, we, we still, will do… we still need to do some benchmarking there to… to get the final numbers as well. And we have, some work starting as well on specs and tests. And finally, for the data EIPs, so increasing the costs, Tony has done the PR on the specs.

And, so I think that that EIP is the one that is more advanced, and also Tony published a proposal to change the base numbers to 6464 for the, the the worst-case blocks, when the blocks are using most call data. So this should also help, Reduce the… The worst blocks there. So yeah, so this is the general overview. If you have any questions, just let me know, and… yeah, we'll try to keep the to-dos as much updated as possible, so you are also aware of what's happening. And if you have any comments, if you want to work more on these EIPs, just, you can come to the breakout, which will be next week, Wednesday. And, yeah, we're happy to… to have you there.

Mario Vega: Excellent, thank you, thank you so much. Any questions for Maria regarding the gas reprisings? Alright, if not, thank you, Maria, again. Oh, Dragon.

Dragan Rakita: Sorry, I'm a bit late on raising the hand. Do you have plan on bot DevNet? Do we expect this to, like, land? Or basically, is it expected to, like, implement everything for the DevNet, or can be partially implemented, or something like that? It seems to look.

Maria Silva: Yeah.

Dragan Rakita: A lot of changes there, so, yeah.

Maria Silva: Yeah, so that's… that's a great question, and I think it's also a point I raised in… as a comment that I wanted to get to, and I'm not sure if this is the right place to have the discussion, but in my mind, the ideal thing would be And I have this, so if you open the to-dos, I have a general timeline of when things will be ready. And so, for all the EIPs, excluding the state growth EIPs, so 1837, so that one we could do earlier, for all the others. They do depend on benchmarking things and their block level. least, optimized clients, and so we only expect to have the final numbers by the end of March. So that means that

it would make sense to add those EIPs after we have those final numbers. However, for 8037, given that first this is the most complex EIP implementation-wise. I think it would be better to have it sooner, so ideally we would have it in the next DevNet. And then we could add all the others. in the DevNet after. And I think that would be okay, because all of the others are mostly doing just perimeter changes. An open question is what we do with 2780, which also has some implementation logic. Not just the… Changing the parameters, and then we could discuss, maybe it may say… makes sense to add them both in DevNet 3, and then leave the… all the others for the next DevNet.

I think for the data repricing, it may also make sense to add it a bit earlier, just because of the discussion we had previously, so it would help us test higher block limits without getting the bottleneck of the… of the size of the block. But yeah, so this is, the general… I think what would make sense to do is essentially do, the state growth EIP as soon as possible, and then we have a bit more leeway for the others, and, especially for the compute reprice and the state access reprice, I think we should only add it after March.

Dragan Rakita: Okay, sounds good. So, internal, We depend… state taxes and state creation depend on the ball, basically on the ball data, how it will behave, so we need to wait there. Exactly. Another IP needs Could be implemented before, but it will depend on, like, Yep. If… Clients have time, or basically… organization in general. Okay, cool. Exactly.

Maria Silva: Exactly, exactly, yeah. But I would argue to prioritize it, like, to make it the state growth EIP, so 8037, to make it a priority for the next DevNet.

Dragan Rakita: Yeah, that's the state… State creation gas cost increase.

Maria Silva: Exactly, exactly. Okay, cool.

Mario Vega: Alright, but yeah, no, I don't think there's, this is the time for making a decision, because we're still, like, running DevNet 2, so I think if we can bring up that discussion at a later call, it would be… It would be great, just to… See how we can coordinate. Alright. Thank you. So then, thanks, Maria, and thanks, thanks, Rogan. Yeah, for the last topic of today, we have a benchmarker update, and demo, sorry. I think Rafael is the go-to person here? Yeah, thank you.

Rafael Matias: Yes. Yeah, I mean, it has been mentioned already by Camille, Benchmarker, so what is Benchmarker? In the past, like, NetherMite has been running these gas benchmarks. And Benchmarker is highly inspired by this, like, at least on the approach. How to basically execute EL benchmarks to, like. verify the MGAS per second, and so on, and see which tests are, like, the slowest, and so on. So we created Benchmarker, which is under the ETH PandaOps Benchmarker repo. It is able to… so one thing it does not do is, like, the test generation, so that this up to the, EEST team. to EEST, so it can consume, like, there are East fixtures for benchmarking, it can consume those fixtures, it can also just consume raw JSON files that, like, do engine calls.

new payloads and FCUs. And yeah, then it… you basically have, like, these set of tests. For example, the release that was mentioned before by, I think, Lewis, they released, this version. And, Thanks for the shout-out for the logo. Yeah, they released this version, so Benchmarker can directly, consume this, I think it's a zip file. And parse the tests, and yeah. And it can also do, like, raw JSON false, as I mentioned before. And let me actually show you, like… it's very configurable, so yeah, for all… there's a YAML file, of course, that you can, like, configure. One thing you can say, for example, I'm gonna show you this example, and we're going to execute this.

is, you can say where the test source is from. In this case, we are, like, using this raw JSON approach, because we are doing some stateful tests. But in the future, we want to probably have, like, these East fixtures, like, as the standard for everything. And, yeah, it kind of looks like this, like, test source, that you tell it. You can also, like, you know, tell… give some specific Genesis files to the clients, resource limits, this is important because there's, like, this EIP, like, for these standard specs for the… for an Ethereum node, so we want to, like, be very close to that. So we can do resource limits, also on the CPU clock. And,

one thing… this thing can run, like, stateless sets, so you don't have, like, any, local state, but an interesting part is also it can also run, stateful tests, or, like, tests against, like, some existing state. And what we have done also is, in this case, you can specify, like, data directories. And these are from our snapshots. We downloaded it for PerfdevNet2, and for… I think all of the clients are only… I think it's between one… 2… 1.2 and 1.6 terabytes. On each client, like, on the data deer size, I have them locally, like, for testing purposes, and you basically can define where they are. So Benchmarker will pick this up. We support multi… like, CFS, this is, like, a mechanism for file system snapshots.

Because every time when you run a test, you don't, you know, you don't want to, like, copy the data yet somewhere else, because otherwise this is going to take a lot of time just copying. So we do support CFS, Overlay FS, and… yeah, raw copy if you want to do that. And then you have, like, the clients themselves, right? You can… yeah, images, argument, extra arguments, NVARs, whatever. You can do all of that, and we are actually gonna run this one, just to see… how this works. So I'm saying, yeah, run the config file, and I'm just limiting it to guess, because otherwise this is gonna take… A bit longer. And this thing is gonna execute, so it's gonna fetch those tests that we referenced there. It runs them against the engine API, and at the end, it will basically create a set of files. It will dump a bunch of files into a results directory.

And while this is running… what is gonna happen afterwards… so after we have those results, I think they appear in here. Yeah, there. But afterwards, the interesting part is… Yeah, you want to see the logo big again, right? No, let me go here. So there's also a UI that is able to, like, render the… the results afterwards. So we saw the most recent one was, like, a Reth run. Let me just… let's see if it finished. Oh, there's a new one, right? This one… Yeah, it was now, at least based on my clock. And let's actually check the console again, just to verify that it finished. Yeah, it finished, okay. So if we then, like, click on this, you can see the run details. So you see that it ran 70 tests, you can, like, see little boxes here.

There's, like, this mega gas limit that… I don't know if we want to go giga gas. This doesn't look good. So we have to improve. There's also, yeah, more, like, the configuration, which is always important, like, for people, they ask, like, how did you run this, right? So it's kind of clear, you see it here. Also very important, when something does not work, you want to see logs, right? So you can, like. Press here, and you have… in this case, it's nice because you have the benchmarker logs and the client logs in one. So you can see, kind of, what is happening. Like, the blue ones are benchmarker, and the purple ones are the client.

Yeah, and there's, yeah, the generated files that I've mentioned, it generates a bunch of them, but yeah, you don't have to worry about this, the UI kind of knows how to render these. This is also interesting. like, this graph in here. This is based on something that, Carlos did. There's, like, this, there's also a link here, so for this research. It's basically something that blocks slow blocks on the, execution clients. They will basically dump a JSON payload in their logs, and if your client implements this, then Benchmarker can also pick that up and kind of generate disk information. Also, like, caching information. Yeah, you can see cash-hit ratios and so on. What else? Yeah, you have a test… you also have additional, like, you see… yeah, all this… this table is also based on that… on that JSON payload that is being logged.

This currently just works on GEF. I think the other clients have not merged this yet. You have resource information over time, and also important, like, if, let's say, let's scroll up, right? You are like, oh, why, I don't know. let's maybe scroll this… why is this one orange, right? Why is this slow? So you can click it. You have, yeah, some more metrics. You can also see the raw requests that were done and responses. Yeah, so there's, like, a lot of information to debug. set… there's, like, test steps and setup steps. Setups are done before the test, it's just, yeah, sometimes you have… Some calls you have to do before the test. Yeah, and… There's, yeah, there's also this definition of sweet.

So you see that this test belongs to this suite. It's… a suite is just basically a set of the same tests, so that, you know, you can actually compare them between clients, like, how something looks like. This is basically, you can see more information about what it's doing, or where the test came from. And it's… yeah. It's just doing some stateful tests, and I'm just doing Xen. Yikes, just Xen. Yeah, this one. And I think that's more or less… yeah, I think when you go… yeah, this is also interesting, like, when you go to the suite page, you see the test heatmap, so you see the slowest tests across, like, all clients, Yeah, and then you can, like, go dig in deeper if you wanna.

And, yeah, at the bottom, you see every run for that suite. Then you can, like, breath. Yeah, and REST doesn't have, like, those other graphs because they don't implement the JSON log yet. Yeah, I think that's more or less it, just like a… Quick glance on this front. Yeah, if there are any questions, feel free to ask. Or message me, like, on Discord.

Mario Vega: Thank you so much, Rafael. I think we only have one question in the chat, and it is, are you getting the results on all the runs, or there is a way to have the runs for warm-up, and then use other runs for getting the metrics? This comes from Amazon.

Rafael Matias: if I'm getting the… so, I'm not sure if I understood your question correctly, But is this more, like, about optimizing the runtime? Like, doing, like, warm-up, if there are warm-up phases to them, and then, like, you know, snapshot and do, like, the test phase?

Ameziane Hamlat: Yeah, so… so this is more related to Baesu. Maybe, at some point with another mind. So when you restart base, so if you try to benchmark directly, you are getting wrong results, because Java needs some warm-up, needs to execute some methods at some level, so there are some thresholds, and when we hit those thresholds. Those methods are, are compiled to, to assembly, basically, but before they are interpreted. So we need to, to basically have that, that warm-up that is already done. to… to get… to get the results. So, Nethermind benchmarks. does that, actually, at some point. So I wanted to know if that was replicated in those benchmarks. But by the way, great work, great rate, really impressive.

Rafael Matias: Thank you. Yes, and regarding that warm-up thingy, so I also have seen that on the gas benchmarks, and yes, we can also do that, So it's basically, if I remember correctly, it's just executing a test, but generating, like, a wrong, State root hash or something on the new payload so that it kind of warms up. And yes, we can do that. It's already supported. Camille, you can correct me if I said something.

Kamil Chodoła: Yeah, exactly, just wanted to confirm, and as you've seen, for now, this is utilizing still the benchmarks which are either generated by ELs, or by gas benchmarks for stateful operations, or stateful tests in general. So, yeah, so warm-ups are still… are there, and I'm not sure, probably, it is either not yet enabled, or… there is just a small bit which needs to be added, and the benchmark should be, like, feature complete with gas benchmarks, and… extra things on top of that. So, like, this is, like, an evolution of gas benchmarks, so all the features which were before should be there.

Rafael Matias: Yes.

Ameziane Hamlat: Okay, so how about, how about stateful tests? So, when we have a snapshot. Then we, we start a node, and, on top of that snapshot, and we generate those new blocks. How does the warm-up work in this case? Work in this case.

Kamil Chodoła: So, warm-up will be still exactly the same, so, we are generating the tests on top of state, also with the usage of overlay or ZTFS, and we are generating also the… warm-ups in the exact same way. So we are stoning up the overlay on top of the database, creating bunch of, executing a bunch of tests with mismatch of state root, and saving the… saving everything into the file, so it is also still… Capable to execute in the exact same way as we did on stateless tests. So, differences… There is no difference except that those are generated on top of state.

Ameziane Hamlat: Okay, thanks.

Mario Vega: Right. Thank you, thank you, Rafael. Are there any more questions, regarding this? If not, thanks again, Rafael, for the amazing presentation. I think that's it for today. Thanks, everyone, for joining. Sorry for going over time. Alright, see you in the next call. Thank you.

Kamil Chodoła: Okay, bye.

Rafael Matias: Aye!

Louis Tsai: Bye.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.