All Core Devs - Testing #91

Transcript

Mario Vega: Hello?

Justin Traglia: Good morning.

Marcos A. Maceo: Good morning, guys. How you doing?

Mario Vega: All good, Marcos. How are you doing?

Marcos A. Maceo: How good, Mario, Florentine.

Mario Vega: Yeah, indeed. It's been a long time. Justine, we have just one topic for the breakouts. So, one single SL topic. I propose that we just stay in this call for this EL breakout, because there are no EL breakout topics. Is that okay, or… What do you think?

Justin Traglia: Yeah, that's fine. But if… if, for whatever reason, there ends up being more, we could just break out. Wow. Yeah, that should be fine. Let's give it a shot.

Mario Vega: Yeah, for sure. Sounds good.

Saulius Grigaitis: Here we go.

Mario Vega: Hello, Solis.

Hello everyone, let's give just one more minute to give some time to people to join, and then we can start.

Alright, I think we have enough people in the call to get started. So let's get going. Welcome, everyone, to ACDT number 91. Today is August 10, 2026. Today we have a slightly light agenda, I would say, and just a few topics, mainly discussion of the planning of DevNet 8. So yeah, let's get started, the main… the first, item in the agenda is, DevNet 7. I don't know if you have Stefan or somebody else who would like to chime in and give some updates about the status of DevNet 7?

Stefan Starflinger: Yeah, sure. So, overall, it's been going pretty well. We haven't had finality stall, during the last few days. We've had, TISM nodes, testing many different scenarios, that have now been turned off. I think there was some sinking stall, with the Prysm nodes, pretty recently. That, is being investigated, or might have already been fixed. But overall, it's going pretty well, I think. Yeah, should be fixed. That sounds great. But overall, it's going pretty well. I might do another round of spamming on the network to kind of get the performance report. On that, again, but otherwise, maybe someone else would want to chime in on… DevNet 7. But otherwise, I think most of the… interesting things will be 11 at 8.

Mario Vega: Yeah, thank you. Barnabas, would you mind explaining on Toni's attack?

Barnabas Busa: Maybe Toni's on the call?

Toni Wahrstätter: Yes, I can… I can briefly explain what I have been trying. So, I used Builder to win a bit, and then I spent a lot of payloads. And the payloads were, in a correctly signed envelope. This was important, and also the envelope referenced the right beacon block, as well as Had the right, block hash. So I think what happens is that you can send, like, thousands of payloads into the network, and Lighthouse and Prysm would still propagate all of them, as well as passing them to the EL. And it seems like, you can cause missed slots and missed payloads through that. I'm not sure if that's the only impact. There was also something at Nimbus, which I wasn't sure about. Maybe someone from Nimbus can explain, can expand on that.

I wasn't really sure what exact impact this caused at clients. But maybe clients can tell us.

Dustin Brody: I guess I can address the Nimbus side. So… Nimbus does… it's not clear to me that Nimbus was directly affected by what's called the First Order version of the… propagation issue that… that you described. At least I didn't see direct evidence of that. what I did see evidence of… is that… mmm… there's still a relatively high volume of… of these, sort of, payloads being sent to ELs, and it was still enough to, Delay, the… sort of… let's say, the results from the EL from other… necessary traffic that Nimbus sends to ELs. And that causes the delay in envelope acceptance in a couple of cases, and that appeared to be the mechanism. And then Nimbus had issues a little bit with recovering from that, which…

Are a little more subtle and sort of Nimbus-specific, and also sort of in the process of being fixed for other reasons. So… as sort of… Yeah, that… Sorry, it's not like, a great summary, I guess, but but yeah, it's… There was no one, individual terrible bug that I saw with, the Nimbus behavior. It was a number of, sort of, sort of. Not quite ideal things.

Toni Wahrstätter: Oh yeah, this is interesting, because what the payloads also do, they have, like, I think you posted this into the Discord at some point, they have, 95,000 transactions in them, so… the EL would first have to put them all into the transaction. try, and then calculate the try, and of course, first, RLP, decode everything. And this is a lot of work, and also, I should have added, the payloads are all 9.5 megabytes big. So that's quite a lot of… traffic, happening, because basically I win one bit, and then I publish, like, as many of those… of those 9.5 megabyte payloads as possible. And I can at least see in the metrics of my own Lighthouse and Prysm notes that, I can keep them busy for, like.

One or two slots. And at the same time, there are, like, missing slots appearing right after, so I would assume that This might be the cause. And I'm wondering, like, of course clients cannot, like, reject payloads that come outside of the slot in which they should have come, because, that would, like, enshrine, like, synchrony, I guess. But I'm wondering if there is any other defense from… Payloads being published, not in the original slot, where they should have been published, but later into a different slot.

Mario Vega: Yeah, jochem?

Jochem Brouwer: Yeah, Toni, one question, because… Did I just hear it correctly that you said 95,000 transactions?

Toni Wahrstätter: Yeah, but this is more or less garbage. So the attack is mostly against the SEALs.

Jochem Brouwer: I see, but that… this should already mean that the EL should fail when it's early, right? Because it's not possible to construct 95,000 valid transactions on a network where the gas limit is 300 million, right?

Toni Wahrstätter: I'm not sure how many transactions you can maximally fit in, but, I think the COs are also not, I think they see the transaction spied, yeah, but, I don't think there is such a check happening. But of course, you could then just go right below that check. And there are probably many transactions that fit within today's 60 million gas limit.

Jochem Brouwer: Yeah, then maybe to add on this, like, is there a way how we can add this to, like, some benchmark infrastructure? Or is there, like, a way how we can easily fix this? Because… I will not say that this is, like, a trivial Way to do this, but this sounds like something that we should easily make testable, at least.

Mario Vega: terence?

Terence Tsao: Yeah, a few questions. The first one is that I imagine here the bottleneck, you see it's from the EL, but do you see any bottleneck on the EL side? That is when we do, like, the state Transition function. on the CL, do you see, for example, like, a bottleneck on Het Street Root and stuff like that? That's the first question. The second question is that I imagine here what's gonna happen is that… Because of the delay that the SLOAD it's empty, but the payload is later imported on, say, M plus 1 or M plus 2, so clients should eventually sync it. But the question is that, like, how does that affect the client when they're seeking a payload that is late? I can say for Prysm, yes, because of something like this, we did find, like, random

but rather than bots, it's more like something we can do better on dealing with edge cases like that, and that has been addressed in the later PR, so… So, I'm looking forward to see how we do now in those cases.

Toni Wahrstätter: Okay, yeah, this sounds good, I can test it again on DevNet 8. Yeah, I think the main work that comes with those payloads is BLS verification. Checking the fields, so calculating, the SSZ route, I guess, and… of course, the repropagation, I think that's the main work that is caused on the CO, but of course, like, many of those, because… I can send, like, thousands of payloads with slightly different, values in the EL payload, and thereby cause, like, the EL to not deduplicate. But yeah, I'm happy to… to test this, On definite, on definite age.

Terence Tsao: Can you let me know, or everyone know, when you start doing that, or just set a time when you start doing that, so we can capture live profile? I think that also might be useful, that… Every client can do very client-specific capturing or monitoring, and we can find bottlenecks.

Toni Wahrstätter: Perfect, yeah, let's do that.

Mario Vega: Just to raise one of the comments from the chat, Kev asks whether there's a write-up for this issue or not.

Toni Wahrstätter: Not really, this is something I've, started, testing on Friday, but I can, I can produce such.

Mario Vega: Perfect. So, if I understand correctly, next steps is to reproduce this again in DevNet 8, and then just communicate to all clients when this is happening, so they can start doing, Proper data collection, right? Is that… is that okay?

Toni Wahrstätter: Sounds good.

Mario Vega: Alright, let me see… Is there… is there any comments? Any other comments from other clients regarding this, or…

Dustin Brody: I guess I'm curious, where… was it measured where the, kind of. bottlenecks in the… were actually occurring? I mean, you're saying, like, in the CL, you think the BLS signature verification, etc. I mean, how, Empirical are these statements?

Toni Wahrstätter: No, this was not measured. The only goal I tried to achieve was getting those payloads repropagated by other nodes. But nothing measured. I just was looking at my own metrics at the same time to see, like, If lines are spiking.

Dustin Brody: Okay?

Mario Vega: And just to raise a comment from the chat, Nico wonders if there's a bug in propagation. Nico, would you like to expand on this?

Nico Flaig: Yeah, I was just looking at the Lodestar code, and we should, for example, only propagate the first validly signed execution payload envelope we see from the builder. So at least this repropagation should not be happening, I think. Or is there a valid reason why clients propagate many payloads?

Eitan Seri-Levi: Yeah, Lighthouse is repropagating envelopes it's already seeing. We're… actually just built out a fix for that, and it should be merged in, like, a day or two, so maybe it's our fault.

Nico Flaig: But yeah, to me, it seems like proposal equivocation, and there we also do not, re-gossip the second, signed Beacon block we see from the proposal, so I would think it should be the same here.

Mario Vega: Nice, okay, just… I think I jumped the gun a little bit on the retesting of this issue. Toni, would you think it would be valuable to retest on DevNet 7, or would we want to wait until DevNet 8? What's your take on this?

Toni Wahrstätter: No, I think definitely it is fun. As we just discussed in the chat, this seems like it is specced, but just not implemented at some clients, so we should probably just do it at DevNet 8 and see if this repropagation is still occurring.

Mario Vega: Sounds good. Okay. Great, great to hear. Thank you, Toni, for raising this. All right, yeah, let's move on. I think these… oh, okay, before we jump, is there any other topics on DevNet 7 that we… anyone else wants to cover? Or should we go into the merit? Alright, so yeah, let's just jump directly into DevNet 8. There are, two subtopics, but first, Stefan, is there anything that you, other than these two subtopics that you want to raise? Or, how would you like to go on with the narrative?

Stefan Starflinger: Wait, let me just check… Or can you quickly summarize the two subtopics?

Mario Vega: Absolutely, yeah. So the first one is basically just, there's an ELS, release. 8.1.0. So, Spencer, if you'd like to just quickly summarize what this contains?

Spencer Taylor-Brown: Hey, yeah, this, we released this on Wednesday, it just matches the repricing numbers in the existing EIPs repo. And add some more test coverage, and that's it. But, yeah, I… I have a feeling the numbers are gonna change again. is that right?

Mario Vega: Yeah, that's the main… this is the other discussion topic. So, yeah, for the second one, basically, this is… this is true. Spencer, I'm not sure if the numbers are gonna change. So, for that, jochem-brouwer wanted to raise a… A topic on discussion of the… of the current numbers, and if they're gonna change or not. So, jochem-brouwer, do you like to go ahead?

Jochem Brouwer: Yes, so, since last ACDT, we from the repricing have some new insights, and we can currently not confirm that the repricing numbers, what we gave last week, are actually the final ones. I think there's also the understanding that Glamsterdam DevNet 8 is going to be the final definite before we… or at least the intent, that this is the final definite, before we are going to launch a public definite. Of course, in the public, definitely want these numbers to be final, because, well, app users, contracts, like, whatever, like, you want to ensure that whatever your gas cost of your transactions are, that they are actually correct, and they do not change, that you can actively test your

Own products against the, well, the specs that we are actually going to fork. So, therefore, what we are currently doing is we are ensuring that these new numbers, or at least that we get more certain that the numbers that we are presenting are actually the final ones. And therefore, I'm not sure what the Definite EIP timeline is, but we are aiming to, to give, like, a… at least an indication of, no, I should, I should be, be careful how we word this. We aim for, for Thursday, that, We either confirm that the current numbers are the correct numbers, or we say that we likely have to change these numbers, and therefore the proposal is that

Okay, I… get some conflicting comments from Barnabas in the chat now, but we can get over it. In a minute. But that by ACDE, we wanted to signal, like, okay, we do not want to change the numbers, or we are not going to change the numbers, or we are going to change the numbers, and we are on top of this, and that's what I wanted to say. Because the other risk is that we then launch this DevNet 8, And then we reprice this, again, and that's something I think we should not do. So therefore, I say that we at least wait until ACDE, before we can give, like, another indication of, If these numbers are final or not.

Parithosh Jayanthi: But what is happening between now and ACDE that would change the numbers? Like, what is the exact task being done?

Jochem Brouwer: Yes, I can lighten on what, actually, these news insights are. So, we have been running on a different state than we were running the repricing Nimbus on before. And we got to the insight, or at least the indication, that this state, what we are running on, or at least the testing infrastructure, might not result in the correct numbers, which are, like, representative of the actual mainnet numbers, which is therefore also the reason why we do not want to say currently that, we… These numbers are the final numbers, because we first need to get the, the analysis, the test, the benchmark. up and running on the, well, the original, I would say, the mainnet infrastructure. There could be two outcomes, it's very simple, it's binary, basically. It means that either the numbers, the benchmarks weren't the same, or they do not. If they weren't the same, then we say, okay, that's nice, we can now comfortably sign off on these repricing numbers, and if they change, then that means we have to change the repricing numbers again.

And they can go further in depth if wanted to, like, what these insights are, but I think this is the condensed version of it.

Mario Vega: Bespronounced?

Barnabas Busa: Baby. My main question would be, in case we go to Thursday, we get to Thursday, and we find out that the numbers that you guys proposed are not the correct ones. What do we do? Do we just… Delay another week, until we find, hopefully the correct numbers.

Jochem Brouwer: This is a good question. So, in my understanding, Definite 8 was actually not a public testnet yet. I start to understand from your chat that this actually is the public testnet, so I think we should really aim that these numbers are final. So… That means that on Thursday, if the numbers, Given the fact that we get these Nimbus running. we say, okay, these numbers are fine, then we can go directly to, with a plata, Plata, which I'm likely pronounced, wrong, or we have to delay it again, but by now I cannot give any indication, like, if that's 10%, 90%, 100%, I don't know. But yeah, just that everyone is aware that we are on top of it.

And I… sorry, Barnabas, but I hope that this answered your question, and if not, I can, Retry to enter it.

Barnabas Busa: No, it does, yeah. I'm, I'm, I'm just curious if, It would make more sense to… stick with the repricing numbers that we have proposed now, and possibly do another round of repricings in H-star to get to the final, final, final numbers.

Jochem Brouwer: I would not say that we should do this, because that means we have to PFI and repricing EIP for Hegota, and I think this is not possible anymore. And I also think that like, I know that I'm now saying, like, okay, yeah, we are not sure, and that this might sound like, okay, we might need, like, another month. This is not the case. I would really say that we have to do this final check, we have to see how this looks, and then we have to sign this off. Because to, to, basically to… to reprice it now with the wrong… with somewhat the wrong numbers, and then do it again via Hegota, I think this is, like,

well, I would not say, like, a waste of energy, but a time we should not spend to do this thing again, I would say.

Mario Vega: And also Tim.

Jochem Brouwer: Sorry, just to answer, like, the questions in the chat. So, to give an indication, and I'm not going to give, like, any percentage or, like, numbers, we do not expect these numbers to change drastically. But we are also not comfortable right now to sign off on the current numbers, and we want to get these numbers right, and not, like, half right, and this is the reason why I'm… we are proposing this, this hopefully slight delay. I know this sounds, like, not super optimistic, but we are really aiming to get, like, this up and running by ACDE, and then we say, okay, it looks either fine or does not, and then we propose, like, these new numbers. But for that, again, I cannot give, like, a percentage of, of, like, what's the chance that the numbers will change.

Mario Vega: Alright, thank you very much, jochem, for this update. I would like to, ask the clients, any thoughts, or comments or opposition to this, Yeah. any EL or CL clients, I would say. Alright, I think there's no comments. Does this mean that we are comfortable with delaying? That is my read at the moment, but please correct me if I'm wrong. Alright. I think that's the case then. So, jochem, if I understand correctly, this is something that you're gonna present in ACDE, and then we'll have more information, right?

Okay. Yeah, so next steps seems like we just wait until ACDE to have the final numbers, and then… Probably after that, if those numbers are final, we're gonna have an… yet another ELS release with the final, final numbers, and then, we can start testing, and then, we can schedule that merit. cool. That sounds, Sounds like a plan. Any comments? All right, yeah, cool. Okay, so yeah, let's keep going. I think that's everything for DevNet 8, so I'll just jump into the next topics.

Stefan Starflinger: I added something else just now, if I can interject quickly.

Mario Vega: Yeah, for sure, go for it.

Stefan Starflinger: So the request was that we, test with a big state again, because we had, some SSZ, encoding issues at the fork, loss fork for DevNet 7. So my question would be, since DevNet 8 is supposed to be public-facing, how big can the Genesis SSZ be? And, we currently, for DevNet 7, used around 70 megabytes, but if clients want to add it into their binaries, or we serve it over S3 or something similar, is there kind of, like, a limit that clients would be uncomfortable with if it's too high? Or… yeah, just general feedback on that. Is 70 megabytes good, or is it too much?

Mario Vega: Just a correction cayman on the… on the comment. It's not 17, I think it's 70 megabytes.

cayman: That's fine, too.

Parithosh Jayanthi: Do you guys fetch it from the ETH client's repo, or do you have your own S3 buckets where you fetch it from, or would you fetch it from the ETH PandaOpps S3 bucket? What's the preferred option?

cayman: We can… we can, fetch it from… a PandaOps or, like, a trusted bucket, I think that's… Fine.

Mario Vega: Alright, any other SEAL clients have an opinion on this, or pushback? Alright, Dustin is highlighting that it would be large.

Dustin Brody: Yeah, I mean, we have the infrastructure to handle the sort of dynamically downloaded stuff that cayman is describing to, so we could reuse that. So, I mean, if it's useful for testing, that's… that's… Fine. Yeah, I mean, because this is what we do for Hoodi now, we also don't include the Hoodi Genesis state, either way.

Barnabas Busa: What's the biggest, chances value would include in a binary?

Dustin Brody: Practically on the order… Couple dozen, megabytes?

Barnabas Busa: I guess they're up to, like, 25 megs.

Dustin Brody: Yeah, on… yeah, around there. We definitely have a bias in that… in that direction, though. About, sort of… especially if it's temporary binary bloating, Then that will go away once we… once we drop. the DevNet 8 as a, kind of, Quasi-public target.

Barnabas Busa: Right, so the idea was that we're gonna be loading the state with some exit validators. I'm just not sure if it's worth the extra megabytes, for the exit Validator state. If we're gonna do a… Finally, that's not anyway where we're gonna be able to do a very large number of, Validators. In the coming weeks, anyway. So the question is whether there's any use case for, bloating the Genesis state, with accessibility errors.

Stefan Starflinger: I mean, the use case is to test, what didn't work in DevNet 7, and to redo that again. Independent. See if every client is fixed.

Barnabas Busa: Right, Besu can do that in kurtosis now.

Stefan Starflinger: I mean, we could do everything in kurtosis.

Barnabas Busa: like, a big initial genesis state test that is SSV-related that That can really just be done locally. We really don't need to force the public to download the 70MBs estate for For this.

Stefan Starflinger: Yeah, and my opinion is that 70 megabytes is nothing.

Matthew Keil: Yeah, I tend to agree. I mean, especially with broadband speeds now, it's… I mean, it's fast.

Mario Vega: Good.

Dustin Brody: We do see people… oh, sorry.

Mario Vega: Go for it, go for it.

Dustin Brody: As far as the… I'll address the, broadband speed, like, I think 70 is certainly within the realm of, I mean, what for dynamically downloadable ones is… has proven properly reasonable. That being said, we definitely see people, because we see the… toporphyQuest in our Discord, of… who have trouble, honestly, within a span of several minutes, which is kind of Nimbus' default. timeout for this. For downloading the 200-odd megabyte state. they, at least for Hoodi, they at least think they're going to be able to run Hoodi, but they time out on the state download And… I mean, one can argue whether, I guess, they're actually going to be able to run Hoodi, or whether… it's clear. Sometimes what seems to be happening is some,

Other network oddities, like some proxy or traffic shaping is hitting the… that HTTP request, they may not be hitting their, sort of, peer-to-peer traffic. It's hard to say always, because the remaining ones are kind of pathologically slow downloads for the state, but I would not simply, in any case assume that it's quite quite nothing. That said, I… we don't have any real serious objections to 70… a 70 megabyte state, but it's still not tiny.

Matthew Keil: But I would guess the people trying to run Hoodi might be, like, a first-time you know, someone just getting into the space, the person who's gonna run a DevNet to test, because they're doing some kind of a production test, I think would probably have better hardware and connection, generally, but just… yeah, it just seems like a different type of operator, in my… in my opinion.

Dustin Brody: That's probably true. And… and practically, in the cases where I've asked them to run, because this is the usual next troubleshooting step when they come. Just run the same sort of equivalent curl command on the same machine, so… so… to isolate you. Then, so far, like, if you're saying this is half the size, I think that would put it back in the realm of working, even with our default kind of timeouts and everything else. So far, we haven't seen it, you know, five times too slow. It just kind of… even with a 150-odd megabyte Hoodi Genesis, it's… it's… You know, a couple times too slow, but… Like, again, this is not a major issue either way. Nimbus can work with either

20 or 50 or 70 in this context, but… but it's, Yeah, I mean, I would just say we have seen some… a little bit of tension in… with some of the users, but I agree that a lot of the people testing this are likely to be a little bit more aware and on higher spec.

Mario Vega: Alright, Stefan, do you have all the info you need for this, or…

Stefan Starflinger: Yeah, I mean, for me, I would prefer to have it in DevNet 8, since it was an issue in DevNet 7, so we can just check it off, and even if we test even bigger states in the NFT DevNets, I think it would make sense to have that, especially because we have EIPs that are related. to this. And I don't think there's a reason to not do it, and miss out on this, kind of, chance.

Mario Vega: Sounds good. Then, what's the, path forward? Do you see, launching 70, or… Or not.

Stefan Starflinger: Yeah, I think as long as it's under 100, I think I would prefer. If I can convince Barnabas, that would be the way to… to go.

Mario Vega: Sounds good. Okay, so, yep. Any last, questions for this? I think we should just move to the next topic. All right. So yeah, let's go into discussions not related to, Demeter 8. The first one is, a 200 megas, bump at the fork pantry. This was proposed by Barnabas. in the ET R&D Discord, server. So, Barnabas, do you… would you like to kick it off with this proposal?

Barnabas Busa: Yeah, so the idea is that at the, glass fork, we're gonna be bumping the default gas limit on the EL side. And because now the EL is the one that's driving the gas limit, everybody at the fork boundary would basically default to the new gas limit that we have agreed on. So this… would basically allow us to basically take advantage of the new repricing values, and yeah, because state now becomes a lot more expensive, we can make sure that, No new transactions will… or new… no old transaction will, fail, even after the fork, just because the gas limit is too low. Because I think there's gonna be a few cases of, Where… a transaction that we would have submitted now.

with Fulu, it would have failed, just because a single transaction would be over 60 million gas. So we want to make sure that, That won't happen to anyone. Realistically within the first day. It would still take about 5 hours to… Go up to 200 million, I think.

Justin Traglia: Is it possible for CL clients to schedule this? Jun in the past.

Barnabas Busa: No, we can't, because the CL clients… only start communicating this over the Engine API at default boundary.

Justin Traglia: Oh, okay. I'm unaware.

Barnabas Busa: So… Yeah, it's a new feature in Gloas.

Justin Traglia: Okay, great.

Barnabas Busa: But, like, right now, the ELs don't drive the gas limit, right? It would be on the EL. But would you want to increase the gas limit pre-fork?

Justin Traglia: Oh, I was just saying that if… When CR clients, cut releases for Glamsterdam, they would include the $200 million gas limit bump. It might accidentally happen before Glamsterdam. Which I wouldn't.

Barnabas Busa: Would it be safe? Would it be safe, though?

Justin Traglia: No, I don't think so. Or I don't know. Sorry, someone else will need to chime in.

Toni Wahrstätter: Yeah, no, it wouldn't be, because, from FBoost, we use the CL gas limit that is communicated to the builder, so if you pump it up earlier. It will actually go up earlier.

Dragan Rakita: Because the gas limit doesn't require hard fork, can we do the bumps after the Glamsterdam? It doesn't need to be hit on the same block. It can be done in the literacy, that yellow SLA, too.

Dustin Brody: Yeah, this was pretty complicated to figure out the… because of… the… like, the UX for… because you would need… we would need two gas limits. We cannot… Yeah. Because we'd have to keep this 60 until then, and then… Yeah.

Dragan Rakita: why it's complicated? I don't… I don't understand it. Basically, we need to have current-present gas limit until glamsterdam. That means the… releases the client made, basic, they don't need to touch gas limit, but releases after glamsterdam, where Glamsterdam is… basically, it can be day after Glamsterdam, we can bump gas limit for everybody. That means that it'd be gradually increased, depending on the amount of the clients very… Clients that had gas limit above.

Dustin Brody: Right, the way I see this is, this is not simply asking for a change in gas limit, this is asking for a qualitatively new feature, which is a dual gas limit setup. That's…

Dragan Rakita: Circle. I don't understand that. My proposal is basically… do it gradually after, and… The machinery that we currently have.

Dustin Brody: Oh, yes, no, your proposal, I agree, I agree. But I'm saying, the Barn… what Barnabas is asking for. It is effectively asking for two gas commits, And for clients to kind of maintain both, and… And then the question becomes, okay, but that… even the 200 is being overridden right now, to 300. in the DevNets, which again, that's fine, but that means, well, that overridden… the overriding would have to, what, still be possible? Which means that now you have a setting, I guess, which goes away? It's like that, which is immediately on… in Glamsterdam, a structurally new setting, which is just bad, I'm sorry, and… it's a mess to… to… because… because you can't do this safely until Glamsterdam.

And, yeah, unless it's really harmful to, avoid… to have this sort of first few blocks be a little… bumpy regarding large state transactions, I think it's just… Do the incremental thing.

Barnabas Busa: Yeah, so the idea is basically once we have, gloves live on Mainnet, every client, we basically make a new Release. And then people would need to upgrade right after they just upgraded to mainnet. In order for them to upgrade to the new gas limit.

Dustin Brody: the configurations can still be changed. Like, the gas limit…

Barnabas Busa: Yeah, yeah. You can overwrite it all the time. That's… that's not what I'm talking about. What I'm talking about is that probably 90% of the people are running default gas limits. So whatever the client will make a release with. will be what's gonna be running on mainnet for probably the foreseeable future, at least for 3 or 4 more months. So… If we now say that the default gas limit at Gloas is still gonna be 60 million. and we just rely on everyone to manually override it, or on the people that will upgrade a day after mainnet goes live, then we are realistically Not gonna see a increase in gasoline rate this year.

Ben Adams: The danger also with asking people to change the defaults is now… They're not obeying the default, so any future Default changes won't apply.

Mario Vega: Toni?

Toni Wahrstätter: Yeah, since we know that, nodes are generally lazy when it comes to upgrading, I would say we put the $200 million into the CLs right at the fork release.

Mario Vega: Okay, I'm sensing that this is exactly what the CL clients, are pushing back against. If I understand correctly, that's Dustin's comment, that it's basically a wasted. a new feature, and also from Nico on the comments, he mentions that it's… this is not… this is not possible.

Toni Wahrstätter: Why is this a new feature? See, our clients do have a gas limit flag today, right? Can't we just set that flag to a default?

Dustin Brody: No. You can't.

Nico Flaig: You want to avoid this being used before Gloas, right? Otherwise, it's a bit dangerous.

Toni Wahrstätter: I see, so you would want to have, like, the value of this flag only be activated with Gloas, basically.

Dustin Brody: Okay, but then, right now, what hap… so right now, clients have CLs, as far as I know, I mean, correct me if I'm wrong, have one gas limit, flag configuration that can be overridden, yes, and that has a default. This makes that 2, And then every… and then the question becomes, well, they can each have their defaults, but if this also has to be overridden, because the DevNets want… the Gloas wants to be 300, well, that is now a new user-visible feature that disappears, like, immediately. And, and… Yeah. that… Because it cannot safely activate, as has been discussed before Glamsterdam. So, what is being asked for is not just a bump slightly earlier or later, it is having two gas limits.

and having full UX for two gas limits.

Toni Wahrstätter: Yeah, I can see that. Yeah, it's not… it's not clean, neither. I think we have to think more about this, and find a proper solution. It would be… a waste of, having control over that field in the hard work if we don't use it, and then we have to chase node operators to upgrade their clients. So it would be nice if we end up at a solution where this happens automatically.

Mario Vega: Yeah, Terence, I saw your hand raised, you still have to chime in?

Terence Tsao: Yeah, I think I… I mean, like, the previous conversation between Dustin and Toni kind of summarized my point. I think this is very doable, you just have to flex, and then… After the fork… after some time, you deprecate the other flag and default back to the Gloas flag, but for the fork transition. you do need two flags. It is doable, and then the question is, how acceptable is that US? I personally don't think it's that bad. I find it worse that we don't scale because of this limitation, just have one flag, so yeah.

Dustin Brody: But it also is this question for… and one of the example UX questions that comes up is, what if the two flags do different things? or sorry, one of the two are sort of become both present, right? So somebody sets their overrides, like, Glamsterdam flag, and now they have… well, these are answerable. Clients can even have their own policies, what happens, to this. But… I mean, it is… It's a lot of messiness to avoid. sort of… and then I don't think we'd wait for months and months, either, for… I mean, people don't… people do usually upgrade clients.

Barnabas Busa: Yeah, but if we do the fork in the end of December, people will… I guarantee you that there will be not that many people that will upgrade their releases right before Christmas. Like, people will not touch their notes if they don't have to. And if you just say that, hey, this is a gas limit change, I don't think that many people will update. But yeah, maybe I… maybe I see it's wrong.

Dragan Rakita: I think this is question, do we want… Validators community users, so the clients to navigate, or basically choose the gas limit, or do we want client teams to choose the gas limit. There is, like, slight difference preference here. For the glam system, we want clients to choose the limit. While for the… For, at least presently, the users are the ones that choose the limit, and they gradually, like, increase it. I… don't mind, anyway, if we introduce Flag or not. I think the… Users are going to download newer client, newer version, and with that, gradually increase the gas limit to the point that are okay for us. So, yeah, either way, I think we can do it async and just choose something.

Either way, it would work.

Mario Vega: Alright, I would like to timebox this discussion, because we only have, 10 minutes, and we still have two topics to go through. Barnabas, would you like to take on this and, draft another proposal, or… if I understand correctly, we don't have a proper solution now, right? So we have to draft something.

Barnabas Busa: Yeah, I will… I will think of something, and see if we can get everybody on the same page. Without having to have two releases.

Mario Vega: Alright. Sounds good. Thank you so much. Yeah, let's go quickly into the… Last, general topic for today, that's… this is the EIP-8070, discussion for that rate. So, last ACDE, we had, oh, sorry, ACDC, we had a, discussion whether this is gonna be, obligatory for Demand 8, and there was not enough consensus on this, and we decided that today was gonna be the last call if anyone had objections about this EIP being, been mandatory for DevNet 8. So, what I'd like to hear from today is if there's objections for this, otherwise we can go ahead and make it mandatory for DevNet 8. Let me just share a link of the EIP. Just for everyone's, to be… to have context.

Yep. Either objections or comments on this are welcome, so please feel free to… Chime in?

Barnabas Busa: We shouldn't make it mandatory. We have an extra week to work on it, so… Just edit.

Mario Vega: Yep, Yale clients, Yale clients, anybody has opposition to this?

Marius van der Wijden: mandatory what? Mandatory… chestnut flag?

Mario Vega: No, this is, this is another discussion topic, this is, EIP-8070. To make it mandatory for DevNet8.

Marius van der Wijden: Oh, yeah. Yeah, it's fine.

Mario Vega: Alright, last call for objections. Anybody objects to this? Or we can just, Go ahead with it for DevNet 8. Alright, yeah, so, okay, so no objections to this, we should be proceeding with EIP-8070 mandatory for DevNet 8. Yeah, great. Cool. And for this, let's just go into the last topic. So this is a CL breakout topic, basically, but I don't think we have any topics on the EL breakout type. So, my suggestion is that we just stay here and discuss circuit breaker design for this. Alright, so…

Barnabas Busa: Just quickly, before we go into that, could I maybe raise that we would also optionally want to test SNAP2? And sell-level datas on DevNet 8. So, if you have Salvable Delta's implementation, already done in your master branch, make sure that you also rebase that on your .NET branch. And the same true for, Snap 2. So we can… we can test out the ball-based, state here. And if you have no idea what I'm talking about, I just pasted the EIP numbers in the chat now.

Mario Vega: Alright, thank you. Please take a look on this topic. Yeah, okay, so moving to Secret Breaker, let's just jump into that topic now. I believe, Barnabas, you're the person for this, or Parithosh? Please stick it away if you want to. Discuss this now.

Barnabas Busa: Yeah, I think Berry would have more context on it. I was, out of office when this was discussed.

Parithosh Jayanthi: Yeah, so most of it is, trying to get an update on where clients are with their circuit breaker designs. I've seen a document from Potuz describing the Prysm Circuit breaker. I think Nikos posted a bunch about the Lodestar Circuit Breaker. Do we have a similar document for Lighthouse, Teku, Nimbus, any other clients? Or where are you guys on your implementations?

Mehdi Aouadi: I can give an update for Teku. Actually, we already have a Circuit Breaker implementation, it's already in our DevNet 8 branch. We don't have a documentation yet. But it's basically a couple of options that we need to enable, to make it work.

Parithosh Jayanthi: Awesome. Is it enabled then by default on DevNet 8, or do we have to add some flags to enable it?

Mehdi Aouadi: It's enabled by default, but the values are not very tight, so it should not be triggered, I guess, so I think we need to… To override some, some options values, yeah.

Parithosh Jayanthi: Awesome. Could you maybe give us the flag so that we can try triggering it on 11?

Mehdi Aouadi: Yeah, yeah, sure. I'll do that.

Parithosh Jayanthi: Yeah, could we get an update from Lighthouse and Nimbus, maybe? Is the plan that, everyone ships Circuit Breaker by DevNet 8, or release it before testnets? I think just getting an idea of the plan would be enough for now.

Dustin Brody: So, I guess I could say for Nimbus, we're planning on using a very similar design as, obviously, transported to the payloads rather than the outwalks, but as we have for, currently the… the Fusaka… It's just to start with, and we can… See how that works, and elaborate on that as useful. But yeah, it's our… roughly our current kind of design, but… Ported.

Parithosh Jayanthi: Okay, thanks. I don't know if anyone from the lighthouse is here to speak, though. Okay, we might have to take Lighthouse later on. Thank you.

Mario Vega: Alright, was that… was that it for this topic, or…

Parithosh Jayanthi: Yeah, I think so. We just have to trigger it, and then test it out on clients, probably play around with the values a bit, but… Yeah, that should be it.

Mario Vega: Okay, perfect. So that's… that's it for today's topics. Any last comments before we jump out the call? Yeah, jochem?

Jochem Brouwer: Yes, one question for a light client or an EIP editor. There's one EIP what we need to get merged, so we can also ship the updates to EIP-8282. But this needs additional editor approval, so lightclient, if you could do that, I dropped it in the chat. Go request 12021, if you could approve that, that would be very nice.

Mario Vega: Excellent, thank you. Alright, so that's it for today call. Any last words, comments, or should we just jump out? Okay, let's go. Yeah, thank you everyone for coming, and see you in ACDE this week. Thank you.

Jochem Brouwer: Thank you. Bye-bye.

Justin Traglia: Hi, everyone.

Marcos A. Maceo: Bye.

Marius van der Wijden: I agree.

Spencer Taylor-Brown: Thanks.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.