All Core Devs - Testing #98 (CL Breakout)

Transcript

Justin Traglia: Hello, hello. I just wanted to make sure that they had a moderator before leaving. Alright, we don't have a lot on the CL breakout agenda. I would like… To discuss this edge case payment without… sorry, this edge case, Regarding builder payments, without discussing How to potentially exploit it in any way. Potuz and Nico. Sorry, is Nico here? I don't see him. Oh! Yeah. Potuz, would you like to discuss this? I think you're… you probably understand it best.

Potuz: I it's So, when we remove this partial state transition on 5094, That happened when we import the payload, then we… We… We have… we moved into the system in which the block processes the requests at the beginning from the previous payload. So, if a block builds on full. It processes the request of the previous payload, and it performs specifically the state transition that would have been… would have happened on the payload, and then it continues processing its own state transition on the consensus side. The problem is that, A request could include an exit for a builder. And this is processed unconditionally. And, Afterwards, we process the payments from the previous, payload.

So that's part of the state transition, because you just built on top of full. So now, in a normal scenario, the situation is that, you process an exit for a builder, the exit is put there, and then you process, the payments, and… the payments are either from the current epoch or from the previous epoch, if the block was there, and then you just… if there was a trustless payment, you just add a withdrawal for the… you add a withdrawal for the proposer. Now, timings can be set in a very specific way, if a lot of things have to happen, but it can be put up in a situation in which the withdrawal for the builder is put in the queue at the beginning of the processing, and then later, after that.

There's a withdrawal output for the proposer, also on the queue, and both of them are paid unconditionally on the. Both of them are paid unconditionally, if certain things happen. In that situation, the builder would be, minting money. And there's a bunch of solutions that are possible for this. One of them is to not pay the proposer. Another one is to burn, not only not pay the proposer, but actually burning that, that ETH. Another one is to be careful with how to, what exit from the builder is, what's the withdrawal that is set for the builder. Be careful in the sense of, like, noticing that there might be in flight some payment for the proposers, and then deducting that from any potential exit from the builder.

And another one is to change the ordering, so perform first part of the consensus state transition, and then the request from the previous payload. I personally prefer the simplest one, which is just not pay the proposer, and that's it. And I opened a PR for that.

Justin Traglia: Yep, thank you, and thank you so much for the… brief overview. James, I see that you're unmuted. Were you gonna say something?

james: I'm sorry. Yeah, that was an accident. Sorry.

Justin Traglia: Sorry, no worries. Okay, Nico, you have an alternative. How do you feel about the PR that Potuz opened, which I guess is probably the simplest potential solution? And others, like, how do you feel, or what do you think? I'm very curious to hear opinions.

Nico Flaig: Yeah, what was the biggest issue with the ordering? I mean, my PR, my change is also really simple, right? I didn't find any problem with changing the ordering.

Justin Traglia: I'm unaware of any problems with that, either. Yeah, and Potuz, maybe a question for you. Like, with your PR, how easy or difficult would it be for a builder to abuse this? Like. Could they wait for a very valuable MEV block, make a big bid for it, and then somehow cause the next two epochs of… Slots to be empty, and therefore not be required to pay a proposer. Like, is that theoretical? theoretically possible or practical.

Potuz: It is, and that's why burning is the most aligned. Yeah, it is, and that's why burning is most aligned. But… I think still the trade-off of changing the state transition and performing part of the state transition from the consensus side before the state transition from the previous envelope. opens for… a bunch of subtleties that might happen, that, yeah, I haven't spent time analyzing, it's just the fact that we need to analyze them, and then try to find if there's something that is exploitable on this change. It's scarier than just making sure… I mean, definitely not paying the proposer does not come with minting money.

Justin Traglia: Yep, I mean, that is true. But it does have other side effects or whatever, like I tried mentioning. Should we just evaluate, or, like, spend a day evaluating? like Nico's suggestion, and see if we can think of any edge cases, and if we can't. Maybe prefer that? I definitely…

Potuz: prefer not. Like, if I were going to do edge cases, I would rather burn that in. I mean, if you really think that it's exploitable to not… to put up a payload, beat a lot, and then reveal it. And make sure that it's full after 64 epochs, after 64 slots. Also notice that the next proposal can always reorder that payload. So.

Justin Traglia: Yeah. I mean, I would also prefer to burn instead of just not pay the proposer.

Potuz: But burning comes with also subtleties because then it could be a consensus split by computing incorrectly how much you need to burn.

Justin Traglia: I agree, and if there are subtleties. I mean, then I feel like Nico's PR might just be fine. We just have to… Address the subtleties, and… Whatnot. Mmm. But, again, to, like, to Potuz's simple PR fix, like, this is pretty difficult to trigger, I don't know if it's… a realistic issue that we would face, and as long as we're not minting ETH, it's… Like, we just need to document it. Be aware. Potentially. I'm not sure what the right thing to do is. I'd like to hear from clients more. or anyone else. They don't have to be a client. Sorry.

Is there… I mean, just to agree, I mean, there's no reason to delay Sepolia over this, correct? Yeah. Potuz shook his head. Totally agree. Sepolia is going to die. Pretty shortly after this upgrade cycle, correct?

Barnabas Busa: Yeah, I…

Potuz: It doesn't matter, you cannot trigger it anyways.

Barnabas Busa: Okay.

Justin Traglia: True, but I think if we do. Actually, maybe Nico can correct me, but if we do Nico's PR, it's technically… Sure. It would be different, like divergent from Hoodi and Mainnet, correct? Or does it behave the exact same way?

Potuz: It would be the same as long as this is not being triggered.

Justin Traglia: Okay.

Potuz: You can do mine or Nico's, as long as this situation doesn't get triggered. We can do this even after the fork. You could just change clients' releases after the fork.

Justin Traglia: See? Yeah. And because it's so hard to trigger, I I'm not really worried about Sepolia. It's also just like a testnet. I would I think we should aim to have this fixed before Hoodi releases, though. Does that sound reasonable to you, Potuz?

Potuz: Sure.

Justin Traglia: Any other thoughts on this? I don't want to, like, waste people's time. Do we need more time to think about this async over the next day or two?

Nico Flaig: Yeah, but just to be clear, so the two spec changes require 64 slots of missed blocks, right? And the minting bug itself requires 64 epochs, right? Is that the correct understanding?

Potuz: Yeah, it depends on how long the churns are, but yeah, so it requires many more.

Nico Flaig: Yeah, maybe.

Potuz: It actually means it requires many more blocks that are missed.

Barnabas Busa: And we need to also miss the CL, or just the EL payload?

Justin Traglia: My understanding is for the… there needs to be, I think, like. Sorry, 64 empty slots, and then… 60… 3 epochs with non-full payloads.

Potuz: No, no, so no, no, no. For to trigger the double the double withdrawal, you just need the the missing payloads. Because.

Barnabas Busa: So that's quite cheap, right?

Potuz: What is it?

Barnabas Busa: You could just buy the Blocks and not include a payload.

Potuz: Yeah, so that's that that's why I got worried with triggering that thing, because well, I mean, Justin didn't want to talk about this.

Justin Traglia: I mean, it's okay, as long as we fix it before, Hoodi, I don't really care. I mean, if someone else doesn't want to, we can…

Potuz: I think it's practically impossible, even in that scenario, that it's free to buy blocks. Because, hopefully, circuit breakers from clients are actually going to trigger, and we're going to fall back to self-building before this happens. And at least…

Barnabas Busa: the solution that we take.

Potuz: Good afternoon. I'm sorry?

Barnabas Busa: Could we test this on devnet 8? DeForest.

Potuz: Yeah, we can. Yeah, we can.

Justin Traglia: Yeah, and like Potuz was saying, it only requires a single client circuit breaker, right? Assuming that they have a proposal during that period. So… I agree, it'd be very, very difficult.

Barnabas Busa: But yeah, but this is the 64 epoch bug. But there's also 64 slot bug right?

Justin Traglia: It might be the same one. I don't fully understand the requirements for this. I'm sorry. Potuz will know.

Potuz: Yeah, so the only thing… so there's an if branch. And that if branch just checks whether or not There is a pending payment for the previous payload. And that pending payload for previous payload is going to be gone after two. epochs worth of. empty, of empty payloads. So, if you just have two epochs of empty payloads, regardless if you have the blocks or not, the consensus blocks or not. that payment is gone from the queue. And the branch, the last branch of that function, what it does is just unconditionally pays the proposer that is building… so it pays for the proposer, the proposer block, whose payload is now being included, because you're just building on top of full.

And, yeah, so we can talk about whether or not the disincentive aligned or not for a builder to try to trigger this. to try to get a payload, included, and at the same time miss all the other payloads, I would just say that a builder that can actually make all of the next payloads to be missing can always avoid paying.

Barnabas Busa: Right, but, like, anyone could trigger this, right? Like, if I'm… if I register a builder for 1 ETH, and I outbid everyone for 64 consecutive slots. Then I can just trigger this.

Potuz: you can always win the auction, and that's why I think every single client will… should fall back to banning you first, immediately. Prysm would ban you just on the first offense. But then, after a few offenses from either… from any builder, Prysm would just self-build. And I think every client has this already in place.

Barnabas Busa: Right, because otherwise, for 64 ETH, you could just have 64 builders registered, and you just… Cycle them through.

Potuz: But I think every client has a circuit breaker that falls back to self-building when a lot of payloads are missing.

Barnabas Busa: Yeah, we definitely need to test this, I think, before Sepolia.

Justin Traglia: You could also just fix the issue. Oh, sorry. Yeah, we can test it before Sepolia if you want, but we can also just fix the issue. So it's not a problem in Hoodi and mainnet. But yeah. Nico, I have an ask for you. Do you mind making your… er like to the ethereum consensus-specs repo. So it's a bit easier for others to review, and I can then share. Make sorry. I can then create a thread on EthR&D Discord, or Somewhere else to discuss it. We could continue discussing it on signal if you prefer. But A true PR would be helpful. Any other… I mean any other comments on this topic, or New topics that you guys would like to discuss.

Mikhail Kalinin: I… I have a question about this RPC-style discussion that happened, but yeah, happy to move it to the end of the call if we have anything to discuss now.

Justin Traglia: I have something to say, but it's not like important. I just learned that MEV-Boost will plan on updating for Gloas. I don't think that I don't think those should be any issues regarding that. But just Fyi, I I thought it was going to be deprecated, but it doesn't sound like it will be.

Potuz: Can I ask what's the update? I I I want to know I'm just curious. I mean, if it's if it relates as a if it acts via the builder API, we're gonna be able to connect. Otherwise, I don't see how we're gonna be able to connect.

Justin Traglia: I actually have no idea, but I can try to share the plan with you. I only saw the… a note. Or, like, the… a generous. Oh my god.

Potuz: Do we have someone from Flashbots here or from the maintainers?

Justin Traglia: I don't think so. But Faheel would be the person working on it. I can try to. If I don't make a link while someone else does talks. Also, I can't invite them to, like… Discuss this on… EthR&D Discord, or at the next ACDT call. Okay. Well, this is a very high level. God! This is a very high-level overview of the changes that they plan to make for MEV-Boost regarding Gloas.

It's a bit worrying to me that we're doing this so late in the game, and it won't really be tested for Sepolia or Hoodi. And then it sounds like they're… intending to have this ready for mainnet and that. will essentially be untested.

Potuz: So this looks to me like it's… it's going… it looks to me like it's, it's just doing the… The multicasting that clients are already doing anyways. So, it might be a simple way, perhaps, of adding new builders, if MEV-Boost itself, like, connects to some centralized server and gets new builders and just sends them. But other than that, I suspect that it's just the same that clients already are implementing.

Enrico Del Fante: Maybe I'm missing something, but don't we sign the URL?

Potuz: Yeah, but it's… I mean, if you're just connecting and you just sign over anything, you can just sign MEV-Boost.

Enrico Del Fante: Yeah, you sign local, but if you are trying to add something that checks the signature, it will fail, so you reduce the option. Sure.

Potuz: But I don't think anyone is going to… I mean, I don't think any builder, like. working on a relay is going to be like checking signature.

Enrico Del Fante: Oh, yeah.

Potuz: I mean, they are already in a public auction anyways.

Nico Flaig: Yeah, but this can be done, right? You can have the custom auth data if you need to sign over something different than the URL. But yeah, I don't see why anyone would use MEV-Boost, like, with the basic functionality it has today.

Potuz: Well, it could be this, right? Like, if… I mean, that thing can be doing off-protocol whatever it wants. It can connect online to whatever it wants. So it can grab an ever-growing list of builders that your node is not going to do because you're not automating it.

Enrico Del Fante: Yeah, it's, it's a deployment, feature, at least externalize the list and managed by something else that would be gonna know.

Potuz: And it's something that we want to add anyways on Prysm at least wants to add it.

Justin Traglia: There's a good question from the chat regarding CommitBoost Barnabas or anyone else. Have we tested this? Do we know if CommitBoost is ready for Glamsterdam?

Barnabas Busa: We have not tested it.

Justin Traglia: Do you know if it's ready, theoretically?

Nico Flaig: I know it works, but it's Jason here. So I know he tested it with Lodestar, but only on local Kurtosis. So.

Justin Traglia: Okay. I mean, that's… of… League is ahead of MEV-Boost at the moment, so that's great. Okay. I don't have anything else to say. Mikhail, would you like to discuss the thing you wanted to discuss?

Mikhail Kalinin: Yeah, thanks, Justin. Yeah, I just wanted to clarify, Potos, you were suggesting basically to For the RPC providers to enable the fast confirmation for For a subset of the nodes they run in their fleet, and, serve the. Last confirmed? Oh, what was that?

Potuz: No, no, not really. I mean, sure, that's gonna work, it's just that that really is more work for them. I assume that if they are requesting that there is a tag, or there's a way of, like, having both, it's because they really want that. And that's fine. And we can make it optional, both on the CL and on the EL, Without touching any node. And without pressuring us to have releases for this, you just have a flag on Geth that says you are allowed… you are going to run with this extra type for FastConfirm, and you have a flag on Prysm that says, send this on FCU, and we make it. And the node operator for an RPC needs to make sure that it's running that combo that already supports this.

Mikhail Kalinin: Yeah, I see. You mean like this will help to avoid any? Specific coordination, or any… kind of, like, reading capabilities in the CL, right? Yeah, I see, I see. Got it.

Potuz: I mean, because it's only, like, the 10 guys that are going to be running this, so okay, fine, those guys are already involved enough that they should know how to deal with this.

Mikhail Kalinin: Yeah, yeah, makes sense to me. The other, thing that Barnabas was suggesting is basically to have a Beacon API. That will… respond with the… fast confirmed block hash, I guess. But yeah, that's what.

Potuz: Yeah, but that's worse, right? Because they really want to have this as part of the ELSIN.

Mikhail Kalinin: Yeah. Yeah, I guess so. All right, thanks. That's… Cool. Thank you.

Justin Traglia: Alright, Jihoon, Nico, Kev, would you like to discuss the, lower epoch retention thing?

Kevaundray Wedderburn: Yeah, I can… Discuss it, so we have a Discord thread. Where we've just tried to ask all the clients, what are their thoughts. Nico's opened, I think, 2 PRs now. Just… on the thread, I believe Sproul said he's okay with it. tersec is also… Dustin's also okay with it, sorry. Enrico is okay, I believe, but I think Paul from Teku is not. We haven't heard from Grandine and. Lodestar is okay with it. Potuz, I believe, is okay with it, as long as it doesn't mess around with the EL's, like, minimum requirements. And yeah, so I think it seems like folks are okay with this change, apart from Paul from Teku.

Enrico Del Fante: I cannot, I did a very quick chat, because he's off these days. So he's… he'd like, and I kind of agree, that if it is possible to… to… to make this change backward, back to phase zero, so we don't have to deal with the parameter change at the particular fork, it's just something that changes.

Kevaundray Wedderburn: As far as I know, it should be okay. Nico, is there any reason we can't do that? I thought that's what we would do, actually.

Nico Flaig: I mean, I don't really see how it changes anything, how clients handle this, because, I mean. So the only risk I see, at least on Lodestar's side, looking at this, is that if you use the fork epoch, and then you get into Gloas, and then suddenly you prune 3 months' worth of data. That's…

Enrico Del Fante: Awesome.

Nico Flaig: That's…

Enrico Del Fante: That's, that's another thing. And in any way, you… it's orthogonal. The way you prune is… Just… A technical thing that the client needs to deal with anyway. As soon as you change the parameter, if you can change it at the next restart, because the parameter is set for… forklass parameter, or… You just want to have all the machinery that changes that parameter at a particular fork. I believe this is for simplicity. Could be better to have just one parameter and not being fork-aware, and then the next release, the client We'll deal with the pruning as as we as we probably all clients do good pruning, not trying to delete gigabyte of Stuff in one shot.

Nico Flaig: Yeah, I mean, one thing from the spec side is there is no constant anymore. It's basically a function that is defined in the spec. I guess the only thing is if we backport that to phase zero, there's no real rationale or formula you can use that makes sense with the phase zero spec.

Enrico Del Fante: Yeah, that's fair.

Nico Flaig: I think, anyway, the safest route is probably to do this one release after the mainnet fork. Like, there's no reason to directly prune this. I think one signal we need now at least is, for ELs, so they know how long they need to retain the data. For BALs. Other than that, I'm not sure we need to decide anything before the fork whether or not clients should implement this, because it anyway only defines the minimum period you need to serve data, so you're automatically spec compliant if you don't do anything.

Kevaundray Wedderburn: yeah, so I guess it seems like everyone's okay with just modifying it, given 8061. 14K, and then… If people want to keep more data, then it's, like, fine.

Justin Traglia: All right. Anything else? All right. Oh, you go, sorry.

Jihoon Song: Sorry. When it comes to the rollout plan, from my understanding, clients can implement this. At their preferences. What I mean by that is, even if we merge this PR, some clients can release it after we ship Gloas, so… I would suggest to merge the PR in, and then let clients to… Implement that changes if they want to. It's just something to make sure we Merge the PR or not.

Kevaundray Wedderburn: Right. Right, yeah, so the PR can go in, and then clients can do, whatever they want, because as Nico said, they'll be automatically spec compliant if they do nothing at all.

Jihoon Song: Yeah, exactly.

Justin Traglia: All right, let's merge the PR then. Barnabas is asking if anyone started working on shorter slots. Quick thoughts.

Nico Flaig: So for Lodestar, I have a branch that works, but it's not yet on the latest spec. So there might still be, like, the churn limit updates missing and stuff.

Barnabas Busa: Do we need… Heze to be activated for that. Or what is the current method for it? Or could we just.

Justin Traglia: Based on Heze.

Barnabas Busa: Oh, it is Visa, so we would need to activate it, Heze, first.

Justin Traglia: Correct.

Nico Flaig: Oh, but I thought we tested separately, right? With a custom EIP fork.

Barnabas Busa: Yeah, we haven't talked.

Nico Flaig: FOCIL stuff in there.

Barnabas Busa: Yeah. That's my… Question?

Potuz: I would suggest that we test these kind of things on top of FOCIL. I mean, particularly for the slots, because that's the one that has the most interaction with the timing on FOCIL.

Barnabas Busa: Right, so we would need to have that set as a prerequisite, basically, to have FOCIL enabled, and then possibly later we fork into the shorter slots.

Justin Traglia: Yep, that's right. Also.

Potuz: Wait, why… oh, so you want to do… you want to do two forks?

Barnabas Busa: Yeah.

Justin Traglia: Yeah.

Barnabas Busa: Right now, right now, it's two separate forks in the config. Correct.

Potuz: Oh, I see. So this is the issue that having the fork like an EIP kind of fork. Yeah, that's fine.

Barnabas Busa: Yeah.

Potuz: Then just make it based on Heze, right?

Justin Traglia: It is, yeah, it's a feature, yeah, a feature spec based on Heze. Yeah, and FYI, that PR was merged. I'm not sure if everyone's seen it or not, but Yeah, if you see issues, please open PRs or issues. And we can get that fixed.

Barnabas Busa: Yeah, so this just changes quite some stuff on our tooling side. That's why I would like to be Early on this, because, let's say that we're gonna do a future BPO that's after… The shorter slots, then we will have to calculate the time till the previous BPO with 12 seconds, and then I have the adjusted time in going forward. And there's going to be also a bunch of other tooling, of course. But this is just on the protocol level.

Justin Traglia: I'll go this.

Potuz: I want to go back to the issue of settling the payments. There's something I said wrong. I still believe that the easiest is just to remove the last branch, and just not do anything. But, but there's something I said wrong, which is that changing the order would change, would add a little bit of the consensus state transition and the payload state transition. And in fact, both of those were already part of the payload state transition, both the settling of the payment and the processing of the requests. So, in principle, we could have just done in the different… in the separate order, also, on the… Also, when we had two different state transitions, so it's not a good argument against Nico's PR. I still prefer the simplest one, which is just not to pay.

But in principle, if there's no attacks, it should be fine. Also, it keeps the separation of concerns of having, them in the different orders.

Justin Traglia: Yep. Let's let's discuss this more on discord and signal. Are you fine with that? Alright. I think we should end the call. We're already 3 minutes over, and I don't want to waste anyone's time. Thank you. Thank you all for attending. I'll talk to you guys in a few days at ACDC? Yep. So bye everyone.

Kevaundray Wedderburn: See you guys.

Potuz: event.

Kevaundray Wedderburn: See you. Bye.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.