All Core Devs - Testing #92 (CL Breakout)

Transcript

Justin Traglia: Hello again. Just one second. Not concerning. So, I don't think Barnabas is here to discuss it, but he wanted us to make a decision on whether or not to include the, bug fix, called Fix Builder Payment Double… sorry, Fixed Payment Weight Double Count Under Target Equivocation. That was the one by Luca. How do we… we merged the PR, end of last week, or yesterday, I forget, whenever. How do we feel about including this? Indefinitely. Like, it's not consensus-breaking, and it's unlikely to be triggered. Does anyone have an opinion? Oh, I'm sorry, it is consensus-breaking, but it's… Yeah.

But it's hard to trigger.

Nico Flaig: Yeah, I mean, for loads that we already merged the change, and it's deployed on the DevNet. So I guess if someone triggers that, that would be a consensus split. But others don't deploy it.

Justin Traglia: Yeah. I think as long as, ETH PandaOps. As long as all the client teams provided the updated DevNet 8 image, and ETH PandaOps updates their, like, stable set of nodes to use those images, that would probably be fine. Eitan from Lighthouse says that we can't include it. Do others have opinions? Stefan from TechU says that… They'll include it after the call, happy with it. That's great. Would you guys like for us to make an Alpha 14 release with this change? The other little changes we've made. Like, would it make it easier to include this in DevNet 8 if we did that?

Okay. Any other concerns on… okay. Eitan says that he doesn't think that we need another release. Okay. I guess this, point is dealt with. We will add it to the images for DevNet8. So, just do it whenever you can, and then we'll let ETH PandaOps know to update images. on the notes. Mikhail, are you here to give us an update on compliance tests for Gloss?

Mikhail Kalinin: Yes, can you hear me?

Justin Traglia: Yeah, I can hear you just fine.

Mikhail Kalinin: Great. Hey everyone. Quick status updates on the compliance testing for gloves. As, as you all know, we had this fork choice compliance testing suit. And yeah, we're currently working on the state transition suite, And, the approach to generate those tests are basically similar. It's model-based. Kind of describe, outline a model of… of a builder, of a… Execution payload bid, and whatever we have in the state transition, we outline it, with constraints, and then transpile to to the mini-sync, then we kind of ask a constraint solver to produce All the potential… all the possible outputs for… for a given model.

So, it's all combined as inputs to, say, process execution payload bid, as an example. And, yeah, after that, this, all these solutions and all these combinations of, of inputs, they are materialized to a real, Beacon state and, real execution payload bid. And, this is how we actually get, one test case. And, yeah, by doing this exercise, How many times for all of the inputs, we get the tests viewed for for, say, for the process execution period, and the same for all the operations. So this is kind of in progress. We already have a subset of tests, that we will have at the end. So there is, like, a small sweet witch where currently run in… with Deku, just to…

Understand how it looks like, how those tests run, what potential problems and issues. With the test generation itself, we can find there. We don't expect any change to the Testing format, so we'll use process operation, test format for all the operations in the state transition. Same for… for the EPOC processing. And, once we've finished with, like, each operation as a standalone thing. We plan to have, tests with multiple operations per block. Or per… per, sequence of blocks, that may affect each other. Like, Builder deposit and builder top-up, they are dependent. On each other, so… Yeah, that's the, the update on the State Transition Compliance Testing. What else I would like to, ask, here, those clients that, tried, to run fork choice compliance testing.

Tests that we have, already. Would be great to get your feedback. Any blockers, if you don't run them, then just applying blockers that prevents you from doing that. Any issues with the output, with running them as well, would be very valuable, or any potential ideas on how we can improve the test generation, that would be really, really very much appreciated. So please, just reach out on Discord. I believe that… I… yeah, the process of, like, the state transition compliance testing generation is… Kind of a bit slow, because there are many, entities in the state transition, and It's quite… not… it's quite a sophisticated task to, you know, to model them all. But we expect something to have, like, in mid-September.

That's rough, estimation. Alright, thanks for your attention. Happy to answer any questions if you have.

Justin Traglia: Thank you, Mikhail. Are there any questions regarding compliance testing from clients? Okay. There was a discussion item from Kev about the safety decay parameters for historical block retention, but he's not here. I think he might still be in the EL call. I could have misunderstood, butch. Or, sorry, which layer this was for. I'll give him time to join later, I guess. And then I just had a question about POTUS's PR on allowing EPOC boundary reorgs and gloss. It was this PR.

At ACDT, like, 84 or something, we, we discussed this, and At the moment, we decided to… just rename it. From a different function. Perform a different name to this, and keep it as a check. But it seems that POTUS is suggesting we just simply remove it. At the time, Nico, you had thoughts and opinions, What do you think about this? I was really hoping POTUS would be here to provide more justification.

Nico Flaig: Yeah, I mean, the original PR was more or less a refactoring, right? So I guess, yeah, we can discuss it more explicitly if we want this reorg behavior or not. I mean, my main concern was, right, that it harms the proposal more than anyone else, or it also harms the attesters, but yeah. I'm not sure, like, if Lighthouse is already doing that on mainnet, then it doesn't matter, like, every client can just do it as well, because the proposal itself will already prepare on the right, On the right head, and do the epoch transition, but attesters will not, and when they see the block, they will have to recompute the epoch transition, so… So if one client does it, and especially if Lighthouse does it, which… because they have a lot of network share, right, I think everyone should just do it.

Justin Traglia: Mikhail.

Mikhail Kalinin: I think that, what Bot was meant in the PR is basically that, after full, we have a stable proposer look ahead. And, this, EPOC boundary work, If it happens, it would affect the, proposer, shuffling. And, yeah. That's why post… post-fulloit, this check doesn't make sense. But that's just my… my understanding. I… I don't… because the dependent route for attesters, is not the current EPOC boundary, it's the EPOC boundary, the previous one, so… Should be fine to remove these, as far as I can see.

Justin Traglia: Nico or Eitan, could you clarify, does Lighthouse… Have this checked, or did they remove it already?

Eitan Seri-Levi: We can reorg at Epoch Foundries.

Justin Traglia: Okay, I see. So you do not have this check. Like, you do… you follow what POTUS is suggesting?

Eitan Seri-Levi: Yeah.

Justin Traglia: Okay. Thank you. Okay. Others, please take a look at the PR and leave a review if you have time. We'll probably… we'll probably merge it, unless there's… Sorry, we'll probably merge it unless a client or two say that they don't like it.

Nico Flaig: Yeah, the misleading part is just, in Chloa's part. I guess it's not really relevant, like, maybe you should state after Fulu or something. But yeah, every network, as far as I know, is on Fulu, so it doesn't really matter, I think.

Justin Traglia: Okay, can do. Is there anything?

Jihoon Song: concern, coming from other… EIP processing operations. Because I removed this shuffling check in the previous PR, but I added it back because there was a feedback that it is not just about shuffling, but it's also about epoch processing. Just going to double check if there's a problem.

Nico Flaig: I mean, right currently on mainnet, we already have a lot of missed SLOT0 blocks, and this just increases it more, so I'm pretty sure if Lighthouse does this reorg on mainnet, they will just orphan the block anyway. So it's just self-harming the proposal.

Justin Traglia: Okay. Other clients, please take a look at that PR when you have time in whatever GitHub. Starts working again. Alright, I guess that's the end of that discussion. I see that Kev is here. Would you like to discuss the safety decay parameters for historical block retention, Kev?

Kevaundray Wedderburn: Yep, hey guys, yeah, so I was looking into this constant because, we agreed on the EL side that they will just copy whatever the CL does for historical block retention. And this is mainly because some of the CLs rely on the EL to have the execution payload. So when I look at this… constant. For checkpoint syncing, we've… we've made the safety decay 10, but for historical blocks, we've made the safety decay 100. And, I was trying to go through the rationale for this, but it seems like this is overly conservative, and it just means that we're storing, like. Four times the amount of, historical blocks than we need. And yeah, if you… I don't know if you guys saw the message, but after 33, if I understand correctly, there was actually no extra security that's, added.

So I was wondering, does anyone know why we chose 100, and is… are folks against just making it match what we have for checkpoint syncing? Or just, like, double it, like, for operational purposes, I think that was what I proposed. Right, so the min-epochs for, block requests, that's based on the safety decay, of 100. But after 33, there's, like, no… Actual… actual benefits. And if you're only ever gonna checkpoint sync, for a safety decay of 10, it means that your checkpoint syncing, like, file is only ever gonna be, like, 2 weeks old. So you only… Ever would need to be, Serving blocks that are, like, 2 weeks old, but you can just say 4 months, or whatever… sorry, 1 month, or whatever.

Mmm… Mikhail?

Mikhail Kalinin: A question, Kev, are you asking from the CL perspective only, or maybe there is another reason from the EL side, or from the user side, or whatever? That's… Actually results in these 5 months.

Kevaundray Wedderburn: Yeah, I was just auction from the CL perspective. Sorry. From the EL side, it's, the EL is just gonna copy what the CL does, that's what we agreed. But is the… yeah, is the… I don't know if anyone has any questions about safety decay, or, Or like, yeah, what it's, for, or whatever, yeah, can explain some more. And yeah, I had a second question, which is that some clients seem to actually not be backfilling to this 5 months, But, yeah, I first wanted to… Ask about this safety decay, if anyone has any opinions on it or whatever. Or maybe the CL Devs who usually look at it are not in this particular call.

Mmm… Okay, I can move on to the next, oh, sorry.

Mikhail Kalinin: Yeah, just a quick thing. I think that from the security perspective, 100% decay doesn't make sense. So.

Kevaundray Wedderburn: Right?

Mikhail Kalinin: If you're a checkpoint sinking from 5 months ago, it's gonna be… Likely insecure, unless you trust the Actually, yeah, it's… anyway, it's kind of insecure, so… but maybe there are some… I'm just trying to… to guess, mate. Maybe there are some sync modes, that some clients support, or whatever. That's actually do lockstep. That's just, like, the only thing that comes to my mind, why we would need these, you know, number of blocks. served by a network. So it's like lock-stepping from, you know, I don't know, from some point in time, and if you have, like, error files or whatever, so you can do that and, you know, get all these 5 months of history. from… From the network, yeah, but… Sounds like… quite unlikely.

Kevaundray Wedderburn: Yeah, I'm not, not sure. Hmm. Yeah, I guess if it was, even to do with that, I would probably say it makes sense to have two constants then, like, one for the safety decay, and then add, like, a plus four months, or whatever for whatever this other thing we're trying to account for. But yeah, I'm not sure it's that dear. Yeah, and I gave some links, but I think Prism, Lodestar, and Grandine don't actually do this by default. If I just copy this… Mmm… Paste it in here from GitHub. So I think the Prism one, unless the comment is outdated and I've just not read it properly, is kind of self-explanatory. But yeah, it also seems that Lodestar doesn't do it, and Grandine…

I'm guessing because you don't… you don't really need to, right? Like, backfill that much. Yeah, I guess if you don't do it by default, then I don't know if it makes sense for the EL to now be storing 5 months. Worth of blocks, like, Yeah, which is why I'm trying to… trying to change this constant, if possible. Right? Okay, cool. Yeah, I can make an EIP, just to codify this, to see if… it'll probably be informational, I guess, see if anyone… has any issues, but, essentially, I was proposing to make it 8192 epochs. I just took the safety decay of 10, doubled it for, like, operational purposes, because in case someone can't make a checkpoint every two weeks or whatever.

And made it a power of two, just… It's a bit easier to think about, but happy to change it to whatever, but just that 100 doesn't really make any sense. So it'll be, like, the retention policy would be just 2 times whatever the blob retention policy is, like, one way to think about it. And the blob retention is, like, roughly around safety decay of 10, but that's not really why we chose it. In retrospect, we could say… we could say it. Hmm… Okay. Cool, yeah, I'll make the EIP, and then, try to pass it around, and if CLs agree, then we can just have the ELs go down to this constant instead of Using the safety decay of 10 Consulates that we have.

Which means, yeah, just storing less blocks for, yeah.

Justin Traglia: Okay, great, thank you, Kev. Yeah, please show the EIP whenever it's ready. Luca had a question in the chat about, when we should merge the Beacon API changes. Or are there any blockers or open questions left for 6.30? nico, would you like to chat about this, or I could just read your comments?

Nico Flaig: Yeah, I mean, there was some… so, the major redesign, the last one was last week, I guess. But yeah, I'm not sure anyone besides me even reviewed the latest, and I guess Jason, but it would be good if people agree on the… or can have a look at the latest state. So I implemented, like, there's the other two PRs, which is… one is on the builder spec, the other one is on the key manager, so I have this fully implemented end-to-end. So it seems to check out the whole flow. There's also an implementation guide that Jason has written. Which is also helpful in case someone is implementing that. So yeah, yeah, once GitHub loads, I guess, we can look into…

Getting this merged, this week. Yeah, I don't… I think they're in a good shape, chasing, right? I would have to give it another pass, though.

Jason Vranek: Yeah, I think the only outstanding one is on 88. You asked a question if the Get endpoint should return the pub key or not to be consistent with the previous endpoints.

Nico Flaig: Yeah, maybe James has an opinion on this. I don't even know why the current APIs do this, to be honest. It's a bit redundant, but yeah.

james: Yeah, I, I responded. No strong opinions. I think when we first started doing this, PRISM had some restrictions because of… this gRPC gateway thing, but it doesn't matter anymore. So… so we can include it or not include it? J- I guess Jason can make the call.

Justin Traglia: Jason says we'll keep it as is, then. Cool. On this topic, how are… how are implementations of the DirectBuilder API in clients going? Mike, does anyone have this yet?

Nico Flaig: Yeah, so in Lodza, I have it implemented end-to-end now, based on the latest spec from Jason. I found a few issues with Builder, so there was some outdated API… yeah, I guess nothing… I cannot share anything. It's not loading, but So there was some stuff in Builder, the submit Block API, which is now fixed, so it used some stale URL that was there previously. So if you use the latest builder image, everything should work if you have it implemented correctly.

Justin Traglia: Cool. Thanks for the update. James says not end-to-end yet. Yeah. Implemented KeyManager 88, but not… 630 Beacon APIs. Eitan says that Mark did a full implementation, we just haven't reviewed and merged it yet for Lighthouse. That's great. Are we gonna try to test this in DevNet 8? I assume we are. Right? Have we chatted with builders about this as well? Or, Nico, you were gonna say something?

Nico Flaig: I mean, I at least plan to have this before the fork, so I can see if the transition works around the fork boundary, but also, yeah, I'm not sure there's any builder that is exposing an API to test. some…

Justin Traglia: Yep.

Nico Flaig: Yeah, not sure. Ideally, what we want to see is that, there's continuous blocks via the API, and I guess with clause then starting… Ideally at clua SLOT 0, I suppose.

Justin Traglia: You're hoping to do it. Sorry, to be clear, you're hoping to have a direct… Sorry. how do I say it? A bid from a direct builder for slot zero after the glass upgrade.

Nico Flaig: Yeah, exactly, yeah. So we can test if that works smoothly. But yeah, I also noticed there's some, like… you need to… like, SLOT 0 will be maybe a bit hard, I'm not sure, because you also need to discover your builder right at the fork boundary, like, you need to see what index you get assigned, so you need to be really quick to submit a bit for slot zero. Maybe we are fine with having a local block there, I guess. DFI needs to wait 12 seconds.

Justin Traglia: I mean, I would at least like to see a… like a… a builder bid for Sot Zero after the upgrade. I'm not sure if it would be direct or… P2P. Yeah, that'll be fun to test. Or to figure out. Sorry to call out names, but, like, Teku, have you guys worked on this at all? The DirectBuilder API?

Stefan Bratanov: Yeah, a little bit, we're still in progress. Hopefully… this week or something, we may have some workshop.

Justin Traglia: Okay. Sounds good. Given… So, Nico, given that you've implemented it in Lodestar, do we feel comfortable merging the builder-specs PR related to this? Or should we wait, like, another week or two for other clients to implement?

Nico Flaig: Yeah, I still need maybe one day to review our implementation. A bit more, to feel, like, comfortable. But yeah, I think it's in a good shape, so just seeing if there's, like, from others, just some approvals would be nice.

Justin Traglia: Yeah. Okay, cool, That's it for the agenda. Is there anything that we'd like to discuss that's not on the agenda? Alright, it was great to chat with you guys, talk to you guys next time. I have none.

Eitan Seri-Levi: Right.

Nico Flaig: See you.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.