Fast Confirmation Rule Breakout #9

Transcript

Mikhail Kalinin: Hello, everyone. Can you hear me?

Mercy Boma Naps-Nkari: Hello?

Roberto Saltini: Aye. Aye.

Mikhail Kalinin: Yeah, let's wait for… 3 to 4 more minutes and get started. Pain Azar…

Hi, Nico.

Nico Flaig: Hey…

Mikhail Kalinin: Alright, so let's move ahead and get started. Let me just drop the agenda. So, today, I think, will be quick. And we will start with, client updates. Does anyone want to go first?

Nazar Hussain: I can go. Can you guys in?

Mikhail Kalinin: Sure, please.

Nazar Hussain: Yeah, from my upside, like, We have deployed our task information in our Test fleet and in RC. And, working fine so far. And, hopefully in, coming release, around this week or start of next week. We will include into the public release, and we'll also publish a blog post about it. And, related to the metrics we are for fast information. I have reviewed and approved it, but I will integrate those matrices in those charts, during this phase. But that falls from my side.

Mikhail Kalinin: Yeah, so, will these metrics be in the release as well?

Nazar Hussain: I'm very confident that there will be. Because I'm just working right now on this metric PR.

Nico Flaig: Wait, wait, Nasa. No, no. Like, we wanted to cut the release today, right? Like, literally in a few hours.

Nazar Hussain: That's thrillers.

Nico Flaig: But if we want them in the release, I think we can maybe sneak in a last-minute commit in the RC, like, do another RC.

Nazar Hussain: But I think it's… it's fine. We can… have a PR, and then test it on our fleet of And can include into the upcoming release pipelines.

Mikhail Kalinin: Yeah, I think, it's… it's up to you guys, but I think it's, Not super urgent to have them, those metrics, in the release. But, I mean, like, if it will be in the next release, I think it's fine. Because, like. if one wants to run… to measure those… to get… collect those metrics, on Lodestar, can it just run the master version of it? Like, the build from the most recent master branch.

Nico Flaig: Yeah, they could just run that, Yeah, maybe if Nassar is fast, we can sneak it in.

Nazar Hussain: Yeah, we will discuss it, async. But even if we don't include it, I think it's not urgent, and once the PR is going through the review process and merged. Anyone can… Build it from the master branch.

Mikhail Kalinin: Yeah, because I, I just think that, releasing fast confirmation, like… Without metrics, and without delaying it is more important, you know, than delayed because of the metrics.

Nazar Hussain: That's true.

Mikhail Kalinin: Yeah, great, congratulations, that's great progress. Cool. Do you… do you see any, like, any… Unusual behavior, like reversions to finality or reorgs. Of the confirmed block on your… on your node you're running.

Nazar Hussain: When Nico observed one behavior, So… I think it was after restarting, the node was… The first confirmed block was… Delayed for some time, but after Two epoch 6K accounts, and then afterward, it was just one slot demand. Okay. This was the observation, we haven't yet… Diagnosed it, properly, for the reason, but that was one observation.

Mikhail Kalinin: Yeah, I see, but yeah, that might be fine, because if you just started a node,

Nazar Hussain: And…

Mikhail Kalinin: There is, like… Reason all fork choice information, so fork choice store is not yet.

Nazar Hussain: yet?

Mikhail Kalinin: Caught up with the head. It will likely be a two epochs delay, because first you have to, to… To finish the first epoch, the end of the first epoch, to capture the… Greatest and realized justified checkpoint you will start from. And then in the next epoch, you can search from it, if it meets other conditions. I mean, like, if it's, like, from… From the most recent epoch, from the, On the previous epoch, to be more specific. So, probably two epoch delay is fine.

Nazar Hussain: Yeah, once we have these, metrics, then we will have better observability.

Mikhail Kalinin: Yep.

Nico Flaig: Yeah, but it's not even to epochs, so what happens is the node boots up, it swings to head, and then for, like, 10 or 15 slots, it… the fast confirmation, or the confirmed route, stays at the finalized block. And then there's a jump from the finalized to usually the current slot minus 1. Yeah, I didn't observe any reversion yet. But, yeah, we didn't, like, we probably should hook up a node and continuously monitor this. Or probably, yeah, getting the metrics is the right way to measure this.

Mikhail Kalinin: Yeah, yeah, probably. I mean, like, metrics are, you know… designed for this case as well, like, to… to measure any reversions. To get an idea how much… how many reversions happens, if at all.

Nazar Hussain: Okay.

Mikhail Kalinin: Great, So next, to Grandine.

Mercy Boma Naps-Nkari: Okay, hello, good afternoon. So… I did make some changes on the, the metrics, and then the correction we had, we talked about the last time we checked in. And then, the only thing is, I'm waiting for review from the maintainers. I'm here to get a proper review from them and move forward with the PR, so… that is also something I will be needing help with, because I don't really know how to… poke them around on getting the necessary review I need, and then also, like, deploying that and more. monitoring day. The implementation and all of that, so that's all from my own end.

Mikhail Kalinin: Gracie, could you please repeat, what do you need help with?

Mercy Boma Naps-Nkari: So, I've made the necessary, all the new metrics implementation and all of that, so I need, the maintainers to review it. The only person that is… kind of, like, reviewing the PR is at-up, and he doesn't like… he can't merge a PR or approve it on, Grandine. Although he works for Grinding, but he said the merging has to come from, Samas, or the other person, I can't record the name, so… I don't really have access to, like, push them to reviewing the PR properly, because they have, like, a lot of things they are doing with EPBS and then other, glamster, work going on, so…

Mikhail Kalinin: Yeah, I see, I see. So it's… it's about priorities, right? It's more about.

Mercy Boma Naps-Nkari: Okay.

Mikhail Kalinin: Okay. Yeah. Yeah. Alright, I see, Yeah, thanks for… thanks for your update. If you need any help from… from our side, just please let us know. Yeah, thanks, Nico. Yeah, that's great to know. Are there any other clients' updates? If not, I have… I can provide some, from, from, Lighthouse. We have been recently looking into Likely a regression. In their code, there are some reversions. to finality. This is why I was asking you guys as well, do you observe anything like that or not? But, with the previous version of Lighthouse, there were… there was no such issue, so this is, like, we're investigating.

If it's a regression, and… Yeah, it will likely need to… We will likely need to write a test to cover this case. Obviously, we don't have a test for the case that Lighthouse is experiencing right now, so… Yeah. Alright. The spec and testing updates are next. So… There are no spec updates. Since the last time, so the spec is pretty stable. is good. Tests, testing. So there is a PR, That enables tests for loss and hazard, and also this PR disables PLS. It's kind of, like, dude… Yeah, the outcome is twofold, for disabling PBS, AP, BLS, sorry. So the first is that it just becomes a bit faster to run those tests.

On the spec. And the other one is that the shuffling becomes stable, which means it's not only stable. Across different validator sets. at Genesis, but also across different forks, which is good, because some of those tests are shuffling dependent. Specifically, there are cases where you have a validator vote in a support of a block, in the last slot of current epoch, and in the first slot of the next epoch, and this validator Does not… contribute to… like… Yeah, it's not a new value, so if we have this, the support is kind of, like, a bit lower than in the case where these two committees are disjointed completely. So, yeah, this is one of the cases.

To give an idea why it's shot independent. So, like, yeah, disabling BLS gives us stable shuffling. And makes those tests deterministic across forks, which is good. Also, on the testing of the… on the other side of testing. Update is… I'm kind of working on… Like, edge case tests. And, planning to open APR soon. Like, maybe tomorrow? There are a couple of tests to… yeah, to be written, and After that, we can consider that, you know. The task work is completely done for… for the false confirmation rule. We could do buzzing and other things, but that's out of the scope of the initial testing suit. Yeah, that's for, for tests, API and metrics. Yeah, any question with testing or anything to add here?

Alright, API and metrics, so… I believe that… The PR… Yeah, so… let me check. Yeah, this PR by Nazar has been merged, been merged. Thanks, everyone, for… thanks, Nazar, for working on it. So it basically add current slot. To, to the, event stream. Which is… I think it's quite useful. Also, you've all seen these metrics. PR, thanks for taking a look, and thanks for approval. I, I will… Does anyone know whom to reach out to merge this one?

To ask to merge this one, basically.

Nico Flaig: I think it's Panda, or someone.

Mikhail Kalinin: Okay, cool.

Nico Flaig: maybe Perry Barnabas, or something.

Mikhail Kalinin: Alright, cool. Then I'll just reach out to them. In order to get this one merged. Alright, so… things that solve for API metrics. As well. And… The last, but not least, research updates. Roberto, do you want to?

Roberto Saltini: Yeah, not much from the 8th, we… Likely. I released the paper this week, To be honest, not much work was done over the past, Four weeks, and we've been busy with… Mainly fast finality and other projects. But, I'm back to it this week, so I don't see any particular blocker from publishing this week. That's all.

Mikhail Kalinin: Cool. Thanks, Roberto. All right, is anything else we want to discuss today?

Nico Flaig: Yeah, just worth noting, so we do have this metric already, maybe Nassar can correct me if this is what we measure, but we have this metric that tracks the resets total. It's currently a load star internal metric, but it should mirror what you have on the Beacon AP, the beacon metrics. And on my note, at least, I see in the last 7 days, there were 3 resets. Total.

Mikhail Kalinin: Hmm.

Nico Flaig: I'm not sure if that's, I have not digged into it at all, why that is the case, but…

Mikhail Kalinin: Yeah.

Nico Flaig: So it should match the fast confirmation underscore restarts total, I think. Should be the same metric. I think that's the one you're looking for, right?

Nazar Hussain: That's correct.

Mikhail Kalinin: Yeah, does it count the initial start as well?

Nazar Hussain: No, it doesn't.

Mikhail Kalinin: Hmm. So there are 3 restarts, as you said. Yo. It would be great to investigate what caused the fallback to finality and then restart.

Nico Flaig: Yeah, I think our metric is not great, also, it's just, counter that goes up. So if this happens at startup, it could be that it already increments it by 1, and then we just keep it at 1. So it doesn't… like, if I change the rate in the wild to, like. higher, like 7 days, and then the startup happened at… at 8 days ago, I think it would still measure that. We need to check how we display this. But yeah, I didn't restart my node in the last, 5 days. But it shows that there are definitely two more resets.

Mikhail Kalinin: Yeah, I see. Yeah. It might be useful to have, you know, some… More debugged information for these cases. Just for us to dig into that.

Nazar Hussain: Yeah, I'll check and investigate a little bit. There's something, like, related to the special word shape, and I will say everything.

Mikhail Kalinin: Cool, thank you. Yeah, mercy.

Mercy Boma Naps-Nkari: Yeah, so I don't know if this is far-fetched, but I wanted to ask, do we, like. how do we, sort of, run different clients' implementation of FCR to actually, like. See if we can get these edge cases out. I'm not sure if my question is.

Mikhail Kalinin: So, yeah, sorry, could you rephrase, please?

Mercy Boma Naps-Nkari: So, I'm saying… is there, like, a resource for, like, who can get from, its founder Ops to, like, run all the client implementation of, FCR for, like, a longer time, and then see how… if similar age cases can, like, suffice, or if the… for example, what Nico is saying, if it's similar to Grandine, or something else is happening, to be able to identify all these… cases.

Mikhail Kalinin: Oh, yeah, I see. Yeah, you mean, like, on the historical data, or just in a, you know… in the, in the runtime.

Mercy Boma Naps-Nkari: Yeah, on runtime, like, for us to, like, run it longer on a server or something to monitor, because.

Mikhail Kalinin: Sorry, can't hear you. You're talking?

Mercy Boma Naps-Nkari: Okay, sorry, I was saying, if we can, like, have a server to run all this implementation on a server… I don't know what I'm saying, like, intense.

Mikhail Kalinin: Yeah, I… I think, yeah, I need to talk to expand the ops about that, probably.

Nico Flaig: There would be another thing, which I'm not sure is necessary, but, so they also have the contributor tool, which you can run as a sidecar. Which is then, used by them to measure, like, block arrival times, From, like, different parts of the world, basically. I'm not sure if that's… relevant. I guess you could see if nodes outside of the network, like, further out at the edge of the network, emit FCR later, I guess? I'm not sure if that's that relevant, but…

Mikhail Kalinin: Yeah, I see.

Mercy Boma Naps-Nkari: I don't… I don't know about this who you're talking about, can you share the link?

Nico Flaig: You mean, like, the Panda Ops? How do they call it? Yeah, let me find the link around…

Mikhail Kalinin: I think we can ask around the hopes also to, to do another… Comparison test between clients. between client implementations. To be fed with, literally the same data. each client. Something that they did before her.

Nico Flaig: Yeah, so I shared the website, I meant. But it's only, like, they measure only the data based on SSE events that clients emit?

Mikhail Kalinin: Oh, okay. Interesting. Yeah, we'll just investigate in that. All right Is anything else? for today.

Roberto Saltini: Yeah, I wanted to ask one question about APIs. I… What was the actual decision about the tax for the EL? side, in terms of safe and don't recall what was decided.

Mikhail Kalinin: We decided to use SAFE. And we also, decided to reach out to infrastructure, to RPC providers. And ask them if anyone If anyone relies on the safe today. And their users.

Roberto Saltini: Okay. And have we reached out to them?

Mikhail Kalinin: No, I'm not sure, but dude.

Roberto Saltini: Okay.

Mikhail Kalinin: I don't think so. But, yeah.

Roberto Saltini: Okay, okay, so…

Mikhail Kalinin: But we haven't, haven't received any, like, we haven't heard that anyone uses it. Before.

Roberto Saltini: Okay.

Mikhail Kalinin: Do you think Yeah, do you think it's important to do… to beat on Roberto?

Roberto Saltini: I don't know, like, I… It's like, it's like a sort of insurance, in a way, like, you… Like, we don't know if there are cases where this can… it's unlikely. Very unlikely, I think, that this… and cozy ships. I guess the problem is that we'll need And half work, right? If we're going to introduce a new tag. That'd be correct?

Mikhail Kalinin: Yeah, that's true. That's likely the case. We… Yeah, to be… formally, we don't need the hard work to make the engine API, change, but… I… I doubt that. I doubt that, actually. Yeah, probably in this case, we would need this, yeah, we would need some coordination to… Hmm. doing CL and DL as well. Because if you introduce a new one, it means that CL as well will have to Just send something like Fast Confirm. Hash, or whatever. aside to.

Roberto Saltini: Excuse me.

Mikhail Kalinin: cash.

Roberto Saltini: Yeah. These can be…

Mikhail Kalinin: Yeah, and this can be done only if EL supports, so this EL will just basically wait for EL to support this. This new data. Hmm. And…

Roberto Saltini: So there is quite a bit of… work on top of Glamsterdam. Absolutely.

Mikhail Kalinin: Yes, yes, so… I would… yeah, really, I think it would be hard to, you know, to… to do this before Glamsterdam, honestly. Right. I'll try to figure out something, about the usage.

Roberto Saltini: Yeah.

Mikhail Kalinin: save block. Oh, okay.

Roberto Saltini: Okay.

Mikhail Kalinin: Right, thanks, everyone. And we'll see each other in a couple of clicks.

Roberto Saltini: Oops. But…

Mikhail Kalinin: Bye.

Nico Flaig: See ya.

Roberto Saltini: Thank you, Kurt.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.