All Core Devs - Testing #86

Transcript

Saulius Grigaitis: Hello.

Justin Traglia: Good morning, Sally. Yes.

Good morning, everyone. A few more minutes before we get started. And, PK, Barnabas is gonna be a couple minutes late. Would you want to give the DevNet status update, or just wait for him?

pk910: Let's wait for him, I guess.

Justin Traglia: Works for me. Good morning, everyone. Just wait a few more minutes.

Sam | Nimbus: It'll be like…

Justin Traglia: Hey, Sam, you're unmuted.

Sam | Nimbus: Yes. Okay.

Justin Traglia: Just one or two more minutes. We're just waiting for, Barnabas to join. He's running a couple minutes late.

Sam | Nimbus: What's up, mom?

Justin Traglia: I don't have the ability to.

Sam | Nimbus: L-O-B-E-D-E.

Justin Traglia: It's…

Sam | Nimbus: And…

Justin Traglia: Fant.

Sam | Nimbus: I've been experiencing some discomfort in my joints. So, I'm thinking… People… Say no, no more. Brilliant. Okay. Okay. You can join us! It's right Every week. Okay.

Justin Florentine: Is Sam your mic.

Justin Traglia: this. Barnabas, are you in the call? We'll start after Sam mutes.

Sam | Nimbus: I just… Okay.

Barnabas Busa: Okay.

Sam | Nimbus: Anyway, this is cool. You got that.

Justin Traglia: Can you mute Sam? I don't know who has access.

Barnabas Busa: Oh, yeah, that'd be true. That should be good. Can you guys hear me?

Justin Traglia: Yeah, we can hear you just fine. OK, great. Thank you. Let's get started. All right. Welcome, everyone, to ACDT 86. Our honor business here, would you like to give some definite status updates?

Barnabas Busa: Yeah, sure. So Instagram doesn't have 6, had some issues with, Besu? I have since… recovered most of the busy nodes, but we still have a few nodes that are not happy. Teco Ergon is one of them, and old Star Gas is another. Mainly the issue for Besu was that it was stuck in a weird case, so it required us to resync the databases. Maybe someone from Besu can also give us an update on what they have fixed throughout the weekend.

Karim Taam: Yes, so we did a fix because Prism was sending multiple new payloads without FCU, so I tried to do a fix because this new behavior was adding some latency in Besu. But I had some news recently from Prysm, so it seems that they will revert this modification, and we will go back to receive FCU after new payload, so… So maybe we'll need to revert the last fix I did. And I hope we will go back to something more stable.

Barnabas Busa: Right, but, like, this morning, we had, I think all, all the business notes offline. Was it all related to the same thing?

Karim Taam: I think yes, but we need to maybe revert and test with… because I have really some doubt about this fix. I think it could be the reason of the issue you had. So I think it will be better just to revert this fix and go back to something that I trust more. And if we still have issues like that, I think, we will investigate more. But for the moment, I really think it could be, Because of the last fix I did, yeah. So I will propose a PR to revert, and and we will see.

Barnabas Busa: Okay. Any other client noticed anything weird happening on the Devnet. Mario also is asking where you would be able to share some of the issues, if it was speculated or something else.

Mario Vega: Sorry, that was a question for Karim. Just wanted to know if this is a spec-related issue or was something in the syncing of the node? Or some other detail.

Karim Taam: It was for me. Sorry. No, no, it's… for the moment, I don't think it's a spec-related issue, I think it's more… Because we are trying to match the Prysm behavior, and I think we are creating a state inconsistency. And it's why we have this issue. I don't think for the moment it's related to the spec, but we'll.

Justin Traglia: I'll just.

Potuz: Yeah, I'm a little bit… surprised that this caused issues with Bessel, because this is… our behavior has been like that for months, and it's already released on mainnet, so… and even the fix that I pushed It's only for glass, and it's not… And it's not for Fulu. So if this is affecting Besu even on mainnet, I'll have to also change our Fulu behavior.

Karim Taam: Yeah, we need to check, but I don't think we saw that before. I think we only saw that on the recent DevNets.

Potuz: Yeah, but it's surprising. I mean, this is several months ago. I did move to sending FCU on the background, and then I even removed it completely from init-sync, and it was several months ago.

Karim Taam: Yeah, honestly, I don't know, no, because we are… Testing a lot numbers. Prism, I don't know if we did some tests recently. Yeah. But yeah, we can also check on my net to see if we have the same issue, but yeah. I didn't notice that.

Ameziane Hamlat: Karim, maybe just to add something. So this is related to a new syncing node. So if the node is already in sync. We shouldn't see the issue, so it is…

Potuz: Not really. So any node that starts, it doesn't matter if it's on a checkpoint sync, we're going to hit this path of sending batches.

Ameziane Hamlat: So, okay, let's say I have been running a node, right? And I upgrade my, let's say my best version, I restart. So we have, like, let's say, maybe 5, 10, whatever notes to recover, then once I recover. I should receive for each new payload an FCU, right? Like, when the node is in sync.

Potuz: Yeah, but if the CL client has been running continuously, then that's what's gonna happen. But if the CL client is restarted again, so if the CL client is restarted, then you're gonna hit this path where we send you a batch without a single FCU.

Karim Taam: It depends also regarding the size of the batch, so if it's only a batch of 2-3 blocks, I don't think you will see any issue.

Potuz: It should be at least 64.

Karim Taam: 64, yeah.

Ameziane Hamlat: Yeah, we need to double check, yeah. We need to double check when that happens and if there is anything to fix on mainnet as well.

Justin Traglia: Any other updates for. Definitely 6. Have there been.

Barnabas Busa: Okay.

Justin Traglia: Okay. Sorry, Lodestar, Death Star paintings.

Barnabas Busa: Yeah, it's been… so, Lodestar HX1 is still running the Death Star, that's the image.

Justin Traglia: Awesome. And I see that those are missed, like orphaned, which is the expected behavior, correct?

Barnabas Busa: Yeah, that's correct. And we also know that some clients didn't fully implement fork choice yet, and they would have some issues building on top of these blocks. I think Nimbus is currently still having some issues regarding these.

Justin Traglia: Got it. Okay, sorry, what were you gonna say before? I got you all.

Barnabas Busa: Yeah. So that that was my next point that the new, the numbers for choice issues. I think they're working on it now. Maybe someone from the numbers team can give us an update.

Dustin Brody: Sure. Yeah. We're working on it. We we know the issue. We know the cause. We know, broadly speaking, the fix. I say we. I I'm not going to be writing the PR for this, so I can't promise any specific Timeline, but I guess… Caleb is… is… Like we will, and I don't know if he's in this Anyway… It's it's tractable.

Barnabas Busa: And we still have a bunch of nodes that are self-building only. So I can… I can see a few now. It's… it's mainly Nimbus pairs. That are self-proposing blocks, and they're not using the remote builder, so it would be nice to Nice to get a chain where we use only the builders. Here and there.

Dustin Brody: So, one comment about that, and I don't… is, Nimbus is, currently using the same, like, 10-odd percent, Hmm. Maybe it's higher, but but 10 odd percent sort of Local block, value boost. that we do for the builder API. And so… I don't know what the how valuable the external builder blocks are. here, but, right, that's gonna mask things potentially.

Barnabas Busa: So it's… 0.15E for external, and… Close to zero ETH for local.

Dustin Brody: Okay. Sure. So, anyway, a solvable problem. If that is the issue, then that could be resolved, clearly, but yeah, we will… look at that. The odd thing is that some of the Nimbus nodes definitely do, have… use external block building. Sometimes. Like, it's not completely absent, so… I I can't speak to the exact details of why or why not right now, but It's there's some nuance here.

Justin Traglia: Okay, yeah, something to look into. What do you mean, Barnabas, that we seem to be flip-flopping a bit, like between self-built and…

Barnabas Busa: Yeah, yeah, so, like, for example, Nimbus Ref 1, It has managed to build a few blocks. Like three hours ago, it managed to build three different blocks, but self-building is like pretty much every other. block is submitted. So, like, every hour, it seems to be able to do one or two. Builder, better blocks. So that's why it's a bit, That's why it's a bit.

Justin Traglia: Okay. I'll stick into this. Anything else for DevNow 6?

Barnabas Busa: We now have every single client on board, so that's very good news.

Justin Traglia: Awesome. Okay, maybe good to, now's a good time to discuss plans for DevNet 7. Are we still planning to launch that this week if the spec updates come out and clients are updated?

Barnabas Busa: Yeah, I would target Wednesday.

Justin Traglia: Okay.

Barnabas Busa: Not sure how reasonable that is. That's partially because of personal reasons as well. I will be off for two and a half weeks after that. So, that's why I would push forward to Wednesday. In case that's not doable, maybe we can take a look at launching on Thursday. Or,

Potuz: The breaking changes that we need to push.

Barnabas Busa: I'.

Justin Traglia: I mean… That's the biggest one, yeah. Stable containers?

Potuz: Anything else.

Toni Wahrstätter: There are a few. There is one thing on the EL with the stipend. I think we will get to that.

Justin Traglia: Yeah, I think there's, like, on the CL, I think there's, like, 5 breaking changes, but they're fairly simple compared to 7, 6, 8, 8 Spencer?

Spencer Taylor-Brown: I guess I'd ask, do we want this DevNet to kind of be the final… Devnet, because I think for the EL, there's changes for 2780, 8037, I think, block access lists, and then also the 8282 with the… And the bike codes.

Justin Traglia: I would like for this to be the last of them. What do others think?

Barnabas Busa: Yeah, Spotify is for sure. I would do a mild spec freeze after we launch this.

Maria Silva: Just a quick note there, I think we still might need to make changes to the Prices for… 8038, 20, and 2780, so… I mean… As I explained in the last ACDE, the numbers are, I would say, stable. But there could be something we find during testing that points that the numbers need to change. But, again, those should be very small, like, I… just constant changes, not really spec… like, not… Design… big design changes.

Barnabas Busa: I think single parameter changes should be fine. But like adding a new method to AD37 or changing the whole structure of AD37.

Maria Silva: No, no, no, it would be mostly. Yeah, it would be mostly just changing parameters like the gas cost for account. Right? Or something like that needs to be higher for some reason, so that that would be the change type of change. I mean.

Barnabas Busa: I think that's fine to… Definitely for those kind of changes.

Justin Traglia: Yeah, I wouldn't be surprised if there were a consensus layer specification issue as well. Might require… date.

Gary Rong: Yeah, and also I have a question for mostly, like, Spencer and Mario. So, we found that, the current spell test is diverse from the EIP2780, because, you know, SPED tests, they use, transaction-based calls. as the input for the floor, floor gas calculation. But it's the EIP… but it's the EIP say we should use the sum of… of the regular gas, primitive as the input. So, I can share the link, and I guess it's, like, only the issue of the spare test, and it would be nice to… To fix it for the devnet 7.

Spencer Taylor-Brown: Yeah, no, thanks, thanks for flagging that. I think… I think it is an issue on our end, so we can, we can prioritize that for… For DevNet 7. I think I was planning to take a look after the call.

Justin Traglia: Cool. Okay. Barnabas, anything else you'd like to mention about Glamsterdam devnet 7?

Barnabas Busa: Well, we definitely gonna need the offer 12 release on the sales side. And Use backtest on the ER side for the changes that we agree on today.

Justin Traglia: Yep. OK, I guess that's a good time to talk about those. On the CL side, we do intend to make a release today or tomorrow as soon as we merge all of the PRs and Performa. some tests. hopefully running the reference test on a client beforehand. Oh. Mario or Spencer, are you expecting to make a release today or tomorrow?

Mario Vega: Yeah, I think so. The only remaining questions, I have two open. EAP changes that I was hoping we can discuss today. I think they're in the EL breakout agenda, so we can discuss them there. So once those are finalized, we can start implementing the changes. And I don't think we have to write new tests, but just make these spec changes and we should be able to release. I think by the latest on Wednesday, I would say, yeah.

Justin Traglia: Great. Yeah, same. Pretty much the same timeline for us as well. Hopefully today or tomorrow if something goes wrong Wednesday. Any questions or concerns about this back before the breakout calls? Or any other topics we'd like to discuss. Okay. Barnabas, could you help us by creating the breakout? Groups.

Barnabas Busa: Yes, sir.

Justin Traglia: Alright, while he does that… Reminder that the SSZ Engine API breakout call will be this Friday. Please attend if you can.

Barnabas Busa: It should be open.

Justin Traglia: Thank you, Barnabas. I'll be in the CL breakout call by everyone else. Talk to you guys next week.

Barnabas Busa: Pooja, do you know how we could do the recording for these breakout channels?

Pooja Ranjan: We can definitely make use of OBS. Do you want to? Do it right now.

Barnabas Busa: Yeah, so it would be nice to have the recording of these as well in the, in forecast.

Pooja Ranjan: Mmm. Yeah, same.

Justin Traglia: for the CL. I'll join the CL and try to figure it out.

Pooja Ranjan: Yeah, ma'.

Justin Traglia: Never.

Pooja Ranjan: I'm I… I'm not sure. I think, Mark Wolovin is there in the, He may still have access to record it. Let me quickly coordinate with him and get back on it.

Barnabas Busa: Okay, thank you. Yeah, we're gonna… we're gonna… one of us is gonna record the CI session, and then, If Mark can record the session, then that should be good.

Pooja Ranjan: Yep, let me ask him. Cool.

Barnabas Busa: Does anyone have issues joining into a breakout room? Please let me know.

Pooja Ranjan: Barnabas, if you could maybe guide me there. You, you want me to record the EL1 or CL1?

Barnabas Busa: The EL one, please.

Pooja Ranjan: Okay, I'm a good linchman.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.