All Core Devs - Testing #38

Transcript

Loading developers...

[Music] Perfect. Um hello everyone. Welcome to ACDT number 38. Today's uh May uh 26th and um we have just three items in the agenda and the main one is uh regarding the Fusaka DevNet. Um basically um I think we should start with that one just to see how clients are uh looking up to uh join the devnet and we can uh continue with pure das and then his

Introduction to ACDT 38 Agenda

expiry if there's anything to talk about also there's another topic by uh that was raised by Anders I think this was uh left incomplete on ACDE and we can um just uh finish it here if if he joins the call. All right. Um so yeah, starting with DevNet zero. Um I think we can uh start with test net test status and then we can follow up with devet status and so on and so forth. Um Barry, do you mind going first or should I I just have a a quick uh uh basically update about the tests. Uh but do you want to All right. Yeah, basically the first

update is that we have a release uh from east side u about the um uh zero it it contains the uh sified uh uh uh EIP basically mod x only the the the two mod xp related uh EIPs they are inside in in in this release and um it's ready for for everyone to just uh run it and we can we also have a hive uh hive instance is already running. Uh but at as as as at the current moment we only have uh Go Ethereum live in here. So please uh check it and if you want and you have a release or you have a branch that you

want to use to run your client in this uh please reach out to Partosh and and they they can or or someone in the E band team and they can they can add you to this to this uh hive instance. Um we have another another uh simulator for hive uh that is getting ready is not uh is is not quite ready yet is regarding the blob transactions and the engine API endpoints that we want to test inside of east. Um the reason it's not ready yet is because we had to make a lot of modifications to east for it to be be able to um send the blob transaction to the clients and also um uh request

engine API uh requests that uh the gap blobs basically um so that's the reason why it's not ready uh but we're working on it and we should be ready in the following days. Uh that's that's it from my side. Uh, Partosh, do you have any anything else? Uh, yeah. So, uh, we still have to update the hive instance with the, um, client images that we're using in Fusaka Devet Zero. Um, I can do that after the call is done. Um, but yeah, Banus is already on the call and I think he has updates on Fusaka Devet Zero. So, I'd let him go first.

Yes, we just had Genesis about an hour ago. uh we decided to schedule Ulu a day from now and the first BPU a day after that. This way we are going to have enough time for every client team to take a closer look if everything's still working. It also allows the new client teams to still uh produce a full ready image before we hit tomorrow. Currently, we're still waiting for Nimbus and Lighthouse on the C side and Aragon on the A side to have a a new image that should be flow ready.

And I think regarding 37, we plan to shut it off tomorrow after the flow fork is done. Okay, thank you. Um I think um we can switch it up now to hear from clients if there's any concerns about uh joining the dev nets. Uh how's everyone looking? Um particularly Argon I think it's uh the one that you mentioned that is missing.

Is there is there any updates from their side? They have a working branch or a work in progress branch. I have other chat. Marius, can you hear us? I think the issues on his side. Um, yeah.

All right. So, and from Nimbus on the CL side, um, do we have any updates? I'm not sure if you're asking me or someone from the Limbus team, but uh, basically they also have a an image that is currently under testing. And from Lighthouse, uh, I haven't heard anything yet. Yeah, I think the last estimate was Lighthouse will give us an image tomorrow.

Okay, sounds good. All right. Is there anything else that we have to um discuss regarding Devet Zero? Is everyone aware of the changes that should go in or have any questions or discussions to be made about DevNet zero? Uh just to be clear about uh the BPO uh the BPO changes it's just uh the EIP as is without any of the changes that have been proposed with respect to the four question changes. Right.

Yeah. Exactly. So the fork version changes and any anything else is still in flux and we would do that for devet one or potentially devet 2. Sounds good. Um all right. If there's nothing else for DevNet zero, should we discuss DevNet one? Um I think there there were two SFIs SFI um EIPs in the last ACDE.

is there anything that we want to discuss that that now or should we defer to to next week when we have uh uh deployed 110? Well, there was one other thing related to the last question on the DevNet zero fork, but one of the things or have been discussing is a config change where we don't necessarily include the um like Electra and Fulu epochs in the blob schedule.

um like we had initially decided to do that, but we've had discussions that maybe that's not a good idea just because it could lead to changes. Um so or it could lead to like inconsistencies in the config, but is the DevNet zero version expected to have those in the config? I'm sorry, could you repeat this? Um, so the config may or may not have the in in the blob schedule, it may or may not have the Electra and Fulu blob parameters. That's one one of the things we've been discussing. So, not only do we need to know what the spec is for

blob for BO forks on DevNet zero, but we need to know what the config is going to look like for DevNet Zero because that may change in future forks. Yeah, I mean here's I just posted the current version for net zero. We already had Genesis. So this is what we have in the B schedule and we still uh we we do not have uh full max blobs but we still have electra because we haven't merged that PR in yet. we're discussing about having the Electra uh block schedule

in the next potential uh test. Yeah, it actually looks like the Electra and DAB parameters are not there in this which I think is what we're planning to do in the future anyway. Yeah. Um yeah, it's still under discussion. That's why we didn't bother to to add them here for now. Okay, that's fine. Also, if we ever going to include those officially, then it's going to be really painful to test anything because then we would need to always go through those in a way. We couldn't just start from

Genesis, right? That was one of the concerns was that we'd really have to make sure that especially across different test nets and stuff like it's a it's a duplication of the existing it's putting the same thing in the config twice and have to really make sure those match probably later not to do that. Yeah. Yeah. So in future future they're not I would assume that we just going to agree on okay which BPO we start the gen rate we're going to have epoch zero set to whatever 18 and then go from There any other comments regarding the config?

Configuration and EIP Discussion

The BPO config? Sorry. the rest of the of the team's uh clear on how to how to proceed. All right. And um yeah, Pari shared the link to the Faka one u specs. I think I think someone mentioned in the chat that 7918 is also included and as as far as I know I think we we speify that uh last week and then there was only a couple of uh changes to be to be made to the EIP

but yeah I think I I think it is suffite. Um is does that mean we want to definitely have this in Fusaka devnet one or do we want to delay that for Fusaka devet 2 and the main reason for delaying would be in case we want to um have the devet up as soon as possible because we would like to have full devet one ready by um the interrupt event in bullet yeah I the the the main reason to select it for devet one was just that it looked

like there was only I think a single new addition to devet one um on on the on the dev call or code of call and we had a few ones already pending for the addition to devnet 2. So it was really just a pragmatic um choice nothing principles behind it. So if it actually makes more sense to push it to DevNet 2, I don't think that would be an issue. And I think the other argument was that uh there was a constant that was missing. Uh I mean it it was the only thing that was left to define. And since the constant is not the ma most uh uh risky part of the EIP, I think we could include inde one. And if the constant

changes, we can change the constant in uh the minute 2. I think that was uh the decision if I remember correctly. Yeah, that's my understanding as well. There was one last change that needed to be done to the EIP. And is on the call. Maybe Anders, you could briefly summarize whether you made that change already or whether that's still pending. Yeah, so I updated uh the EIP uh yesterday. So now there is a constant we we are still going to probably discuss the exact setting for this constant but we have a specification now and I posted it in the uh PM for the for this call also and you can just see the EIP there's a specification now we can use

thanks Anders is the EIP already updated or does it need an update It's already updated. Sounds good. I I think we have everything to proceed with the also the testing uh for this EIP. Okay. Um I've added it to the EIP list for dev one. Excellent. Yeah. Thanks. All right. Um, anything left to discuss uh comments or status from any clients uh for dem 0 or dem one.

I think Dimmitri asked about a consensus pack um PR that is still open. I'm just linking it. I don't have the current status. Do we have someone looking at this? Yeah. Do you mean 4323? Yeah. Ah, yes. Okay. Um, is anybody aware of the status of this PR? I'm not sure if G11 Tech is on this call, but maybe um if Justin or I see Pawan's posted about it.

Yeah, post it on the PR like at least for lighthouse I don't think it that would be useful in any way because we do not request multiple roots in a single uh like columns by root request. So uh yeah that's what I posted on the PR but uh yeah I'm not I'm not also sure like how exactly it will give us uh any benefits in terms of like better bandwidth consumption or anything like that. Not convinced that it's helpful. Yes Gender.

Consensus Pack PR and Client Feedback

Yeah. Uh thanks. Uh so basically uh the aim of the PR was to make the request size smaller and uh it would benefit definitely in the sense that okay if you have a list of block tools basically you know it I don't think that would save time in amount of creating the request but definitely if you have you are requesting a lot of peers then you can send the request faster and they can inspect it earlier and say that okay these are the columns that they might not have and they might also to reject the request earlier. So in that sense the latency of such requests would come down. But uh uh uh given the discussion that has happened many clients uh sort

of uh pull uh different blocks that they get the blocks that they get from gossip and uh definitely their columns might be different. So in that sense uh I don't think that this format is going to serve that mechanism where uh you request the where you are basically requesting uh the columns for those that you have not received in gossip uh although in lots we had a different mechanism going on uh where uh for the uh blocks for which DA is not complete we basically request them one by one. So uh so that was my context for making this particular PR.

But if other CL other clients are basically requesting uh uh the uh the nonDA blocks altogether uh which they were which were to be DA in gossipet uh then basically this request doesn't serve that purpose. So in that sense I guess we can leave this PR. Any comments from other teams um about this PR for Prism?

We do not yet request multiple um data columns by root for different um data column in indices but we plan to do it and so yes in this sense uh this pull request won't help us and we should we would like not to see this pull request merged. So the current status is is okay for us for prison. Thank you.

And Ginder mentions that he's fine uh closing the PR um given the future usage plans. Do you want to wait for other client teams? We I think we we have comments from only two. Do you want to wait for uh maybe other clients uh to chime in before closing? Okay, cool. Great. Um All right. Um yeah, if there are no more comments uh regarding the zero, um we can jump into peer desk uh right now.

Let me see. Who is in the call? Um, does anybody have any updates uh for Pieras in this call? Um, I can talk about prison. Uh so right now uh still a lot lot of code in prism for pureas our is in our um dedicated pure branch and we are slowly merging it into our

stable branch namely develop. There is still a lot of work to do but it's progressing quite well. Yes. Ban and then uh Sunnyside Labs do they want to share u results? Yeah, for lighthouse we've been uh I think other than like we haven't completed uh adding BPO to our uh terminate ready branch and uh we are still working on validator custody. Uh but that shouldn't be required as such for devet uh for the dev net zero. Uh other than that we have

merged a few sync fixes and we have also we had a bug with uh get blobs v2 which we have fixed and merged into unstable. So hopefully we will uh have a branch uh ready to go by the time like the full lu fork happens on net zero. That's it for us. Thank you. Um, Sunnyside Labs, do you want to talk about the the results? Yeah. Hi. So, um, I'm Jay from Sunnyside Labs and we've done some testings um, as shared last week and also there

Network Performance and Testing. Discussing node performance and blobs handling

are some additional tests that we'd like to discuss. So um last week um we were investigating like um where we are on podas in terms of like how many number of blobs can we reach as a whole. So um we ran bit more nodes than the original dev net 7 back by e pandops. So we ran 60 nodes with six different CLS and three different ELS and we were able to reach up to nearly 50 blocks per block.

So um compared to the previous lab um it's an achievement we think because like previously we couldn't even mix this many CL and ELS together at all and we were able to reach like the best results out of like all the nets we ran and we've um after this um update we continued looking into um what can be improved so that we can reach 70 72 blobs per block

and then um we were looking at some bottlenecks and comparing like whether there's an impact on different st strategies on um get blob speed v2 and um like the um consequences of having um get blobs or not having get blobs. Um so far um there hasn't been much difference enabling or disabling get blocks. Um and we also found that um most of the CL's just request once per block

um if they can't get it from P2P network. um only Grand was requesting more than once but um we were not able to run um Grand Stalon to compare the results because of because it was bit unstable. Um but anyway um for lighthouse and prism even with and without um get blocks they were achieving much higher than before. So which which were like 53 blobs per block for lighthouse which previously was 43 and 44 prison which previously was 25.

So yeah, we will keep post like any updates and any findings we have from on the Discord channel. Did you guys happen to look into why get blobs wasn't making a big difference or um yeah because that does that seems a bit counterintuitive for a small Um, I think um we I I really wanted to test Grandine because they were requesting more than once, but I guess because it's a small network, they were able to get it on via P2P.

Um, we still have to look into it. Okay, cool. Yeah. Yeah, I also had another question like did you also happen to test any syncing uh like any node in the network like with a fresh sync and if it caught up to head with these parameters? yeah, syncing was um a challenge because um sometimes we had to restart the entire network because many of the nodes

failed to reync. yeah. Um is there a way that we can manually forward sync? yeah you can do that on lighthouse like you mean uh try to sync without checkpoint sync or understand what you mean. yeah at any point like when a node fails to catch up the aim.

yeah I I I guess we can check um talk offline regarding like those kind of things. Well, and there's a question about um the resource utilization for super nodes. Um whether you know what's the reason behind the high uh resource utilization in those super nodes. High resource utilization. sorry, one second.

Resource Utilization in Super Nodes. Issues with high CPU and RAM usage

Yeah, like in the report it says the super nodes were often running uh with like extremely high um like the CPU being like very much utilized like uh utilizing like a ton of RAM and like these kind of things. I was wondering if you maybe knew if it was like mostly due to column verification or or what exactly? Um just because there had been already in the past this issue that it seemed like column verification was much more expensive that it's supposed to be like just by looking at what what the library um should um like what are the verification times uh for for the library and so it's kind of yeah I'm just wondering if that's still the problem or um if it's something else um the resource utilization problem was

particularly on load Super nodes. Um, it hasn't been an issue with other clients tested. Oh, I see. Okay. Yeah, I I I misread. I thought you just said I didn't realize it said Lar. Right. Right. Sorry about that. Thank you and thanks for sharing the link. Uh if anyone has further questions, I think we can uh um take a look into the report and yeah just uh uh reach out to Sunnyside for questions.

Yeah, thank you. Um is there anything else that we want to discuss regarding pure? Uh yeah, I wanted to ask if there's some specific things client teams want us to be testing on podas devet 7 before we get rid of it. Um I think a natural test would be just making sure we can sync to head with all cls. Um besides that, is there anything else we should check? Yeah, I would say syncing in both in a network which is finalizing and not finalizing.

Um I think it's important to test both because sometimes at least it is the case in prism the cut pass is not exactly the same when you want to sync in the finalizing PL compared to a non-finalizing PL. So yes, would be useful to test this. Yeah, sounds good. We can make it non-final. We can do a basic sync test when it's finalizing, make it non-finalizing and do another round of sync tests. I think that should be straightforward and yeah and especially also I I said this for the previous dev net um I think it's important to run at least a few hours of um

blobs transaction before going to fulu because there is a electra to fulu transaction which can cause some bugs as well because electra has blobs and fu has data columns Uh I was talking about Pas Devet 7 up until now. So that one is the one we would be deprecating this week. Yeah. Yeah. But it's still the case. I mean we we can run uh Electra uh for a few hours in in Pas Devet 7 before uh activating FU or maybe I'm missing something.

I mean Pas 77 already has full sorry sorry okay but I agree what what you're saying is we can apply the you can do that yeah something um Marius I think we might need a malicious get to do that right to create a block with set up transactions. I can I can build you one.

Awesome. Yeah, let's do it. But could you could you expand a little bit more on what you mean? I didn't quite get it. Um, so my idea is to send um like as I as I said in the interrupt chat um already, we should try to make sure that all of the transaction pools are rejecting the the sale proof transactions before fulu. And we made sure um but now we should create a block that has a transaction in that uh that has the sale proofs. So if someone mines a faulty block with a with a uh that those blocks get rejected by the CL.

Yes, Francisco. Uh but what does that mean with like the the cell proofs would be in the network wrapper? So, how could you even put them in in the block? Well, I I mean with a with a transaction where this where the network rapper is is the the the the new subroof. What I mean is like the like today the the current proof type doesn't go in the block anyway. So it would it would be like I mean you could somehow do it but it would be like more than Yeah, it does go

like we don't really have a name for the for the object that we pass through the engine API to the consensus layer. It's Oh, in that sense. Okay, I see. Yeah. Okay, I see what you mean. Yeah. Yeah. So, okay. I didn't understand you meant through the engine API. Okay. Y But how how the rest of the clients would detect this? The CL should reject the this the proof.

I think it would be like a des serialization failure when you get it over the engine API. I think we can explore also doing it in hive. Uh we have a new type of testing in east that we basically use get blobs but I think maybe we can tweak it a little bit to see if we can uh do this test case. I'm not sure though. I'm going to I'm going to check it in in east.

Well, yeah, but get blobs that's different, right? Like this is not the consensus path. Um like we should definitely make sure that if we return cell proofs on get blob v1 or or blob proofs on get blob v2 get blobs v2 um then the the client should reject them. But um this is more about like the consensus pass. Got it. Thank you.

All right. Um, yeah, thank you. Thank you for that. Um, do we have anything else from PE for Peras that we want to discuss? If not, um the last topic that I have in the agenda is history expiry. Uh I left this because we talked about it in the in the previous meeting. So maybe there was something left that we have to discuss. Um but if not that that's okay.

History Expiry Discussion

Uh do we have anything uh that someone wants to share about the update and his expiry? Okay. If if not, uh um is there anything else any other topics that were missing from the agenda that anyone wants to bring up?

Okay. Otherwise we can um yeah the agenda is normally shared in the PM repo uh as an issue. So you can uh just look up there uh ACDT. This is the current agenda. Um just go into the PM repo, go to issues and just look up ACDT which is this meeting. And then you can either comment uh if you want to discuss something during the meeting or um yeah you can revisit the topics afterwards and there should be a um basically meeting notes in here uh after the meeting also. But yeah.

All right. Yeah. If there's nothing else, I think we can uh wrap it up now and uh just wait next couple of days that that net zero uh is launched and we can uh pick it up from there. See if there is any issues. Thank you. Thank you everyone. Have a nice day. Bye. See you. Thank you. Bye. Take care. Bye. Thanks guys. Thanks. Bye. What's up?