All Core Devs - Testing #47

Transcript

Mario Vega: Hello, everyone! Let's keep people a couple more minutes before we start. Thank you.

Thinking we can move this room over and get started.

Pooja Ranjan: We are, live.

Mario Vega: Thank you so much. Hello, everyone. Welcome to Acdt number 47. Today's August 4.th We have a couple of items in the agenda starting by Fussaka. So yeah, let's just get started. I'm gonna ask, is Barnabas online? I think so. Hey? Yes.

Barnabas Busa: Okay.

Mario Vega: Updates on Fuzaka. Devnet 3.

Barnabas Busa: Yeah, currently, we have 2 that are struggling on business, 3 nimbus and roadstar. And also captain, seems to be offline right now, we have 90% participation. So every other client seems to be doing pretty well the all the initial issues with the suggested values seem to have been resolved. There's only one more issue on Low Star, where, right after the startup, the values are not set according to the finalized epoch in the Genesis State. But I've notified them about this issue. Maybe someone from Losar can give us an update in.

Matthew Keil: We're looking at getting the update for going off the genesis validator set. We're looking at how we can actually implement that. And then the Sync bug, I am still actively working on it. It's been proven to be pretty difficult. We're doing some refactoring in order to just improve the code base, because, it's what's making it challenging in order to actually get it resolved. But we are actively working on it. And we have a good plan to going forward, so we should be should be there soon.

Barnabas Busa: Right, and on the number side we seem to have some super nodes that are having hard time getting peers. So most likely they got rate limited or banned by others. Maybe someone from the nimbus team can also speak up.

Mario Vega: I don't think it's anyone from from nimus team at the moment. Maybe you can get back to this after they join. If they do anything else on the my, my question is, is vpo activated. Any Vpo activated on the.

Barnabas Busa: Yeah, all all 5 have been activated. And we are currently hitting the 20 blobs. Usually, we had some issues regarding the mev relay. There's some transaction propagation issues. And hopefully, we can get sorted. But also the builder workflow seem to be working on every single client fair, which is very good.

Mario Vega: Anything that we need to test on the Meb workflow for Bpo. My, I mean, I guess it was fine on this. But is there any lesson learned, I guess, from activate the Mepo on on the Mev side? Or was it like smooth.

Barnabas Busa: No, that's that's pretty smooth right now. The major issue on the Mev side is we're trying to create a private pool. And we send the transactions the spammers, sending all the transactions to just the mev relay machine that is running the rest node. And this ras node is unable to share transaction with the R builder. And that's where we are struggling to basically rebroadcast these transactions and include them.

Bharath Vedartham: Yeah, I have a, I like, basically an update on that. I think so, yeah, so basically, like, the our builder uses Eth new pending transactions, the subscription Api to get transactions from the Red Mempool and so we've disabled transaction pool like local transaction propagation in the red node and be. And you know, today I was just talking to the red team. And it's because of that flag transactions are not being streamed. So we currently have like, 2 ways to get around this right now. One is yeah. Our builder has implements et Ross and transaction. It implements its own Json. Rpc, server. I. We are like, we can try like sending transactions from spam or to that

Rpc, to that Rpc server, and that doesn't go through the Red Mem pool. So that's an that's our bill. Their own transaction pool, if we can. That's 1 way yeah. So and the other way is like, like, I'm seeing I'm just talking to the red folks like, if they can probably like, just like, remove that flag. And we can implement. So that currently doesn't have a flag to disable transaction. Gossip can somehow get that flag and could probably remove the local transaction propagation flag and things should work. So we currently have, like a couple of ways forward. There's some complications with spam, or, I guess, with directly sending it to the builder. But but yeah, like, we've identified like, why, transactions are not getting included in the block with

when we send only to the Mev relay machine as Barnabas mentioned. Yeah, I'm currently like exploring, like, what are the best ways forward? I think we have like 2 ways, and I'll I'll I'll see like I'll give an update like when it when we finalized on something.

Mario Vega: Everyone.

Roman: Yeah, quick question. You said that rath has a flag to disable the expo local propagation. You said that it's not working.

Bharath Vedartham: No, no, that's working that's working

Roman: Okay.

Bharath Vedartham: I've put in the Devnet. Mav. Tg, chat. There's another there's a flag for which you there's implemented for optimism to disable transaction gossip in the peer to peer network, which is not, which is not there for the red. Like the ethereum. l. 1 el that flag. Maybe, like you can have that flag. But yeah, the transaction pool flag to disable local transaction propagation that was working fine like.

Roman: Okay.

Bharath Vedartham: You, we can follow like Roman, you can follow the the chat in the Devnet Mab testing. Tg, I think you have context there, like.

Mario Vega: Thank you. Any other findings on Defnet 3. From any other clients. debugging issues or or similar? Alright, was was that everything that you wanted to share Barnabas, or was there anything else that you want to touch upon?

Barnabas Busa: May. Maybe Agnes can give us an update now from Nimbus as well. cause he was just joining.

Mario Vega: Thank you.

Agnish Ghosh: Hi, so we had a couple of issues. Last week. One was, I think, each das guardian we were having a custody column mismatch. So whenever it does guardian requested us with all columns, if we were a full node, we would return with our custody. The columns we would custody. But surprisingly, somehow, the column indices don't match. I think the indices don't match after a couple of validator custody updates. We'll just check into our code. If there is some hash function stuff going on such that the indices are being inaccurate after couple of epochs, because I think we do pass ef test vectors. So so I'm trying to looking, I'm trying to look into that apart from that. Yes, yeah. The the earliest available slot

it was was not moving. So right now that thing looks fixed to me. We have a build triggered. I think it should be picked up in Devnet 3 in a while and general updates. We had mev issues they are fixed now. So we, I think Bharat can confirm, or somebody maybe Barnabas can confirm. Nimbus is. I think it. They are proposing Mev Block. Yeah. So what else? I don't think anything else I mean whatever. Oh, yeah, okay. So there's been an issue being raised in discord where members aragon one and get one are having a tough time to sync and and of course,

we are. We are basically being asked, why are we proposing? Even when we are not caught up to the head. But I think that's like that's something we have by design. If we are short range thinking, and we are like not 2 to 3 parks away. We don't turn off validator duties. Yeah. So if we are unable to sync, it's definitely a da or a peering issue, either, we are not getting all the required columns on time, or we are not being able to peer. Well. So I'm currently looking into that numbers. Aragon one get one. They have some sort of peering issues. Yeah. so yeah, nothing else. Nothing else from our side. Yeah. Overall. I think there were a couple

couple of peering issues. Mev issues indefinite 3. They're all fixed now. And yeah, hopefully, I'll fix the das guardian thing. By next couple of days, and what else? I think there is a there is a crash Barnabas sent me it was a seg fault. It's happening in a very specific scenario. I suspect it's happening because of indicating the earliest available slot incorrectly or something of that thought. I'm currently looking into that. So yeah, nothing else from us. Yeah, that's it.

Mario Vega: Thank you. Mark.

marek: Yeah. I just want to mention that. Never mind has an early integration with our builders. So it would be great if we can try this as well. And by early, I mean, of course, it does require more testing, and so on. And it is not being run on production. And also it's like integration with State. So never mind, can work with our builder as a State provider, but it still uses our evm. Us. For execution. So it's it's only replaces the state. Yeah. But just wanted to mention, because I heard some discussion about our builder and this kind of stuff let me share the link.

Alright, that's all from my side. Is it fully ready? So, as I, as I mentioned the state is is ready, of course. We would appreciate more testing here and then that would be great here. And but Evm part is not being done.

Mario Vega: Thank you. Yeah. Any other updates from anyone else on any any team? If not. Yeah. Thanks. Thanks for the updates. And it seems like we're doing fine on the military. Then. What's the follow up plan here? Do you know, Barnabas, do we plan to keep definitely running? And what's the next steps? Or

Barnabas Busa: So ideally, we want to have Nimbus and North Star. basically bug free before we go on to the next step. And we have 2 more decisions to make today. And based on that, we're gonna decide. If you want them at 4 or not.

Mario Vega: Perfect awesome. Yeah? All right. Yeah, then, let's just move on to the next discussion topic, which is related to Eip 7,825, the transaction gas limit gap there was a discussion on last Thursday about rolling back the gap well up, rolling up the limit to 30 million instead of the 16 million that we have. Currently, I think there was an update from the person that shared this on act, and whose major concern was raised on Acd. Sorry on icde last week. and I can share that response here. If this person is is in the call. Currently but otherwise it seems like the main concern is not really concerned from from the read of the text. is there anyone here that can elaborate on this?

I don't think I don't think this person is on the call.

Toni Wahrstätter: Yeah, maybe I can. I can stay a little.

Mario Vega: Okay.

Toni Wahrstätter: Yeah, it was a concern that there might be some apps that might be affected. For example, very complex defi transactions that go through many different pools. and Duncan brought it up, and that of course, it's it could be a backwards compatibility issues. But it seems like, after waiting. Yeah, the trade-offs. We can just keep it as is as we have it on Defnet 3.

Mario Vega: Sounds perfect. Any other opinion on on this matter, or this is gonna be like a very short discussion? Alright, yeah, if nothing else. I think this is the way to go. We keep the limit in 16 million. and which is great news for testing, because we don't have to perform. The analysis of the testing back doors are breaking again. So yeah, great thanks. All right. Next topic is, this is a recurring topic on gas limit testing updates. I think Barry is the main person to reach out to this. Or do you have any updates, Barnabas, that you want to share about Gatlin testing, or there's nothing to discuss. Maybe today.

Barnabas Busa: Not from my point of view. No. And I think part is available today.

Mario Vega: Alright, no worries.

Kamil Chodoła: Only thing.

Mario Vega: Come in. Yeah.

Kamil Chodoła: I can chime in, yeah. Hi, Albert, Hi, everyone. So yeah, basically, we are still trying to achieve the stateful testing being capable. But we are tackling that from various perspectives, and we still need to verify few things. Marching also is trying to do some work around our test. Generators in gas benchmarks to see if this will be any more helpful and will speed up the development. In meantime, esta integration working with you guys to to make it to runnable on our infrastructure, and I I believe I am almost done with that, like I need maybe one or 2 more days just facing some unexpected issues with that. And in the meantime Pari was discussing with with all of the teams, checking our readiness, maybe for any other gas limit upgrade in future.

So we have another version of gaff checking. If modex scenarios are being improved. and in general I would say yes. A lot of modex scenarios are even twice or 3 times better than it was before, but still the worst cases are not being properly attacked. Maybe there is some some tweaks which need to be added on the, on the adjustment made on Gef. Plus there are some regressions. So there are scenarios which are performing 50% worse. Comparing to the to the previous results. So, yeah. so this would be like the major concern about any further increase. But if this will came up, being good, which is, looks like it is right now, but just needs few more tweaks. Then we can consider doing kind of thing forward.

But still, also, the stateful scenarios are really really needed. And also we have a bloatnet which have already stayed on 2 x 2 x size, and we are preparing for some test cases for some testing. Yeah, so hopefully, some results will be this week.

Mario Vega: I can also update on the East side. So what we propose on doing there on the stateful testing is so we have the execute command that we use to run our test in in in running devnets. And the idea here is that we prepare some test cases where we rely on a contract that has a very large storage or state size to be already present in the chain, which I think is the case for Blotnet, and a lot of the contracts are in there. So basically, it's just a matter of updating our repository to receive some flag.

Milen Filatov: To point.

Mario Vega: To point it to to the contracts that are already existing there. So it's gonna be. It's not a trivial change, but it's also not not not breaking. But yeah, we'll continue working on that. And and we can keep you updated. Marshine brick. Yeah.

Marcin Sobczak: Hi, I would like to say that we are developing one more tour tool to to check the potential issues with modex repricing. It's Rpc. Endpoint price block. But with custom spec. So what we are doing we have a script which is getting a trace from like current chain. So on, Petra and another one doing the same tries, but with Fussaka spec. and we are comparing receipts like. The difference in gas usage is expected but we are trying to to check if there will be some other changes like broken contracts, etc. for now it's I think, like proof of concept. It's it's working. But we didn't perform the actual test yet. But yes, we are planning to do it soon. So we'll keep you updated.

Mario Vega: Thank you. Men.

Ben Adams: Would it be 60 million? The next level that we'd consider.

Kamil Chodoła: I believe if we'll consider anything like 60 should be the next step. But we need to perform a lot of testing around that. But yeah. small but still significant. Jump.

Mario Vega: Yeah, I think we can prepare testing for next couple of weeks and gather some benchmarks before continuing our decision. Is there anything that you need from the tests side, Camille, a new release? Or how are you looking? There.

Kamil Chodoła: As I mentioned on some of the Tg channels, is like new release with new tests. Which you you added Louis added, around modex as well, plus change of the gas limits. But this is also like being discussed right now on on how many of those we need. But yeah, to unify the it a little bit more. And yeah, basically the recording stuff around genesis files which I'm fighting right now. But yeah, after after being able to run those tests on our infra. I will let you know if there is anything else to be improved. And then from the statefuls perspective. Yeah, we need to think a little bit more around that, and hope to help to make it. The workaround, like execute the capability, is a good workaround, for now, which probably we can use, but will be very time consuming to to execute everything properly.

But yeah. we just need to figure out we, and also the tool which Martin is trying to revive and adjust, to, to be able to generate on top of existing state and the custom Genesis files would be also helpful there. But yeah, we we are fighting with that right now. So whenever there will be anything to to to help us with, I will let you know.

Mario Vega: Excellent. Thank you so much. Great and just bringing this up, surfacing this this message from Barry. He kind of speed right now, but it seems like the get team would take some time to upspring any of the changes if they decide. Is there anyone from the get team here that can a little bit more context to that. Or if not, that's not a problem. I don't see anyone from Geth. All right. I think we can just follow this Async. Alright cool. All right. I think. We can just pass until on on the next Next discussion topic, I'll leave the intervals per slot discussion topic at the end. We can just jump into right into the eips, 79, 51 benchmarking results. These were shared from the Bessu team. Anyone from the Bessu team that would like to expand on this

this topic.

Justin Florentine: Hey there! Unfortunately, the folks that are curious about that are in the aus time time zone. This is in reference to the gas improvement that we found with the r. 1 curve. Is that correct?

Mario Vega: Yeah. Exactly. Yep.

Justin Florentine: Yeah, I don't really have anything in addition to add other than the notes that have been posted. Sorry for that.

Mario Vega: Yeah, no worries, but it seems to me I I've read through the comment, and it seems to me that it's It is not a concern of how the the new precompile is. Is is what's the cost on it? So it is not. It doesn't seem that it's a concern. Actually, it seems like the The cost is is too high, maybe from the new precompile. But I think that's not something that we're willing to discuss today. Maybe. Yeah, if there's nothing else on this topic. Maybe we can continue to sync.

Justin Florentine: I think that would be fine. Thank you.

Mario Vega: Yeah, alright. Thank you so much. Yeah. With that, I think we can just jump into the intervals per slot. Justin, do you want to. Take it up.

Justin Traglia: Sure. Hello, everyone. I would just like to raise awareness for this. Pr, I just shared the link. It essentially defines explicit slot component times in milliseconds as a percentage of the slot duration time and as a configuration variable instead of a preset like youth, Panda Ops requested, and potus I'm not asking for us to decide on this today, but I would like to make a decision, maybe at Acdc. Whether or not to merge it sooner than like 3 months, 3 or 4 months from now. So that we can start working on other specifications. It would be beneficial for epbs like Efp, 7, 7, 3, 2, and 7, 7, 8, 2 shorter slot times. it wouldn't actually change anything. It would just add additional configuration values or sorry. Yeah, additional configuration values.

but no substantial changes. It would deprecate intervals per slot. But once again, you don't have to stop using that immediately, so I see no reason why we couldn't merge it now. Fucking down.

Mario Vega: I think the main question here is whether this breaks Fussaka or changes anything in Fussaka.

Justin Traglia: It really shouldn't. No, I mean, clients might have to update code to handle the new configuration values, but they could essentially ignore them until Clamsterdam. or until we start working on claim studio. Yeah. Shouldn't do clients have any opinions on this or maybe just look into it and discuss it later. Please leave reviews.

Barnabas Busa: There was another question regarding possibly moving some values from config to presets especially the number of columns. I think. Yeah, this is.

Justin Traglia: That is a separate issue.

Barnabas Busa: But yeah, yeah, it's a separate issue. But I'm I'm just curious. If we should actually take a look at all the different config values. And consider moving some of them to preset, because I feel like there's more than just one. Something in the.

Justin Traglia: Yeah, a number of columns could be a preset, probably should be. It also doesn't actually have to exist, since it will be all it'll always be the same as Salisberg's, tended Bob, but it is a nicer, variable name.

Barnabas Busa: Yeah, then maybe we can just get rid of the unused variables. It would also just clear things up a bit.

Justin Traglia: Yep.

Mario Vega: And Pawan mentions in the chat that he will review it soon. but it looks good at 1st glance.

Justin Traglia: Thank you.

Mario Vega: Yeah, if the rest of the teams can just give it a look. And I guess Acdc is the way to continue this conversation and see if it gets merged.

Justin Traglia: Yes, that's right.

Mario Vega: Cool. Yeah, thank you very much. alright. I think that's all of the topics we had to today. So I can open up the floor to any other points or discussions that we have.

Barnabas Busa: There's 1 more

Mario Vega: Yeah.

Justin Traglia: Yeah.

Barnabas Busa: Regarding 4, 4, 7, 7.

Mario Vega: Is this related to sorry.

Barnabas Busa: This is related to Futaka. Basically the the proposal was from Lido to increase the balance for additional custody groups. So basically, every single 128 Eth. Would increase the number of columns that the validator would need to custody, and they push the decision making to today, instead of making a decision on Thursday, because it was very last minute on Thursday.

Ben Adams: And this would need a new devnet.

Justin Traglia: Personally, I'm against this change. I think we just keep it. How it is currently.

Mario Vega: Yeah, I think this is the only feature that requires a new right. It was before this call. It was this, and also 78, 25. But now it's just reduced to this only change for a new. So with that in mind, with that in mind, do we? Is anybody Bro. Doing this at this call the ones to raise the the point.

Barnabas Busa: There was lots of support from different cl devs. So that's why I thought this should probably bring it up again.

Mario Vega: I see I think the moment is now, because otherwise, if we delay this further, I don't think it's gonna be any any good to the development processing. It's already too late. So either we. either someone from the Cl that is supporting this speaks up now, or we simply just reject it? If so, if there's anyone in this call that wants to support this, please speak up now.

Csaba Kiraly: I just just wanted to say the time I'm against the change, because it's so. The this is last minute. because it we might with Bpo scale to a high number, and we think that with the Bpos we will not have a chance to do it later, so we would have to do it. Now. my take on. This is that if if we, if we will need these kind of changes. then we should do this together with the Bpos, and not quickly now. So that's why I would, I would say, don't do it now, and.

Barnabas Busa: I'm actually super against doing that. I wouldn't want to include anything else in Bps.

Mario Vega: Yeah, if we add this, it would be like a new fork. Actually, I don't. I don't see how it could be. Only vpo.

Csaba Kiraly: Yeah, it wouldn't be proposed, because not. It's not not a broad parameter. So by definition, it's not a Bpo. Then I'm just saying I'm I'm skeptical about Bpos being Bpos. For because of these and and and the other things that will come up similar to this.

Barnabas Busa: I think if we say that we're gonna reject this from Saka, we might consider it for Amsterdam, but not before that. I wouldn't consider any substantial code change for any Bpos.

Mario Vega: Yeah, I think that's the general sentiment right now. I don't see anyone pro Agnish. First, st Anisha nico.

Agnish Ghosh: Yeah. So I'm against the change. Simply because I think if we raise it to 1, 28, the number of super nodes decrease, and the number of nodes with a variable dynamic da of sorts which would keep swaying based on attached validators. That thing is tricky, usually like on on a Cl implementation level. I have discussed this with multiple people, Jimmy. Manu It's tricky for all of us to sort of declare da and keep blocks resolved unresolved across various stores. Based on the dynamic da. I have. I have probably raised concerns about this with Francesco and a lot of other people over the last couple of weeks. I know that. Okay, we have this design and this design is there for a reason. But

I would definitely want to see more super nods. Hence I'm against this change. I I just don't want more nodes with this swaying, dynamic da condition.

Mario Vega: Got it. Nico.

Nico Flaig: Yeah. Just want to bring up that. Because I saw in Glamsterdam we could change the value. But I think, since we cannot reduce the Cgc. I think this would not be possible right. because the existing notes would just keep the previous custody they had. But I'm also not strongly for this change. I think it's a bit late, and I guess, too, too little analysis is done to do this. Now I think.

Mario Vega: Thank you. Alright. Yeah. It seems like the general consensus is that we do not go forward this change so that would mean that we wouldn't have them before. That means that just going back to the point that was raised earlier by Barnabas, we only need nimburs and load. Start to be bug free before continuing. So I guess instead of any changes, we just focus on fixing box this week, and we follow up next week with the next steps on this. All right. Yeah, thank you. Thank you. Everyone. Sunnyside lives. They wanted to share something. Is anyone from Sunnyside lives here that wants to present.

Jae Hee Lee: Yeah. Hi, so I think I shared the link a while ago. So I will just reshare it. Yeah, we've just released a new report, which contains mainly on high block loads, plus regular transactions, and perfect perfect pdos. So for the 1st one, we sent high block Britwood on the around 22 million gas. and we were able to find find that we could reach up to around 60 blocks per block with that load, and there wasn't big impact on bandwidth usage. So it was going quite well. And then we further limited the bandwidth of each node for both shipper node and full nodes as per eip. 7, 8, 7 0 hardware requirements.

and it the network performed poorly that it only got 20 blocks per block, but later Francesco suggested that maybe we should be less restrictive on supernodes, that 100 mega megabits per second for download and 50 for upload is too low, so from next round we will retest this with unrestricted bounds for sugar notes. Assuming supernodes will be available for one Gigabit network will be available for Schmutz and for the second network for the perfect periods which is having 16 8 validator nodes uniquely custoding one each column.

and we were also able to achieve 60 blocks per block on the 22 million gas usage, and also 30 about 30 blocks per block with bandwidth limits. However, we found that Genesis sync was failing from full app books on perfect theaters network. So We are looking into it, but it seems like There are only 8 peers on sync nodes, and the sync nodes are trying to find the columns their peers don't have, and it just fails at the very early stage at full we also did backfill test on Teku we found an issue reported to tech team, and they are now fixing it. We also would like to test other sales backfill once they are ready.

and very lastly, for nimbus and load star. Please let us know when you guys are ready for block group protesting, and other validate validate the cost of the test tests that we have, because for numbers I heard that in versus having pending optimizations so that hype high block triplet isn't available yet, and for loadstar. Full notes are subscribing to more than 8 validators. Nearly as same as super nodes. So please let us know when it's not ready? Yeah. Oh, sorry. I say, hands up. So.

Mario Vega: Yeah, not. Just.

Jae Hee Lee: First, st yeah.

Mario Vega: Yeah. Manu Manu was 1st and then Agnesh. Thank you.

Emmanuel Nalepa: Hi, I can quickly give an explanation about perfect pureness, about syncing in perfect pureness. why it fails, and why at least, prism fails. So by the definition of perfect, peerless another has absolutely 0 peer clustering the same command than the node. So if you start from Genesis, or even from from even from a checkpoint. Then you want to fetch some some columns. And yeah, you don't find any peer. So the only possibility you have the knowledge to is to retrieve any 64 Coltons, you know any one of them from from your peers and then reconstruct the colon. You you need yeah, prism doesn't do that. Currently, I'm I'm we are working on it. Target is this week and end of this week.

And I'm not sure that a lot of other clients already implemented. This this feature, sinking in perfect purdues.

Jae Hee Lee: Yeah. We also tested lighthouse, and Granddyne and lighthouse doesn't seem to sync any fully slots, while Grandine, I think Granddyne also stops at very early fully. Epoke.

Emmanuel Nalepa: Actually, I think everybody can can sync flu, but without any blob. So the the connection, which is not okay now is post flu with blobs.

Jae Hee Lee: Right, right, right.

Mario Vega: Yeah, thank, you.

Jae Hee Lee: I'm.

Agnish Ghosh: Yeah. So currently, members, supports up to 50, I think 52 blobs from the last from the last of test that Sunnyside labs ran, so is there. Is there an actual priority to actually work on optimization that can raise up like upwards of 50 right now, like for Fusaka. I I just wanted some clarity on that, because. Yeah. Yeah. Okay, go ahead.

Jae Hee Lee: Yeah. But I wouldn't say that was a stable achievement. that the changes failed after that flow right.

Agnish Ghosh: Right, right.

Jae Hee Lee: I'd say 40 plus is a safe margin for Nimbus when we test it.

Agnish Ghosh: Okay.

Jae Hee Lee: At that time. But yeah, are we targeting something higher, at least for in-in the idea that not seen scenarios, I think we should get something higher.

Agnish Ghosh: Do we actually need something above 40 for Fusaka mainnet like? That's the open question. I'm not. I'm not sure.

Jae Hee Lee: I I think someone else could clarify.

Agnish Ghosh: Yeah, exactly. If someone else can let me know, because, this work can be done. It's just not on priority. So oh, yeah.

Mario Vega: I'm gonna pick on you, Barnabas. Do you feel like this is a requirement for next devnets or main shadow forks, or

Barnabas Busa: That really depends on what kind of Bpo values we expect to do before Amsterdam. We haven't really been down the rabbit hole of deciding Bpo values. Yet I think it's gonna be a totally different question. But I would expect that we want to be able to do 72 blots before transgender them.

Mario Vega: So it feels like right now. It would not be a blocker, because we have not decided on Bpo. But before deciding on Bpo's we have to have this working right?

Barnabas Busa: Yeah, I mean, we don't need to ship right now. 72 blob support. But technically, without any physical hard work, we should be able to scale up to 72.

Mario Vega: Okay. Manu.

Emmanuel Nalepa: Yeah. Sorry.

Mario Vega: Yes.

Emmanuel Nalepa: It was because it was before. So, thank you. Actually, yeah, there's a question. There is some question in chat about syncing, in thinking, in perfect paradise. Francesco is asking if it is. If do we really need this thinking in perfect, careless mode? I think we need? I mean, I will be more comfortable launching flu with this feature. Because I think that Evan is superno that are really helpful. I think a nod should be able to think. even without any super node in its peer. Maybe because not enough supernode will be in the network. Maybe so, maybe super node will reject incoming peers. And so that's why I think that we should be able to think in perfect parallel mode.

Jae Hee Lee: Yeah. Adding some data that I got which which is also mentioned in the report. But what what I saw was what we could guarantee so far was that with the tested Ceos for the 1st 1,000 slots. Every node seems to have their columns correctly so. But it would be definitely much more feeling, much more safer to have this feature and imperfect theaters think available.

Mario Vega: Okay? And there's another comment from Francesco. Francesco, do you want to raise this comment or.

Francesco D'Amato: Yeah, I mean, basically, I was just saying that I'm not saying that you necessarily need to have or like, I wouldn't expect that. Okay, let me backtrack a second. Yeah, I I wouldn't want the requirement that you need a super node in your peers that that's not what I'm saying more just. I'm wondering if it is reasonable to expect things to work when you don't have anyone in your peers that has the columns that you want, and not only you don't have in your peers, but you cannot find someone in the network to be your peer that has those columns like. I just don't understand how you would even exist in the network afterwards. Like you want columns, you know, 1, 7 and 8, and you can find anyone that wants those columns that look like also kind of has those columns, and generally wants those columns like you could maybe get the other ones and and sync by doing this trick of getting more of them and reconstructing. But eventually you'd just not be able to

keep going on the network.

Barnabas Busa: So you're basically saying that a full node shouldn't be required to reconstruct in order to sync.

Francesco D'Amato: I just unclear to me that there is necessarily that much value to it like I seems to me that if you cannot find anyone that is willing to give you the Comms that you have. You have a much bigger problem. Like. You're just even if you manage to to solve the kind of immediate problem by doing this reconstruction thing that doesn't. That still doesn't seem like it leaves you in any kind of a good position like in in this perfect beard. Us? Setting. That's maybe okay. Because yeah, by by design, there isn't anyone that has your columns, and I don't know the only you know you have, like. yeah, you'll you'll be like, alone in your topic. And you, basically, I guess you just receive the columns directly from the block proposer every time, but like in a realistic setting that doesn't really make sense for you to just like exist independently without anyone that you connect with in the topics that you care about.

Mario Vega: Alright just for some out a little bit. It seems like we don't have the capability right now to go up to 30 plus blobs for Fussaka So we don't need this feature right now. But we do need to ensure right now that we are gonna perform this testing again the moment that we want to. Increase the Vpo limit right.

Barnabas Busa: This has nothing to do with that. as far as I understand. Like, it doesn't matter if you have 4 blobs, or 30 blobs, or 300 blobs. If you if your full node cannot reconstruct and sync up using columns with the user co-coding, then that's yeah. That's that's a totally different topic. That's a perfect topic. Now.

Mario Vega: Alright. Then do we have any other opinions on other sales about this matter? How do they fill chicken, Fussaka without this without this salt? Yeah. Agnesh.

Agnish Ghosh: So if a full node chooses to reconstruct because it doesn't find the accurate peers. I don't know. How is that a problem? Because if it catches up to head, then it just again progresses normally right from like by getting columns from either the blob pool or the proposal. No, I mean, I mean, it's a it's a it's a problem when it's sinking, of course, but reconstruction has several sort of guarantees to end. Let's say the da of a slot within a given timeframe. So I would. I would obviously prefer reconstructing than actually actively going and trying to find peers, because I think the latter would take me more time. Yeah, I agree with Manu. Actually.

Mario Vega: Alright. It feels like the sentiment right now is that we do need this feature. Is that is my reading correct?

Agnish Ghosh: Yes, yes.

Mario Vega: Okay? So yeah, I would. Like the Cls to speak up and and say, how much time do they need to to get this implemented. It seems like it's a blocker issue for Fuseka. Any other Cls that have not expressed anything today about this issue. No, we don't, Jacob.

Pawan Dhananjay: We don't have this feature right now, but I think, like we have the mechanism to implement this. so I'll have to look into. I'll have to look into it. Maybe I'll take a look this week, and by next week I'll have something ready, or like at least know how long it'll take.

Mario Vega: Thank you anybody else. Alright! I feel like this is pressing enough that we have to Talk about it again on Thursday. Please to the rest of the Cl teams. Please look into how much time it would take to implement this, because it seems like it's a blocker issue, and we need to talk about this on on Thursday, so we can. We can touch upon the topic yet. There! Again, and see the raise the status. Does that sound? Okay for everyone.

Jae Hee Lee: Yeah. And if you guys need testing, we can just launch new Dev notes at any time. Or just please let us know.

Mario Vega: Thank you. Excellent, all right. thank you. Is there anything else that we need to talk about? Right now? Any other.

Barnabas Busa: Did you choose?

Mario Vega: Sorry. Backfield. Okay.

Barnabas Busa: Backfill. Yeah, I'm curious. If any clients have implemented backfill for lapse.

Mario Vega: Manu says, prism is in progress, same from lighthouse.

Matthew Keil: Lowestar is actively working on it, but it's a pretty big body of work, like everybody else.

Barnabas Busa: Do we want to actually wait for backfield implementations to be done before we launch them at 4, in case we do want to launch them at 4.

Mario Vega: It seems so at.

Barnabas Busa: It would. It would be good if we had at least 2 clients that have done it, and then we could test those at least. And then we can actually go on the Shadow Fork and whatnot could we get like an eta on when clients think they would have backfill implemented.

Mario Vega: Any of the clients. Is this bandwidth issue? Or is it like something intrinsically complex about the the feature that anyone wants to share.

Agnish Ghosh: I would like to go ahead. So the issue with backfill is more of a CPU usage thing. I would say, like it really depends on how quick do we want to get this backfill done like, if we want to get it quick enough, then we are basically allocating a significant amount of the CPU to do this thing rather than focus on. Let's say, increasing. yeah, it doesn't. Yeah. Okay, so rather than focus on let's say, higher block count or a bit more stressed environment when full is actively running. So yeah, that is, that is one second is while the backfilling do we really want to advertise earliest available slot in a really very granular way? Or are people just doing it? Something like, let's say, when backfilling is in progress. I just keep it at local head slot, and then, when backfilling is done, I again

push it back to the slots behind. I I was really curious about this. The current crashes. Some of the current crashes in members is because we are trying to sandal, the earliest available slot in a very granular way. Like every slot we complete backfilling, we want to update the earliest available slot that way. It's just more helpful for the network. I know there's no incentive. But yeah, so I was really curious about that as well.

Mario Vega: Yes, pawan.

Pawan Dhananjay: Yeah for us, like, we fetch the earliest available slot from our database. So every time backfill updates the earliest available slot in a backfill. Basically what we advertise also updates simultaneously. So at least for us, like we, we basically had to change a dB a little bit to add that particular earliest available slot point. So right now, it isn't a problem for us to keep updating it after every backfill batch is processed.

Mario Vega: Thank you. We have less than a minute left. Ainish. Do you want the response or.

Agnish Ghosh: Yeah, just a question for pawan. What is an average backfill bat size? I'm just curious.

Pawan Dhananjay: It's 1 epoch for us.

Agnish Ghosh: Okay. Okay.

Mario Vega: All right. We are out of time. I think we're going to close it now and thanks everyone for joining, please, on Acdc. This Thursday, we need updates on perfect beer desk syncing. So please, teams focus on that. And thank you. Everyone for joining. And yeah, have a good week.

Pawan Dhananjay: Call queue.

Mario Vega: Thank you. Everyone.

Agnish Ghosh: Thank you. Bye, bye.

Justin Traglia: Bye, bye, everyone.

Fredrik Svantes: Well.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.