PeerDAS Breakout #21

Transcript

yeah okay um let's start off with Cent updates I guess um I can see Manu on the call do you want to go yeah sure uh good morning um so as you may know prism still has an issue about um the number of files uh on uh its database so with a current database designed for col prism for supern needs like more than 10 million different files which explode literally the E not the in not count I not count on disk and you have some prism super n which are down right now on the devet uh so mainly my task of the last week was

to um work and to implement uh a new database design to save all data columns for a block is in only one f and not in 125 128 FES forut for example I still hope to have a PO request for today um so it was mainly my main task and also Frances from Bas uh worked on engine get blow V2 um yes it is still in progress uh that's all for Me Maybe um who's the next C

team I see TI uh yeah te uh we are working on validat custody we are finishing the feature and starting to testing it interrupt uh te with Teo but not finished yet that's all okay um Agnes do you want to go for nus we finishing off on column syncing without supern as well and we have made some progress in validator custody but uh we are also implementing get V2 get V2 should be

finished off him like by next two days and I'll see if we can it up and I think validated cust would take a bit more time because I'm like I'm kind of working on the columns back filling thing or yeah some more time May by of this week yeah okay um Matthew do you want to go for Lota uh not a ton of updates I was on vacation this week uh gajinder did uh address a bunch of the comments on on

the pr body uh for making the uh validator custody static and uh still waiting to see where that lands sounds good um that plan for Lighthouse maybe what do you mean maybe still here yeah not sure if you were doing the updates or not for Li us yeah we are we are currently stuck with sync issues that's our main problem as of now and we suspect that some clients are faulty and our scoring logic is not

there yet we are not aggressive enough penalizing bad behavior so we get stuck with bugy clients and we cannot progress syn so we have a lot of uh improvements and refactors that we know we have to do that it's quite a bit of stuff uh and we are implementing them now but it's going to take a while but hopefully that should sort uh these issues in the case that the network is at veral or clients have bugs so for example as of now we try to sing from a pier that doesn't serve as columns and then we try again from the same pier and then we get stuck there where when we

should just penalize the perer away besides that are these buggy behaviors maybe um written down somewhere for example which pier was advertising columns that they don't have and question is we should also be fixing set P right yeah exactly I think Jimmy reported I think it was prism yeah which is great like thank you thank you for giving us testing cases you're welcome yeah I just asked to Jimmy to give me the the parameters of the syn um colum by one request uh which was 4y for me to

to produce a yeah I think some of the like inability to serve column range requests was because of the like two and fr switch I don't know like if it's Pro but at least do switch from u a super node to full node or vice versa because um if we do that um we are I think not currently backfilling so I asked this question to Jimmy a couple of days ago and he said that if we don't resync the BN shouldn't start up um in my opinion I think we shouldn't do that

we should have an external like not external of course but a column backfilling Loop group that can use the block database to like Advance slots and build the excess columns in that scenario would you advertise the exess columns before you finish the back filling or would you update your enr ler I would I would update my enr once I have backfilled up to the fin at least the finalized head at least okay um the other kind think is the idea that each team chooses what they approach what the

approaches uh okay I just want to say something yeah in my opinion I think uh inhibiting from a BN startup and asking the user to resync is not a good idea because it just adds I mean I mean the the node is just not ready and we need to actually like kind of figure out the odds of um getting a range request like that moreover if we advertise our enr later we won't be down scored so I don't think there's point in inhibiting the BN to start up I think the BN should start up as soon as it

normally starts up and then it should slowly backfill the problems and then advertise the up yeah I think as long as we follow that approach it sounds same um yeah we're just trying to make the parallel to the E world where that exists right now for example if you switch a full note with the archive flag it doesn't backfill anything it would essentially require you to resync um but yeah I get that the cl world doesn't for example have to backfill to Genesis or something no I think it should it should back fill from minimum like 4 Z last

496 yeah exactly till the amount that is in school basically yeah yeah in in our case we want to have strong databasing variance where we are very confident about the state of the DB with respect to what columns are there and what columns are not there and at the same time unless it's strictly necessary we don't want to implement a dedicated back filling routine for columns that's why if the user tries to switch back and forth from supernode and non supernode we easily check what's the set of columns that I'm supposed to have in the DB and what's the set of columns that you want to have me now and if they don't match then we will not allow the user to

start it feels like this is something that we can let the client decide as long as we align on the fact that each node only advertises The Columns it has rather than not the columns that it will have as long as we have that shared understanding I feel like it's fine to just leave it to each client to decide what approach they want yeah I think that's fine but we we have to be aware that all clients whenever they checkping they will always break that rule uh because they won't yet have the The Columns right they would have to first backfill yeah when when you checkpoint seeing your theb is empty and there is no way in the status message for you to

express that state and what is being done right now say I checkpoint sync a new note I'm just advertising columns I don't have nothing like this is the same issue if you checkpoint sync and you try to back fill blocks like today in in the neb and you you connect to another checkpoint S node that's a little bit ahead of you you will maybe try to get blocks from it that it claims to have and doesn't have and then you will disconnect from it which is fine like if you really want to backfill you want to you want the blocks so you want to have at least some beers

that have blocks for you to complete your task yeah AES yeah exactly I am uh exactly what Manu wrote uh in the comment section we can have an indicator U that should solve the problem yeah but what so this pointer will show the the progress in terms of blocks in terms of columns like what of in terms of side cars so in terms of columns or LS and what about blocks in my opinion it should show the progress in term of everything so as long as there is as long as all blocks

and blobs and columns are not correctly backfilled it should indicate that backf is still in progress but I mean I can already tell you how I'm going to be malicious with this I will basically advertise that I'm always backfill in progress and then you never down SC me and I continue getting data from you forever yeah exactly would down score you once we receive bad stuff right like once you Bluff uh we are happy like we yeah but if I'm back in progress the understanding is you never receive stuff from me so you're not requesting stuff from me okay so you basically created a situation A malicious yeah I can be a malicious not on the network forever you

will never down you can be a manici not on the network but validator custody already you can modify the the client and in a way that you will never custody more than four uh subnets it's already possible sure yeah yeah with validator custody you can be that I agree with but here I could spin up a th000 nodes all thousand nodes say I'm backfill in progress so I continue leeching data from every other good pure but I never provide anything back to the network I mean you could add more than just that flag of uh backfill in progress where you'd have state like how far you've backfilled and if it if it

stay static uh you know that person is lying yeah I guess but then the question is is this complexity worth it no so consider this case you you advertise a specific value with back filling progress and then you query blocks and then you get no blocks this is equivalent to just quering the blocks and getting no blocks so you you still have to query the data and check that the data is there or not so you can just issue queries and if you don't get what you expect because you like this is historical data you can easily detail you can easily know what you're supposed get so in my in my view doesn't add much

information if you can just ask for the data you need and check if if you get it or not yeah it feels like PS scoring already accounts for this so there's no real need to add more logic right yeah like in lith house what we want to do is not uh nearly Willy downcore anyone that we try to backfill from but at least allow so unless we have X amount of peers that we can backfill from then we are happy and not force everyone to be on the same backfilling progress as we are that makes

sense yeah um unless guess yeah sorry I guess a tldr of the discussion would basically be don't advertise anything you don't have right well my my point I was bringing up is that with the current spec you will always break that rule at least for some time and I think that's fine but we have to be aware of that yeah yeah we probably have to do some Edge case stuff um where a whole portion of the network is sinking after nonf finality or whatever but that feels like the problem we have on meet anyway um we can have a bit more aggressive

Downs scoring if uh malicious not relies about back fill in progress is there maybe some way we can first find out if what we have is broken like if any action is required at all no I mean let's say um a node is lying about backfill in progress and uh we still get bad blocks or we don't get in blocks at all we have a separate down

scoring for um basically misinforming we can do that but do we actually need this backfilling progress in DNR or or do we not actually have to do anything because current peer scoring algorithms should be able to handle it so for for columns I don't think like in itself because um updating the enr advertising the higher enr after back filling is fine but yeah as like

lion mentioned the checkpoint sync that situation it might be useful but okay yeah yeah um I think to sort of EO what Franchesco also wrote um it this is kind of an invariant or like this is how the network behaves today anyway it doesn't feel like we're changing this behavior in pias even today when you checkpoints think you're advertising that you have blobs that you don't really have um so it it feels like a lateral shift that we don't need to fix unless we find an

issue we need to fix but everybody has the blobs and the blocks not everybody has the columns so you have a much smaller subset appears that you could sync from in order to get the columns got it so I guess the argument there is that we're basically making it harder to sync so we need to have a more robust way to sync or a more robust indicator of who is good to sync from okay um do we think this is something we can figure out if it's a problem to begin with I'm I'm not sure what sort of test

can help us figure this out but it feels like something we should first try and figure out before we jump into Solutions yeah I would say let's let's try the approach of today like just just try to do a request and if that's not enough then we can study this we've actually been adding metrics about this exact topic uh try to understand like how successful there our our requests are yeah but this doesn't this doesn't add peers with more columns it just it just gives you a shortcut to obtain an information that you can obtain anyway today with a current spec I think there is a relevance of the

flag with regard to debugging because you know when we are debugging and we are trying to syn and we basically are always confused that okay this particular peer advertised that it had this custody but we are not getting it and it basically makes debugging whole lot difficult so this flag can be used in that uh Direction but obviously no assumption should be made on top of this particular flag so if you want to disconnect a peer that can't serve you your custody uh because you want to add other peers who could serve you then you should do that on at level of client but for debugging purposes this definitely makes sense because uh when we have this so many cases that you know uh we we

don't get the matching columns and then we are confused as to why this particular no is not providing us that data sorry black yeah let's have this as the last exchange and then maybe we move on to the next yeah yeah let's move on for me it's good so for now we do nothing basically we just going investigate Poss exactly and I think one of the uh last points I also want to mention is that I think we might over index on a really small Network we literally have p d networks with 20 nodes I feel like this problem goes away at larger numbers um yeah uh adish do you want I

mean I would uh I would basically not see it as a problem I would basically want that okay you know this is an indicator which can help us in debugging the network so there is no point just adding it in and if the clients can indicate whether they have the or not I think that is a great information to be shared with the peers um in general we need to increase the probability of the intersecting columns right so either we uh either we have like randomly round robing uh query peers or we basically cut down the event space of by like that way we also

increase the probability of finding intersecting column custody so I think cutting down the event space or like you can basically uh like if you have backfill and progress flag it basically helps you to cut down the event space faster when you're column syncing and it would basically make the column syncing more robust yeah and of course debugging would be easier too okay um um yeah let's maybe get a bit more data on this and then we can figure out how to proceed it could be that validator custody plus bigger networks also give us better Insight on

how well this works but yeah acknowledge that we might need to figure out how to more robustly figure out who has what columns yeah sorry it's last point you can just collect metrics from every client yeah on what's the C and backfill progress yeah that should be are clients offering this at a metric end point I would say probably we just need to find what metric is who I think we standard as the metrix so it should actually be the same one but yeah let's uh time box this I

think we might have already spent a bit too much on the topic um so the next is updates on devet 5 um so we have been spending the last week making the network a bit more malicious there's roughly three nodes on the network that are either withholding columns or delaying block publishing or delaying column publishing uh by roughly 2 seconds and I think we uh have a bug in prison do you want to expand on that Manu are you talking about the bug about um the database why was that just the database or was there any other prison B um so at the moment I'm aware of at least two

prism bugs it's the one of the database so uh that's why uh prism super no are now down because there is not enough IOD available I'm working on it to define a new database type type and the other one is a bug um about prism not uh serving correctly data columns by rench that Jimmy uh and uh grounding uh raised that's all maybe there is not maybe maybe the are B but I'm not aware them okay that's uh good to know so basically the withholding stuff has not

triggered any other cases that we worried up about yeah okay that's good so we'd probably wait for the fixes to come in um both on Lighthouse as well as prism and then add more withholding notes to the network nice um also we have roughly 60% full nodes on the network which um they also seem to be stable of L of late so all good signs um anything else for Petra devet 5 that we want to discuss sorry uh p f okay um then the next topic would be

debet 6 so what's uh happening in the world of validator custody I think most clients are still working on implementing it could we get a status update and if there's a blocker or is it just requiring more time for the code to be written um so for prism uh it's implemented um uh with the new version of prism hopefully I have today with corrected database um this verion will include validator custody as well uh so that means that current full nodes on the P devet will probably become by default

super no U because I think they have too many validators um 64 if I'm not wrong um but right now prism so uh implements validator custody but does not Implement Back Field does not Implement backfield at all that mean that when you start for when you start for checkpoints there is no back field for data columns and when you increase the number of validators there is no back fill um as well and it's my next task when I'm done with the database issue okay that sounds good um is the

idea that we deploy the prism changes already on dnet 5 or are you putting it in a branch that we only Deploy on D 6 no we we can we can deploy it onet 5 no issue but the only things to know is if the rator has um more than 64 is sorry is there is more 64 validators attached to the Bon so this become no will become a super n yeah I think that's the Assumption we have with all clients basically that yeah it's it's a principle of Val valid exactly yeah I think we take this into account when we're designing tet 6 anyway so we should be okay cool um any other CS updates on updates or blockers

on validator custody yeah Agnes so for us uh I think in itself implementing validator custody is not too complicated but the bigger complexity is the backfill and the the complexity that we had till date in Nimbus is uh we had one single block processor which would uh basically be directly kind of depend dependent on the side cars as well um I think I have discussed this with Manu several times like since November but the thing is um this entire

architecture is being changed and side cards are being looked upon as a like independent it's basically being processed independently uh why because of uh mostly because of this reason because of this um backfill feature that we need in terms of validator custody so because of that we are rewriting sync we are actually also in like we are actually also introducing a column back filler routine um which would handle back chelling on two cases one is the as I said the switching from Super node to full node or vice versa and also the validator

custody situation so in terms of blocker it's not exactly blocker but these things the syncing and the back filling is taking us more time than implementing validator custody itself because once these things are done I don't think validator custody like finishing it off and taking it to a testable stage would be that time consuming so yeah that's that's pretty much from n okay some good um any other points on validator custody yeah

kaj yes since no clients are giving update I can talk about the pr and uh yes so the uh response on the pr has been that okay for the ux purposes uh clients would not like to go for a stat custody but I think it should be left to the client whether they want to go for a static or they want to go for a u Dynamic custody as long as they Implement uh uh as implemented in the sense that they only support the validators uh which basically can be supported by that particular static or dynamic custody and uh with regard to that uh so I I I'm still want to push for change in Spec how it's written not

to have validat indices mentioned in the spec but but only for example have a supported stake mentioned in the spec where we are trying to figure out what is a custody that a particular validator needs to be supported and then uh there could be uh so there there is another helper to figure out how much supported uh stake there is from the attached valid data indices U and this can basically help uh client Implement in both the way while staying as close as to the spec uh as possible will you post a link to the pr that

you're discussing I think it's in the agenda once again Yeah so basically mention in his yeah go ahead yes sorry go ahead I just wanted to ask if any client teams had a chance to review this to begin

with yeah so I was saying that lion reviewed it and basically has sort of I mean it's a valid uh uh point that line has raised regarding the ux but I think we can leave the ux to the clients uh what they want to support and uh whether they support that particular ux is worth the complexity uh maybe all clients will end up supporting Dynamic custody as well but if we have spec written in a different way then it would be possible uh for uh Dynamic and static custody both to be supported all right um have has any do

any other clients have thoughts on this by the way yeah I R it and I I agree with Lion about the the uix side um as a former node operator I agree with this uh comment but is there some reason we can't do automatic with the flag being an override um no I think no no I'm I'm not asking people to uh support the flag I'm saying that if we modify the spec uh uh basically have supported stake inside

the uh used inside the spec rather than Val data indices uh where we figure out how what is a valid data custody and I think there is another PR in which is available data is also using valid data indes so instead of using valid data indes if we introduce supported stake as the point as a reference point and uh then suppored St can be dynamically or statically uh uh provided depending upon the client implementations so uh if we do this change basically that's what I'm saying is that we should we would be able to support both approaches in terms of spec and clients who want to do static can also stay close to the

spec if you can do static then you can also lie right you can always lie for this as well if you uh change uh uh change you do the dev you you Fork out the client you can always do it because there is no real validation in in terms of custody but what what static also proposes is that on the on the validat apis it is enforced so uh the honest clients will enforce it in even for static uh custody as well where the duties of the validators who are going beyond the support stake are rejected I don't really get the point of

adding this to be honest because it feels like the beon node will still need to verify somehow how many validator clients are connected yes basically the point is that there is separation of concerns while you are syncing the beacon node and while you're supporting uh the validator apis it is a a bit cleaner in my opinion there is less pollution of attached validators into Beacon out operation and the complexity is definitely a lower for this particular implementation and we can leave it to clients what kind of ux they want to support and what kind of complexity they would want to undertake for that ux I think this should be up to decline

things and up to implementation instead of yes so for that I'm asking that can we modify the spec not to use valid data indices in the spec and use supported stake in it because supported stake can be dynamically or statically uh calculated by the clients depending upon what their approaches uh but having valid data indices in the spec basically takes uh the static uh custody clients away from the spec in terms of uh implementation uh and basically in terms of Maintenance and if somebody is trying to read so code readability as well yeah I think the change itself

makes sense because even after the change to the spec client can decide to do what they do um based on whatever option they want um so if this is good I can basically refactor the pr to do this thing and remove static as such but just uh provide the supported strike and uh is there any strong opposition to this change by the way or I mean it feels like the C the change can be made and clients can continue to do what they're doing today anyway so either they take the ux G or they

don't yeah I will need to re RW not not a not a big fun okay um then maybe we can block based on another round round of a reviews and then in the meantime gajender can refine the pr and then we take it from there perfect thank you um cool the next topic I wanted to bring up was um cellproof computation um I know that get as well as nethermind have been engaging with the topic um does anyone from either team teams slcl teams who've been

engaging on the topic want to give an update on how stuff is going uh I I can give an update so uh hi this is fr from base uh I have been working on Implement implementing the self Pro computation to transaction Center on prism and I have a PR kind of like up and uh it's just need some kind of like touch up and fix it adding some tests um this on the the E side um the I think Mario had a PR regard like with like get BL V2 but didn't have get payload V5 so I'm trying to basically finish that part as well so that we can have a full endtoend test soon hopefully

this week and once that is kind of like tested out ex I will make follow up PRS to the consensus spec and the Builder specs uh accordingly to update the relevant kind of like apis and stuff so overall the change is not like super big in scope the big I guess the big pain Point currently in implementation is just that there's like P like get Blob and proof V2 blob bundle V2 and like uh what Beacon body contents V2 something like that there a lot of like a new version of like protos across different specs uh Builder uh beacon beacon to e and like e

specs so that part makes implementation a little bit hectic but overall manageable I think that's the own thing and we are hoping to W with the kind of like implemented prism and gas pair we can see like the benefit of it and really flashh out the entire like uh path of the spec and then help everybody else to start implementing on it awesome thank you um I also saw there were a bunch bu of discussions about the API to be used in general um do you want to expand if there was any changes to be made this week just in case other

client teams are working on implementations and need an update uh I'm sorry like API being used in what sorry not API being used but I see there a new ckcg release and then you need to use 2.1 um is this yeah yeah yeah that is not problem like it's just that they are the new compute sales function is going to be released in that version so like if you are using that like vkg Library you just need to update to it I got it okay so it's nothing major yeah okay um are other client teams also looking into implementations if so where

are you guys yes so you never mind uh uh we are working on it it's uh not ready yet but yeah we are actively working on it so so should be done as soon okay any other CLS looking into implementation and I also see Romans on the call has re got some uh updates as well uh yes so we we started adding the related buus changes one question I did want to Brint up is are there any test factors

available for for the new network message type uh I'm not sure um Mario do you happen to know if there's any test vectors no we don't have anything yet um we're still finishing up at this moment and we have to take a look into the into the into the spec changes and also we need eels uh implementation that um I'm not sure if anyone else is working on that uh but yeah to fill the test vectors we're going to need need implementation um I'm going to take a look if to see what the status of that is uh and we'll get back you Francis this this something you

might also be able to help out with um would H would be happy to but I'm I guess I I don't know what a like test Factory is I mean we we sorry yeah I mean yeah thanks uh I think the this vectors we can handle um uh once we have the implementation uh but the eels uh implementation is necessary for us to even start filling uh the tests yeah that's that's the first first part got it uh hopefully we can finish the prism implementation like um early this week if not like at at the end of this week so that we can have something for you

guys yeah also I I think I mean if you have also a get implementation already I think we can it should be minimal changes that we need in the evm tool from G um to to be able to F test also so that that could work too uh as a fallback cool um I see Francis also had a message about uh renaming cell proofs to yeah on returning sell proofs in the original ke sorry yeah do you want to bring that up no um I I I think like I basically took a deeper look into that I Curr my feeling that there's like already too many things that needs to be

changed it might as well be kind of like be consistent and like upload up update the version of the apis anyway instead of like some of them are updated and some of them are keeping the same name it can be a little bit confusing so yeah maybe scratch that okay perfect um good cool so ideally this week or early next week we should have some um implementations ready and we should have some initial numbers on how if Cell proof computations actually helping um speed stuff up right and in the meantime other clients would continue looking at implementations along with Val data space okay that sounds good um

and what are the other there's also one other open PR I think that we wanted to discuss on the spec I'm just posting it here this is from Franchesco do you want to expand on appear this appear to I dip uh yeah I mean it's just basically kind of defining thep what the what the new rer for it is uh and like motivation for for that uh I added the version I think I call it rapper version or something like Felix wanted um yeah other than that there was basically just a question from Felix

around um what exactly how exactly should we Define like P2P acceptance of uh the old and the new version around the fork um like basically saying that usually P2P doesn't like doesn't quite like uh doesn't have a concept of fors necessarily so like it would not be like a natural thing to just like immediately deprecate something at at the or not deprecate because it would still be used in other ways but specifically for like uh Gossip acceptance so I'm not entirely sure like what exactly to write there uh about this and uh yeah then the other question is like uh should something

about the NG API be be uh added there um I I think yeah definitely like link into the to the engine pii uh PR like should be done somewhere but yeah basically kind of it would be nice to to to finalize this at some point so just have to answer like these kind of few things of like basically what needs to be added what needs to be uh what else needs to be like more precisely specified um is this something we would need Felix for or is that information something we can get from this room right now I mean that thing neily Felix but

like just people that are generally familiar with like um yell kind of mle practices and how things are done there which is not really me um yeah I mean Felix has definitely been like the most opinionated person on on this so I think it would definitely be helpful for to get um yeah to get an opinion from him um but other than that I don't know I mean any anyone I guess you know if you have a uh want to go comment on uh below Felix messages and or or just like propose um like a like a final change to the uh in the in the pr then like feel

free to do so yeah sounds good sounds like we would be able to get some information from Felix SL everyone else async and hopefully we much in the pr by the next week okay um besides this I know the Sunnyside lab guys have been doing some experiments uh do you want to expand on experiments yeah hello guys I'm in from sunight Labs so during the past like one two weeks we have been running different tests on the pias and Petra net in order

to uh test and see what effects of increasing the blob counts on the current uh devet implementations look like so uh I have sent the uh reports on the test in the uh perod Discord so I would appreciate it if you guys could kind of take a look into it and uh some feedbacks that would be really helpful for us to continue is to uh one is uh like what sort of experiments or other analysis that uh core developers would uh prefer or would be helpful for you guys so like what tests you do you want to see like for like the cell

proofs we can also run tests with different implementation versions and like see how the difference is in uh the dev Nets and another thing would yeah and then another that already um just to uh help with setting the stage I think it helps if um we're able to com so with networking a lot of stuff changes between mainnet and Dev Nets purely because you will never have such optimal puring on mainnet but on dev Nets they definitely do uh so one of the approaches that tends to help is if you have a baseline so you can have Petra with regular 69 then you can have pure

do with 69 and you can showcase just how uh pias as a change has affected the Baseline and then you can look at increase in blobs so essentially just look at the differences because the Baseline will be different from Main net if that makes sense right so yeah yeah definitely so we did the Baseline test on Petra devet last week so we were increasing the block block count per block from like one to 72 uh in the Petra devet and we know noticed that in like the most optimal uh networking conditions the blops were being filled in uh without much problem so we were

also doing the similar test with the current Pier Das implementation in the P 7 5 the client version in the appear 75 and we noticed that uh we started seeing like the networking breaking apart at around envelops but the thing is that uh there are much more configuration related to kedas like the custody requirements or like the uh ratio of full nodes or super nodes or how many like uh different node types there are in the network so I think it'll be also help for us helpful for us to know like which type like how what is the best optimal setup for us to

make in the damet to optimally test per yes yeah so I think you we can definitely discuss this in a thread on P testing but I think you would want to see essentially two or three different types of setups so one would be as similar to mainnet as possible and I think the pro blab guys have a really nice dashboard where they sample different structures on mainnet essentially like 40% super nod 60% full nodes as well as a bandwidth distribution for that uh um the second one would be flat network with really low uh limits so this is essentially worst case what happens if every massive

node on the network goes away so this is absolute worst case scenario and then I think you would have some sort of um optimal scenario where it's higher bandwidth limitation but this is what we recommend on mainnet so uh there's a post by Kev about B uh bandwidth requirements and essentially a flat net work with sban with requirements would be would make sense so like three different scenarios yeah definitely okay we can do that I'll follow up in the Discord Channel afterwards as well perfect and I think ancar also has a point on the El mol side um yeah essentially of potentially limiting meol

bandwidth right I don't know if ancar has a mic but um yeah yeah sorry um I mean it was just a very small thing but like in discussions with SK the other day they came up this idea of basically optionally offering like a flag of sorts to limit um mol um load um throttle it in some way that the most direct way would be like a direct bandwidth kind of upper upper limit it could also be if that's too complicated could just be like a maximum number of blob transactions per second or something um just basically because we we expect obviously like that the the the background kind of mol load uh on clients will will scale with with the

number of blob transactions or like blobs overall in in the network and um for some nodes um it's it's it's not a concern of course like in terms of burst like when the blog is published but just there background kind of bandwidth strain um that could could become a concern for for we weaker bandwidth nodes and so if there would be like an easy flag for those notes to basically set and that that that that might be very useful yeah I'm not sure we have the critical mass of um El devs to have that discussion on this call but it does make sense to try and have it on ecd perhaps can do that as well

yeah maybe a m will break up um yes thank you very much for uh for the experiments I guess we would engage a bit more on uh Discord to figure out what the scenario should be and it would be great if client teams can also um start posting what they would want to see um maybe the next steps for that topic might be presenting on ACD um I'm not sure if to this week might make sense but potentially plan to do it sooner sooner rather than later um purely because I think a lot of people who might have uh contrary view/ opinions on what data they want to see might not be the ones who are implementing podas might be the extended

cev Community um so it might make sense not to hide the data I haven't follow up on the ACD schedule but yeah uh we can definitely work on something in the coming weeks as soon as possible perfect thank you very much um is there any other topic we have to discuss today yeah do you want to go um flcl on engine RPC perhaps uh no just uh was talking about a Mar about

that uh we could prepare some methods to to work uh according to new proof uh in coding and uh probably would be tested in hi uh got it yeah yeah we need we need we need a transition Tool uh support U that I mean yeah it it seem rout transaction could be useful but it's like uh way later in in the testing cycle in my opinion um yeah we need we need a transition to First yeah um ancar you had your hands

hand raised yeah but not not on SPID realized that those was this one thck so just one last thing I wanted to say yeah I think the topic of hi is um over I think we would just need the transition tool first so there's nothing else to really discuss okay yeah then I just want to very briefly we already talked about it quite extensively in the chat but just for anyone who didn't PID much attention there and there is this topic of this this this batch publishing proposal by saaba um and several people um which we are I think it's not quite sure yet but we we have the feeling that it might actually

be very critical on the Block building side to basically to not end up in a state where with pasas any normally kind of normal bandwidth node at home will only be able to to to build like very low blob count um blocks I think you basically you have to like leave 90 95% of The Blob section empty if you if you build locally whereas with with with with badge publishing hopefully that that means you even normal normal kind of bandwidth notes can still build reasonably sized um blob sections um and so this might end up making a difference uh in terms of what two numbers we can we can chip at depending on what you know qus in the end feel like what what is reasonable for for for for local

building to to be a requirement there um and so just for C clients that haven't looked at that at all yet um just yeah something where like ideally this this this this could be should be considered yeah Francis go ahead um yeah um quick thoughts on that um so for like the blog publishing side uh there's like distributed blog publishing that could potentially help that depends on like the E like get Block V2 and points so basically the idea is you publish the blog first and you don't quite need to publish all the columns out like all the other clients maybe supern on the network when they have all the blobs all

the transaction blobs all the blobs transactions they can like help you publish all the uh blobs out or the column columns out and there are I don't I don't quite remember this the first thing but there are some other like mechanism that could help alleviate the problem as well so I think yeah the the G is that we should probably like add as many like Network Implement Network optimizations as possible uh but um like in my opinion it is not like strictly necessary to have this like gossip stop um like implement mation there but yeah we will definitely try to like add that if it's possible um has anyone thought of how this PR sorry this either the search

post goes into a PR or is the idea that we prototype it in X client and then first check if it's actually beneficial like what are the next steps okay yeah I mean I don't know that much about this I just know that both um Lighthouse and prism have looked into this a little bit and this ISP from I think nation and age um I think this was you know very much like early stage kind of like looking into this but yeah I think this it's what exists right now yeah to add go ahead sorry to add a little bit into that um prism seems like from the prism has a implementation

Lighthouse is working on one and base on base side we will contribute one engineer to this to try to like we can be the status tracker for for this call and to really like try to like drive this forward but yeah I think it's just starting okay perfect so this could still be a topic that is auxiliary to pias um either it makes it into the critical part or not but it's still too early to say yeah and uh I think yeah of course it would be to waste effort on this if it's not on the critical path so I think we also like on the on theory side we maybe look a bit more into like how much do we think this this this might have to be load bearing and so yeah we can follow up on this but uh

just want to like it for now perfect thank you then We'll add it to the tracking list as Francis said and then we can get status updates and decide uh probably down the line if it's in the critical P awesome um we're already over time um so thank you all for joining and uh keep in touch on P testing thank you byebye bye bye