PeerDAS Breakout #8

Transcript

up e e e

Ando e hello everyone

hello hello hello yeah I think um shall we can make it to the call today um so um I'll help with

um moderating the call um maybe we get started in one or two minutes when everyone's here seems like we've got um the like enough people maybe most of the usual crew um so maybe we can start

and just save a bit of everyone's time um and we can probably start with um client updates um I can go with um Lighthouse um for lighthouse in the last two weeks um we have mostly just been working on interop testing and de issues with other clients um I think from initial testing it looks like most client um I mean the ones that I tested with at least um seems to be okay of interrupting each other um and um the issue is whenever we have issues with resources like whenever

we have six blob C cars it's like it's more likely that we we missed the miss the block and um and then and then we fail to recover um that seems to happen a few times for a few clients um I think we probably want to just make it a bit more robust and that um but it's most likely a resource issue so we're mostly doing local laptop testing and maybe we're not assigning enough CPU um um so I'm not sure if it's a real issue but it will be ideal if we can recover from those scenarios and I think on top of that with the decentralized block building I think that could Al also solve the issue

um if a no is not able to compute Pro quickly um super no essentially it would help with that sry yeah um I guess the question is um when I was trying it on kubernetes I was still running into the same issues and there was no resource constraint there so there's probably something else in the mix right yeah probably um do you um have you looked at um the the proof computation time I think it would be interesting to look at the metrics whenever that starts to happen ah okay so it's sort of like a spike that's unable to be done within a certain amount of time okay okay yeah probably though find trying to compute

proof for six blobs and if that happened too late then the no do you have a metric I could maybe use or a dashboard or something um I think kosis itself comes with a a p Das dashboard that's only for Lighthouse um that's built by Kya um I've been using that myself um I think we'll probably need to add a few more things there but I think that that's should be good enough for now uh then I'll rerun it and have a look there cool yeah thanks um and um we've also been working on making the max block block configur R um got open PR for that it's under review um that should be ready soon um it's been tested so that that that's also a good good

thing um and we've also been working on capturing metrics for um distributed block building and we we're looking to have a block blog post out soon so maybe we can share some of the metrics that we found find there um that's all from Lighthouse maybe prison want to go first um go next yes sure soing the last two weeks uh yes we had a issue with um big CPO user um in a small devet basically the NS needed to needed a lot of CPU to look at other n in the network and if there is not enough n network the cpu's

consumption keeps running and running which does not help for sure for block for construction if needed uh so we we we we fixed this issue um also we went from uh mix between subnet sampling and peer samplings to subnet sampling only as defined by the latest specification so remove the the Ping and we increase number of um subet subscription um yes also we had an some issues um when uh reorg appeared sometimes uh prism node were not able to

to rej the main Fork um the issue was it was happening only after a few hours of test so what I did I I added a new hidden flag in in prism in order to um ignore data colums sometimes from time to times coming from the network so for example every I don't know every 15 slots uh a given node will ignore all the data colons and this no May um create um create a reor a few slot laters and so like like that we are able to test if everything is going all right in case ofor and with that I have a p request where uh where tonight I was

able to run um devet only with prism supern and and full n uh during a little bit more than 10 hours I mean I mean I stopped after so um yes it's basically that and now I want to test again for a very long time so let's say 10 or 12 hours with prism and other clients um with stress tests so like um with some colums witholding when we propose a block and also when with some colons ignore when pris node receive some colons and it ignore them on

purpose um that's pretty it for the last two weeks um maybe te yeah uh so uh te we have refactored P DB access layer and it does not stink like a prototype now also we added paration to two tasks uh proof calculation during uh block construction during proposal when we uh do Colum when we are making col cars and the boot mostly depends on number of threads but it's like five times on my laptop and also we added

paration to kg verification in in availability check uh we did some changes to theb back login so it's less noisy now and also improve as a prod login in few things but the main thing is is that we identified like four reasons of why Tech of fails to join canonical chain on any splits due to block delays or availability check fails and so we found uh for issues and we already fixed one of uh this issues and free Ste to uh to go and we don't want

to join any devet until these issues are resolved and I I think we could finish it during this week uh that's all nice thanks thank dri um see Solas maybe granting go next yes hello so so the guy from fellowship program he was working uh on fixing some uh uh some issues and it seems that the one issue that we still have is for some reason we get down scored by Lighthouse and eventually get disconnected from

Lighthouse however I think it's it's uh slowly getting this the state where it seems It's it makes sense to to start testing so I think I'll I'll give a party a link to uh to the to that branch and uh I think after part does some initial testing we can see maybe we will be quite ready to join uh the network even if there is some uh some issues with u with Lighthouse down scoring so yeah that's the update so far great thanks

alas um do we have ners here on no ST it doesn't look like it um yeah maybe we can go to debet update um I think Perry's been helping us with testing um Perry do you want to give an update yeah um yeah most of the work has been spent on um local testing I think the the only real find last week has been the load star bug with uh with the end bug but besides that the one we're all talking

about with um potentially proof computation Etc is still an open topic uh and just surfacing some things from the chat um I think there's two things left to try um one would be yeah just trying what we did the last time but paying attention to the metrics that you mentioned second would be trying a network with zero blobs just to see if it's still breaking at some point and then goomi allows um f grain blob spamming so we can ensure that the network only has one slot one blob per slot or two blob per slot and so on so I guess we can increase that number to see if um the behavior at one blob per slot versus six blob per slot is actually

different um and then I guess that'll also give us a bit more information to try and debug what's going on here um and also just another um short update Barabas is back this week so I will be switching back to full-time Petra and he will be doing beas as of this week again cool thanks thanks f um yeah I know you've been doing some cuties testing um does does the um the time that it takes to crash juice with that or it's the same is it the same it seems

to be roughly the same so it tends to happen between Epoch 12 to 17 there's no real exact number right thanks yeah for me when I was testing I think it last until 7 or 800 slots before it started working um yeah I really don't know if it's a resource issue but um I feel like we should be able to recover from those issues if it is a resource issue but um yeah we should look into it a bit deeper um yeah and one more sort of ask is that um it seems like the metrics that Kya

has built are really helpful for trying to debug this but as far as I know it's not um present in all clients yet uh there should be a PR open to the like there's a metric spec um maybe client teams can spend some time implementing that as well so that we have a bit more visibility into what's going on nice um do you have to link to the the um the spec PR for the metric yeah just looking for it but I think Kya is also on the call in case she has it a bit more ready than I do yeah yeah I can share the link oh um I see bers is back if you got

something to share or too soon I got married last week really congratulations con yeah I I wasn't very up to date with the all the peress changes uh but it sounds like everyone was still debugging so always good you didn't miss anything I will also add a gossip sub and U rras metrics in into this list maybe today

nice um so um do you plan to open a PR for all of these changes maybe it's uh going to get a bit more visibility that way because right now I feel like this is just a fork of the repo I think if if you would have an open PR with a with the draft or even whatever then people can refer to that yeah sure then I will I will end with the other metrics and PR so if if I can comment on the metrics we had the chat with Kia in person yesterday and I mentioned that we should distinguish between metrics that we suggest for

clients to have because they are interesting and metrics that we need to standardize with the same um meaning so the ones that you talk about bar would is would it be important for the metrics to be identical or we just need them to exist in order to Deb back each client individually uh I think I have a lose preference for trying to make it such that all like there's one metric name and it means one thing across the clients because we have the situation today that everyone has different client uh different metric names for everything and that makes it extremely hard to get a global view of how the network is

looking without looking at six different dashboards um or the approach we've taken is building an exporter at which point we don't need the client to implement the metric anyway um so I have a loose preference for trying to standardize it but I think what you're trying to imply is that there's like for example the verification time for a column that's something that we don't need running all the time on Main net right so it's just increase in cardinality for no reason um in that case maybe it makes sense to only do it when there's debug logging enabled or a debug metric flag or something like that such that we can expand the cardinality set for users that want it and by default we're not expanding it right so I'm yeah sorry I'm currently

trying to implement uh metrics with the same names uh like house and Teo and uh we will have an example so I can share it later and we will see if it really works good right so what what I was meaning is even if you want to go to the approach of having uh six dashboards if clients don't even have any metric at all of the thing you want to measure then too bad so that was that would be kind of the first step just hey clients we need to measure these things please can you have a metric and the second one would be to standardize so uh for Katy and maybe par you can comment there was a an effort long time go to standardized metrics and

it didn't work out so if you guys want to do that um yeah you have to be much more push pushy I would say just keep in mind that it will it will never be a priority so if you want this to succeed you have to both um make the spec simple enough that it doesn't take too much time on the clients and it's something that could be uh feasible in terms of priorization and then you need to be like really really rough in a way so be on top of the things have a way to um enforce these standardization somehow across versions and basically be on top of everyone or otherwise I would just abandon the

effort yeah totally totally agreed um also just a bit more context on that um earlier standardization attempt I think that was my first like 3 months at the EF and I think I have a lot more experience and ability to crack the whip now um a few years later than I did back then uh it's definitely something that's on the blade and there was just other battles to fight back then but I think it's worth fighting right now so I would I would push for a St got it I think related to that somehow we have the Bon API spec uh but I don't think anyone is tracking compatibility besides users just randomly having compatibility issues between clients so that would be kind of an example that yes we have a

spec yes everyone agrees on the spec but there is no enforceability or enforcing outside of maybe as forgetting to look at it every once in a while so that would be something that would be valuable if there is a spec do we have an automated way to make sure that LTS complies yeah that's been an open topic as well um so the execution layer has this approach with using Hive for spec compliance but the issue so not not necessarily spec compliance but making sure that the API is comp uh it's just no one currently owns it so I was kind of hesitant to add the beacon API into a stack that no one is owning anyway um but yeah if someone is

interested in working on that happy to help support you but otherwise I guess once um work changes slow down a bit we would definitely take it up it's it's a standard we want to help fix uh but the stack to test it exists and of tent Sol you got your hand up yeah so so regarding the metrics uh so couple of ideas that may help to to push this this thing so so one way which is a very hard way is just to implement uh the required functionality in the clients and open PR uh so I'm I'm pretty sure that the pr will be decent you will you would get uh

uh this accepted uh fast and that that would but of course it's it's it's hard to do that uh to add the metrics to all the clients by uh yourself and another option would be try to show that uh the metrics help to um help debug uh this functionality uh because we we sometimes actually spot some uh some quite interesting uh bugs or issues when uh when checking just dashboards and comparing especially if the dashboards are exactly the same it's it's not always but for pust you likely can can do the exactly same dashboards for all the clients and by checking the visual differences between the

dashboards uh between the different clients could be a good way to debug uh the peer do and so far it seems that the bugging of the peer do was was quite a big challenge uh for the client so so maybe you can push through this angle somehow yeah definitely definitely makes sense I think I think this is a good use case for why the standard works and will I guess help people also see that we should standardize other stuff thanks yeah I think it's a good time to do it as well because um I think like what Demitri mentioned that we can't change the metrics once it's been introduced otherwise it'll be a breaking

change for users so we tend to avoid that in Lighthouse as well so it didn't appear as it's new so it's it's a it's a opportunity to to standardize something early on so please review my PRS ready for review metrix PRS so we can go further with it cool anything related to devet um I guess we need more testing before we can decide on when to launch

right yeah exactly I'd like to um get to the bottom of whatever is going on with this spontaneous combustion of Dev Nets before we launch a public one um and one point I do want to bring up was um it was mentioned last week that a few of you at least have local machines as the bottleneck um so if you guys want I think we're happy to provision like a more beefy machine but uh one thing we did want to mention is we don't want this to be like a permanent thing because I do think local testing is valuable and all of your independent client teams should have some sort of infrastructure to help you guys fix it but at least for the specific issue if

it just gets the ball rolling we're happy to set up a machine and then give you guys all access and you can run your kosis configs there and I guess the main thing we wanted to is is that something that's useful or is it not at all and you guys are happy with the status quo anyone want to say something I know Jimmy you run a MacBook Air hopefully now you have a beier machine yeah I don't but um did request

one um internally so just for p testing um I've got a yeah exactly temporary machine as well cool so I guess it it will be by request um if we if any anyone needs needs a machine um we could reach out to to De Bops um guess yeah but I we don't want to make this like a every you know run it for a long time so ideally each each of the client teams should handle this internally but if someone needs something quick like you know by end of to day then we can do something

quickly but yeah just let us know yeah it's more to just help us get to devet to faster and figure out whatever the bottleneck is until then awesome thanks um so that's definite update um any back questions anyone I don't think we have any agenda items today um if anyone wants to um Jimmy maybe you want to bring up the um Network debugging stuff we were

talking about and potential debug API oh yeah um yeah there was a little rabbit hole that I went to last week um kind of like a little side project there um those last week we were debugging a few um issues between clients and we're looking at the RPC requests sent between clients and I kind of got interested in trying to build a debugging tool to see to visualize the PTP traffic between clients but I ended up finding out that's like almost impossible because of the way we encrypt the um the the network packets with noise

protocol so it it seems like it's it's it's quite quite hard to do with just capturing the network packet and decrypting it um so but you have had a bit of discussion with Perry and we think this thing could still be useful um but easier way could be to implement um like debug events um at the event API I guess um and potentially just spit out like all the RPC requests going out and coming in um and then we could have some visibility on um the network activity but yeah kind of want to hear if anyone

feel that this would be interesting because it's kind of like a little side side quest that I went for but anded up failing if if the tool know in advance the private key of the private key of the of the pier like I I know in some clients you can specify a given um private key and so if the tool know this private key as well normally it should be able to decrypt all the the message in coming between between pi yeah the problem is the the messages are not encrypted with the with the node ID

private key um it's actually encrypted with a a key that that's it's encrypted with a symmetric key that derived between two two peers when they establish a connection yeah yeah yeah yeah yeah so it's it's in memory and it's um it's if you like there's no way we can get the the private key use for Computing that symmetry keys so it's a bit hard with encryption like I feel like the only the easier way would be to disable encryption during testing or potentially split out uh debug events but only if it's useful so there's um I'm hugely in favor of the debug events also just thinking

about it um a bit longer since the last discussion about it um main reasoning is that there's scenario where the sort of having access to this raw Network event stream is also useful um there's a bunch of researchers who work with Tyler and the security team who are trying to do some formal modeling of uh the P2P layer and currently their approach is to use Trace logging and then using a log ingestor to understand which pier sent which message to whom and it feels like there's enough use cases for people to build on stop this is something that'll never be enabled by default anyway it's probably really deep in the debug space that if client teams can actually

support it it might be helpful long term for pias um essentially the moment you get a message you push it to a debug event stream SL Network or whatever um and then you forget about it and then we can build tooling around that expectation to handle more complicated stuff like visualization or understanding what's going on Etc and this would also kind of give us access to invalid messages that show up on the P2P layer which would never show up on the beacon event stream cuz that's post validation I guess my main question on the topic is is is this some fundamental reason why this is a bad idea or clients

are architected in a way that would make it impossible or we're talking about like a few months worth of work to get this to work I want to Echo that I think traditionally like this kind of local debug thing is how debugging is happening instead of going like you know the hardcore way of uh disabling encryption I think nobody really does the disabling encryption thing because it means that clients need to go to the critical security path and add

conditional code that like you know disables encryption or uses a null key or something under testing scenarios and I think that is usually considered very risky Behavior because you know if you mess it up you end up with no encryption in real life so I think this kind of local debugging thing is how usually things are done at least as far as I know yeah I feel like PR's idea of using the debug events could be useful because we could use it on real networks as well

even for like Hy or or maget um um yeah the decry the encryption the the not encrypting idea is not great um but yeah I don't think it's a huge risk because you can't really talk to any other peers with without encryption unless entire network is not is not encrypted um yeah but but I do think that the debu events is useful um yeah it shouldn't be too hard to build um but I don't know if we want to have this um I think it's probably worth having a spec first and then we can discuss Perry are you

volunteering I would like to not but uh happy to help get the ball rolling at least yeah can have um like some basic PR up and running and I guess we can like shed on that and outcome could still be that it's a bad idea and there a smarter way to do things but I guess the cost of trying is low yeah I agree um yeah happy to collaborate on that as well um if I can help um but if there's no like objections I think I think that's a good good start I guess um be good to to hear

some feedback though um do people find this useful um besides testing and like besides develop and testing yeah and I think just in general we're doing so many things to get pias shipped faster that making testing and Dearing faster is also a way to ship your faster so I guess my selling point is always going to be there yeah babus I think the um the reason why we want to have um thebug

events is is that I want to see like like the actual dollar being sent or the actual request being sent yeah I I I I totally get it like I totally understand why we want the debug API as well but uh at this point I would really want to have the general metrics done before we go deep dive into the debug API because I feel like this debug API would potentially be a much deeper Rabbit Hole than we expect uh also we don't necessarily need the de API from every client but the the P metrix would probably be a higher value low hanging fruit

potentially yeah so I miss I I misund what you mean um yeah but yeah I do agree AG agree it would be ideal to have both I guess yeah yeah yeah we we totally need both eventually but uh like the next thing to focus on I I feel like uh we should we should push for the metrics and then once we have a good uh implementation of all the metrics then yeah or if anyone else has some free time then they can recommend the the API as well but I I just feel like we have maybe one or possibly two uh devs per team and that's why it might be hard to push for both at the same

time yeah I think the yeah I think the DB MATC is not necessarily like part of pias it could be it could help with other other areas as well it's not really appeared as specific thing yeah yeah yeah for sure I have a question regarding um spec I think there was a discussion on Discord last week uh regard in um the sampling without P sampling PR um are we agreeing to something for DEET 2

or are we just just um just aiming for the current behavior for definite 2 raise implemented the sub samplings so eight so prison is now um subscriber to eight subnets and custodies four subnets and prism does not do anymore Pier sampling I see what I mean now so um so do you advertise those subet as well

when you in your metadata no only so in the metadata and in no record it's only four subnets which are advertised however uh we are subscribed to eight subnets and um in order to consider a block as available we need the eight the eight columns basically to be to be available to us I mean like the like the the specs uh States but where do you get the four from um is that based on the the this the number of samples in the configuration can you repeat

please um where did you get the four number from so so you add four more subnets right yeah yeah yeah yeah yeah actually you just uh in actually the spec is very clear um it's exactly the all the function you have to run are really written for you you have you don't have to to think at all uh yeah yeah um so so basically before you have the four subnets you should custody um coming from a function like custody subnet something like that and you you you you you call this function with argument four and now you just call this function with argument eight that's all okay thanks that's cool um yeah we

haven't imple we hav't Implement that Lighthouse I we'll try to implement in the next yeah it's really it took me maybe a five minutes to implement actually oh nice can you do onl house too that's cool that's great um I have a question about this um I'm not sure um currently for the four extra columns we do save them we do save them into into our database as well I guess it's not mandatory to save them because we are not supposed to custody them um yeah I guess this Spirit of it would

be that you only keep the things that you custody um but I mean if you're participating in the subnets and like [Music] you're like people will not make a distinction between the fact that you're c customing a subnet or not considering subnet like as as part of just you being in in the subnet subnet like if there were to be peer sampling then um people would only ask you for columns your your regular peers um if they knew that you were customing them but um while you're you know you're just in the subnet you're subscribed to subnet people might still ask you for historical um dat like historical

columns so I think you should still keep them um but yeah I don't know I guess in principle you could rotate those subscriptions um but then You' have to SN how yeah I think it probably just easiest to keep them I would assume um to just to not have to deal with this problem of uh people asking you for historical columns that you don't have yeah so I see just Just for information a prism never ask from we never ask from colons a pier does not advertise our as I mean we we do not we we of course we know which pier is subscribed to which shop net but uh we

will only ask for columns which are advertised through either not record or through either metadata I see and this is even for syncing like you you join subnet okay yeah yeah um okay but I guess I don't know is this not sure if this is specifying in the spec U so I think if we want this Behavior we we should really um clarify it in the spec so that there's no issue around different clients doing this differently and you know some client drops the data and some client asks for it or things like that um I mean I guess it could be up to each client to to decide to keep or not to keep those scon in in the database but I

really think that a client should not ask EV for ining um should never ask to another Pier a colon which is clearly not advertised as a custody colon basically we should not um a n should not rely on the subscribers on the subscribed subet of its P okay I mean yeah that makes sense um I think we can I mean I think if we agree that that's the right thing to do um then that that thing we should uh yeah specify um you're right like I think the keeping or not keeping it sure clients could decide to do it either way maybe we can have a recommendation not to keep

possibly I think that I mean if if there's no behavior of asking for that data then maybe there's no reason to keep it um but yeah so I guess do we how do other people feel like is is this something that we should uh specify do other people feel as strongly as as you know that being kind of the correct Behavior the lighthouse also only ask peers that um has advertise um the the column count in their metad data so if they if

they're not advertised then we we never we never um cery the peer for those columns I see um okay then yeah maybe let's just had um a note somewhere in the spec that that makes this explicit that that this how it should work unless there is one already Maybe I'm just missing and it's already there but I'll take a look right thanks any other spec questions I guess we're targeting for this change for definite 2 I think that

was the the previous um consensus on Discord I guess cool um um then we move forward um guess there's no other spec question than any other open questions anyone um I'm curious if people are happy with the direction the uh proof computations uh have gone and are going like uh are people okay it was raised at least two weeks ago and I think you know there I think today is the first meeting for the distributed proof computation

are people happy with that too early to tell for me okay I'm saying because it feels like uh uh we've moved to the distributed proof computation approach over the um let's use the previous slot to pre compute things approach and I think that's a sensible choice to make and better long term I'm just wondering if I don't know

people feel uncomfortable about it or yeah I think it's a good idea um I come make it to the call later today but I think the precomputation is is is the way forward I guess um we've got some numbers on um the optimizations that that we've done in Lighthouse with um fetching bloves from E and the proof computation time for 16 blobs actually looks pretty good and but but I think with precomputation we could even like like make it even better so um yeah I think I think I'm I'm in general in

support of um precomputation um distributed block building I think they they're potentially two two phases one is the precomputation um where um no could participate in group building of um non lot from the year and there's another one where we we could like after the blocks been proposed if it wasn't pre-computed then um the other noes could also participate in distributive building of the of the of

the blobs that that that they he about and so so I guess the two part of distri block building L house has implemented the latest part where we block e after the blocks been proposed and that's pretty straightforward I think it's um probably the less complicated ones and with pretty big gain as well um but pre compation could bring us even further I think yeah we're hoping to share some metrics um in the next week um whenever we already we've tested with 16 bloves it seems like um it's it's it's

working pretty well here mean distributed block building right not pre computation yeah that's right yeah but but even with precomputation that that's also like distributed block building as well but just before the proposal um the proof building part Co um any other open questions yeah just wanted to get a temp

check on how people are feeling about splitting backra into two forks I think it'll be a bigger topic on ACD this week in case people have concerns or whatever that might need some time to think about before ACD yeah I wanted to bring it up uh if we split we are definitely going to ship the next Fork so fusaka in q1 2026 um is that too late for PR d I don't actually think it'll be 26 to be honest I actually think that we'll ship faster if we split the V into two then just

one so how are you going to prevent usaka from just swallowing a bunch of randoms yeah that's the hard part that I've also been bringing up on act um I genuinely think we just have to get better at saying no to more eips and I think a big problem of that is also just people not wanting to be the but I guess more voices saying no more eips is a good thing but yeah definitely a point you should also bring up on ACD yeah I think faka scope could

probably be just limited to us and P death uh without anything else and then that way we could Pro possibly push it for like Q3 next year so like two smaller Forks rather than one massive one yeah I think the argument I've also heard being used is that we've kind of already scoped the fork so we're not going to open the scope foring discussion again any other comments on

splitting does anyone heavily oppose this idea of leaving P death out of pectra I mean I'm opposed if it means that it ends up being BM bundled with other stuff that he that really doesn't need to be bundled with I already feel slightly like not so at ease with the fact that it's going to end up being bundled withf um and I hope that it doesn't have to be the way like if you ends up being delayed for whatever reason I think we should just say we just do a for quit peer do um because I think it's just like strategically much

more important um but other than that yeah if we can like really you know still keep in mind that it is a very important thing that we should ship as soon as we can after this kind of initial smaller Fork then I I don't have any problem with that I think yeah um yeah I GRE with Fresco I'm kind of concerned that we lose the momentum of hereas because I think we've been moving pry well um I think the my current concern with I mean at the moment I'm kind of agreeing that we we don't do p as in the

first P for because just feel like we're not ready yet for TI one um 2025 um and I guess that brings to the next question that I have in my mind um I really wanted to understand like what are the remaining features that we have um in order for us to ship p as like because we do have a few more spec changes and I want to understand like what's the the minimum viable features that we need to implement before yeah is shippable Chang are you talking about uh yeah like things like theid data custody and custody groups those we haven't work on work on them them yet and I can want

to understand are those like required for the first um iteration of p and is there any any more things that we should include before we can sh p so I think I mean if if you know this presented a problem we can also not do custody groups I I don't think it's a it's an urgent thing um it's it's more just like a kind of yeah cleanup thing I guess while custody I feel some strongly that we should do it um because I really think that it will actually make the the um roll out like much safer um and uh but I mean like really the

basic validator custody not not any other like I don't I don't plan further to be like further um like iterations on it for now and I think yeah that this basic thing of just saying you subscribe to one subnet uh per uh 32 e that you have basically um one one extra sub you custody one extra subnet per 3 tws so I'm hoping that that shouldn't be too too uh kind of uh much more to Implement on top of the existing kind of custody infrastructure um other than that I I don't think that there's anything else um I think the only things it's more just this uh you know distributed block building and Etc ET Etc kind of optimizations for uh trying to get u

local block building as viable as possible with as many blobs as possible but in terms of the spec I I don't think that the that it should change I guess that's the question then right is sharded mol is it on the critical path or is it un nice to have that yeah it doesn't matter if we don't have it before we want to Shi at us right yeah that's that is the other question I mean I think um it's uh yeah I think we should probably not consider it to be on the critical path um like just because I don't know it's seems seems a bit late

um for being confident that we we can do it something like that in I mean I don't know there's ways to do it I think which are not necessarily like a huge lift um but yeah there's like basically hardly any kind of effort that has been started in that direction except some like preliminary thinking so I think um we we we should not consider it to be in the critical path and just accept that um it it might it might mean that um we cannot get the full benefit benefits of of doing Charing on the c um but I think we can still get some benefits and we can as we kind of um have the well as we can test spe do better and better we can understand maybe uh

what it means in terms of blob count like what's the impact of the not having a shal mle on the blob count that that we can actually support and if you know if it turns out that we cannot increase the blog count by by a lot without uh doing some form of Y sharding uh then yeah then so be it I think or I mean maybe eventually we can reassess and decide okay let's let's wait a bit longer uh until we can do this but uh yeah I think maybe this is also something that we'll have a bit more clarity soon because I think we're going to I mean we've been talking about it and um we want to talk to Felix about it uh more and I mean whoever else on the yell is interested so maybe this is one thing that yeah we'll know a bit more in

a few weeks great thanksa um you just added that um from my testing with 16 16 blobs um I think the the ear bandwidth doesn't seem too bad um so yeah if we're just ining to 16 probably it's probably not like a blocking issue but if we increase it to something like 64 then we probably probably will need it and also want to like understand like whether we need all of these optimizations like like do we need all

the features of distributed block building in the the fork or is it something that we can do after the for because it doesn't really impact um the consensus um I mean my personal opinion is that we could ship p d without uh this distribut block building infrastructure or like maybe with like a minimal amount of it maybe just with uh pre computation um and uh yeah or like anyway with without this infrastructure being able to support like um a very high blob count um and for for two reasons one reason is that I don't think

we're going to increase the blob count either way by so much um like you know if we just I know double The Blob count still or and I mean we could even do less than that but um that that's still we're not talking about some crazy high low numbers um so that's one thing and uh yeah the other thing is that it doesn't really um in terms of Revenue it's barely a factor like it's you can be like a it basically doesn't like impact at all profitability for for someone to not uh include a lot of blobs and even from the network perspective if it doesn't matter too much if like a small percentage of network like home stakers don't include a lot of blobs

like if they only include it up to six or eight or whatever it is um that would still be fine like from the nwork perspective just because with the 1559 mechanism you still recover the the throughput that you lose uh in like these slots where you have like a lower resource proposer so it's my again my personal opinion is like this shouldn't be um also a blocking issue like with can go ahead um ship like a conservative version of P do where there's not going to be any crazy uh blond count increases and kind of iterate from there improve the distributed block building infrastructure improve the Y mle we need and and try to you know do things in

this way like iteratively and not trying to have like a perfect thing where like uh you know we have like a super high block count and and local block Builders can also um propose like as many blobs as as possible from from day one like that just doesn't seem to me like a super uh neither realistic nor like super useful goal and uh again I don't think that this is going to like kill local bu building um but again this is my personal opinion great thanks Franchesca that's really helpful um I guess we're on time now um there's any further discussions we can continue on a sync on Discord um but that's all for

today um thanks everyone I'll see you guys in two weeks thanks thank you thank you thank you bye bye bye thank you bye bye