All Core Devs - Execution #176

Transcript

[Music] J [Applause]

for yeah

Tim Beiko: um welcome everyone to uh

ac176 um so main thing today uh is the denune upgrad so uh be good to get some updates on the devet from the different kind teams and also to try and figure out um uh what we want to do over the next couple weeks and potentially over the holidays um as we have an ACDC scheduled next week uh acde two weeks from now um and then I don't know if we'll have ACDC over the holidays and and uh around sort of New Year so uh the last couple calls we have to uh sort of coordinate um and then after that I just

have a couple small process things around meta which we discussed last time and then starting the discussion around the next upgrade um but yeah to kick it off uh I know we launched defnet 12 uh we've been running some tests on it so maybe uh we can start with just a status update on devet 12 and then go into the various client teams um and and where they're at I don't know Perry or if Barnabas either of them is on the call

Parithosh Jayanthi: I can go yeah uh so we've had DEET uh 12 up since last week it's been finalizing uh during regular operation we did find a red park that's been patched now um on Friday uh we launched

Mario's blobber tool along with a new action it can perform that triggers slashing so that's the new uh equivocation condition and via the test we found I think one or two Lighthouse slasher related bugs um on Monday those bugs were patched and now we do have slashing sunchine so if you go to Dora there's a section there where you can see some notes are slashed um once we did have uh once we did confirm that slashing works we turned on me on more nodes and we did find at least one uh Mev relay related bug that's also been patched now and

we've rolled out Mev to a lot more notes you can see that there's a healthy distribution of delivered payloads uh we're still seeing like onoff errors on the relay so nothing's fundamentally broken but there's still something broken um that would probably be dug Deeper by the flashboot team um and I see there some chatter on interrupt now with JY posting a potential map bu with Lighthouse I think I think that's uh pretty much everything we have on dev networks

Tim Beiko: what can you just clarify what's an onof error

uh we

Parithosh Jayanthi: we're noticing like a few um

error logs on the relay we're not necessarily sure why um there's something about slots in the past um I think sh from flash Bots mentioned it could happen when there are no transactions that the Builder would be POS potentially building a block with but that doesn't happen very often

Tim Beiko: got it uh thank you um and there's a comment from that about trying to test their fall back uh to the relays by shutting them off from time to time have we done that or

Parithosh Jayanthi: uh we haven't yet we need to still test the circuit breaker but yeah

Tim Beiko: awesome um yeah any client team want to

share some uh more granular detail from from

Daren

Terence Tsao: still working um um revamping some of our original design with regards to like blobs catching and verification later so I think we're we should be able to join death 12 maybe next week if not if not in two weeks so that's just an update from our end

Tim Beiko: got it thanks hey

Danny Ryan: and I know um I think David was talking to Casey yesterday but essentially a lot of the the security researchers are like super eager to help do reviews um you know the testing team and and devop team are super eager to

integrate yall so if there's anything you know that people can do to help support yall um to get you to the the Finish Line just make sure to to reach out

Terence Tsao: yeah thanks we have got um a bunch of good uh feedback from Justin from the from the security team already so thank you so much

Tim Beiko: awesome thanks Terence any other team so if there's no specific updates um I guess what the teams feel like the right next step is at this point once uh

you know prism joins the death Nets and obviously uh you know we we find if we find any issues there we we fix them um do teams feel comfortable moving to test Nets um or at least the first one like Gordy do we want to do potentially a shadow fork in the next couple weeks as well to to see if we get more data out of that um yeah how do people feel we should be moving forward at this point

base is ready full steam ahead yeah same never mind Aron is

ready star is also ready

Ben Edgington: no concerns on teu side yeah have to move forward

Tim Beiko: I guess is there any team aside from uh prism who's like uh feels like there's something biger or sign ific that they still need to work on obviously every team probably has some bugs and some um some small small things to fix but is there anything else that people are are concerned with um and Perry has a comment about testing the slashers are there other clients which have slashers that ready that we could test on the devnet right

now

Danny Ryan: I'm pretty sure that's just prism and Lighthouse

future

Tim Beiko: okay so we can test pris as soon as we we we have it um so I think we if if uh prism needs a couple more weeks and you know potentially we find some issues uh and and and you know fix them um and also not wanting to Fork like a test net over the holidays um I feel like we we might be in a spot where we can aim to coordinate a fork on gordi as soon as people get back from holidays um again assuming we don't find some

other crazy issues that you know or or something that delays things by a few weeks um but generally does that seem reasonable to people that like you know I don't know when would be the first 2024 call but using that as a time to uh set the fork date um and then potentially having the fork you know two weeks outside from that um so if if the first call is uh you know on the first or second week of January this means we'd um this means we' we'd be forking Gordy sometime mid January you know something like that does that seem reasonable to people um at this

point and again pending like obviously I know prism has some you know some changes we need to test on the death net and like that could still reveal an issue but assuming that uh after a couple weeks it's integrated and it's it's also tested um yeah

Danny Ryan: yeah I'm pretty Pro conditionally doing that you assuming we can get prism integrated in and battle testing got it

Tim Beiko: um and then so if if we if we're aiming for that oh sorry Ben please

uh I

Ben Edgington: just wanted to check status on something so so one of our concerns

moving from targeting two to three blobs per block was around Network propagation are Are We Now satisfied that this is going to be fine on Main net have we done the the have we gathered enough data or is that something we need to do on test Nets and and where do we fit this into the the program and the planning

Danny Ryan: so from my understanding um both the main that big block experiments as well as the organic Big Blocks on the order of even 1.5 sometimes near two megabytes that we've seen on Main uh were really solid and test Nets continue to be solid

and Shadow Forks may not Shadow Forks would be telling um at least to some degree although all of those test nut related items are on the order of small um I would be supportive of between now and mid January uh trying to do some multi- region pretty large node uh experiments on the or either either with like um potentially even on on Main that shadow Fork level uh just to really get a view on it let's see what is Perry say yeah so Perry you said 300 nodes

Parithosh Jayanthi: yeah the uh plan is that once every Clan's ready for uh de 12 we're going to do one really big Shadow for so something like 300 notes um and we have the tooling mostly ready for that but we'll find out once that shadow for is up um and the is sorry go ahead

Danny Ryan: the cost differential on going on doing that same thing on main that is is on the order of many multiples

Parithosh Jayanthi: surprisingly not that much G is really really big right now

Danny Ryan: okay uh interesting yeah I guess the the main thing the main thing that we'd want because I you know in in the gossip flow is not

really execution against the main that state um but the big thing is the gossip kind of competing with main net level probably transaction gossip is the main other thing that would be kind of telling and probably switching from Gorly to mainnet although we could potentially uh make sure that a Gorly mainnet Shadow Fork had a lot of transaction gossip but anyway I I would be supportive of 300 or even ramping up to more if if it's reasonable in terms of man hours and costs even if it's like an hour long test

or whatever

Parithosh Jayanthi: yeah I think we'd have to

keep the test really small for sure um but at least one thing we've noticed in the past is none or all the shadow foxs Etc that we've done are too pristine um we're only going to get any real data for a P2P related upgrade from natural test net so at some point we're going to have to buy the bullettin to girly and I would make the case that we spend any extra money we were going to spend on better monitoring setup for G and do it earlier

Danny Ryan: do we have an estimate of how many nodes are on gorley proper

Parithosh Jayanthi: a crawler I don't have that at hand but I can find that out

all

Danny Ryan: yeah it'd be really interesting to get like

just even order magnitude crawler data because if we're on the order of like thousand I think that that would be very valuable data if we're on the order of 300 then it's like not much much different than what we're going to get

Tim Beiko: the newest C has about 3K on gordi versus almost 9k on 9k on Main net so I don't know how reliable the numbers are but if if the ratio between them is is is good then it's like still a third of my net which is it's just pretty decent

Danny Ryan: that's really good especially with the way gossip should scale obviously you have degenerate cases but should scare with the log of the size so it's not actually terribly different in

terms of Pops yeah

Tim Beiko: oh actually sorry so if uh and okay no no yeah it's about 3x ratio um yeah um so okay so we can aim to do at least one Gordy Shadow Fork um by the end of the month then trying to Fork Gordy next uh early in January and then um that'll be probably the time where we would uh change the blob count basically like if we saw something going wrong on

gordi that was sort of attributable to that but aside from that there isn't really I and I mean you know we might see something on sepolia as well but all of those networks have like a smaller like if it works on on gordi it should probably work on sepolia and and on hsky um but gordi is probably the last significant test where we'd learn something that uh makes us change the blob count before we go to

main net

Danny Ryan: yeah makes sense it's our we're sacrificing it

Tim Beiko: hopefully not but yeah anyway

Danny Ryan: it's it's it's okay if it is sacrificed yeah um yeah

Tim Beiko: Ben does that uh satisfy your

question

Ben Edgington: yeah I think so so I mean to to summarize uh girly network is is likely where we'll get the best data on this um so as long as we got the tooling and the kind of time to to do that analysis at that point um I think that's fine

Danny Ryan: and knowing that a Gorly Shadow Fork before we will be spamming and monitoring for this condition specifically uh so that you know if it fails there we know it's probably almost certainly feel like really

Tim Beiko: um sweet

um okay so I guess yeah so in terms of next steps yeah let's uh let's get some more testing on on devet 12 get uh a Gordy Shadow Fork up and running as soon as we're back next year we can uh finalize the gordi fork date uh potentially we can also have some proposals async on the Discord and you know um use the call only to like finalize things but um we'll know a bit better over the next couple weeks how how things are looking um but the aim would be that in January gordi gets fored um I see Terren you have a PR uh in the

chat about uh uh an execution API uh change and do you want to bring that up now or is there anything else that people feel is more urgent with regards to

Danny Ryan: I have one more question on the

Gorly stuff let's Y

Danny Ryan: is the node distribution similar toet not necessarily the validator but the node the

Tim Beiko: um yeah I mean I'm just looking at this ethernet's uh crawler so I mean if I mean

Danny Ryan: if it were way out of whack

not going to necessarily perform like me

Tim Beiko: but it doesn't feel crazy different but this is for E not uh CL so I I I don't know what the SE looks like

um yeah okay

and see if we can find some

Tim Beiko: yeah in terms of the countries it also feels roughly similar so like yeah it feels like of the three three test Nets by far the closest um yeah yeah okay um see anything else on gordi or Shadow Forks okay um Terren uh yeah you want to talk about the execution apis

right

Terence Tsao: so little background in Cancun um when you um try to get a

payload from the C to the E there is this optional parameter it's called overwrite meaning that as a EO client you can trigger C client by telling there if there's some sort of like um censorship going on on the nwor so censorship it's a very um um subjective term here this is CL client has sorry e Cent may have like different design implementation on how to OB observe this this this sort of censorship like for example like a transaction that's been paying higher than normal priority fee but it has been stuck in the main pool

for a while and the builders are not including it blah blah blah right so um so since this is available in um Cancun I wonder if any EO client has look into this and then and implementing this sort of override feature and the reason I am asking this is that because by as as as we have seen over the last few weeks right um there are there are staing pools there are actively discussing U um delaying um blog um publication just to it it's TR more me and this may become a problem once we

have blob a block transaction because block transaction also delay the block gossip right so imagine today we have block transaction that is in the main pool but it doesn't really like make that much money because they are cheap block transaction they are boring and then as a station pool I will have the decision whether to include block transaction or whether to or whether to not include it instead of just wait a little bit longer and get more M so there's some sort of this type of game that the state state can play and it's not ideal so I think having some sort of like um censorship o override um with some sort of respect on

block transaction could be fairly useful so I wonder if any client team has um thinking about this or yeah thinking about implementing this oh yeah like so like has the comment saying that Mario's implemented it but but then they don't think they will merge in the near future yeah I wonder if anyone has any more like feedback comments on this

Marek Moraczyński: so we haven't implemented it yet in

NE mind Basu also has not implemented it yet got it

Terence Tsao: thanks I don't yeah I can

also add another comments on Discord later just to eer said yeah I still think it's very important to have this early on because like 95% of the blocks today are build by Builders and Builder has a lot of power on just like including like blob transaction and choosing not to include it and I suspect in the beginning blob transactions are going to be very very cheap such that the buers may not even care and yeah so there may be some weird information a symetry there

Danny Ryan: yeah and I guess to be fair as Roman mentioned like on the local setup if a pool really wants to be playing these games they can just override um but I guess also if we do see this uh

the equilibrium of what the priority fee in the normal case for l2s um is probably going to shift to much higher um so hopefully you can find some sort of balance but I I agree some of these Dynamics are pretty concerning

bis

Potuz: so I was thinking along the lines of what Danny just said that uh that it just might be Market uh this block transactions will have to pay and off so has to offset whatever the fee is required so that they the proposer actually includes them because also it's something that that risks their block to be to be reared more but I think it's different than with usual transactions

because this is not now playing with these pools that are either publicly or privately delaying blog production as much as possible so now blobs not only need to compete with fees but they also need to compete with that delay itself and all of the M that you can get out of delaying your block um I mean this this is a market that I think this wasn't prevented this wasn't thought of when when the fee mechanism for blobs was

the sign so yeah I mean I think this is this is true

Dankrad Feist: but um blobs can use the same tiip as normal transactions in order to

compensate for the uh increased offering risk

okay

Tim Beiko: um I guess what's the best place to continue conversation on this um obviously the pr is like closed but is there uh okay yeah Terren will bring you up on Discord um yeah which channel do you think makes sense or was there already a conversation on the Discord in one of the channels okay we'll use

ACD

Danny Ryan: um there's

also a good post um recently by Mike and Casper on E3 search kind of digging deeper into the time game stuff going on in general if people want to take a look there to put some more brain power towards it

Tim Beiko: awesome uh yeah thanks for bringing this up Terence any final thoughts comments okay um anything else on denune in general okay if not um I have just three

quick uh updates on the meta e stuff that we talked about uh last week but um we have a meta draft for denune uh I forget why but there's some reason why it's not merged yet I think it just needs an EIP Editor to merge it um and then we didn't use meta eips for a couple years uh so instead of backfilling all of The Meta eips I created one to um reference all of the specs for the previous hard Forks um so that it it doesn't look like the The Meta eips were always there uh but it's saying like for this chunk of forks

where we didn't use Mets here's where you can find the canonical specs um it includes both the E and cl Forks um so if people want to leave comments on that um I'm hoping to get it merged soon as well and then lastly um so I I created this thread on eth magicians to start uh discussing the scope for the next Network upgrade uh I know there was also an issue on the CL side on GitHub for where people could uh post suggestions uh but there started to be some conversation on it and I think uh yeah as we um as as we start wrapping up at thenon uh then we can spend more and

more time thinking about uh how to scope and uh prioritize things for uh the next upgrade and I think one thing that's probably worth it for client teams to Think Through um now is like are there uh proposals that uh would cause the upgrade to be coupled or decoupled across the El and cl um so the past couple Forks uh were obviously coupled uh because we shipped stuff that that sort of required it um but it's there's at least a possibility I think for the next upgrade that this is not the case um and so it's worth thinking through that like is this something we we you know we we want to to explore and and um

are if there are things that couple you know um how important are they and and you know do they risk delaying a bunch of other stuff um yeah I think that's uh that's yeah coupling sort of happened by default the last three times but we should uh yeah El trigger Bol exits is probably the the only one that I'm aware that requires the fork to be coupled um okay it seems It's a okay there's more popping up in the chat okay

Danny Ryan: yeah I I mean I I'd really love to see wherever veral goes to not be coupled uh because simultaneously I think we'll probably work on some sort

of data availability sampling upgrade um and those are very independent uh but obviously then there's a lot of these small things that might be

Tim Beiko: yeah and one thing maybe to Think Through is also um how early late in the in the process do we want a couple things like you know say that we did veral and Das um and they're decoupled and then we choose to add El triggered uh exits you know can we couple stuff later in the process what does that look like um um yeah I don't know I these other teams

or people have thoughts generally about approaching the next fork or how should we think about that um I don't think we should start discussing specific eips on this call but yeah at a higher level any thoughts and I guess maybe okay so last question then these past couple calls have ended a bit on the earlier side um so would people want to start discussing the next fork in more details like in the next couple weeks before the holidays or do people prefer to table

that until uh early next year and so they stay focused on uh on the fork oh like clients wants to get into it now we can do that too if yeah does okay it seems only like client wants to discuss right now

dapplion: um I'm down

Tim Beiko: I mean okay so yeah I think we should probably do most of this oh after

the holidays um I'm fine giving people a couple minutes now if they want to bring some stuff up but

pus I

Potuz: I do agree that we should probably defer this to after the holidays uh but I I very much find it unlikely that this is going to be de coupled or at least on the CL side there are some kind of urgent matters with the growing validator set so I think like the few PRS that or ideas that are being discussed on how to uh lower the Bator slice or to reuse indices or I mean to set up the stage so that we can actually lower the validator side we should start working towards that

Danny Ryan: yeah to be to be fair on

70002 or not on the uh Max CV proposal you could potentially introduce additional uh CL messages to counteract the ability for needing the e um but there's not like 100%

Tim Beiko: I'm happy with

Potuz: the CL only fork issues that I don't yeah I don't see I don't I'm sure there's going to be El changes that are also important

Tim Beiko: and to be care I wasn't trying to to push necessarily for a decoupled one but I I just want us to make make that like a conscious decision not just default the coupling because we've gotten used to doing that

the past couple Forks um yeah um Okay

g

Guillaume Ballet: uh yeah just uh regarding the coupling actually veral is mostly an e thing but since we're putting uh Pro like we we're Distributing Witnesses and so far the the decision we okay it's not decision but the approach we have made was to to pass this uh this proofs and Witnesses via the CL so it would actually be coupled

Danny Ryan: is that P2P only or is that a consensus change as

well

Guillaume Ballet: um no it's uh I mean it's an execution engine change and then an EI sorry a peer to-peer um it's not actually I don't know how how Mac and gajinder implemented that in the state so I don't think it's a consensus change no but it's um I might be wrong it depends on the implementation detail I'm not a I'm not 100%

Tim Beiko: oh got it uh Justin

Justin Florentine: uh I just wanted to U I posted this comment in the chat has the ship

already sailed on using meta eips for hard work scheduling are we all firm on that

Tim Beiko: what is the what is the argument against it yeah

Justin Florentine: um well I just wanted to again I apologize for not being here last session it was holid um but uh it it seemed a little counterintuitive since we just spent a lot of time deciding you know ercs and eips should not be in the same repo because they have different audiences and so I was wondering like why not just simply create a separate EIP for for hard works and scheduling I'm sorry not an EIP a separate repository rather for hard works and scheduling

Tim Beiko: so maybe my like practical

answer to that will be because there's just one of those that happens every year or maybe two um and it's kind of weird to have this separate repo for like 10 eips that said I'm not completely like if I don't know it yeah I'm not super against it I do think we want them to be like closely coupled the eips like where they were before was like randomly throughout the specs repo and that's kind of hard for people to find versus like meta eips you can just go to eips doe.org and search for London and you know there's like it pops up there um so

that's like the main argument I see in favor of metes it's just like they're easier to reference um whether or not they like live in a separate GitHub rep is maybe not super corated with that but um yeah I think that's the argument for like by default having them in the same

rep

Justin Florentine: um okay so traditionally when I think of meta um blank right it's it's usually um you know specs about specs for instance EIP 1 right this is where we Define the the EIP process and how future eips are to be written etc etc so that's when we say meta eips that is more along the lines of of of what I think yeah

Tim Beiko: so we've been talking about renaming them which I'm totally fine with

but it

Danny Ryan: to be fair it is kind of a

metas because it says this specification for this upgrade is these five specifications or something like that so by referencing an EIP it is med but I I don't care about the name

Tim Beiko: yeah I yeah I care more about the proximity than the name and yeah some EIP editors have made the same argument as you Justin so if we decide to rename meta eips to like hard I don't know Fork the scriptor eips I'm I'm totally whatever name we come up with I think is is fine but I I think not having like first of all having a list of what's in the hard Fork I think is really important and we didn't quite have that the past like the the list ended up being the blog. ethereum.org

article which is not great um 100% and then and then having that list be close to the actual eips but I agree the name is is you know potentially not uh not great um we Can bik Shed this off the call though

Justin Florentine: yeah yeah I'm going to drop it I just want to it's not so much about the name it's about you know the repo that's all yeah so all right thanks

Gajinder Singh: just to chime in with respect to work being uh uh consensus uh Fork on clite yes it will be because uh execution witness is now part of the execution header which will basically

require uh us to uh have consensus on

Danny Ryan: okay and I guess to to be clear and to be fair like some some of these things are more like insert the new field into the data structure rather than uh dealing with like a a feature beyond that um and so you're right in in a lot of decoupled forks there might still be some of that like process upgrade that needs to happen without like a ton of effort happen so we should at least like keep that in mind when we're talking about

decoupling because I could have still imagined like brokel could quote be decoupled meaning the consensus ler makes sure to like give them a branch with this extra extra field in it um which could take not very long um for testing and then kind of come in together and make sure everything lock tight

again

Gajinder Singh: that's correct because State transition has not changed uh on the seal site because of execution witness being added also uh there could be a different mechanism in which execution witness can part can be part of blobs so basically it doesn't even need to have that kind of upgrade

Danny Ryan: yeah and I guess just another Point like if we did sorry like we had a data availability dominated consensus layer Fork there'd probably be an upgrade to the data Gap uh but on the execution layer but like I'd so largely call that decoupled

Tim Beiko: okay I think um yeah Bas anyways based on this

conversation uh I think for Prague and Electra we should probably do is then have people I guess signal now to people that if they want something consider it's like the right time to start raising it and that over the next month I suspect people will like bring up a bunch of proposals if client teams can keep an eye on both The eth Magicians and and GitHub thread um then we can sort of kick off the calls next year's getting your feeling from the client teams about all the different proposals you know their relative value and kind of go from there um that seems probably more productive than trying to start

discussing random eips now um and I think in parallel to that one thing we discussed in the context of Def connect is uh trying to do some sessions uh on like specific topics uh remotely after after def connected like get people up to speed on stuff so if things like uh veral or potentially like a Das Fork on the CL are being considered and they like pretty significant changes um it might make sense to organize a session for at least the two of those and I don't know if there's something else um because I I I suspect those will take up like more time to go over and have like a much bigger degree of of

discussion than um sort of a a a random small EIP um yeah does that seem reasonable to people no objections one thumbs up and I guess yeah we can try to roll with that okay anything else anyone wanted to discuss before we wrap up today okay if not uh well yeah we can uh

close out here thank you everyone and and and we will see you on the testing call on Monday and then uh ACDC next Thursday

thanks thanks bye bye thank you by thank you h

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.