All Core Devs - Execution #124

Transcript

[Music] so [Music]

[Music] [Music]

[Music] [Music] [Music] [Music]

so [Music] [Music] [Music]

so [Music] [Music]

[Applause]

Tim Beiko: okay we are live oh and i'm muted okay we're live take two uh welcome to awkwardev's number one two four uh let me post the agenda in a chat here um so basically two big things on the agenda one updates on the merge um and then second uh decisions about the aero glacier and uh the difficulty bomb push back um to start i guess with updates on the merge um we had a face-to-face uh meeting last week with uh most of the

client teams in person uh to work on the merge um i'm not sure if someone wants to give kind of a recap i know a lot of the people on the call were here um yeah i can give like a very high level one but i think a dev would be better

you have to do it you need to nominate someone okay so

Tim Beiko: i'll give a high level and then um yeah uh people can fill in the technical details um but basically the goal of the event was to try and get uh the different team on the consensus and

execution layer to interoperate with each other um and to iron out any kind of remaining issues in the spec and uh basically the the consensus back execution as well as the the api between uh both layers um we did this through like a series of of milestones so basically uh they went from just simply implement the spec in your client and pass a set of reference tests all the way to build a multi-client proof of work to proof of stake devnet which runs through the entire transition and we managed to get most teams through most of the milestones um so we set up uh several devnets during

the week basically starting one to one so one consensus clients to one execution client and then kind of growing you know uh many to many or passing multiple one-on-ones um and yeah like i mentioned towards the end of the week we were able to set up a kind of temporary devnet which had i believe 100 nodes which ran 10 000 validators across a different set of clients and um it started on a proof of work network uh went through kind of the transition to proof of stake and then managed to finalize on the other side um so it was kind of a really good uh success in that you know it showed that we could get all these clients interoperating with each other uh that we could get the spec mostly ironed out

um and uh that we had kind of a clear set of of things we needed to fix going forward um talking about that i guess um we've launched a just a more stable version of this this final devnet called uh pythos ptos i'm not sure how you pronounce it um which people can join if they want to kind of really experiment with uh with what the post-merge infrastructure would look like there's still some spec changes to be done so this is probably not the the final version but i suspect towards the end of this month we should be done with those um and uh yeah i think that's a kind of a high

level anything else that i missed that people wanted to add

Diederik Loerakker: yes about beatles um we started with amphora during the entrap event this was on a minimal configuration in ethereum 2 which turned out to be actually more stressful than the mainnet configuration because of the sheer amount of metadatas mainnet configuration is built for this type of nut so we decided to launch pitos which is also an infohara just a larger one hence the name and thanks again paratorch for helping with the devops here if you need notes or need keys to

validate on this network let us know i'll try facilitate that and we're trying to stabilize the remaining clients so far we have three consensus clients running really stable on the network and gaff on the execution side then we're working on getting bezu and netermind running

though as well cool

Tim Beiko: thanks for sharing anything else

Micah Zoltu: what is the appropriate coordination channel for the pizzas test net would that just be the interop channel in the merge category on discord

yes that works okay

cool

Tim Beiko: any final things about the workshop that people want to share

um i

Marius van der Wijden: i just wanted to say that i finished a differential fuzzer for the engine api so if execution layer teams are interested they can pass their implementations against

gas

Tim Beiko: can you share do you have a public repo for it already or

Marius van der Wijden: yes okay i can share a link

Tim Beiko: awesome thanks um

cool anything else on the

face-to-face also

Diederik Loerakker: one thing i'd like to share where i think part of the success of the face-to-face is that we had these clear milestones as coordination points so we could get clients in the same boat and figure out like how to test with each other i think we should keep this thing up where we have points of coordination on which spec version are we at and these type of things because the spec will keep changing we're changing the api a little bit

um this would really help i think

Tim Beiko: yeah that's a really good point so i think um yeah i was talking with danny and mikhail yesterday i think there's like at least one more large round of changes where they want to fix a bunch of small things once that's up we should probably make that kind of the next uh the next milestone for for people to aim for and and make it clear that you know this is the point where we where we head towards and if there's other changes we can kind of implement it afterwards cool anything else um

one thing i'll ask uh i know there was some progress on the sink uh at the face-to-face and that the basically the uh the implementation is quite different from what felix had proposed a while back is there like a repo or a spec or something that like kind of documents that or is it basically the get pr

Péter Szilágyi: uh no there isn't yet uh spec so we do want to update it but i kind of there are actually two steps towards that one i need to have a long chat with felix be aware of all the all the details and whatnot and the other thing is that we're also

trying to figure out one or two open questions before committing to writing it down okay so we'll try to get it up somewhere asap but there's no updated box yet

Tim Beiko: okay got it thanks any other just thoughts comments questions about the merge in general okay um and yeah one final thing we put out a blog post on blog.ethereum.org today kind of

also going over the the face-to-face so if people are interested in kind of seeing all the details there um yeah i recommend just reading the blog post it should be the last the most recent one on blog.ethereum.org um cool so the other thing uh we need to go over today is um the difficulty bomb pushback um so as uh we know the difficulty bomb is is gonna go live uh sometime in december and so if we want to like uh agree to how far we push it back and and when we do so and put out client releases uh that needs to happen kind of soon

um so i put together a bunch of different options um and shared them on github and basically there's two kind of decisions we need to make one is when do we want to push the difficulty bomb back to um and when do we actually want to have that that upgrade um so for the actual uh ice age offset um i'd first propose kind of a value that would push it around basically mid april of next year you would start to see uh some noticeable increase in in the block times and then around mid may you would see like a second or more increase

in block time which is um uh which starts to be kind of the point where we absolutely have to push it back um and then uh i i also uh kind of got tj rush uh to to sanity check these numbers he's done a lot of work on the difficulty bomb before um and he agreed that this you know this would have these numbers were right but he suggested we should probably push it back a little bit farther so his proposal was we push it back instead of 10.5 million 10.7 which means the bomb starts the show around mid-may and then we would have to push it back around mid-june um yeah and then there's a middle value there where if if you don't want to do 10.5 or 10.7 you could do 10.6 which means early may the bomb starts the show

and then late may the bomb um the bomb would have to be pushed back um yeah i'm curious uh what do people think about this uh generally

um well since no one else is speaking uh i so as i see

Martin Holst Swende: this should be the last pushback

of the bomb i think it's we should not push it back too little uh and run the risk of like having the bomb disturb when we and and then interfere with the scheduling of the merge and so i think we should rather aim for like yeah towards the summer next next one when hopefully we should be done with with the

merch

Tim Beiko: got it uh andrew you have your hand up

um yeah i also like unfortunately

Andrew Ashikhmin: in aragon we haven't been able to concentrate on the nudge yet so from our perspective it probably makes more sense to have a later to to to push it to a later date yeah so something like out of the the options i would choose the longest

i guess does anyone feel like

Tim Beiko: uh does anyone have a uh uh opposition to the 10.7 which would mean around mid-may um which is kind of the farthest we we had there but like would people want to go we could obviously do like 11 million also and

that's you know every every 100 000 blocks is like very roughly two weeks um yeah are people kind of generally okay with say 10.7 or would we want to go even even further to make sure we don't have to push it back again

Péter Szilágyi: personally i think pushing back again is still the better option versus just putting it out way into the future and then right so i mean yeah sure pushing it out is is an unfavorable

thing that nobody wants to do but it's still the better option to that requires kind of developers actively doing something versus we just cannot do something because yeah my vote is to push it but don't push it too far

Tim Beiko: got it uh micah you have your hand

uh and scars first oh sorry yeah go ahead and then okay yes uh i think i

Ansgar Dietrichs: i just generally agree with peter in that um like i mean the whole point of the difficulty bump is to to have some incentive to do to try and do things as quickly as safely possible right

otherwise we could just completely remove the difficulty from and say we do the merge whenever it's ready and so if we wanted to keep it around as it is some token of our promise to do this as like as fast as as a safe merch can be then i think it only makes sense if it has like a meaningful timeline attached to it and say it only starts to come into effect in the summer i just like i mean i think last alcott's there was still we were still saying that we couldn't 100 say that we wouldn't do it this year even and now like just basically like if we already sent this by six months it just i think it becomes mostly meaningless at that point i'd prefer just removing the difficulty bump all together so my weak preference would be to to to

go with the april um timeline and then that already gives us room to to by to the end of may or something and then if we really they can't get the merge done by the end of may which i mean yeah that's definitely an option but then i feel like it's more honest in a sense to then have like a another kind of like chosen um delay instead of just yeah effectively i i think everything else just makes the difficulty bump kind of mean meaningless which is always okay but i think we should be honest about that

Micah Zoltu: so the difficulty bomb has two purposes one is to make sure that we are regularly exercising our release process

and getting things out on a regular cadence etc and the other one is to make sure that when the merge happens if you want to run a fork of ethereum you need to have a release pipeline i think pushing it out until summer addresses the any competitor or any fork needed to release pipeline it still addresses that because even if you do run your run your fork of ethereum without proof of stake it is only going gonna last you for you know three months so it doesn't do anybody any good you need a release pipeline and you get an extra three three months to get the release pipeline out but you still need it and so i think pretty much any date we choose in 2022 solves that second problem as far as the cadence problem that one i think the argument favors doing it sooner because

like against guard just said it forces us to behave on a cadence

Lightclient: um i think it also might be helpful to work backwards a little bit like when do we expect to like how much time between the final release of the execution clients do we need before the fork goes live because just saying mid april that kind of almost implies that the the merge is ready in the code basis by the beginning of march

Tim Beiko: yeah i i would plus one that like i think at the very least you kind of need a month right um and people have uh kind of complained in the past that

like where those delays could be longer um so like yeah if you take say the farthest option right now which is like which is mid-may that basically means mid-april we need like a client release right um on the execution layer

slide

Lightclient: but also for test nets like how long is each test net going to run what is our thinking on these things

Tim Beiko: yeah peter

Péter Szilágyi: so um i would argue that the merge kind of requires a much longer timeline than just a one-month thing because generally when we say that people just want to give people one month to upgrade

that upgrading is mostly just download the new code and swap out your old instances and of course for operators who run multiple instances that kind of means that okay you need to upgrade one node see if it works then just spin down another node upgrade etc etc and we essentially tell them that we expect this to take one month worth of effort so that they can find the time to do this which is a relatively simple task versus with the merge all of a sudden you need everybody to figure out how the beacon clients work to figure out how to set up one and the other to figure out how to make them communicate between each other so it's a

you're a session you're essentially all of a sudden all your beacon users that are just on the ethernet chain now need to learn how to run a test election and all the consensus people who are here and one now need to learn how to run a beacon chain client and i think you don't want to rush that

part

Tim Beiko: right yeah i i definitely agree you don't want to rush it i think one thing uh we're planning to do is also is one we start having these devnets hopefully start to get people running on the devnets before so that if they are a large infrastructure provider they don't learn about this and start kind of setting up their infrastructure when the main net releases are out but i do agree we probably want like a longer uh delay um

and there's a comment to the chat why do we have to pick a block number now so the reason for that is based on the bomb going up in december and depending on when we want to push it back we also need client releases to be available like at least kind of a couple weeks before the upgrade so um so we can probably kind of uh fork anytime in like december later december will be a bit harder because um the bomb might actually start the show and also it's the holidays which is just not a great time so that means that you know if say we wanted to fork december 1st that means you probably want client releases around november 1st

which is two weeks from now and similarly if we wanted to fork say the week after that you probably want releases kind of the second week of november which is like three weeks from now already so i think basically making that decision today means we then have a few weeks for clients to actually implement that change and write tests for it and you know make sure that we're in a good spot um whereas yeah if if yeah if we if we wait uh much longer then um yeah it it it'll just be a very last minute uh upgrade yeah uh like client you have your hand

Lightclient: it seems like there's two

parts of work that need to be done before the merge goes live and one is kind of a dynamic amount of work and that's actually finishing all of the things that need to go into a code release and there's a fixed portion of work that needs to be done to allow for people to upgrade and to upgrade all the test nets and it's not clear to me how long that fixed portion is and i think that it would be easier if we defined exactly what this fixed portion is and then we target how long we want to spend on the dynamic portion of work

right um i mean

Tim Beiko: off the top of my head you know like say you

have to merge at date x um you probably want say you you add like some buffers it means you probably want some like mainnet client releases two months before say we do twice as much as as as we usually do that means you probably also want like some test net releases a month before that um and so that's like three months before the actual merge and that means that you basically want to have the code mostly ready and then the spot where you can shift it again say like a month so that's like four months before the actual merge on magnet very

roughly

Lightclient: um yeah so that basically means if we're gonna ship in mid april or so that we need to launch these code bases by the beginning

of january which i don't think is reasonable

Tim Beiko: yeah i would agree there i think micah you you said something

oh

Micah Zoltu: i was just trolling but i do have something to say um what was the final decision on the total terminal total difficulty strategy that we're going to go with last i heard but it was slow in the air i'm guessing you guys decided at the off site like are we releasing multiple doing multiple client releases with you know first with the ttd and then another one i'm sorry first with all the code then another one with the ttd or what was the decision on

that

well i guess that's how we also did all the tests that you have the code and then you just ship the dtd for a test and see what happens if the desktop breaks then you just ship some fixes and another clip for another test and eventually you just put the ttd information

Micah Zoltu: so will all clients have some way of providing that via config or will people need to download a new version once the ttd is decided

Péter Szilágyi: i mean i guess both so at least currently what the gas does is that in for normal forms that you have

the fork block number and it's not set for mainnet but you can just we have a flag override for example we had override london where you could well the point of the flag was to postpone the fork if something goes wrong but you could also enable it if everything goes right servicing so my guess would be that we could do something similar for the ttv where it's part of the genesis pack but uh you can control it with a flag

so you could in theory just oh we already have the flag over right so the

Micah Zoltu: the concern is again maybe i'm a little out of date because i haven't

followed super closely in the last two weeks um back when you're talking about this previously the idea was we want to choose the tt the total total difficulty like a week or two before it happens because we didn't want to wait too long because we're worried about that could like the further in advance we try to guess it the more uncertainty we get and so we wanted to wait until like a week or two before before to choose the ttd and then the idea was is that we would just announce to everybody you know two weeks prior to the that date whatever that is okay this is the exact ttd um which in the the discussion was does that mean we have another release two weeks before the actual merge or do we have like config override or is there some other strategy that we're going

with and it sounds like we're going with like the a mix between new release and config override

Péter Szilágyi: well my best guess is that it could be both so 100kgps announced clients could do their releases because for most people it's a lot simpler to just update than to mess with command line tracks but i would definitely say that the command time flap should also be available because it's always a good backup to if something happens

right

Micah Zoltu: okay i just the reason i bring it up in this context of this conversation is just because it does add uh complexity to the entire merge process for integrators and so for for people infrastructure providers etc so when

they are getting their stuff up even if we release the clients two months before um there will still be chaos you know near the merge as people have to do that last minute upgrade or config change or whatever so just keep that in mind when we're right like this this the merge is going to be significantly more complicated for everybody than previous um things

yeah and it's it's yeah

Tim Beiko: i'm not sure that's something where even if we added an extra month of buffer it actually helps people because it's like you just need to do this thing close to the merge right

Micah Zoltu: um yeah it's just like in terms of messaging the stuff like the kind of the more we drag it out and spread it out it's just easier to make it clear like okay do this one thing this week and then next week you'll have another thing to do versus okay do this today do this tomorrow it's more complicated for people as well

yeah that makes sense

Lightclient: so four months seems like a pretty generous amount of time to allow people to upgrade and do all of the test nuts what is what do client teams think of reasonable targets to have the merge code ready to release

four test nuts would be

Tim Beiko: the hard question

um spec changes right yep

Tim Beiko: yeah so as i understand it you know there's still some some work on the spec that hopefully should be wrapped up by like the end of this month um then you know we basically have like all of november probably like half of december before people start going away and if we also have uh kind of the aero glacier fork in december that also eats away at kind of the time there um

so we kind of have you know one two one and a half months prior to the holidays um and then people kind of kind of uh yeah yeah then people come and we we have some more time in the

new years um hello oh

Tim Beiko: hi danny uh we're just asking when is the merge going to be done um [Laughter] no yeah maybe something live something you can maybe confirm is uh you know when do you expect kind of the specs to start settling down uh

you mentioned towards the month to me is that is that still reasonable

Danny Ryan: yeah we had a number of uh spec changes that came out of the interop most of them were actually simplifications and most of them have to do with the engine api um and we're working on those right now i'm doing most of that mikhail is on vacation this week but we'll be picking that up and the target is to be done with all those changes by the end of the month um and the general agreed upon target by client teams uh to get an a next version of like a persistent test net up would be a month following so the end of november um and then there's a lot of security and

testing work and hardening of engineering to do so then as we get into the new year after the holidays we make decisions there

oh

Micah Zoltu: we're trying to decide on the ice age which means we have to make some guesses today

Danny Ryan: right um the optimistic launch if everything goes well was end of march um and so ice age in april make sense but we could debate that i'm sure

yes i think one thing is that

Micah Zoltu: is it optimistic code complete or optimistic code complete and gone through all the test nets and a couple of months for people to infrastructure providers to upgrade and download and messaging

Danny Ryan: um no not optimistic code complete optimistic we you know do the launch at the end of march um realistic is probably

plus something what you're getting there

Micah Zoltu: what's your guess on code complete ignoring all the other stuff that comes after code complete

i um i

Danny Ryan: i'm not building the code so that's fair if we have uh if we have specs at the end of october what does co-complete look like based off of people's experience on the work that needs to be done also knowing that pretty much everything is the same

except for ensuring some some issues in the engine api fixing smash cases and things like that and i think adding the random op code which is supersedes difficulty

oh lucas

Łukasz Rozmej: so for from my side the hardest part or saying when what will be called complete ready etcetera from the implementer's point of view is all those disaster recovery edge cases which like even though there's but maybe we do know but we might miss some etcetera and that those will just come in in testing

[Music] general specs are you can more or less tell how much it might take but those those edge cases will be extremely hard to pinpoint and fix all the things there

Tim Beiko: got it um there's a couple comments in the chat about like what's worse is it you know pushing back the bombs two times or you know like what like alienates the community um my opinion there is like a bad merge is what alienates the

community by far the most uh so like that should obviously you know people will take a good merge with two difficulty bomb pushbacks over you know a bad merge with one because we had to get it out two weeks earlier um and i think we kind of saw that also with london where some people were uh a bit uh like unhappy with how uh quickly we we went to mainnet after we we found that last issue um and then you know obviously people want to merge and like they want it as quick as possible but like there's no there's no way to like expedite it beyond just doing the work and making sure that it's safe

and yeah so i i don't based on all of this it does seem like trying to get a date that's far enough in the future to give us some buffer ideally not far enough that it's like completely irrelevant um and you know that uh that we we kind of forget about the difficulty bomb um and i don't think it would be the end of the world to push back the difficulty bomb a second time from like the community point of view um where you know anyways at that point it's like if we've taken a reasonable delay for the difficulty bomb and we push it back it kind of means that uh the merge is late and people will be upset about that not about the fact that

we're pushing the difficulty bomb back um so yeah i i guess this is like a long-winded way of saying i think i kind of agree with what peter said at the very beginning that like something that's like far off but not completely far off based on this if you want like a rough um kind of a rough four months from like the code quality to like actually going on main net it seems like probably somewhere around like june-ish to have the bomb goes off makes sense which means you kind of need the code done around february-ish um and obviously if we get there and we realize that's just not going to be the case

then we push back the bomb again but like i i feel like the yeah the expectation we sell is around the merge not around like the difficulty bomb and if we have to push it again that's less worse than having to like ship the merge too quickly or something like that oh andrew

Andrew Ashikhmin: um right so i would like to say that um if we commit to this bomb delay it doesn't mean that we we are postponing the merge because as was discussed if if the bomb goes off in in in say august it's we we

can it doesn't mean that we have to wait with the marriage until until august or something like that so my preference would be to have it to move it to mid summer so something like a nice number like 11 million uh would move it to mid-summer and uh we can if we are happy with the notch and everything goes smooth with the notch we can do it earlier than mid-summer by all means so the bomb does not doesn't force us force us to do the merge late

Tim Beiko: right yeah and then 11 million would mean the bomb starts being noticeable early july basically

Andrew Ashikhmin: um

yeah sounds good

and yeah yeah i mean uh yeah go ahead i

Danny Ryan: i want to piggyback on that there certainly a week ago there was a lot of optimism as to um timelines and and so i do i don't want to making a longer wait on the bomb uh shift our expectations on like what we're trying to deliver here

Tim Beiko: yeah so i think there's like two ways like one it's either you go with say i don't think it matters if it's like late june or july and because it's kind of far in a way you call it like the generous bomb or you have like a very aggressive one that there's like a a a high chance we we push back if we

don't meet the optimistic timeline um yeah and you know like you there's kind of no use i guess debating between like mid-june and july it's like if we go in the summer we might as well do it and then the other option is like do we do kind of more like april or may right um and then assuming we don't meet kind of optimistic to realistic timelines then we'll have to push it back again one more time um yeah um and i guess yeah one thing also worth noting is uh uh right before you came danny we were talking about how like what's the delay you want between having the actual code done and um and going live on mainnet um so

ideally say you go you know live on mainnet at datex you probably want something close to two months of like having the releases out so that people can actually upgrade their clients and like run an execution client for the first time uh and and run a consensus client for the first time on the execution layer um if you do that that means you probably want to fork the test nets in like the month before and that means you probably need to put out the releases kind of a few weeks to a month before that so you have like a kind of generous four months between like the code is done and we go on main net um we might be able to do like slightly less than that but that's kind of the yeah the the buffer

um yeah buffer timeline that makes sense

um yeah

Ansgar Dietrichs: i just wanted to to uh say one more time that i i think it makes total sense to basically on the on the code complete side of course that we think there's no way we could try and rush this this just takes the time it takes but then on the i just don't think for months that doesn't seem necessary to me um like for one like the once we have the test net releases that's already at the moment where people can actually start uh like getting used to running both clients and everything everything at that point will

basically be frozen and you already know what steps to go through so i don't understand why we basically have to wait until the test has a fork before people could even start with their two months of time that we think they need so i think that's just like another extra slack month that just doesn't need to be there um and then even on the two months side as michael was saying right like if we only choose the total difficulty relatively close to the merge anyway and there's like the need for another update anyway very close to the merge maybe like even the two months uh kind of slightly don't don't really make a lot of sense so i just feel like the four months uh at least a month if not two

too much but yeah i just wanted to twist that

right and i think

Danny Ryan: also something that came up go ahead then yeah in conversations is that running certainly running a test net for a certain amount of time is very valuable um but one of the most critical elements of this is likely the transition process itself and so having nightly builds and hive and running it through the ringer the transitions of the ringer multiple times is as much you know equally of importance if not higher importance for me on um being being ready

and so um just just so i guess

Tim Beiko: just to kind of eliminate some options like are any of the client teams like comfortable with the bomb starting to show around mid april it does seem like kind of from what everyone has been saying mid april is probably too soon unless we go with like the most optimistic timeline

um and then yeah so i'll stop there like you know there's anyone kind of pushing for the bomb to go off like mid-april

Péter Szilágyi: so one thing to keep in mind is that the difficulty bomb so if essentially if we expect the merge to go

cleanly then we might also see miners dropping off earlier simply maybe you just want to get rid of your hardware i don't know so that is just an option that we might want to keep open that we might see some hash rate drop before emerge and if there is a hash rate drop then the effect of the difficulty bomb is much more pronounced so right you could actually speed it up so that's also i have absolutely no idea how to take this into consideration it's just

Tim Beiko: uh i think yeah that's true and somebody

else had mentioned that on the the awkwardest discord also um so it's almost like and and i think the the the good thing there is like if we do need to we can coordinate another push for the bomb very quickly like we did with muir glacier so like we probably don't need to assume you know like it that you know if if it started showing up it would take us months to fix um but we probably want like at least a couple weeks of like hash rate buffer in whatever number we choose so that if the hash rate drops um and we kind of move through the bomb quicker

uh yeah we're not we're not stuck um and is your hands still up or did you

forget to i'm not no sorry okay [Music] i guess definitely i'm

Danny Ryan: i'm a bit more pro putting the bomb in may rather than june or july i i think uh there was a lot of general agreement last week that uh a moderately aggressive timeline could be achieved and i i don't want to take that momentum out you know i think if we set the bomb to june or july then like i think that's the earliest that the merge is going to happen um just because that's the way things go

Lightclient: so that means

roughly code complete by the end of january beginning of february yeah that's what is that something that client teams feel is reasonable

Danny Ryan: i got a lot of head nods of something of a persistent test net at the end of november for current specifications i know that doesn't mean code complete and i know we have holidays then after so then it's a you know is it realistic to come up come back in in january and harden things such that you're feeling like the code is relatively complete at the end of january beginning of february

Gary Schulte: from from a basic perspective i think

what we're most concerned about is adversarial conditions at transition uh i think we could be code complete uh or think we're code complete fairly easily and not necessarily account for all the adversarial conditions i think that's that plays into uh our what we think is readiness more so than having code complete

Tim Beiko: right

so

Tomasz Stańczak: like possibly go well ahead it might be more about name ecosystem testing and integrations testing and users testing everything rather than us only so now maybe maybe a bit lighter bomb is reasonable already

because even even with this november test net um this would leave just four months like end of march or february would just be three or four months for everyone else to test how it behaves and seems tricky

yeah

Tim Beiko: yeah and for sure that's something we're going to be proactive about so like uh i guess starting in the next couple weeks we'll already start to reach out to like large infrastructure providers and try to get them using ketos and see what like their feedback is and you know what breaks uh when they're using it and what not um so it seems there's like proposals i i

don't know from like around april all the way to like july um and being mindful of like the the impact that like a lowering hash rate might have

um i oh sorry go ahead mika does does anybody

Micah Zoltu: um is anyone willing to die on this hill like or would everybody here be satisfied with whatever vote turns out

Gary Schulte: we can always push it further so it doesn't seem like this is a hill that needs to be died on

Danny Ryan: i'm on the edge of dying on the hill but i also am not writing the software so

i think yeah

Tim Beiko: my proposal would be to follow what tj rush proposed which is 10.7 million it means the bomb starts to show around mid-may there's a world in which the hash way drops and it starts to show around mid-april and we need to coordinate either uh uh you know pushing the bomb back or releasing the merge um and if we really you know if we're really close to the merge and whatnot we might be able to wait until to not push it back and have the merge happen in in june and still be okay um so it seems to me like a sort of average of the different opinions it's obviously

something we can push back um it's not you know and again i'll stress that like the most important thing is actually getting the merge done well and like out and no one will care about the difficulty bomb if the merge goes sporty and no one will remember that we pushed it back twice a year from now if the merge goes well does anyone feel very strongly against 10.7

how

Lightclient: how distracting would it be to have discussions about diffusing the bomb in march to merge work because if we get into a situation where we're like not super comfortable about hitting the target and we need to start

discussing like okay what is our backup plan to diffusing the bomb is this something that we're gonna be wasting time that is a really important moment of of working on the merge and we're spending all court that was talking about timelines rather than issues at hand

Danny Ryan: i think regardless that's what we're going to be discussing is timelines at that point you know are we ready are we not and and the merge being the difficulty bomb being there is is gonna force our hand in having those conversations but we're gonna have to have those conversations anyway

uh thomas yeah finger

Tomasz Stańczak: to some extent i'll repeat what danny said i think this conversation now is particularly important it gives the lots of

information to how confident teams are what are they afraid of and what their timelines are and give the insight into for the community to

to understand where we go so about this minute obviously i have a stack of emails to go through sorry mike's life yeah yeah greg you were unmuted and yeah thomas were you were you done oh it's fine it's fine i was actually yeah i finished slightly more abruptly because of greg but i think i was finishing my sentence anyways right and one thing yeah so like kind

you know

Tim Beiko: you're concerned about like spending all the time on our core devs discussing the bomb i think that's something that's also kind of easy to discuss out of band and you know for me to come back on these calls and summarize you know hey i talk to all the client teams and like half of them think this makes sense and the other half is not ready or something like that um and yeah i i i also don't want to spend like uh yeah spend several all awkward devs just discussing the difficulty bomb um and then there's a comment about having 10.8 instead of 10.7 so that it's like a clean month and we could have a

cold complete date of february first so 10.7 danny maps to mid-june uh sorry mid-may the bomb starts to show so they point less than 0.5 seconds extra per block um and so point 10.7 million means we start to see the bomb in may 10.8 million we start to see it early june again assuming a relatively constant hash rate if the hash rate drops we might see it before that um yeah and and usually it takes about a month from the bomb starting to show to the to the delay being like around a second or more

which starts to be very uh negative

uh enzgar

Ansgar Dietrichs: yeah just just just a clarification question so um because you're saying if the hasher drops then that would be earlier but my understanding is that the because if it's triggered by a block height then that would be later right because it would take longer time to get that book and then once we hit it the effects would be more severe sooner but like basically the actual the moment of start of the difficulty one would actually be further out not sooner is that right

uh if the hash rate drop well the

Tim Beiko: not necessarily because the the difficulty will readjust right to the hash rate um so like there's probably a point at

which both effects cancel each other out i'm not sure what that one is

yeah

Ansgar Dietrichs: right but the adjustments are always like they're always lagging behind so basically just this right now we have like a 14 seconds or something right roughly block time instead of 15 seconds so then we would have like a 16 seconds or something uh approaching the the merge so i don't know i don't think it makes a big difference

but i guess okay so yeah there's

Tim Beiko: some

comments in the chat between 10.7 and like 11 um do any of the client teams feel strongly that like 11 would be much better for them

okay

Andrew Ashikhmin: uh well i think foreign 11 would be better but we can live with 10.7 probably

Tim Beiko: got it so yeah just because there's like a pretty wide range of opinions here i'm kind of tempted to go in the middle with

10.7 and that means that basically in february we need to have a conversation about like how ready are we and do we think we want to push this back or do we think we're on the right track to not push it back um and you know ideally the best of worlds we obviously are ready by february um does that seem reasonable to people

Danny Ryan: i personally want to keep this train moving

Tim Beiko: great so danny will make sure we are ready by february uh so will i um cool let's do 10.7 um

i doubt we're going to get a cleaner consensus than this um and now the other kind of uh big decision we need to make is when do we actually have this upgrade so the bomb is scheduled to start showing sometime in december um we probably need a month between the client releases and the actual upgrade on mainnet uh because that means kind of probably three weeks by the time we announce them and whatnot and and and the fork happens um so maybe starting like backwards like given that we have the number to push back the bomb for today how quickly can client teams have a release out that has the bomb number and

obviously where we also have tests for for it

this

Danny Ryan: is a one week project two week project four-week project

Tim Beiko: i can start my preference so my uh as a non-client dev uh i think december 8 would probably be like the nice sweet spot where like we're not just rushing all this in the next two weeks uh we have like an extra week of buffer um and that means we basically we need

releases out uh yeah three weeks from now roughly um and the fourth block for that would be one three seven seven three thousand uh so 13.773 million does anyone disagree with that one uh thomas sanskar i don't know if your hand is still up from last time or if you you want to add something okay last chance for this agreement otherwise

we're going for 10.7 million uh delay to the bomb and having aero glacier go live on block 13 million seven hundred thousand seventy three thousand seven hundred and seventy three thousand um which should be midweek uh on the on december eight uh but obviously these things are hard to uh to predict cool that was easy

um those were the two big things on the agenda today as the last small thing was uh eip3607 uh on the last call we said uh

we wanted to move it the final uh but we wanted to wait to make sure that all the clients had implemented it uh so i don't know do any client teams has a have an update for this or against the question any client team have a issue with moving the eip to final sorry which here what's that 3607 this is the this is the the one with like the reject transactions from senders if there's already called deployed at the address right no objections from me

andrew anything else you wanted to add on that or we're just good to move at the front

Andrew Ashikhmin: no we have implemented it in evergone and we are cool with it

sweet

Marius van der Wijden: um also there should be some tests merged pretty soonish that will require it in the clients so if you haven't uh implemented it in your client and the tests uh break in the next in the next test release then you know that you should do it it's it's three lines in geth code

so

Tim Beiko: okay um yeah that's all i had on the

agenda um anything else anybody wanted to bring you up

Danny Ryan: i would just say sorry i was 30 minutes late um but if there was any additional questions on merge interop updates i'm happy to answer those but you can also take it offline

Yuga Cohler: um one quick question about the december upgrade so obviously it's a small upgrade just changing the difficulty bomb block um i know there are still like 20 of open ethereum nodes operating and open ethereum's been deprecated do you think that open ethereum would be upgraded for this purpose or is the expectation that uh anyone depending on open ethereum would switch by then

Marius van der Wijden: the expectation is that you should have already switched by now but i think it's it's it's not a big change to update on ethereum and i think someone from the community will probably do it but that shouldn't mean that you should still run open ethereum you should have switched by now

Tim Beiko: yeah i wouldn't be surprised if somebody just made the pr to open ethereum um but the client is no longer officially maintained so um yeah i and i don't think there's anyone on the call who's still a maintainer of open ethereum who can like give more

cutter um yeah and we can ask in awkward as after if if there is someone who's like willing to to do that but yeah generally yeah the client doesn't have a a full-time team working on it anymore got it thanks cool any other questions comments okay well yeah i guess that was it then

thanks everybody and i'll make a pr to the eip for the for aero glacier the eip for the difficulty bomb and the aero glacier spec uh today so that we have the updated numbers in them um yeah thanks a lot for

thank you thank you thank you thank you thank you [Music] [Music]

[Music] [Music] [Music]

[Applause] [Music] [Music] so

so [Music] [Music] [Music] so [Music] [Music]

[Music] [Music] [Music] [Music] you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.