All Core Devs - Consensus #67

Transcript

[Music] [Music] [Applause] [Music] [Music]

[Music] [Music] okay

welcome to the call

Terence Tsao: call number 67. um i'll tear focus obviously let's do uh client updates with focus on altair status then we will discuss a bit about altair planning and then general research updates and open discussion from there today let's go ahead and get started with

prism hey guys um so terence here we have been mostly working on a tear so we aligned with the latest bag we implemented all the beacon node and the validator interactions

we are almost done with the networking side of things but we have started interrupt internally already so that is pretty much working there's a few bugs to iron out here and there so we very soon will be interrupt with the tattoo um chant that the adrian set up so uh thank you for that so yeah that's pretty much it and other than that just 15 uh phase zero magnetic bars here and there but yeah most of our efforts are on the uh uh front these days okay thank you terence lighthouse hey everyone similarly to prism

Adrian Manning: we've

been working on alte um so i think we've recently finished all the validator client features um we're increasing testing at fork boundaries and eyeing out bugs in in the vc there we've done all the alpha eight altair changes so they've all been implemented um so we have we've been compatible with tekker which has been confirmed on the on the devnet that teku set up so we're ready for multi-client altair test net when when all the other clients are ready to do that um we've also been upgrading a number of uh some of our sub networking components so discovery and gossip sub i've got a big networking refactor which should come out in our next release which will be 1.5

there's a fair amount of testing to do on that still that's pretty much it from us

great thanks age taku

Adrian Sutton: uh yeah we're ready for for altar as well uh we actually haven't had too much more to do on that uh the latest gossip changes from alpha eight are in and seem to be working pretty well um our big stuff has been uh we've added in batch signature verification for outer station gossip which has reduced cpu usage quite a lot um so that's in the 21.6.1 release that was out yesterday the day before um uh we've also enabled i think i

mentioned it last call the the time-based e1 head tracking which just aligns us with the spec correctly um it didn't doesn't make much difference on net because there are 2048 blocks in the time allotted anyway the blocks come faster than we expected um [Music] sorry again um uh we've been doing a bunch of work on discovery um thanks age for pointing out that we weren't weren't filling our buckets as well as we could there

so there's been been a few improvements there we're kind of continuing testing that it'll take a little while to roll out but that's coming and we're finally actually going to switch over to using the dependent routes that we added to the standard api um we've taken quite a while to work out why there was a very small reduction in rewards earned for them um and it came down to the particular timing of when the full fork choice was run and making sure that because the beacon node and validator client were not as much in sync you've got to make sure that that ties together well um so it's there this ben actually wrote some code which was nice and it's paging of the g2 check so that's another

performance improvement and there's been it's actually the rta work we have been doing is uh getting web3 signer ready for that so the external sign is underway i think it should be pretty close to ready to support out there um yeah so the one bit of sad news i'll take a moment to thank meredith she's actually moved on from our team she's off to be a cto uh which is hard to argue with really it's really a great opportunity for her um but that that leaves a hole for us and meredith has been a huge huge asset for our team and done a lot to push things forward so i want to thank her

um and uh we're hiring

if anyone's interested

Danny Ryan: thank you and uh thank you meredith good luck on the next journey lodestar

Cayman Nava: hey everybody on the altair front uh we're on alpha 7 and we are updating to alpha 8 right now um at least on the alpha 7 i think everything is implemented and looking stable on our devnets um other than that we've uh not not alter related we

updated our logo and branding refreshed that it looks pretty nice i think and we've also made a benchmark tracking tool for our ci to track benchmarks across commits and tell us when we have regressions in things it's been really helpful even so just a few days we've had it so far also adding adding more metrics database metrics and four choice metrics to try to just get a better handle of what's actually going on and we've improved uh improved performance uh in our gossip attestation validation

it's not the not the batching the signatures we're gonna look into that next but it's just fixing up some stupid things we're doing uh the last thing is we're uh trying to get together a little demo of our the light client work that we have been working that we've done so far or figuring out the like client initialization and how to cache and serve the proof as necessary to like initialize the light client with the sync committees and genesis information to kind of verify the rest of the rest of everything and that's it for us

nice thank you cayman you know the uh

regressions and performance in ci that's that's a good idea numbers hi um so regarding altair

Mamy Ratsimbazafy: we are passing the alpha 8 tests we are updating uh or uh fuzzing interface so that uh my prime can plug nimbus into their fuzzing framework with the altair changes uh otherwise we released uh ten days ago nimbus 1.4 so it has a couple of improvements regarding attestation effectiveness and cpu usage

mostly coming from improvement in database and caches and we have also updated or added a rest api endpoints and fixed a false positive on the pergola detection when we restart nimbus fast and that's it for us

Danny Ryan: got it thank you and salias has joined us who um is on the team that just dropped the client what is the name grand dean uh sally's do you want to just do a quick intro and let us know what's going on there

Saulius Grigaitis: yeah hi everyone this is soulless from lithuania lithium so yeah it was a bit error start with the binary release that we have now i hope that people tried it a bit and if you have some feedback then let us know well the key things as you probably see in the readme that we have some some cross platform support for for currently for windows and linux and macos so we actually test on linux only and if

you find the if you if you test on other platforms uh there are high chance that something doesn't work really so please report if you uh if you you have some time to test it and check it and we have some some other things that there is a basic uh user interface uh which was very helpful for me personally especially for debugging the what is at the stator's performance so it's uh nice to have feature maybe for somebody but for me was very helpful so you let me try to to check that too so probably that's

all for now as you probably know it's at the moment it's only binary version but but we are looking for ways to uh to open source it yeah so if you have any question feel free to ask it now cool thank you

a

Danny Ryan: quick update on testing before we do altair planning um we are going through altair tests with a fine tooth comb to make sure that we have um complete coverage uh shall we is also looking to continue to enhance the

fork choice test there's an open pr we're doing some review on right now so generally we might see some like small non-feature modification releases that just continue to enhance testing and someone asked on the ethernet discord if when we move to beta i think once we have all clients interrupting on a devnet we can call uh we can call the subsequent release of beta until then still some blind spots i'd say um cool is there any other testing updates

might eat anything on the beacon fuzz front you want to share

Mehdi Zerouali: um not really not much that i can share really we've been reaching out to client

teams that um for which we've identified uh potential vulnerabilities and um yeah got some good feedback there um things are going quite well we will be kicking off the actual differential part of the fuzzing effort um in the next couple of days really we're just waiting for most clients to stabilize and be up to uh alpha seven alpha eight which uh has been the case for the last few days um you can definitely see that the consensus code has been a lot more stable um yeah going well hoping to get a bunch more uh fuzzing cycles before we start forking uh test nets but i'm hopeful that we'll be able to fit those in

Danny Ryan: great any other testing related updates before

we move on okay uh planning we are on the cusp um i know everyone's kind of really just getting the pieces together um and the final pieces of altair together thank you adrian again for standing up that small devnet i think that's been really useful um i think the next two major things to do are to do a probably slightly larger more coordinated devnet i and the other would be to probably fork

piermont um to give us an idea of working one of these test nets in production uh peer knot is slated for to be moonlit after our sunset i don't know after the altair fork um and so if we break it it's not too too bad um my current thoughts on both of those and i will ultimately defer to your judgment on timing is to do a coordinated small but more distributed in terms of validator count and node count uh devnet at the end of next week and perry can can help us give us a hand with that and then set a uh a fork date for for pyrmont

in that like three to four week time horizon um and then that we can use that as a target to really iron out and get things ready for that and then the success of that we can then uh define a pratter and and mainnet date um if that of course is successful is that all saying in terms of timelines or are we jumping the gun i'd love some feedback

here

Saulius Grigaitis: so you would like to to fork pyrmont first

yes piermont's the smaller one correct

Saulius Grigaitis: okay this is great as uh us as we run

only a few all the latest validators of payment and we are not ready for altera yeah so this will be very good for us

yeah so pyramid

Danny Ryan: is um i think the last thing we will do with pyramid other than maybe some like leave some nodes on for a really long time and see what happens even things are not finalizing uh the last thing we want to do is is do an altair fork um if it breaks if we kill it that's that's fine because it is planning on dying uh so that'll be the fourth the first one and kind of dictate i think uh prouder and mainnet depending on it goes

so

Adrian Manning: so that's your plan for the the first

multi-client test net um for altair on to do it on

pm1

Danny Ryan: no so in i suggest that we do a smaller devnet in a week's time yeah okay um but to set a date today for piermont to keep things rolling

Mehdi Zerouali: my take would be to wait another two weeks before setting it down on pmo just to see how the clients behave on the actual devnet

yeah fair

Danny Ryan: any other input on that one

Adrian Sutton: and it sounds like go ahead

i guess it sounds like most clients have have tried to be sinking euron kelly so we've got a pretty good chance of this working um so i'd be inclined to to set a fork date for pm on fairly optimistically and we should be okay it's only

payment same here

Danny Ryan: okay

Jacek Sieka: so i mean what we can do is set a tentative date yeah and then as long as it's after the next meeting during the next meeting we can confirm it

Danny Ryan: how what percentage of the network do we control on piermont is it

Jacek Sieka: it's over two-thirds still yeah significantly yeah okay so even if we give people not that great of lead time the network can can still carry itself

Danny Ryan: um okay so i would i would suggest that we do the pyrmont fork pretty much three weeks from today and we will do a final confirmation we'll do a confirmation of that two weeks from today and pick a precise epoch number assuming that devnet or devnets have gone well between now and then

Mehdi Zerouali: okay um from the conversation that we've had with the team i again do think it's too short

Danny Ryan: too short because

Mehdi Zerouali: because it'd be great to just have one they've net with all four clients first and see how that behaves and set up a a bit later on uh i i anticipate a fair bit of issues once we have all four clients on the same network but you know that's just me might be wrong

Danny Ryan: fair but given that we plan i i think having a having a

target week will keep things moving and so the next in two weeks we can kind of confirm that um i think that if we fail to get a devnet standing up or we have many issues then in two weeks time we'll say you know what uh pyrmont needs to be pushed back this amount of time um i just i do think that we need to at least have some targets in place um not that we're cutting epoch numbers into releases currently and that target

might be aggressive in here

Jacek Sieka: yeah sorry so one thing to consider is that fcc is coming up uh that's in three weeks time

exactly

let's see is that the 22nd

Jacek Sieka: yeah i think 20 to 20 seconds right that might not be the best week for upgrading per month okay so let's do this so i would aim for a week after that which which helps both uh you have time to to rehearse this and also avoid uh forking when people are pregnant okay

Danny Ryan: so let's say this i think that we should target doing the pyramid

upgrade at the end of by the end of this month target and that the devnets over the next two weeks will inform whether that target is actually realistic and then on the 15th on this call we can take some time to discuss what's happened in the prior two weeks and reassess that target if not nail it down

Mehdi Zerouali: that sounds very reasonable

okay

Danny Ryan: great um and what happens over the course of this month will inform what we do with um pratter and mainnet um

obviously i think there is a lot of testing to do there's a lot of fuzzing to do it's a lot of ironing out to do i don't want to do this too quickly but i do want to begin to to kind of move into that production version of altair okay um [Music] perry proto and i will discuss and propose like a configuration for a devnet launch at the end of next week we will propose it soon for the end of next week that's all i had on on altair planning

um we can just open the discussion for if anyone has any other points to discuss with respect to altair planning alter testing altair features or other issues that we want to bring up anything on altair today

Saulius Grigaitis: maybe a question for me is so this is the first um if it is hot fork and maybe teams could share their like biggest lessons that they learn in terms of older organization did you figure out something

you know something extremely valuable that you would like to share

Danny Ryan: and if not here we can take it to some async conversation

eath r d adrian did you have anything to share yeah i was just gonna say i think it wound up being different for each client because different languages had different capabilities so yeah boiled down to that a lot

Adrian Sutton: yeah um we leveraged what what we've done in base a lot so yeah the the the thing that thing that

Saulius Grigaitis: we are exploring now is kind of how completely separated uh runtimes actually not completed but with some isolation between it so as as it looks like uh next hot forex may be more complex and it's it's always a bit complicated to support to branch code basically and support the old version and basically support all the forks there and there was just thinking maybe somebody else also had this idea uh to to try to do uh something like a different uh almost

separate runtimes for each uh fork that lives uh together uh as long as it's needed and later switched to the new uh fork uh i'm just sort of wondering was this idea explored and if somebody explored it maybe maybe you already know that it doesn't work

Jacek Sieka: i think what most of us have done in general is actually look at the code of the other clients and sort of judge that for ourselves i mean we've learned a lot from

all the other clients really um looking at the prs that implement particular features and those vr's represent what works in different languages and designs right that's probably the best source to be looking uh which is nice with the open source model that we're using

Saulius Grigaitis: okay thanks

Adrian Manning: we haven't considered running or having two different run times and switching over i think the the fork changes allow you to um i guess seamlessly move through the fork

on a single runtime so i think that's the pathway going

Danny Ryan: yeah and i haven't seen that done on the eth1 side either although i've i've contemplated it before but i'm not a client engineer

we just do

Adrian Sutton: everything with range by abstraction so it just kind of fits naturally with the way we develop anyway

Micah Zoltu: okay thanks just slight tension but uh isn't there a bug that was introduced recently because it wasn't a consensus failure because everybody copied the code rather than the spec as we've discussed and i think 82 and

the r d discord earlier

today

Danny Ryan: that was a um an edge case on the fork choice implementation yes

yeah so just a comment on like one of the big things with the theory that makes it great is the multi-client idea and everybody building a client independently you're more likely to notice when someone introduces a bug if everybody is just copying you know pyramid or whatever influence thing first we lose some of that benefit because we don't have independent implementations anymore i'm just going to think about i mean obviously the strategy that actually gets code written is the best one um just keep that in mind but

Saulius Grigaitis: do you apply this to my question

or this is a that was a different

uh no sorry not really your question this was one of the answers just think about it oh

Saulius Grigaitis: okay i mean this this if if it's actually relates somehow to my question then i mean these uh these runtimes let's say let's just call this separate runtimes i don't think this introduces any more copying because if you have one uh runtime then you then for the code that is no different between the hot warps then you still have the same code or this is a for copied code in case of uh of the runtimes

of two separate runtimes so yeah just my note yeah

Danny Ryan: and i certainly see the desired relax and you can keep your single kind of code base relatively clean and just refer to old releases i think some issues you might run into are cross fork boundary api calls um and other types of things like that uh where you ultimately would probably need some sort of like meta orchestration between run times that are maybe running at the same time in perpetuity

Saulius Grigaitis: um yeah yeah they've definitely would would be some type of routers or something like this which uh

which is common for uh for these uh multiple

runtimes

Danny Ryan: yeah yeah i mean i'd i'd be really interested to see uh that method of forking explored

Saulius Grigaitis: okay cool

Danny Ryan: but i cannot claim if there are any critical pitfalls that you're gonna run into

Saulius Grigaitis: yeah i understand that this is a experiment that may lead to just uh wasted resources would be some okay

i think it'd

Micah Zoltu: be great to like write a blog post whether it works or not um i've also thought about that model and wondered why no one did it

Danny Ryan: okay thank you um anything

other altair related okay move on to research updates we had the emerge call uh an hour prior to this call that has been recorded and will be shared i'm sure relatively soon the um i think in terms of things that are related to this call there will be the merge code will be rebased on altair sometime in the coming weeks now that altair is stable and they're beginning to be robust implementations there and the execution payload i believe needs to be modified

partially with the um for london so both of those changes will happen sometime the next month we will also be continually expanding test vectors on the merge and figuring out how to just better test it in general those are the probably relevant merge updates here um again check out the call check out the notes if you want to dig in deeper to what happened there are there any other research updates that people like to share today

um just a comment on the metrics centralization uh we finally got some set of standard

of metrics uh to standardize across all the clients i think pari has already pr ready to to be merged and yeah so thank you to all the people that work on this and this could be that's just the first set of metrics and then we are after we implement this uh we plan to move on to into different ones that are not um implemented by all clients at this point um but then we will go on those slowly later that's all

Danny Ryan: thank you and remind me leo where where is this standardization happening which repo

uh party uh did you have the

Parithosh Jayanthi: yeah it's on the each 2.0 metrics repo i think i think i still need access to push

there

Danny Ryan: okay cool i can do that right after the call can you drop a link in the chat just for a refresher sure thanks cool any other research updates any other updates at all okay uh general open discussion anything spec related closing remarks anything on anyone's mind

i

Adrian Sutton: put up a pr for to change the default vote for for each one data that the honest validator spec sets um currently it says to to use the eth1 data from the state which i think most clients don't actually do and it causes problems because it leads everyone to vote for no change basically when you waste a voting period so the two options are basically either vote for i thank you um either vote for a random block um which means you can't tell whether you just didn't have that block or whether that's just they didn't have any blocks or each one note at all

i've suggested setting it to zero so it becomes very obvious uh but it does then mean that we've got to add a check in to the state transition to make sure that we don't ever actually vote that in if nobody has e1 data um we can do it as a soft fork but that's slightly more complex than i was kind of hoping for so it'd be good to get people's thoughts on that i could go either way to be honest i do find it very useful to be able to tell the difference between unknown block and

and not voting basically

Danny Ryan: gotcha yeah i saw that uh right when i woke up this morning but haven't had a chance to dig in

um i'm happy to review but i don't have much to comment on before i take a deeper look

Adrian Sutton: yeah yeah i don't expect there'll be much to say now but it'd be good to get people's important thoughts on it um and it would be a soft i'm just trying to push it back it's been hanging around a little while

that

Danny Ryan: conditional could be a soft work because we haven't actually ever had a voting period that votes for zeros

yes we should double check that but i'm pretty sure that's true it would be a very old thing pretty sure that's true as well

Danny Ryan: okay well please review um adrian's pr and let's have some conversation there

okay uh anything else today before we close

uh sorry you mentioned it's useful um what's it useful for for

Adrian Sutton: debugging um so for instance we have always had a bug in the way we track each one head and so which is the latest block we'll consider and you can't tell because if we don't have the last few blocks in a period um it just looks like the other node didn't know what they were talking about um so it's really useful to be able to have a metric that says hey we have this number of votes that we just don't recognize they look like invalid and that probably should be zero normally um

versus we have this many default votes you just can't distinguish them at the moment so you can't tell whether your node is is actually seeing the same equine train or whether there's just a lot of people out there with no

ethernet chat

Danny Ryan: is not a big deal is voting on the previous eth1 data does that give just as much information and the problem is people just don't follow that rule or is there a distinct value in voting zero rather than just voting status quo oh you said uh so too many people voting status quo at the beginning can lead to entrenchment of status quo such that it is

um it

Adrian Sutton: yeah so it's so with the way the spec is currently

the status quo is considered a valid vote um so if the first block in the voting period doesn't have an e1 chain everyone winds up following that and votes for no change which is kind of the problem we've seen in a couple of cases the other issue is that you lose visibility of it once you get a block actually has 50 of the votes because it then becomes the current state even in the middle of the period and so then everyone is just voting for that which is you know fine it's not a big deal uh but it it just adds a little complication there in terms of you can't rule out you can't say don't vote for the default

but you also need to if that makes sense like at the start of the period you need to save don't vote for the default but once it's got 50 of the votes you should keep voting for the default right

does

Danny Ryan: one single vote for status quo at the beginning of the period and i i need to go look at the logic here um does that and does all honest votes after that would that be to vote the status quo or is that just something lazy that's happening

no

Adrian Sutton: that should be what the spec says to be to the honest votes would follow that because it's now the majority vote and it is a valid candidate so you'd follow is it a valid candidate

Danny Ryan: because it's

within that the the period the the period that's yes i think so okay but if the period was restricted by half or something then it would no longer be a valid vote unless there were no votes unless there were no blocks

to vote

Adrian Sutton: yeah okay i mean that that probably is the simple thing of

Danny Ryan: make sure status quo is no longer in voting period i'm just thinking out loud logic

Adrian Sutton: yeah yeah i mean if there's a simple solution i'd feel for it

Danny Ryan: yeah yeah the fact that there's overlap makes it unclear if somebody is voting nothing or voting

for status quo but if there were no overlap that might that might help let's uh i need to look at the logic

sorry

Jacek Sieka: one more question then what if a majority votes for zero i don't know during and he's one yeah that

Adrian Sutton: that was what i realized when i was writing up the pr this evening so that's why there is a soft fork required to actually in the state transition ignore that case so you never set the e1 data in state to zero basically and it's a it's a very annoying

complication but i was i didn't didn't know about when i was thinking this is the perfect solution and it's it's not a perfect solution anymore

right

Danny Ryan: that's i think that's why at least status quo was used as kind of the default there because it's not dangerous if it you know because that just means no up on on deposit processing but um okay

Adrian Sutton: i yeah i i think i'd lean towards random as the other option it would kind of be my second pick and it's very much on par and it's been working because i think the majority of clients actually do that

Jacek Sieka: now i mean you could do random with you know if you bite zero if you really want to get down to it

yeah or even just setting the deposit count to zero deliberately might be a way to pick it rather than just doing an off block cache as well anyway i think i've known snapped enough we can take your eyes offline thanks adrian and anything else for today

Pooja Ranjan: uh i just want to share something last week we recorded an episode on take with adrian and in the past month we did a couple of more um episodes on ulta special uh here is the playlist for all the be contained proposals these are good uh like uh introductory material for altair for people who are trying to understand

Danny Ryan: great thank you pooja and i will drop that in the youtube chat oops

okay cool anything else great thank you we will uh coordinate over this coming week on that devnet and keep things moving thank you everyone talk to you

all soon thanks so much guys thank you hi everyone thanks everyone thanks man hey everyone [Music]

[Music] [Music] you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.