All Core Devs - Consensus #63

Transcript

[Music] so [Music] [Music] so

[Music] [Music]

okay great string to be transferred over this is the e2

Danny Ryan: or consensus layer proof of stakes necessary call number 63. the agenda is issue 20 217. nothing crazy today client updates i put a position in the agenda for uh discussion in the incident if we have anything else to discuss i think we've discussed this a ton offline and um there's a lot of like public updates and discussions around it um but if anybody hasn't had a place there altair uh general engineering progress uh spec in testing planning we're going to research updates spec discussion and any closing remarks

um just a reminder we can talk about uh some merch stuff here but there's super active discussion going on in the rainism chat and uh there are still merge specific calls on the opposite week of this so we won't go too deep but you know if there's interesting stuff to discuss we can touch on it okay we will just go ahead and get started then on client updates if you can give us definitely a picture of where you stand on on some of the merge progress and uh more importantly for this call where you stand on altair that'd be great starting with lodestar

Cayman Nava: hey uh so as far as altair i'm still working on it i think let's see so we've added the new gossip and recresp methods um and we've also been working on the light client side uh specifically now we're able to generate sync objects and consume them and generate to consume state proofs i think the big open item for all the altair like being able to run a test net locally or anything is uh just um updating the validator interaction

and other than that uh just generally we've been adding more metrics to leadstar it's really i've been helpful um and updating our microfauna dashboard and that's it for us

Danny Ryan: great and in terms of the validator interactions it's primarily additive as long as you have the data structures correct and that i guess a block producer could do the same role and just not be paying attention to those um sync committee and similarly someone could just not pay attention to their security so you might even be able to stand up a testament today um but great progress

let's move on to

lighthouse

Paul Hauner: hello paul here um so when it comes to altair we have our consensus changes awaiting review um we're adding some caching to our beacon chain for sync committee so getting the the fast verification of that down pat uh and we're also going we've got the network protocols under review then when it comes to rainism we're just passing the merged test sectors today um generally we're aiming towards at the 1.4.0 release it's probably going gonna be the next few weeks but it rolls in a bunch of features that we've been working on um including beta window support um big reduction memory footprint uh

doppelganger service uh we're reducing our outbound f1 calls and it will also have the altair um structure definitions and the the uh the mechanics to to choose between the two forks that's also going to be included in that that big release so it's coming in a few weeks um we're also planning to share some validated performance stats it's not uh comparing clients but it's you can compare a group of validators just to the global average for some span of time i'm just trying to um try and get the the broader staking community to be a little bit more aware of the details of how their annotation rewards go so they can come back to their clients with more info um and michael also has a pr open on

spec repo um for lsat balance carryover oh and that validated performance thing i'll i'll release it on like reddit or twitter or something at some point soon just a little spreadsheet uh that's it for me

Danny Ryan: yeah cool we can talk about michael's issue in the altar section thank you nimbus

Mamy Ratsimbazafy: hi um so last monday we released uh 1. 2.2 it includes more efficient queuing which kind of reduced power consumption we also have a stabler a more stable rest api we had issues with i think it was rocket pool that

made large queries and sometimes instructed the beacon node for a long time and we also fixed some b2b timer issues that caused us to give scoring penalties to peers that were actually good on the research side [Music] we didn't make much progress on altair in april it was de-prioritized so that we could advance on reaganism but this would be a priority starting from may and we have also merged some pr that we had pending for over a month regarding improved

slash in protection performance so that we knew that everyone updated to our v2 of the session database to prevent issues if they have to roll back on the social media side and documentation we have uh improved our nimbus guide in a lot of areas uh following feedbacks from uh especially a rocket pool users and we have a blog post called if two is green which made headlines on twitter especially and was well received also on a lot of discord

and that was with gaff plus numbers plus a rocket pool running 10 validators so for 10 hours on a raspberry pi that was powered by an external phone battery

Danny Ryan: nice great work thank you

prism hey guys parents here so

Terence Tsao: um we finally merged the optimized slasher implementation and that will go into a release in about a week and a half and uh a terrorist fat test is finally passing and uh now we're working on optimization of sync committee like paul said

also working on networking and the validator to be connect interactions and um so we participate in the merge uh the the testnet unfortunately we had some consensus error and that was confirmed using the merge spread test uh product put out yesterday so uh thank you for that so uh we failed at the uh execution payloads uh transactions and we're looking into the hashtag implementation of that i've also started working on implementing shardings back i'll be asking also questions to protocol at the and the um and turn on that and uh yeah that's it from us thank you

Danny Ryan: great yeah i'm in retrospect it was crazy that

we tried to do that devnet without consensus vectors uh i'm surprised at one of those it did uh great thank you

and

hey guys um so we're pretty much caught up with the latest alpha spec release with respect to altair um reference tests are passing except for a few altair state upgrade test cases that we need to debug um all the sync committee related validator duties have been implemented we integrated the altair state upgrade logic so that we can transition across fort boundaries we added logic to update the enr fork id field at fork boundaries uh we have sync committee subnet

subscription updates wired through to update the new in our sync nets field this isn't in an official release but we went ahead and implemented the version two of the get metadata rpc request which adds a new sync nets field and we're continuing to migrate jim mcdonald's proposed sync committee apis to the standard spec and implementing those within tekku as we go and um we also had a community contribution for a custom rest endpoint to query peer gossips course and that's it for me

Danny Ryan: great thank you uh next up two weekends ago we had the incident seventy percent of

uh block proposals went offline i don't know if there's anything discussed here just leaving room and kiss this okay uh if you're listening in uh prism has a great uh incident report and a number of others have been discussing this and you can probably easily find a lot of that thank you uh and thank you for everyone that worked their ass off through the weekend

on that one altair

Danny Ryan: uh so we got general engineering progress seems like things move forward not too much issue um i did want to have another pre-release out

now-ish i actually had eye surgery one week ago and was uh a little more optimistic and how much i was going to be able to read and do computer work in the past week but many people from my team have stepped up there's a lot of prs out for kind of not a final release but um a wave of kind of cleanups and testing primarily

a lot of

Danny Ryan: a lot of testing to be added um so we are aiming to get a lot of that review done today and try to get respect um pre-release out tomorrow there is this item that michael proposed um that before he put this out we were essentially dropping an epoch's worth of

attestation participation which as michael pointed out as others agree this would probably not look and feel good from the perspective of validators losing rewards on that one epoch and so there's the simple mitigation might be just give a plus a tiny bit of reward uh but the slightly more complicated but not very complicated method which michael proposed um is to translate the current epoch out to state pending stations into flags at that epoch crossing um the one major not major but the primary downside to this is

just more testing needs to happen on that fork boundary but looking at his proposed changes i don't personally think it's extremely complicated i think we could get a reasonable amount of test factors in there without too much issue but while we enhance the testing the next couple days on that pr i would suggest if you haven't taken a look in your client team to take a look at that does anybody have any thoughts and opinions on that or should we take this async into that issue

Vitalik Buterin: all right so this would um only basic affect

like attestations created in the last pre-4 key box that get included in the first post for epoch right

Danny Ryan: well no it's um it's pending attestations in state

Vitalik Buterin: so if nothing gets done then like there will just be one e-box that gets

unincentivized there'll

Danny Ryan: be one e-book of it will look like essentially empty uh participation and a minor reward a minor uh drop in potential

intuitively it doesn't seem worth the effort to even try solving that problem right because that's like one e-book um that was my gut michael does have a pr up that reuses the functionality of uh prosthetization in a modular way so that it's primarily just for using functionality my yet is that it does introduce consensus complexity and a lot of potential for error at that fork boundary which is like this single instance but open for discussion i don't feel super strongly one radio i mean i guess from the first from like

the financial analysis it's probably a very small amount of money one way or the other and so but a reasonable amount of dev time so that's kind of a funny way to look at it but there's also it's definitely probably more correct in the sense to just do differentiation

Dankrad Feist: yeah i mean it just like one one epoch rewards are like so tiny from another perspective i yeah i would also tend towards saying it's not worth introducing any complexity for this

i think michael's the main consent

there's

Paul Hauner: also i guess some complexity in

um breaking the you know the variant that you know if you get your intestation included in blocks in the chain then you get rewarded um and and that these you know um participation metrics stay the same so you know i guess there's a there's a there's a complexity that lives there too if we choose not to address it in the spec

Dankrad Feist: so what complexity is that you're saying like there's a complexity because um because the there are no rewards like but even if they wear like one pock where nobody participated um because there are no rewards for it that wouldn't do very much right

yeah

Paul Hauner: yeah i'm not saying that the financial

um difference is is a lot but it's just you know it's it's just a thing where the system doesn't it just has this epoch where you include your attestations and you don't get rewarded

um and it

Dankrad Feist: just doesn't tell me where where where does that introduce complexity i don't see that it changes an invariant

Paul Hauner: uh yeah that's right users expect

Danny Ryan: it could potentially introduce complexity in third-party tooling that is making some sort of assumption based off of that uh but i i would imagine block explorer

Dankrad Feist: well i don't i don't feel like that's true because there can be epochs with no rewards

like that's right

Vitalik Buterin: that never really wasn't any variance because there was always the possibility that like network

latency would just temporarily increase the four minutes

Danny Ryan: okay right i mean in very like you actually get your things on change but doctor you can imagine and i i'm not i'm just pointing down there

i can't

Dankrad Feist: i cannot imagine just cannot imagine a tool where this introduces complexity that's what i'm saying

Danny Ryan: i could imagine a block explorer that uses on-chain attestations to cap to display granular rewards and that getting out of sync with what was actually given uh because what is given is very is is bulk and so but that's that's not necessary

Dankrad Feist: but i mean that but that explorer would have to use so my my assumption is that there's

something broken just in the way we compute rewards but i mean there that explorer has to do something to like uh over the fork boundary anyway

so yeah

Mehdi Zerouali: i think one of michael's point sorry one of michael's point i think was that a lot of validators will actually be watching uh this quite closely that particular epoch quite closely and monitoring their rewards and noticing that they're not actually getting any might you know trigger a whole heap of uh users complaining on our discord channels and uh probably

Dankrad Feist: but i mean are we serious i mean we're talking about one one epoch that is like cents how much do you get in one epoch

Danny Ryan: but doctor there is there is clearly demonstrated psychology

that the others care a lot if they have one epoch of red

Dankrad Feist: yes i agree but they are much much worse things i think the issue i think yeah we all agree on i think there's like there's a midge here that is blown into an elephant no i agree but there are like 1 000 times worse problems with that that

Danny Ryan: we haven't seen i agree maybe maybe a hundred thousand maybe

Dankrad Feist: no no no i mean like with with rewards being with rewards being erratic we have like 1 000 times worse problems than just like one

Micah Zoltu: i think the issue is more that if someone sees something happen that's unexpected

they trigger a retrospective they trigger a dev time they start investigating and if they're not prepared to for this to happen they may devote you know hundreds of hours of engineering effort to try to figure out why did they miss the cpoc and it turns out just because it's a little bug that was never bothered to be fixed they didn't know about um it can be that i think that's real the real issue is had nothing to do with the money has to do with the when you see something go wrong you research

Dankrad Feist: right and and you believe that that time will exceed the time that we will spend on fixing

that that's the right the right question to ask

Danny Ryan: i think this really is that that seems a question of complexity and uh there are four or five teams here that are going to implement that complexity and

ultimately i'd like to hear more input from the various teams it seems like the lighthouse team is keen on adding

this amount of complexity for it but i'd like to hear what other people say i'm

Paul Hauner: not sure that any of the lighthouse castles down the hill for fyi um but we still stand by what we said i think

Danny Ryan: this is a one-off writer it's not going to be useful for future forks this is just a purely solution for this one right yeah yeah my gut feel is i would rather not do the work but um no stronger opinion on it

Jacek Sieka: yeah i think no strong opinion from us

either nimbus um i mean we will have to maintain the boundary code between the two fourths for a while anyway thinking and so on so the first time we'll be able to remove any of this complexity would be the next work potentially but

Danny Ryan: i think the big risk is not the complexity being sitting in the code but the testing that we have to do to ensure we don't accidentally get a consensus split through this like one-off complexity there

Jacek Sieka: isn't that kind of issue the same though i mean one behavior or the other it's a behavior the behavior

Danny Ryan: uh the current behavior is

extremely simple it's wipe an array replace it with a different array this is like a translation function taking uh the state of a current array and then mapping it into like a parallel state in the other array

Jacek Sieka: yeah but we still need to we still need to match the pre and post balances before and after the fork right that's the number that we're checking actually when we're when we're doing the testing

and and uh i no there's

Danny Ryan: yes you need the state you need the pre and post state to match like everyone to agree on it but the pre and post state the way you get from this one to the other is extremely

simple versus a slightly more complex mapping but we seem to be talking about each other a little bit

Jacek Sieka: yeah whatever

Danny Ryan: okay if you haven't taken a look at the open pr please take a look at the opr it represents additional complexity to solve uh this minor issue and would certainly require a bit more testing to make sure that we can get that for transition correct

um let's can every team please take a look at it it's a simple pr um we just need to make calling and i will knock on your door on monday okay um has anyone stood up a an altair testament

with the current specification locally or even any sort of like ci that stands up a testament

Ben Edgington: uh i believe from slack messages that adrian got something kind of running uh with teku uh yesterday or last night my time but that that is all i know

Jacek Sieka: yeah we spin up little test nets in our in our altair pr but they're they're tiny really they just run a few epochs to finalize the national concerns down

Terence Tsao: yeah same here we have our end to end test which is similar like the test net

but we only run for a few epochs that's not nothing fancy there yeah same for us

Danny Ryan: um okay so i think we need to make a few more decisions look at these lasso cleanups through um and we will discuss again in two weeks on plan any other altair items

Ben Edgington: yeah i just um just a little concerned about timing if we're uh punting two more weeks before we make

a decision uh it just so june is definitely kind of outright if we're going to run a six week or so test net um at that stage then we're sort of into mid-july and we've got things going on on the eth1 network and so on um time starts slipping away i mean should we start planning a date because it it always there's a latency between sort of planning a test note and getting everything um uh ready and prepared uh for that um and so if we don't decide for until everyone's ready then we've got this sort of uh latent period

Danny Ryan: right i think the one another compounding factor is that we

don't we have like five open pr's on this the spec repo so we could put a i think if we put a test net target date we also have to put a spec freeze target date which we probably should so if we're looking at a calendar let's at least do the theoretical um could do spec freeze the 14th first test net last week of may 1st week of june i'm just gauging temperature what are

people think about that last week of may is this is for an altea test now right yeah so i think the sequence would probably be we have these two public test sets that we can fork uh but we should probably do a multi-client short lived test set before then would be my guess

Vitalik Buterin: given that we're still like occasionally finding small things maybe spec freeze on the 21st instead of 14s would be a little bit safer

Danny Ryan: yeah probably more realistic

i'd

Paul Hauner: need to chat to the team the broader team in order to commit to anything um but i i'd really be leaning towards the later rather than sooner right it's in like the next month instead of like this one

Danny Ryan: so like second week of first second week of june is short-lived multi-client test that's moving towards end of june on working the actual test nets and doing a hard fork in july or august

Paul Hauner: yeah i think so um i would have to check with everyone else before i could come in

Danny Ryan: yeah actually um on the fuzzing effort and fuzzing

infrastructure when what's the status in that i know yes

opening it back up

Mehdi Zerouali: no good um so yeah we're spending a lot of time patching uh youth chufas and basically incorporating the latest changes of various clients that we have in there so that's taken actually a lot more time than i anticipated um but that timeline will probably you know work for us uh i'm i'm expecting us to have uh both the differential fuzzing and the um coverage guided fuzzing up by you know june early june so hopefully we can get a decent amount of fuzzing cycles in before we or around the time we'll be forking

the test nets

Danny Ryan: okay so i'm based off of that spec freeze in two weeks forking one of the public test sets by the end of june and the other one shortly after and coordinating the actual main altair fork depending on kind of in conjunction with with london to be staggered at least some amount i guess the nice part is uh that our validators could

update both of their nodes at the same time to deal with altair and london rather than having to do one and then do the other okay so i think that does put us at a reasonable sketch of a timeline that we can commit to and in the two-week time we will have the frozen spec and can get an engineering update and harden that a bit more

Ben Edgington: cool thanks uh for considering that i think it helps just to have some idea of uh what what a plan might look

like even if it's you know somewhat vague absolutely

Danny Ryan: okay um anything else on altair moving on any research updates for today

Vitalik Buterin: i started working on an annotated spec for old town

Danny Ryan: okay anything else people want to share

today on research side okay specs can you let us know um what the state of that testing release was

Protolambda: sure so um right now we have a lot of new features in the left branch like in respect to testing both altair and the merge we don't have a release out just yet but so yesterday i made two different pre-releases or pre-releases of the pre-release really that

enable all the client implementers to move ah to go ahead and test the merge and try the latest alter changes and then i expect like end of this week or maybe like start next week it will cut an official release

Danny Ryan: thanks create any other uh just back related items

Protolambda: one more thing so i'm currently writing

a proposal for the new way i would like to handle configuration within the clients and so i am working on updating the specs to separate constants configuration and presets and the idea here really is to try and separate the things that are really intended for test builds and these kind of more static configuration things and separate them from the more dynamic things that we change with almost every testnet like the fork versions uh forward planning like the timing of these works and a few of

these common configuration variables that we would want to change in

test nuts

Danny Ryan: kind of like a configuration preset versus a chain instantiation right so like minimal versus like when do we do the fork what's the time stamp that genesis is not going to say

Protolambda: right so the idea here is that it's much it's less scary to try and change the configuration for different test nets and you can rely on the runtime capability of clients for different local tests and test nuts and then settle on like a few set presets that change more of the configuration

for testing purposes but which won't have to you won't have to go beyond those presets you want to support them in binary so we can have compiled time configuration for a lot of things

Danny Ryan: right which i think a lot of clients do and this just kind of makes it cleaner in this operation

Jacek Sieka: i mean speaking of which do we even want to maintain the minimal configurations anymore um i feel that they were kind of a hacks because we didn't know very much about performance back then and now

Danny Ryan: we do they're used extensively on

python spec testing um it's very important for our ci just because we can't wait the time to run the the mainnet uh configuration testing and i do think it's good to be able to different have different configurations up there

Protolambda: right so i think we should maintain the minimal preset that really just define it does this one preset so we have mainnet we have minimal these things only change during compile time and then we do specify like which things are like part of a preset so clients can opt to define additional presets but we don't require clients

to define more than these two spec presets and we just try and isolate the parts of the configuration that we do want to configure as a user

Danny Ryan: all right keep an eye out for your proposal all right anything else spec related or anything at all i want to bring up and chat about today

Jacek Sieka: yeah coming back to those constants i

think as a general rule we find it easier to reason about compile time stuff from a security point of view it's always easier to size things if we can analyze it statically but yeah that's right as well

Protolambda: so for um for the shape of the of the beacon state and many of the other types these constants are very important to know in compile time to both optimize for them it also just reason about the security and then there are these other types of applications that you'll see with the light client hard fork a lot more where we want to do

miracle proofs over the state and so i think it's important that this this shape of the state is very consistent so yeah we'll make and try and be on the conservative side of these types of configuration variables and make them compile time

Jacek Sieka: perfect and we're on the same page

Danny Ryan: great anything else today

uh yes uh any craving yep yep uh yeah so used to uh quick

maintenance quickly mentioned that um we have set up a group uh for working on the standardization of the um style metrics uh so i would take this opportunity to thank uh the client teams to help us on this effort and we are starting to make some progress into uh selecting a subset of metrics that is already implemented on our clients and trying to standardize those so that we can track many things across clients and another thing that i want to mention is that together with the at staker community we are starting an effort in order to track plan diversity and version progress over time in a

frequent basis yeah so that's it thank you

Danny Ryan: got it thank you where is this working group gathering is that is there some sort of chat or is it on the uh discord

yes so we have a discord group for both efforts one for each

Danny Ryan: okay anything else today before we close excellent okay uh that

tentative altair timeline is the goal let's work towards it um i'll talk to you all very soon appreciate everyone joining and for all the updates and conversation take care thanks

thank you thank you [Music] [Music]

[Music] [Music] [Music] [Applause]

[Music] [Music]

so [Music] you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.