All Core Devs - Consensus #62

Transcript

[Music] [Applause] [Music] so

[Music] [Music] [Music]

[Music] do [Music] okay great uh the stream has been transferred over

this is the agenda should be pretty short and sweet love to get some client updates and then have a conversation about where we stand on engineering on altair um and also give you a quick update on some of the stuff going on in spec right now uh as we move towards like a final release of that uh but definitely waiting on thumbs up on engineering side before we do so let's begin with client updates and we can start with tekku okay that's me uh so alto work is going well uh i expect we'll be done with it by the end of our current sprint which is in two weeks um specifically on

altair uh reference and fork choice tests are all passing uh gossip changes are all done we're currently working on the validator api and duties and for this we're following the api format proposed by jim mcdonald and that's working out pretty well we've got logic to pass the altair fork info from the configuration file and construct four conform schedule objects that transition at the specified slot and we are almost done with blocks by range and blocks by root rpc methods for alte so that's the main thing we've been working on but there is other news um we will be migrating our default

database from rocks db to level db over the next couple of releases we've been running level db in some of our test infrastructure for a couple of months and it seems altogether more pleasant than rocks in particular memory consumption is much improved and we're going to make this as transparent as possible for teku users of course we have release coming tomorrow with a few features worth noting first is a nice community contribution for specifying multiple ethon endpoints and automatic failover between them second infurer is now providing beacon chain state snapshots

via their api so on a fresh teku install you'll be able to point uh directly to infurer's api endpoint and be fully synced and running in under a minute we've had snapshot sync for a while now if if you have another running node to point it out or a recent state file around but being able to start up directly from infuria is super convenient and finally we are now storing orphaned blocks uh in archive mode and providing an api to retrieve them which was a feature request from some of our users that's all i got got it thank you nimbus

um mommy's sorry either yeah mommy usually does it well i guess i should take that um i was expecting mommy to be here but he's off this week so i guess this is a bit of a misunderstanding but anyway i'll make a brief update uh we've released two releases very recently 1.2 and 1.21 they represent a very impactful improvement in performance of nimbus with the introduction of batched verification of incoming gospel stations we've achieved roughly

between 30 and 50 percent total reduction of cpu usage of the client and then around right now for example we can process all the app stations on the broader network on the nodes that are running 2 and 500 validators without maxing out the cpu um other updates just in a moment another major new feature is that we

finally have an implementation of the official rest api which is currently in beta and we just like taku we have implemented a failover webtrip providers and with another notable improvement is that we have a new back at a station packing logic a new algorithm that works in a more lazy fashion it computes

the setup attestation that will be including the block in the last possible moment and this has led to significant improvements in the rewards that we are seeing in the blocks and i guess that's the major improvements for this week got it and how is altair development going on the outdoor development front we are making progress and we are still keeping everything in a branch actually the focus has shifted a little bit towards the ryanism test nets and we are at this point ready to

participate in the first night that's net i believe got it thank you and uh let's start hey uh so as far as altair goes um we have implemented the fork choice tests and the spec tests and they're both passing the consensus tests that are passing on like a naive implementation of a state transition so it's not fully ready but it's at least a first

step um we're still working on updating the networking stuff for altair and the validator interaction um we just cut a new release 0.20 it is kind of a minor release just some stability updates but what we're going to be working on in the coming weeks is we're working on the hackathon we're working on the light client um during the the uh scaling ethereum hackathon so we thought about participating in ryan ism but we thought it might be more prudent for

us to kind of go our own way and push forward on unlike clients during the hackathon nice very cool and um hey guys that's me so on the attire front we finished implementing all the state transition functions that ended up took us a while um we're working on networking and validated stuff right now so the good news is that the spec test is already passing for all the state transition related of operations so we're very excited about

that um on top of that we have a new slasher implementation that raul has been working on so that is planned to go in release in the next week or next next week highly encourage people to try that out it is a lot more performant and on the merge front um with the renaissance we were able to interrupt with the catalyst so we have the interruption there is able to advance um unfortunately there's one issue to work on which um our history route for nestedness is not correct right now so we're working on that today tomorrow and last but not least we have a new uh casey so welcome casey and the uh

and uh i'm sure you guys will meet him soon so yeah thank you great thank you and lighthouse hey everyone um so we've also been working on altera as you can imagine uh we've been passing ef tests for quite a quite a while now uh and we're planning on merging the consensus changes to our unstable branch very soon massive pr just needs a thorough review um we've had a lot of refactoring to do to handle the forks in general especially around those tests so we put a lot of effort there um the remaining features uh from the consensus side of things are the sync committee aggregation and the validate updates uh on the networking front

uh the rpc and gossip changes have all been implemented they're ready for a review and we started the work on the discovery um so there's two issues that we opened on the specs repo that we would be very grateful if people could take a look at uh it's issue 23 14 and issue 23 47 i'll link them just after this the first one is related to the participation rewards for the epoch prior to the actual art upgrade and the second one is a pretty much an open question on whether we should be adding uh the sync net field um to the metadata the networking side of things uh we will be participating in rayonism

rayonism uh the merge hackathon uh so we're building a merge chain with catalyst and uh have submitted box to protos uh merge net tutorial repo uh we're planning a big release in the coming weeks with the doppelganger support and significant memory usage improvements uh we've also actually had a release v 1.3.0 about 9 10 days ago also known as pencil vesta it brings additional validation conditions that are applied to the aggregate stations that essentially prevent incorrect attestations from filling up the ops pool and we've also added improvements to our block packing which essentially

results in better block rewards for our stakers we are also starting the development of our user interface in may and that's about it lighthouse really great thank you um yeah 2314. michael originally from lighthouse originally opened that um i left a comment seven days ago and there's a few options on how we should proceed i do agree that doing nothing is probably not a good call but i also think that doing a map of like previous epoch attestations to flags like increases the during that fork transition increases like the amount of testing and consensus

complexity and things that go wrong with the transition by quite a bit so i prefer to do something simpler but please do take a look at 2314. um and i'll take a look at 2347 today um there's been a little conversation about that another thread on that front altair thanks for the the general updates it sounds like we are getting there um but it also sounds like uh there's at least a couple of weeks of good work to do before we're working on some transient test sets is that you know where we're at yeah that sounds great okay so that puts

us two weeks from now puts us a week into may um the original kind of like end of june target then becomes six seven weeks away um which i think given that this first hard fork that we'll be doing plenty of like test net testing and have a lot of other moving parts with randomism and things going on that that is not a likely target right end of june last week of june seems early at this point i think so okay um

especially with realism also happening yeah there's a lot of homework right now um okay so london is slated for end of july although it doesn't have a date uh it could go as into maybe mid-august but they have the difficulty bomb that is creeping up so that will happen in that time frame it looks like we will be i think the target would be in a similar time frame and so let's keep an eye on when that date gets set and we can think about dates then um i don't think it's prudent for us to go much

farther than that just because if ryanism goes well it'd be a good point for kind of the unified developmental merge but this is also our first hard fork so i think that it is very important that we do it right rather than do a fast uh a quick comment on your comment gossip i think that we have to do a fork before we do the merge because uh there's too much at stake to try this out the first time then i know you're joking um okay so continue conversation over the

next couple weeks we'll hopefully have some like pretty strong progress in engineering and and can begin to talk about what tests look like two weeks from now uh sound good any other comments on this let us know in like a shared channel when anyone has a feature complete version of altair uh that can spin up nets and we can begin playing with that over the next couple weeks okay um on the altair spec front we have these two open issues that betty

pointed to i love some input there there's also been uh some refinement of spec tests some additional tests that were added to cover a couple of corner cases um and a fixed test i think the um in the sanity test historical batch routes the sync committees were being prop calculated incorrectly because they weren't using bls properly so that is is coming um and we're also going to be adding a couple of like actual sync transition tests that have blocks before and after the port is more of an integration test um so we're looking on doing another pre-release

probably early this coming week and so there'll be new test factors coming out but no features have been changed it's been primarily refinements in language and fixing up and adding tests i will definitely be knocking on doors on these couple of issues so that we can um get some feedback any other things on altair that people want to discuss today uh i had a question for for ben really regarding the snapshots and then the sync within pura i'm curious when when somebody starts up

their clients and and thinks that way it doesn't get any blocks right so what happens next when somebody wants to sync with uh uh we can possibly uh discuss offline but uh uh it's just like starting from genesis again i mean it's basically you download your sse which is the state it's about 90 megabytes and uh it says you you need to appear and start um taking blocks oh i see what your question is the backfill of blocks yeah we do uh we do that i don't think we're currently backfilling blocks but i need to check on that

yeah are you thinking about peers scoring each other down for not having the block history or not being able to serve the history yeah both scoring and you know uh once nobody serves blocks then we don't have them anymore yeah uh we we're not currently planning to implement that we don't see that as a huge risk i mean it would only be an issue if um people are um simultaneously everybody on the network you know drops all the whole history so um and you pretty quickly get beyond the weeks of subjectivity period i mean it's not so uh we are not

currently i think um very concerned about it but definitely willing to express uh to discuss not concerned about backfilling the genesis or not concerned about starting before you backfill the genesis uh so we are happy to start with a blank slate and just move forward from there and if we are asked to serve blocks before that i think we just say we don't have the blocks uh currently okay we need to un cover the old pr i think that the network should probably

have some promises at least back going through a max recently period um but if that pr is not merged then we don't have anything to stand on on that yeah i think i think we might be back filling to whatever we think the current um weak subjectivity period is i need to check that we're definitely not going back to genesis i believe that y'all are working off of that pr and that pr has like a max receptivity period um which is like uh it's either four months or eight months um based off of like maximality or set sizes and so that you know even in the worst case uh you you're

filling it out but you're not having to promise to get blocks from genesis um me open that up i know that there's there's also there's two questions there one is how far you backfill and and what the promise of the network is and two is kind of the order of operations can you start validating before you backfill and how you would your score around that and who can lie and under what conditions um so let's take the conversation there i mean the final question there is that older versions of all softwares i think generally start from genesis and sing from there right now regardless of which week's

objectivity right so those if there is if right but albeit i'll altera is a good place to because everyone has to upgrade out of hard fork and so changing like you're going to fall off anyway um and so you can't use those old versions of software anyway so that's those potential coordination points are good places to investigate that yeah of course it's more for like now if a lot of people start i mean network can sustain it to a certain degree and a lot of people start using too much things now and then we'll have issues for people running all the versions for sure

yeah i agree with that in theory but i also think that on like a three month time horizon it's very unlikely that you saw such degradation before a potential coordination point uh but in the extreme you definitely cool let's take it to the yeah are there any other comments or questions regarding this and one more question on how you're

validating that state um is it just a hey inferior give me the exhibitivity state or is there like a checkpoint cut into tekku releases that provides baseline validation it can be overridden what's the security model on that yeah i think when we have all agreed what a suitable distribution method for checkpoint states looks like then i think we'll incorporate some safety net there but currently it's just simple trust um we just take whatever's given so that's not i think the long-term sustainable model but we don't yet have um i think the uh um understanding of what that network of

getting a trusted um uh checkpoint looks like so yeah that's tbd sorry how do you currently uh make sure or authenticate that the state is actually coming from insurer uh it's so you provide your uh inferior url so presumably it could be hijacked in some way but it's you know secured by the normal means um right it's a personal end point right okay so you're just relying on on the ssl site like sorry are you hardcoding the ssl certificate of infuria on the client side uh so the client

provides their url for their uh inferior accounts so they could right and that that's introduction right now uh that is in the release coming tomorrow right i mean um you probably could verify the stay root after right just to verify this date is correct yeah for sure yeah yeah so i mean it's just a simplification at the moment you can download a dot ssd file from another clients and and load it in as a file so this you know cuts out the download step and just makes it super convenient okay um this is

an issue we all need to solve and has like a lot of subtle little trade-offs so let's keep talking about it okay moving on um anything else on altair specifically great um on alter optimizations is there any

any gotchas people run into or anything they want to share any questions they have i know croto has also been thinking about the alastair optimizations when he implemented in serenity um some of them are subtly different than uh prior optimizations so on comments here um there are two optimizations i was thinking of one is with the three of the parts the participation records um so currently um it does this random kind of access to this tree and this means even though we reduced the size of the three with previous type

changes it is still rather a lot of hashing so if any client has tried to optimize this yet please share i'll try and explain the attestations that come in have random committees and so the uh whenever the adaptation is processed it does random access throughout the whole tree and it does this for a number of stations for each block and basically replaces the complete tree uh there's not much you can really cache there so that's one point we could try and optimize and then another thing the obvious one is the

sync committee computation we have been pre-computing the committees and the proposers in phase zero so it's only logical to do the same kind of thing for the sync committee but note here that the computation differs and that we are computing a committee ahead of time for the future period and this is a different time frame outside of the epoch processing compared to the proposed proposal computation and the committee computation thanks for any follow-up or questions

for product great let's move on to research updates uh there's a lot going on um merge is in active r d there's the um call that happens every two weeks so a lot of what is happening there was covered on that call we can answer questions here if you have any uh proto still is managing i think three office hour calls every week for ranism um and a lot of progress there

but beyond those two are there any particular research updates already updates that people would like to share one thing about decals uh tomorrow will be a little bit different we'll have this eve global emerge summit so no varianism of our call likely or we will move it because it collides with the its conflicts with the presentations and you are welcome to to watch the presentation of course great any other research items people

would like to share okay there's general spec discussion we've kind of had a little bit of that throughout um just another i think we said this last time but if you do want to take a look at the like latest charting designs um did a bunch of work to modularize those uh and they contain like the current latest and greatest on on those designs so if you want to digest that other than that um like i said we've

chatted about the specs a decent knot throughout anything that people want to discuss now we're looking at potentially reviving beacon fuzz and having another cracker uh fuzzing your state transitions in preparation for altair so yeah i might hit you guys up within the next few days weeks um to try to yeah identify those new state transition functions and perhaps also have a crack at your weak subjectivity sync for those of us for those of you who have it uh ready or about to be pushed through production

thanks man um we're looking at potentially reviving the fuzz and having another cracker uh fuzzing um that was very weird great all right moving on anything else anybody wants to discuss today great uh there's a lot of moving parts uh the merged calls i think have been very productive especially on the youth one side um

there is an extra awkward dad's call tomorrow uh if you want to attend just let me know and then next week again uh will be all for dips and the merch um i believe monday or tuesday we will have a very short-lived rainism merge testnet launch uh and then maybe the following week or later that week do something a little bit more long-standing uh good work everyone on that it's pretty exciting to see it come together and that's it thank you talk to you all soon thank you bye-bye [Music]

[Music] so [Music] [Music] so [Music]

[Music] [Applause] [Music] [Music]

[Music] [Music] [Music]

[Music] [Music] [Music] [Music]

[Music] this [Music] [Music] so

[Music] [Music] [Applause] [Music]

[Music] [Music] [Music] so

[Music] [Music] [Music] [Music]

so [Music] [Music] [Music]

you