All Core Devs - Consensus #53

Transcript

[Music] [Applause] [Music] yes sir yes boss um okay we should be transferring over um pull up the live chat so I can talk to the people of the internet okay cool if you're on YouTube

and you are in fact listening and can hear us let me know here's the agenda someone is spamming the hell out of that chat call 53 been doing this for a while um great so if youall weren't aware May night launched um call 53 new deck of cards New Era yes indeed uh actually we're at the Joker there's two jokers in this deck um but anyway yeah I I think that y'all may may or may not be aware um

May night launch obviously you can debate taxes you can debate um what actually launched uh but yes we bootstrapped the proof of State consensus and it's running stably and um very proud of everyone it was awesome um and now there's plenty of work to do but hopefully you can enjoy the holidays so testing release updates nothing nothing going on here uh we do have some testing reforms that we want to do and uh Fork Choice stuff that we want to get in uh both of which the testing reforms kind of enhance enable the fork Choice um a lot of this is

going to be kind of D duplication of repeated stuff within the testing outputs so that this thing doesn't blow up to be like 30 40 gigabytes it's already very large um so that's in progress uh obviously it's sto being prioritized in the past three four weeks uh but that is in progress um there's also some specification work going on that is related to a couple of upgrades I think that we'd want to see early on and maybe kind of eyeballing for first hard Fork um the feature-wise the the big one would be and it's not big it's actually very

small uh is adding a light client sync committee uh to add my client support as a first class citizen it's pretty much reusing um all types of infrastructure we've already used signature aggregation uh subnets that kind of stuff uh but for a this particular type of committee that changes more slowly um so keep your eye on that uh probably in January we might release some test sectors for that if people want to uh begin building um but I also know that um you know there are op ation and stability things that people are still working on um as the main net gets bigger we want to make sure we stay stable any

other testing updates a quick note about API testing yeah um I've been working on API bindings for the standard API uh Source test vectors and trying to um run through the full surface of the lighthouse API um if anyone else is working on bindings or likes to talk about API tests then please reach out on this card great thank can those bindings be

generated from from the definition so there's two parts of this um so tests and client tests um for the client tests I run through whole lot of variants that's called the server collect request and response data and then mock the that interaction to test the bindings uh server testing is more complicated since it depends on lot of dynamic conditions um there's I'm checking status codes right now and checking the format of each of these responses make sure things are right but then again it's a lot more to go through

with server testing we can talk about this in this Cur cool anything else on that um I also think we're probably inches away from a release on those apis um that's I know it's something I need to get back back engage with but if you are uh regularly engaged with apis um if there are any kind of open issues or anything else that you want to see uh before this release Now's the Time okay cool uh test net main net Pont

looks good main net looks good uh again congratulations it's been a wild ride any comments thoughts anything here um I have a question has there been a discussion on starting a test net but with finality dis but with finality disabled I'm asking that because of um there's lot of great lesson learned during the nonf finality period at the Pont and the definitely clent made a lot of optimizations around there but feel like we can use more optimizations yeah I agree um some of us

were discussing this this morning um on either we were discussing it from an angle of like just running some load tests um and seeing what we find but an extended test net uh devet without finality might make sense adjus in your Unity have some thoughts no I just going to say that yeah I agree it's um high priority and it's um I mean for for those who may not know we onboarded Parry par TSH theim Foundation helping you know at the intersection of of devops and security and um you know that's one of the things that we we really want to

prioritize a test net with no finality yeah um just a survey I'm curious uh do in terms of client performance if there's no finality but incredible stability um is that a tough scenario or is it when there's no finality and also a reasonable amount of forking or orphans and things is where does the load in your estimation uh become induced here I think when there is a forking and non finality uh clients will not prune the

the dag in memory and then you will have us AG great so but if in in the Assumption if there was Zero forking uh do you have the same issue uh no prunning but uh since we don't have Forks to keep in memory uh that's fine I supp okay so it might make sense also uh if we're going to do this to have some subset of the nodes maybe have a clock skew so they regularly uh are getting blocks orphaned uh to try to capture some of the load in a in a more loadful manner um anything else on this uh definitely

something I think we we should discuss in the coming couple of weeks and maybe Target for early January I think another interesting case is is long periods where there are no blocks at all uh I mean when prism had the rough time issue we didn't have any blocks for uh I think maybe two EO was it three EPO I don't remember right now the numbers but um there was a good test for the sync protocol to ensure that we can handle an entire Epoch without blocks correctly yeah I got you um that's also that kind of stuff's also

on the list of some load tests that we're like to engage with um but we could also induce that on like a public devet I suppose everyone turn your nodes off everyone turn your notes back on um okay cool anything else on this can we talk about future of um test Nets in general uh getting lots of questions is one is it longed or will there be something else thoughts so my intuition and I'm happy to waver on this uh depending on what

people think but my intuition is uh Pont Works um and at least probably through the end of January I don't I don't see any reason to disrupt it obviously that doesn't mean it's quote the one um there are some techniques at our disposal uh to ensure that a test net stays more stable and we haven't the only thing that we really have at Paramount right now is we ask people to not spam it and we run 100,000 validators um this may be enough for a while uh but ultimately like it might it might just degrade um and so we can start a test net with a much higher ejection balance like uh the prism guys

did many months ago ago probably a year ago um and we could also tune the leak parameters uh to ensure that the network even if it has a little bit of instabilities kind of stabilizes more quickly uh we can also alter the Q links to kind of have a higher quality of service so that people can get get in although then you're now changing the user experience from Main net so maybe that's not a great thing maybe changing the exit cue um but that's actually not easy to tune independently of the activation queue so there's there's some stuff to discuss about what we would do in that test net but um my my intuition is essentially probably pont's good enough

for the next two three months um but we should consider our options and maybe do a a fresh start next year um q1 Q2 but that's just my opinion right now I I don't know what what do people think on on this yeah since ask question uh sounds good to me I think it's uh because two three weeks ago we were talking about taking a view on this and people just want to know where they stand um I think you're right Pont is serving the the function at the moment does anyone else have any strong opinions on what we might do here

okay so I think we Pont serves the purpose right now uh we'll keep our eye on it through New Year and keep an open mind towards how we might have um a more stable test net in the future uh if and when pamod degrades mommy you something yes um one thing is that pment um uh in a month will start being quite slow to s and um we might want to have uh the wick subjectivity think uh in place uh well in a months also yeah agreed um and I know pont's a higher load than main net as well but

that's probably a good thing rather than a Bad Thing uh but you're right yeah to weak subjectivity sync from a from a a recent state is um probably a pretty high value feature um not just from security but from a ux standpoint to get out in the next couple months okay anything else here well I only add that we'll be on Main it where permont is not too long from now because you know we never Prov the valid dator set even when they exit so right although I expect pretty low

exits on Main net yeah but nonetheless like yeah it'll happen yeah I mean and there's there's multiple types of load right one is that validator set even if exits happen um grows in memory but if exits happen and and the validator set is half of what you have in the registry the active validator set then that doesn't induce as much load on networking side so there's there's a few different angles there okay client updates let's start with

nimus okay so uh well uh main net loue uh we have uh following that uh a couple of things uh to to done uh in particular documentation but we still have work to do there to try to clarify because uh right now it's very detailed um we have a hot fix yesterday we have another one uh scheduled today so people using the Embers get track of our Discord and we'll put in place mailing lists and the ash of releases uh for for security we want to thank uh all supporters uh in

particular who supported us on gitcoin over a year ago uh we still thinking about you and uh we'll have something for you soon and uh otherwise uh on the software side uh we are still using Import and Export of slashing databases the V3 uh so we plan to do V5 uh soon and uh on the networking side um we uh for the launch we raised the number of peers requested by default from uh 79 to 160 uh which had a very good impact uh because we have now a much better

attestation inclusion delay um usually it's zero now uh one thing we are concerned with is that um over time we see a drop of Lighthouse clients while uh in know p connection while uh teu and prism are stable uh also not related directly to numus but uh for everyone benefits uh the team at blst they released um a very significant Improvement to uh the square root function which uh implies uh a benefit of 40% on the h2g2 and uh since this is almost a third of uh

verification uh this should Translate to to 15% perff Improvement uh for every clients using blsd and uh that's it for me m is that Improvement in the latest release was it 0.3.1 uh or is that I'm putting a link right now this was released 16 hours ago super thank you very cool um load star hey uh so we finally um got our BLS te integration at the door um working

closely with with their team they were very helpful we did something similar to a lot of other teams where we can U create a common interface between uh blst and harumi her's Library uh so we can um use blst and node.js and harumi in in in the browser we're working on getting um the uh blst to compile to wum of course it's not going to use the super fast um assembly but we're very curious to see what the performance is going to be like between harumi compiled to wasum and blst wasm so we'll keep everyone informed when we get some results on

that um other than that we did finally cut a release day before main net uh I would say that L start experimental level robustness um we are are we do have some nodes on mainnet and they're they're staying sync there's some few performance issues that are still uh kind of plaguing us um 100% CPU usage verifying a bunch of aggregate proofs but staying on the network um so yeah that's that's it for us cool and you're not attached you don't have validators attached to those

nodes right now no no yeah no no economic value attached to those noes we we're g to we need to do a lot more testing yeah um on Pont and other things so so I suppose it's just an enduser node you don't even need to listen to the aggregate improved Channel although it it helps you in some like CIP resistance uh scenarios but you know it's probably worth listening to that channel to induce the load and and figure out how to optimize it right right yeah just from a readon kind of Beacon dode staying up to the head um see seems pretty stable right now cool congrats thanks let's do prism hey guys so yeah Manet launched um

before Manet we will mostly working on our documentations uh Mana Readiness uh fixing user reported bugs we have two or three patch relas from that and uh yeah so thank you to our users and contributors for um working um with us on that and uh post Manet um for the this week next week we'll be working on um Import and Export of validated client exchange format that's the number one priority right now which uh we don't support and uh in parallel we're working on with subjectivity sync from the checkpoint state and also working on um implementing the um E2 API and um that's it

thanks great thank you let's do teu right that is me uh so we have released this uh week's objectivity checkpoints Inc uh feature so you can pass an initial State teu via the CLI and you can start your sync from there so um um this is a great user experience we in never mind the weak sub subjectivity side of things if you can get a recent state from a node that you know is reliable you can be often running in under a minute um so it's great for cold standby and stuff like that anyway Meredith has made a great video that demos this and explains the uh longrange attack issue very nicely

which ought to be out soon so keep an eye out for that um we've also implemented a historical sync that will backfill the blocks from Genesis up to the initial state that you specify uh in other news um we fixed an issue where our attestation sometimes failed on the first slot of an Epoch on Piermont it's a kind of race condition where with I get the wrong Source on occasion so the attestation doesn't get included um we have a remaining issue about delayed block production after Epoch processing um we're fixing the it doesn't affect main net at all at its current scale it seems to be just a

scale of Pont that that exposes this uh we also fixed an issue where we failed to gossip voluntary exits and slashings we now gossip both uh correctly um I'm aware of the conversations about gossiping slashings or not um but we do that now by default uh and the main work in the last three or four days around mayet launch has been finding ways to pull data more reliably from uh eth1 nodes on Main net um turns out it's very different from uh gly uh there's lots of gu around on slow drives or limited resources uh we've had to work around nethermind and open ethereum not downloading chain history or not providing um the data in eth get Block events U so we're just finishing

up the last testing of our fixes for that uh as a workaround we've been recommended to any users that have trouble just spending 10 minutes sying up with infura before switching over to their own E1 node as the load is much lighter after the initial catchup um aside from small bug fixes and bits and pieces that's it from te awesome great work uh I'm excited to see that video um and I'll mention this in the network I know there's the open issue on exactly how to do back fill and sync and stuff if you're starting from that state um and I will take a look at I'll open that back up today and we get it merched

uh and Lighthouse is there anyone from Lighthouse here H uh yeah cool they launched on mainnet as well and seem to be running okay uh and we will catch up with them later um and Tomas you're joining us today do you have any uh up updates on your end you'd like to share all good okay um research updates anybody

have anything for us sorry sorry Dian it takes me ages to to unlock the phone to be able to good entering the passwords and searching the unmute uh no I joined mostly to to hear about like was the follow up the feedback what kind of problems you had with with ethereum one neres so our ethereum T obviously is on hold and obviously congratulations to everyone as I said on the channels as well and yeah I'm just listen Okay cool um is there any any particular uh comment you'd like to make about nethermind in this context or any questions for Tomas while we're here okay um I I do have a question so I saw

on Reddit that uh a full well uh a never mind node that can be used to serve um the if1 monitoring would only take um 45 or 60 gigabytes uh that was in uh 2019 uh how much uh because uh since uh we target 100 gigabytes now okay so this this is still much better than G at 300 gigabytes uh or Trio I think is also in the same range uh yeah for the minimal support so what you need for minimal support is the receipts uh since the uh deposit contract was deployed on etherum one

network so this is uh 11, 50,000 blocks um so te had this trouble because they was doing binary search where they had reasonable approach to just find that deployment of the contract by searching when it appears uh but if you hardcoded then you know that you don't need any receipts from the past um and what you need is just like 10 15 gigabytes of headers uh around 30 gigabytes of receipts from the recent history and 60 gigabytes of state and this is this is the absolute minimum to support uh ethereum too

gotcha anything else here uh NE mind eth1 thinking that kind of stuff before we move on is anyone working on a you know on infrastructure to remove the requirement for the E1 node where we have this this super light client that just um you know syncs the the headers and then we have have an ad hoc gossip Network specifically for the deposits and the merel associated Merkel paths um it would be just so much lighter than the current situation for validators a followup has anyone exper

experimented with using the Lees protocol the light protocol we were trying to implement Le in that mind but obviously is this good question for for eum to client so sorry the the beaming client potentially can serve all your all your requests and it would be smaller in a way that it could go down to 40 gigs um if we just beam sync to the to the recent State and download receipts and we are working on it for uh for r key support and it also would work with ethereum too so you can actually go a bit below what we have now and with Trinity team we working on the witness

protocol that actually the the beam sync can exchange information much faster with the nodes and the network so I can go below that as for the less client light client thinking if it requires receipts from the past then obviously it will be heavy download of the receipts you need to have it unless someone delivers verifiable provable bundles of receipts historical receipts for ethereum to just for the deposit contract so this would be other approach um to just be able to download those and get them verified maybe store them somewhere and and be able to trust that this is the correct bundle

gotcha okay let's move on to some research updates short note about week subjectivity stuff there's a new week subjectivity calculation uh wherein the period also depends on the average validator balance um this Branch for it expected to get um merged in the next release um on on a different note uh Alex and I have been working on this slashing protection rebuild thing where um it well depend it uses a few different methods to rebuild the slashing protection interchange file um we actually did not know that clients don't

support uh V5 of the slashing interchange format yet but if there's any other nuances that uh we should know about please let us know it also uses the E2 API so super important that uh clients support all of that infrastructure for software upwards in the stack cool uh other research updates so I have a a couple of PRS in progress and so there has been that light client one which I think well most most recently I saw sh we has been doing

a lot of work on trying to make it executable and I've been reviewing that just now the um second one is the incentive accounting reforms um so that is p well I will just get the get the pr number for that right now at 240 um and the goal there is to basically um take some of these simplifications the rewards that we had in the big humongous phase one candidate uh and just move them over to a separate patch which could potentially be included much more quickly so the goal is to get rid of pending at the stations

and basically replace them with a bitfield data structure um and this should simplify this back increase efficiency of accounting things and also set the stage for things that I think would be good to do later but that I haven't uh well yet written the um written the specs for um which is basically there's a couple of kind of wish list items I have one is to redesign the speca bit so that empty Epoch and long strings of empty Epoch don't have ofn for Epoch processing time the second is reforming how the inactivity leak works and then the third

is potentially using polinomial commitments or other magic to remove most of the Merling overhead from the uh from Epoch transitions so that stuff's a longer term but like in addition to just being like simplifying and making and making it easier to add more incentives in the future that's another thing that this um PR is is designed to kind of set the stage for in the long term so the pr exists um and the the longer term things I that I mentioned they all exist as kind of scattered issues as well but I'll

probably write a more and publish a more Cur document that just talks about all of that stuff quick comment there more implementation focused all or most clients are already optimizing the EO transition in a way that's very similar where it first collects uh the stats data of all the validators of the previous two EO and then those other reward calculations um so this change there with the bit field may not actually actually be that big U even is looking at implementing that optimization uh to improve their

ebook transition uh times okay in that case you might even end up just simplifying The Code by a lot because the the data structure that makes sense from an efficiency perspective would also just be the data structure that gets hashed great um any other updates um yes um the executable Bon chain proposal has been published uh thanks everybody for the great feedback uh there are some pain points um to address like block hash issue some

others uh we'll keep working on them um in the meantime um the Prototype and is already in progress um so soon likely soon we'll see te client with with one execution in it um Gom is also working on dating get list um so the Prototype is the goal is to to uh Implement and evaluate uh the core um the core proposal um yep and uh probably r a test n in the midterm so

that's thanks Mel other updates here one quick update is some some good news regarding uh blst so um an optimization was found U for doing square roots over extension field and uh that means that the the hash to G2 is going to be 40% faster so quite a significant and nice optimization there yeah Mommy uh mentioned that in the nimous update maybe you were oh away from the computer which translates into a 15% speed update on signature verification I think see as he said yeah

because in my benchmarking um ash2 G2 is uh one3 uh because you have h2g2 then pairing and uh it's one3 two3 in terms of uh uh proportions time spent okay cool yeah killer Library thanks supernational um okay other updates here um yes um just very quickly can you hear me then yep okay uh we yeah with respect to the figures uh so that I showed last um last meeting uh we have

written a document um kind of showing all the um things we found uh and it relates to many of the things that we discussed at the beginning of this meeting uh so how clients perform um during periods of non fatality so for finality um uh when there is a long period of no blocks and these kind of things um so um I have shared this with uh some of you and I already got some feedback so thank you for those that gave feedback um if not then we are always waiting and very happy to have feedback and if um somebody here didn't receive the document yet and you would be interested please contact me so that uh we' be happy to share it with you thanks

cool thanks anything else okay great uh networking um some of us have been discussing uh having a networking call next week to kind of catch up to speed on all the the various things uh it's tentatively Wednesday morning I'm going to look at the calendar and put it on the calendar right after this call um I know I mentioned earlier there's a a standing PR uh with some comments that we still need to be addressed on um we subjectivity in the context of sync and

and what is a good and bad Noe in this respect um and I will get to those comments today and hopefully we can get that merge relatively soon um are there any other pressing networking items people would like to bring up right now otherwise we'll have a a long form conversation in about a week okay yeah I would perhaps just repeat that uh it's good not to have very low peer settings uh overall that makes it difficult to find to form a suitable

mesh on on all the attestation topics that we're supposed to subscribe to makes it harder to find clients and it kind of degrades the network performance in general so hoping that users that listen to this just keep your maxp settings reasonable and not something like 30 that will significantly degrade your station performance right by reasonable I mean in the range of 70 80 100 gotcha um anything on networking before we one so about bearing um there was

this recent issue or kind of improvements we could make to connecting and onboarding is this where the a client should not be strictly dialating at the exact maximum of their per ing that rather keep some buffer or some kind of elasticity to allow for new pairs to join them even then slowly rotating them out or handling this turn in some way or another but this would help on kind of situation of may not where one or two client types uh make on boarding others uh rather difficult right so essentially

allowing for elasticity on the upper end and then on some interval pruning down to a target number rather than having a hard limit at a target number okay uh anything else on networking before we move on great uh um General spec discussion any questions comments thoughts here um I if if we wants to talk more about uh some of the things that I

mentioned always always happy to talk talk in more detail too this is around changes to incentive accounting and then um empt optimizations and a couple of other um types of optimizations um yeah any questions here for falic know maybe I'll just like publish a more um readable allinone Place document so people have something to refer to and we can also talk about those things that

well though I didn't I do think that we should start some of some of those discuss discussions sooner rather than later as well and especially if we wants to move toward having a having some of these things in or in a a hard fork in a fairly reasonable amount of time yeah I agreed I mean I think if you get it into it I just people read okay I will do this um I have a question uh regarding the uh phase one and phase two spec well

uh everything after phase zero uh what's the state of the uh the spec and the tests uh because will likely uh dedicate now people uh to those phases and uh we'd like to start with some low hanging freit uh let's say right so the big things that are being in R&D right now are the uh data availability charting and um merge spec uh both of which I think are going to probably spend the next four weeks doing some testing and refinement of the ideas um at which point I think we'd start

seeing them form into more concrete specs um on the data availability stuff uh there's kind of two components here there's a number of components here uh but um we'll likely be leveraging uh past lookups in a DHT uh we'll be leveraging uh some new cryptography uh called Kate commitments um and uh be needing to kind of manage additional validator d and some extended uh Beacon chain spec items um and and block essentially managing blocks in parallel to the beacon chain

um so before we can do for example a cut commitments um we would need to push the BLS library implementers to expose the group operations um we already have those huh they expose everything already um not last last time I checked at least BLS blst and heri and many others like they don't expose the basic operations they expose only the signing infrastructure um not actually they do expose uh even the FP fp2 fp6 fp12 G1 and G2

right right I mean the question is whether you can do like um multiplications and additions in G1 and G2 that's the main thing you'll need yes we can and general pairings then then you can start but at least last time I checked with BLS like blst I couldn't do it in Python okay comments so it depends on the bindings what is exposed roomi and go exposes quite a lot but not quite everything and then B in go U is quite limited so the bindings are limited and the exposure but the underlying BLC

Library isn't necessarily well no the library can do everything like you need to be able to do these things um but yeah like the question is whether that's in the API I mean there are questions here around how to design the API to be to make the best use of the library because sometimes The Binding you know can can have quite a bit of overhead and you want to minimize communication for example between the binding and the core but yeah that's a discussion we can push forward with supernational trying to design an API that makes sense and implement it for

all the bindings um another thing to think about is that we are going to need um like specific of BLS things optimized for C based data availability sampling probably the two most important ones are one is fast to linear combinations um AKA multi exponentiations and the other one is the generate end proofs in and log in time thing right I mean the L one is like a specific thing that we'll have to implement

ourselves I don't think we'll be part of a BLS library but the former I mean it's so General and like any zero knowledge proof system will need this so yeah we should prioritize this somehow um for the for the former um I've talked with the the guy at blst and they were already looking into that uh six months is ago uh or even be on so this is uh one of a priority right and and they they've been working on what they call ZK blst um which I guess is blst for these linear combinations for the

SNS um I guess it's a matter of finishing the work polishing it and and Publishing it but they have done a lot of the heavy lifting in terms of finding the best multi- exponentiation algorithms depending on the size of the multi exponentiation because it turns out if you're doing small medium or large it's a totally different algorithm that's optimal for these linear combinations okay um okay I think maybe the next couple weeks we'll spend some time sharing

resources and things with client teams uh so we can kind of come to a plan to begin to be effective on these things in the new year um we're not quite in Spec mode but there are going to be components I think that would be valuable um and some of the teams might be able to help with some of the the R&D um on these efforts over the next few months I mean in terms of lwh hanging fruit to answer the question I think the like client is probably the most low hanging fruit in terms of being easy to implement you know and also being like relevant for the first Fork that we intend to

do yeah absolutely um there's a PR right up up right now if your team wants to review it uh that not only has the slight modifications to the beacon chain but has the um discussion of the like client sync protocol so an implementation of that early implementation of that I think be valuable aged okay um anything else here on spec discussion okay okay um General

discussion open discussion closing remarks anything else people would like to discuss today okay cool sorry I'm distracted by the stuff in the chat uh it does look like someone was scammed and people are upset and I'm that's very unfortunate um okay cool thank you everyone uh congratulations again on the

launch uh stay diligent on monitoring the test net and uh we will talk soon take care everyone bye thanks by thank you thanks everyone bye-bye [Music] m

[Music] [Music] J [Music]