All Core Devs - Consensus #47

Transcript

[Music] [Applause] [Music] [Music]

so [Music] sharing the agenda in the chat and here we go and um if you're on youtube let me know you can hear me okay um i've spent a lot of time working on uh phase one testing um so that's good we're putting the pedal to metal on getting um some refinements in and getting that into a place we can work on um thank you i know a couple of teams

have taken a look and given us a lot of awesome feedback on it so far um we are honing in on what would look like a version one release any potential parameter changes and things like that age is doing some work on figuring out the scoring parameters for gossip sub v11 that's a big item that is not completed yet these scoring parameters are there's no defaults in place they're very specific to the expectations

of the network and the channels that you're operating on you know the expected load and messages and things like that um so there's some work being done using the protocol labs test ground simulator to pick these parameters and i hope that we'll have these done in the next week and a half or so um on that note i think that we're overdue for just a general networking call um there's a lot going on a lot of moving parts and a lot of things people have learned and have questions about so i think wednesday in six days we're gonna do a networking call i'll pop up a suggested time for that um on the

[Music] test net front natasha looks good and everyone's working their asses off still and getting in awesome refinements and things we are based on feedback and based on discussions we've had here we do intend to do a address rehearsal of a of a test net launch with like a two or three day end of life so that people can run through the motions try out the different tools and things before a main net deposit contracts put up currently based off of i think one of the one of the bottlenecks here is

getting some of the bug fixes and refinements in the launch pad which is being actively worked on um but so that's that's why we're not doing this tomorrow uh but the intention i think at this point is to in the second to last week of september post the deposit contract for this um sorry skynet's attract distracting me on the chat it just put a joke um anyway dress rehearsal test net launch uh the intention is to put up the configuration parameters and the launch pad associated with that and all the other things in place um around the like 21st of september that week

and to do the dress rehearsal test at lunch one week later we can talk about that now but we can talk about the preparations and things kind of out of band any questions or thoughts on that dress rehearsal test at launch the expectation would be we take it very seriously we give the community a chance to run through a testament launch again um but then two days after so we have no commitment to keeping up our nodes or um you know block explorers and all that kind of stuff it also gives a chance for block explorers and other tooling providers and stuff to try their

stuff out again questions thoughts cool um so that's a few weeks from now so over the next week we will do some more stuff on that okay um that's generally what's going on with testing and releases that's the kind of next iterative plan on test that's been you have something for us uh yeah i just wanted to ask if we plan to do

a practice hard fork at some time uh on modasher and whether we should start planning it yeah um been thinking about that one too so there's two options here one would be to do it before we do a mainnet launch uh the other would be to do it before we have a fork on main net um i see the value in doing one soon i also see it as um you know an additional complicating factor in everything but i don't know what are you where are your thoughts on

prioritizing that in the next few months uh either is fine with me um don't know what other views are um given that there is an existing strong preference toward launching them toward not doing things that delay the main net does it make sense to test the um dasha hard for um hard forking machinery uh go long before we'll need the hard forking machinery on mainnet that's kind of where i lie but i do also think that someone should do a little a quick at least write up on what that would even

look like in terms of conditioning it upon an epoch and things like that at least so we have a clear plan when the time comes um i'm i mean i'm of the mind that when a hard fork does need to happen on main net it will um we will do it on kind of some local transient test nets and then we'll do it on something like madasha or whatever exists at the time so we will certainly practice but i don't know if practice is the priority in the next few months if we did practice a hard fork are we

talking about perhaps just changing the um the uh the fork struct or are we talking about like kicking out a validator or rejecting a block or um i mean at the bare minimum changing some parameter um yeah that we could kick all your back that's like something too yeah that seems like the most achievable thing if we were gonna do it i'm not even sold that we should but if we were gonna do it um i would suggest that as opposed to trying to like take a shot at some more complicated scenario uh but we might need to blacklist a block as well um so maybe we want to to do that if that

becomes a requirement to be a client that's more or less i guess we could we could also there's all sorts of things we could do i think we perhaps don't want to just speculate on you know things that we're unsure about there's a lot there's a large range of things we might want to do but if people feel strongly about block we can i don't know i'd be leaning towards not getting too complicated with um trying to simulate forks when we when we've already got a lot to do before launch okay

anything else testing releases testaments uh quick updates here so the attack net has a few problems with this backup three clients of that are finalizing gun 100 participation then i'm working on the reef temp network testing and reviewing some clients cool thanks brother okay moving on to client updates uh

let's start with prism uh hey guys so um we'll be receiving really good feedbacks on dos vectors so we have been addressing them one by one just trying to do a really good job on those with also wrapping up addressing all the feedbacks from the coin stamp audit so so this will this will help us to get ready for the future audit we started working on implementing uh exploring the uh e2 api so the current content tentative plan is to have this supported for for a minute and then we started um researching on uh which subjectivity sync

so trying to uh scroll up the work um on just how much there is we've been fixing the slashing bug the slasher box we optimized the slasher from the modesha accident so we're also making sure our slashing protection db is um interchangeable and we're also implementing the ux for the validator client exits end to end workflow and then on top of that um we fix a few things for our interrupt mode so that should be working today and that's it thank you great thank you let's start hey uh so uh let's see the past few

weeks um we've fixed a big um memory issue that was preventing us from sinking madasha um now we uh can we prune unfinalized states whereas before we were holding them all in memory um and we've also fixed it a separate bug that was also preventing us from sinking madoja we had a recursion in our fork choice and the long period of unfinality caused the stack limit to be reached so we fixed that as well and we're looking at just replacing the four choice entirely with

something a little bit more separate and we are also working on adding the phase one types and parameters so we can at least start doing things with them um at least you know making proofs or whatever um that'd be a first step for that other than that we're still basically working off of madosha trying to get synced and trying to [Music] propose a block i don't i don't think we've managed to do that we are our eth1 processing is still not i don't think good enough to propose a

block so that's kind of one of the final blockers i think for that nice so are you all able to get to the head at this point and not quite uh we we so those bugs just fixed them um yesterday so i haven't actually gone to the head yet i haven't turned it on today but um we'll see i i think i think yes but um remains to be verified at least by me yeah sounds good thank you um teku hi everyone this is jem from techo uh along with the uh miners along with like

a couple of minor several bugs we fixed user reported block proposed letters and which seems to have been caused by missing deposits from the e1 node so we added that validation uh around that and also there was block of probable proposal letters that were due to inclusion of attestations from different forks and that became apparent when we were when the medalla was like not finalizing for a long a good amount and we fixed that as well uh we've had huge improvements in attestation effectiveness we're almost there we also are thinking about our fluid publishing locally produced at the stations and using the mesh one propagate using the mesh for only propagating messages from other peers we're curious if other teams are already

doing this we've implemented the unicode password normalization rules that went into eip2335 lastly we're up to date with the v3 version of like the slashing protection interchange format yeah on the the intention is to um when we go and specify all the v11 uh parameters to have blood flood publishing set to true i i think that might be the default um i don't know if anyone has that enabled or not by now for now yeah it would be good to get yeah yeah would we get to know what other clients

are doing but yeah that's probably why okay numbers hi so um let's start with the audit uh we had all face to audit that was in part on the cryptography and also the consensus part of nbc crypto is still ongoing and regarding the consensus part uh we the auditors highlighted several potential for malicious attestations or blocks uh

to exhaust the system memory because we have a quarantine system and there are a couple of situations where an attacker could make the quarantine absolutely huge so now we are moving to using bounded buffers and explicitly dropping either the peers or the blocks once we are filling because that's normally a corner case and it shouldn't happen normally uh otherwise uh we have we still have a couple of issues on medasha or current focus are uh to be able to run

numbers without restarting every once in a while for that we need to address uh three uh issues number one is the log file that is filling disks for end user and also uh taking a couple of dozens even hundreds of gigabytes on our aws node so we will move the default login to something much less verbose and we will raise people have been asking uh to differentiate their own attestations and blocks versus the noise coming from the network so we will implement that uh second issue is a

deep a database that is filling disk uh so far we didn't have uh database printing and this is something that we are working on and the last one is a long-standing problem it's uh it has been significantly improved but we still have a ram usage issue so it's still probably in lib p2p with some asynchronous timers that are holding objects and not freeing them and then this is holding ram lastly we are still maintaining on maintenance scripts prism is working

so we are now able to test for example a very small network with uh uh a split of eight uh times 64 lighthouse nimbus and prison node uh to check if we can reach a finality so i encourage uh maybe uh as deku and stars team to look at those scripts and maybe try to add their own client and of course we can help you on that it's in the if to client multinet repo great um on the

are you all just manually triggering those network tests at this point or are you looking into doing some sort of ci with it um well the thing is uh or ci we are already at one point half hour of ci so [Laughter] it's a bit difficult especially with um uh main net params yeah i'm definitely interested in doing some sort of nightly build on something like that so we'll take a look at what you'll have okay great um lighthouse

hello paul here um so been up to a fair bit in the past couple of weeks uh we fixed a deadlock that had been plaguing us for quite a while um it was a sneaky little red white lock somewhere um interesting scenario we have a write-up on the blog if you're interested in or use russell tokyo um we fixed the bug in our pruning that caused the database to get too big um we've been implementing the http api the standard one um got through the beacon endpoints now um sort of chugging on downwards through um we've been trying to looking into attestation inclusion um some metrics to determine uh kind of what is good and what is bad um we're working on some ux fixes just

trying to generally make lighthouse a little bit easier to use um addressing people's feedback we have a new team member starting next week which is pretty exciting i let them introduce themselves when they're in but we're looking forward to having someone with their experience uh on board we have gossip 1.1 in master and adrian has been working to define some parameters for the specification um our slasher found some unique flashings that we don't think were found by anyone else on the network yet at least haven't been slashed so we're waiting to see them go through and i think that might be some of some of the first last things we've done with our slasher which is kind of cool um we i've been working on defining the

slashing protection interchange format so we're implementing that as we go um and we're kind of just chugging along uh because we have two audits kind of one and a half to order it's starting uh in early october so trying to get everything in before that that's it from me awesome and was the person here that what you wanted to introduce sorry no they're not they will be here perhaps in the next meeting they're very cool i'm gonna get some suspense going thank you paul and trinity everyone i have an announcement today

we're actually going to move forward with tabling the e2f person trinity for the time being those resources can go into the imminent mainnet launch that being said the project is definitely open to open source contribution so if anyone listening wants to get involved please reach out we'll miss you trinity we're just going away for the time being the project's still there perfect sweet thank you alex okay um any research updates do you want to go over do we want to talk about the phase one

changes um um yeah i mean i'd give a quick on it uh so we had a if you're following phase one um there was this kind of uh there's a requirement that for a crosslink to be formed it must have come from the immediate prior slot the committee from the entire slot um although this um this was to avoid some of the complexities and trying to accounting to accounting across multiple slots um this ends up being this is pretty pretty brittle design so there's currently a couple of pr's up in phase

one that does allow for cross-linking in the non-optimal case and in doing so we've actually revamped some the way that attestations are being accounted for over time in the beacon state in general um instead of storing all these pending attestations uh we instead store uh flags bit flags related to each validator and and their participation although this is a certainly a divergence from phase zero it actually ends up being a much cleaner way to do the accounting changes related to both of those are being tested and under active review if you want to take a look take a look

if you're following the phase 1 stuff there's semi significant changes but it in terms of the optimal kind of validator case uh it's actually the same if not simplifying um and most of the like public interfaces in the way that you think about cross-linking forming are generally the same um it's more about like internal accounting or not optimal cases but happy to dig in deeper if anybody as people review if you're taking a look at that stuff do we have any back envelope number on what the state saving is like after um

removing pending adaptation um state size savings um it's it's negligible compared to the validator set size um that was definitely not a driving factor in in doing that it was more of the way that we're tracking crosslinks it definitely was a simplifying um factor on the crosstank accounting um yeah that i i don't know the back of the envelope calculations there might be like a one-half savings or something or

three-quarter savings on that uh particular accounting data but it's not it's definitely not the driving factor got it other research things other questions um just a quick update from our site um we are uh trying to build a crawler uh using rumors uh so we have uh contacted proto with uh for to help us with this um and uh yeah we have right now uh first implementation that uses the discovery protocol get a list of peers and at this point we are

not doing gossip so but that would be like the next step and the idea is to keep track of the score of the of the nodes on the network and have this uh kind of monitoring system in the network so this is uh right now other research updates great um terence asks a question about weak subjectivity sync

um yes or no is it a main requirement uh the prioritization of future requests today um and the progress in having this documentary to be sick um having something in place at or very soon after made at launch is the requirement um due to the way that you can construct alternate histories if you don't have any sort of weak subjectivity when you show up to the network uh a matter of weeks maybe months somebody can construct alternate histories for free and at least make it an annoyance to sync um there are obviously protections you might need to be semi-eclipsed for

somebody to be able to get this get this off on you but if we don't add something relatively soon then there will an attacker can cheaply make sync kind of a nightmare um in terms of being able to expose you to chains that finalize that or not quote canonical chain um the easiest thing to do is to have just a finalized route and to reject chains that don't have that finalized route and still do block sync from genesis but ultimately having some sort of starting from a

state is going to be a better ux and if one client does it everyone's probably gonna have to do it in terms of the documenting things in the p2p spec what we can do is document kind of the security requirement here um but i don't want to put a full requirement on how that's handled at this point um and then in terms of the prioritization of the feature requests uh i'd say it's a low priority but very high reward so and it's definitely treated being

treated as a client side like tool or utility it's not really a network level spec uh at least in the current version of how we are doing weak subjectivity checkpointing uh when we do eventually move to weak subjectivity sync starting from specific states we'll need some major changes to the p2p spec in regards to how we identify nodes which have blocks under specific height or not but until then i guess it might just be better to treat this outside the p2p spec and as a client-side tool right

um yeah paul yeah i think we we haven't um made a firm decision on what to do but regarding this but i think at this stage we'd probably lean towards doing the um like you know checkpoint state somewhere in the future and then if you don't sync to that then you reject the chain um for now because there's a lot of edge cases with like um like a teacher said there's a lot of edge cases um regarding syncing and networking that we have to address in order to do a proper you know start in the middle of the chain and sink backwards and we'll keep going forward so i think i think yeah we're learning towards um taking the easy route um to get us to mainnet at this stage uh

and then implementing it in later the full version in later so um question um should all the clients come together and somehow standardize this ux just for example the flag name and and we pass in the head string and stuff like that just so that when people switch client it becomes [Music] yeah that sounds awesome the other thing we we need to agree on is like so one of them is as you mentioned the flag name the other is the format for whatever these checkpoints are so that they don't have to be client specific sounds good

yeah okay um another thing dany mentioned about this before is it might really help to mitigate some of the sync issues if we encounter any future hiccups on the test net or the main net in the future uh because you can kind of reject all other branches except this one specific thing that the user is inputting yeah i mean that that base ability um is definitely a coordination tool beyond just the weak subjectivity security requirement so it does open the door for conflict resolution if we need it okay um aditya and i will

make sure to give uh this some love in the first few days of next week so that we uh don't remain blocked on that okay networking like i mentioned earlier in the call we're going to do a call on wednesday there's a number of things to discuss v11 updates uh progress on param choices um along with um definitely some like potential dos vectors that proto's been investigating that kind of sit right in the networking layer along with some of the network monitoring tools and other things that people are working on that said are the things that people

would like to discuss today okay it works for me um expect discussion any questions or comments on spike things in general okay and closing remarks does anybody have

anything they'd like to share or discuss today okay quick meeting thank you everyone um sincerely keep up the great work um i know y'all are working your asses off and i know that the clients have gotten much better in the past month so keep it up thank you everyone thank you thanks danny thank you bye

everyone thanks bye [Music] [Music] so

[Music] [Music] [Music] you