All Core Devs - Consensus #49

Transcript

[Music] [Applause] [Music] so

[Music] [Music] [Music] the agenda is shared in the chat and if anyone on youtube wants to tell me they can hear us

let me know okay cool um so today we're gonna primarily talk well i'll talk about whatever we want to talk about uh but we'll talk briefly about spadina and zinken african correct me on that pronunciation um and general updates and kind of figure out things from them um testing release updates i am personally working on a v10 candidate release there is a branch up and like a work in progress pr this includes the lsb4 update it's going to include a couple of networking things we've been

working on um and i'm doing a pass on the consensus test to see what holes there might be i believe that proto has been continuing work on the networking test stack but has uh it's kind of continually interrupted by other higher priority items that show up and as for the um for choice tests they still sit in that independent repo and you can consume that if you want the product any updates on networking side on the testing you know so it kind of suppressed so up until they'll go i was working on this thing where you could have

nemo files define different clients and test network setups to then have some kind of integration testing but then this kind of starts because of higher priority things i've been reviewing clients lately but without results but this will take more time and i'll go through disclosures and things like that great uh maddie you have an update for us right yep thanks danny hey everyone um i just pushed a new fuzzing blog post uh describing the latest uh updates on beacon fuzz about an hour ago now i will link it to the chat shortly um we've been doing some bls

differential fuzzing and i guess um we can disclose some of the bugs that we found uh because they've been uh patched um so we identified two critical bugs on blast the supranational library that um can effectively allow signature malleability um so they've they've been fixed they've been patched we're all good uh we've only been targeting the rust findings at this stage and hopefully the next few weeks will make some time to have a crack at the other languages as well we've been running our fuzzers on a dedicated aws infrastructure we caught two minor discrepancies between clients on slashing processing so both the tester slashing and they're not directly exploitable per se

but um they're pretty pretty interesting um highly recommend going through the blog post that i'll be linking shortly i guess the good news is that we're seeing less and less exploitable vulnerabilities so good job everyone we've also added teku to uh beacon fuzz v2 our structural differential fuzzer um it was not straightforward but we got there in the end thanks nat for your all the all the hard work there and thanks uh the taker team for uh for their support throughout the process uh we also plan on starting the work on our own custom fuzzing engine to replace sleep fuzz honk fuzz and afl and that's about it i'm happy to take any questions what's the advantage of a custom custom

budgeting engine here so there's a few things in terms of coverage uh this will allow us to rotate and cycle through beacon states so the way it works at the moment with afl and lip fuzzer we're only mutating the actual uh ssz container that we're processing so if you're fuzzing the process at the station functions for example you're effectively mutating an attestation container and by having our own custom uh fuzzing engine we'll be able to essentially dictate how often we want to rotate and uh swap beacon states another benefit is going to be fusing speed so uh we'll be able to load the beacon state

in memory and not have to perform um ios disk ios for each implementation that we that we fuzz and also we will be able to have both structural fuzzing and mutation based fuzzing within the same targets so yeah keen to see how that will impact coverage and we'll be working on this next few weeks got it thank you any questions from eddie uh yeah one question nonetheless which implementations do you plan on on supporting for the bls

housing um i think we'll focus on blast seems to be the dominant library amongst all client implementations so the idea is to make sure that the bindings are sort of um the same across all languages but the current bls fuzzers that we have exercise um apache milagro uh blast and also the z cache library i think lodestar is using mcl because uh it's only library with a wasn't target yeah that's right yep um it is something that we wanted to do

uh adding uh harumi uh we didn't get to it um hopefully we will great thanks maddie um other testing release items great okay uh moving on to spadina uh spadina actually looks very stable uh once we got about 70 epochs in um but as you were all very aware i had about one third participation at

genesis and i think this was the compounding of a number of um errors kind of in the outer configuration and release process um i don't know if we need to belabor the point right now uh but the um i think the result is that we are going to do this again uh zinken and i have a blog post coming out uh shortly uh the launch pad will be up today genesis deposits in the next week and a nintendo genesis on monday the 12th obviously there's always the chance that validators just aren't going to turn their notes on

um i don't think that's really what we're the issue that we're looking for i think we're instead looking for a lack of kind of errors being reported by validators that do turn their nodes on uh which uh after the first few hours of the spadina launch became pretty clear if you're monitoring some of the discord channels that um you know there were a number of validators that had trouble getting things running and not just because of their own personal issues so um afri you had some thoughts about maybe doing a call outside of this call to just kind of discuss release processes and things you want to just give a quick on that if you are available i know that

sometimes you don't have the ability to talk on these calls yeah sure um i i think i had these thoughts and i um arrested a couple of weeks ago i think early august um i will strongly believe that clients need to have some kind of stable release management so that we can have some kind of code freeze and work towards uh hardening these clients and um uh yeah i i was offering that we can maybe have a breakout call for just uh talking best practices here uh uh because uh most of the clients uh never have

been used in production uh so we need to get towards stable releases if we plan to release the main net soon and yeah i was just offering that we can like a short breakout call um i'm down i would suggest maybe we do this early next week um i'll you and i can suggest the time and see if people are available yeah sure great um anything people want to talk about while we're here on spadina uh i know client updates we'll probably talk about some of the individual issues that we saw um or anything is moved towards zinken

i i have one more comment um we we saw that two days guinness delay is quite quite of um short and i was wondering if there are any thoughts in uh maybe in increasing the default genesis delay significantly for future networks or maybe even the default value i guess yeah so we um as you know to set the config up in the zinc and release we're going to do a four day and that's also that's partially to have a longer delay and partially to kind of keep things moving and not have a seven day delay but in my 1 0 candidate branch based on experiences with the two

day delay and kind of just based on what we expect with genesis we're going to move towards a seven day delay um as the default in the mainnet configuration and if we do any sort of rehearsals between now and genesis i would i would suspect that we do the seven day delay so i think everyone is generally in agreement that the two-day scramble is not great so default more for sure some users were confused by the delay also so maybe we need better communication but some said oh i deposited on monday and i'm not in genesis why right

now in a few places um i have a um the zinc and blog post i'm going to put out right after this call and i have a few dates noted very explicitly about uh and hopefully that that will help with the communication but yeah there's definitely some communication uh confusion there and maybe one generic comment would be that um like maybe client teams need to think how they keep in touch with their users so that they can quickly notify users when there's a problem with a release and that they should update i don't know i feel like that was probably the most yeah the biggest issue

yeah getting it off the ground absolutely and a concrete follow-up on that is that in a couple of different conversations i've had with uh staking providers that have provided services in other chains they said obviously there's a lot of noise you're following github you're following discord you're following twitter but the one thing that they always see is if um some software they're using has in email lists and email updates they kind of religiously will check their email um so an opt-in email list might be a good way to get get in touch in case of uh critical issues yeah and maybe display that list on um startup of the client or something so that you really

remember that this is like an essential thing to sign up to if you are serious about staking so i've been collected the uh some feedback of the spatina rehearsal lunch and this is the notes that i've shared to some people and so thanks for people for giving the feedback and if you have some more input to help us to improve the progress and feel free to comment or just edit the note um and also um uh

thanks to people on the east stacker discord that we suggest that we can have a checklist for the stakers to check um what else uh what thing i have to do uh before genesis is generated or after the genesis is generated so i and we are the if researchers are writing drafting the checklist and um hope to share with the clients maybe next week yeah great thank you

anything else on spadina or zinken um and we will plan on maybe having a breakout call on monday tuesday about just release processes and information sharing around that let's talk about some best practices okay great um moving on client updates let's start with prism hey guys terence from prison perspective labs so after spadina we had the postmodern and we did share this so if you haven't had a link please check out please check out

our discord so we basically reveal the items that we thought we could have done better such as being more prepared in terms of the dogs and better communication and the more vocal to the stakers so we created a general testnet readiness checklist and then we shared that among the client teams and we're looking for feedback on that we also align um the code to spec version 12.3 and that's running on master right now we added the wii subjectivity checkpoint support so we input the blog root and the epoch and when the node seems to that point and and say that block rule is different if hit up it will fail right away the volunt

voluntary access feature already about the client and the doc is ready on that so feel free to try that out we also go through our second round of audit so we're getting all the feedbacks coming so we're going um we're fixing the box and then enhancing our test suite of that so last but not least uh just uh just um general but fixes so yeah thanks great thank you terence um nimbus hi um so we had our last phase of audit that ended uh next step is fixes and a fixed review

that is scheduled uh two weeks from now we also uh fixed um and significantly improved stability uh with a lot of resource liquid fix down and now according to estats.io for those who participate um nimbus is apparently using less memory on average than other clients we are also in a heavy round of polish so that the user experience is significantly improved in particular regarding to logging and dix disk space and like prism we have our weak subjectivity sync that is a

work in progress it's now living in a pr and we also updated the collaborative repo with some medasha trusted checkpoint so that it can be tested cool thank you mommy um lodestar hey uh so uh we participated tried to participate in some small way on spadina we were running uh four nodes and um our kind of our progress is that basically our node is still not stable

and so um we were unable to participate with these four nodes um we were able to create the genesis properly but uh we're able to stay up to the head so um that's just ongoing work we're working on to to get our nodes stable um as far as what we've been working on recently we added weak subjectivity chain start where you can pass in a state unknown finalized good state and start chain from there uh we finished up the standard api implementing all the endpoints and we've been doing a lot of refactoring of our chain

just kind of the core of everything adding custom errors um and just cleaning things up and going forward with the things that are still in progress and we're going into or our gossip sub 1.1 integration still not done almost there and we're working going to be working on slashing protection interchange and the disk v5.1 upgrade as well cool thanks kevin do you have an intuition on the current bottlenecks in sinking spadina uh yeah it's um like i think that

uh just our sinking strategy in general hasn't been too great because we've not really known i guess what's happening to the blocks as we're feeding them into our chain and so i'm hopeful that um like now that we're getting kind of proper errors as to like what's wrong we'll be able to know okay we don't actually have a the parent so we need to like we can try to do like a parent sink backwards or we didn't have enough information to really make a smart uh syncing algorithm but now we have those errors we have um some like rough peer scoring um and so i think it's going to be a lot

easier to revisit just the whole algorithm it great thank you uh lighthouse hello um so this bedina launch went pretty smoothly on our end uh our validators are performing quite well as can be seen on the fly valley they're a little broad i guess the only hiccup is that at least one user didn't set up the data there correctly and as a result couldn't find peers uh their node was still using the media setup we are working towards uh pre-made net audits from trail of bits and ncc both starting next week and we're planning a big breaking release a v 0.3.0 release which will include the

standard http api so we have the partial implementation partial implementation ready and paul is working towards the full implementation i will also include the validator client api that will be consumed by our web user interface will incorporate a new directory structure which will hopefully mitigate the hiccup i mentioned earlier um we'll include as well the slashing interchange format that michael has been working on herman has also been working on a remote sign-off for the valley there a client we're hoping to uh integrate that into this release as well uh same as the other clients i guess been working on a basic weak

subjectivity verification and some caching for validator keys to mitigate the extensive uh script loading times with large validator accounts uh in networking land we're up to date with 0.2.3 age has implemented this v5.1 or i guess up to the changes made to the hey by felix um it's not in lighthouse yet we still want to do some more interrupt testing before uh we've been testing our gossip sub 1.1 scoring parameters on medasha and spadina uh edge also just completed a few hours ago the upnp support and we've also been fixing a couple of sync bugs

that's about it cool thank you maddie taku as me apologies if i'm slurring my words i just had a tooth extracted and half my face is numb forgive me the main things uh we've been working on discovery v 5.1 is basically done i am not not yet running by default but nearly there by the last few updates a lot of work on weeks subjectivity sync we now have hidden cli options to handle the checkpoint we have internal weak subjectivity checks in place that

we we check the the the sink against a given checkpoint we check that our peers are consistent with the checkpoint and we handle the case that the last finalized state is outside the week subjectivity period uh next step is to be able to start syncing from a given checkpoint state itself um we've been implementing some validator performance metrics uh just you know in our console for sort of immediate feedback and of course doing all the spadina uh launch prep um we got our draft audit report from quant stamp uh yesterday so we're just digesting that nothing too scary there so it should be fairly straightforward to deal with

next work is um finish implementing the standard api and finish up make our kind of validator split out uh bring it up to production quality um that that's it for for techyou the australians seem to be in holiday season at the moment so we've had a little less progress than usual gotcha thank you ben and uh congratulations on the tooth pull thank you um and looks like we have another mine today right or no we don't never mind um great okay i think that was

everyone um moving on research updates um on phase one spec stuff there's still a pr i think i mentioned this last time the intention is to we're going to leave that pr out of dev and out of master until we do a 1-0 release just kind of keep things cleaner but we are now generating um phase one tests i think those are out in the last release so you can check that out if you're following along progress there um otherwise uh other yeah mommy please uh just a quick note about the phase one

tests uh they made us an issue on azir because they take three gigabytes and we went over a budget um three gigabytes one over what which budget uh when decompress uh you have uh 10 gigabytes on azure so if you want to use phase one tests um you likely need to un uh uh maybe it's better to separate phase zero and phase one so that we can download them separately we also um this highlights kind of a fundamental issue and i think we're just going to

keep hitting it um about the structure of our uh the tests and how there are largely especially in like start states there's a largely duplication in that data which i think we're hitting the point in which it starts to become a problem so we might spend a little bit of time after phase zero launch thinking about some of the structures and testing and how we might de-duplicate data so essentially have like a repository in the tests of start states and reference them instead of dumping them in every single test um so but that's not going to happen in the next two months but thanks for the uh new warning there um well one immediate solution here or

about hotfix uh is to compress the stats because most of these stats just discontent a lot of serious right okay um but on to actual research updates um does anybody have some stuff sure um uh here let me see you just i want to report um i don't know if this is free social network um but uh basically we continue to work on

the crowley um network roller uh and we are based and basing this on um we do um we are managed to get a certain number of um fears and get the disservice messages from the topics um i will be posting uh the link to the the crawler that we are developing um and we have been we have we found a couple of small bugs that we have already commented to proto and they have been fixed uh we also propose a couple of features and and changes in the gossip messages and we are working on on those currently

so that's for myself cool thanks to you um on the phase one uh east wanting to merge work um i think kind of had a three to four week hiatus on digging into that stuff once we got the the demo done people working on other work uh but we've identified a list of like five or six items um that are the next things to attack on the r d um so we'll be digging into those and probably writing a little bit more about what's going on in each research to update people all right anything else here

great networking um there's still kind of some movement around the five one uh this five one um i know that felix is trying to get this thing merged and get that wire format uh completed very very soon but keep your eye on there if there's any place to give some feedback and then do some interop testing other networking items okay any questions comments about spec

in general fantastic um any anything anyone wants to talk about any lingering concerns any issues they want to bring up without thinking anything at all okay great i'm going to put out a blog post about thinking today and um we will handle things moving forward next week and plan for a genesis on the following monday october the 12th

thank you everyone i will talk to you all very soon take care [Music] thanks [Music] [Music] [Music]

so [Music] [Music] [Music] [Applause]

[Music] you