All Core Devs - Consensus #54

Transcript

h [Music] [Applause] [Music] [Music]

for [Music] stream should be transfer

hopefully things are quiet on the chat it was kind of chaos last time all right um here's

Danny Ryan: the agenda I do expect today to be short but I guess you never know on testing releases there is some activity on the spec repo um there are some minor features that are being worked on that we expect to come out in something of like a minor hard fork early mid next year um the actual

discussion of what's going to go in there I'd say let's wait until January um we can spend some time making it kind of clear what the intentions are and we can debate and go back and forth a little bit um I think the lwh hanging fruit here are a little bit of accounting reform um on how attestations and things are stored in state adding a sync committee or light client sync committee which reuses some of the generally similar committee functionality both on the networking level level and in the in the spec to add a um like Client First support and some other small stuff but again I think

our time we better spent formalizing some of that stuff writing it down getting people up to speed and then we can discuss it in January um additionally there's ongoing work in the spec on

Vitalik Buterin: I should interrupt in just those specific ISS like there are um GitHub issues and PLL requests and the E specs that you can uh look at already so on the accounting reform stuff if you just go into the poll section of you two spec so the the most recent recent two from myself I forget the numbers but one says like accounting reform and the other says um quotient um reform um or something like that highly recommends for client developers to look at them and at least famili familiarize

yourself with the proposals and comments if have things to say

Danny Ryan: right and some of that in there uh moves some of the state accounting to the way it's generally optimized on Epoch transitions anyway uh so I think this would be a net benefit helps for things like empty Epoch Transitions and stuff which are currently probably a a Dos Spector um cool uh so ongoing work there uh there's also um to make this testing a little bit more sustainable I want to use a little bit different model and I expect that to come out early next year uh didn't really want to pull out the pull the rug on you on testing infrastructure right near launch but

um that's where at uh let's see anything else on testing and releases medy you got anything for us

Mehdi Zerouali: not much crashes lately which is good I guess um we had to fix Pixar fuzzers after a few client updates messed them up still have one um small issue with Hon fuzz um mutation based fuzzing engine not the structural one it's spitting out way too many false positives to be useful um so we need to basically Implement some timeouts to give it enough time to initialize all the clients

um but

Mehdi Zerouali: yeah it's been good we've been working with Parry um on deploying the fers to the AWS infrastructure he's got some great

anible and terraform scripts um but yeah pretty pretty quiet I'd say um no crashes to report which is good

Danny Ryan: good thanks okay anything else here

Hsiao-Wei Wang: um so uh about the test vectors um I have a proposal about adding the uh BS calization test uh posting here um think I got some feedback from the Cent and the BS um the P the library

implementers so I think it will uh continue to implement it in our test Vector but uh the the interface might change if I found something new but basically is the test will will uh focus on the three significant bights and the verifications

Danny Ryan: great and this is after you found a couple of issues in PCC around some of the serialization right shall

we

uh yes thanks to CR and it's our uh is to B Hunter and helped the IC a lot

Danny Ryan: great okay let's move on to client updates we can start with

te hey

Cem Ozer: everyone uh from the te side we've been focusing on optimizing Epoch processing uh creating the first block of the epoch is still slower than others so there's potential for further

Improvement but it's significantly better than it was and we're not like we're not seeing blocks get orphaned on Pon anymore uh attestations for the first slot is also now much more accurate because block import times are faster uh We've modified E1 Genesis block search to be more tolerant of missing historical blocks we' reduced be can say backing tree Heap consumption it will significantly sped up as a seed serializing serialized and we fix fixed a bug which caused TECO to produce invel attestation Aggregates that's about it

nice and prism

Terence Tsao: hey guys Terence here so we've

added uh database backups at runtime for both n validator client we're also supporting um reading graffiti from file and this is for the validator side uh this was a high request our user we have Pier scoring enabl by default we updated it to goand 1.5.6 and with dis IO for Val client and this results in safer attestation safer ATT tester sling protection which is great last but not least we've added e one fallback option for the uh for the um e one node and this was also highly requested by by our user and that's it thank you

I'm

Danny Ryan: curious on the fallback option um is that if the API the connection fails or does it try to detect if maybe the the main option is following the head

uh we're just doing a API protection fail got cool yeah I wonder

Danny Ryan: I wonder um maybe if you like look at peer count or some other stuff that might think that it's bad I I was just setting that up the other day and didn't know exactly what was going to happen but thank you uh Lighthouse

all

Mehdi Zerouali: um first of all apologies for missing uh the last call two weeks ago uh we been working on enhancing our

slasher so we fixed a um Mis propos A slashing bug and we added the uh option for users to broadcast their slashings uh we fixed a couple of IO failures that were uh causing potential uh database Corruptions uh I think like a lot of people there's been some F1 client issues so gu Works uh seemingly and seems to work really well but um everything else seems to struggle so we um we also added an F1 fallback node um a little while ago and we also now have a manual flag for users to purge their E1 Cas uh for doy clients um we simplified the beacon lock files uh we're now using OS uh locks um we almost completed um the standard HTP API the

server scent events are implemented now uh when then corrected um some light attestations uh we've been testing and analyzing gossip cell data metrics and and scoring so we found a little discrepancy in the expected message propagation um which will be corrected very soon in future PR um and yeah we're currently working on reducing our block proposal time cting it down from 500 mes to about 100 mes um Paul and Benedict are working on a beacon node fallback so kind of similar to the F1 fallback giving users the ability to specify multiple Beacon notes um and we started experimenting with a new scoring

parameter um to essentially help detect sensoring nodes and that's about it

Danny Ryan: great

Mamy Ratsimbazafy: hi so yesterday we released uh or uh version 1.0.4 uh we didn't announce it but we will be announcing it soon though some uh users found it still so the biggest change that we had were um properly handling uh the termination signals uh for example making sure that the slashing protection database is correctly flushed on disk and uh closed uh we added also uh new uh performance scoring so that you can uh

check uh if your system can keep up with the small requirement of the if2 chain and um also importantly we fixed uh some uh a bug where we uh didn't F aggregation we received and we forwarded them to uh the network and we ended up uh getting bad score and we also fixed isue in Gossip sub which led to um a slightly worse attestation Effectiveness so uh right now uh we are focusing on a couple of performance bottleneck one is a dis iio so uh when we uh we have some IO when we do

attestation of block proposal that we would like to reduce and uh accelerating uh State uh transition via having something called State diff instead of uh storing the wall uh the wall State and um lastly multi Fring um that we plan to have we have The Primitives done and we plan to uh refactor uh or uh verific benefits from multi threading at two different levels and for maybe three weeks from now

Danny Ryan: great and

Cayman

Cayman Nava: hello everyone uh so last few weeks um we turns out we hadn't implemented the API the standard API but uh we implemented those two thing two uh endpoints um we finally got on implementing and integrating the BLS batch verification um which really helped helped us U and we've just been going going through um profiling um Runing Beacon node and finding finding low hanging fruits some things as simple as you know like in Li P2P getting peers in in JZ Le top is

kind of a takes a while so um just doing our hey did I lose Cayman or am I disconnected um we also the lost you for about like 20 seconds or maybe 12 12 seconds oh your internet connection is unstable um yeah um uh sorry was last thing you heard BLS batch verification profiling uh how to get PI right um so we're just doing last try

to get some final things together um that are really stopping us from having a responsive Beacon node all the time um and we've decided to to cut cut releases every two weeks so we're just going to do like a a um that kind of release schedule so that's it from us

uh can people try load star on Pont

Cayman Nava: yes so it works um I will say currently um so you could pull it down from either Master build it or from npm but um one thing uh it will like kind of pause every few

minutes um but it will it will it will stay on the head but it will not be responsive at all times so uh we're we're working through those last things

Danny Ryan: great I lost my agenda page but I'm pretty sure next up is research updates anybody want to go

Vitalik Buterin: um so on the um shorting front I the not sure if I mentioned this in the

previous call but the well but the data availability sampling sharting is now a uh APR it's also one of the very more recent ones in the APR list uh so that includes the uh the beacon chain transitions uh and there's also a GU a separate document that the that the pr links to that has been around for a while that basically talks about like what it how the data availability sampling Works um and in Dan gr has been doing some excellent work on optimizing proof generation making it look like

generating the inclusion proofs for or or the correctness proofs for branches of a block is going to be much more practical possibly some um somewhere around um eight to uh uh eight time eight times faster or more than we um than we had thought before which is a potentially good news because it just killed because it reduces the headaches that we might expect from needing from a block proposing otherwise block proposing just be a a significantly simpler process

Danny Ryan: can you help us understand the complexity of implementing Kate commitments and of like what type like how much is built

off of existing BLS libraries and things versus

Vitalik Buterin: uh really good question I think one really important thing is that you need to have a library that exposes BLS operations so multiplication and addition being the most important ones and ideally you want this fast linear combinations AKA multi exponentiation which basically just takes a whole bunch of points and a whole bunch of coefficients multiplies each point by the Coe sums them all up and gives you the output basically there's of optimized ways of doing that that are potentially up to something like 10 times faster for or for for large inputs than doing it the kind of

naive way so if you have libraries that do those things saying if you don't have the fast onear combinations you then you could also just code it yourself so if you do have things it's actually not that hard and so making a c commitment for a piece of data just literally means step um you have a kind of pre-existing set of points which would basically be the Trust basically an fft of The Trusted setup pre compute fairly easily and you just take the first object you just basically do a fast linear combination of the setup to get um to get Block so so you do the first point multiplied by the first piece of data plus the second

Point multip by the second piece of data and so forth so commit data is easy um gener verifying a proof is um and inclusion proof is also easy and basically it's just a single pairing check uh and now generating all inclusion proofs is thing that danr has been working a lot on optimizing but the good news is that it's still something like 40 line or to 50 lines of python code or so and so you can probably just take the uh take the python code and translate it so my opinion is that it really it should not be all that

difficult actually

and is

Danny Ryan: this qu is this a known enough quantity that it can be worked on today

Vitalik Buterin: I think so

uh just

Dankrad Feist: once again if we have the BLS Library support right I mean I think uh for blast I think you have to wait for them to expose those

Mamy Ratsimbazafy: uh I can give you uh some comments about the library support but uh just to make sure we don't need f F because it's simple enough that we can just use multiscalar multiplication right

Vitalik Buterin: um correct so the fft is needed in a couple of places so one place is that to generate all all inclusion proofs for a block so to

generate n inclusion proofs in um n log n time instead of in N squar time um you need the the fancy um algorithm which Dan C of created by adapting this earlier thing from Demetri and one other person and that uses ffts internally and the second thing is that the um CAD a trusted setup that you normally see um comes in the form of a sequence of powers but in order to just commit to a Blog you want one that comes in the form of being a sequence of evaluations and converting from one to the other also requires an fft

Mamy Ratsimbazafy: okay so in terms of the state of the BLS libraries that we can use remember

that uh with my very first talk with supranational back in April that was one of the thing they wanted to look at the multiscalar M but I don't think it's ready yet however uh consensus has a library called gof and Gnar for uh snarks and 12 uh 381 uh and they have the multiscalar multiplication that is also multi threaded and that they optimized heavily and uh the zxa project has also so a library in Rust uh recently they renamed to artworks uh and they also have a heavily optimized uh BLS 12381 um including fft um and

multiscalar M and I think it can also even work on GPU

but

Vitalik Buterin: oh excellent so and looking at the the libraries that are being used for snarks definitely seems like a potentially a fruitful route to take um but there one thing I want to stress is that there definitely is value from working on the um Kate pieces sooner the reason basically is that um we we really wants to just get the statistics on how um what what is the runtime of one of

these things in practice um basically just um like if we have that statistic it I think it significantly der risks the whole thing

Dankrad Feist: also um I'm very happy to uh like create a little intro to the fft on group stuff like the method to create all the proofs um like so if as soon as anyone is kind of saying they want to start working on this I would yeah I would create something to like get them on board

Danny Ryan: yeah I think maybe in in January it might be a good time to do some various like and knowledge transfer so we can all kind of get up to speed on this and some other

stuff that we're looking to do next year all right mean to totally

deril either

Vitalik Buterin: um I think that's the main the main new stuff happen on the research front um no can't think of anything else

though

Danny Ryan: okay uh anyone else

Justin Drake: I mean related to the data availability sampling I guess one aspect is the then possibly you know slightly modified dhts with new properties like

low latency and I guess one of the things we're looking to do is is higher a networking expert could who could maybe help us at the FM Foundation investigate these issues so if you know anyone please pass them on

Danny Ryan: all right um any other updates here all right great

um I might have had my mute backwards there any other updates all right um great we had a networking call one week ago uh there's a to-do list that I need to get to thank you for taking notes Ben um is there any other uh networking components people want to discuss today and general spec discussion uh like I said there is plenty of movement on the spec I think it it makes sense to have at least one person on your team keeping up on the various developments and engaging and will have plenty of good stuff to talk about on maybe like a

first hard fork and like I said I think it'd be a good time to maybe get on some calls and make sure we like write down a l a lot of stuff to just do some knowledge transfer and get people up to speed because there's plenty of um engineering R&D tasks to begin to um tackle here other than that anything people want to discuss today

Vitalik Buterin: uh do we want to talk about Fork Choice things at all or is that just the kind of like keep quietly working and figure it out

sort of

thing

Danny Ryan: um let's uh I'd say let's pick that up in the New Year

Vitalik Buterin: yep makes sense

now

Vitalik Buterin: I mean for anyone listening the quick update is basically that there's some multiple small tweaks to the fork Choice rule that we've been mulling over in order to uh address all of the issues that some academic people have found with the fourth choice role like basically some theoretical issues with livess so yeah we will like the idea the the ideas on exactly what to do have been around for a couple of months now but just kind of pushing them

forward to something close to a ready to go State and kind of underneath

Danny Ryan: that is more of a a general just contemplation of livess in general because as you aware the fork choice you know is lmd Ghost but then has a number of these like tweaks and fixes to make it more alive under the attacks found um which is a game of whack-a-mole which is maybe not that like okay um anything else at all that people want to discuss today before we take a look off at least for this

call can we talk proile for 40 minutes um yeah yeah you can start us off balic can you start us off with a 10-minute diet tribe yeah okay fair uh thank you um excellent work as always happy to see main net very stable and um have a great holidays talk to you all soon

thanks every thanks all cheers

Dankrad Feist: thank you bye

bye

[Music] [Music] Happ

[Music] J [Music] [Music] [Music] [Music]

[Applause] h [Music] for

[Music] [Music]

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.