All Core Devs - Consensus #14

Transcript

[Music] [Music] [Music]

[Music] [Music]

[Music] [Music] [Music] [Music]

[Music] okay great the stream is switched over and the guys on the YouTube chat box

will let us know soon for the confirmation please okay welcome to to implement our call number 14 we have a lot of us and a decent-size agenda so we will keep the client although we have a lot of exciting client updates we'll keep them brief and try to get into some of the agenda items and we will start with testing I've been doing a lot of work on testing one of the big updates coming in be 0.5 which is to be released later today or

tomorrow is that the spec is now fully executable the Python spec and we've done some work to pull that out into like an executable file and run some tests against it and we have we now have some state tests against the actual executable spec which is I think a major achievement and going to allow us to I'll actually be writing the same code soon cool so I shared a sample of that file in the kit ER last night and I'll be getting more of that together today and tomorrow as he released the b05 this

conforms to the standard that I posted to the 2o generators repo a few weeks ago some of the details are probably still up for debate and but now that we have kind of a general format together we can iron out some of those details how many days cool any other testing updates yes kitchen go ahead Dimitri and Paul um shopping PES is updated too zero point four point zero and it's just released thyself and POS fixed as count

Ming soon great and I believe the shuffling algorithm has not changed between 0 4 and 0 5 so that could remain stable to emerge that okay any other updates great we will move on have a client updates Pegasus you know serves off yep and this is gem from Pegasus Artemis so we successfully integrated LMD goes to our state processor we started

working on our honest validator client instrumentation and currently we can successfully generate blocks which in turn our beacon node successfully run state transition on we implemented a data provider inside our client to output information to it a CSV or JSON files we started profiling our code to look at performance bottlenecks and visualize the call graph and lastly we're working with that call from white block and Anton's wound on getting a test net set up and implementing the hobbits wire protocol great thank you so much about prismatic incorrectly proposed and throughout their lifecycle

and people know concurrently process not far apart we fix a few bars here and there and they're mostly related to for the written they're mostly related to boundary conditions and we also started testing a multi no setups by multi no setup I mean there's four validate this connectivity you know there's other favela this mid without people node and then we made sure they sink in war and then the Bella data's can fail over you won't be hotels so so so we'll so we will have more update on that next time for the test result and we also implemented lmd ghost framework we have some preliminary batch numbers and what we to

optimize it so we're looking at our photos with reports in series since we did a great great job on that other than that yeah just basically just testing yeah great thank you how about youth um so we've kind of taken a break from working on the actual client and we're currently looking into implementing lippy dopey and Swift so we can start working on networking stuff cool thank you

parody priorities so right now we are mostly focusing around testing I think we will continue to do that at least for recently so the rationale is we still plan to do a lot of refactoring for the runtime but we kind of want to make sure that our refactoring is correct and we don't accidentally break since so of course matching the and the who state test is our unco but right now we are just trying to do you need tests so so

we're basically just manually getting the license back file and try to random and Carol the output and put a function to the output of the Python fell in flower last codes it mostly works and we also try to whom this is not related to testing but for the other thing we are trying to move adapter layer for our subject from is in the execute block from above the execute block so say execute block function can be like standalone without substrate so this can

hopefully makes a runtime more flexible in different situations so that's for us awesome thank you have a lodestar hey sorry yes nothing likes major we pretty much scaffolded all of our like auxilary stuff inside the client like the database like no we're p2p stuffs gonna handle and all that other good stuff we pretty much we finished up all our state transitions and we're gonna start like actually testing everything kind of

with a fake 1.0 1x deposit just we we're almost we're pretty close on getting lip p2p done JavaScript we ran a few PRS and hopefully once those get done we have a better understanding of like where we stand with that and we're working towards we're starting up the validator client right now other than that it's kind of a wrap awesome thank you harmony we've been working on a simulator and released its first version this version is based on this back as is the young optimization we have the errors improved

shaft and algorithm simulator works well with hundreds of validators but with thousands of them simulation becomes insanely slow this is basically because everything is running the single thread so far the next step would be to make work with let's say 50 thousands for the daters and due to some benchmarks on these big numbers this is a work in progress so far during doing benchmarks on a small number for the daters we have found several bottlenecks and already sorted

them out these bottlenecks are not even related to this packets some issues with our implementation for those who interested in trying out the simulator I'll post a link and if feedback would be much appreciated also we started to get integrated with distance use made a small PR it fixes shufflin fast generators and also encountered an issue with sharply Schieffelin swaps and mandates yes we will talk either about that so that's pretty much it great thank you and if you've implemented the spec as is as you said are you caching the shufflings perky bug no okay

that is probably the the most major bottleneck I would one of the one of the most major bottlenecks probably the easiest things thing to resolve at this point yeah that's this what we have already discovered yep cool and okay all right thank you let's light house yep so we've been building at our runtime I'm working on syncing we've been trying to get kind of DeFazio a wire protocol in very roughly aware that it's moving target we did a bunch of benchmarking so we made a document it's in the in the PM reaper

like the issue for this call which has a whole bunch of information and how it's useful to people that are trying to optimize we did 16k 300k and 4 million validated benchmarks we've got a full time resource on the validator client we've got a part time resource I'm working on an efficient operations for its it for us great thank you lembas hi so let's start with Li p2p we've been working on the p2p named to replace go demon so on that try deep restore are done we are considering

bounties to progress further and we're working on it refreshes and fixed or discovery on whisper testing now regarding lipid to be in the beacon chain using lipid 2p demon so we right now use the Arabic X but we also have a stimulation that kind of work using a lipid to be demon and we will provide an option to select both lpx only p2p for test net on sink for the beacon chain we have an open question on the handshake and we want to we are using arrow peace organization for the wire protocol that

we implemented and we want to kill error be in that regarding the state in the past month we froze or spec to 0.3 we moved to 0.4 this week and we are eagerly waiting for the 0.5 so that we are able to compare all block hashes with the executive of spec and we've been working on a Roth optimization and to remove production behaviors that we found in the become state and in terms of preventing bug we have been using an intact system to create like a pouch and slot types and have the compiler tell us

that we are mixing both so that's that's quite nice we have a dev a date blog post coming soon it's been written and it's under regime as you know last week I did a talk on at ETH CC on testing and simulation so I will provide the slide on the sobbing guitar channel like in two minutes and lastly a big attempt for the week was integrating naive LMG girls in our simulation because before we were using some kind of folk choice called

latest resolve block like the latest block will received we consider that it was the one to use and it's working fine we have a lot to do to optimize it but we are already considering a proto lambdas implementation and also bitwise and empty goals so next week will be on that and also on testing great and to be clear bitwise lmd ghost is not exactly the same it generally returns the same results but in certain corner cases might return differential so that's something we would only all use probably as a group

decision rather than as a lot yeah I've seen that but I think it's good to have like to know the performance trade-off oh absolutely yeah so yeah great you should be Eric yep convert bitwise ghost into standard down buddy goes quite easily you get the same optimizations okay ready hello and so this two weeks the spec

sinking and now we're targeting at version zero zero point five zero hopefully that's that's the one that we can finish the spec sinking and on the other side I'm going updating is that with move the POS module to the PCC library and since the us signatures verification is like our biggest bottleneck of in Python area so I'm trying to either it's as we can and also

the the p2p integration is still ongoing beside the Trinity implementation about there's two interface updates of the deposit contract and so if you using the latest deposit control every deposit contract repository as to PR has been merged this week wise here 21 we updated the deposit event interface and another one is the hundred - which updated the is to genesis event interface another

thing that you want to give update is that since there's a proposal about changing the hash functions to sha-256 by protein is very supportive left edit sha-256 you'd in function to the latest hyper release so yeah if that's something we are considering it's okay to change it in the contract thank you thank you cool did I miss anyone great I think I got it okay next I'm

going to read a quick update from Robert Paul apps he says we're working to add deprecation notices to the areas of the spec that are outdated we're making significant strides on the new Docs live p2p i/o and writing a non normative walkthrough of the blood PDP stack that everyone used as a reference and engaging in various debates on github Raul is generally very responsive if you have questions and things and he'll probably join us next time thanks Raul I threw the lighthouse benchmarks update on here was there anything of note that you didn't get to in your the lighthouse

update that you wanted to share with us regarding the benchmarks yeah yeah yeah I guess some I guess a trap for a lot of people is there's a lot of like like Oh N squared stuff in there like you fee if you just implement it straight out of the spec like especially inclusion distance it's pretty obvious but if you just do a copy out of the spec that's that can be a massive slowdown and we did a lot of parallelization with some all the processing in that save just a fair bit of time all right yeah I guess one of the interesting things we did was was do rewards processing as a parallel map to validated balances and the spec was really well designed for it but if

you want to check it out this and maybe some information and something you were surprised about was that decompressing a point in g1 cost the same amount as aggregating a point to points in g1 correct yeah yeah that's right so we we kind of had this problem where we wanted to processing a deposit we want to see if a public key exists in the validator registry or not so we were keeping a map of like a building a hash map of hub keys validator index and then when we're doing that we found that you know going to bytes was really slows and then we were like oh well why don't we

store the just instead of storing it as like the the point for our BLS library will store as uncompressed by its which made a hash map really faster then made our SSN serialization really slow because then we had to like compress it yeah so we kind of found interesting thing and I spoke to Danny about it um was just kind of wondering why that we're compressing the public key bytes when it seems like everyone's gonna need it in uncompressed form anyway the tell you you had a comment on that I think the only comment I had is that I don't think it would be a good idea to alter the spec to store em uncompressed

points basically because that would just require us to standardize serialization for uncompressed points in addition the serialization of our compressed points yeah sure right but good like clients given especially given that the uncompressed points doesn't actually store like and the doesn't really get changed or access like it might even be reasonable for a client implementation to just be basically as soon as you get the undepressed points you would just hash it store the hash for like triage purposes and that's and then immediately uncompressed and then store the uncompressed version yeah that's what we've started doing basically we just

maintain that hash map that deposit so we don't have to just do it all at once and times it's not a big deal for us anymore no one really have other more general thing to keep in mind by the way s like we start like exploring different efficiency trade-offs as like in this case I think it might actually be fine but it's actually for the beacon chain it might not be too bad but especially once we go into the shard chains keep like there's going to be a lot of clients that are running like five shards ten shards and fifty shards and so we want to keep in mind like what the marginal costs of being a validator on one shard are and one there's trade-offs like makes it like have a coup your model of like what

we think the cost is of a percentage CPU power versus a gigabyte of ram versus a gigabyte of SSD and like actually seeing whether or not it makes sense to go in some direction basically because I guess it's but we do have an important task of making sure the running a validator is like marginally cheap because it like if it's not that it's going to significantly eat into a validator prone in profits and and which could mcgraw which could both really hurt participation and encourage things like stake willing but but it's like sometimes counterintuitive look what

actually is expensive and what actually is not expensive yeah it's something that I did in the bench box was bench on my laptop and on my desktop it's quite interesting to see the difference between the two starting to look like just the way that we've got it written it seems like the more calls you have it really improves the speed for you so something to think about if someone has like a 12-course only I'd love to see benches from it the other thing the benchmark on is just probably a minimal cost of EPS again I guess mm-hmm maybe we don't mean we don't necessarily want courage people to stay convey

beyond VPSs but it's still like oh if you have to target if you have some code I can run like easily I can give you benchmarks on my 18 cores oh I think earth big fellah benchmarks talks there's like a bit about running the benchmark so I'd love to see it you can just copy and paste output if you want don't do the markdown table thing it sucks actually the other kind of like I think we should definitely we really should avoid the trap of running benchmarks on powerful hardware because like we don't want to repeat the mistake of ether ether one and like having really high spec requirements it would be good to get numbers on Nagbe $200.00

laptops we try to run those on Raspberry Pi as well okay yeah sure I've got some crappy laptop so I'll try then mm-hmm okay chose one yes yep thank you the if you haven't taken a look at those benchmarks do they're pretty exciting and a little wizard okay next up is leap seconds and time drift I believe Justin added a note about how leap seconds is handled in UNIX time and how were conforming to you next time I

believe and so there were a couple questions on on wire and what exactly is happening there and maybe a discussion on time drew general Justin can you give us just a quick on that yes so basically we need some sort of notion of time in if you're m2 and I guess we have two options one is kind of you next time which subtracts number of leap seconds so in and then there's this other thing which is much more esoteric which is called the international atomic time or something like that now I guess there's several

reasons why we're favoring you next time over this atomic time one is that it's it's much more common ly accessible in from programming languages so I guess ease of use for the programmers is one thing the other thing is that it's compatible with if if one time stamp I believe if one uses UNIX timestamps and you have a nice thing is that it it provides a nice invariance that at basically midnight UTC the the slot number is going to be a multiple of

fourteen thousand four hundred I believe so what one of the things that that we've done with with if to genesis is to basically have the genesis be at midnight UTC and so this invariant would remain over time if we take into account the leap seconds I guess by having this invariant did the trade-off is that we lose the invariant that every slot is exactly 6 seconds or whatever you know constant we set it to in the future i I think this is the the right trade-off to make but I'm happy to hear counsel arguments and to be clear I'm just gonna say that as far as

programming or ease I mean like this a lot of us are just you know literally setting six second timers and letting the slot stick I mean so so that that sort of surprised me he said that we were accounting for the eight seconds so so we're literally just gonna have to use a system clock or or something like that is that is that what you're suggesting here a timer without going back and checking some sort of system clock sounds dangerous and sounds like you would get much drift over time like right yes that's why I asked about drift to like the kind of bug that I would see from like relying on just for doing timers without it going back to a

system call I guess like what if thought it was at some point you have some your computer temporarily slowed us down a lot and it takes you like 19 seconds to verify some block and then like does that delay your whole thing by by 19 seconds so like we don't know you're like a loop that says wait until or some trigger that says we didn't tell the system time it's like six more six times then it seems maybe even just a few computer sleeps and wake up yeah by the way another nice thing about I just realized is that 14400 the number of slots in a day is also a multiple of 64

so it also means that we get like really nice debug boundaries every at midnight every time just one thing there is Linux 5.0 has been started working on fixing a year 2038 bug which is basically because the unix time was using you in 32 and in 2038 will run out of you in 32 so we need to be you in 64 right from the start time sync attacks it might be a stupid question but sax do

you mean things like attacks on in GPU yeah or like let's say that I control a majority of the nervous and I say that six seconds or my version of whatever time is becomes like the standard so there's two kinds of attacks there that you have to unpack right one of them is the meta content EP and another one is a 51% attack where the 51% attack brings basically a kind of implements a chain that pretends that the time is say two minutes in the future you know in the case I did write that ether research post and I talked about this at Stanford I think about Network adjusted time stamps which is basically like this hybrid between relying on local clock time and relying

like relying on like basically deltas between blog or weeding sometime after messages that you see from other nodes and I think if you just like see Google for a networker adjust the timestamps if the area I meet they're probably you can probably find it and I'm like I mean this is both something that probably should be implemented I mean unless people come up with reasons why it's not a good idea or why there's something better so I and I definitely like to encourage people to read the post and most of my comments on it the the second so that on base the kind of practical

effect of that is that it makes some NTP attacks not effective unless they're coupled with at least 33% attacks on the underlying systems or on the group of stake and then as far as 51% attacks with that kind of push time forward go that's not really like that's not something like that is also something that exists with proof of work and so we're not introducing any fundamentally new issues with reverse take it is like the kind of attack that's a bit more dangerous that I've thought about is a 51% attack where instead of being two minutes in the future you're like two seconds in the future and then you try to sort of pull a portion of honest the

nodes with you and that is the sort of thing that probably should be analyzed more by as I said it's like technically just as doable of current work as improve Stig okay another another possible solution is like implementation of like NTP v4 but I'll just like write something about it rather than boring you guys is doesn't ipv4 rely on some new like some trusted consortium or like what's the security model basic way no I mean but that's another thing is like if everybody needs to agree on time then that needs to be somewhat of a function

within the consensus mechanism itself adjust the timestamps kind of does that go ahead well without what I was sleeping is like we could actually implement base functionality using NT bb4 within the actual like wire protocol so when they're engaging in that handshake ones the trusted consortium being the validators yeah I mean that's actually like pretty close to what netwo what my network adjust the timestamp repose is like it's probably simpler than whatever the NCP protocol is but it's still but that's just because like

it doesn't need to have like email us like millisecond precision yeah well the thing is is that this is NT PV for this is something that exists now that we could probably easily transfer latency and things like that so you know well the wire protocols kind of agnostic to that as long as you're just accounting for what the definition of that message type is then that's there should be the responsibility of either the PTP stack

itself or the application layer okay okay okay so just to sum up we should just be using system time in order to determine or slots basically currently yes and currently using system time that conforms to UNIX which would adjust for leap seconds over time okay and then oh sorry oh I mean potentially an implementation might use some some some combination of timers this is some time that makes sure

that they conform over time depending on what you're looking for but again you need to kind of be going back just as some time on some interval okay and then and then we're not continue talaq said that we're not concerned about some sort of NTP attack just because it's no different than what you might have and improve of work right we don't we don't have we don't have special reasons to be more concerned about 51% attacks that have to do with timing than we did before but like proof of simian we do new who wants to like me we do want to take the timestamp problems seriously

because proof of stake does kind of depend on time stamps more so like if you can if you can get people to believe wrong things about the time you can cause more a tree you can wreak more havoc more quickly in proof of stake improve stakes that improve reward okay but yeah that's what my wrap my head around and then you said you had a you had a post about about Justin said that he posted in the night you know I'd encourage everyone to like read it and either implements it or come up with reasons why we should use something else I be also curious to see if this end right yeah just to comment on the

implementation of the simulator that that I did for the supercomputer implemented local timers for each one of the audio nodes and so kind of simulating a UNIX system but I also implemented some kind of global synchronization every five seconds of this local timer being five seconds a parameter that can be customized at the the configuration of the execution yeah so maybe this is some kind of combination that yeah mixes both UNIX time and the word time and I also saw some kind of weird things happened when design colonization period is too long

yeah some we have message from the future started to appear and some company interesting so with you if you were relying on timers for too long started getting pretty out of sync yeah yeah yeah so that's what I said at this point for most of the wrong side I said global synchronization of five seconds assuming that most of benevolent non attack nodes would have a maximum delay or time difference of five seconds and that is also smaller than the H dot time and so that could allow for more or less got it thanks very cool we're gonna

move on from the timing stuff I skipped research updates quick research updates if we have any yes we have a few months ago and Nikolas posted a pause on if research p2p about using quick for all Peters communication is new protocol initially designed by Google to improved Cpl to improve on TCP and TLS and we in the last few days we just replaced our you the UDP communication but quick and we wanted to see what will happen and we saw like the performance

degradation which we expected is because quick does more things it's and it does the encryption and authorization of authentication by default and also our like AWS instances are very small so we we are overloaded CPU but we didn't have bigger problems with integrations and also we are using the quick implementation by protocol labs they are still working on it is this feature of zero round-trip handshake which we wanted to check but it's not available yet they are working on it but I've we think this can improve also the

performance great so in general positive or negative yeah I think in general is positive I think we can keep thinking about using quick the thing is that I saw that now we will have implementation in the go and Google has implementation in C++ I didn't find implementation in other languages and was the additional overhead on the the initial handshake of the protocol or was the additional overhead on this continuous now so the thing is that we don't do session

management so we handshake on every packet and that's why I think this is the the main okay for the performance degradation yeah okay thank you any other research updates would research up updates include expect right yeah and of ready yeah a couple of cents back for my formatting issues that came up one of them is mid turn so that were doing shuffling in kind of two different

in two different ways at least as it's currently written in phase zero and in phase one and the distinction is basically like is shuffling a function of like as in via the the integer going in being the rule and the integer coming out being about being the validator performing the rule or is the function that the input is the validator in the output is like the index of the role that it's performing because in phase zero what we're doing is we're shuffling and then splitting the shuffle which basically says in this evaluator indices go in and girls come out but in phase one reviewed though we're using get per muted index which basically says that goals come in and validator indices come out so again like it would be good both

for kind of queenís and for just general total complexity reduction to understand agree on one or the other I would err on the side of doing what phase zero is doing is in phase zero is a bit it's kind of less elegant in that but we defined get Bermuda the index and then we defined shuffle and then we take a slice of shuffle so in some ways the stack has written kind of has kind of combines two inefficiencies in that it

doesn't yet get me an efficiency from doing shuffle with the optimal algorithm and then it also has any fish and see is because you're doing and complete shuffle and then taking a slice out of it so like I mean I personally actually find like it's not just the kind of forward versus backward a probe backward thing it's also just about the kind of aesthetic of whether you want to do a shuffle at that take a slice versus do you wants to a call Jett get an index for a range of indices you know I'd argue there's like aesthetic reasons why be a get per muted index for a range of indices approaches a might even might be cleaner too but yeah I

mean we don't need to decide this now but there's kind of two issues that are in the specs that are related to it I think one is 79 and the others I think 774 but I forget the exact number yeah it's 77479 another spec formatting thing is that when we actually after this well get to a real research update we do we're doing hash tree we look currently in a lot of places we're doing hash tree roots taking one argument and with the kind of implication being that from the

one argument you can infer what the type is but there are some cases where you can't infer the type from the arguments and probably the most agree just one of those is lists like because you can't distinguish between a static list and a dynamic list so one possible solution to the assist in hash would beat me good so that hash tree root just always consistently takes both an object and a type the other approach is that we basically add wrapper classes for static lists and dynamic lists and Nick applied yeah apply them more consistently you know and this is more probably on the Python side than anything because a lot

of the a lot of these languages are gonna have that internal distinction and they're typing anyway yeah and it's I mean like I get the feeling that languages other than Python are probably gonna take the 1 or do an approach because because everything is statically typed but this is also just kind of respect consistency you know ok so the real research update after the tiny after respect nitpicks is the number 766 so this says we a poll request having like client related files so the idea here is basically and there

it this includes a couple of different kind of new ideas one of them is the concept of a merkel multi proof which is basically a theoretically optimal way to combine basically Myka Merkel proof for multiple objects and it's or like this potentially has something like a good it's somewhere in between like twenty to thirty percent percent or potentially even more efficient than just having them just having Merkel proofs for a separate Merkel groups for every value so the idea basically is that whenever you have a Merkel proof in the Merkel

proofs they're kind of redundant and share hashes or one of the hashes can be computed from the other hashes it removes all all of the redundancy and it just includes the minimum possible data that you need to compute everything and like code complexity wise it's actually really not hard at all there's a I mean there's both the light client spec file and then there's also a github repo research repo that talks about and I put it in a telegram that just makes a minimal implementation of creating and verifying multi proofs in the in 50 lines of code so that so that's the

first thing and then the second thing is that basically there is an algorithm called that get generalized embassies and the idea here is that you can represent in arbitrary SSC hash trees to an arbitrator like a hash tree of an arbitrary objects that yet that you use to give you the hash tree root as a binary Merkel tree where the depths and different locations might be different and so you can represent a path so a path basically being a function that says oh given a blog as an input return the kind of the public key of the hundred and ninety seven validator or return the length of the list of open challenges or something like that so and it so it takes it takes

a path and it turns it into a kind of generalized index in the Merkle tree which basically says like where do you go left or go right kind of Express how deep see you go expressed as a number and from there it becomes really easy to just basically make a multi proof of multiple accesses of object of arbitrarily deep XM accesses in going into SSE objects and this makes it just really easy to kind of define the light client protocols for basically determining the minimal be creating minimal multi SS merkel proofs for

calculating any any kind of function that you might need so like seven immediate example might be if he wants to calculate some proposal persistent committee if you want to calculate some like previous state route or some previous index route or something like that and then there's I also introduced this notion of a Merkel partial which is basically a yet enough set of Merkel proofs that you can then use in place of an SSD objects and kind of treat as an SSD object so the the goal of all of this is basically to have a basic framework that you can use to do pretty much anything that you might want that

you might want to do with as a light client and then the other file that's so this is all still in Merkel proof MV in 766 and then the other thing in there is the saying is sync protocol that MV and that basically describes a beacon chain like client syncing protocol and the idea basically is that if you have some walk header that you currently recognized as the head from there you can ask for a Merkel partial which tells you what the hakeem you know what the earlier and what the later persistent Committee is from there you have a function that you can use to compute the shuffled Committee of some points in the future and then from there you can you can ask for future

vlog headers and then you can verify future vlog headers by basically verifying the episode of verifying the persistent committee and the data complexity of this is something like about 80 kilobytes every nine days plus a couple of hundred bytes for a blog so it's actually really nice um you know it does depends on like one small patch that needs to be and that needs to be included into phase one which is basically reducing the person committee size but actually it doesn't really require anything else great and to be clear so these these protocols require

some of the functionality from phase one yes so the white coin protocol requires a functionality so one of them is just calculating the persistent committee and by the way just to kind of explain why the reason why it uses persistent committees is because they change only once every nine days instead of changing every every epoch and so the verification of a committee which has what takes 80 kilobytes only needs to be done once a week and you can amortize it over the week instead of like we're having to do it every six minutes and now that and the other thing that you need is you actually need a testers to be and if acting as members of

persistent committees and signing some kind of chard walk so if we decide we can't really care about my clients and we could implement a kind of minimal version of phase one that just basically gets people to keep on signing like signing FC shard blocks and that would give us enough but on face phase zero if you tell you can't italic you cut up metallic mm-hmm you cut out yes you said you think oh sorry you're totally I think I stopped by

saying that basically it does require a kind of minimal stubby part of a portion of phase one which is like some kind of shard blocks header structure like even if the data is empty that's fine but shard block headers need to exist and people need to be signing them and they need to be cross-linked I know but okay they need to be able to be served yeah they need to be available over here area network okay thank you it's like so I was looking at the composition of a block right now is that there's a beacon

block with a header portion and then it becomes block buddy in the work holder what polls recently they there's a mention of the Piton blog header which is actually not defined or specified anywhere else and it says it's for like mine friendliness it's possible for people to propagate slug hitters who's out having too much all right kind of like approach because it's really easy for me just get the bytes for bunch of errors and just hope I get that to network so replayability on the network level is pretty good but you have different opinion about how to go about that in the dev branch there is a beacon block header to find which is simply the beacon block but instead

of serving the body it's serving the hash tree root of the body okay third branch yeah I can't remember if that's unnecessary now but that's to be released in the Bureau of pp0 visor in your in the light client protocol documents I also define a more general notion of an expansion and a summary of a bla of an of an SSE object which is basically the idea then given an object an SSD object you can create another SSD object where parts of the object are replaced by their hash tree roots or where

alternatively where hash tree roots are and if replaced by the thing that they stand for and like part of the reason why that's useful is that let's use talk about like an object that's part of the state that's part of the block even though like as technically written in the protocol blocks don't have States to only have state roads okay but like the client thing is definitely still very open to feedback so I'd encourage people to the reader wouldn't offer any comments I mean my biggest thing is the

dating and all the block getting proofs and lettuce that'd be great but I think the version of a client a lifetime you just did a good job at replaying blocks and headers or the network and maybe having a way to just consult for five twenty years make sure it's not serving garbage directory model to make the network more reliable by you know just just getting more replayability between different clients okay that's all okay a relative on any other quick research updates so the quick research

update I wanted to share is basically an improvement to the challenge game in the the custody bed scheme so the custody bid scheme basically in the optimistic case where there's no challenge is basically you get this for free it's just one extra bit in the in the other stations and its really really nice but in the worst case there's this challenge that gain that happens and it turns out that the this meant that the phase one spec kind of grew much more complex than we would have liked it so the good news is that we have a new challenge game

which drastically simplifies the communication complexity where a challenge happens and it was looking like we can basically have a challenge which is just a two-step thing where there's a challenge and a response so it's like a single round of challenge response which will allow us to do things like simplify the way we do in sensitization and the way we I don't know handled and the mechanics of the of the challenge so hopefully there were hours to have you know the phase 1 spec be as simple as we want it and hopefully I'm thinking on the order of 4 to 8

times simpler than the phase zeroes bag we've also made kind of a bunch of progress taking off kind of the laundry list of small design points that are listed in issue 675 and hopefully for the 0.6 we'll have almost all of these checked off great it's just okay next up I'd like to talk about the or give us a chance to talk about the networks back I know the so Matt slipper posted an PR on the specs repo and I know that's very active

right now so if the conversations mainly happening there and there's good feedback there we don't need to go into it too deeply right now but I did want to give just a second a minute of time if anybody has some quick comments questions feedback etcetera I think there was a active question about the use of Secchia or TLS 1.3 in networking in general we might want to address here too right Zak was that your question well yeah but we started working on a kind of like just building out a lightweight proof-of-concept type of wire protocol we're calling it aa 'but

or there and back again but the idea is to kind of just create something that works right now and we've just about finished implementing it Matt's working on that with me antoine is as well there's like a few people that are contributing to it right now but I just wanted to kind of get you guys this feedback and like see what you think about that thing Matt do you want to talk about that yes sir sorry okay so did you link the repo we're just having hearing a lot of feedback from people

creating the clients that there's not a lot of activity going on as far as like testing peers talking to each other and yet and I know that there's a lot of work going on the TTP to figure a lot of things regarding finding the piers in different parts of that higher-level network stack but a lot of these are like maybe in the application layer details of application layer so we were thinking that maybe there should be like a different layer at the bottom which is just like a simple simple wire protocol which is extensible

enough to allow people to start iterating and start communicating and so what we've done is I've created a EB and F grammar which is basically it's it's kind of like sort of inspired by HTTP but it's very very minimalistic and narrow to the use case of sending like binary payloads in an RPC matter I looked at the specification I can't remember what's anywhere that other serialization might be that's going on Zack yeah yeah okay so there's SSE going right so SSD so fos which is this like proposed alternative right so so SSD as

far as I can tell is basically very very similar to beasts on and far as the types it supports but it has this additional tree hash type and so my concern there is rework that's moving in a direction where is sort of bit you're you're putting application schema into the protocol and you're making a sort of one monolith now over time the types of data structures that might need to be supported will evolve and it's just nice fuel to negotiate some of these things if if you have to iterate over time with different clients that may not be at the same level of being up-to-date so what we're thinking is beats on provides the

the basic primitives you want like integer float things like that but but what you get is also a binary data structure and inside these binary data structures which you know can be tagged is like you names you can use it you can bake in your SSD if you really want to or any of your like actual application specific data payloads but I don't see I I think it's it's it's a dangerous to kind of go in the direction of baking your application logic into the wire protocol itself because it makes it much harder to evolve and negotiate between different capabilities with some clients I'm curious like what do you mean by a

baking application logic into the serialization protocol okay so so like there's this tree hash that's an SSO like so tree house I think it's definitely a misnomer to call it SS each rehash because honestly SSD and tree hash are both like functions of the same kind of data structure but like you can totally swap us to see for some for something else and keep triage exotic residues right so there's a lot of experimentation the prime-minister go on and what the most optimal pickling format is but in the meantime you know maybe you can get rolling faster if we have a simple protocol and figure out the data structures as we're iterating

inception is specify the protocol completely upfront by the protocol here you mean the wire protocol like not because that's a fire right yeah yeah totally totally the wire protocol is let me be clear my definition here is the wire protocol is responsible for making a common agree to brand messaging format which can support basically RPC type commands it's like we're all while this is getting conflated with the holded p2p thing where it's like that's gonna solve everything so everything just gonna be implemented through that but that makes it you know it's it's like it could be very simple it's like oh all you got to

do is parse this simple thing and here's an EB enough grammar so it's like easy understand what to do and you can start interacting as opposed to having it's just like oh well you have to just install that p2p enough doesn't make any sense yeah so go ahead just like would PTP def PT pls PT p stacks are really designed specifically to accommodate for the application layer the wire protocol should be something completely separate like RL px stuff like that is kind of just a mechanism to establish a handshake and kind of define the rules of engagement for communicating with those peers and then layers on top of that are responsible for adding additional logic to how those messages

are formatted all of that application logic that says like this message means this the wire protocol should be like a dumb protocol that's kind of agnostic to any of that additional logic so if that makes makes sense you guys know what additional like Viking about though like I'm still like whoa like the negotiation for like how particular massive messages are going to be interpreted or like with like pickling format is going to be etc so we're saying why your protocol should be in we should be able to like change though yeah yeah matters all that matters is the we have these messages

and these bytes right all them up is that each client agrees that this particular byte sequence correlates to this particular action or application logic the wire protocol itself doesn't need to account for any of that so it should be lightweight it shouldn't we shouldn't have to worry about TLS we shouldn't have to worry about like any of those additional features you know like those are those are just gonna bloat the protocol itself and it's gonna take much longer to try to get something out that's actually functional we should just have something that's like the bare minimum and and try to prevent you know let's like follow the UNIX philosophy of designed and kind of just like make something that's simple functional and versatile the reason why I'm confused is

that like SS Z by itself is pretty simple and versatile and I guess as he as a thing doesn't really depend much on like application level decisions at this point the idea is that is that we might want something other than SSC or we might need to expand beyond SSC so don't pay any of that level of the encoding into the wire protocol and the wire protocol essentially just represents messages bytes right run right now look at the protocol yes this is an SSD message or this is a wire protocol this currently part against the specs repository have

we yes sorry yeah because confusion here indeed the wire protocol for the the current like Network specification is effectively just SSE but the reason why like if you look at the for example there's a messaging like specification and basically just has a version bit that represents your compression protocol your or your compression scheme your encoding scheme and then some encoded body that can be SSD or whatever the fact that we're using Lib p2p does kind of matter here because that's going to determine the way that we negotiate

which compression and encoding formats we're going to be using right that shouldn't be on the wire protocol that's an application in your decision well your wire protocol in that case right is just going to be whatever the negotiated format it's like in any in any world the wire protocol for an RPC call for example right just needs to describe what data structure are you sending over the wire whether you're negotiating upfront for you know to use s as e or not happens somewhere else but the fact that we're using lib p2p means that like we need some place to define where that negotiation is going to happen and so effectively all the like RPC like

specification is saying is these are the data structures that are being exchanged and here's how you can find out whether or not like they're encoded using SSD and how they're compressed like that's it like I think it's easy I guess it's easy to like it looks I think a lot more complicated than it really is all it is just effectively one lip PDP protocol ID and it's used to identify messages that perform RPC RPC calls that's all it is one thing maybe in terms of organization should the people working on the web protocol maybe have a call every week so that we only have reports on the conclusion on turns a

frame to implement a call and also to prepare for ethical because I suppose then you will have a workshop on that yeah so we were talking about we about starting a working group so I mean there are about eight of us working on this stuff right now and I mean rather than like kind of like you know duplicating any effort so yeah we should probably just start a working group that we can all contribute to and discuss these things together yeah just let me know the best way to contact your gonna be yeah we'll probably be at Ed John I don't really want to travel but yeah I will regrettably be at a cop yeah I've seen you unmute like eight times you

know comments I just didn't really know when to jump in properly so I just wanted to quickly circle back to that to that earlier question of like can't we just have like a simple it's the simple sort of protocol like that that's the thing that originally kind of got me started but and you guys already kind of reach a like reasonable conclusion so I guess what I wanted to say is that basically weapon voices concern many times it said basically I think that it's totally fine to represent consensus objects whenever they're transferred using using SSE or any other serialization format but that might actually be the best choice to you know encode say like they the message frame that carries the RPC command or anything

like that but it can totally be useful and I think Matthew is kind of right in saying that it does it does it does matter a bit you know if two is relying on only p2p because because that already defines how how these RPC commands are encoded on the wire and things like that so I think that the high level one a protocol specification for it to really only needs to include the names of these RPC messages and then they're sort of like payload and how that's encoded and I think that's pretty much what it does so it is kind of it is kind of alright right now yes so I just wanted to mention a few more things I didn't have a chance to finish so the the wire

protocol specifies we would like to feedback on it but the thing is is all it really is responsible for right now is like Antoine was saying identifies the name of the commands negotiates compression and and then there's two separate headers and payload these are just basic protocol optimizations if you put everything in one serialized payload like an one SSE then that means that the whole thing has to be pretty much decoded before maybe you can partially do it perhaps but still like you have to do a lot of work to like look at partial or the whole thing and so you these little optimizations and protocols where

you have in the first 10 bytes or first 20 bytes all the information you need to make a decision sometimes makes a difference because you don't even have to realize payload but that's exactly the issue we had with our LP and why we want it SSE where we could put or maybe as a offsets as realization where we could know at which offset was the data we needed without dissing everything yeah you know if you know the SSE the frame that contains the RPC message just has an ID and a method name and they're both like has to see objects so you can bytes like you have to read all this

bytes sometimes you don't need them all if that makes any sense like look into so we've been doing like like in on the DEF p2p side we've had like streaming messaging and things like for kind of a long time I can kind of tell you that we haven't actually needed it very often so we do it in boy theorem in the deputy implementation the way we handle it is that basically all of the message decoding is streaming we process basically we unleash your deserialize as far as they need into the message to really figure out if it's valid and stuff like that and that hasn't actually for us like any extra speed or anything so I wouldn't necessarily say this like super important to tooth to have that right

now right now isn't everything sort of tied to let btp like e 2.0 protocol is leaked gp2u P as far as I could yeah that's actually that's that's a normal point that we want to make which is there is a an excellent PR from my few sleeper a method that would allow exchanging data between clients and they require RPC level methods and some intense behind them but that it's actually being pointed and that's in that document in in one of a comment that Gus it would be used to propagate blocks and headers and that seems to be kind of a magic bullet that people just want to use for for the p2p propagation I don't know that is

going to be such a great way to do it and I would want to see better flows on actually with the lifecycle of how you would want to propagate blocks and Irish Wars or clears and how would that be how would we be mindful about that and those actually would actually have to do with the rest of your domain lifecycle and your consensus in your block time right because if anything if you're pushing or pulling depending how fast we can see propagation and network the block time of six seconds is going to be interesting right so we need to meet us I mean the the performance of that networking layer is really really critical and maybe even so then some of the you know freaks you can play with

icing and all that and I think to so one thing that the hobbit protocol does is that it does not really enforce encryption for example so you can really replace between two peers by just getting the bikes and just take this message as is and just push it to another socket so if we talk about gossiping you just making it really noisy we could do that and it's kind of because it's bare-bones it's more open to iterative development in changes where most of the updates of this call so far have been people trying to implement Leiby to be in a fashion that they can leverage it so like to be such a huge deal yeah it's just I need

yourself to Lippe to be roadmap makes it that you're going to only deliver after Libby chippies done right there's nothing new here software engineering so I guess it will be really beneficial at this point to just get together and find this like minimal insecure unencrypted wire protocol transport mechanism that allows you to do to do cross client testing and I think this is something that you know the to implement a community should look into actively to just kind of make something work it doesn't have to be perfect doesn't have the world's like best and most secure and most optimal transport you know it can just be something where basically

you can you could do basic interoperability testing maybe unlike a test that and that's and that's totally gonna define and then you know there's gonna be so much time later to figure out what the actual perfect transport mechanism is and you know there are literally like hundreds of options available for that and don't think it's like the right time now in the implementation cycle of it to to worry about what the final transport protocol is going to be or what the final sort of you know discovery mechanism is gonna be because basically we're all just you know chewing away at our own sort of like you know favorite optimization topics but you know at the end of the day what really matters now is just you

know getting it trying to cross yeah what I wanted thank you for saying now that's exactly what I was gonna try to say like what we're not trying to preserve it's gonna like override anything we want to get something out the door very quickly that'll allow us to actually start doing network testing and getting clients to one another rather than operating on these like simulations and these like monolithic applications this will help us push the ball forward it's something that we can implement like before the next call like we're already working with Artemis team to get this working and we're specking it out and we can have this functional we could have we can have multi client testing much quicker by doing this and just you know

we'll just start building you know so this is also something that this is also some some lesson learned that I can share from you know the development of each one so the the way that each one was developed throughout the entire POC serious there was no RL TX there was nothing we have this all we had was like unencrypted TCP connections carrying our LP and that took us like really really far and then we only added encryption I mean in just you know like kind of late before the launch I mean we're happy with our px is kind of a shitty protocol but it's definitely you know something that you know got

this really really really far to just to just to find the most minimal thing possible and then and then you know take it from there so I think this is something that I that I would really like to see is to have this like super super basic protocol that you can you can implement in like three days yeah I love you thank you any more comments on networking right now so maybe it's good time for me to give like my usual update about the whole discovery stuff is this place not so much episode I've actually not really been busy with discovery last two weeks because we tried to push the eath version 64 discussion forward so that is

something that must be concerns if one and you're gonna maybe hear about it tomorrow in the all quarters call we did revamp all of the e3 if one related specification so we have a really really good specification of the each particle and a sort of complete specification of the f1 like line protocol now but that doesn't mean anything for you because your homework you need to and when it comes to the discovery what kind of happened is that I am still implementing discovery version five as sort of specified and the specifications live in the in the repo and we've had a pretty annoying issue where basically

since the whole thing is supposed to support like different sort of kinds of node identity at the same time we kind of have to find a way to verify package signatures on the on the individual UDP packets sort of like depending on the crypto system that the individual node uses and currently there's no super good solution for that so there is an issue open on the def PDP repo that kind of tries to explain the problem and if anyone's interested fun UDP drop the proverbs and you know you can come there and help out otherwise before we gonna have a solution in time for like next fall or something I don't know so that's the that's current update so started implementing and we're starting to hit

like the first sort of like real-world issues with the whole thing great thank you okay on networking yes there's value in getting some these minimal protocols out and if a team or two want to drive that forward and want to begin experiment an arrived ability but in that respect go for it I don't think that clients should experiment with interoperability on the wire until they're conforming to state tests which I plan on releasing today but for b05 that said we're going to continue to work on a more concrete high

level spec in parallel to some of these like rapid iterating smaller specs yeah I mean don't don't let the size of the spec that you know I wrote like scary you I guess like all it is is just a SSD through a live p2p protocol that's kind of it and from that if you want to work on something more minimal look at those message types look at those RPC communications and just build them minimally because those are at least in the direction of what is going to be required for the communications protocol okay Piper has some serialization

benchmarks that he wants to share I think I linked to it just recently in the agenda Piper boom this will be short I'm glad that there's other people working on the actual wire protocol part because other people are more knowledgeable than me on that I just have been like digging into serialization and sort of at the more at the application layer but with thoughts towards the network layer and I ran a bunch of benchmarks and there's data that you can go look at it's pretty extensive but the gist is that ROP is actually really terrible for eath to data structures it almost doubles the

size because of all of the little things exes and the deep nesting of things like blocks and even as I say and essentially I've just been looking at ssz which is I think actually an old version of the SSE spec I've got my own sort of really highly tight compact serialization format that I've been tinkering with and I've also been reasoning about SOS that style and I think was interesting but Felix said earlier about rarely actually meeting to index into some of these data structures however what I'm currently leaning towards and I'm not I do not feel like

but I don't know enough about this yet to have like a really strong opinion but where a meaning right now is pushing to modify the SSD spec to include the SOS style offset pattern so that we can have a serialization format that also works as a contract ABI and I believe that at least those two things combined together give us reasonably compact messages that can also be used to talk directly into contracts that give us that fast indexing into data structures which at the application level may not be useful

but inside of the context of like the EVM or any sort of like and I say that in the most generic like EVM he was and whatever being able to reach into these things and grab the data that you need in those contexts is actually useful like Vitalik said earlier tree hashmaps well under the top of just about any of these formats and then the last thing that I'll say is that experiments with ssz suggests that there's maybe nine to ten percent size gains that can be improved but it's it's reasonably efficient from the get-go so even as like not as the wire protocol as

in like what we were just talking about but as like the the inner part of it I think it's an acceptable format but we could still get message size down by about 10% with some slightly different ways to pack things I I'm happy to answer anybody's questions but I'm not at a point where I feel confident that I just know so how do you do 19% I'm sorry sighs so resulting sighs like 9 or 10 percent size reduction in overall messages similarly the like but to get that 9 or

10 percent you also gain the ability to do streaming encoding and decoding but you lose the ability to dynamically acts index into a data structure without decoding it's at least all the way up to the point where your index so there's trade-offs across the board just to comment on your document which looks fantastic you say problem statement when talking about solution we need to be looking at two primary use case networking and protocol yeah I think you should rename protocol to consensus because as with anything in in July when we are talking about Sur ization because posterization comments like every two months is the proto before whatever and we use consensus do you know about the

CPU utilization when you do that they're big I I suspect that that the more compact version is less CPU intensive assuming you're decoding the whole thing just because it includes less information but then again so I'm using so one of the primary place where these size gains are achieved is using le be 128 encoding for integers which just dynamically scales the size of the integer and now that I'm saying this out loud I suspect that there's some CPU hits there because you actually have to like iterate across each of the integer bytes until you find the one with a high bit set and so there's there's going to

be some some performance it's there while decoding but I do not know what they are I will start looking into what the difference is okay take a take a minute to review Piper's excellent document it seems like making some modifications S is said to accommodate for it being more useful in a VM makes sense okay we are at the end of the call we are meeting in person for whoever it happens to be in Sydney on

April 9th from 95 I will share location I'm gonna send a doc around so we can get a head count and figure out what we want to work on on that day that is the middle day of the hackathon so we won't miss the opening or closing ceremonies and then the actual addcom starts the 11th usually we have time for any expect discussion and stuff I'm gonna be dropping to 0-5 today with some tests and can handle any questions in the getter as usual so let's skip that today all right any just like quick closing remark anything that needs to be said I

think we just we talked to repeating what we met we talked before we like instinctively in for him in favor of like separate of having like class trappers to separate static ends and a dynamic west so that they have like different firearm types yeah because I think that kind of actually intuitively translates more over to these statically typed languages anyway right okay that sounds good and we don't need to make any wrappers for numbers because it seems like so far we've been we haven't even even made the evaluator balance decision in the direction of if you went 64 being the one true you went right

okay thank you I just had a quick note on the shuffling so basically we talked to our time today and and the way it looks from from our perspective is basically that we schedule a handler and it looks at like what's what's the time now oh and what do I have to do so if we're changing the shuffling around let's like the question that let me answer and we have to answer it in two ways like what do I have to do right now you know what I'm like what am I going to have to do in the future so that I can prepare by for example thinking some some charge right yeah one thing there is that what I have to do now my chain just my hand just write it we have these deep reorg so that's something to consider as well and selecting which which side of the

coin the shuffling or with aesthetic for the shuffling we should use mm-hmm right and it would be nice to have something page 1 page 2 right so does that way of looking at it imply one direction or the other it sort of implies to face one direction if I if I remember it right like I have a sloth what should I do the face one direction is the input

number referring to the role so like basic which basically just means like what team which committee you're in and VM output being know what what's the validator that's responsible for that role which can translate to get here's a slot what's what are the validators yes because like my role I just like yeah yeah there's a set of rules there for it to the other set of roles Bruce watch yeah also like that Vinny was just just hearing it that feels more natural it also has the advantage that I think

we save like a tiny amount of hashing because the like maybe something like 10% because if we're calculating the akima saladna die committee members first members for some committee then at least in the first few rounds of hashing they'll be concentrated and like what a fairly small set of ranges of the full validator said and so the hashes he needs expects only for the shuffling are gonna be shared but that's like maybe a 10% gain and just as long as we also have a way to anticipate what we're going to do you know in the future yeah the nice thing is that the shuffling the permutation algorithm is like trivially

reversible so not a problem yeah well then that's perfect yeah cool okay yeah a little exact same thing but ever but you just run the rounds in reverse order well maybe I'll mention one one other thing I talked briefly to Danny about this but some of these invariants that hold across for example where which are fixed from the finalized block or which are fixed from a justified doc and so on in the future would understand a lot clearer which ones we can rely on to be maintained in the future for example you know committee won't change for an epoch from

from just ready for it like we can reverse engineer that from the spec sometimes but if they were spelled out would be much more comfortable relying on them for optimizations and the like right so doing a having the research team doing a pass on adding explicit invariants would be useful yeah exactly agreed okay we're gonna close six minutes over thank you everyone really good discussion b05 coming out today and talk soon thanks off Thanks thank you

[Music] [Music] [Music]

[Music] [Music]

[Music]