P2P Performance

Transcript

We're good for other trying to see how we can see how light can pick up on that. >> Cool. Are we ready to get started? >> Cool. Uh, so my name is Ronald. I work at EF. I am leading the peer-to-peer networking team. I uh joined 5 months ago and I've been mostly focused on on layer 1. So it's kind of like I'm joining lean these days and plugging into it uh with more attention. So um I was kind of like figuring out how to best lead the session given that I'm not like very much involved in the

day-to-day. Um, so I came up with this agenda um of kind of like maybe these could be potential goals for this session, but it's entirely malleable. So like let's figure it out here what would be most useful uh to everybody in this room and in general for lean and people who are outside this room as well. Um so I just like let's use this as a starting point. I thought that I could share some context and insights on peerto-peer from the L1 today. um how it works and kind of like some of the learnings there to then influence you know what we do in lean uh ideally I would like us to agree on peer-to-peer requirements for different devet stages so if we can set a road map and kind of like figure out

um how the requirements are going to evolve and potentially how they're going to affect the design that would be extremely helpful because that would give us a moving target that then peer-to-peer efforts can plug into to like change the design over time to cater to those requirements. So it can be versioned. So we can do a v 0 and then like you know this is deliberately something that is just a sketch and we can continue iterating on. Um I would like us to agree on design principles and the design space in general. So, for example, some trade-offs in the L1 um I mentioned earlier are that we uh that there's no spec no kind like validator privacy by design, but also we're not

leaking IP addresses or ports by design. So, what do we want to do in le because we have very aggressive throughput requirements which then could thwart or kind of like you know close off a part of the design space. for example, mix nets, onion routing and so on might be out of the picture uh for for lean. I don't know. So just kind of like want to agree on on some of those and then we can just like um go into discussing solutions and potential prototypes of like building blocks that we might want to prototype early to see how they would integrate into the peer-to-peer stack for Le. How does that sound as a starting point? Sounds good.

>> Okay, sweet. Uh, by the way, I think we all like most people in this room were in the previous session, so we kind like already introduced ourselves, but I think we have two people here that weren't. I don't know if I was >> you were Oh, you were. Okay. Sorry. >> Maybe I was sorry I was hacking on the slide. So, maybe that was your your address. Um, cool. So, just wanted to give a little bit of an overview uh for a bird's eye like view of uh L1 networking. We actually have two networking stacks running in parallel.

Uh there's the CL stack and there is the E stack. So the CL stack is the libby stack that most people that lean is adopting by default right now. Um and libby has transport handshake and Muxer modularity. Uh we currently use two different kind of like compositions of this. So we use GCP, noise and gammax or mlex um as one um kind of like transport stack and we use quick which embeds a multiplexer and a handshake in it and it's based on TLSB uh 1.3.

Then uh we use gossip sub as the broadcast framework for all objects and this is a massive problem and uh like I can go into that uh later. We use discovery version 5 uh for UDP. This is interesting because it was a protocol developed like in in death B2B which then still debrief before and can probably talk more about this ended up also finding so much in that B2B that we ended up implementing like adopting the CL before the EL could um and this is interestingly this is UDP based um and this is at at a point where um where the CL uh did where Quip itself did not implement or did not have

support for datagramgrams. Um so it is kind of like a one like one more parallel protocol that does not run on top of DP2B. It's just UDP based and we have two domains. We have gossip and request response. Um the EL has DEP P2P which uses RPX to handshake authenticate and uh to conduct u multiplexing in in a way and uh uses this discovery V4 for sorry this this V4 for discovery which is also based so this is what the CL stack looks like today. It looks daunting, right? It's very complex. It's more complex than it should, right? So,

this is one of the things that I think lean can do can leanify the the stack, right? And um this is something that we are also actively working on in the peer-to-peer networking team for the L1. So this is one of those cases where I think we can merge like join forces to develop um a leaner more um say let's let's just say more modern stack um that is Ethereum optimized at least in kind of like the bottom layer of the stack where then lean can go and develop customized protocols for lean and the L1 can go and continue iterating on blob hubs, blocks and other objects

that we disseminate in the O1. Um, so this is what it looks like. By the way, the things that are that are dotted are actually uh components available in Libby2P that we don't use in Ethereum. And this is what takes me to the next how the stack how we planning to evolve the stack. So there are we're shooting to towards this is by the way you're getting very early alpha because I haven't talked about this publicly. So uh so it's probably like there's going to be some dev connect announcements around this just like you know this is very early so it's probably going to change along the way. Please keep it

like between us for now and people who are watching. Uh but but this is kind of like very early, right? Um the idea behind ETHB2 is that it would be the next generation Ethereum specific networking stack and I'll go into like what those building blocks are. And as a inter as an intermediate point we're working we're already working on min2b a go implementation of min p2b. So this would be a libby compatible minimum networking stack that removes the pieces that and the abstractions that we don't use from libby um and actually takes some more

opinionated decisions while remaining libby compatible. Right? So that's an intermediate step. You can think of it as like going from a from your perspective going from a heavy stack into something minimal to then like slightly open up again into more specialized protocols. And the opening up actually happens by way of taking this component which is gossip cell which is a general purpose uh broadcast uh protocol and actually starting to reason about dedicated broadcast protocols that are very aware of their slot timing when it happens of their required

latencies of the networking workload right that they're placing. So we start thinking about what is the optimal protocol for attestation broadcast. It looks nothing like the optimal protocol for blob broadcast. Right now they're converging and both using gossip subub and to add uh insult to injury they're actually using the same parameters as well. So, so you're already seeing that there's a lot of inefficiencies in the peer-to-peer uh stack right now in the L1. Just by improving these things, we could actually make the L1 as well way more performant, right? Um then there's things like traffic shaping and

purposeful peering and um potentially a built-in mixet uh which might be something we want to adopt for the L1. I don't know if it's viable for you. Right. Cool. Um, so I'll stop there because this is not a presentation, this is a workshop. So yeah, just kind of like don't know what do people think about this direction like how do we think lean could couple with this leverage? >> Yeah question like I still see like two transport uh what's the difference?

>> Cool. So in libby we have this notion of transport modularity where uh a a endpoint can actually or node can actually expose any any set of transports and they're expressed by this um standard called multi adder right um the reality so this was actually a deliberate choice back when we adopted libby for e2 because back then we didn't know if light lines like how light lines would interoperate would operate with the network interact with the network.

So it was thought well you know at that time we wanted to have maximum flexibility. Uh it is possible that like clients and specifically loadar was working like one of the original visions was actually running a e2 beacon node inside the browser right so it was kind of like you know part of that like browser interacting from the browser from and and status as well I don't know mommy I don't see mommy here the status was uh also like targeting embedded devices which is like another set of devices that also run um have other transport requirements because there might be like in the wild there might be like you know this thought of like hey running devices in nature as well right and like these were legitimate use

cases. So having the transport full transport modularity where you also allow transports like web RTC, web soockets and like the newer web transport and so on to actually um be exposed as endpoints in the L1 network and the core network. That was a part of the design space that we wanted to reserve the right to, right? We did not use that and like that is not the way that we would approach edge devices today, right? This is very inefficient to have a mobile device actually be an attestation propagator in the network.

Like it's not it's not a reliable device. It has very high latencies and it's just not going to work right. So today if I were to like revisit that thinking I would say keep the core network really tight and then plan for edge layers that allow these larger variety of devices to participate efficiently uh without delegating trust like so this layer has to be trustless right so then backing out into how this part of the stack changes and evolves now we think about two different transport profiles so we have uh two substacks right that compose uh okay and sorry just there's limit also gives you

moxer modularity and handshake modularity right which like this has another set of problems that I won't go into um so what we're doing is like collapsing those choices into just two transports the quick transport which already comes with everything built in and the legacy transport which is you know in some some environments UDP has problems. It's blocked or like you know you can't use it in the way you want it to. So that's why um this looks like that. And then for ETHB P2P currently so min is going to support streams but ETH B2B will support unreliable data rounds over quick as well which gives us access

to like another uh opens up another set of like the design space as well for building protocols. >> Does that answer your question? Um I have a few questions. Uh the first one is conceptually why is east P2P smaller than mid P2P? I mean okay I understand it's a legacy thing that you want to be able to >> Yeah. >> Um how do you see and this is purely an L1 question. How do you see the transition happening? because I don't know when we met back in 2018 >> I was trying to get P2P to work with G and we had this prototype that worked >> but really what u forced Peter to say drove Peter to say no let's not do that

>> is that he was saying like we're not going to be able to >> you know if someone starts building on one network creating a block we need to make sure that this block is going to be broadcast on P2P on dev P2P as well >> it's a big mess >> uh how do you see the introduction of I mean P2P is P2P to that stack. >> Yeah. So um so I see it P2P I haven't discussed with uh with GE and other like team members about a potential unification. >> So I'm approaching this from more like the CL perspective. I think there's a lot that we can do to unify the text stack and I know that Felix and others are thinking along the lines of quick as

well. So, we need to like actually I haven't formulated this idea in a good enough proposal that I'm happy to present to them yet, >> but I do think that once we do, we probably have pretty good synergy there. >> And um I think we would like maybe try to converge to the same tech stack even though the network planes might still be separate, right? And in terms of a transition, that's where MintB2P really really comes in because it's kind of like that transition vehicle for us where we have a libby compatible stack, right? And then the idea there's two different approaches here. We can keep this initially libby compatible as well.

Um and or just use a fork to transition over really >> there's a fork like we can transition over definitely in the fork or we can use like this many >> yeah this negotiation as well that can happen there >> oh and that okay because I was thinking like before you already have this in the sense that some people are going to upgrade later than others and there's always a few people that forgot to upgrade their client >> yeah but okay that's their problem um but yeah okay So you would activate the switch at the moment of the form.

>> Yeah. So there have to be one version of the client, right? That would implement that would have both stacks and because we're building this from scratch, we can like figure out a way that is not obnoxious, >> right? >> Uh for that period and then like that would be the transplant, right? We would transplant the network over to the to the new protocols and then from the next version over we would just kill the previous just remove the code parts at the previous step. >> Right. that would be like as I said this is very early we haven't thought about it all the but that's kind of like some directional thinking >> do you have any optimistic plan for I mean in terms of timeline >> yeah yeah so I think men p2B is already

starting to get worked on um we have two one person actively like working on it right now and a fraction of another person Um so this is something that and and specifically we're doing it in go because prism is currently using go lib2p and go lib2p is very heavy because it was actually it's kind of like the reference implementation of lib2p so it like ended up being the kitchen sync uh and we just want to reduce a lot of that surface right >> but do you build the go main P2P library from scratch or >> yeah Um, oh okay, please.

>> Right. Yeah, I was just just wanted to confirm. So, um, so Quick, uh, is a relatively new transport, right? >> And it's gaining adoption. So, is there potential plans for deprecating completely TCP and Yamix and NLX and noise? Like for example, we recently built CI P2P. >> Mhm. And um we were just reading discussions online that um like things like TCP, Yamix, they were considered baggage in the consent there. Um and now that clients like whilst their P2B supports quick transport.

>> Yeah. >> Um I think Golang Golang does as well. Um so should new implementations of the P2B deprecate like you know TCP because it will make the codebase a lot more manageable if we just kept quick. >> Yeah. So I think I have to look at the latest stats from Cloudflare. By the way, if somebody can pull them up, Cloudflare has a really nice dashboard. >> You mean about number of quick notes or what? >> Uh, no. So I'm looking at Yeah, if you can pull up the Cloudflare dashboard, global dashboard that shows the amount of traffic of internet traffic that runs on Quake, >> that would be really useful because I think it's like still at a 30 something 33% or something like that. and it's

kind of like plateaued for a little bit and I think what's happening there is that there is significant oscification in the form of middle boxes that are assuming TCP and that are blocking like specific UDP patterns and this is one of the problems that quick is going to um face and the group the like the working group is actively working towards this. Uh so I do think we still need to keep the TCP option because UDP will not be universally like supported on the same level as TCP. Uh I think you're still going to have to you're still going to encounter encounter edge cases there.

Now the interesting bit here is that uh this should be actually this should read TLS instead of noise TLS 1.3. we already have a TLS handshake and um the interesting bit there is that we're just we don't like implement that stack does not consider MLEX you should just if you're implementing from scratch you should just do TCP YAMX and TLS 1.3 and then quick and then the interesting bit is that the quick working group is actually uh working on QMX which is a version of the quick multiplexer that can run on TCP This is like blurring the lines a lot,

right? It's kind of like, wait, this is a multiplexer to like TCP. It's quick as UDP. Like, what the hell is going on here? But the conceptual idea is that you have a single API, right? That is a quick like API that that that deals with streams and under the hood, it's using the UDP flavor or it's using the Q the the the TCP flavor with QMox. So it's it's it's a nice idea and then like once that is available there's a draft once that is like fully um agreed on uh we can we can replace y with cubs but for now I would not implement templates I would implement. So yeah, I guess like uh I just had a one more

question. >> No, please. Yeah. >> So basically um with this sort of stack, sorry. >> Um we basically have been relying on test plans which is the lip. >> Yeah. >> So is this going to have its own interrupt test repo or should should is the expectation that um test plans or universal connectivity will be updated to support? >> Yeah. Uh the idea is that uh it would there's no support needed. uh there's just tests that you wouldn't run but everything that runs that is compatible with min2p or that is the profile which like test plans is actually all of the p2p right so we really need need to narrow test plans to what's applicable

to ethereum and that should run on on a min2p implementation it there's no protocol changes right this is libby compatible it's entirely lip compatible on the wire it just looks like a like go lip tob it's just like a redesigned for like dramatically less complexity. >> So it's basically a new a new hype and I mean a new implementation of lib spec. >> Yes, in a way. Yes. Which is more like narrow. >> Yeah. >> Yeah. It's a profile of the spec. You can put it that way.

>> What do you think? >> We did the same with C++ for the I mean we don't have because we made it quick >> and that simplified things a lot. I mean we don't need service at all. >> Yeah. Exactly. Yeah, >> that's the that's the notion. Yeah. >> But then for us, we're going to be using that to prototype Ethern right as the reference notation. And that's what's >> I guess like embedded ping is that is that protocol different from the normal ping protocol that >> yeah so the idea is that um you have so if you use yamx and you use quick already has um native RT measurements because they use for the uh for um pacing and uh for the window sizing and

Yamx already does RT measurements as well so why have another protocol it's just leading up the stack. >> Okay. >> Yeah. >> Um Yeah. So, um I assume that for lean you would want something close to east P2P, not P2P. >> Yeah. >> Uh yeah. >> Yeah. I think like in some cases, yes. In others like I don't know this I don't know what the latest blob like you know stands on blobs is. um what blocks actually look like you know it depends but at least I would say yeah like this the broadcast protocol would be a

dedicated protocol for signatures aggregations like there's maybe three protocols right there is the signature like emission protocol then there is an aggregation protocol and there's like the global aggregation protocol >> right >> uh okay so that's fair uh thank you for clarifying this uh so my my question is uh purely in terms of do we want that for uh you know DevNet one for example which is uh um if we want to do this you know currently I mean in terms of the Z client we are reimplementing everything all of the P2P in Z >> if we don't have to that's even better >> uh the question is first how much of what we did could be reused like what

components could be easily reused yeah >> the first thing the second one is could we have you know this is much uh this is a much lower amount of modules you have to implement but it's still significant for something I want to do in December >> um so can we take a shortcut >> maybe just create the specific signature aggregation and be done with it >> yes >> yeah I would do that I would basically take kind of like your rewrite of sake and think of just supporting these two transport stacks >> and then everything in between like the machinery if you're using ABP as like your model then like look at the I wouldn't implement the network switch or sorry I can't the swarm as it is today I

wouldn't implement the store the peer store as it is today like just rethink you know the minimal like what the minimal implementations uh could be right >> um so here we we're thinking about a diner and a listener one problem of gobp for example is that it inverts the control so it views the networking stack as like something autonomous that maybe the application like gets call backs on or it gets to plug into to but like in reality the application should be in control of the networking stack. So that's what the new gen API is is like you know a more an API that put that gets control. So like just rethink it and kind of like the thin layer that I think is the interperability layer is

pure ids multi hash and the transport choices everything above and obviously the dissemination protocol right and kind of like the protocols built on it everything else is the guts of the implementation that you can like implement as you want >> okay >> but we can we can discuss more about this like I can I can transfer some more like learnings that we've add an insight so that we build a minimal necessary thing from the get go. >> Okay. >> Yeah. >> Just as as we start implementing it basically in P2P layer uh how much in your opinion it makes sense maybe to implement it in a way how we implemented cryptography today. So we have some single basic implementation and then we have I don't know bindings adapters that

can actually reuse that. So that we basically have >> yeah so the thing about the networking layer is that if you have to cross an FFI like frequently you're yeah you're going to be leaking performance a lot and then like if you integrate C go like if you're using C go from go then like you become like single threaded and like it has the serialization like problems. I don't think that's viable honestly. Okay. >> Yeah. And especially then like each programming language has a different set of um like idiomatic >> yeah synchronic >> principles of how you deal with IO how you deal with network timeouts like faults. So just building a single implementation is just going to feel

super unnatural when you adopt it as well. >> I think maybe you can reconcile that at the binding level but it's still like going to feel very clunky I think. Yeah, cool. Um, we spent way more time on this than I wanted to. So, I just wanted to kind of give a little bit more inspiration of what the like what I see as like maybe a road map that would really help us. So, if we place the different demite iterations in columns and then like think about like all the parameters that define it and some of these can be like fully uh fully constant right over time. If we just say if we set on forced second slots then that's it and that's it. If we're going to be using 3SF finality then that's it and that's it which I don't think it is

but then um like also which environments we targeting. Um and then like this allows us to like use this as a target to design against right and say okay now the broadcast protocol needs to change now X needs to change and so on. So just kind of like you know as we um as we think about about dead nets this would be tremendously useful. Cool. Um so now I have a laundry list of things here which is kind like you know some collection of thoughts and I just like want to stop there for for a second. So this is not a model not supposed to be.

Is this being useful to people? Would we like to like go into protocol design, network design specifically for lean >> or like some context setting is is useful first? >> I have a very stupid question. Um, >> what are your main bottlenecks right now? >> Right. um several so there is so right now we're making very poor use of available bandwidth >> okay >> and this is for several reasons so gossip subub um produces a high number of duplicates

which is being improved over time with item once and other other mechanisms right mostly I don't wants and and like there's there's other ideas like choke and there's other ideas about making some topics pull based instead of push based like there's a bunch of things there but basically the idea is you have available bandwidth the like conceptually speaking you have available bandwidth every like if you're not maxing out on bandwidth to support the throughput of the protocol you're wasting resources >> right that's well I guess it depends on like you know if you speak to different

operators like the kinds of um like bandwidth purchases that they make and like you know traffic egress ingress purchases that they make but I would say from my perspective if if we're not using all available bandwidth then we're we're wasting resources so uh block propagation for example like in most cases it will finish in 500 milliseconds now um builders and um meth builders are using the rest of the time to actually continue playing timing games, >> right? So some of them like will publish their block at second three knowing and maybe they're hyperconnected as well. So they like have a high guarantee because in the cost model they can like very

aggressively push egress and uh they can like on the first hop they can maximally reach like a high number of nodes. So maybe they use that as a as a game to delay their block propagation, right? um attestation propagation is extremely wasteful right now. So I I have a whole doc on this we can go over it but basically what's happening is that in a sub in so the network is divided in 64 subnets um which correspond to committees um these every validator is assigned to one committee at every slot sorry not at every slot in the epoch and that means that you're going to be having 500

around 500 in the in the validator like set today you'll be having 500 validators per committee. Um, and in total in every uh slot you're going to be having 33,000 attestations circulating. In every subnet you expect 500. In the whole network you expect 33,000. Every addestation is only 190 something bytes compressed with snappy, right? And gossip sub is we're using gossip sub. So we're reasoning about each attestation um by hashing it by trying to reduce duplicates by um like doing all this accounting of who sent me what and like

who should I forward things to and all of this traffic is arriving in a very pulsatile manner right it's arriving in two spikes it's arriving there's two so clients u beacon beacon nodes uh some implementations attest to the first valid block that they see as soon as they see Others wait until second four to a test. So you see two spikes, right? And these are very pulsatile prop like broadcasts because the tiny objects they're have to reach everybody in the subnet because you don't know who the aggregators are.

Ideally, you would be able to have some directed broadcast where you know the aggregator is like you you preferentially send to the aggregators but that means knowing who they are. So becomes you know the anonymization vector. Uh so basically you have like for these pulsatile objects you're doing duplicate accounting you're doing scoring you're doing all this thing for each one of them you're doing hashing so it's just easier to and you know which ones you have to receive like you know them ahead of time so it's way easier to just send data around except the duplicates in that case maybe right and like you we prototyping and we're kind of like measuring things these these

days. Uh but my guess is that we'll see uh a faster throughput in attestation. Now all of this is reserved like atestation you have 4 seconds reserved for attestation. So like if you're currently it happens I think atestation broadcast happens in like a second point 2 or something like that. You can actually compress it further by optimizing everything that I'm saying. There's other inefficiencies like topic names are represented as strings. So every attestation carries a topic name which is like 27 47 bytes over 192 is like 25% of the message already just for the topic. It's just like there's there's decisions that haven't been optimized that would optimizing would like help a lot. And then uh on the

third sublot there there is uh which is aggregation time. There's just like way too much time spent there for something that is very light. I see. Thank you. The reason I'm asking is um because at least from the perspective of um looking at how you can scale the TPS and sort of make everything reach performance levels that are really high. Um so from I'm coming from theory, so some of my question would be a little bit off.

>> Um >> one of the first questions I would ask is um that you know you you have like the DA stuff, right? you have the actual transaction sizes and that's really what informs throughput and then sort of the ideal situation >> the size of the pipes the size of the bandwidth sort of is dictate dictates the TPS >> I'm just trying to understand so after all the optimizations are pushed through or if there's some sort of theoretical lower bound >> like what percentage of the pipe or the bandwidth um is taken up by um non-transaction related information like more consens message related or like heartbeats or you know I don't know what exactly goes goes in there but um like

do you have like a rough understanding or at least some kind of like aim that you want to get to where you would say that like you know I know you mentioned like 4 seconds initially stations like how do you consider these things to split up like because I agree with you taking the full bandwidth is >> like super important like what percentage of that >> goes to sort of bookkeeping and how much of that really shows up in the TPS. >> Yeah. Right. So how much is goodput essentially? >> Sorry.

>> Goodput in like in networking terms we have this like you have throughput but you have goodput which is how much is actually dedicated to making progress right. Yeah. Yeah. Exactly. Yeah. Yeah. Um so I would say like there's not so much control information generally flowing. It's actually useful information um use useful data except for the duplicates. Okay, I see. Yeah. >> So that's a fundamental sort of issue that needs to be resolved. >> Uh that's a fundamental issue. But then um yeah, then there is like the compute overhead as well of um of things that might throttle you at the networking layer. So if you spend too much time due to an implementation inefficiency like

processing a validation and there's like natural things that you can optimize in the logic of the application that then like also percolate back up to to that and a very like there hasn't been a lot of attention in the CL >> to performance engineering. Uh so like I think we can go a long way by by investing time there and effort. >> Um >> sounds great. Yeah. >> Are any other teams sort of crack that issue? You guys can lift any any code from anyone.

>> Uh which issue specifically? >> I mean the question of sort of optimizing the good foot or or sort of dealing with all massive >> I think this goes into like you know what what the design space is. Um and yeah we've looked into like you know Salana Monad and others and all of them like I think the the key element here is that you want to view the networking as a part of the protocol. >> Yeah. >> Right now we have this like duality where the networking stack is like this opaque thing that the protocol puts bytes on and expects to receive bytes from. Um and then you have the protocol which is like autonomous as well like tendling to put things. So if you design network the networking stack with

protocol like with knowledge of the protocol understanding of the pacing which payloads are important at which point in time what is the critical path right then you can do things like traffic shaping so you were talking about for example transaction how much goes to transaction how much goes to blobs how much goes to blocks the interesting thing is that um we don't actually currently throttle or prioritize different different classes of of traffic. They're all competing at the same time. And what's even worse is that there's actually constraints. The constraints are set by EIP7870, right? Which uh Camille cited earlier today. Um and they represent both EL and CL like is the totality. Now the EL is

doing his thing and the CL is doing his thing and they're both doing different like they're both competing for the same bandwidth. So they're not synchronizing. um is the EL can for example throttle traffic while the block is still arriving or while we're doing antistations. Uh the CL can throttle traffic in other cases during like the rest of the slots so that the EL has more time to propagate the memp pool and the blob pool like better mechanical sympathy is is is good. >> I see. >> Yeah, makes a lot of sense. >> Yeah. So maybe some ideas here just very quickly. Uh go gossip sub no one say no one size fits all. This is like a key

learning right. So we want to design broadcast layer the broadcast layer from first principles. Uh we're starting to reason about specialized broadcast protocols for each like type of object. Um and in lean this could mean signature like signature dispatch protocol a subnet aggregation protocol. if there are subnets maybe there are no subnets so therefore like you know maybe this >> is really great >> yeah so that's something we have to decide um a global aggregation protocol so basically understanding the real like the actual network load of each of these objects and optimizing them for bandwidth efficiency so some ideas to consider as well or some like directions in terms of routing are we a structured

or unstructured routing do we attempt to be directed or undirected do we do subnetting? Do we like do we incorporate latency as a primitive given that we now are measuring latency? Maybe there's like a an interesting protocol here that we can uh that we've been thinking about in the L1 and we still have to prototype, but a node could actually classify peers in different latency bands and then like specifically send to different regions like with more awareness of where those peers are and what the latencies are. they could decide to like send to different layers so that messages start propagating regionally as well like in different

regions at the same as soon as possible right um yeah so revisit the bandwidth constraints purposeful purposeful peering and scoring this this is uh this is actually very interesting because right now we have um a peers that were connected to and then we try to score them in a linear way and whatever it is it is. Instead, what we're looking into is if we have a per table that is segmented in four different purposes.

Block propagation, atestation propagation, um blob propagation. we can then find the best peers in each of those categories instead of having all those use cases compete for the same share. So that's something that you know I would recommend we think about as well in lean um how constrained devices are going to interact with the network. Are they going to be first class citizens or are they going to interact through an edge layer right like Justin specifically Justin mentioned that one of the like north stars is having mobile devices uh validate the network as well.

How do we like a mobile device is not it's intermittently connected has high latency it is you cannot rely on it for any dissemination. This is actually one one of the problems that we have with a with the L1 today is if you start a node anybody starts a node in their laptop they will instantly like be in service of the network which is which is a problem because it gives you a lot of turn. Yeah, I think there was a project portal network that was working on solving that >> but I guess with like clients on the consens front >> mainly um the reason I'm not sure what the adoption today looks like but um >> mainly the big limitation for having

browser support was that um the clients themselves they were on TCP browsers they only support websockets >> so it was very hard to um get a browser extension running which should directly fetch the data from the P2P network. >> Mhm. Um but like if browser support is to add is to be added does that mean like uh web socket becomes like a additional component in that uh e P2P section or that min P2P section you showed or or just >> it it could be I doubt it because I think architecturally speaking what makes most sense is actually having a

specific edge layer that such that those agents wouldn't would connect to the core network but they would actually connect to the edge layer, right? And that edge layer should be trustless. So very similar to portal maybe, right? Uh at least in like design principles. Um and then like think about a mobile device should probably be receiving a request to sign a payload to validate a pro a proof and to sign and to sign um to sign the to send an attestation. Um and then should be receiving the aggregate at the end. And those are the two interactions that you really care about if you're going to be having a browser extension or mobile device, right? Um yeah. So, but then there is also kind of like how are clients going

to generally use the chain? How they going to submit transactions? How they going to follow the state which I'm not sure exactly um if this is the stage at which those details are being worked out. >> Yeah. >> What do you what do you guys think? Like do you see uh these constraint devices as close to participants on the network or >> first class? Definitely not. I I know the whole uh the whole like sales pitch is I want to validate on my watch on my smartwatch.

>> Uh that's probably going to drive your smartwatch anyway. >> Yeah. I mean uh I think that ties a lot into rainbow staking, right? I mean it's >> maybe there's a rainbow staking but there's also rainbow many things like rainbow participating do you want to be um like you know your Raspberry Pi yes he has maybe your Raspberry Pi can do some validation but it cannot do RPC RPC things like this >> um >> so I think like to address this specific problem of having the edge edge layer you're going to have to know what the whole zoom of uh of uh you know actors on the network are going to look like

>> and I think this is way downstream of of reno sticking >> um so like the of course that's a lot of awesome ideas there are some like this one we can definitely not today >> um but uh yeah maybe uh I I also don't want to preemp the question if anybody else has something but what I would like to have and once again we can wait to address is >> uh is you know if if that were you why would you how would you make it because you know unfortunately I I don't think any of us is a peer-to-peer expert except maybe camel >> um we you know I could I can give you my generalist opinion of having developed

execution for for a long time >> um but uh honestly I >> defer to you to tell to tell us what you think and then we get criticized because of this or that aspect of >> Yeah. >> Constructively of course. >> Yeah. Yeah. Yeah. Yeah. Yeah. That makes sense. Uh >> I mean one one addition maybe regarding the mobile devices and and your touch on this already. I think the >> the latest or I think the broader consensus of mobile devices here is the power of a mobile device.

>> Mhm. >> So not necessarily >> Yeah. Right. Yeah. uh like from like a the processor of a smartwatch. >> Yeah. >> And less sort of the Wi-Fi connection of a smart a smartwatch. >> Yeah. Yeah. >> So that's probably the other. >> Yeah. I mean, I think like we should think about are we really intent on like having like concretely these devices service the network or is it more of a design principle that limited like constrained devices should be able to participate if they want to but they're not going to like we don't expect them to. You know, sometimes you do like you you set these

goals to like design to influence the design, but then like you maybe don't actually intend to or you don't see that that scenario panning out. I don't know. Um but yeah, I think this maybe something we can defer to later. Um, and and yeah, like I can I can give an opinion once I have a better understanding the protocol evolves and and like we're in a better place to to actually think about this. >> I mean I I can tell you one use case where you do want to have the edge layer. It's uh think of a dystopia. It's illegal to run the network.

>> You want to still be able to resist that censorship. >> You want to be able to hide those things. I mean those those notes >> are going to be moving. They're going to be very light, very easy to conceal. >> Do we want that? Absolutely. Do we have to uh you know, do we have to fix it right now? >> I think there are more pressing. Thankfully, there are more pressing >> more. Yeah. >> I also think that we shouldn't expect any batterydriven device >> to actually participate in the peer-to-peer network. Mhm.

>> It's probably not realistic, >> right? >> Mhm. >> Yeah. >> I think one point you brought up which was interesting is sort of um removing uh advantages of collocation. >> Mhm. >> Uh due to peer-to-peer techniques. I think this is a topic which the which we will face in the future. Yeah. Um and even as FDM said like assuming like rainbow staking and sort of uh like admitting that it it's okay for builders as an entity in the block content to to be colloccated. Um like let's let's assume that sort of we

we get to rainbow speaking we like maybe there's interesting part we can discuss to allow for at least the attesters to be like uh have ways to communicate which allows them to be >> fairly um or more widely geo distributed than we have today. >> Right. And I'm not sure I understood the point you had about like uh certain latency latency assumptions how our testers communicate could could be >> um sort of allow for >> Yeah.

>> Um >> like not not putting them at a disadvantage if they are like >> w distributors. >> Yeah. Yeah. Yeah. >> So like an equal footing from like a networking like latency perspective even though they are mighty >> right. I think this is a good segue into into the next point by the way. So um I one one question is Camila are you taking notes or somebody taking notes? >> I thought it's recorded. >> It's being recorded. >> I know. Yeah. Would be good if someone had >> Yeah. Sorry I forgot to ask for it at the top of the session. But um maybe you can spin up a doc and we can wait for

like 3 minutes you can share the link for people to like brain dump what we've heard so far to kind of like make up for um for this this time. >> Uh >> does that make sense? >> No. >> Uh given that nobody has been note takingaking. >> Yeah. But we we recorded right? >> But we it's not super reliable is it? Yeah, it would be good to have both actually like bullet point note takingaking and a >> request. Okay, so let's do that. Uh Camille, can you can you spin up a hackd and just share the link in the in the telegram channel for testn net for devet?

>> Yeah. Okay. >> So >> and then we and then we spend just like three minutes now typing up our notes from the session >> each person. So we crowdsource note takingaking m like making up for the la for the last hour if you remember. If you don't then that's fine. >> I mean that's the thing like maybe what we should do is now try okay I I can put my computer in the Google uh sorry the thing record everything that is being said.

>> Yeah. >> So we have at least we record what's left. >> Okay. And then we do that, right? But I'm going to have to reflect like I'm sure I can I can remember some things. I don't >> It's just uh >> Yeah. >> Yeah. No, it's uh it's fine. Otherwise, I can kind of like um I think I I remember worst or at least the key points and then we can complement it with with the segue like what what >> Yeah. So the segue is onto Okay. So right now the L1 is essentially a flat network. there is no sense of who's a validator and who isn't. Ideally, you want you would want a method to create a fast path. Uh Camille has like mentioned

those words and I think they're right. You create a fast path amongst validators so that they can communicate uh so they so they prioritize communication amongst them uh while still allowing others to participate and observe. Right? Right? So think about like read access and read write access to the to the gossip layer and one potential primitive for that is proof of validator. So I just talked about this I think all of you were were here in the previous session. Um so that would be one way of actually creating kind of like an overlay um and prioritizing traffic amongst validators right and that's why I said like it segus right into into that point

because basically um what you what you were saying is you have mobile devices or constrained devices that want to be first class participants of the network in a way >> maybe I'm thinking a check like yeah If I if I have like fast path and sort of like these terms they like in my mental model they are not necessarily not necessarily good. >> Yeah. because I don't know we have we like what we want is like this um the non-correlation of the test let's say like assuming rainbow thinking we like we want non-correlated testas and so if we >> and correct me if I get the idea right

yeah like if we had a like a privileged quotequote privileged network of just >> ah that's the key word not necessarily privileged in the sense that nobody else can join it or right >> or observe it but you just know who of your peers are actually um attesters or like have a role or are actual nodes validators. So you preferentially send to them, right? And that allows um specifically aggregators, right, to collect the payloads that they need to collect uh to run it through aggregation sooner than a random person running a

laptop would receive it because that's what you really want. >> Okay. >> So you want the critical path of the protocol, >> right? >> Right. to run as efficiently as it can without deanonymizing the raw ideally without deanonymizing who's who's who >> with this uh I'm just worried about like um centralization of stake like if if I have a lot of stake and now I'm sort of not privileged but this like I have a better >> even better view of the test set so I have a I guess a way to like leverage economies of scale because of that then I would be like hesitant in sort of

pushing forward but um >> yeah that's that's one of the difficulties of organizing something like this where when you create the the construct like you could either if you don't represent the balance or the stake >> right you could end up with sib like a sibling out of of this layer because then I can like split my my available balance to across many validators and like you know flood the network um and if you represent it then you have a deanmization vector so it's like you want something in between uh I think like on a conceptual level it is basically like you want I think I would vouch for something like

this together with graceful degradation. So if you have a fast path right it just emerges on the network but the network is able to operate without it maybe at a reduced throughput. It's like baking it baking in this principle which is also like touches on um I think like some of the like doomsday scenarios that we do want to uh cater for as a design principle because you want you want Ethereum to survive such a scenario just lower it's fine maybe right for confidence because it's not going to >> yeah so if you have compromised infrastructure then you want it to continue operating >> but it's fine for that period to for it

to have a hiccup in the sense that you It is running slower. Yeah. >> Okay. >> Yeah. So, uh some more uh traffic shaping coupled to slot timing with back pressure. So, back pressure on specific concepts on specific objects. Um you want ideally like a self-organizing network with emerging roles. So there was some talk about what if we allocate GPU super nodes uh to perform the aggregation. Now of course if you identify them some protocols can afford it because they have like this optimized infrastructure for those cases and they

have a permission network and everything but we're assuming we don't right. So ideally you would want like these roles to be kind of like self-emergent in the network and for the network to continue operating without them as well, right? So is there a way in which you can measure the quality of service of nodes in the network locally, right? and just locally decide that you're going to send your signatures first to this node in your subnet because consistently over time it has appeared to aggregate faster maybe and therefore you think that you know they have a higher like they they they have higher chances of including

your your your signature um and therefore you send so it's kind of like you know you let the structure emerge versus like creating it top Um yeah, so I already mentioned the next point which is getting rid of the networking versus consensus consensus duality. What I call what I tend to call is like aiming for zero idle time in terms of resources. So you maximize available resource utilization. I think we kind of like touched on this and and agreed. Um yeah, something that we learned is like really actually specifically spending time optimizing peer-to-peer networking stacks is valuable because otherwise uh we see a specific performance and we

assume like it's it's it's very limited by design and not by implementation. So now we're actually spending a lot of time looking into the in profiling uh profiling of uh implementations and we're actually discovering several bottlenecks there not necessarily in the peer-to-peer networking stack but also in the in the clients. So we don't want to be in this place where we just take implementation um like implementation hotspots to be protocol design flaws. Um and then yeah just what do we want to do with like our essential principles right so do we want to have some form of embedded validator privacy can we have mix nets can we have onion routing can

we have like some of these methods comma dangling comma for like other other methods um what do we do in terms of censorship resistance right like these are things that I think we should be thinking about as well um which like you know are usually at the other end of the trade-off spectra, which is the problem, right? Because now if you want to introduce onion routing, well, you're you're introducing extra hops, right? So now you're introducing latency and therefore that penalizes the overall throughput of the protocol unless you find a way of like doing maybe delayed execution of some kind where you give things more space to like

I don't know uh to become um like you was the pipelining there right where you can make this work maybe I don't know >> yeah the only problem with this uh delay execution is that if you give more slack people are going to start using it >> and you're going to have to find >> more timely. It's a losing game, right? Losing battle. Yeah. >> Well, so that's kind like I think everything that I had today. It was a lot. Um, so this >> pressure >> Oh, back pressure is the notion that if your application is slower then you your producer you rate limit your producer as well. So you're kind of

like filling up a queue up until a point where you can't take anymore. And at that point the other uh party knows that you're in that stage, right? And they stop sending and they stop dropping messages and they start dropping messages, right? Um so the idea there is for example we're like rate we're prioritizing right now signature propagation for example then we might want to like that will if we have limited bandwidth and we have like too many signatures to propagate that's going to naturally back pressure everything else right but you can only achieve that like like application level back pressuring by actually designing for it and your

protocol because you need to structure your protocol in that way. Um so for example, gossip of today would not be a good example because all topics are flowing through the same streams but um a like if you had different streams for different purposes then with Quake Quake would like uh use up a entire window when there is back pressure and the the the producer would know. >> Yeah. >> Yeah. So if I remember correctly, we have dedicated session for matrix tomorrow. And so do you have any uh advices for uh bootstrapping this metrics regarding P2P?

>> Oh yeah, a lot. >> Yeah, a lot. Yeah, there's there's like a lot of a lot of metrics there that you want. You want them um at a lower level networking stack like kernel metrics in terms of um um you want to have congestion control metrics, you want to have like connectivity metrics and and other things. Um you bandwidth utilization per like peer as well. Then you want to have um like the actual networking stack metric. So uh what transports you using? Uh what is your churn rate? what is the size of your peer table like all these things and then you want to have like metrics uh for the protocols themselves. So if

you're using attestation like if you're using broadcast protocols then like metrics for those duplicate rate you want to have like messages uh peers per group and then you want to have application level ones. So yeah we can different layers there. Um sorry just uh rebounding on what you said um do you know how to collect those metrics from a regular like machine? >> Yeah uh there's a few libraries and like if you look at um net data for example uh which is actually a full web app >> right uh that does like the whole if you look at how what it does under the hood

that would be a good thing. Another another good starting place would be the Prometheus exporter, the node exporter, >> which exports system stats as well. I don't know if it goes to a pure level, but I've seen like there's libraries in most programming languages that that already do. >> I mean the peer level is at the app level this it's just I mean I used to do a lot of experimental development. Yeah, >> I never looked at the network side of it. So that's where actually >> I think you collect them through netlink uh through the net link interface. Okay.

>> I think you have a bunch of messages that you can send through that that socket to >> Okay. >> to collect uh stats. >> Okay. >> Yeah. >> Understood. >> Cuz you don't want to go the EVPF way because that would >> That's true. You can do that. >> You can you can Yeah. >> Yeah. Okay. >> Yeah. So out of like these ideas like is there a way to to group them into like in terms of client implementation like a base like minimum design that you want to test out?

>> Yeah. Um we we can with your help. Um we're a small team in in the EF so like we don't want to centralize this design. Like my my goal here is not to tell you what I'm like I would like to do myself or my team. It's more like transfer the knowledge so that we can self-organize as a community to actually put together a spec that hits on that learns from the past and kind of like uh aligns with the direction of the L1 as well uh so that we can like work together.

Right. >> But I mean given idea from earlier like since we have you as an expert. >> Yeah. Yeah. >> Um and what we can say is sort of we at this stage at least let's say we care about like two three four things. >> Let's say for example >> sticking to EIP780. So let's say N's law may not apply to the globe equally. So we will always have like >> uh whatever certain regions which will be stuck at 50. >> Yeah. So >> and with the so with that in mind and sort of with the goal and then assuming sort of the rainbow staking so we we may what we care about is like the the geo diversify the test set maybe that's

another thing we >> from a networking we can also like prioritize >> what we can add is >> uh maybe valid data privacy is good but maybe not a must so if we like have like three four five six assumptions. >> Yeah. Um, I would say, okay, this would be really cool to at least start with quote. Like, how would you how would you start or where would you where would you say, "Okay, guys, okay, um, let's start with this." >> Yeah. Yeah. I think what I would do is um I would take and structure probably kind of like a spec with a lot of gaps that we should with with options right

inside each of those gaps. So we know for example just to give a very clear example with um uh signature propagation right we have a like very ample design space which includes subnets maybe not there's a structure topologies there's the grid topology there is gossip sub like we know gossip server is probably not the the the thing to use um so like that would be I think we I I can inform like different directions that I would like us to explore basically in each of these areas and then we can work together and create a set of projects right to go and prototype those directions in a more comprehensive way.

>> I think would that help? >> Yeah. >> Yeah. >> I think what could all we call could also do is like headliners like hard forks where we come up with like some simple headliners where what we want to achieve with this networking design and then group all those like features that we try out into >> try. I was thinking like for if we want to place them in a sequence specifically, uh like having something like this would be really useful >> cuz like ultimately we want to prioritize depending on where each devet what each devet wants to shoot for.

>> But yeah, I'm totally happy like you know helping structure all of this. >> Yeah. So for for the dev net too we already need to have applications. So we should have some at least direction like what we should be able to implement. >> Yeah, >> as I presented just having cosep up I'm afraid is not might not be enough if we target like 8k 16k validators. >> If it's if uh devet 2 is going to be the first devet that's going to have the me like the that's going to test out the mechanics of aggregation. Yes, >> I would keep it really simple from a networking perspective. just do the dumb thing from a networking perspective because what you want to test there is that aggregation works and you want to

get an understanding of the performance profile before you go and like optimize the network for right because you probably want to like you're going to have many rounds of compute optimization of of aggregation right >> until you get to like something that is >> the simplest thing would be just rely still on gossip just reimplement what we have for BLS aggregation Yeah, >> like kind of the same type of the >> uh subnets maybe have maybe not at the scale that Justin and everyone talks not 8K not but maybe I don't know maybe thousand with >> each committee or each thousand of maybe 128 volator something like that

>> that could be just one thing if we just want to test aggregation if >> and then yeah we we kind of get the numbers like uh I don't know how long it takes to aggregate >> maybe for the DevNet 3 then target some fast path priority fast path topologies I don't know >> so two two questions here um what are we doing for telemetry or is this the subject of tomorrow because as we test things out we want to capture telemetry of every single node to like roll up to a dashboard and understand how things are performing in the wild um and like it could be totally fine for now. I would say if you actually have a top-

down um if if we have a top down network in the sense that there's no P to there's no P2P per se that could be rate limiting. I think we're at this stage maybe we're at the discovery of how fast like we can go in terms of if you remove the network from the picture like a peer-to-peer network and you just have normal one hop networking imagine right and you have like a centralization there so say for example I have a a thousand node network right and I want to test out what a optimal subnet uh design would be like if we have 16 subnets then a a node would connect to like a server that would place it in a

subnet and would just like do a mesh without gossip sub or like it would maybe like even like centralized through a server. I don't know like I think I want to first like understand how fast these protocols can go um sorry this these implementations can go of the crypto side of things because that's going to ultimately determine like what the like what is it's also going to determine like what the network >> I think signatures per second and 10 starts per second. So that's the compute target that we Yeah, but we don't know over time like how that is going to achie be achieved

right in in each devet. >> Yeah, >> I don't know. It's just a wild idea out there. >> So that means we would be downstream like this decision would come downstream of actually knowing uh the crypto. So that means for techn one we would be doing like signature aggregation except uh we were talking about using BLS instead of of this X number 50 X MMS or XMMSS >> um >> but uh okay so that >> yeah uh what I mean is we're starting to see a tree of dependencies of what we have to figure out that I'm afraid has some loops.

>> Yeah. Um, >> yeah. >> And uh, yeah, we're going to have to break that somehow. >> Yeah. Yeah. Yeah. I was thinking this is just like we're just brainstorming, right? So, guess I don't want to feel like I'm giving any any any instructions here. But like my uh my thought here is I would I would really like to have a baseline of what is like with a million node network or like whatever we're shooting for ultimately uh if I have a central coordinator that just simulates different topologies >> then I can like try out every epoch or every like day I can try

out different formulas of organizing that work >> without actually having to like implement something and roll it out and like the the feedback loop becomes way way shorter. And then once we manage to organize nodes in a specific way, we can then figure out like how a potentially how we can like how the the peer-to-peer equivalent of that organization would work and b like how much we're willing to compromise and leave on the table to like get the peer-to-peer to actually transition to peerto-peer.

>> So we have a good baseline. you know like bare metal like fast fast like you know the the the baseline like you know remove the network overhead out of the picture um then we simulate network overheads by dispatching and like introducing latencies and so on and command like you know >> command and control wait uh and then we actually with all that data we actually go and and figure things out because otherwise I think we're just like trying out things and shooting in the dark a little bit and the feedback loop would be maybe like too long.

>> So how do we build a simulator? Can we reuse what you guys have done or >> Yes. >> Yeah. >> I mean yeah and what you said that we're trying things. This is where again coming back to structured topologies they could be really helpful. I mean for them it's much easier to predict like how much uh how many hopes we have. We cannot say this about gossip for example. Uh so some parts yeah we can predict pretty accurately. Some parts we need to simulate before we actually understand. >> Yeah.

>> So yeah we still need simulators. >> Okay. So maybe because we're are reaching uh we're at time. >> Sorry. Do do you think that like this in the wild actual like command and control kind of like way of organizing the network would be useful? Do you think that with a simulator only like where you have a good enough data? >> Yeah. Could you repeat like >> Yeah. So essentially uh we're trying to like get to an understanding of what network organization >> Yes.

>> would be best. >> Right. >> Yes. >> Uh so is a simulator enough or like actually seeing it in the wild and literally organizing the network differently every day because we have a command and control way of doing it would uh be >> faster. I mean we >> Yeah. Yeah, the only thing is yeah, it's it's hard to verify like how accurate it is and we're also taking some shortcuts in simulator to actually make it run faster. It's uh very resource consuming to uh simulate large networks. So for example when we simulate for example we couldn't simulate quick it was just too slow because it was we integrated quick into NS3 but it was doing all these

cryptography operations to which we actually did need for our performance metrics we assume they usually very fast. Uh yeah there are some shortcuts. Uh there ideally would be like we have um yeah this simulator like we wrote then we have clients uh then we have uh ideally they are compatible with shadow so we can test different assumptions with different networking bandwidth latencies all of that. Yeah. >> And after that, yeah, we're actually confident. We know like how liquid behaves under different conditions.

>> And then we can test it out. Yeah. >> Yeah. >> In some real environments, >> shadow is is great, but >> yeah, >> it's difficult to run in like a resource >> which is why we start with first uh yeah simpler simulators like ours resource. Yeah. So this thing that you said about structured I think this is a very important decision to make um in terms of structured topologies or not. Um >> yeah it's not like structure or not.

It's like whether we can integrate them uh as a fast path maybe or in addition to existing topologies. Uh yeah >> yeah. The thing is that with a structured topology you would specifically designate like you you would need to know like you would need to know the end points of of the of the pure. So this like >> this >> we should >> undermines other data. >> We should always make an assumption that uh whoever has uh their address exposed they might be under attack. they might not be accessible. Uh so which is why they only but still in

most cases in optimistic scenario they should work but we should work also without them somehow. >> Yeah. Yeah. This is something that the L1 has been avoiding for a long time. >> Yes. >> So I don't I don't think that like it's a I I wouldn't be comfortable making that compromise for for >> Yeah. It's something that well I just don't don't know how to achieve these aggregation targets faster. I mean we have 4 seconds we have 3 kilobyte signatures. Uh >> is there a way that we can find um like we can construct a structured topology.

>> Mhm. >> Um just by I think by by observing and deciding. So, Gossip Sub in reality is kind of semistructured. It is not unstructured in the sense that it does have a scoring algorithm that prioritizes latency as one of the >> um as one of the par the the parameters. So, it ends up in theory constructing a it can end up constructing a latency optimized mesh if you configure it right. Um but not consistently because with with gossip hub you have every slot has a different producer

>> only gives scores to your neighbors. >> Yeah. But >> exactly and then you evaluate other neighbors as well as you connect to them. So you're using uh like as you connect to them you're incorporating them in your non-mesh. on your P mesh and I think you can promote them right to replace another peer if you're hearing I haves from them >> before you're actually receiving messages from your peers >> I mean okay yeah that's the problem is >> yeah so it's so it's a little bit kind

of like this adaptive kind of like MP sub sort of like in structured topology as well. I mean, we see that >> um Yeah. >> who's following the rules. If we have, for example, read, who is sending messages faster? >> Yeah. >> Cuz for the L1, we're actually the the direction that we're really keen on right now is a racial coded broadcast, but it does not apply to >> But it's for blocks probably. >> Exactly. It does not apply to many like simultaneous producers. It might be doesn't have to apply it for uh global snark aggregation which has size of 128 kilobytes.

>> Yeah. >> Yeah. So this is so this is where we might actually take that. >> Yeah. Yes. Absolutely. >> Yeah. Because for now it's uh yeah distribute. That's another question which we didn't explore. For that we can actually uh go the Salana style >> and actually split up the payload into MTU sized chunks. >> Yeah. >> And then distribute them over UDP. So it's very like very fast. >> Let's try this in one of the dev nets. I mean we can so so basically we have player one as >> testing this uh encoding stuff which we never did. that this is the fastest like you know I tell you like this is going to be the fastest yeah that we have

great let's introduce this to >> global aggregation is like redifying isn't it or >> I mean no because global it's not necessarily global aggregation it's just once we have some aggregation it has size 128 kilob >> okay okay got it got it >> even even if we have just a single subnet we need to distribute the final start to everyone. >> Yeah. Yeah. Yeah. >> Yep. >> So to do that test in a dev net uh would we need the actual signature thing for at least you know the size of >> No, we we would need just some data of of size 128 kilobytes to test this or even larger size.

>> Okay. So we could take we could take just a PL signature to test the application. We can just take I propose this on one of the calls just message trust me I collected enough signatures I'm good I'm okay and f the remaining part with zero so that make it 128 kilobytes >> um I I'm not sure whether we really need to complicate things with BLS to be honest >> yeah it's just we we need to if we want to test P2P layer we just need the size of the message particular size of this message. Uh >> so okay so could we make uh could we say that's that's the question we're trying

to answer. Could we say death one is going to be exactly about this so that we collect exactly the data that we want >> and we can move on uh you know we can make a decision during the debrief of one so we fill this this column somewhere that corresponds to time. >> Yeah. Yeah. So for the for the devet zero for example we don't have we don't really have the signature size of 3 kilobytes. So maybe for devet one we can target this >> and then we can collect the necessary data like how long it takes to >> whether we can for example collect enough signatures to do fast confirms for the 3sf >> stuff. So then you you don't actually have the aggregation itself but you have

the like something simulating it and it's deterministic so you can verify that each validator is able to decode right. >> Yeah. And then we can move to the next two and make them faster but actually collect data. >> That would be awesome. So yeah uh I think uh >> uh the other thing is we're at time and uh I'm hungry. Uh uh we uh yeah I would like to say I would still love you to describe your ideal thing because we're going to recon tomorrow and you know we decided we would do some coding and stuff like this but we can also take the time to uh you know to collect I mean like do this by the way

my recording has failed so we're still going to have to go through the hack ID and everybody has their stuff that's okay >> uh and then tomorrow we we convene and say you know you you send you try to send out your ideal thing and tomorrow we reconvene and we say okay we want that we don't want that you know >> like just to have a baseline so that we can >> yeah I would really want to like have understanding for the death too like we have some kind of aggregation how we actually do this I mean >> I I'm trying I I still don't have that understanding so for definite one we do we do aggregation just for a single subnet. Is it correct or uh

>> uh so that's the thing like they were saying earlier someone told me earlier today we're not going to have realistically if you want to do aggregations they don't know how long it's going to take. Yeah, I mean let's keep crypto. Let's avoid cryptography just some dump aggregation with some mock data. >> Okay, fair. >> Yeah. Can we do this for the defet one or >> Yeah. >> And then for devet two my understanding was that we actually going to start implementing uh the subnets things like that. Uh but for that we need to have if this if this is correct we have to understand like what topology we should have and all of that. But that it's still not clear for me yet.

>> Um I would say I mean yeah we're not going to decide today probably what two would look like. Uh but I think if we have an agreement then on the networking side one looks exactly like that like this like 120 kilobyte messages. >> Yeah. >> Uh some kind of trust me bro approach doesn't doesn't matter at the moment because we won't have the crypto. We just get rid of BLS and that's fine. Um and then on the what's that called you know we still have to decide decide what def one is going to look like in terms of other things like what you know what payload do we put in the blocks like uh I'm sure there's more stuff we can put that is not network related

>> but I think if we zoom in into the network uh aspect and we already decide this is what we want we can uh we can implement >> this you know this Yeah. So, >> oh, >> okay. So, for Yeah, great. Maybe you have devet one with some dump data. Should we experiment with this encoding stuff for death one? >> Right. Exactly. That's what I was going to say. Sorry, I just wanted to ask in that u you know east P2P thing could we have a minimal subset that of things we could implement that would be ready for def one that would still let us you know stuff we know is most likely going to be there in the final product

>> even if it's not the whole stack. >> Yeah. So we want a we want a spec minimal minimal spec would be ideal right of a racial coded broadcast that we can implement with with synthetic like random payloads. Yeah. Deterministic random payloads and determin for dev one. >> Exactly. >> Right. >> Yep. Uh is it fine like are we okay just not using lip for this specific test? I would be really >> I mean we still use lip but just to connect to send message >> or or what?

>> Um >> well how are you? >> No I would I would just use quick a do we all have a quick library available? Uh >> yes >> that's good. Oh okay. >> That that way we get like the closest baseline to what the protocol would be without involving the B2B or the networking like a networking specific networking part. >> Can't we just Sorry. >> Well, I was just questioning uh so um if we're not going to use lip PP but use quick, we'll still have peer ids and multi addresses and >> yeah so uh that that's the idea. So we'll be adopting like maybe what we can do is adopt the quick transport from

libby tob that does the the handshake and the form that libby does it today with a pure ID. So we'll still be having this um but then we can just consume the quick uh transport directly instead of like having to go through the abstractions of lip tob which obstructions >> um so >> because I think quick handles most of the abstractions we don't have yanox there >> exactly that's that's what I mean so we don't do so we don't use the libby abstractions we just go to quick directly but we use the libby handshake for quick which uses uh the TLS 1.3 the ALPN from Libby tob and everything.

>> Mhm. >> Um and authenticates the public key against the pair. >> Does that make sense? >> Okay. I was not implementing quick myself so I might but it's like it sounds to me like even if it's part of the li P2P it's already good enough so we can use it. I mean >> just to open a stream to some beer using quick but over li P2P not avoiding it it doesn't make much difference using it as part of li2p or without lip but with lip it would be a bit simpler because we already have the integration >> yes we use okay so we use libp >> I mean everyone every client here has their I mean for devet zero we target to

use only quick so and and we already use quick transfer >> okay that's good Yeah. >> Yeah. I just didn't know if you were like doing the whole spiel with LibyP. Okay. If we just >> We're only using We don't have like >> Yes. We use LibyP. >> Yes. >> And we build a custom protocol that is no costub. >> Yes. >> That does um the eraser code broadcast. >> Yeah. Fine. >> Y >> So we are targeting for devet one for the custom uh networking stack. um >> uh not necessarily custom but on top of the li P2P streams for example stream it's gonna be some stream open >> yeah li2p streams so I would say uh for now lean should continue using lib >> uh just don't build a lot on it right

now because you know we are evolving the networking stack and like lib is going to be there in mainet for at least a year right so it's it's not going away like we shouldn't move away from it um in in lean and it's also like you know we want to make sure that like we explain why like you know we put the right messaging out and everything in terms of Ethereum has a that Ethereum as a community understands why we're like changing our networking stack in the future when it comes to that. So we don't want to like you know jump the gun. Uh so let's just continue using L2P but just the bare bones of L2P which is quick whether it's like with a with a

streams um and that's it basically. Okay, >> I think there's a timeline question as well whether like implementing the the P26 plus the networking changes >> how would that affect the timeline whether whether that's actually >> what is the time for one >> well from original discussion we hope to do it before connect >> is it feasible to do >> I mean >> I mean I know it's it looks good to do it before connect but if we actually do something of value and we miss the deadline like I don't care. See what I mean? I mean, I'm sure Justin might be might not like it so much, but uh as far

as I'm concerned, I'd rather bring stuff, you know, >> we can do this as part of Defet 2. If I mean, for example, if coding part would slow us down. >> Yeah. But do we have lots of task remaining in the PG integration? So there's a library provided. Yeah, my main concern is like we this is good to do because it fills another part of the stack. Um, but for me the biggest unknown is the first part which is the actual signature dispatch and the initial aggregation to the initial aggregator that that's like one unknown but I think that's going to deserve a lot more um like a lot more research.

Okay, so just to recap uh and so that we have we we use the P2P as is today even with our Rust implementation which I'd love to delete because it takes forever to compile. Um but okay, we do this but then we add a new protocol on top of it which uh I guess we could reuse if worst case we reuse it from Rust if we don't have a better interface with our stuff. Uh I don't know did you how um I know I know it it's mostly cut or ged these days but would you think do you think it would be possible easy to do at least in >> uh I think with most we have most of the quick stuff already that we don't have to be captions on top of it

>> so that would save some time >> okay >> so we said this is for dev net one and for dev net 2 we want to start using real aggregate Is that the idea >> would be I as I saw this on dev net 2 we actually want to start experimenting with the subnets so that we also have local aggregation and global aggregation maybe it's too challenging for to do to finish before the I don't know what was the target end of the year or >> the two would be before the end of the year uh does anybody know like uh Okay.

>> Okay. Well, it can be completely it doesn't have to be the crypto that we're depending on whatever happens in the other room in terms of crypto and how fast they can deliver. So, all that matters is that we you know some >> I mean once they figure out the crypto provide us with bindings I mean >> and if we tested everything else then I mean we're good. We should check like P2P layers works correctly. We have interrog Yeah. I mean replacing stuff with crypto stuff should be >> if they don't have the crypto already, we can still do something meaningful to >> Yeah.

>> Okay. >> Okay. Okay. So I think the so my main takeaway here and by the way just of this part of the discussion I just at this end I posted the hack and link in the telegram group on devet in the devet channel. >> Um so please go ahead. I've left room for you to copy and just uh your to put in your name and dump your notes. Yeah. So, let's all take maybe 5 minutes to do this to make sure that we captured like most important bits of the conversation.

>> Could you copy your slides as well? >> Yeah. >> To help. >> Let me just remember. >> So, this is the the Yeah, I see that people are on it already. Should we wrap up the official session then? >> Yeah. >> Or anything. >> Just Yeah, just give us five minutes to like finish dumping our thoughts in the doc and then I think we we're done.

>> Yeah. >> Oh, okay. But just in terms of like the >> Why is this not the async part of the Probably >> I think what we can do is if people have time because we're over time already, right? >> We over time and we we need to go to the like at least like punting starts at 5:30. >> What is >> hunting on the cam to the to the to dinner >> a small boat? >> No, like you >> it's like Venus but better. Anyway, I think we can finish the session like for now and then sort of just dump the notes and then continue.

>> Yeah, >> sure. So uh but then we have to