Devnet Breakout (p2p)

Transcript

thing that they're shooting for but probably a more likely realistic value for now to model against is to 156 >> double >> double so I'm just I'm going to change that because it's changed in the code as well okay cool >> this is for global aggregation >> this for global aggregation so I've kept this part like this is the end goal really like local aggregation is this is I view it as the means to get to the end which is this cuz ideally if you if you were able to have a flat network and just aggregate once that would be the ideal scenario but because this is not viable we'll need to introduce layers so um

>> it might be viable with TPUs but yeah >> I mean you still okay so if you look at the raw I I I I doubt it because if you look at if you do the very like simple multiplication of how much traffic this requires it's like around 3 gig if you're shooting for 1 million validators, right? So, I think you are necessarily going to need to layer and probably layer in more than one level if you're shooting for that, right? If you're shooting for 16K, uh then that would be like 49 50 meg >> still, >> which is, you know, probably okay in a setting where you know who you're

sending to. um and that node is fine paying for the ingress of that traffic which is yeah I I I don't know if that is a viable scenario but we can run the numbers and like figure out if that is if that is potentially viable in the future I don't I don't know cool um and then the idea here is to distribute the result back to all validators before the next law begins right so you want to you want validators to have the aggregate the aggregate signature so that they can build or verify the right state transition next.

Cool. Uh so just um a few numbers here that we can we can shoot for and this is work in progress but is this parts that I haven't figured out yet. Um so let's say that we're shooting for in this prototype for 16k validator 16k nodes which is also the size of the consensus layer network today like around the same magnitude right. Uh we assume that we have 32 global aggregators. >> Is that a fair number? >> Okay. Sorry, I just had a quick question before you asked that.

>> Yes. >> You said that's the validator count, but you say it's 16k nodes. >> It's not 16k validators. I think >> no. So I'm assuming a one one mapping here. >> Okay. >> Yeah. So 16 validator nodes, 16 nodes. We can call this node count as well if we prefer. But yeah, so I'm assuming a onetoone correspondence here. basically deciding validators you can say. >> Yeah. And the reason I'm doing this is because um multi many validators behind having many validators behind a beacon node as a centralization vector and kind of like one of the goals of lean as well is to commoditize you know the protocol so that even constrained devices can potentially participate. So

I'm assuming like you know the ideal scenario is you have one validator in one device assuming that >> model right um if there is for example say 10 validators behind a node then that node would have to transmit uh three times 10 right um signatures maybe they can compress them in some way but whatever so is that good do we agree with like this modeling for now like all of this is Yeah. Um, okay. Global aggregators, 32 global aggregators. Does that seem like a fair number to model against?

>> We did 100 in our simulations, but I don't know. We just picked it out of nowhere. >> Yeah. So we can we can try we can do like different different parameter sets as well and like 164 they are permissionless or uh they should somehow be you know registered kind of type or anything. So one thing that would be that like this is I think we should have dynamic aggregators that are selected by the protocol similar to the way that the beacon node behaves today and simulate with that and use that as a baseline. So take the beacon like the beacon uh the the the beacon chain as a as a baseline and kind like

do what the beacon chain does today and then we can start changing things from there. >> Um what is a global creator versus a local creator. >> So this is so the idea is to use the same protocol in the two layers of aggregation. So for both global aggregates and I'll go into this in a second and also for kind of like the local aggregations in the subnet like yeah >> um the question for both kind of global and local aggregators is whether they are also validators or not.

Well, the it doesn't matter that much. Yeah, >> doesn't matter that much for now. Yeah, just for information. Justin said that yeah, in theory they might run some valators might run some kind of tests to prove that they are powerful enough to actually become aggregators and then they also aggregate. Yeah, >> that was just some idea but >> yeah which uh for example in Sana in Aville >> they actually today aortion bandwidth share based on stake >> so they require validators to have enough to supply enough bandwidth capacity to the network in proportion to the stake

>> I don't know how they track this >> but would be good because I think they it would be good to look into it because I think they they um they published their their p their code recently. So we can look into how does how to do how they do this. >> So does the local aggregators are out of 256 validator in in single subnet? Do I understand correctly? >> I would assume yes. >> Yeah. Um so basically the idea is that if you're able to just like today in the in the network you assign validators to committees.

>> Uh what you can do is then give them the role of the aggregator as well which is like very similar to what happens today the the beacon chain. It's like if you look at the beacon chain it's very similar. It's you have aggregators in a subnet and then you have sync aggregates which are global. >> Yeah. >> So it would be very similar to that. Um we also would be modeling against different subnet counts I think and this would be could be actually a proportion like this could be dynamic and it could be a formula based on the valid data count and um um and like some desired target for local aggregators for how for local sorry local nodes that we want to have in a subnet at each point in time.

Um yeah and then >> quick do we need to Um, it's recording. >> Oh, great. Hey, I didn't know I was being recorded. >> Great. >> Oh, yeah. >> Surprise. >> It's being streamed live. >> It's not stream. It's not stream. >> Yeah. Okay, cool. Uh, so then that would give us a subnet validator count of around 256, which seems like a interesting target um to to pick. Now, uh, payload sizing. So modeling against the aggregate size the raw size of 256 which is >> I commented before we went too far from that.

>> Yeah. >> Uh just for information with subnet count of 64 uh as I said like the expected aggregation rate for snarks is 10 snarks per second. So that means we have 64 snarks which we need to aggregate >> with 10 seconds per slot it's 10 snarks per second is like six seconds. So that's should be ideally should be all right. >> But if you have GPUs, of course, we have different snack 64 and viable.

>> Let's just say that for now. Um, cool. And then are we are we good on this stock? Yeah. Cool. And then we have an aggregate. So let's move on to the next block. This is the payload sizing. So what we're actually going to send is an aggregate. The aggregate currently is a realistic size for it is 256. If we extend it for erasia coding broadcast um a racial coded broadcast with read Solomon codes would be having twice the size. So really what we' be transmitting is 200 512 kilobytes.

>> Cool. Good there. >> Y. >> Okay. Cool. So now one network approach that we can take here is to split this payload into MTUized chunks and use UDP to spray these chunks around within the subnet uh and at a at a global level. Mhm. >> This is a very interesting like it the I really like this design uh avenue because it parallelizes it allows us to parallelize uh like basically all network parts because you just have data that's disseminating. The important bit there is that it has to be authenticatable, right? Because like otherwise you could be open to just

receiving garbage. Uh and you don't know because it's erasia cod you don't know what to expect, right? So that's the important bit. Um a safe MTU. So this would be basically the are we all familiar with what MTU um is? This is basically the network transmission like unit. Um which is when you look at UDP usually there is like you can just send a single frame. Um if you can send larger payloads but that you like you have to build the framing and the chunking in your own and then the retransmissions and so on in in your own application protocol. So really the best way to use uh to use UDP and and quick datagramgrams is to uh just target MTU size and we want to pick an MTU. So

quick actually dynamically discovers the MTU of every every every path. Um and uh like a safe value because we want to pick a value that is static because we're going to be designing a protocol against it. Even though quick calculates a runtime like a dynamic MTU for the path, we can't use that um because not all paths are guaranteed to be same in the network. So we have to settle on some safe MTU and that would be um if we want to support IB IPv6 which which we should absolutely should it it would be it would be uh 1,280 uh bytes which then would need to

contain not just the chunk data but also the framing around the chunk and enough contextual information to determine whether the chunk is valid without before integrating it into the decoding matrix. >> Uh sorry if I I just uh you want to be too fast like why is IPv6 worth an IP4 for this thing? >> I I I have to look into the details of this. I'm not exactly sure why, but I think it's because of the extended uh like the the length of the IPv6 address takes up more space in the frag in the in the >> in the network fragment essentially. Um and like once you start looking at so the values are different for TCP and UDP

because they take up different like they have different framings kind of like you and you start detracting right subtracting uh you end up with this like that being a safe margin a safe value for >> IPv6 the worst case scenario essentially which is IPv6 >> uh with UDP I think and not TCP um and like some other variable there. >> Cool. Uh so within that we would need to fit the actual chunk data but also the framing and the enveloping of the of the chunk which would include a signature. I think this is like we're going into like a specific protocol design now. So this is kind of like the context right

>> and now a specific uh schema here for this would be to have a signature where that is that is sent by the aggre the signature of the aggregator essentially so the aggregator signs over the every chunk so it's authenticated at origin the network or the subnet and the network knows who the aggregators are so they can validate that the chunk is coming from the right source Um, a potential attack vector here could be the aggregator could sign garbage and you wouldn't be able to decode. Is that slashable? Is that penalizable? I don't know if it's even worth it because you

know presumably the aggregator would be having some incentive to aggregate and they would forfeate on that incentive if their aggregate does not get integrated into the chain. So I don't know if this is the case but uh but I would assume it goes in that direction. Um so then we take the MTU and we start subtracting the signature size and some overhead that I'm reserving here just to account for some framing overhead. And we can go into what the framing could be. Um and that would give us a specific chunk count which I think if we resolve for this it we do the ceiling division um of the aggregate size over the raw size of the chunk and that gives us around 432 chunks per slot per aggregate in a slot.

>> Do we use BLS to sign a chunk? >> Yeah. So this is one question I had for for you guys. What are we using as a identity um I guess an authentication scheme here for identity? >> I guess we cannot use XS because it's just one signature slot. >> For sure. >> Yeah. >> What do we need for what what do we expect for this signature scheme? >> Can we use P2P for that? Yeah, >> if you can map the P2P identity to the aggregator to the aggregator identity as it's like mapped in the protocol as it's uh registered recorded in the protocol then yes I assume that the protocol is

going to be tracking a public key >> what is that public key >> we don't I thought one of the principle was that we don't want this traffic to exist >> exactly I I don't I don't think that's the that's the right choice >> yes okay >> I think it should be the protocol node and the pup key. >> Huh? >> Between the node and the validator key. >> Exactly. Yeah. >> Mhm. >> So the validator pop key is right now what scheme are we using >> except Yeah. So it's >> okay. And it's only one shot signatures that we can produce. Can we produce uh s like one shot signatures per one signature per chunk?

>> Can we nest? that would increase the size of the uh key like chunk size number of chunk types and the keys they already pretty big so we need to like >> u yeah we can talk to Benedict he's know better about that but yeah like we need to spend >> multiple days to produce enough keys for the >> I don't know maybe eight years >> and now if we say that okay we actually want those signed chunks Yeah.

>> Yeah. >> If the aggregator signing uh like if the aggregator is a sub if aggregators are a subset of all valid datas and uh the aggregator does not sign does not is not in a tester in the same slot where they are. >> They could be both the testers and the beginners >> for the same slot. >> Why not? >> Yeah. I mean today >> Yeah. Today it is the case. Yeah. >> Yeah. But like maybe there are reasons not to be the case.

>> Yeah. >> In the future. >> It's it's not super improbable. It's not super probable. I think um that they are but it could be the case. Yeah. The protocol doesn't prevent it. >> Um >> I guess this was just to say that sort of >> if if sort of they don't test and aggregate >> in the same slot like there's no dependency for the key size or key. >> Honestly, I don't think only said only sign was >> just to like iterate a little bit. I don't think we need postquantum properties for this kind of authentication >> because it's ephemeral. It's a message like what you care about is the end result. Uh this is you're just authenticated data authenticating data

while it's in transit. So I don't think you need we need postquantum uh properties for for this postquantum security but um that would require still that a validator have two keys one that is PQ and one that isn't. So for ephemeral operations like this they would use their and this could be this would be recorded both keys would be recorded in their entry in the protocol. Okay, I think we can just put this as to do maybe figure out what key to use. But now it sounds like there should be a third key magician key.

>> Yeah. Yep. It it could be that. Um yeah, I mean at that point like if there Yeah, we can we can figure it out. uh for now. I think the problem with so does the BL I think like if we have a ephemeral key that we just use for network operations and it always remains constant isn't it just like a pure like like a networking public key it's just equivalent in terms of behavior >> to a networking key uh which then like maps to a pure ID and like so maybe this rotates

um like We need to figure out how we do this mapping in a way that it does not map back to a particular node in a network identity permanently. So maybe there's like some some like key generation here that scheme that we can use to cycle this key. I don't know sign this key signature key of >> yeah because the thing is if you rotate I mean I assume I don't even know if let you do some key derivation does >> but if you do that and you use an quantum secure sign >> ultimately you could go back to the to the previous like to the actual owner

>> and figure out what that is. Yeah. >> Uh the other thing I I was just thinking can you if uh you say those those things are transient. >> Yeah. >> Right. I guess this is why you want to to rotate the keys but uh because if you don't you find yourself uh potentially someone with a quantum computer could eventually start uh signing messages and pretend to be you even though uh >> they don't need to know who you are.

They cannot break the key that fast even quantum even when quantities >> it's impossible right now but >> no it's mean when even when it's going to be possible it's going to take not like 1 second or two seconds it's like >> yeah so if you >> right >> yeah but you can reuse like if you know that the key does not change then you can amortize that compute and use that key like you do it once and then you can reuse that key as many times that Yeah. >> Yeah. So, we need to never reuse the keys. What I'm saying? Yeah.

>> Yeah. >> Um, but yes, maybe uh maybe I missed that part. But did you did you want to use BLS in the final? >> No, it's just a placeholder honestly. Like we can we can use BLS for now for the prototype and then figure out what we do with this. Uh, >> but then uh that's what Camil was saying yesterday. Can we just do like a signature by you know we just trust it to be true? we're interested is the the network conditions. We're not really interested in uh >> yeah it it so we can reserve like if we don't want to do any signing I think we should do the signing >> because it gives us a better understanding of the performance um of of of a particular scheme or we could just simulate that we like spend some

time signing or whatever but also this becomes important for the sizing of the chunks right because it's not the same to have like 300 chunks or to have 500 chunks so having understanding like the overhead. It could be a placeholder um is is important. >> Cool. Um right. So for the scope just kind of like to touch on on scope this protocol. >> Yeah. >> When we split into chunks do we have a thing that we need to uh also build some Merkel tree Merkel rule then?

>> No. No. So this would be >> I no I I I've left this out of the equation. So there's no proofs of of any kind that we're emitting for the chunks because I'm assuming that we're authenticating them through a signature and that's it. And we know who the who the aggregator is ahead of time. So we just um validate we just verify chunk incoming chunks by signature and that's it. >> Right. Um yeah. Yeah. So just to like continue uh we would be using this protocol for the two use cases that we mentioned. So that would be for global aggregation broadcast and local aggregation broadcasts. And these have different producers, right? So we have local aggregator producers and global

aggregator producers which right now are 32 each. So you know pretty much the same from a produc from a producer standpoint. But for consumers, this is something I I want to check with you guys if this makes sense. So the critical path really is for global aggregators to receive the local aggregates. This is really the the critical part, >> right? >> Cool. >> Why? >> Because local local aggregators are interested in like local nodes in the subnet would be interested in knowing if their signature has been included, but they can really not do anything about it. local validators, >> local validators, sorry.

>> Yeah. So, submit validators would be interested in knowing if the signature has been included as soon as they can, but like they can't really they can't really do anything about it if they if it hasn't. >> I mean, do we care if they interested or not? Eventually, they we see global snark. Yeah. Which proves signatures for reward, right? agreeing reward that is based on the inclusion of there. By the way, it's a question. Do we Yeah, once we have a global snark, >> yeah, >> does it have any maybe it has ZK properties which we actually don't want to have. So that we cannot map. Oh, does it just proves that the keys uh that

were signed or the keys uh they actually belong to the proper validators? Uh I mean the keys that we aggregated, they actually belong to the proper validators. But we don't have this property. So we need to ask cryptobase um >> we need to make sure like that the adistation has been included into a block whether that's proven with decation but aggregators aggregation >> yeah but but you were questioning like the the local validator >> I I think like the local validators are still okay so let's maybe move on very briefly to the global and then we can switch back to the local so in the global aggregator I think it's this one is very clear there's going to be X well

global aggregate is producers and uh the entire network is interested in consuming the aggregate >> yes >> which includes the validators that were in the subnet and this is how they know if their signature was ultimately included or not because this is what counts >> right the output of this >> so really the local yeah >> just curious at which point are we making sure that if those local aggregators If two global aggregators connect the same at a station that those are not counted twice >> then that means a global aggregator would have to detect that >> uh Google whenever they create uh aggregation they also sign maybe sub

subnet ID add to the me well they add to the message the subnet ID and the global aggregators they might have a rule which I was proposing that they only observe one local aggregation per subnet and they can check it by this subnet ID and then they ignore once they received the first aggre local aggregation from particular subnet then they ignore all other aggregations from this subnet that was my proposal but yeah >> okay just making sure this is >> so can you can you repeat your question >> no I mean as I as a validator would >> val send two validation to two local aggregators,

>> right? >> That's that's >> you send an attest you add at a test station to two aggregators. >> Yeah. Well, you actually so this is something to figure out. Do we does the node send or does the node publish >> because the way it works today in the protocol is there is a topic and aggregators are listening to that topic. >> Yeah. >> You don't know who they are but they're listening to that topic. Everybody else receives the aggregation uh receives every single attestation. M >> um and some of those consumers are >> okay >> aggregators. So then so that is the push that is the publish like pops up model.

>> There is another model here which is you alleviate the entire network from having to or the subnet from having to serve as a forwarder >> and you just if you know who the aggregators are you just send to them. It is more efficient. Um but it requires you knowing ahead of time who the aggregator is the IP addresses and the like the endpoint of the aggregators and this is the the deanonymization problem again >> and also this demization from the perspective of validators because now we see >> who's from aggregators see who exactly sent you know the signatures >> but you could you could also do some something like tour like where you could just have three three hops >> add more hops

>> but that too many of Yeah, >> it but the problem is that it adds latency. >> Sure. >> But like what I mean is between the full gossip server you don't know what like I don't know what the average number of hops is but maybe three hops is actually much better. >> Yeah. >> And if we have that >> so one interesting idea there is to to create a mini mix net. >> Yeah. >> Inside each subnet >> Mhm. that is um gated is permissioned >> right >> to by presenting a proof of validator >> right >> so you only allow validators to participate in that mixnet >> mhm >> you don't know who the validators are ahead of time >> you just know they're participating

uh you would need some association to know like if they are supposed to be participating in the subnet which is where it becomes difficult because now you deanonymize right >> um but like it I like that that solution space because it's like gossip already has you know hops >> so why not use those hops to to anonymize by having some onion routing for example right that's one possibility >> um onion routing is probably not the idea like it's it's inviable because it requires you knowing who you're selling to again >> um but yeah some form of like what like maybe oblivious HTTP is a good example here where they just have two hops

um one that anonymizes the sender, one that decrypts the content essentially and then that would be delivered to a mesh of some kind. Um >> yeah, that's that's maybe like another but all of these solutions end up adding latency, right? And that is the trade-off, the big trade. >> Yeah. How do you how do you avoid the liveness failure on the aggregator side if you have a direct like you if you have an expected relationship between the local >> local and the uh >> well you have redundancy right because you have 32 aggregators so if one of them fails right you would

be having redundancy there is that what you're asking or >> yeah okay so okay so the aggregator or have to have to maintain a connection or or to like to multip Yeah. >> All validators in subnet. >> Yes. They would validators and subnet would connect to them and like some in some way prove that they are a validator >> inside that subnet that that is associated with that subnet. Of course, there's like way too many like in some form like you know some open like a lot of unknowns here of how that would happen. Um but I do think that like even if you do the direct communication over

time you can deanonymize validators. Um so you would need some form of like I think if you want to guarantee privacy and you want to guarantee um yeah you want to avoid deanmization then you would need to have like some form of oblivious routing there. I was thinking about witness uh uh and yeah we have like local aggregators who everyone's connected to so they send their signatures to them but if we also have like for example sub which we have today uh and in addition to these aggregators we also have which I call them private validators they do not

actually private aggregators they do not say they are aggregators they behave just like validator because but in fact uh they are also aggregating uh nothing of them they purely alistic and uh but that's more like a backup just in case uh well if these aggregators for example we decide that we need to expose their address and for some reason they be become target for the DOS attack well then our network kind of become slower But since we have this private validators that only participate in gossip out then they still can uh well they behave as a backup. So they

create aggregation that could be just one one option >> introduce this private aggregator but >> they will not be incentivized. Well I think publicators also will not be uh and yeah we have we cannot guarantee that they actually exist. So um yeah, >> so this is more or less how the L1 works today where you have the aggregator duty and they just are subscribed to the subnet because they're assigned by the protocol and you just don't know who they like which node it is >> um ahead of time. I think you might be saying that you don't even have a protocol assignment there, right? You just have >> altruistic. Yeah, I mean they to observe uh on signatures anyway.

>> So maybe they can also if they are I don't know they decide for the reason to also be aggregator. >> Yeah, >> they will be a bit slower because they do not really use the protocol that we trying to create for aggregators but >> they exist more like a backup just in case we have some people attack. >> Yeah. one uh so I think we should like stay focused on this protocol because I think the the like this this is another rapid I think this is the complex piece right of like we haven't cracked it yet of how we're going to be doing the individual uh test station broadcast um and probably like that should be a topic for we should continue working on but uh in parallel to this uh just one thing I

would say is that that might be a good reason to have validators >> receive also subscribe to their local aggregates because over time they can score um their inclusion probability as aggregators have selected in the subnet right so say for example I consistently see over time that a particular aggregator fails to include my my attestation I know that like I will send my attestation anyway but it should be the last validator I send it to like I should depp prioritize that validator

because it's consistently over time failed to include my attestation for example so that it creates a feedback loop which I think is interesting that we might want to leverage um yeah that's but you can maybe also get that information from the global aggregate maybe not I don't know because yeah depends depends on how you send it. >> It's not really a lot of information. It's just bit fields. >> Yeah. Who created? >> Yeah.

>> Cool. So, any other ideas, thoughts or directions that might be before we move on like before we go back to eraser coding? >> Okay, cool. Uh so >> just as a general comment like it seems like the trade-off like you're walking here is like either like completely like untrusted relationship among the complete validator set like independent from validator local aggregator or global aggregator and sort of peerto-peer efficiency and sort of all the tricks optimizations like that.

>> So that's sort of the >> that's always a dichotomy right. Yeah. Yeah. I mean if you look at for example Salana like they rely on fully permissioned networks and that's why they can optimize they can optim like they can make certain decisions at the networking level like for example in turbine they have a um every chunk has a deterministic propagation tree and every node in that tree knows who they need to send to next and how many peers and like exactly the peers uh that they need to send to next. So of course you can do you can play these tricks I guess just as a com I mean we're optimizing this whole thing for like maximum uncorrelation.

>> So the pico pie whatever logic this is sort of the the ultimate meme for uncorrelation. So it's like >> like I don't I don't know what it has to mean eventually but just from like if you look at it from the above it's like okay if we >> we then will have like direct connection like super optimized >> um direct links then like why why do we even like worry about like this pog yeah yeah yeah the tradeoff is really you want the redundancy part of things and the censorship resistance and you want the um I think like in general the like the resilience of the network

>> right >> when you don't rely on a single endpoint right or a single peer to like provide service a particular duty but you can decide to like you can get the same property without using pubsub necessarily right >> okay yeah >> you you can get exact so that that would be like the the thing to say Okay, how can we uh get the best of both worlds essentially? >> Okay. >> Um and there might like you know I think like ultimately if we keep pushing the boundary for throughput and we keep like staying you know enforcing these values we're going to need network breakthroughs like this is not going to it's not going to you know like the traditional protocols are not going to

be enough. >> Uh I'm sorry to report it's already no hour. Sorry I thought Oh, there's another session or >> No, no, it's just that we have the group brief and my computer is still in >> European time. >> Yeah. Yeah. Yeah. >> European. >> Okay, cool. Um, so just to >> like go into arasia coding itself. >> Yeah. >> Okay. So the idea is keep it simple for now. Uh just adopt read solomon erasia coding. Um and the idea is to reuse the standard functions that we created for blob encoding. Uh so there's no reason to reinvent the wheel here. There is a

library. It is proven. Uh it is audited and it has binding in all languages. I don't know if in Zigg but definitely in Russ. >> Oh so yeah it can bind straight. Yeah. Yeah, I already have a Z. >> Nice. >> We already shipped. >> Nice. Nice. Yeah, like predicted the need. Yeah. Nice. >> Uh I have a couple questions for this. Um the first one is um I'm not sure exactly how the uh because I don't remember. I've seen the the presentation about like this stuff uh maybe five years, three years ago.

>> Yeah. >> Uh but there was a two-dimensional side. >> Yeah. Are we going to do two dimensional things? >> No, it's just one D here. >> Okay, so one dimension. That's cool. Um except that means if we go in one dimension, but yeah, we should have enough. We just need to make sure we have enough elliptic curve points to to do that thing. >> Yeah. Right. So this is basically what we adapt now. Uh so these are like the the thing is that the the library has some statics there has some constants that are pretty uh like specific to the blob scen uh use case.

>> Yeah. which do not like match what we need. So we would need to fork the library and override those constants uh or like we can provide them as built parameters or something like that and like just make change the library. It does have it does like some of these actually specifically talk about blob. So like the name contains the word blob and cell which are like very domain specific. Uh but for now I think for the prototype just working it is good enough. >> Wait, wait, wait. Because what do you need? Is it small? This is smaller, right?

>> Uh, this is larger. Like we're Yeah, cuz the in blobs we have 128 kilob blobs. >> Here we have double >> and we're actually generating and sizing the cells to stay below MTU. So we don't we no longer can use the same parameters because the the library uh splits cells into two kilobytes. >> Yeah. So the 38 number is came from the uh same MPU. >> Exactly. Yeah. Yeah. Yeah. Yeah. So this is coming from basically so we're using um like 32 bytes is the scala representation right and then like you multiply by 38 and you stay below MTU or like bit way below a little bit even

below MTU um to leave space for the P to leave space for the framing >> if we if it's larger >> then do we have enough u like um uh s your powers of the secret to cover that thing. um if it's large sorry can you say >> yeah the I mean the you know I forgot all those names but the idea of KCG is it's a polomial you come into and >> but we're not using KCG we're not using the KCG part of this like it's it's a misnomer this this library is called KCG but it also does the eraser coding >> so we're not actually okay we do not

generate or verify KCG subs >> because we don't like we don't yeah we don't do a commitment because this authent the because this is just for networking it's not so a commitment is necessary because we commit the blobs to a block right in this case this is just ephemeral >> like in transit um like the eraser coding for to move the data around >> right >> and for that we just need authentic like source authentication >> okay >> so we don't need to do any sort of >> no no no >> exactly Yeah, we are just coding.

>> It's just a ratio coding. It's a misnomer. The library is called KCG, but if you look at it and actually the the methods, the functions themselves >> are recover and prove or like they already involve proving, but then if you pass null values to the specific field, >> it will disable it will not do any proving. It's kind like well I mean this makes sense because like when you do when you build a library a C library that is going to be exposed through FFI you want to minimize the over the FFI overhead. So you just like really overfitit the library to how it's going to be used which makes sense. Um but for us uh we don't need a big portion of this. So eventually we'll have to like

if we go if we decide to commit to this direction uh we will have to build a like fork this library and actually make it smaller and like adapt the parameters. >> I think that's the first action for dependent one. So we need to >> Yeah. simplified library. >> Yeah. Yeah. And this could be done very quickly. So yeah. Yeah. Because it's just overriding something new just simplifying. >> Exactly. Yeah. >> Cool. So ju just jumping back to the fact we're not using actual KZG stuff. We don't need to worry about quantum resistance, right? The sol would work, right? Because otherwise fry would not be posed.

>> Yeah. >> Yeah. I don't I think it's orthogonal. Um because it's also like in just in transit. This is the chunks are not going to be persisted in the protocol in any way, >> right? >> Yeah. Um, so one thing is like something that I hated was that this is not uh binary aligned. Uh, but I don't think it's going to cause any problems because of the way that like it's not going to penalize performance because I don't think that like this would affect cache CPU caches or memory or anything like that. But we should benchmark against different values anyway in case we prefer like we see there's a significant gap there then we might prefer to use

other parameters and like lose some edge on the networking side of things if it's just a >> cool um so I this is like where I kind of stopped a little bit uh but like maybe we can think about this together and I can continue working on the on the framing. thing. So what I try to capture here is what would a packet look like on the network, right? So I think we will be having a chunk slot number. We would be having a chunk index.

We'll be having the chunk data and then a frame signature. So a signature for over these three fields >> that is produced by the aggregator. So sorry this is a bit messy. It was just like halfway through. Um so do we think that we need any other value here? >> Quick question.

>> Yes. >> What are we coding? What is the object again? >> The aggre the aggregator signature. Uh sorry the the aggregate signature. Yeah, in future it could be also the block itself. >> That's smart. >> That could be true. >> It could. Yeah. >> So, >> yeah, but that's like something that erasia coded to block propagation is something that we're doing already in the L1. It's going to like we're already >> I mean block propagation. >> You mean for block propagation? We're already using the >> No, no, we're looking into it. >> Oh, looking. Yes.

>> Yeah. So, I don't think that lean would need to look into it specifically as well. We'll be duplicating. Yeah framing is it important to know which validator in the protocol did the aggregation or if not then we don't need to validate next >> right um so depends on whether you can recover from the signature you can recover their identity from the signature you might just be able to um to recover it because like what's going to happen here as well and this is something that we should probably talk about that is that there's going to be competing aggre aggregates in the network.

>> Right? >> So at a global level, it's possible that you'll be having different aggregators produce different snarks because they collected different numbers of quantities of uh local aggregates. We use this reging on uh only for global or for local aggregators as well. >> We use it for both. >> We want to use it for both. >> Yes. Now the something that we could do is each chunk could also carry a commitment to the whole payload and therefore if you have matching

um matching hashes or commitments. So this is maybe where the KCG commitment would would become like interesting or maybe just like a hash is is good enough because you don't need to like prove every chunk maybe I don't know. Uh what you can do is like create many lanes as you receive these chunks and you have they are coming from up to 32 aggregators. Imagine that like 31 of those aggregators arrive to the same aggregation result and just only one of them arrived to a different one because they missed maybe a local a local snark.

Right? Then you could actually repurp you could re you could combine and reuse the chunk data inside those like no matter which of those 31 aggregators you're receiving it from because they actually arrive to the same result. >> This is impossible they will arrive to the same result. >> Okay. >> Because what we have like subnets >> for example thousand nodes in the subnet let's say in the future. >> Yeah. And uh each aggregator they will not wake they will not wait like 100% of signatures to be received before they start aggregating because if this is the case then we have one slow or malicious validator who is not producing the signature and then yeah we stuck and we

never come up with aggregation. So we should create some kind of threshold uh like for example 90% of signatures that we want that each aggregator is waiting for before it starts aggregating >> and each aggregator more likely than not we receive different different set of 90% signatures. Yeah, before so right now I just to like back up I was talking about global aggregates. >> Okay. Okay. >> Yeah. Where I think it's way different because you do have more likelihood that you'll be having overlap there because a the data that you need to receive is only 120 256 kilobytes per subnet and you'll be having a limited number of subnets. Right? So if you have 64 like

what did I define 64 128 what did I go with 64 right so you have 64 * 256 * 2 well 256 because this is what you need uh you just need to receive uh 16 megabytes of um of aggregates right if you're assuming that you're going to be receiving all aggregates from all subnets uh before you actually go and produce the global aggregate. Um, so I think that's like the the the domain is way smaller than with local sub local aggregates and local and subnets, >> right? So I do think there's a high

likelihood that we can parallelize and like actually reuse the the the aggregates between global aggregators way less than local aggregators because that is 3 kilobytes over like 500. I use is for example uh some global aggregator let's say we have for simplicity just eight subnets and some global aggregator for example already aggregated subnets from one to four >> and another aggregator aggregated from five to eight >> and then you mean this is what you mean by >> oh no no so I I would assume that global aggregators

>> yes >> are attempting to aggregate all of them >> yes >> right so if I I have eight subnets. >> Yes. >> Then and I've received all eight. >> Yes. >> And I have 32 global aggregators and only one of them has not received um like say subnet 7, >> right? >> Then which is the last one, right? Um starting from zero zero indexed. Um so they have seven subnets and 31 aggregators are aggregating um eight subnets. Mhm.

>> The chunks of that those aggregators are going to be publishing should be reusable >> because they're aggregating the same data. They're arriving to the same result right? >> Okay. Yeah. They will be reusable. >> Yeah. >> Because the problem is that with the schema, you don't know ahead of time, right? So maybe we should add a commitment to the aggregate and maybe not even to every chunk maybe to like even or odd chunks to amortize.

So you just really need one chunk from a given validator from a given aggregator. if you can recover their identity from the signature to map which hash like which aggregate they're actually uh publishing. So you don't need this um or you can intersperse it in the in the data itself like you don't need every chunk to carry the commitment because using probability you're going to receive like x number of chunks like say you receive 10% >> of chunks from in time of every aggregator you can just like inter interperse it >> right uh you mean interpolate

>> uh >> well yeah >> I I mean okay you can you can recover. Okay. Um >> yeah. Okay. This way you can not really interpolate like we could do a simple simple thing here which is you just publish every module like you put the commitment every modular chunks. >> Uh but yeah but that gives you like what do you use the remaining payload for because you need a >> you need a fixed schema. You don't like variable schema suck.

Yeah, maybe maybe we just commit it on every every chunk or we do the interpolation, we just like put it in the chunk data somehow like instead of graphically, >> right? Um, >> yeah, but how do you hide something in the >> open? No, I don't I don't know if uh Um >> just one clarifying question. When you say payload, you mean CL block?

>> No, we're talking about aggregate aggregates here. We're talking about signature aggregates, >> right? That's sort of when we talk about the Yeah. Okay. So, so pay off for you is synonymous synonym synonym for uh for >> it's yeah I mean in networking terms is like the data that you're going to carry which in this case is the aggregate the the signature aggregate which is what this protocol is for it's just for aggregates >> okay right because we have the term payload like we use it

>> execution payload yeah right I know Yeah. Yeah. >> So, payload obviously only meant to refer to the >> Yeah. >> So, so the chunk is the chunk of the aggregate. >> Yeah. Yeah. >> And then I do have one question as well. >> Yeah. >> So, you mentioned that this >> you mentioned about the commitment, right? And that's for >> the hot path between local aggregators to global aggregator. >> But does that mean that if if you want to have that consistent schema, does that mean that local aggregators also have to have the also have to commit.

>> Yeah, I mean do the global aggregators I think so. Yeah, they would have to commit. It would be useful for them to commit. I think so because that way the global aggregators can use the same technique right of leaning uh and reusing maybe not like I think this is something we'll need to test and it will depend on parameters as Camille was saying is like it's way more likely if you have like 500 validators in a subnet and just missing one signature just having like one one signature diff is going produce a different aggregate then there's very low potential for reuse

across the aggregates that all the aggregators within a subnet are going to produce. So maybe we omit it in that case because we already know ahead of time it's like the the potential for reuse is very low. >> So I suppose like we follow the same schema but then we don't have to do that. >> Yeah. Yeah. We just Yeah. or just make it an optional field or something. But then the problem there might be like two different flavors of this protocol because the parameters will be different, right? Because if you want to maximize and pack as much data as you can inside the MTU and you know that you're always going to not have this field, then you can

>> you can pack more data inside and and reduce the number of chunks. So it would probably be two parameterizations of the same protocol. Yeah.