EIP-4444 Bound Historical Data Part 2 (Breakout #18)

Transcript

uh all right guys this is the second breakout room for eip4s i have some things on the agenda here that we can discuss i'll post the link in the chat here but i think i don't know we only have about 30 minutes i have to go to a call um at the top of the hour people are free to stay on afterwards but this is like roughly the things that i i've been thinking about um i don't know if other people have like other specific things to uh to talk about sorry let me post the link again for them because they're asking for the passcode okay

does anybody have something that they want to discuss before we talk about like this roadmap the rough idea and then talk about these answer these questions i feel like we should start all these meetings with the discussion about how to pronounce eip144444 i'm pretty sad because vitalik pronounced it on the um whatever podcast he was on like last week eip4444 and i feel like that's just really hurting the branding yeah i mean it's just important that we're all on the same page with this

because if we can't even be on the same page how to pronounce it there's literally no chance that we're gonna be able to ship this to vayne head wow that that is an incredible mustache yeah oh my god holy welcome alex's mustache the call hello everyone hello alex uh i just i was just saying that um i posted this agenda i basically have this road map which we've kind of like roughly talked about in the past and some questions i don't know if there are other specific things that people haven't added that they want to talk about but if not we can just talk about this road map and then answer those questions where is it oh so let me post the link again it's on

the it's not that's not the link it's on the github agenda there's a good of agenda on pm issues okay i'll go find it i sent it on zoom thanks okay so in my mind i think this is like roughly what what proto had talked about in one of the first meetings that we had is you know we need to define this ssd object that's going to contain the rlp block bodies we need to then implement some sort of

mechanism in the execution clients to verify those sse objects and also create them and then actually import and execute them so that you can get to the uh the latest state with those the historical um data and then like farther along is we need to talk about how we're going to distribute the data over bittorrent like are we going to have you know a client that kind of you know uses some sort of bittorrent library or is this going to be more on the user's end to just to take that file and seat it themselves or connect some sort of magnet link and then download and see it themselves

and then the final thing is like to actually implement four fours which i think we're you know pretty far away from doing right now since it's not something that i think is going to be accepted into a client for at least six months so if we do implement it now we'll just have to keep rebasing against changes for clients just so that's kind of roughly what i see is like a road map i don't know um thoughts on that so the question i have is how confident are you that uh this is like what we need to do like have we validated that like if we do this thing then everyone will accept what we're trying to do and it will all be good and happy

i'm sure it would be nice if we knew that what we were building was going to be used before they built it well because like there was like more i think philosophical push back from like example the guest team so the questions like uh you know is there stuff we can do because we can go and build all this like that's not really a problem like for sure everyone here is very capable to do that the question is like on this i think the general pushback has been yeah the general pushback has been that bittorrent is not as a distributed store of information as they would like i think which i think the more i i think i mean that's mario's position but i think we can argue him out of that if he can just sit down with me for an hour

um i think the bigger pushback from the guest theme i'm confident uh i think the bigger pushback from the guest team is that they don't want the impression i got is they don't want it to be something that is not in the clients like they want the guest client if i understand correctly someone else may really crack them here they want the guest client to be able to do it which means if we do do bitcoin they want the torrent you know capabilities within the guth client and that's where things get hard it's it's much easier yeah so it's much easier if we can you know download it a via bittorrent and then just import it into the clients um but i think they wanted you know a one monolithic thing and

maurice had some milestone but if we hit that milestone then he thought it'd be okay to drop it i forgot what it was if i didn't remember it i don't know we're getting a little bit ahead of ourselves here with how the solution should look like we should look at how we want to deploy and how we get there and i believe we like even in this issue where we define this difference between sse and rlp in the end release these sub components that we should just give people ownership offer and not

try and prioritize over just the bigger thing so i think henry is planning to spend some time specifically on the first two steps i think you've looked into ssd a little bit and i i think i understand was at some point you would come up with say this is what i think the actual definition of the ssd object should be and we can talk about and make sure that that's like what we all are sort of assuming like i think we all sort of think that this is going to be some sort of ssd list that has you know rlp data on the leaves there's like some small questions about should we serialize both the block and the uncle

the block body the uncles and the transactions together or should they be separate and then after that after we like agree on that i think henry is also going to implement this into an execution client to do the verification the import and the export so the rest of us are really not doing anything yeah that that's kind of my idea and by the way for those i know i've met some of you on a recent call but for those i haven't i'm kind of new to this whole thing i'm relatively recent convert and working you know looking to get into working on ethereum i've done a bunch of database stuff previous to that um and so having worked with tim he suggested i

could help out on implementing so some of this as a nice way into um into ethereum development um as a background but yeah to um to what you were saying that's my understanding i think uh and that's what i intended to get started with the the two first items it sounds like unless there's some i mean it sounds like we can get going with those two without having to settle the broader question of will this of the bittorrent and of the distribution mechanism and whether or not that's integrated into clients and and all of that for sure that's something that we should do in parallel yeah uh and um yeah i mean i have a few implementation

related open questions et cetera i i don't know if we want to get into that now or later but um kind of time yeah i mean let's i'm happy to just start talking about implementation specific questions on step one okay yeah so um well you just the the outline that you had is is more or less what i was thinking and i think what we discussed last time um i guess well one of the questions i had so i started looking at this more seriously um last week and this week a little bit getting into ssc um and uh one question which is which i need to figure out pretty soon uh that's kind of

related to number two is which execution client would be the best candidate for this and and the reason i'm interested in that is there's a bunch of ssc implementations in in different languages and i'm kind of looked a little bit at the fast ssc and go and i know there's a python one um i don't know if there's a kind of natural obvious good candidate for the execution client to use the proof of concept or not but that would that would be helpful yeah usually guess is the preferred one okay so yes death is the preferred one i would say because it is dominant but we're really trying to make it not the dominant client like

the way ethereum works we don't want a dominant client we specifically want people to use other things and so people use guest because guess is what they know and so if you know nothing i would weakly encourage doing something else like any other client just to kind of just incentivize more growth out into other places um again if if go is your favorite language though then gath is a good choice if c is your favorite language another mind if um you really love java then maybe you've worked the best basic team probably go but in that case so would aragon be a good choice or that that's probably not the easiest one um it's harder to work with them sometimes to get things upstreamed okay that's my

experience i mean not that it's necessarily easy for guesth but slightly harder with aragon okay well get was what i was thinking is the default choice so it sounds like that kind of confirms but i mean given that yeah you know go i'm i'm way more productive right now and go than uh if you've got a language reference i'm just struggling with reference yeah i think that's really yeah it's a good choice then just a chat i shared a list of libraries recommendations so just browse that and you'll find something to work with yeah that's it i of course i've worked differently

before i have a few implementations but i'm not necessarily saying you should use ssc at all like what are we really trying to achieve do we want some kind of proof to be accessible or we don't want some kind of serialization property and these are things are not specified right now so maybe we should do that first yeah this is one of the main questions that i had and i think the strongest um you know proponent for sse was actually piper when we originally spoke and it you know i think like for me the the two main reasons to go with ssd is number one ssd

is sort of the next generation serialization format and i think that the more new things that we do in ssd the easier it will be to transition over time everything to ssd even if that takes a long time so i think that is in itself a pretty strong reason to go that direction and the other reason is is that if we are going to basically be creating these checkpoints that are creating um you know large segments of blocks that we're committing to then we can now prove something about a historical block and we don't really have that functionality in ethereum right now this is like something that will be out of the protocol but we will have a way where similar to

having like a weak subjectivity checkpoint where we say this is something i trust i can now say i trust this route and now i can prove something about a historical block so that kind of gives us more functionality the thing is is that we're not there's not a lot of people who are requesting that functionality um there are people who want to have it in the protocol in a smart contract context but i don't think that this will be like easily like maybe we can have a smart contract where we post these routes to like they are you know checkpoints and you can like sort of prove them but they're not something that you're you know ascertaining from the protocol itself so there's like a weaker a weaker guarantee there so to me those are the two reasons to go with ssc i don't know if there are other reasons

that people have or if there are reasons that people have to not go with soc i think the reason you gave are the exact reasons i was going to give just to play devil's advocate a little bit reason for going with rlp is just because um that's what everything today uses and so integration it's going to be a little faster and easier like if you don't you know convert the blocks all to sse you just have an rlp we already have them all in rlp so when you import them into guests it's just a direct hey guess here's the block as you understand it you don't have to convert between the two um fairly minor i think in the grand scheme of things and i agree with matt overall just i think that's dark i guess there is like this one you know specific point

that the leaves of this sse object are still rlp encoded that was one decision that we had made to to avoid exactly what you're saying where now you have to define the ssd spec for every block all historical types of blocks and the clients have to support um you know the manipulation of the sse object into the rlp object whereas if we just commit to the ro to the ssc list we do get a lot of the functionality of being able to prove things about historical blocks we don't have the granularity of being able to prove a specific thing in a block we have to prove the whole block but we can't prove down into like the nonce but maybe you could move down like the header as an opaque rlp yeah i can't

like easily create a proof against some specific value of the block i can create a proof to the block but i feel like this was a good trade-off and that we get to use ssd for most of the things um but we also don't have to rewrite a lot of stuff in clients to let them import so this is i think perhaps the reason i don't see the future is i have always imagined this to be something that sits external to the clients and so you would have some little thin thing that knows how to download a torrent and knows how to do some data manipulation and you would use that to do the bulk file download and then you would just have a button in there that says export to another mind or export to

guess or export to whatever and so and the reason for that is because specifically so we can avoid having to convince clients to let us integrate into them we can just build this entirely isolated from the clients and just say hey you know we're done you know you can now sync guess by running this little script or whatever this little tool hit a button and now your your brand new guest client is synced up to january first 2021 or 2022 or whatever um just my understanding one question that to do if we were doing what you described to do the you know press the button and import into geth you'd need some new support for that on get site right well

most clients support this functionality this like import sub command that lets you import a like directory of rlp blocks oh okay no we didn't know that um we can already probably do it like micah is saying you know this i guess another question like exactly how is this integration going to go i would say one client does have to have four force integrated because we want to be able to export the historical data into these structures and we want to be able to verify i think verification is important because whenever you know some entity says this is the you know this is the root of this section of blocks like we should have people who can you know very trivially retrieve them and verify them yeah

and you wouldn't count like just running gas after running this import as verification i guess like when you're running that import into guess i believe it verifies in that step doesn't it um yes but the time it takes to execute is a lot greater than the time it would take to just verify the you know historical blocks like if you are running a full client you know if you're interested i'm just saying that when you do that import it um tries to fully verify it whereas you want like a light verification where you're just like hey do these headers make sense well i'm just imagining that you know we create this new checkpoint this period like we

haven't expired these blocks yet and so we still have them locally and this is the proposed checkpoint of the those historical blocks so i should be able to download that and then compare them to what i have locally and so yes you could execute them and make sure that you're at the you know the canonical head of the chain um but that's a lot more time consuming than just verifying that you are aware of those blocks we should try and move to [Music] action points or things to to do after call before we run out of time um

yeah henry i guess like you are the you're the person who's really going to be driving a lot of this initial implementation so are there things that we can do or questions that you need answered so that you can move forward like on our front we'll continue we'll talk with the guest teams where we'll continue like working on some of the more like philosophical questions about you know what is okay to distribute how to distribute data um but i want to make sure that you're unblocked to move forward yeah thanks um i i think i am that this has been helpful and and i think i mean given that the immediate first step is the is writing out or coming up with the first roman um layout i i can get started on that right away and then i

guess as it i'll update as it gets as it moves forward and and there's something to get feedback on okay great we've got eight minutes left to other things that we should talk about i see metallics here so go ahead alex i was just gonna say like uh in parallel with implementation work we can like you know start drafting some kind of vip-ish thing um because like the more concretely we can write down what we're trying to do the more we can say hey uh get team hey piper like you know how does this sound and the more specific you know the more productive we can narrow down to a specific design so

so that's something that we should also be doing that actually describes the distribution distribution of the data and like what the format of that data will be in additions like what we have with four fours those should be separate things or yeah i guess yeah maybe like it's not quite an eip but just like writing down like our thinking so far and like documenting that process for other people different because like it's like i mean i'm pretty sure people said like like four fours i was written like would never ship right like i feel like maybe george coleman or someone said that like i just feel like there's like we agree with the separation as proposed before where we have

execution or sorry vft uh formats we have the import export and we have the distribution yeah that makes sense to me okay let's split that into eip stuff cool so i would say you know how whatever format you want to write this up in henry as you like work through this then we can like work together and put that into something that's like an eip um so that'll be the first step for the actual uh data format and then we'll you know the distribution format is something we'll continue working on discussing yeah sounds good to

just like curiosity is anyone here right now opposed to torrents or think there's that's a bad choice or just just people in it's like probably a great way to start like maybe we like get six months down the road and various is like we can never do torrents and then you know we have to tweak it but if we start now thinking it thinking of it as this like modular bit right to this design then we'll swap it out when we need to yeah i guess one thing that was not added on the road map that you know is another thing that's that's farther out but some sort of like bridge client that allows you to like maintain the archive functionality

that's you know i think this is something that's you know at least 18 months 24 months away because i don't think that geth is looking to remove any any consensus code you know in the near future but it is something in the law the medium to longer term that they will want to do i do think that it's pretty important that we like maintain the ability to generate all historical artifacts so that's something to keep considering i think like you know it's not really a theoretical problem that needs a solution like it seems relatively clear like we just run certain client versions and have some sort of you know multiplexer that is able to route request to

the version of the client that understands the information it's just going to be an engineering challenge all right five minutes is that it i can easily fill the time i was trying to respect the goal of actually getting out of here on time well you've got four minutes if there's something else you wanted to discuss on four fours i would love to go down the rabbit hole of uh ssc all the way down like i i feel pretty strongly that would be a big boon

long term you think that we should ssc the actual block without the value blocks transactions uncles everything like the we just get so much power from being able to prove everything arbitrarily right but is it strange to just leave that as a detail for now and at least define the d3 of all the blocks and then whatever the leaf may look like it can change later yeah yeah i'm okay with moving forward just like using rlp as a placeholder for the blocks for now um i just you know in my experience what once you have a placeholder in it gains strength the longer the placeholder sits there

and you know if you let a placeholder oh it's just a placeholder and then you know four years later yeah that placeholder is permanent whatever move it a i do is that there is a natural showing point for moving everything over to z even if we start with rlp because like eventually we are going to turn even the the beacon block that goes into the or the execution box that goes into the beacon block into an sbc format right okay so how about we switch it throughout we started as a c for the block and did a referred to rlp if it doesn't work for clients yeah i'm okay with that just flipping

slides here the uh advantage to of course to rlp first is that there's less engineering work like we already have decoders and encoders for rlp box you just copy and paste microsoft execution photos as well uh i mean don't we you should all leave and let like argue with yourself this meeting would never end uh so after the merge i mean we're going to be able to prove things about historical execution and payloads like with the granularity of ssc anyways so is it that i give a boom i mean there are the historical like accumulators in the beacon i want

to say if i want to create a witness of this transaction occurred in this block and these are the transactions or like in this state entry i think the transactions are still rlp encoded though but you could prove a certain transaction you say i could prove a transaction hash is this because the with the merge are they switching the block header to ssc yeah well at least there's a representation of it and then transactions are still opaque and the ideas and another fork or two will sort of switch fully right right like i think uh with the merge what happens is that the transaction

this the list of transactions becomes uh ssd but each transaction is still opaque and then the receipts are still um okay and then obviously the state is still opaque um and uh i guess we wants to eventually um moving the receipts over that's disease should not be super hard and then eventually the one move transaction internal structure over receipts to a format that can do proofs over those receipts would be really really useful for our protocols yes i was actually having some discussions about crossway or two bridging and one of the big hurdles that actually came up was uh basically the

fact that that if he wants to prove something in what one way or two to another way or two then like first of all there's stuff that the layer two is haven't done that they really should do like make the api l1 block cache accessible that's an op code but even going beyond that like it requires these proofs going into a receipt um that contains the l2 state route and then you have to do the same thing inside of the l2 and the thing with roll-ups in particular is that with roll-ups um execution becomes much cheaper but data does not become cheaper at all and so merkel proofs could easily become like the biggest cost factor of cross if a secure crossroad stuff becomes the norm and so like might it even make sense to

skip ssd and go directly into putting receipts into like either a verbal tree or a ksig commitment i don't know i can think of two more things that hurt layer twos with the current receipts design but we should probably focus on the force now [Music] map where hopefully at the end of the 10 years we have everything including the the execution and beacon state and receipts as part of the same structure

but i guess we can leave that until later yes so i think i i i i really like the idea of using this opportunity this is an opportunity to get data structures into a for like a better word more modern place so like we've got vertical trees instead of merkel trees we've got ssc instead of rlp there's lots of things that if we were to start over with ethereum today we probably would not do the way they are and so this this is a kind of an interesting opportunity to take on that and say you know hey this is the new standard way we do blocks in ethereum we still have the legacy way that all the clients use but you know

going forward if you're building something new use this new new data format of course as i was arguing earlier that's you know the more we do write the more engineering we have to do to get it right like it's much easier just to do the status quo than it is to do something new um so i'm conflicted on that we dive into the future of receipts and everything we can improve about proofs do we want to end the recording here for forest purposes or just add it as an extra are from us fine with recording free bonus to anyone who's working on this

all right so above those receipts the this is like the information about the execution output of the layer one this is really different from all the input information in that we rely solely on the receipt root of the layer one to interpret the execution as a layer two so this really distinction it's not just about data commitments but about knowing about execution and the problem here is the receipts can be huge they can be larger than you can load into the evm making optimistic roll-ups for example really difficult

and they are mercurized in a way that's also quite ugly with the mercury doing index data like this is the type of tree where we have consecutive in the protective leaves where we can just simplify the dimmer construction so understand correctly um the idea here is that if we could easily prove receipts something like a zk roll up where some some roll up somewhere optimistic or ck might see a transaction that updates a state route but it doesn't have a way to verify that the evm actually did successfully update that state route it wasn't just one line

right like you can send a transaction with the state route update it doesn't mean it was accepted whereas the receipt actually says yes this state route was accepted on layer one and by having access to that receipt in some roll up you can then prove that and say yes we verified that so this is actually only if they do not have to re-execute the same layer 1 process is to just prove the receipt and that's quite inefficient right now and nearly yeah there are its cases for optimistic rollups that make it even more difficult receipts can contain many many locks and instead of merklizing and logs for example they're all just concatenated

and coded and hashed him on go which sounds sufficient but for layer twos and these these protocols don't interpret individual looks it's really difficult to prove right so if we had ssc and vertical trees everywhere you would have be able to get a nice little witness of a single log potentially i mean if you really want to go all the way a single data point from a single log and just say hey i'm going to prove to you that this log occurred with this as parameter 3 or whatever and you can be confident in that just with the witness and a single like block cache set for example exactly that let's improve the usability of layer 1 information for layer 2 by a

lot yeah it's it's really compelling like like i said like if engineering was free i would 100 percent uh be pushing this super hard and say i mean we do have resources at optimism for this type of stuff and i'm sure other tools are interested in such changes as well anyway maybe we should just open an eip or something for this as a separate thing and then we're back to the sse

discussion this is one example where look information in a different shape or different commitment type is really useful i'm sure there are others and if you know of any any of these you should aggregate those and maybe you can include the same eip or at least add this to the eip discussion about the format for the historical blocks i mean there's i'm guessing someone could make basically more or less the same argument broadly for um

vertical of verticalification of the state right like you want to prove some piece of state and you just have a block cash i mean i i've written contracts that do this and it's hell you're paying 600 000 gas just for the proof like alone not even invalid just getting the proof on chain is half a million gas um if i understand correctly a vertical proof would be way smaller maybe if you thought it has the time to comment um so what was the question again whether or not we should just directly skip to focal trees for looks and receipts and such interesting

um it feels like something worth considering right but i don't but i also think that maybe we should not like we should not commit to that decision until like we do have some broader discussion of like what does the endgame here look like um because i guess like the thing with ssd right is that if we just move things to ssd then like they become an ssd thing along with all of the other things that are in ssd um but the uh thing with verbal trees is um like basically it would be it would be nice if possible like if we start moving moving things over to verbal trees like

if we want to just standardize and say virgo trees are where everything is going um and then there's obviously a lot of a lot of benefits that come with that um but then there's the question of like we want to make sure that we get standardized on like the same sort of structure as we're fine at as we're introducing in other places like in the state just to make sure we don't accidentally like create two slightly different implementations and like blow up to like four different kinds of structures instead of three hmm yeah yes you can see even as you see and you get more future compatibility

with frequencies at the cost of this longer experimental period i guess i'm just thinking like like well what would need to be standardized with vocal trees probably the main thing is that in a virgo stage tree like we have this thing where the keys get assigned randomly but in one of these virgo trees it would be like the keys would be assigned sequentially i guess randomly as opposed to just the case t commitments or um

oh one other question is do we have an uh do we have an upper limit on how many verbal or how many receipts or transactions there can be in a block i guess guess i spouse right right it would be nice right so for for receipts and and transactions they're currently like we store them in merkle trees but if we switch over to ssa is that necessary like doesn't ssc handle lists better like it does to store them in a tree at all there are two problems i currently deal with they would already be solved with sse

here in the best of your scenes but i like future compatibility as well so i mean if there was a way to verify these circle commitments or these kind of factor commitments whatever trees are built on top of at this moment if we had that in the evm or as like ready libraries for outside evm as well then i think we're good to go and just directly skip their maybe skip as you see so sorry so zayn if if we had all the infrastructure for both you would prefer to have the receipts in a vertical tree rather than just necessary yes i mean the layer two will eventually

have to verify it in the evm or outside of the evm and in either of those places we have good tooling for freckle trees right now and for the case of sse it's just a matter of using the most popular hash function with concatenation of 32 bytes and 32 binds and that's it the mercury station is really really simple what's what's the advantage of the vertical tree for a just a list of receipts why would you want that if you had all the tooling i suppose we can verbalize the home block at that point it would be great good for zika run ups for the optimistic case i don't think

there's any special benefits maybe proof size why does it help the zk rollups to have a vertical list instead of an ssd list because hash functions are expensive right the fergol tree is uh deliberately designed with a specific curve which is like the exact same or the field modulus is the order of the ls1231 which makes them makings you get snarks over it really easy i see and we lose that feature if we just do an ssa list and so therefore what would

have been a very easy proof to generate is now a more expensive proof that's accurate i mean the current design is crazy expensive anyways because of the rlp overheads all the logic gates around encoding the rlp before mercalizing at each step ssz is simple but still expensive because of the ash function and then vertical trees or any kind of factor commitment in that area of research would be an improvement for ck layer 2 usage just a quick pie in the sky question um does it make sense to vocal for you the entire block just like from the head or down

eventually i mean i think like eventually like i think uh if we standardize and say virgo trees are the thing that we're doing then uh like it'll make sense to move beacon blocks over to it and move like everything over to it um do we still with verbal trees like that do we still get the ability to do like uh proving just arbitrary pieces yes yes the proofs for arbitrary pieces become even shorter hmm so now i'm now i'm rolling through my head well why are we stopping by ssc on

the way to the verticalization of all the things yeah i mean i guess one argument for it is that like if we at least shove things into one format then it becomes easy to upgrade that orbit to a better format all at once um not sure like right now that format is i guess that format right now is rlp plus uh right well right now we have uh well right now we have a really complicated mix of or we're moving toward having a complicated mix of uh what was it there's some um rlp some sfz

um some vertical and some uh bls4 3d 1k zg also defense encoding so once we move all the encodings over to the same system then we can redefine the mercurization or the vertical fertilization later but at least that part will be consistent and right now we are really messy situation where the encoding rlp is part of the competition 3 and thus also still dependency if we were to move to everything to sse we would have consistent encoding

everywhere and then we would not have special encoding rule rules in the murphy station and then ideally we just swap that out for verbal trees okay does my little road map there sound like something that is hypothetically reasonable and align with what you guys are saying mercury leaves is a little bit fake for those that are out of the loop but yes i guess i mean you mean merkley's [Music] the transaction list merchandise the

receipts list i guess inside of that technically you've then got rlp so it's not the the the leaf most thing i guess technically is rlp that's yeah so okay we have these three iaps for for us one for format one for the import export functionality one for the distribution at the very first we have this description that's right the funds there we should try and see how ssc exclusion payloads work if it's worth trying just improves the improved situation of our historical

block data which could help projects stack that from piper and then we can have that or not anyways we have the virtualization of sse on top and we could look at faculty fertilization here but i think for fur for us it's out of scope does it sound right yeah that sounds reasonable yep okay happy tuesday uh i'm leaving now let's go um i'm lost you want to continue to call

i'll make make a house now i'm just going to stay in here for the next seven hours myself be all alone see you guys later thanks later you