Stateless Implementers Call #32

Transcript

Okay. Um, welcome everyone to the stateless implementers call number 32. Uh, I will share the agenda in the chats. So, as a summary into in today's agenda, we have um team updates as usual. Um I will be presenting some 7748 um execution spec tests uh document and talk about uh that for a bit. Uh then we have two uh testing topic

discussions. Um then Carlos will be talking about postquantum burkel and GM wanted to touch on Xcode copy behavior. uh when targeting system contracts. So let's start with team updates. Um I can start. So I've been working uh a lot on yeah EIP7748 uh execution spec tests. These are basically the tree conversion. Uh this is a true conversion EIP.

uh as we discussed in previous SE calls um it's quite important that we have some confidence on people um implementations of DCIP um so one of the ways of helping Y clients um test their implementation is actually having execution spec tests uh because there are a lot of um yeah things to test corner cases uh things like that. So, uh yeah, I decided to start working on this and I will share more videos later.

Anybody else want to share Tima please? Uh I think I can go next. So I'm just finalizing the transition implementation in Nethermine and hopefully I'll be able to run the spec test by start of next week on Bezus side. Could you hear me? Yeah. Yeah. Yes. Yeah. Okay. Uh on BU side uh still working also on stateless. Uh we stopped to work on the vertical cryptographic part. We are just implementing state test without this part and in the transition we have a first implementation. So thanks to Ignatio for his test, I was able to fix

several issue in my implementation and uh I'm trying to finish uh all of the HKS like that when we'll have a devet in the future it should be work and yeah I'm also waiting for the new test from natur to to detect if we have other problem in the implementation. Good. I can go next. Uh, Ethereumjs, we began working on the conversion uh implementation using uh underlying like MPT and vertical. Uh, we have pushed back a bit our you know target date uh for releasing that because we've been focused on that like

the yearly release that we do a lot of last minute work for that. But we're going to resume work on that this week. And yeah, hoping to be able to start iterating on the tests. Uh maybe not next week, but the week after would be more realistic for us. Um, anyone else want to share some updates? Okay. So then we can move to

the next agenda item which is um a tree conversion test document. and available fixtures. So I will present that a bit. Uh it will be pretty quick I think. Uh but first I will share my screen. Um okay hopefully you can see my screen. Yeah. Yes. Yes. Okay. So uh I will share the link in the chat first. So um okay these three conversion um tests are pretty complex in the sense

that there are a lot of things to cover. So I won't go through this document in detail. Uh but the goal of this document is to really explain um what we are testing and what is the status of the testing. Um so this helps like first it helps me to reorganize the thoughts and really understand which are all the things that I should cover. Uh and also I think will be helpful in the future for the testing team to really understand like my thinking in going through all the test cases and just simply document document things better than expecting them to read the Python tests which can be confusing.

So um yeah, if I don't expect everyone to read all this, but if you want to have a sense or a sensation on like the main um reasons why we need to test this carefully are um you probably want to see this bullet points at the top of the document. I think that with that anyone can build a a good intuition of what can go wrong um in the tree conversion. So basically the rest of the document is trying to cover a lot of things that can happen in each bullet. Um I also added this big picture plan section which I simply wrote some

quick thoughts about how we can test the tree conversion in different steps. So for example, the first step is basically um building this execution spec test that I've been talking about and letting EL clients run them to have some really quick way of um testing that the implementation seems to be correct. And yeah, execution spec tests are really nice for building really weird cases that are really complicated to really simulate in in real chains or devets.

So I think that after we have some reasonable coverage here and we feel confident, we can jump into maybe running a devet uh trying to bootstrap it with a reasonable amount of genesis data in the MPT. So we can define maybe the uh like the the the the AP will activate in some specific block or time stamp and let uh all the clients go through it and see what happens.

Uh I suspect this is mostly for um testing uh stuff that the execution spec test can't really cover like chain configurations and maybe the engine API on some stuff. Um but anyway um I I kind of put three faces here. I don't think we really need to do all of them. I just wrote them for just as an idea. We can create this uh devet with some synthetic data. We can generate some random account and contracts data.

Uh we can also try copying existing data from test nets. Uh that's a bit more interesting or we can have some kind of randomly sample data from mainet. Um I know that's like an idea. uh after running a dev net and seeing that things can uh like work correctly, we can do some kind of shallow fork um of test nets or mainets. Um so I don't know I I don't think it makes sense to go through all of these. Uh maybe it's a lot. Um maybe after execution spec test we realize

that running only one variant of this in a deet is okay and then we can jump directly to trying things on mainet or something like that. Um so yeah take this as like just as an idea that that that I have about like what is the big picture of how to move from one kind of test to the next. uh uh basically the rest of the document is um describing multiple testing scenarios. So if you only want to see like on the high level just read the title. So we have no known partial account conversions which basically

means testing cases in which accounts are migrated are converted within a single block. So for example, EOA accounts are always converted in a single block. Contracts kind of depend um if they have not that many conversion unit can be converted in a single block. Um and things like that. Then we have partial account conversion which start to be a bit more interesting because if a a contract is big enough converting uh it fully will span multiple blocks. So it's important to cover all the cases in which um uh there's like boundaries in the

account conversion logic that can match exactly blocks or or not. So yeah, if you want to know more about that, you can see it here. Um there are some code chunks try correct accounting tests. um because the way that the IP is constructed today, we always migrate the whole u by code data in a single go. Uh but we still account those transaction units to put a bound on the worst case. So I'm trying to cover that. Uh there are some special accounts that we should cover. um like accounts that we shouldn't migrate as

defined in the IP and things like that. There are some really weird cases that can happen exactly at the blog where the conversion ends. So it's really good to cover those. Um then we have test cases where uh things start to get a bit more complicated. So for example, you have accounts that already had rights by previous transactions. Um so whenever you move the data you have to be checking that you are not writing stale data. So there are a lot of things there. Then we have blocks where um the account that you will be

converting in that block also has a transaction in that same block that write to that same account. So yeah, some things there. [Music] Um, and there's like another case which basically doing state accesses to check that actually the converted data is read read correctly. Um yeah, this is a bit has a big overlap with um um with EIP 7612, but I will talk about that maybe

later. And finally we have to test for reorgs and being sure that your clients are managing the conversion internal state like the like managed correctly whenever a reorg happens with the conversion progress. Um so yeah this is a bit more advanced and actually we need to do some um testing framework changes to really support these kind of tests but yeah uh I think we will leave this to the end.

Um but yeah that's kind of a big picture of of of all these I use emojis to signal which ones are I already implemented which ones are pending to implement and yeah which ones I can't implement. because we have to do some testing framework changes. So all the ones that you see a green tick are ones that I already work on and yeah can share uh with all of you. Uh so today doing all these accounts for around 14ish um fixtures.

Um I have already a zip file with those um that I will share maybe in the matrix channel or somewhere. Unfortunately, those can't be um like included in this formal releases that we do because to do all this work, I had to do some um quick hacks in the testing framework and in G um because um yeah, there are still some things that we have to support um in the testing framework. Um so to not really be blocked with that. I have to do some changes there that we will eventually um kind of formalize with Spencer. But

yeah um I will share the the file with with you guys and you can simply run the tests and we we can see what happens. Bear in mind that uh there might be cases where GU has a bug. So um I already shared before like around 88 tests with Karim and he already run those and he passed them. So there's some reasonable amount of certainty about some tests but many other tests that I wrote after uh it might be the case that G has a bug and since get is being used for failing tests um yeah yeah that that can happen so whenever you see a failing test don't

assume that is your fault uh maybe get fault um so yeah let's just collaborate a sync and try to make progress and get some confidence. After that, we have like another client that fully passes the test, then yes, like there's a higher chance that any anyone failing on a test means that your client has a bug. Um, yeah, I think that's it for this agenda item. I don't know if anyone has a any question.

Yes, uh I don't have any question. I have a comment regarding the potential DevNet launch and uh also talking about bugs. Uh there is currently a bug that was introduced in GU. So it no longer like the shadow fork currently is broken. Um we know exactly where it was introduced. So it has to do with the serialization format we we changed. Um all I'm saying is if you want to build some test net uh on your own to do your own testing as preparation for for um for doing like such a shadow fork or transitioning test net. Uh G has a bug

that needs to be investigated. Uh I mean sorry it has been investigated. It needs to be fixed. Um yeah I I hope to fix it. uh shortly, but for your information, yeah, if it doesn't work, it's normal. Any other question? If not, we can move to the next uh general item. Okay. So the next topic is um EIP7612 uh transaction writing to the contract storage log indirectly convert specific

data. Okay, this is something that I I will explain a bit better now. So when I was writing the test I realized some fact that might not be surprising in hindsight but I just wanted to surface here uh just to be sure that I get others opinions around this topic. So the situation is the following. Um let's assume that we have this overlay tree. So we have the frozen MPT with data and we are using the U new tree on top being vertical or binary it doesn't make a difference. Uh and we have a transaction that writes to a writes to um contract storage slot. Okay really

simple case. So something that will happen at least what happens in G is that okay um clearly this new storage slot right will happen in the new tree because when the overlay tree is activated any new right goes to the new tree. But this also indirectly means that the storage route of this contract will get updated right. Uh and this also means that um the way GU get works will also migrate the account header to the new tree which at first was somewhat

surprising to me because I was expecting okay if a transaction only touches the storage slot of a contract it will only write the storage slots to the new tree But uh I get the point that since the storage route was also updated, it kind of interprets that in technically speaking the account header is also touched. Um but it's a bit weird because the storage root is not a field that we have in the new tree, right? So actually the right this account header right in the new tree is exactly with the same data that is in the

MPD. Um so you know bucketing this as a uh a conversion that happened because of a right is a bit handwavy right. So anyway, um maybe it's a way that everyone agrees that this should how this should work. Um I think that would be the best for get um um but yeah, I just wanted to surface this like fact uh which might not be intuitive. Um and yeah, I don't know if anyone else has any comments. That's mostly my point.

Um, I do have a question. When you say you've write something to the storage, um, and you said the storage route gets updated, that's the part I'm a bit confused about and I Yeah, I should have asked that before. um how like the storage route itself does not get updated, right? Because the MPT itself is frozen and so there's no reason to update the state root. Um so yeah, I I don't know. I'm confused by this. Could you please detail what you mean here?

it um this is what get interprets that uh should happen in the current implementation of the overlay tree. So it detects that the state the storage routt should change and for that reason includes the account header as something that changed and should be like refreshed, right? Okay. Yeah. So it's okay understood. In any case, this is something that will become apparent whenever people run the tests. Um because if your client is not

doing this, clearly you won't agree on the new uh tree storage uh in the new tree route. So I mean eventually we will find out if anyone else uh doesn't agree with this how this should behave but I just wanted to raise this topic um just for people to be aware um another question I have is is there any alternative um to doing what guest does really like in in your mind is there anything else that could be done because I can't think of any way to do it.

Yeah. Actually, not migrating, not converting the account header, right? Because it didn't change, right? Um Yeah. Yeah. Okay. Yeah, that's true. We could do this. We could try to do this. Um H Okay, understood. So, ju just to be sure I understood. So when you are writing a slot uh you are migrating this account to the to the new tree and the state root in the the storage route in the partition tree will not be modified right so technically it shouldn't be modified because the MPT is frozen but like the way that get decides on

which things got touched um is kind still assuming that the MPT is kind of active in a way. Uh so it considers that the the the account data is touched and is doing the right because in theory the storage route should have changed but actually the right happens in the new tree but this field doesn't exist there. So there's this weird case. So yeah. Okay. So let's simply wait for people running the test and we can

double check if any like there's any other client that doesn't agree with this and we can talk about this again. I don't think it makes sense to maybe discuss it further. Uh the next agenda item is um testing framework support for pre-im images. So this is a topic that we chat with Karen when he was doing some alpha testing of my fixtures and he was raising the point that maybe it makes sense for the fixture to provide the pre-im images for the tree conversion.

Um okay so the way that get runs the test today is basically that whenever it receives the pre-state to run the tests it records all the pre-im images of the tree. So whenever they have to do the tree conversion logic it already has the pre images. Uh another possibility um is that these primages are provided by the fixture JSON file and you can simply ingest them and use them for the tree conversion. My take on this is that we shouldn't add this to the testing framework. Uh I know that it might simplify some EL clients logic but I

simply believe these three images are redundant data because whenever you ingress the pre-state into your database for running the test you can actually do this recording and yeah I just want to prevent some you know extra blow up in the testing framework of just keep adding fields That's can be avoided. Um um yes Kajind there. Yeah, sorry to just take the conversation back to the uh tree migration. So uh what do we think of having uh basically persistently

migrating trees where after every epoch we migrate the tree so that there is stiry also going on. Basically the previous tree is frozen and then there is a new tree where things are getting migrated to. Uh what do you think of that particular uh mechanism in respect to uh our work here? I'm not sure I quite get what you are proposing. Is this related to testing or something else?

No, no. So this is basically that I'm proposing with regard to T migration and it's not me. I'm proposing basically somebody else proposed and I just remember it. So I'm just putting it out there that uh one way uh they said that uh one way basically to uh to have uh the new trees is to have basically you know uh also to have uh uh state expiry along with uh the new mechanisms. is that that every epoch we basically put in a new tree and expire and freeze the old tree and then m and basically do the same migration mechanism each time we see something happening uh some account or some something touching apart from

the first migration that totally migrates the tree but uh then there are these uh these trees every some epochs that you create and you freeze the old tree and sort of expire it. Uh so what do we think of that mechanism with regard to uh the work that we want to do in in the stateless? Um I mean I think there's a pretty strong agreement uh with everybody I talked to at least uh except maybe you now but uh that uh we shouldn't do state expiry right off the bat.

Um, so yeah, I think I think that's going to be a big uh yeah like scare it's going to scare a lot of people to to do this because you have to go through the transition somehow and on top of that so you you have to handle like two proof system which is already huge on the RPC uh providers and um every API that you can think of and then on top of that you need to implement the API to handle the state expiry Um so that would be I mean yeah that would be my opinion about this and my at least my understanding of everybody's opinion as well that uh anything doing state expiry while the

tree is being converted is is a no go no no I'm saying that once the tree is converted then we basically consistently keep on expiring trees in this particular manner I mean this is just state expiry and maybe an independent topic but uh I want to know what we think of it uh in terms of uh since we are designing this migration mechanism then basically it makes sense that it can be extended to do state expiry after obviously our migration is complete.

Yeah I see. Um yeah is there a write up with this because you say someone told you about that. So is there a write up so that we can dig a bit and uh form an opinion? I think uh Sophia only mentioned it in uh PR PR MD group. Uh so I'll check I'll check and get back to you on this. Um so sorry I think uh Ginder I only caught like half of uh what you were saying. if we had a way to do um state xray with separate state trees per epic like um was discussed I don't know five years ago I think then we wouldn't

migrate the old MPT and that would be very nice both because it's like kind of an ideal form of state x-free and you avoid having to do the migration um but that was ruled out a long time ago because address space extension or contraction is kind of like a UX nightmare Um, I've only seen one proposal of how to kind of avoid that and limit proof sizes for resurrection. Um, but it was never really explored. So yeah. So I'm not uh talking now in uh in the uh in the context of MP3 MPT but I'm now basically talking in the context of

the stateless tree uh worker or uh binary tree. So, right. But if you if we had a viable way to do state Xray that way, then I think what we'd want to do is to not migrate the MBT at all because it saves you from having to do that. Um, but there were other serious problems in doing that kind of state Xree in general, which is why it was ruled out um back then and the only work since has been the leaf level for vertical.

I still think there might be possible ways to make that work, but uh no one has really been working on it. Um I can link to sort of the last discussion of it. Um it's this external period registry thing where you keep a separate much smaller binary tree tracking uh hot state. Yeah, basically I was thinking uh of the same thing how to uh sort of modify the proposal to just do that in the current uh uh in the new migrated tree uh so that the pitfalls that you mentioned uh would anyway be avoided and because we would have anyway migrated a tree out of

MPT so MPD pitfalls won't be there anymore. No, I I mean I think the problems I'm talking about don't have to do with different tree types because even back then it was proposed that uh you could freeze the old MPT and just make the new ones vertical. Um, uh, I know Guiam just mentioned that having two proof types, but I think that the the bigger issue is that these proposals, all except that not really fleshed out one I just linked to, uh, require modifying the address space in some way. And, um, I think that's probably a hard no. Oh, so basically modifying address

space to indicate whether it has been frozen or not because I don't understand how address space is basically uh coming into the picture when we are talking about uh post migration system for doing epoch level I think I I had to sort of piece this together from discussions and um uh old writeups but I think it it's to um uh identify like which um which old tree state is in when you're resurrecting it um as part of the proofs. Um let me link to something else. So my memory isn't

really quite fresh on that. Um I think this is the most comprehensive uh summary of the different like approaches that involve separate separate state trees. Thank you Sophia. We'll check it out. Yeah. So um this is this is an interesting topic and we should definitely come back to this in a further s call with more time. Uh um I was actually talking about something else in the agenda. Uh but I I get that this topic is interesting. So let's try to also allocate maybe in the

next call if people want to talk about this um as like an alternative way of doing the tree conversion if we find some viable strategy. Um so coming back to what I was talking about about the pre-image supporting testing framework I don't know if anyone has an opinion around that. Uh if that's not the case we can move to the next agenda item. Yes, Spencer. Yeah, I uh just wanted to check. So, um how does the pre-image differ from the pre-state in the fixtures of the tests? So, the pre-im images are only

um okay, let me think about this for a second. Yeah, I think will be mostly the same. I'm trying to think if there's some extra case, but yeah, I think it's mostly the same. Maybe it's like has a semantic difference as a separate field. Um, but like technically is what GU does today is just take the pre-state and just use that information to have a separate database. So I I don't think like formalizing the pre images as a separate field to have a specific interpretation adds any adds any value.

Yeah. Okay. Um the next topic is something uh that Carlos asked to be added which is postquantum verical variants. Um maybe Carlos, you want to talk about this? Yeah. Um I'll try to not take uh too much time. Um so yes so basically when I've I've been revisiting a bit uh the reasons why we like Burkel and why we started Berkel and there's quite a lot of the reasons why we like Berkel that no other alternative is giving us like uh binary trees are not giving us uh small proof

sizes we lack the capability of aggregating uh the openings or having like decently sized opening And the the the more we raise the gas limit and the more we allow to touch uh state within a single block the the worse it will become. Um so I basically started to think about what if we could uh make happen vertical uh still and take profit of some of its uh properties but actually just trying to replace IPA and uh the vector commitments um for something that is actually postquantum secure.

So I've been like taking a look to all the state-of-the-art of the cryptographic papers and this kind of stuff and I basically made this write up which I will try to publish today on his research. Um so far I've been in touch with Arancha from the uh research team and Jako Mafeni and some other authors uh well-known authors from the snark uh and vector commitment landscape and I think there's like while I've been researching that it's obvious to me that there's more people that is actually looking into this like far more than I expected and I anticipated for sure and that does make believe that it

might not be a completely crazy thing to actually look at. It will also be nice in the sense that if we find this viable alternative then a lot of the stuff that has been coded so far is still valid. uh namely some of the stuff like for instance um elliptic curve uh related things we can completely throw it away and replace it with whatever it is but general uh tree layout and interactions with it and all this kind of stuff that that should remain the same. So we could reuse uh quite some work let's say.

Uh and yes, the idea is to start testing the physibility of that and try to get some benchmarks uh to actually see if it makes sense to invest more time on that and like consider consider it like a serious contender to actually propose some EIP or something or instead we just discover that that in fact there's not much to do and and we can discard it and focus 100% on binary trees or whatever we want to go forward with. So yeah, don't expect anyone to have much questions for now as the right extends. Uh but yeah, uh feel free to please uh give opinions or feedback or ask any questions or whatever. I'd be happy to chat about that.

Anyone wants to ask any question about topic? We can come back to this uh in further sequels after people read the document of course. Okay. So the last um agenda item is uh point raised by GOM which is how X code copy uh behaves when the target is a system contract. Uh maybe GM you want to talk about this.

Yeah. Um so that comes from a fairly contentious PR and created in the GU repo by yours truly. Um there's uh yeah there's the problem which actually stems from the guest team's insistence that we should be calling EVM uh like the system contract should be called with the whole EVM over overhead instead of just implementing the the code uh directly which is the intent of EIP779 but yeah there's some push back anyway the point being that uh using the whole EVM

system makes it more difficult to filter out which value which um item should be added to the witness and not uh when you when you call system contracts. Now there's a there's one example which is Xcode copy. Uh what happens if you Xcode copy a system contract? So currently um the argue well the argument is that every stateless client knows about it so there's no point in bloating the witness up with code we already know. Um the counterargument which is pushed by Felix among other people is that these are

just regular contracts and while uh you do this for example we also agree that if you decide to call any system contract with an address that is not the system address um the code should end up in the witness. But now if you do Xcode copy, it's just an instruction and that instruction underneath calls a system contract. Sorry, wrong thing. If you uh if you call Xcode copy, it's an instruction and it behaves like it needs either it needs to know that the system contract is a system contract and not copy it or um we need to just say well

in that case even though it's an instruction um and even though the stateless client knows about the system contract um or at least knows the code of the system contract We are going to add the code to the witness. Yeah. Ingasio. So just to understand the current way that 4762 is written, if I recall correctly, is that if you do an X code copy in a system contract, the code will be in the witness. Okay, that's not my understanding of the So if that's the case, then there's no more problem I would say. I mean there's still a bug in guest but that's uh in guest master but

that that's a different uh different thing. Um if that is the way it's currently specified then there's no problem. My understanding though because I found this bug while trying to sync the test net with uh with guest master. Uh it seems that this is not how it's implemented but maybe I got mixed up. Um yeah. Okay. Gajinder seems to say we should copy generate witness. Okay. I mean I guess it's simpler. So I would be up for this.

Um okay. Maybe at this point the action point is for every client to verify what happens when they when an Xcode copy is uh done on the system contract. And if everybody agrees that then the system contract ends up in the witness then that's the end of the story. And if that's not the case, either we explicitly we make it explicit in the in the spec that this needs to happen or um yeah or we we change the spec to to adapt to to what we like the what I believe is the behavior on the test net which okay Ignasio seems to think is not the case. So yeah, just uh please verify

verify your your code bases and let me know. Yeah. Uh so I was reading this link that I shared in the chat which is the system contract section of 4762 and says that apparently the only cases in which you do not add system contract um by code is whenever you do a system call or resolve a pre-ompile and no code uh it says however any other accesses and other rights should appear in the witness. So right maybe maybe some we are in the case of end of code because it's Xcode hash sorry Xcode copy.

Yes but what does it mean however in any other access all right should appear in the witness. Um so any other access sorry which line are you looking at system? Uh yeah yeah any other access is what it's referring to. uh any other access um refers to calling for example you call the system contract without like from a regular contract not with uh like from a regular transaction for example. Okay.

But then we could just remove to resolve a Yeah, we would have to go over all the up codes, but presumably we could simply change the the line to resolve a pre-ompile/upcode and then uh that would solve the problem. Yeah. Okay. So, yeah, I guess that's a spec change, but everybody seems to agree. So, okay. Gajinder agrees. Tanish. Uh, I don't know. But yeah, we should probably Oh, and Karim, I don't know about Basu either, but I agree. We should change this line. It's It's easier.

We We check. Yep. Okay. Um this was the last uh item in the agenda. I know if anyone else want to ask a question or raise any other topic. Uh if not if that's not the case, we can finish earlier. Okay, I think that's it. Uh thanks everyone and

Let's see us next time.