Verkle Implementers Call #12

Transcript

hello all right welcome to veral implementers call number 12 this is issue 950 and the PM repo uh let me share the agenda quickly in the chat here today we have I believe five agenda items six sorry six uh starting off with client team updates we have some test net updates we also so have uh EIP 2935 blash some updates there uh Gom recently opened a new EIP for the state transition method uh there is a proposal for code packaging in veral and then

assuming we have time and we can get through all this quickly uh some testing team updates at the end and I think that's it uh cool client team updates would anyone like to to go first I think I can give a quick update from NE uh so there is no big changes just we tried to join the testet and we like found out a few issues and the same time trying to Reas changes on um the denone updates and yeah that's it

onum J side uh so we were also able to join the test net and we were able to sync up till uh some 478 block and we sort of uh ran through issues on 479 block and even before that we ran through some issues which we discussed and debugged and sort of applied some patch to sync through constant in4 uh but currently I think uh we would need uh the new network launch so that basically we can uh we can execute stateless on the Miss on the witnesses

which are missing in the current tnet and without which will not be able to uh execute further uh with regard to block hash uh your home has uh sort of it the block has implementation and we'll be able to use it currently uh basically in the current syncing we were not resolving block hash properly as well so that could also be the reason that we might not be able to execute few of the blocks of this particular test net update from the ners team U so the last two weeks we put in some effort to

optimize the deserialization function uh for uh bandwagon base field points uh because it was pretty slow as compared to like go like the go implementation after uh optimizing basically what we implemented is we implemented the uh dlog pre-compute that uh is meant to be done for uh calculating square root of points and uh turn out that currently overall the Nimbus cryptography library that is there in Constantine is at least 19% faster than that we have at go and from the integration perspective uh we are almost done with the stateless

primitive testing uh that is like in consistency with uh the go implementation and we have come to a conclusion that we can start off integrating U into cost via the nimbus execution client first so we are currently looking into that uh that is pretty much the update from the nimas side uh for B Karim from B has left some message to read here so I'll just read it uh We've made changes in Bess to connect to new test net Genesis configuration

Etc uh I think there's a bug about next Fork Block in get already talked to gillom about that EIP 2935 has been implemented allowing the storage of block has history in the state gas cost modification for veral are still in progress there are some bugs but we managed to import more blocks with uh contact creations and transfers uh we also been able to fix roll back uh bugs in Basu important to uh manage reorgs uh we'll soon need to focus a lot on performance but that's in the coming

week weeks so regarding that we need to change the tree to compute uh to partially update the the commitment not to recompute uh all the 256 values every time something is needed to change in the tree so that's next for the performance uh yeah we have uh been working mostly uh to help other clients join the test net so indeed like people already mentioned there's a few bugs that have been uncovered so that's that's good um

none of them will be fixed uh for this instance of the test net because it doesn't look very critical uh I mean you can work around it and we can keep looking for bugs so that's um that's the the way that seems the simplest at the moment uh otherwise we did uh some a lot of uh specking um R wrote some changes for the EIP 4762 the gas cost EIP uh there's been a complete redesign of EIP 2935 and and I also created a first EIP to describe the transition but I will I will cover that

later and uh yeah we also worked on testing tools so um yeah we hit a few roadblocks but we're I I just pushed a couple fixes this morning and uh otherwise yeah ignasio has worked on reactivating the replay so that we can uh produce data from the current height of the of the chain see if we can well first confirm that we have something that works in a more with a more modern payload like real payload and also uh do more performance testing check the size of proofs etc etc and I think uh oh yeah I did a bit

of there's been a bit of rework for the sh Shadow Fork code um yeah no no bug fix just just some pre Preparatory work and yeah that's it um just to extend a bit on that update um and I won't get into the details because I think this is the next my yeah yeah it's about 500k probably okay so um yeah we kind of fix

some bugs that will probably run in the next test net just I think most people here already know about that but I will just quickly mention uh there was like a B for 40 for 2535 uh that Cinder found regarding an desire effect of EAP 158 deleting stuff from the B history uh contract that we created uh he also finds found some other B regarding the no recording withness um data correctly for addresses that were let's say smaller than 20 bytes

uh also um thanish found some other bus regarding uh some of course that were in recording witness accesses such as balance and external goldash so um the fix that to and as said um we are now able to replay um at this moment 5 million blocks in important a chain with theal tree logic and that allow us to find a really nasty B um that caused like an inte inte R flow which caused an infite Lo which goost an out of memory problem um which

is one of the reasons why we like rep playing this long chains uh because this happened like four million blocks after it start um so it's a bit of a n breing kind of H because I is pretty bad but that simply means that we have to do some extra work in testing and this is kind of the bags that are found with some cling and also the fact that go doesn't panic whenever you underflow or under or overflow there are other languages that are much safe on that regard but yeah just just saying cool any other

updates if not uh next up the test net relaunch Gom uh yeah so I would like to bring your attention that now uh Barnabas every time there's a new test net relaunch uh produces so I'm sharing my screen I don't know if you guys see it uh produces a spec for the test net so we have one active where all the bugs are currently described um so there's the yeah the block number so most of them have already been mentioned by uh by everybody affected really but um so you typically when you see when you see when

we find a test net we will TR sorry when we find a bug on the test net we will try to update the the the spec sheet to to specify yeah if it if it got fixed after like if we decide to to fix the bug and continue or if we just decide to wait for the relaunch so so far all the bugs um have been uh this uh have been estimated to not require a relaunch but since they're accumulating at some point we we will try to do this um but yeah one one thing I forgot to mention so we launched uh uh defet three last week but we found already some bugs so we fixed them and we relaunched after after a couple days so number four uh number

four is going strong so far so this is uh this is good um yeah so I just wanted to mention that so if you have any questions or if you find any bug go go there first uh Tanish has found the bug self-destruct or at least thinks there's a bug we are still investigating the the last one we know of and there's also the the one that's not mentioned here that Karim described it seems that gu is sending the wrong Fork next Fork value namely that it sends Prague instead of sending zero when Prague should actually already be active uh but yeah these things are work you can work around them so so currently we we don't fix them I

mean we will fix them for the relaunch but uh yeah it doesn't seem to be a showstopper for now so yeah uh that's pretty much it for for that yeah Barnabas I would also like to mention that CIS is now capable of doing Shadow Forks uh out of the box so there's a way where you can test your uh El CL implementation uh whether it will go through the holos key uh Fork transition

using a state that was uh taken before uh Denon so this is a state that is uh Preen and the state is about 16 gigabytes uh so it will take some time to download that state but we offer it in a S3 bucket and it would be great if different client teams would be able to start using this and uh start testing it uh yeah so just to add to this so I did I did start using it this morning Works uh works really well and gu immediately dies so I mean immediately like after after a dozen blocks um but yeah to support uh to run it you need

you need to be able to support the transition so uh uh but yeah if you want to try to join that that's extremely useful as well indeed you mean it dies before elro is hit it dies before I no the transition starts and then it dies like it produces a block that it can't uh reinsert in itself so yeah and if you find any bugs just uh create an issue or let me know because I can quickly fix it cool okay next up some updates on the

implementation of EIP 2935 gajender thanks Josh so basically we had discussion uh with Tes and G and we basically uh came to a simplification in which what we are doing is we are writing uh the the entire so whenever there is a there is a fork block the first blog of the F we are persisting whatever uh required history is is

needed for that block to execute uh so basically last 256 block hashes and uh in the slot and uh basically this solves the problem that it simplifies and solves the problem that okay you know uh we don't need to detect the fork height anymore and uh so just we just one access to the uh slot one can figure out okay whether you have the block hash or not uh so basically if you look at the ELP uh there is a Pudo code uh that that basically at the start of the block uh you insert uh the parent hash uh of the block into the uh in into the slot and

if this is the first uh block of the fork uh then basically you insert extra 2505 ancestors and uh the way you resolve it so you can now switch uh the resolution of block hash directly on the first for of the blog itself to just uh doing a lookup in the U in the state for that particular slot and uh if you not that now basically you can have a lookup on a bigger range uh which is basically the entire uh Max history window which is basically all the slots that are there so uh you could only look for 256 last

256 blocks but uh with this update you will be able to look at uh the entire range whatever is there till the r around happens so then uh there is a question regarding wound uh and that is what is the basically what is the size of uh the block number because uh slot size is is 256 bits and the block number is 64 bit as per my understanding as of now so AR is going to be of the same size so question is whether we take care uh we basically consider that there is a rant that is

going to happen or maybe in future when uh when that are uh when the block number size uh could be increased then basically we add in this condition so uh thoughts and comments are invited on this um sorry there's something I didn't quite get so you said the wraparound is going to happen at 64 bits right no no wound will happen at 256 bits only but our current uh uh block

number size I think is 64 bits so right I mean I'm not sure how how that situation will plan out whether before this we come to this situation we might increase the size of block number not sure how that is going to happen uh yeah I think we have for a couple million years before that happens um because the order yeah I I would have to to write the math down but if I'm not mist we're we're going to be fine for a long time um yeah okay but okay so if like the the wp around would be 256 I think we're fine right because that's more time than

uh is left in the like in the life of the Universe I think uh yeah exactly that's a lot of time yeah so basically maybe we can then move the r around condition itself that basically checks so there are two there so the condition uh before we actually look into the block is that the AR has to be uh AR has to be less than block number and obviously it should not be WRA around should not be wrapping around so maybe we can remove the wrapping around check but actually no uh I was just thinking because yeah like realistically it will not happen but you have to check

the lower bound because yes it will wrap around if you start asking for a negative block right and that should be maybe what the check is about uh so okay maybe we can yeah so we can check for the negative block and basically make check that AR is if your AR is less than so AR cannot be negative because AR the input AR g z in right so I'm not sure how we'll get a negative AR anyway uhhuh okay uh yeah I guess we need just need to continue talking offline uh thanks okay uh next up we have an EIP

for the uh the base EIP for the state transition method uh Gom oh yeah um right so last time I made a suggestion to have a less ambitious fork and um is that the the right window yes um so I created an nip for that and I'm aware that yes there's been a lot of push back so that's understood and I also won't seek to to push this any further uh however it serves uh I still want wanted to create the IP for two reasons the first one is because uh that's a first uh let's say that can be

used as a base to be extended to create the full-fledged EIP which I'm currently working on but I need to keep uh learning a bit more about how eels is built um because yeah uh presumably we we would use this their terminology to describe the EIP um but so before I do this I was like I'm going to to take the common the common denominator which is basically this EIP because this EIP is the first step that you can then extend with the full-fledged transition and uh the EIP number is 7612 um and uh yeah so I just meant to have this EIP like I said to uh as a

base for the the next EIP that's coming but also in case we during development we change our minds we say no actually doing the just the overl tree and taking care of the conver the conversion later makes more sense at this time then we have ANP to fall back to but like I said uh it's not something I'm I'm pushing for at the moment but yeah if you could just have a look and uh give some feedback if you if you can uh that would be really appreciated because uh yeah it's going to help uh with the next step which is the the full uh the full EIP like the full conversion e cool all right uh next up we have uh a

proposal for code packaging that I believed uh Pavo Gom and a few others have been working on um Pap did you want to start with any thoughts there I don't see him in the ah is he not here no I don't think he got the message okay yeah yeah next week I guess uh yeah I wouldn't be able to to present it but the idea was that um there was a couple ideas that were that were quite interesting I would prefer him to to present it um but the idea was uh to do some packaging of like do the packaging

differently uh namely that you would have chunks of 32 bytes and then you would have a map that says for each block um that starts uh for each chunk that starts with push data uh that map would tell you okay this is where this is where it starts and yeah it it has a few interesting ideas so I encourage people to to go have a look uh clearly it's eof based we could also Implement those ideas without uh eof or more exactly to prepare to pave the way for eof I had a couple of questions for pav myself so I really don't want to be the one presenting this um but I think it's quite interesting so we should uh I mean

uh I think everybody should have a look and uh and try to come up with with questions for pav next time cool okay uh then we can move on to testing team updates Mario are you around yes I'm here um can you guys hear me perfect yeah so we've been um doing some implementations and changes to the execution spec test uh repo that we have uh to accommodate for verl um so far

we've started with the very very Basics which is the transition tool support uh changes that we need for veral uh which is the tool that we use from G to fill every single one of our tests and we've been working with gam to make uh this possible um it's still a work in progress so we at the moment are not able to generate any tests but when we do um I have some um suggestions and I wanted to just uh share them here to make sure that we are on the right track so this is very uh it's very subtle but I wanted to share my screen I don't think I can um nope oh yeah s give me one second

yeah want to do this now it's sharing my screen second SC if it's not possible it's is very little information I can just yeah if you just give me up to you but if you give me one second should be able to I'm trying to find this toggle uh oh sorry okay here you go sorry but you should be good now yeah I can see it now perfect okay so can you see my screen

now yep yeah okay so yeah basically we have this uh definition of forks in the execution specs so we started with this um um assuming that U the current implementation ble is on top of PR and prag is not on top of but it's in on top of Shanghai this is what we Define so we have two important Forks defined and one is the straightforward prag and the other one is I think more important is the shanai to PR ver transition so these are the two forks that we plan on using to implement all of the veral tests um and the main difference between them and I think most of things we will have to

implement something in their Genesis procing tool to be able to consume our tests but the first one is pra when we um start with the Genesis that we'll have PRACK activated with Burl and the purpose of this of this is just simply from the B Genesis we provide a pre-allocation for all of our tests and the idea here is that um the the clients consume this preallocation for the test and it will be it will go straight to the to the veral tree um I think this uh this first approach is just to make sure that everything works I mean all our tests that we Implement they work on um they make the proper modifications to the Verry when

we have already converted to fully fully converted to bur tree and this one this more interesting one is the Shanghai topra bur transition which is um basically just we start with Shanghai or whatever previous Fork uh we are forking to to veral and then on the first block we we Fork to the to the to to the veral uh the veral transition and what this will allow us to test is that we have a Genesis that defines the pre-allocation to all of the state accounts that for example we use the smart contracts that we use to test um every single one of our tests we start with that on the Genesis but the Genesis is pre veral so

we will start with the EMP tree and then on the first block we will go to Burl and I think this is very interesting because it will allow us um to start with the MTT with all our accounts and then on the first block we start send the transactions that the uh tests that we write and that we have written already uh all do the um the execution when we are are already converting uh tole not necessarily converting but also on the overlay tree that would be very interesting to test um and and another good thing about this is that um we have this um we have implemented this this this feature on our repo that would

allow us to not only produce a pre-allocation with the the things that we need but we also can um feed the uh MPT tree to have a lot of information a lot of not random but maybe we can design some interesting patterns for the conversion or or for the overly tree that we can inject into the MPT um so we can test this in in a regression fashion instead of having to for example um sh Fork into something we can first start test here with a big MPT um with a with a special pattern and then we can we can proceed to the um to the to the to the sh forks and and and more so um but yeah the problem is that this still doesn't

work um we're working on it and but yeah this is basically the idea the main idea of how we are thinking on on filling um the tests at the moment and yeah I would love to hear if you guys guys have any any doubts or maybe this doesn't sound um uh reasonable or or anything yeah um I have a couple questions so one of them is uh what what is this interface you showed us is it like some tool of yours or is it uh something CER I don't recogn nice ah yeah sorry this is the um let me

share again um this is the the documentation of the execution spec tests so we have this uh um this is this is basically the the the site that uh renders all of the documentation for tests it contains the definition for all the forks that we use to test and also the the fixture formats that we produce so right now we produce three um three fixture formats so the state test the blockchain test and the blockchain high test so I I think we can feel using uh I mean for the ver veral tree tests we can feel uh the Tre fixture formats and we can uh run them in in different ways I think for vertical transition only blockchain

tests and blockchain hi tests are uh interesting because say test wouldn't be able to do the the conversion at any point but yeah this these are the two interesting one for for ble Y and would it be possible I like when you when we start adding more clients I mean uh I I assume clients could already join the first batches of tests right the the one without the conversion um would it be possible to somehow make you know run all those clients that are available nightly and produce a a result like show a page where where each client is so that we can keep track yeah so I think we can do this with the way that we do it this for

Cancun uh we have the the the high view um uh renderer so we run all the canun tests on Loop every night and this page just keeps updating uh we can do a similar page once we are able to generate the hi the blockchain hi test this ones I think we can run on Loop uh for for verel with all the clients the same way that we do now foring good cool and I had another question that's related to the question I asked you on telegram so in order to perform the conversion I would need to I I need a snapshot so the the question is uh well you you said you know you wouldn't

really be able to pass the snapshot that's fine uh I could regenerate it the question is how much time do those do we have to run those like is there a time out for for these test to run no not at all that that was what I was thinking so um I think we can selectively um generate this tests because I think it would be very intensive to start adding a lot of stuff uh that is that is necess for recal but it's is there's not no timeout but it might take a lot because this is python so it's not very um efficient but anyway I think if we need to add it it's it's possible we might just generate them uh more sporadically

uh as compared with every PR for example right now we generate every single do with every PR so right now I think we can lower that um that period and and just make it uh so that verle tests are generated every every I don't know maybe release or something but it's not a problem regarding to timeout that was basically it um yeah if you guys have any input or ideas on what we

can do um with testing we have now a vertical branch which is out of date right now I'm going to update the uh uh this Branch with um let me just share the link I will update this Branch with uh with the prision Tool uh updates that we're making uh but this will be the branch that we will uh start um adding veral test and everything ver rated um so just um just so you guys know and yeah that's that's basically it awesome thank you Mario okay cool I think we had a or we

do have a last minute addition to the agenda from Kev Kev yeah thanks for that um sorry my audio is a bit bad um essentially I think me and ignasia spoke about it a month or two ago it's replacing the do serialized method in Patterson hash um for basically map to scaler field I wanted to check if anyone's against that and uh yeah basically check if anyone's against it before making it since I can change uh so wait I didn't get it so you want to remove all

sterilizations and just use map to scalar field um I wanted to remove the do serialized method for map to scaler field so you'd still return a 32 by value it's just that um you wouldn't be using I think we lost you Kev may not oh that was it okay got it I dropped a link in the chat I don't

know if anyone else has any questions or thoughts on if not all good cool well we made it through all the agenda items uh ahead of time anyone have any last minute additions thoughts or we can just uh end there cool well thank you everybody and we will see you again in two weeks thanks bye bye bye see you

goodbye