EOF Implementers Call #46

Transcript

now um welcome to eof call number 46 um think our agenda is client updates compiler updates spec updates testing updates and any other items if there's any other items put them in the chat um I guess we'll go ahead and start um as far as client updates um Basu um same status as last week um nothing too exciting has changed uh mostly writing tests um so let's see who else we have on the

call nethermind do you have any status updates you'd like to share uh not really same as last week okay um Diego's uh think he's Basu is who he's calling for or you hi Diego um that's all the clients we have online and uh evm one any updates yeah we still have uh three PRS in progress uh one is init container validation uh this is it works basically we already in more tests like covering it with tests

is kind of an effort there are like many combin ations of op codes and op codes that reference subcontainers so this is in progress another one thing is creation transaction this also progressed somewhat I think maybe close to completion and the removal of TX grate this is easy but also requires test updating so yeah almost finished okay gra re yeah hello uh for my side I remove TS create and and added missing validation for PF a jump F uh what's missing is like new create section but this needs to be speced out

and I need to check what test I'm running R validation but like State tests yeah yeah get that in testing yeah that should be it I think that the spec for for the create transaction is there right yeah there's I need to implement it and look at it okay yeah yeah but just to make sure I also understand where we are okay um I don't see G or ethereum JS is there any other clients we're missing I think that's it um or eels um okay so compiler updates uh solidity is not on the call um Charles any

updates for Viper yeah to jump in go ahead Charles uh no updates but if um UF is like kind of getting finalized then we'll schedule it um I think we can schedule a proof of concept for like the next month or something excellent excellent that's good yeah and uh regarding solidity uh I hope I finish the implementation of the prototype in based on the on the on the Shanghai version implemented by solidity team uh today I I contact also Daniel regarding this he also had a

couple of couple of hints for me yeah so so basically basically it's going well I implemented New Creation uh new calls X calls and a couple of new things like jump F returning functions it looks quite well regarding the process how is it going but we will see uh I will I will try to finally also uh generate some Matrix for uh for the cold size and uh and gas uh gas consumption for between and comparation comparison between hello for solidity and for Viper

this gives us a good opportunity to see actual hard production numbers yeah I think we have some for connection problems did you hear us you cut you cut off for for a second like for 10 seconds sorry uh East African um stability they it's quite good but it's just like the last bit of Ric sentence got cut off we had to we had to get rid of two zebras coming here sorry okay um so yeah that that's good news could point out that the production numbers are like I mean it is a proof of concept and like it's kind of a new

architecture so we're not um so like there's a kind of long tale of optimizations that probably need to be taken [Music] so proof of concept numbers are I would say indicative but it's also like when when we're learning to work with a new architecture it's also like there's a kind of learning or adaptation curve you know yeah that's for sure and uh and I hope this uh more Advan optimization using uh the new OP codes for for for E for evm will uh will be applied by solidity team not by me but they are much more they have much more expertise

in this matter and I guess we're not reing the benefits of the relaxed stack validation yet right uh no those are pretty soon actually I implemented the non returning function already and uh uh and it probably will uh help a little with little bit uh with the stock validation also and an unnecessary hold duplication but uh not sure yet actually so I will think let you know till the end of this cool um kind of at the intersection of

testing and um compiler updates like is there I was wondering if there's any way to Jerry rig um execution test not just to cuz the output gas numbers right like those have to be correct but is there any way to like turn execution tests into benchmarks so we can see um how much time is spent in each op code or each stage for instance um yeah so hold that thought for just a tiny moment um we'll come to that with a testing updates and I think we'll be there quick because do we have any spec

updates anything should we let's go inside yeah there's this is a good idea five minutes ago there's a door right here yeah maybe we'll annoy this poor here but let us know when you're back and safe inside um I think we're safe okay I'm in spea this is the 7 topic right so spec updates think there are still some some items to finish out

right oh yeah so um return data load copy get resolution on that yes um we don't have any resolution on that no um that was one thing in it code uh validation I think it's not merged yet yeah but we agree we want it yeah yeah I would appreciate review on the implementation more confident okay um and there's one item that popped out uh which is this block hash instruction oh yeah and I I actually started writing

the spec update which which is like one word addition but uh I think I want to have PR requests for both eips and meas spec so at least we have can have discussions there the last change on thatp is like will contain same behavior okay and we'll just like FL flush the block hashes to the state and the next Fork after the prod we will enable block hop code that patches those States so it's like two steps yeah that's F Miss yes so that gets tricky if we ban it then solidity might need to

put in the address of the contract to do the reading and set that up okay so yeah let huh why is block has banned so we were having discussions about EIP 2935 um this is something being done for VE for veral because they need to have the block Witnesses of the hashes available in an easy format so we're putting them in a new contract like we did with the beacon chain routes um but then there are some design discussions going back and forth like do we expand block has to 8000k do we preload everything at the transition um and do we alter the gas so one of the things that came to mind is

if we're trying to motivate people to use the contract instead of the op code we could just ban the op code in evm one and make everyone use the contract um since you can get back in with the current setup the contract only gets the contract gets you a full days worth of hashes whereas the op code gets you like an hour or so um for what it's worth I think uh Banning the op code is not a problem from the compiler point of view because like um pre-compiled are really ACC easy to access from user code so it's like like the compiler doesn't even need to implement it anymore I mean user we need to like

maintain this because it might not be available on every chain and it might have different address on chain oh yes if if some like project really needs that it's it's doable definitely so you will still have that yeah for the but yeah but this portable option then like to use the block hash if you only need like previous block hat right so the the the instruction is more portable this because already it's already everywhere so I'm leaning towards don't ban it then what I'm leaning towards don't ban it I mean we can discuss it fully next week but I was thinking of l2s if they deploy eof but they don't deploy the beacon

contract the system contracts then we take away any block cash opportunity yeah yeah so I'm leaning towards not Banning but well these mosquitoes are crazy something maybe everything should be a pre-compile I actually think it should for anything going to um going to the the blockchain system but yeah um but we'll add that to the spec and we'll talk about it in online and more thoroughly next week and find finalize the decision hopefully um so testing updates you had mentioned um benchmarking some of these items right yes and that will also help with

um like any other time that we want to update um you know the pricing tables like it's always like oh do you have data for this and well if they were just like part of the execution test they would be be like free data right um so Bas at least can use the um State tests to uh get some benchmarking that's what I've been using to gauge performance um and any state test in theory could be used on any of the clients um if you can get the timing output you can get the uh overhead removed from the performance

yeah exactly like if there is some like standardized way we can output or even like doesn't even need to be that standard but like just like get total time spent in a test test without um you know setup overhead and then um test spent time spent in each op code would be really nice yeah um that's a big question to address but that was actually one something that was brought up not for the repricing the gas table um but just to make sure that we don't have any major performance regressions um in the meeting we had

yesterday I mean it's the same thing right it is it's the same Park yeah it's the same C like testing regressions is like the same thing same question as is gas priced accurately right so yeah I don't think we can we can go into instruction level because I don't think we have any infrastructure even for legacy VM for that no so I think what would be probably easier and more more meaningful is to have example of the same code um done for EF and for for legacy so maybe solidity can help here if we can I have some benchmarks that I are

generated by by solidity we can pick something else but yeah I think that's that's that's more doable than trying to Benchmark individual instructions because they actually change so much uh and I think don't edms actually don't really also exploit this e eof features so much right now at least we don't we don't we don't skip the the stack validation I believe at fun time yet because it's like there wasn't really point to doing so ah right so yeah definitely yeah and

that's actually a good point because like if UF code gets jitted or something then like you can't even measure time spending your a way to do it I think I did benchmarks before by just removing these these checks and then you can but it's like you get like five 10% U performance I can repeat that but um yeah it's kind of I mean I think it would be super useful to have that level of tracing but also if we have the time for each test like we can actually use like statistics to like yeah but the test the problem with the test is like we have test for Legacy and test for UF and they are different test so you don't compare the same workload you would need to have a test that is kind of doing the

same semantic for legacy for UF and I think we don't have it although we have some idea how to do it maybe but it's like probably kind of longer term yeah but what I'm saying is like you can actually use like statistics like if you have you know a set of tests like 10,000 tests and you know what op codes are in them and you know how long they take you can like use regression to find how how many instruction like what kind of instructions were executed and okay like you have one just 10,000 pushes and then the next one that's 10,000 um ads um yeah you just need to trace the PC Trace yeah because you could count how many of each are in there yeah I know

what you mean um yeah still not sure the actually the the the state test are good good thing for that but we can try because they mostly execute calls because that's the the biggest you have transaction overhead that will be an every transaction could the gasos estimator approach help the what the the one that we done with imop yeah I'm not sure because it's Al also like focus on single instruction and they doesn't change actually so much so oh you could compare I don't know jump to our jump and so forth yeah that would be a that

that would be a like a how do you say a valid comparison yeah I mean if you can reproduce like the full report out of it I think yes but um I'm sure how much work it is but yeah I mean we just need some industrialized way of doing benchmarking that's that's kind of like that's kind of like it but um but it's I don't know what's the status of then how how easy would be to reproduce this these calculations it's been a couple of years well I I think they're actually working on the like fourth report right now fourth oh like some like next I think

it's in progress I lost touch of of of of the what what's changing anymore I for I think they want to just add more more clients and I'm not sure what the scope is but we can check with I mean for this context even one client would probably be makes sense because you don't want to compare it across clients but you want to compare E versus Legacy it's kind of easier um yeah I think even if you can run like a client in Benchmark mode and um it just gives you timings for all the state tests I think that is already like

super useful because at the end of the day that's the timing that matters is what's run in production yeah okay and that also gives you like really like pav pointed out like I think it's pav but um I think like that's like the the the cases in the state test are like really weird but that's like also good because you want to Benchmark the kind of weird cases yeah on the subject of State tests um

we've been aggregating some of the execution spec tests together um so as mentioned last week the future of tests is execution spec tests not ethereum tests um so we've been writing them all in in um in Whatchamacallit in Python and so we've aggregated them here in this uh ipsilon execution spec test repository and includes all of the uh PR zit process because it takes a while for the testing team review it they have some very strong opinions about how their python should look um so this is just a an aggregate of what we have so far um where's Prague oh you need to change the branch

so the main branch is UF SL main got it I guess I can make it the main one on the repo what I I didn't notice this okay it's probably I don't know it's we could just change the default but here we've got the aggregate of the test we have this isn't all of them is it oh yes now it's under this by the way so I need to do some of my changes to get these in um so there's also going to be a layout change in the test directory but I think the more exciting part rather than having to run the python is we're also publishing wow how far back um filled um evm version here it's just the

artifact of the release the artifacts okay we're publishing the fixtures um that include all the state tests so you can just uh throw these into your client's implementation and make sure you know you can see where things differ um one thing that we're going to see out of this I know basu's got some differences in gas costs um that only show up when you run the fixtures and compare against other um other clients because we haven't ridden any of the gas cost tests to to gauge the gas um because there's no gas op code so the the one feature that we put in there is kind of biting us and testing but I think we can figure how to

make it work um we can just hardcode the values I mean we know how much it's going to cost total execution wise in the test we can just test it afterwards it's not going to be too variable okay any other questions about tests any updates going once going twice so other items um this is probably worth the discussion we want to talk about the breakout room uh so things are looking much more positive for getting um eof in Prague um we're we were in in a room with all the

other clients it wasn't unanimous but the majority of clients want to see eof much sooner rather than later um and if it it depends on some of the scheduling with Prague there's um a discussion is we had about what how big Prague is and depending upon what's in Electra or not um some people think that we could just cut what we have now and ship Prague and start scheduling test Nets in a month um if that's the case eof can't make Prague um and in that case there is support for um then doing another Fork after that that includes theof before veral um which was kind of contentious um but if if Prague goes longer and there's some

concern that the um the client the execu the consensus layer teams um aren't as ready to start scheduling test Nets next month um if we get more time then there's room for eof to come in so the Theo that the client teams are working on is just Implement eof and we'll figure out which work it goes in um when we decide to close the doors on Prague and uh whether it goes into Prague or into a fork after Prague um whether we you know call it Osaka and move stuff to Amsterdam or whether we create a new Fork name in between them is is the discussions we had but there's also the distinct possibility that Prague might ship um Q4 this year which means um we

totally got room to get eof in and uh there's some pretty broad support amongst all of the clients in the meeting um yeah and that's I think probably what we could say about the meeting for the most part there'll be more on the call next week um on the acde uh so that's you know tune in bring some popcorn and question is will it be more or less contentious than 7702 yeah actually notied that there new update to 7702 of recent fireworks which it's less compatible with E that we hope for yeah but yeah that's kind of outside of

thought that's not the issue I have but I'm willing to get run over it if I can just have my say on you know make my statement so when things go bad I can say I told you so but we'll see okay um yeah we're running this meeting kind of quick today um is there anything that do I have the chat window open nobody's been chatting me have they um gas cost estimator this one I we we talked about briefly right now with and toate uh yeah take okay take a look at this I'm

sure I'll be awake at 4 in the morning so I can look at that um any other things people want to discuss or should we give people 30 minutes of their life back going once going twice and uh you G have to figure out what to do with the next 30 minutes yourself thanks for coming everyone thanks for all the hard work we've been doing yeah thank you thank you all bye bye bye bye bye