eth_multicall Meeting, May 27, 2024

Transcript

I had a question clar in your experience when we get with gas say with gas same HH is the gas used set to be what do you mean is it set in the transactions so basically I got a hypothesis that gas used would be one of the biggest potential differences and I would nearly bet that when guess used and all the like transaction parameters related to guess are set then we would get same hash and when it is not set we would always get different hash could

you take a look at that and check it yeah I can take a look at it is it that because we because we decide on that algorith or how how the gas limit is calculated if it's not set then maybe there's some disagreement on how to implement it yeah I think it is possible I don't remember what was the logic but it was the logic was to take from the Block whatever amount is set by other transactions and whatever was the rest divide it equally between the transactions that were participating in the block and not have the guess used set in

them yeah is that implementing that I think yes but you have found an issue related to it somehow when it is set to zero and yes we haven't handled I think that as properly as expected but yeah we'll we'll take a look so let's begin uh from our side we had completed thanks to lucash the basic first iteration first pair of eyes look through the code there are a few issues to to be uh to be closed and uh basically yeah code is code is more or less good enough and from the trailer

issues we are still having some return codes issues and basic checks issues that will fix soon and yeah that's pretty much us I think the updated release will be deliv it to you guys today later today or tomorrow morning so yeah hopefully most of the issues will be fixed by then uh I would suggest to like prioritize the nonv validation mode for now so that we could like first complete the non validation mode issues be able to release it if needed and then focus on the differences in validation mode

on what do you think sounds good to me I don't think I have equal amount of tests for both since it's easier to test the non validation mode so I think we have more test for that I always have to set up everything and that's why I think many of the issues that we have in the non validation mode also would happen in the validation mode but then you also need to set more parameters so that's why I don't have that many tests for those MH by the way I wondered when we return blocks from the valid non non I think you most of the time would get more or

less complete parameters like of real of real usage and stuff like that maybe it can be helpful somehow because you get basically all the details of a block and all the details of the transactions filled in when you do non validation mode request yes that's I believe isn't that even speced out in this back you should return all the properties of the blockader yeah but it's a little bit like uh relaxed as you can have stuff set to zero and stuff like that right we still return the values that were actually used right yes yes of

course but the use of zero values are allowed in nonv validation mode and yes are forbidden in validation mode here yeah I guess I could do validation mode test in a way that I run them first in non validation mode and then I get those parameters and then I set them and explicitly and then run it again yeah and basically after these basic fixes we are planning to do some performance testing with kar's blocks that he sent to us and we'll get the like the estimate I think by the

beginning of the next week of how performant or how lucky in performance we are but currently I'm optimistic on the performance I think there will be no issues it's entirely in Ram and they were not serialized in any way any up from CA yeah um I think so I looked at some issues that were on trailer board I think that was last week um and yeah PR some fixes

um and that's that's about it really I gotten to do a lot more would you have time at some point to do those trell fixes if they would be done then it would be again easier to compare NE and get to each other because some of those issues are occurring in most of the transaction so it makes it harder to compare are there new issues or which ones are you talking about oh just the ones in the trailer because I did fix some of the trailer issues so there's more that needs to be fixed I guess oh is is there

new version maybe I haven't tried it yeah it seemed like C delivered like in the middle of the last week or like yeah I wrote in the chat that's that's the latest I actually just the eoa one is still there one oh yeah the EA one yeah that's a bit ugly but yeah what we're on the latest version of both guest and nethermind is that correct Kori I haven't checked it up those those notes probably not on the latest

get we should make sure we get that then and see what's left well I'm running them in the test Sho so I'm not running against that note so that's why it's just what the inter is using do c still disagree with the a I think that's something that we have been discussing for a long time no no I mean yeah we decided on it and I don't want no I'm not challenging it just I haven't been very motivated to look at that yeah basically what I need to do is add a second flag

uh because right so because we have the because right now there's either like validation mode or nonv validation mode and validation mode is what we use on Main net like production and now here we need like a validation mode that is has that has this exception right mostly validation mode mostly validation exactly it has that and then it doesn't check the transactions that if they had the private key correct or they signed correctly but I don't know if that's handled somewhere else in the code so yeah it's done together with the

other check so like non check and so on and um in the same place but yeah I just need to like kind of separate that with the second flag yeah have else um um just uh I did have a question I how many of the blockhead fields are we returning I guess I guess what I'm wondering is like we are returning a lot of the blockhead Fields why not just return the whole thing I don't know I don't remember if we discussed this

before or like what was the what was the conclusion there what Fields don't we return because I think we like everything I also thought that we return the whole block header but maybe there's something missing no definitely not the the whole block header I'm pretty sure about that one maybe it's UND thep um wait let me let me try to load it sec but yeah I think it would be useful to

return the whole block header but I think is there some transaction that or some fields that are not really useful but I would imagine all of them would be useful somewhat I mean there's going be some that are not useful just because we all we know exactly what they always are like the uncles for example is always going to be empty though that's true on mainnet as well but mainnet still has a block at um the there is a difficulty yeah the yeah transactions route receipt route there's there's a few fields that we not really returning and I mean I get the idea that basically we don't need them so why return them but I just want to also put

an argument for like having the exact same structure as the header uh because like it's easier for the parsers yeah I support support that for the same reason that's angle has and do we resent the PLO filter I think that's one which one blo filter are returning that oh yeah uh also not that's also not being returned and I think this might actually help with the block hash debugging as well it would neate the a um Small

Change like in that case I think we would or we should move the call results somehow out of it or I don't know how to structure it so that like the header fields are like one object and um and then the other results are like kind of separate I think most pars so I'm go out the limb and assume that most parsers are um agnostic to excess Fields like if there's extra fields that they don't recognize they're just going to parse it anyway so you take the payload give it to your Blockhead or parser and it will parse it even though it's got

this extra transactions results field and then you can give that same payload to your transaction result processor which will ignore all the blocker fields oh yeah yeah then um yeah I guess this assuming we don't have any collisions which I don't think we do no we I think I believe the extra field that we have is uh calls which would entail the call results as well as um as well as logs and things like this are any of those which I'm missing and

useful yeah the difference will be at least on our side quite quite the Bigg thing oh yeah well but I mean you are creating the header already right to compute the hash so this would in theory this should remove a step from to convert the header that we get to but I wonder if we have same like same amount of stuff in Block header and in between get and nethermind as this isn't publicly shared yeah just looking at that code you have a bunch of things in there that um you don't actually that are optional and you don't need um for

example the aaep is only applies to certain types of blocks and won't apply to main net ethereum and similarly the uh I a couple of those in there that just like there extra things that aren't actually part part of the block header they're just extra things that nethermind includes and because they're optional you can just ignore them and leave them out I think the only ones that we need to add that are non-optional are is a pretty small set yeah but basically probably we still will have like separate structure for it because this may may get different I mean I assume structure for the main net header types right it should be exactly the same as every other client

yeah you should be able to just use this data structure and there will be things that are set to null in your code as well which the code already handles yeah but the question would be about serialization if it is done same way for RPC and non RPC but yeah we will figure that out so if you would add more Fields probably just yeah add them we'll add them too Sor was off a bit What fields do we need to add so gu wants to have same block headers as are puted onto the actual blockchain so there are potentially

extra fields that are currently not in the spec so probably we'll need to either add some or just return the same block header as we put onto the chain they use API API compatibility so if someone has a library that can process block headers they can use that same library to process the return return result from an e simulate H it's not a strong opinion from my side it's just that an observation that basically we are adding already most of we're returning most of the fields then why not just go the extra mile and return the exact object that gets uh get

blocked by hash or get blocked by number yeah yeah yeah we can do that should be easy yep what are we missing is the PLO is one this is extra mixed dig digest one other those the state route the receipts route I think we're not recurring those right now right the bloom filter that maybe up I know are we trainning the uncles I might have left Uncle out so turn that even though it's Uncle hash is bought yeah Uncle hash coinbase St rout

transaction route receipts rout logs Bloom difficulty uh there's also nons and mix digest extra why why do those all still exist they went away with proof of St yeah but we we didn't remove them they just the chain is still the same they just have fixed values in them now yes I think mix has is actually used for random R down yeah so I mean I guess it's a question that whether we should because here the the naming of it is mix hash like in the

Json encoding um and now we like in our structure we return prev Rondo so this is actually kind of convincing me a little bit that maybe we should take this opportunity to return a more correct block header than main net returns because main net is apparently still returning block headers that don't make sense anymore like half those fields are deprecated basically or re reused and renamed [Music] yeah well no not half of them like a handful uncles all the uncle

stuff the uncle rout the difficulty difficulty stuff the kns um yeah but they still make sense for the old blocks that's true I guess you can do e simulate against an old block huh yeah MH yeah okay that's fair I'm convinced back to make m is this the whole list that we are missing yes I guess for the receipt has we are not including the eum transactions there the receipt

hash in re that include the logs that includes the logs yes and we are already Computing the receipts to compute the block hash so we could just return that yeah well I can add those this spe do we all agree on all how those are calculated I mean if we don't agree it will show up in the blackash difference so I think this will be this will be Insight Yeah the more we know the

better yeah um so I just want to make sure that we agree on the naming like of mix has tough because I think everything else we use the same name I have to double check is when gu Returns the mix they return two Fields one called pranda one called mixh and they just have the same value or do you return only one of those I believe we only returned uh miash and is that including if someone fetches a block from before Pro mistake

Fork uh yeah I think sorry you said you only return mixh you do not return prando exactly I see okay so you're saying internally you have it as PR now but in terms of how you communicate with the outside world it's miash got it uh yeah I double check we return mix has non is zero uh but yeah mix has the Rondo volume okay well this annoys me greatly but because I hate technical that but I think having a match makes sense so is mix has the same mix digest yeah the Json encoding is mixed

has yeah I have a question do we currently have the block header like data structure in the RPC repo we are working with or will we still have our own like simulate block result as a type that we are defining um the question was if the header is defined already in the specs yep I believe it should be yeah so we can reuse that that's a good point yeah that would be great and we have it defined as a class in code I don't think

we can make a union type that if you had that header plastic calls I don't know if you can make that kind of type that's the point to make it not special to make it as usual as it is used in like all the other parts of ethereum because otherwise if you have a union it would be not not really the same as block header that is used by other parts then we have to separate the block header from the on the calls I guess and modify the structure yeah probably we shall right so here is the let me post a

link yep so it has a block object and the header fields are required and then there's some additional properties that can be empty left out empty and I think those include then the transactions and withdrawals and so on that we can just leave empty oh oh oh I see why you mean by Union because we need to add the call results yeah yeah um I I I think it should be possible there's two options one you can

just run your payload through two different decoders assuming your language allows that easily enough um might not depending on how you architected things or two if your parser can output a object that says it's one of these two things um typ script for example does that really well but I don't think C has a good way of doing that at least last I check my question was more about the yaml I don't think yam has unions oh you're talking specially about uh yeah so um open RPC does and I believe it's even used in the

Json RPC spec already for ethereum for transaction types yeah exactly so we have the sign and sign transaction types and the sign has some extra Fields okay yeah but do we want to make that there's a header object and then we add calls to it or do we want to make header object separately from the calls I think that's question I mean I lean towards I'm separating them does have the advantage of allowing you to kind of parse them separately um but I would be hesitant just because we're so far along i' would be hesitant to change our data structure

at this point unless someone has a strong opinion if we are making it use the block header from the basic RPC spec I would suggest to like separate them from the lists of call results that are relevant to them so that you would get like a list of headers and a list of lists of call results this would basically allow the guys who have the passes for the headers to use them and to pass the simulat C result as as it is used currently what do we do with the

transactions is the block object includes the transaction but we also include the nonsol transactions there and then in the call results then we are having that same array again block header the block header won't have the transactions would it the header wouldn't be but I was wondering if we want to return the blog object that has complete block yeah like she does said the block but not the block header yeah I yeah my opinion only return the header not the body the main reasoning for adding the

body can be for example past the generation of Genesis blocks yeah can there be any other reasons to return the complete block body I guess not so keep them as one object and in terms of calls we can easily add them either with a the type or a custom formatter not a big deal you're say It's relatively easy in nether M to have them kind of intermingled and be able to split them

out yep yeah I guess the question is which one is easier for like ether G or VM for people who are impl the other side of it so in typescript and JavaScript it should be relatively easy I mean I can't say that for sure depending on what libraries are using but in general typescript is quite good at handling unions and also handling extraneous data

so you can just run the same thing through two parsers if you I I have a slight preference to separating them just out of a purist point of view but um fine with either and hello uh new participants I'm not sure how to pronounce d yes uh hi CA thanks for um inviting and all the others so I joined the uh to to contribute to R client the implementation of e simulate and the reference uh let's say document and

implementation that is done for multiple I wanted to understand the the implementation of traits first of all and then eventually uh the uh the implementation of functions so yeah I might had some specific queries to for for for me to ask so the one of the first one is that um let's say in Rust implementation uh do we have to uh have the same let's say trait traits to implement as it's implemented already for E call or can we do uh significant changes let's say for implementation of State rout or the calls

hello uh so do I send your corre question correctly that you're wondering um if the you can start with kind of e call as the basis for Imp in this is that what you're asking yes so um for inance um during the discussion uh in the same PR um one of the another good colleague uh shared a reference let's say tra implementation let's say you have a root where I can share the link precisely for um for for the reference in the chat so um here let's say we have the structures uh root with corresponding

RPC endpoint the method that we want to call the parameters and then subsequently you have state call the the execution function calls with State overrides and other details so my query was is this let's say schema that is defined is it okay for me to start the implementation and if yes uh should we I implement this in the like should I create another let's say file or should I implement it at the same place I think I understand um this this call is for discussing the specification I think you're looking for

the people who managed the rest impation REO that does that sound right I was not able to hear sorry M what was that I was not able to hear your sentence could you sorry are you looking for the people who um Implement W itself like the Paradigm XYZ people um yeah is that direct that yeah so it was just regarding also um wanted to get feedback also if let's say implementation is correct based upon specification um or I see so yeah so we can help you with checking to see if the

implementation matches the specification um this group is made up of people from gu nethermind and uh an app a wallet team and so we can't help you on anything specific to W or rust um but for for questions about um implementation whatnot more or less the it's probably going to look a lot in terms of implementation it'll probably look a lot like eth call um which re already has and so you're basically going to do more or less what eth call does except for with the caveat that the use the multiple transactions can run in a row more or less and also you may have to generate um do some additional work to generate blocks um when those come in okay and um maybe it's the question

next question sounds to be little na do we have have to let's say create um a separate buffer because um for for generated execution traces uh because they are not being committed now so for people to understand like let's say we execute the the transactions parall we will have to maintain a cash right in order to store the intermediate results uh like how does um what are the let's say various steps if if I have to um create a you know timeline for the implementation what are the major points that I have to check out in the client agnostic to whether it's implemented in

G or in r that probably be a good question for Ceno or Oleg like kind of for you guys what did what were the pain points and what were the kind of steps you took to implement this from the ground up Pro tips for new implementer yeah uh so I would suggest in um starting with a fresh implementation I don't think it's really possible to directly use um the whole e call code directly I mean I have abstracted some of the logic of executing a single call uh so that it can be used both in E

call and E multi call but for the most part like is simulate is more complicated it has more logic to do um so basically the like roughly what the steps that need to be taken are that um like you need to Bas on the input like you have to sanitize like the the block information that you get and and kind of prepare some block headers like um the the main block header types uh fill out some

Fields um and you you have to apply the the state overrides um for for each block and and you just like process each block separately and then fill in the fill in the the headers that header Fields some header Fields after the execution um I I don't know it's u i mean maybe easier if I just share the code with you or yeah so if I can summarize in in one sentence in eth call you execute one transaction in E simulate you execute multiple blocks each can containing multiple transactions so it's more like

executing part of a blockchain and not just single transaction yeah basically just to add on top of it you need to know how to create a block and how it is published through to the chain this is the first thing and the second thing is you need to look at is call Api to know how the RPC calls are basically structured so that you'll be able to for example connect these two and after you get these two parts you need to see how it impacts performance and how you can like SP on this in parallel so it that it won't impact your main chain block

production so I think these are the three crucial parts to tr all um thanks uh indeed Lucas and M yeah um because it was kind of very complex I mean on the side of tra implementation it was perfect but later on there were lots of let's say questions that arised um specifically regarding okay do parallel transaction how it will affect like do we have to simulate also the the potential contribution of those different uh you know simulated transactions on the actual State and other steps but uh I will try to um work on the feedback uh if sa if you have

relevant implementation from your side you can share the link that will be great and I will reach out to you on the on the pr with the um you know refering handle thanks yeah so as for as for States basically every transaction that is processed um it will modify the state and you do keep that modified state to process the next transactions and blocks though this is not persisted to disk that's the that's the only difference right but we do need to like kind of progress the state as we go along because the next transactions depend on the state of the previous

ones so yeah same I okay let's I'm happy to share um my branch and my dis like Discord handle in case you have questions perfect thanks we can also invite you to the telegram group can ask from all of us question from there sorry Kary uh it was um I was not able to

hear and we also have telegram group that we can invite you to okay that will be put your put your telegram handle in um the chat here and someone can add you thanks gar and everyone that's it from my side so back to the previous thing with the blog header do we agree on that we had separate block header instead of merging them all together like what m s suck that's blog header field and then that's the calls

field okay let's do that then I'm fine with using just header um instead of block people for that that's what Seena head above I don't know which which one is better block header is bit more clear on what it is and don't remember what we're calling those calls calls I believe so yeah now we have the block St

calls the is yeah the input is called something more else but return value is c yeah J can we fit everything else in the so that there just calls and header I guess yeah I think so yeah I got modified that uh something I just noticed we do have in our um we have a withdrawals object we have a withdrawals list in in the current spec but that's part of the body that's not

part of the header although like I'm checking my code and this is not being used at all it's kind of empty yeah I think by returning empty withdrawals yeah NE never never mind when you get blocked by hash or sorry get blocked by number get blocked by hash NE mind returns withdraws does GU not yes but they are part of the body right they're not part of the header oh yeah so I guess what we're specifying we should probably be clear about this what we're talking about is the shape of the thing returned by get block by hash which does not match the header itself exactly such as you know the transaction bodies and withdrawals which are both part of bodies but they would get

returned by that um right yeah yeah I think we should match to what the get Block matches and not not to the header itself um since currently there's no RPC method of getting the header anyway there is get Header by hash or get Header by number really yeah I never it I guess because get that's get Block header yeah I don't think no yeah NE nethermind doesn't support the get Header by

number oh no oh I think it might be a a new guess thing because I've never heard of it before either no it's not new it's been there for a while but maybe it's not standardized H that's strange it's because everyone is using get Block yeah yeah yeah I yes I mean like the difference is that the the block has two or three extra Fields like transactions the withdrawals and one more yeah this is this is making me lean

more towards I keep I know I keep flip flopping on this um this is making me lean more towards not changing it because we already have basically get blocked by hash which returns a header plus some additional Fields already and all we're saying is we'll return one additional field which is the uh call results so just to be just so I understand you're suggesting we have the header and then we add the calls to that that's exactly what we have right right more or less well it's not the

exact Fields as I had to right like yeah so sorry the I wish there's another name for it the thing that e get blocked by number returns that thing is a block most of the fields from the Block header plus some things from the Block body plus I think some things that are just derived from the two and so um I'm proposing a simulate as it is now should return basically the same structure but along with some other things so we got the the bter plus transactions plus withdrawals plus one more thing which is the call call

field and that's basically what we have right now except we just need to add a few more fields to flush it out I see yeah yeah that's that's okay so essentially our our return return result would looking the shape of the inner the array items in our result be the same shape as e get block by number or get blocked by hash plus it would have one additional field in each block which is the calls which is an array and it would sit right next to um the receipts and the transaction sorry right next to the transactions field which is also added um part of the body oh and uh and the

transactions list is the hashes of the transactions right we don't actually accept the true version uh ah yeah well just which one do we want to do I feel like since the user gave us all the transactions um are we well are we filling in any of the transaction fields or do we I think we do right yeah we filled them yeah it does feel like maybe we should return the transactions so we can let the user know what defaults we used and so that would be the true version of get blocked by hash so we can would include the

transaction list but after filling with defaults right uh only I think the the signature will be probably left and but yeah everything else we can return should we return all the transactions then as well a new object so I was wondering where is this in the ethereum execution API is repo in the standard

one or is it just in gu code right this the link that I shared before but let me let me find it again so this is basically the block object the block object exactly and it has the transactions field and now the question is whether we

return um only the hashes of those transactions or the full body so I mean just one concern is that the header size is relatively fixed um like headers are not that big but transaction bodies they because of the call data they can get rather big you're worried about the just the RPC load and burden yeah that's the only thing I'm worried about but otherwise yeah I mean I guess presumably that's why the clients currently have a Boolean flag for do you really want these but why would we return transaction bodies they are coming from

the call itself right it would so the user can see what defaults got fil they got their transactions got filled with um I don't know if it's worth it given the size that c brings up but that would be the Advantage for the cases when the user fills in like only one or two details which may be really helpful for him to then be able to call with validate mode on yeah the how about this for simulate call V1 let's just do the hashes and then maybe in a later version we can add a Boolean for um the accepts or we can it's

forward compatible so in theory we could add the buan flag later that allows the user to specify and I also want the transaction bodies that you filled in we can have it right now because it's the same as get Block hashes right get by yeah so this is exactly exactly the same as SK block by oh you're saying add the Boolean flag right now yeah it's trival if it's easy for both gu and Ne mind I'm fine with adding the Boolean flag now I was trying to avoid changing the API too much but we keep changing it all the time yeah I'm fine if it's easy enough and everybody agrees which looks like it

from the thumbs up okay so let's add a new field we need a name for it um maybe is it the additional properties thinking get it's called hydrates well so it's gway number has uh parameters in a tupal not an object so there's no actual name for it on the RPC whereas ours have names for all the properties in our in E simulate V1 so we need a name for it that's agreed on so G internal calls it hydrate um I don't know if that's very clear to end users though no this was just in the specs but yeah I I agree like for good bl by number it's kind of clear but in the

case of is simulate we have maybe just return transactions called return full transaction objects if you will say that it is false then users can expect yeah return full transactions are better I like it much more because otherwise you expect that there will be no transactions at all return to yeah that's fair other option would be the inverse like return transaction hashes only and it would default to

true the second one was shorter it is shorter I agree I'm okay with any of those can me return transactions return transactions is a little bit big I think like or getting that because we we always return list of transactions it's just whether we return just the hashes or do we turn the the whole thing I'm on board with almost any of these names not going to hold up anything for it next return full transactions that be

good that's my my favorite of them right now anyone want to vote against return full transactions going once going twice okay let's do return full transactions uh do we want this to sit next so this will be like um I think we have a spot for these sort of parameters right like these Global parameters so validate mode I guess is one of another one so it would sit next

to that and if true then we return the filled in transactions this again same transactions user gave us we just filled them in with the defaults and we return them and if they don't then we just give them back the list of hashes in the transactions field just like get blocked by hash and default to false I guess yes same as get by hash okay um Kor you can update the spec with these things we've discussed right yeah we just need to agree with

the is it the header or block or do we make it a union instead that's I think we were leaning towards maybe this is just in my head but I think we were leaning towards not breaking it AP Parts into multiple things and instead basically the thing we return is the same as get block by hash plus one extra field for each of the blocks which is the Call's field as for how we express that in the spec I'm okay with just saying that it's a union of the whatever it is the object that get blocked by hash returns Plus um calls property or if you want to respecify it

that's also fun and then with the mix digest is something that do we just call it prand now even though it's coming from anywhere in the history I think we should call it mixed digest because that is what the get block by hash calls it but we have the prand out to would be isn't that the same so they they are in the same spot in the block header and depending on whether you ask for a block before the merge or after the merge mix has will either contain the mix has Prem merge or it'll contain the prevend if it's a post merge block and so the field name is called miash no matter what it's just a question of

what's in there and if it's a post merge block it'll be the prevend out value but at the moment we are returning the prev out so I guess we would remove that yeah we should we should fix that to be in alignment with the get block by hash I think we argued about the prev for a long time and now we going back with the mix yeah okay so are we all in agreement then on everything hopefully these are very very minor

changes for the various clients not going to upset anything Famous Last Words yeah I think there will be more work from this yeah I I think it should be fine we can I can probably try to push something by tomorrow or Wednesday okay okay anything else I hope we can freeze the spec someday someday we're getting close like the clients like every week yeah every week we're getting close let's fix the spec next

week I can fix it this week already the modifi the spec to this okay I got another meeting I gota get to I will talk to you guys next week yeah see you all thank you see you bye bye