eth_multicall Meeting, Aug 19, 2024

Transcript

all right so this was a fun week for us we released 128 with basic simulate in it uh we are currently missing some of the heshes in my understanding and we are missing the interesting issue that c found on when contract is terminated need end it sense guess or is to the to somebody we have not checked that yet so we are currently on it also we are on the food transactions to make them show uh show in the more complete View and also C time finalizing

the simulate Primitives into the E call and see of three other calls related to it so that's it with us I know that there was some nice discussion uh this week on the chat maybe we'll get to it after the uh the updates yeah hello uh I can go so um I was um let's see we started the week with um some CI problems so I was looking into that um and generally making sure that the

the test uh effort is is going forward and um I think there was also we found uh yeah one issue was the default time stamp in gu that I fixed um and yeah one of my teammates has reviewed the pr uh left extensive comments so I just half an hour ago finished addressing all of the comments and one of them turned out to be kind of interesting and this is what we discussed on Friday um

and maybe I will just talk about it shortly for Alex benefit yeah yeah please do so that everybody who watches on demand would also know what's happening yeah cool so basically what happened was so we have the state override for every block and when overwriting an account you can also specify if it's a precompile address then you can specify that this pre-compile should be moved to another address uh now what we had an account for in the GI implementation was um that because the state overr

right is a associative array like it's a dictionary so the keys are not ordered and there can be some edge cases where like you specify let's say you say that account one which is the first pre-compile should be moved to account two and then you later on say Okay account two should be moved to account three um now depending on the order of how uh like depending on which order gu sees this overrides and tries to execute them you will get different results in one and there will be an error and in in the other one the promos would actually

move in in the correct order so after some discussion basically it came down to adding some restriction to the move two move precompile two feature and that that is that um basically if you have moved a precompile that is that makes the destination address dirty in the sense that it cannot be that address cannot be overridden anymore uh and that should fix the error and I think it's it sorry I see

somebody's hand raised up yeah yep so my question would be say you want to do a basic swap you want to move one to two and two to one so how do we go about that not allowed you get an error the idea being that there's not really any good reason to do a swap of a pre compile doesn't really make sense and it adds complexity to the implementation to support it so it's easier to just say swapping not allowed does that seem reasonable to everybody I think n might allows the overriding and moving all the contracts and I guess there it might be more useful to use

case but not for pre- compile but the spec is only for pre- compile as well for even for overriding contracts I feel like a swap is unnecessary you can always just move two to five and three to seven and just or you know yeah I can't I strugg to think of an example where you'd want to do a swap like you can just put new code on the Chain using an State override yep yeah with contracts you can replace the code better but yeah with pre- compiles you cannot do that right but like why would you want to replace like a math one math function with another math function like that doesn't

make any sense or like you know ecdsa replace it with uh I don't know Groth 16 or whatever people using these days it's just it's a kind of a nonsensical operation well basically it can be a test of stability of implementation basically you have implementation a you had implementation B you compare how they are going around the tests so but yeah that is like stretched out case yeah I does anyone want to defend that use case strongly again the worry here is that add adds significant complexity to guess implementation at least and I would assume similarly in nether M

because Json objects are not ordered and so you guys probably have the same kind of fundamental problem where you have to like turn it into an ordered list first or something yeah okay so let's just all right so basic idea is that we flag addresses at dirty as soon as they used and we allow them to be used only once here yeah yeah you can move from you can move from a clean address to any other address any address you move something to is considered dirty any address you move something from is considered dirty after the move the address I'm moving to shall be clean or she'll be dirty or can be

dirty it'll become Dirty by moving to it can it be dirty when I'm moving into it no also not Yeah so basically you can touch each address one time in the entire SE yes Lucas please what's the problem is that when we have multiple overrides on same addresses hang yeah multiple overrides that um so you override to by saying I want to move the content of precompile at address two to address three and then later on you say Okay I want to move the content of address three to address 4 depending on the order you execute those two operations you get a

yeah yeah I get it I get it the objects unordered uh to be honest I would just say that if you do this there's a like the order is not guaranteed and that's it because I don't see any like really use case to do this yeah agreed and so the the question is is so we can we can say order is not guaranteed but like in the case of gu it's a difference of one of the operations will succeed the other one will give you an error and fail and so they'd have to write extra code to make the second one actually work at all one of those orders just doesn't work in the way they've implemented it and so or the second thing is just order it by address before we apply all the

overwrites yeah that would be the other option actually would that solve your problem Cena or would you still run into problems it's uh yeah that would also solve that's the alternative I I think I slightly prefer the Restriction to have just yeah to allow ones uh and others to be touched uh but I'm also easily convinced to go for the ordering like like that we Define one order and go with it I do find that okay there's um not a real I mean I couldn't think of any use case

for wanting to do any of these things so the error should not matter to the user and and the the ordering yeah I just don't think that it should be order dependent at all you know I tend to agree and if we did decide to go order dependent though we would have to Define what the expected behavior is if you tried if you did one to two and then um 3 to one let's say what is that behavior do we assume that or let's say you go maybe better example let's say you go and that works yeah so let's say you know you sort them by address and you move whatever is at

one to address two and then you say take whatever is at address two and move it to one does that take what was it addressed to before the override or was it addressed to after the override or does it result in an error it is linear so it will take whatever is in two and place it into one so it's like a simple instruction set that is ordered by the number of the uh what we are moving and yeah it's currently what is implemented at net mine because we pass whatever is given to us by user into an ordered dictionary basically that is sorted by uh the key so it is like it is exactly what we are currently like what

is the option b so uh just to confirm then you're not able to perform the swap also because if you move one to two that means that there's one one has overridden two so when you move two to one then already the first pre compile is being moved to one so both yeah two is going to be cleared there will be nothing in it well there will be the evm code that you're supposed to like also put in the override um and one will have the same pre compile as before it seems to me that like there is a simple solution of doing just a swap instead of just moving so if you swap then there is always

something somewhere and everywhere okay we can also change the algorithm to firstly read all the things that will be moved store it into some temporary uh thing and then just move it to the thing it will be moved too so if the swap would just work then yeah that mean my instinct is that's a lot of complexity for a use case that none of us can think of a legitimate use for so my question to Cena would be if you always execute the swap instead of just move would it solve the issue um I'm not sure I understand what do you mean we change the move operation

to a swap operation yeah we don't want to do that probably you basically have to go through and you uh do two passes first pass where you look at all the things that are being moved from you save what's currently there up in in some temporary location and then you do a second pass where you look okay what is going into each of the locations and then you can then pull from the original code it's it's more complex I would argue against it because we can't it's not for a good reason okay I'm actually like fine with it it's a very simple solution and with the swapping you think it's it simple yeah yeah yeah it's not that hard you just introduced one like

intermediate immediate collection too so we would need to add the swapping and the ordering right because order doesn't matter or doesn't yeah I think it does so if you go one to two and then two to one that two to one is is actually taking the original two not the overridden two yes so order doesn't matter because we always will the take the original or very as original as possible is it a Doos Vector no yeah but you also have to consider that like when we do the one to two like one can be overridden with evm contract and two can be also overridden with evm contract

there's also the semantic of whether something is a precompile at the end of this over right procedure or not that we also have to determine so if you first do the one to two move that means that one is not a pre-compile anymore and two is now a pre-compile and and so like if you do it in the other order when you move two to one first then um two is not a pre-compile anymore and one is so I think yeah this something that we also so I think what we need to do is to take Swap and ordering into account both and then this would work I don't follow right if you move one to two and two to one if you

firstly take the original values of one and two and store it and then apply the move from the from this and not the like temporary swaps then I I don't get it why why is it a problem so the the the example that okay maybe it doesn't demonstrate in this example but there's a if you move one to two and then you want to move two to three mhm and so that means that uh two has been overridden yeah yeah yeah but you're not I think you're not using the algorithm I told about so firstly store all the

froms so if you move one to two and two to three uh then you store one and two temporarily uh at your what you're moving and then you do the pass so if you move one to two you move the original one and then you move two to three you move the original two not the overridden two yeah basically you store everything that is addressed in a bu in a buff and then you move copies yeah kind of M so you move from like the you start the original values of everything you will be moving and and then you use those for moving and not the things that already

over just look should happen when I do one to two and two to three and I'm not saying original one orig so at the three we have what you had uh at two before any move three should have orig two original two so the the second precompile yeah and what about two two should have the uh what was originally at one but it's also being overridden with evm code oh I see the the issue here is that when you do a move you always you leave something behind with the move I

forgot about that so it's not just you move two to three you're moving two to three and then you're putting that same operation is dropping something in two like you're moving three two out of the way so you can replace two with some evm code that you supply these are supplied and so if if you moved one to two that means that we want two to contain whatever used to be in one but then the US user also said move two to three and leave this in address two and so we have two conflicting things that both want to be in address two the thing that used to be at one and the thing the US the evm code the user supplied in the two to three overid okay let's just throw an error then okay yeah I totally forgot that we have that

evm code that gets dropped Left Behind it's not just a simple move okay so if the if I understand correctly the consensus is you can touch a given address one time in the overrides either as a source or as destination doesn't matter which but only one time if you touch any address twice we just throw an error and we say bad parameters or whatever so if you do 1 to two and 2 to three three you're touching the two address twice so it's an error if you do 1 to two and 3 to two also an error because you're touching the address two twice that line with everybody yeah all

right okay I think one big project still ahead is the getting the hashes to match between NE mod and get yeah by the way could you send to me in DM or something the spec for the full transactions that we shall have are the full transaction flag yeah to basically to what shall we shall be changed in the returns return stuff yeah I think it's the spec but I can share it yeah please yeah I guess getting those transaction hases to M matches the first thing we should do and then see if those

fixes again the issue and if not then we need to look more carefully what why they're differing I don't think I have I don't have any test cases where the hashes would be the same between the clients do our current like hashes match if you send blocks without transactions oh that's a good question I don't I don't think it matches just blank blocks here blank blocks okay's we can find out they don't match they has this difference okay we should probably troot

that one first maybe I'll solve all of them yeah so full transactions won't help you a bit as long as the the basic blocks are different well it's about all the values are going to default and then it's about if the default values are the same do you guys do both clients have an easy way of just exporting what the object your hashing is perhaps after rlp encoding or before either one like the actual thing that you're encoding to Hash you like an easy way to just export that for one test I don't really think so but we can do more logs yeah log it out something just so we just get the

the rlp object or the pre rlp version of that same object either one would be fine I think we just compare and they should be obvious where the difference is at that point yeah so what we can do is maybe temporarily add the full object um as a field in the API and then remove it or before the pr is merged it's a dirty hack but so my question would be what are the fields that we are currently not printing out about the block we know basically that the issue is inside the block difference so there is a a lot of information that we print

about every block just like in vanilla simulate so the question would be What fields don't we print out and if every other field that we print out is the same uh the size is different the size and hases are different but yeah with the the size is also hard to say why it's different and this is the only difference yeah fun the only difference in the blocks here mhm quite fun I would say just if each client can just simply do an um log out just the rlp encoded block

you can then just diff the two and you'll see where the difference at I don't we need API for it just log it from your client I'm wondering if the the size refers to the full block or the just the Heather the he is what's hashed right body's not hashed so if so if the size Matters then it's going to be the header size and there's only a few fields that can vary by in size in header do we print out metadata we do right what do you mean

by the special field that every client can set with basically text like this block oh extra data yeah yeah yeah that's not returned but is the default for those different uh it may be production def pleas because assume it's empty no we would have a default like build by NE mind or something like that I think I'm not setting any value

for that so if if NE one has a default that could explain yeah let's uh should we just Define that as empty white array yeah basically the question is are there any other like such fields that we may be Miss certainly seems like the most likely one yeah and it would affect the size probably there's the block author Fields I think there's still a block off the field that may be something that you guys did default

differently one of those fields that will break a hash but isn't going likely to be overridden or set by anybody explicitly yeah we'll check the defaults that we have uh maybe if so there is a nice page that is uh the markdown page that we have on E simulate Maybe we could like uh check if it has complete defaults so that we'll be sure that we have something to compare to yeah it's a good idea we we can probably take care of that K you think just basically make sure that our the specs page or the markdown associated with it has a full block header including all of the

defaults for every field that has defaults should have all about yeah I didn't know about that exra data that that's probably not there we should I'll give link on the official docs for the block header we should make sure everything's in there okay but like that extra data is not something that when you're getting blocked and I don't think it's a returned uh it may not be returned by get Block but it is part of the header and the thing we care about here is the actual header itself but before we have been caring what the get block by number is returning and it's not there I think I don't know

why oh there extra in the and we are not returning it no we have we have return exat so it's there it's and I think both are returning it's empty m yeah we both empty so they differ by size but by nothing visible yeah and the size itself is not part of

the hash so but it's a clue maybe we can check why Pi under the hood yeah but something that is hash then it if it size varies then it also changes the hash that limits it to um the integer fields and the bytes field everything else is fixed size so in that link I just sent the U ins the U 256 and the btes that's extra data time stamp gas limit gas used number and difficulty and blob

an extra block gas one explanation can also be that that the client is lying and it's returning something else then it's actually hashing so that something is hashed but it's not return return that's same but I don't know I don't know the client's CL if that's possible hopefully not because that has been the case before that some gas used values were differing in what was returned what what what what when you sum out all the gas used together it didn't match up to the block gas used n might used to have that kind of bug so I wonder if that kind of issue could be

there would it be difficult that you would get all the feeds that the multic call is returning and then you would calculate outside what the hash of that box should be do we have all the information for that maybe this chor can we so we have G and nethermind disagreeing can you this week um generate a block and hash it using a third algorithm like a JavaScript thing or something and then see we have all the DAT well so if we don't then we should that that'll this exercise will answer that question so you try to construct a block header following that link I just

gave so make sure you fill it in rlp encode it and then hash it and see if you match nether mind or see if you match gu or see if you match neither or see if you don't have enough information to actually fill it in and can we by the way like feel as much as possible of the same Block in a plane like e simulate call so basically to have an e simulate call that fills in like explicitly sets everything to be whatever we want or we are limited in that regards that's a good idea we should have so there there's two two test cases that are I think are good for debugging this one is a test case of a block with no transactions no overrides

nothing right and then another one which is a test case where you have a block that overrides literally everything two very explicit values and so we might see agreement on the lad but not on the former for example so those are be both very good test test cases but in that second one don't include transactions or override still just fill in fields yeah just play an empty block and I don't think we can override everything so there can be still something that we cannot override and then net with net might there's a bug that you cannot override the time stamp for example yeah we'll fix we we should override as much as we can and then compare those and see if we because that'll help us narrow down

which Fields might be the difference since it seems like it's going to be challenged to get rlp encoded blocks from the clients yeah it would be nicer to get what is been hased and then get that printed out and then you'll know right away yeah we would also if we could have a page where we would have this like uh hashes to compare to or compare against that would be perfect yeah I can try to do some test

cases to calculate those I think I have some code lying around somewhere that's will hash block do anything else no probably not all right thank you all for coming and I'll see you all next week thank you guys bye