EVM: From Solidity to byte code, memory and storage

Transcript

Okay, hello and welcome everyone. This is the Ethereum Engineering Group Meetup. Um today David and I are going to talk about um the EVM, Solidity, and what Solidity looks like as a program in the EVM and how it uses um code, storage, stack, memory, um and other things. Um David, do you want to introduce yourself? Uh hi, David Holmeswood from ConsenSys. Uh I work with Peter in the Pegasus um business unit and on the uh on Team X.

Um and uh uh all of us are uh here at Team X are pretty much present, I think. So, if you want to speak with any of the others doing this uh area of work, um uh please uh either ask questions or introduce yourself to people that uh that you may not know. Uh Peter. So, thank you. So, um before I start, I'll say that David and I will brainstormed this talk, and we were going to work on it and do about equal parts, but David's got super busy. So, um you you won't hear David talking as much as I am, but um he still

knows a lot about this stuff, so yeah. All right, let's share my uh Let's do this. So, if I go play there, just a sec. As Google Slides catch up catches up, and so does um crap. Let's move that over there and go present on that screen. Sorry, just getting my screens to work. And now what we'll do is we'll share screen and we'll share desktop, too.

Okay. So, you should all now be able to see my slides. Can people see the slides? Yes. Right, brilliant. Thank you. Okay. So, this is a talk on the EVM. And the um yep put we're put together the um talk and we've had a lot of feedback from these people, which has been very much appreciated. Um we're going to walk through an introduction to the overall topic and then talk about Ethereum transactions and how they relate to it all and walk through um slowly but

surely. If we get to um about 1 and 1/2 hours in and we're not finished, what we'll do is we'll reconvene um this time next week. So, we'll see how we go. Um so, the color There are some examples in this um talk and they they can all be found in the Ethereum engineering group Solidity examples. Um the bytecode output that you'll see was generated using um some tools that we've created and the stack analysis was created using some static code analysis tools we've created.

So, when you think of um the Solidity language, you've got a a file that's something or other.sol and that's your source file. You put it into the compiler and normally you get out the ABI and the bin. You can specify parameters to say what you want to get, but those are the two files you normally end up with. And your binary file has your is your output of the actual byte code. And the ABI is an interface file which tells applications how to interact with the code. So, essentially what functions your code supports and what parameters they take.

Though, you it's worth remembering that Solidity is but one EVM language. So, for instance, there's a language called Viper, which is in alpha at the moment. Um and it's a compiled with the Viper compiler. Um all of that said, um we've done some recent analysis of all of the deployed contracts on mainnet and um the numbers are indicating that it's well over 90% of the contracts on mainnet were created with various versions of Solidity.

Um yeah. And so, um I'm not going to give you exact numbers because our analysis needs just a little bit of tweaking. So, once you've got those that binary file and the ABI, you then put it into some sort of Solidity wrapper generator and that'll generate a say a Java wrapper, which essentially is an interface that you can call from your Java code to call the Solidity code inside of the Ethereum node.

If you were using JavaScript, obviously you end up with a JS file or maybe you can just use the library interact directly. And so, once you've got your application code and your wrapper code, you use a library, some sort of web3 library. And then that talks to your solidity code via the internet. Um and your solidity code will be running inside the EVM inside an ethereum client inside of the ethereum network.

So an ethereum client is an instance of software running on a node in the network. And a client, given we're in a peer-to-peer network, is both the server and the client. So it's the essentially the machine that's operating the And so um yeah, so you've got to think your application is interacting um with the um the solidity code that is now byte code inside of the EVM. Okay.

So I think I've just skipped. Yep. So in this talk we're going to focus on that byte code in that code.bin. So that's the code that gets executed by the EVM. So in ethereum you have transactions. And um these are the things that are sent over the wire between the application and the ethereum node.

And um and also then around the network. And so you've got a nonce and so that's a value that starts at zero. You've got the gas price, which is how much you're pre- prepared to pay for gas in terms of ether um or in particular wei, where a wei is a unit of um 10 to the minus 18 of ether. You've got the gas limit, which is the amount of gas you're prepared to use, the two address, so who you're sending the transaction to, and the value, and the data to be sent to the two address, and the signature.

So, if you're just transferring money um between accounts, then you um have it set up like this, where you you always specify just 21,000 because that is how much gas you need to use for a base transaction, and the address you're sending the ether to, and the value, and the data is usually empty. but it's actually worth while pointing out that the data doesn't have to be empty. It's not mandatory. And so, if all you want to do is record a commitment onto the blockchain, you could put some data there, and then you'll have that as part of the overall blockchain.

So, it's important to remember though that it's not going to be part of world state. So, it's not going to be part of the state tree, but it will be part of information that gets linked into the overall blockchain, because the summary of all the transactions, so the transaction root, is part of the block header. Um if you're going to deploy a contract, the two field has to be empty, and that indicates to the um to the Ethereum client that you want to create a contract.

And the data field contains the initialization code, so essentially a code fragment, which is going to set up the state of the contract and deploy the And if you want to do a function call, then the data field is the uh are the parameters. So, what function do you want to call, and what are the parameters? And um it's really up to the EVM code to process that. So, code that um conforms to the ABI specification takes a function selector and parameters.

However, EVM code does not have to conform to the ABI spec. It could do anything. And as such, that data field is just a blob of data that the EVM code can do stuff with. Yeah, and there's a link to the ABI spec. um stack memory storage. So, all of the locations you can do stuff with in the EVM.

So, the EVM, so Ethereum, is a stack-based processor. And what that means is that there aren't registers. You've only got the stack. And so, when you have to do an operation, you use stuff from the stack, and then you So, you take input from the stack, and you pop that input off, and then you push onto the stack the result. So, add is popping off the stack, stack zero and stack one being seven and two. It's adding them together and pushing the result nine onto the stack.

And so, inside of the EVM, you can access and store information from six different places. So, there is the stack. So, you can push stuff onto the stack and keep it there for a while as temporary storage. Um the stack isn't persistent. So, at the end of the transaction, the stack is gone. You've also got call data. remember the data field that there was in a transaction? So, that information is available to the contract. So, when that transaction comes in, that call data is a read-only memory area.

Memory is a temporary um place of storage, where you can store stuff just for the duration of the transaction. So, it's like a scratch pad. Then you've got storage, which is a persistent data store. You've got code, and that's the code you're executing, but you can also use that for static data storage, as well. And then you've got logs. So, this is a write-only output area. when you're thinking about storage, um and how it all works, um you've got to think you've got a block header, and

it's got, you know, the parent hash, the omers, the block number, and all that sort of stuff. And it's also got something called the state root. And so, this is the root hash of a Merkle Patricia trie of all of the accounts. And if you um don't know too much about this area, um have a look at the YouTube video from 2 weeks ago, where Horatio and I did a talk on um stateless, and we do talk about the Merkle Patricia tries, and how they store information.

Um So, anyway, it within this trie, you've got a key, which is a the 160-bit account address. And so, then within each account, you've got account state. And so, that consists of, say, the nonce, and the balance, and the code hash, and a storage root. And so, within the storage area, you've got a another Merkel Patricia try and this try has got 256 bits of key. So each key is 256 bits.

so when we talk about storage, this is what we're talking about. We're talking about something stored against the account. Okay, over to David to talk about the yellow paper. Uh thanks, Peter. So the Ethereum virtual machine is defined in the Ethereum yellow paper. Um it is and has been kept up to date since the first version and so there have been changes to the yellow paper along the way.

But if you go into GitHub at this URL or just do a search for the Ethereum yellow paper, it'll pop right up to the top. Um the paper was originally written by Gavin Wood at the beginning of the Ethereum project, but since then um many people have contributed uh to this paper. Uh next slide please, Peter. Toward toward the back of the yellow paper, um you will see um that all of the Ethereum opcodes for the Ethereum uh all the Ethereum virtual machine opcodes are defined. This is the the canonical location of this

information. So what's here is the definition of the Ethereum virtual machine. It's a virtual machine that implements the totality of these opcodes and the way they're defined. So if we look at the first few here, uh you'll see that Uh, you've got values in hex that at zero and they just increase. So, the very first op code is 0x00.

That's stop. Next one is 01, that's add, etc. On down the list. Um, all the all the arithmetic is modulo 2 to the 256, which is the same 256 um, number that you saw in that Merkle Patricia try uh, the on Peter's last slide. And the um, uh, description of each op code uh, goes through. You'll see that stop doesn't take any uh, it doesn't take anything off the stack. Doesn't take any input.

It just halts the execution. Add, Peter's already shown you in a previous slide where it takes two items off the stack, stack uh, item zero and stack item one. Adds them together, then pushes back onto the stack. op code two, um, MUL or for multiply, does exactly the same thing as add, except the operation, the computation is to multiply things together instead of add. This is pretty much the way this level of any um, machine code uh, looks like.

Um, and it's going to um, it's going to be very familiar to all of you who had this level of of computer science. It's it's pretty straightforward stuff. Uh, if you if you look through all of the op codes, uh, you would find that it's all the typical sort of things you'd get in a compiler class uh, with the addition of some things that are very specific to the Ethereum virtual machine. Uh, Peter already mentioned that they're stack operations. Uh, it's a stack virtual machine.

And uh, uh, the higher op codes are the ones that are more specific to Ethereum operations. Peter? Thank you, David. yeah. So, now I'm going to talk about contract deployment constructors and the init code fragments. this is some Solidity code. And um, so we've got a constructor, um, which is public.

And we set um, one of the storage variables to three. the init code includes code to deploy the contract plus set up the state. So, in other words, setting val two to three is part of setting up state. And so, you know, you compile the code using that to produce the dot bin file. And you get something like this.

And so, in this code here, you've got the init code fragment and the actual code to be deployed. And so, it doesn't have to necessarily be this arrangement, but for Solidity it is. as I've said before, contracts are deployed using transactions where the two address address is not specified and the data is the init code fragment. Um, and so, the init code is not stored on the blockchain. That makes a lot of sense because it's the constructor. It's only ever used right at the start to set things up. So, it doesn't need to be

stored on the blockchain. this for this particular type of transaction, the data is actually treated as code. So, um you might remember I said that, you know, you can access the call data. So, for a contract deploy transaction, the call data is actually treated as if it's code and executed as code. if you look at that um binary there, you start out with 68. And so, 60, if you looked at the yellow paper, is the op code for push. And pull push one in particular, meaning push just one byte.

And so, that's pushing 0x80 onto the stack. Oops. Yeah. And so, if we completely um if we decompile a bit more, or um turn it into op codes, this is what you've got. So, we're pushing 80. And so, push a word onto the stack. So, the stack has a depth of 1,024. So, you can have 1,024 op um things on the stack.

And so, when we use the term stack zero, that's the top of the stack. And stack one is the one just one below the top of the stack. And the stack items are one word wide, where a one word is 32 bytes. So, when you're pushing one byte onto the stack, by push one, then the top 31 bytes are set to zero. after this has um happened, you've have 80 as the top of the stack.

And then we push 40 onto the stack. um the stack now looks like, you know, 40 at the top and 80 as the next one. And then we have something called MSTORE. So, MSTORE is store a word um to the temporary storage memory. So, that to that memory area. And so, it takes two values off the stack. And so, stack zero is the location to write to. And stack one is the value to write. And so, what this is going to do is it's saying, "Well, set memory location 0x40 to value 80."

And so, what this is doing is setting up what's called the free memory pointer. And so, um the idea of the free memory pointer is that the Solidity code that runs after this can load it up and it'll know that anything after the 0x80 is something that it can use. And so, if it allocates um some information into memory, then it will need to update the free memory pointer. And so, that's the idea. It's interesting to note that 0x40 is a byte offset and not a word offset.

And so, though we are storing whole words, it's actually a byte offset. Call value puts onto the stack what the value field of the transaction was. So, how much wei has been sent. then you have DUP1. So, DUP1 duplicates the top of the stack. And so, now we've got the stack looking like this, where you both the stack zero and stack one contain the value that was sent with the transaction.

And so there's also dup two, three, four, all the way up to 31. And um what they are doing is they will duplicate different levels um into the stack and put them onto the top of the stack. And that can be used for um stack manipulation. So is zero pops a word off the stack. And if the word is zero, it puts one onto the stack. Otherwise, it puts zero. And so um what this is saying then is well, if the value was zero, put one.

But if the value wasn't zero, put zero. then um we push two bytes of the um onto the stack. And so this is So push two means push two bytes. And then jump e is jump if. And so what that is all about is it's saying that if um stack zero is oh, jump to stack zero if stack one is not zero.

it's going to jump to a location or go straight through depending on the value of stack one. And so um yeah, so if that value wasn't zero, then jump, otherwise don't. And then it pushes um zero onto the stack and then duplicates zero. So then you end up with a stack that looks like that, stack zero and stack one both being zero. And then we have a revert instruction. And so revert halts the execution and indicates that a revert has occurred.

And it uses stack zero as a memory location and stack one as a length to indicate the revert reason. So the reason why the code reverted. So the stack had zeros at the top. So in other words, we're saying there is no revert reason. all of that code together is really saying, "Look, this contract, this constructor wasn't payable. And so if someone sent value to the construct as part of the init function, as part of this contract deployment, then we should revert because the contract is not

supposed to have ether sent with the um contract deployment." So when you have a jump, whether it be a absolute jump or a jump um if jump I, you always jump to a jump destination. um the program counter, so we had push two and pushed 0x10 onto the um stack. And then we jumped to it if the value was zero. And so that's going to get you to program counter 0x10.

And so um yeah, if you jump to an invalid jump destination, then um your code will stop and you'll have an error. Pop pops the top the value that's on the top of the stack off and just throws it away. And so, now we're Yeah, so it's worth recalling that we were setting val2, which is the second storage location, to the value three. what we can see here is we push three onto the stack, we push one onto the stack, and then we store it.

And so, um sstore stores a value into storage. And so, it takes um the stack zero is the location to write to, which is going to be one, and it's going to take the value, which was three, and it's going to store it there. So, storage location one becomes three. And so, that matches what we're seeing in the Solidity code there. So, now what we do is we um push some values onto the And so, um and what we're doing is we're getting ready for code copy.

So, you might remember the code um the runtime code followed the init code fragment. And so, what this is doing is it's getting code getting ready for the code copy operation, so we can copy the code from from where it is at the moment um to the right location. So, we set up um 24, and then you can see that um here on the screen after the invalid operation at PC 23, we've got PC 24, which is push 180. And so, that put it from 0x24, that's the start of the runtime code.

And so, code copy copies from the code um that is currently executing to memory and so stack zero is the offset to write to stack one is the offset to read from and stack two is the length of the bytes to be copied. And so the stack ends up looking in so the stack looked like this. So what we're saying is let's write to memory offset zero and start at offset 24 program counter 24 offset and do it for zero x c one bytes so which is the length of the run time code.

And the thing to notice you remember how we set up the free memory pointer so we're just wiping that out so we set that up in memory but now we're just going to copy straight over the top of it and that's not going to matter because we're going to exit but it shows you that setting up that free memory pointer just for this piece of code didn't actually help and it was yeah a bit of a wasted few operations. Yeah. yeah so we copied the code.

Okay so overall that block was copy the contracts code to that's going to execute on the blockchain that we want to have on the blockchain to memory. And then we return and when we return from a in it function so a contract deployment transaction stack zero is the start offset in memory where the result is and so stack one is the end of the result So, that's the same for any return function, but the result that's returned with the return function for a contract deployment is

the code that should be copied into the Um yeah. All right, which is what that just says there. So, store this um code into the blockchain against the new contract's address. So, there's an invalid op code there, too, which should be un- which is unreachable because you've already returned. And so, the return followed by invalid indicates that this is the end of the init code.

in summary, we set up the free memory pointer, which in this case wasn't used. Um we caused a revert if any way was sent with the transaction because the constructor wasn't payable. Um we set up a storage location to a non-zero value, and so we could have arbitrarily set up lots and lots of them and in complex structures, as well. And then, we returned a pointer to the code and the length of the code that needs to be stored into the blockchain. Okay. So, what about function calls?

So, when we look at this code, quiz question, Lucas, how many um how many functions are we going to end up with? Uh say one. Ooh. We're going to end up with three. Oh, yeah, cuz they're the cuz they're the the Yeah, gotcha. Good one, Pete. You got me in the fields of functions. Yep, the fields of functions. So, you want So, when you're writing code, Solidity code, you want to have a very close look at this. So, if you've got um variables, um you can have them private, and then you're not going to have access of

functions in your Solidity code. Um so, if they're private you don't need to access them um programmatically, um then just have them private. If you want to access them, if you just have them public, then essentially getter functions will be automatically created. So, what this means is that we actually have three functions that we should expect to be created for this code. One to fetch val1, one to fetch val2, and one to execute set.

at the start of the code, we're going to set up the free memory pointer again. And then, because none of those functions um are payable, it does the let's check to see that the value is zero again, and if it isn't, let's revert. And then we jump to the next block of code. And so, we caught that the um the code automatically copies the value onto the stack, and I think that's just in case any of these functions were payable, and you need to use that value. But, so it pops it off.

So, call data size is a um op code that tells you the size of the transaction data field. And less than is going to tell you whether stack zero is less than stack one. And it's going to pop the values off and then push on push one onto the stack if they're less than or zero otherwise. what that push one four down to the jump I are all telling you is if the transaction data field it has less than four bytes, then we want to revert.

And the reason why it wants to do that is related to something called function selectors. So, a function selector is a truncated message digest of a function signature. So, you can think we had a set brackets you went 256 function. And so, if you message digest that and then have truncate that down to 32 bits, then that's the function selector for that function.

when we're doing this revert if less than four, what we're really saying is if we don't have a function selector, you know, so if we don't have enough bytes to at least have a function selector, then we should revert. And so, then if you did have enough and more than four bytes, then you go to a jump table. And so, you execute call data load. And so, call data load only loads 32 bytes at a time, so one word at a time onto the stack. And it loads starting at offset pointed to by stack zero.

So, you can see we've pushed zero and then call data load. So, we're loading up the start of that call data. So, load up the first 32 bytes. shr is shift right. So, shifts um the um stack one to the right stack zero times and then pops them off the stack and then puts um the result back on the the um we've had um stack one. So, stack zero had the um yeah.

Up. Yeah, sorry. I should have had a a little diagram here saying what was on the stack. So, call data load has loaded up the value of the call data, first 32 bytes. Then we're um going to shift by 0x e0 um to the right because that's the top of the stack. And um so, essentially what that's going to do is it's going to shift the top 32 bits from the um top of the word to the bottom of the word. And so, what this is really doing is it's loading up your function selector.

then once we've got the function selector, we duplicate it. And then we say, let's push a value onto the stack, I essentially a function selector, and we say, "I will does the function selector that was supplied in call data equal this? And if it does, well, let's jump to offset 41." And so, if you see in code function selectors, you can go to this website here and paste it in here and search and it'll tell you if it you know, what it thinks that function selector might mean.

Now, because it's a 32-bit hash or truncated hash, you're going to have collisions. So, just because you see that it comes up with a certain thing, it might not necessarily be be that. But, if you have um your own um if you you you know, if you have your set of um functions that you think might it might be, then it could help you differentiate between say two or three different functions. So, and then with this next bit of code here, it's going to jump to 5D if the um function selector is um that.

And another jump. if um the function selector isn't one of the ones that we know about, so if if the the function selector passed it in as a parameter isn't one we expect, then we don't know about this function, so we revert. Okay, so this is um Oh, yeah. So, Ben um mentioned something that I didn't realize. So, when you're looking at the um offsets in the bytecode, um they're put in

ascending order. And um so, Ben was saying, "Well, you know, what you could do is you could try and work out a function name that produced a a um function selector um with a low value, so it was going to be higher up. So, your important and oft-used functions would be higher up. Um but um a extra fun fact from myself is that you've got to be careful with that because if you have more than about four functions, so maybe eight functions, it starts to um do a binary search. So essentially it says, if it's greater than this, jump to here, um otherwise go straight ahead.

And I think I'm seeing a few that have got say 32 different functions in a Solidity contract, and I'm pretty sure they actually do a binary search um until they have only about four functions or so. So, yeah, you'd want to actually look at your code before you started doing micro optimizations like that. Okay, let's focus in on this set we've got a parameter on that we're expecting to be passed in via call data, and we're going to set it to that very first um storage location.

So, the first thing code does is it sets up a return address or a end address. And so what this is is that the code invariably has some code that it's going to call right at the end to exit, whether it be just to stop or maybe to return a value. And so invariably the way the compiler is written is such that you um when you enter a function, you push this address that you're going to go to at the end onto the stack. And then what we do is we work out the call data size minus four. So essentially,

what's how much call data do we have if we exclude the function selector? another, before we jump into um analyzing the next block of code, um something to understand is so if you ever see LT followed by is zero, you've got to think that what we're doing is so less than would set one if it was less. And then is zero will set up to one if it's zero or set it to zero if it's not zero. So essentially LT followed by is zero is going to give you not less than or greater than or equal

to. what then So we had the call data minus the size of the function selector. And then we push 20, which is 0x20, which is one word onto the stack. And we say, "Well, look, if we don't at least have so greater than or equal to one word, then we should just keep on going straight ahead and revert." But if we do, go to location 55. So then we pop the a value off the stack, get rid of it.

And call data load. So what we want to do is we want to load the word at offset four from the transaction field onto the And then we push 7B and jump to there. And so that's essentially 7B and then jump is just a hard jump. So yeah, what you can you can think of this is saying is push underscore param onto the stack and then jump. And then once we get to 7B we push zero onto the stack and then S store. So we're storing at location zero the value that was next down on the stack, which

was the param. So store underscore param at location zero. And then we jump to the location we set up earlier, 5b. Which is there, and then we stop. So, stop means end execution indicate indicating that the execution status is successful. Um and don't return any data though. So, this is stop is what you'll get if it's a transaction that doesn't which doesn't return any data.

All right, what about val2, which was returning um uint256? So, focusing on that. So, it's just a view call that's auto-generated and its signature is just val2 brackets. All the view returns and brackets uint256 is not part of the function signature. So, if we jump to there, first thing it does is it sets up its return address, which is 63. it loads the um storage location one and duplicates it.

Actually, dup2, which is So, dup2 is give me the value from not the top stack, but the one below the top stack. So, if you recall, 63 was the um return address essentially. So, we've um sload will have pushed val2 onto the stack, and now we want to get 63, and so we put that there, and then we jump. And once we get to location 63, um now we've got this on the stack.

Um and you'll notice that the um stack one is never going to be used. So, um, but val two will be, so stack zero. So, we load up the free memory pointer. And then we manipulate the stack. And then we duplicate that free memory pointer, so we swap the free memory pointer down and then duplicate it back to the top of the stack. And then what we do is we store val two to that location pointed to by the free memory pointer.

And then what we do is we load up um, yet the free memory pointer again, so we load the free memory pointer, um, again, um, so we've got a copy of it. And we do a bit of um, stack manipulation. then what we do is we say, well, how big we we're essentially creating how big is the return value that we're going to be creating and what's it off it's offset.

after all of that manipulation, we're then ready to return. And so, remembering return says, um, stack zero is the offset of the result and stack one is the length of the result. And so, what you're doing is you're saying, all right, where the free memory pointer was pointing, so 0x80, I've got one word and please return that. Which is the value. So, val two is at that location. So, putting all of that together for that block of code, what it was really doing was it was returning a uint256

that was put into memory. So, let's focus on actually with this is val1. This is an error in the slide. We're now focusing on val1, not val2, cuz we've already looked at val2. Oh, sorry. So, what this is showing you rather is that what we were trying to do is return val2, which is what we've just done. So, now focusing on val1. So, the first thing we're doing here is we're we're setting up the return address again. And something to note is the 0x63

is the same return address that we used for returning val2. So, we use it we're reusing the same code between two different functions because that piece of code all it was doing it was saying, I've got a uint256 and I want to return it as the result. And so, because it's common code, it can be reused between the two functions, the two view calls. And you notice that in when it's getting to 0x87, we're doing an sload from location zero. So, very similar code.

And it's just essentially reusing code, which is interesting. It means your bytecode will be smaller. what about Peter, I have a have a question for you. Is that the result of the compiler optimization? If you were to turn optimization off, would those have been separate? Would the would the duplication have been present in the bytecode. That is a fantastic question. I suspect the answer is correct.

Yes, true a true statement is that the code has been generated with optimization set to on. Yes, so you may you're probably correct that there will be things that duplicate code if you don't have optimization on, I suspect. Though yeah, reverse engineering or you know, like getting the op code from the bytecode would tell you for sure. So, what about if you want to

be able to have people send ether to the contract. Um and so what you do then is yeah, so you could have a function which accept is payable, but if you just want to have normal transfers, normal non-function call transfers, you need to have what's called a fallback function and it needs to be payable. as well, a fallback function is the coverall someone's done a function call to your but the function didn't match any of the ones that it's got. So, you still want to accept that

function call and do something. And in particular, you want to accept the ether sent to it. So, Hey, didn't they make some breaking changes on V6 around uh now? I know you know you compile those four, but um I think they've changed the fallback into two functions now. On version 0.4.20 Yeah, I'll post a link on the chat. Yeah, please do. Yeah, I um compiled this using version 5.something or other.

Um yeah, all right, interesting to know. I know that there've been a few change breaking changes with respect to fallback function. So, fallback functions are one of these things that I know I well, when I first saw them, I thought, "Oh, yuck." You know, I mean, the idea of someone doing a function call to your contract and the function not matching anything that you've got, but you accepting it anyway seemed a bit dodgy. Um but yeah, having a payable fallback function makes sense if it's just to accept a direct ether transfer, but a um function call that didn't really work should just fail.

Yeah, all right. Thank you, Lucas. Yeah, so if you do have that callback function, how does stuff change? um now if the call data size is less than four, then we're going to jump to 0x30, which is now going to rather than being a revert, is just stop. Which means that the ether will be accepted. And additionally, um you've got your normal function block, um but if none of them functions match, you'll just um fall through.

And again, you'll just stop and accept the ether. Another side effect of it is because the the you've got a payable fallback function, but the other uh functions are non-payable. Now, each and every function needs to check whether the call value is zero or not. So, you've you'll end up with more So, the next area of um of the talk is about storage.

storage is set up as 32-byte words, and variables that are smaller than 32 bytes will be stored in the same word. And so, you end up with something like this. Where's the byte is stored by itself in storage location zero, and then you've got a whole word, so that's in storage location one, and then you've got storage location two, and then three. And so, you've because they perfectly um match the right size, you um the you ain't you ain't 32, 64, and address are all pushed in together.

If you had a bit of spare size, so you could imagine you could have moved your storage around, your um your ordering of your variables, so the byte actually could have been part of that word, that storage location two. So, remembering as well that um setting up um these um each of these memory locations to 10, 11, 12, 13, 14, and 15 hex. So, this first block of code is going to set that initial byte to 10.

So, what we do is we load up the location. So, it's even though the only thing stored in this storage location is that byte, the code isn't smart enough to work that out. So, it's still going to assume that this is just one byte of a word that is being used. And so, it loads up just one byte, pushes the 10 0 x 10, the value that we're going to store there. And then, it loads up FF and does not. So, that will end up giving you 0 x FFFF with 0 to 0 0 at the bottom. So, essentially a mask, which you then and with the current value in that storage location, and then

or in the value that you want to put there, and then you store it. So, you've got to because it's only one byte, you're masking it in. If you compare that if you If you had have had that same value as just a you went 256, then you would have only had these three op codes here. So, you would have used up less fuel op codes. I mean, it should be of course noted that push and swap and other op op codes like that only use tiny tiny tiny amounts of gas compared to a store. Um so, it has very

little actual effect cuz a store is um 20,000 or 5,000 depending on if the memory load or storage location has been used before, whereas some of these others are only say three or five or something like that. So, for storage location two, we've actually got quite a lot of code. so, what's happening here is that we loaded up at the top of this code block we mask in each of those fields, each of those values, and then we store it.

So, this is important because every time you do an S store, you're using 20,000 gas or 5,000 if the location's already been used. So, just doing one S store for all that is important. it is also so you're So, having these three variables in one location means if they're being manipulated at the same time, you're going to use fewer S loads and S stores, which is going to affect the amount of gas you're going to use quite substantially. And again, this one here is just a word.

what this is telling you is that well-laid-out storage variables will result in fewer locations, and fewer locations, well, I say will, but it's probably could save you money. It will depend on your usage patterns because it really comes down to how many times do you call that S store and S load operation. Um so, in the example there, um there was no way of um having that val byte, that first byte, in with the other um storage locations. But, um if there had have been a spare room, you could have pushed it in with them so that you would have had fewer storage locations.

Okay, bytes and strings. So, bytes and strings are um variable-length byte arrays, and they behave in the same way. So, you can do something like this where you're um pushing a um a byte onto the end of one array. the byte is allocated a storage slot. And um so if given the code and we only had So here we've only got one value that we're

in having in storage, it'll be stored in storage slot zero. you'll have to excuse this diagram. A few people have pointed out that byte zero off to the far right here should be as big as byte one and all the other bytes. So it's not that byte zero has got more bits than byte one. It's just that it was harder to um draw it. but so if you've only got zero to 31 bytes, then um you have your byte zero has the bottom bit set to zero to indicate this is a small number of bytes.

And it has a length field which is um zero to um 31. And with the top bits being zero. And then it allocates bytes starting at the top of the overall word and going across. And so all of this is stored in the actual storage location um for the variable. However, if more than 32 bytes are stored, then the um marker Sorry about that.

Susie's obviously got some thoughts on um variable length byte arrays. Uh um As well she should, Peter. So um the marker bit indicates that um you've got 32 or more bytes in it in it. And so, what happens now is the storage location zero just holds the length of the overall bytes array. And then the storage location, which is given by the message digest, so Keccak 256 of the storage slot number.

So, we're in storage slot zero. So, that's going to be Keccak of zero. And that is going to be where the bytes are stored. And if you've got, say, more than 32 bytes, then byte 32 and more will be stored at Keccak of the storage slot plus one. if you had two bytes arrays, then one of them will be, say, in storage slot zero, and another one will be in storage slot one. And so, if you do a Keccak 256 of zero and one, you'll end up with a completely different part of the number space. And so, you're not going to have collisions.

So, what does this actually look like in byte code in in opcodes? So, we push that return address onto the stack to start off with. We work out have the get rid of the function selector from the call data size. And then what we do is we check the length of the variable that's been passed, the parameter. So, you might recall we're only after one byte. However, in Solidity and in the ABI, all parameters are passed in as words, so 32 byte words. So, even though we've only got one byte, it's passed in as a

whole word. what we um do is we check that we've got at least 32 bytes. Um and if we don't, then we revert. And so, then what we do is we load up we load up the the call data, the word, and we create a mask. And so, you could go push 32 and 0x00 and lots of F's, or even push 31 and just lots of F's. Um however, these opcodes here, 44 TO 4C,

UH WE SHOULD BE RIGHT NOW, EVERYONE. MY KIDS ARE HOME. Yes. So, um if you look at that you the fact that we are um from 44 to 4c, we're setting up a mask. And um the reason why you're doing it that way, rather than push 31 all F's, is because this ends up being fewer bytes. And so, again to David's point earlier about optimization,

I am pretty sure if you had the optimizer off, you would be pushing um 0xffffffff. we mask off using that end operation to mask off the parameter, and then we jump to 53 hex. And then, we do some stack manipulation, and we load up storage location zero. And we mask off the length part of storage location zero.

So, 3F. So, 3F is um 30 1 shifted by 1 and with the one extra bit. And you end that in. And so, if you can think I'm going to go back a few slides here, if you can think of this length field, we're trying to mask off the length and the bottom bit. And so, then what we do is we push 3E And we say, "Well, is the length is this length plus that bit perfectly equal to 3E?" Cuz if it's

perfectly equal to 3E, then that means that the actual length is 1F and the bottom bit is zero. And so, if it is, then we jump here. then we've got the stack looking like that. And then we store um the byte. Well, we we we store um the um Yeah, we set up the memory location to be zero. Mstore the memory zero to be zero. So, we you we're we're using the um location zero as a scratch pad. So, you remember how we had zero and then we

had um 40 hex was the um function um the free memory pointer. So, location zero is a scratch pad. we use that scratch pad to record the off the essentially the um storage location that we're going to use. And then we use um Keccak-256, which is denoted as SHA-3. Um so, though the op code says SHA-3, it's actually doing a Keccak-256.

And so, what that's doing is it uses the memory location um pointed to by the stack to do the message digest. And so, it's going to do a message digest of um offset zero in memory for 20 bytes or 20 hex bytes, which is that one word. And then what it's going what it does is it um gets rid of the length byte that currently exists, and it stores the current um information that is currently stored in the bytes into that Keccak of um zero.

And um then it stores the upgraded length. And so, the reason why it's doing this is that now we're in a good state to actually process the bytes. So, all of the work that we've done before this has just been saying, "All right, we've gone from having 31 bytes. Now we're going to have 32 bytes, so we need to change the way the length field operates." And so, you can see there's an awful lot of code there to manipulate and get bytes in just the right right location. that's bytes. Another thing we've got is

um variable-length arrays. In so, storage of And so, you can have uint256 arrays, byte arrays. Um you can have arrays of pretty much any any of the types, including complex types. um values for an array are stored at a memory location which is given by um the message digest digest of the storage slot number plus um the um offset.

So, I I say key, but it's really the offset in the array. And so, you store the value into that offset. And so, the number of elements in a dynamic array is stored um using that formula. So, storage and then slot number. And so, if you've Yeah, so as you make the array bigger or smaller, you put it into that value there. Go into that into the storage slot zero, say, for the um array. For mappings, mappings is a way of mapping between um one type and another type in a a sort of key-value store,

where the um value can be arbitrarily complicated. So, for mappings, um what we do there is you have the um key and the storage slot number, and they're concatenated together. if And then you do the keccak of that, and you set the value to that location. you You should note that for mappings, nothing is actually stored at the storage location slot number itself.

So, you can have mappings of mappings and mappings of arrays, arrays of mappings, etc. And so, what you do there is you do recursively do that kick out 256 to work out the location where the actual data should be stored. Have a look at that link for details. So, we've talked a little bit about memory as being a scratch pad. And um you might be thinking, well, I could write some arbitrary um code, EVM code. I could push, I don't know, a huge number onto the stack.

And I could say store zero at that location and go M store. And then you know, these Ethereum clients would have to try and allocate that memory and then they'd crash cuz they'd run out of Um so, um each time you use an extra memory location, it costs gas. So, you have this memory expansion thing. And so, if if you run out of memory, you're going to probably run out of gas or you will run out of gas.

And so, before you cause an out of memory error on the Ethereum client, you're going to run out of gas. And so, I guess what this is saying is that you want to be a bit careful about say allocating huge arrays or huge other variables on as memory because they cost gas. So, the layout of memory is um so, the first, um, 40 hex bytes are a scratchpad and they're used for hashing methods for when you're doing things like creating, um, the working out the location variables should be stored in storage for mappings and arrays.

The next location is that, um, free Then you've got zero and then you've got your, um, location that your actual, um, memory starts at. So, if you say create a uint, um, array in memory, then it'll be stored starting there, for instance. Peter has a quick question on on memory before you, uh, go on.

I don't see any protection in Ethereum, unless it's implemented at the client level that I'm unaware of, um, or maybe, I guess potentially, um, you know, at, uh, uh, maybe the web3 layer. I don't, you know, if if you were to create a client and you had your own implementation of soliditywrapper.java, right? Or, um, or you a web3 library, you could

inadvertently or advertently create a situation where memory just was terribly corrupted. it wouldn't be hard to do that. And I guess Lucas is about ready to find out how hard that is. Right? And so um it's really up to the it's really up to the client to keep the keep any memory corruption issues happening, I guess is my point, right? Uh no, it's um so memory, it's the Solidity compiler.

very much in Lucas's domain um to make sure it's okay. One thing that would be interesting, I think if you tried really hard, I reckon you could get a two storage locations to override each other. So, you know, like if you've got a I I I I haven't tried it, and so maybe there's some protection against it, but if you can imagine that you've got a Keccak-256 of say storage low slot zero is an array um starting at that location, and then you've got another starting at a different location in the overall

storage. So, if you did a certain offset into that array that just happened to match the other um value, um I do wonder whether you wouldn't override it. So, that's storage, but you would have to try hard. Um you know, you would have to Actually, Oh, sorry, go ahead. Actually, there is an attack known attack like that. Turns out that in there is no protection against underflowing the index of an array.

So, even though it's it can be I mean, in principle, they are very far away one from another because of the hashing of the of the addresses, but still it's doable, and I think it was done in some even in some known attack. So, yeah. And the the summary is it can be done. Not not because of hash collision, but just because of using the way in which the index of the array is moved around.

Yeah. Yeah. And so, I guess what that's showing is that um Yeah, you could um yeah, you could create if your Solidity code essentially took an an array index via the API and then fed that in, then yeah, that That'd be an interesting attack to um to do. But, we should do a talk at some point where we specifically write the code to do it or um yeah, I'm sure we could um get that to work.

Um shouldn't be too hard at all. Oh. Shouldn't It'll All right, it'll take a day. Take a day to actually get it to to work and write the slides. How's that? Oh, yeah. Um but, as far as memory goes though There's no memory Oh, no. Look, I I I used a fair bit of Thursday and Friday on these slides. So, um I'm I'm not I've done my slide writing. Anyway, I've got another talk in 2 weeks' time, so that's going to burn time to do.

Um but, no, we we'll do that as another talk for sure. but, um yeah, I think in memory though, you don't have dynamic sized arrays. And so, it's all fixed size. So, I think if you for corrupting the actual memory compared to storage, I think you're going to have to um have a um bug in the Um yeah.

but actually before I go on, there is one extra point to make and that is you might recall how each account storage is completely separate. So, what that means is even if your contract or someone else's contract is a bit buggy and allows people to do all sorts of nasty things, their contract cannot trash your contract because you've got a completely separate storage area, which is really important. So, yet another location that you can access stuff from is code. And so, you can use code copy or external code copy.

So, code copy copies um, information from your contract into memory and we've seen it was used in the init code fragment, but it can use it for other reasons. And you can use external code copy as well, which copies from a different contract. And so, I've seen this used for revert reason error messages. And so, I dare say if you've got any static strings, then um, they were would likely be stored in code and not um, say in storage or yeah.

also though, if you wanted to have your contract as a contract that deploys another contract on demand, you can imagine you would load up all of that from your own um, space. And the reason why you might say use a um, have some static data stored in a contract rather than S store and S load is because S store and S load are dramatically more expensive than um, code copy and um, writing to code. Um before we end, there is one extra area of um to look at. And that's

auxdata, uh auxiliary data or metadata. And so, modern Solidity, and I'm not sure which version this is from, but um has um optionally has um metadata added to it. And um it's bytes that are added to the end of the contract in CBOR encoding. So, that's a binary encoding format. And what it does is it indicates the Swarm or IPFS message digest. So, it or the So, it's the message digest of the source code, and it indicates um that this this code is stored on Swarm or

IPFS at this location. And of course, so you've got to upload the code yourself, but the idea is that in combination with the compiler name and version, which are also there, and whether it's an experimental version, indicates to you how to compile the source code to get this bytecode. And so, you can use it to automatically verify that the deployed contract matches the source code because, you know, you can have these tools automatically scan. if you have a look at the auxdata, you you've got something like this, where you've got a map, and then you've got a length. And so, bzzr0 with Swarm, but you could also have bzzr1

or IPFS. And then you've got a length of your message digest, and then you've got some characters, which might will say say solc, and then the version of the compiler. So, there you are Lucas, I was using 5.10. Nice. Um and have a read there of the docs. So, in summary, um the EVM is a stack-based processor that has access to all sorts of stuff, including the stack.

And so, call data, read only, it's the transaction parameters. Memory, it's a temporary storage area um and it's not persistent. Storage is persistent and it holds the world state or is part of world state. Code, you can have your you've got your source your actual code you're executing, plus it's got static data. And also, um output-wise, you've got a write-only event log. there's so much more to cover that wasn't covered. Um and um you know, like cross-contract

calls, what do they look like? And what you know, why do you use the various types of cross-contract contract calls? What about error handling? So, exceptions and um have been added recently, and we've also got asserts and requires and reverts. So, how do they all interact? I didn't really talk about logging and what do they look like at the bytecode level? Um are they efficient? How do you make them efficient? Um I didn't talk about bytes32 or bytes1 um and compare that to uint256.

Um and so, I probably there's more work to be talked more to be talked about about memory variables. Um so, we we've been doing a project on code metallization and so we've been analyzing all of the contracts on mainnet. And there are some really weird and wonderful contracts out there and you could almost do a talk just looking at and trying to understand what was the developer thinking and what does that contract actually do and why? it might be interesting to look at say a real contract that people use a lot like ERC20.

yeah, or maybe some of the other important contracts and we've already added another bullet point to the list. Try and write a contract that has a storage collision or a storage override. Um, but it'd be interesting for people to have a think about what else they would like talked about and so this will appear on YouTube soon enough and when it does, please tell me if you've you know, if you've got any ideas put comments in. Is there any decent decompilers?

Um, oh from byte code to solidity do you mean? Yeah, or some kind of more readable markup language. It could get messy right so. Yeah, I I haven't seen any but I haven't looked. So I've written some code that does static code analysis which at least will tell you which of the byte code is part of which function. And you know, gives you the call path. It also shows you what the stack looks like at every point of the code execution. So it's not a decompiler but it certainly helps you understand what's going on and it also takes byte code and

gives you that printout that we've seen during this talk. Um but not Solidity, but I Yeah, I don't know. I'd do a web search. Maybe someone's got something, but um I think it'd be pretty hard. Yeah, you couldn't get any function names or that kind of thing, so it'd be You have to kind of make them up, like function one or something, function two. But Yeah, anyway, something to think about. Yeah, I don't know. I think given it's on your 32-bit um space, um you could scour the internet for all of the Solidity um function names for all the open-source projects, and I reckon you would get a pretty big data set that

could be used for reversing what the function could be, and then based on the parameter um usage, um that would give you an idea, as well. Okay. So, um we're going to be doing a few more talks, and um in 2 weeks' time, I'm going to talk about code Merkelization, and then in 4 weeks' time, Prek Avendra is going to talk about polynomial commitments and point proofs. Um so, code Merkelization is part of Ethereum 1.x, and um it is related to stateless. And so, essentially, how do we have the ability to um not have to send all

the state. So, part of the state is the code that's going to execute. So, how do you work out which bits of code to send? And so, that's what code Merkelization's all about. Polynomial commitments is a thought that maybe there's a way of replacing the Merkel Patricia try, and to do proofs. And so, they offer the promise of much smaller commitments. Um then on the 13th, which is a Monday, um we've got a conference coming up. And so we've just finished the call for speakers just um yesterday.

And um so the early bird registration which ends on this Friday is Australian dollars 20 um and standard registration after that will be 40. Um so we've got a whole stack of um potential speakers and um David, Sandra, and I've got to work out who the speakers are actually going to be sometime tomorrow. Um but it looks like it's going to be an awesome um world-leading lineup. So um please consider registering for the conference. in August the 5th we've got Tim is going to give a talk on EIP-1559 um which is all about um fee markets.

And so he's going to talk about the proposal and the various alternatives. And just how um this can be approached. So Tim actually lives in Canada, so we're going to do the call a bit earlier than normal. So 3 hours earlier. Um so please consider coming to those talks. Are there any questions? Oh, Lucas has dropped. We can uh probably safely say it'd be nice to have a future talk from Lucas on his compiler if he gets uh some progress made on it.

For sure. So was that talk Oh, I can see we've got a ton of stuff that's going flying by in the um chat. Ah. Yeah. Anyway, hopefully my audio didn't break up too often. Um, yeah.

It's good. Thank you. All right, cool. Great talk. Uh, thank you. All right. Well, everyone have a great day and I'll um, talk to you all uh, later. Bye-bye. Thanks. Thank you. Bye.