Understanding the Ethereum Blockchain Protocol

Transcript

[Music] hello everyone um welcome to the third day of the of the conference deep dive into ethereum protocols so we started off with a wonderful presentation from Alex on the protocol on top of which the other ethereum protocols run so I fig it makes sense to follow that up with a presentation on the ethereum protocol so you know the idea behind ethereum blockchain first came around back at the end of 2013 when people were getting really excited about Bitcoin people were

getting starting to get excited about blockchain technology and applications Beyond currency and there's a public consensus that blockchains are useful for stuff and not just money so you know blockchains could be used for asset issuance crowdfunding domain registration title registration gambling prediction markets iot you know people are coming up with some new category of applications every week and so you know the problem at the time is that most existing blockchain protocols were designed like this single purpose tools that are that are specifically designed around one

particular application so in bitcoin's case currency in namecoin case domain registration in in Prime coin's case you know trying to do scientific Computing and in my opinion not doing a very good job of it so then we had second generation protocols that looked like this so we had protocols that had somewhere around 55 different transaction types where one transaction type is something like you know create a binary option another one is join a binary option then one for settling a binary option one for making a bet one for settling a bet one for doing some kind of two-party dice game and every time someone came up with a new kind of application that was deemed important

enough to be worth including the people who are creating these protocols would just add yet another transaction type so at the time you know we had protocols that had 55 transaction types I even worked for one of them for a few weeks um after I worked one of them for a few weeks like another two transaction types got added um good old contracts for difference so even still you know it's pretty obvious I think from especially from the standpoint of uh our current generation that this particular way of doing things is just fundamentally limited so what's the solution basically something like a smartphone so you know the idea is that instead of

[Applause] so instead of having you know a a protocol that's designed around one very small set of use cases you just create a general purpose operating system and you let people build whatever they want as applications on top of it so you know if you do that then things suddenly become better you know people maybe when you're inventing the protocol you might be thinking that okay people are going to use this for binary options contracts for difference making making two-party bets doing domain registrations and whatever other 50 applications you are aware of at the time then you know release the white paper and six weeks later you realize

that people are inventing applications in a completely different direction and guess what your protocol does not even have to change one single bit in order to handle them so That's The Power of generality so what is the concept behind ethereum well it's a blockchain see them for six years pretty boring with a few tiny additions so the first important one is this concept of a built-in programming language so in ethereum there is a kind of a builtin turing Turing complete scripting scripting language it's essentially a hybrid between kind of

standard virtual machine architectures bit Bitcoin script and and a few other things and the point is that you know people can write programs in this script or more realistically in high level languages that compile down to the script then people take take their compiled scripts put them into transactions and send the transactions off to the blockchain transaction gets confirmed an address gets generated and you now have a special kind of account at that address called a contract so two types of accounts user accounts controlled by private Keys you know we all the same old tic curve digital signature algorithm we all know and love

second kind of account contracts controlled by code so these two types of accounts in ethereum essentially have completely equal privileges anything that humans have the right to do bots have the right to do so anyone can create an application with any rules by defining it as a contract so this is essentially the basic principle so you know the whole world of ethereum are creating a decentralized DNS system so basically you know the idea here is he wants to create a system where people can register addresses and if they you know addresses could be you

know something something. eth and if they own an address they should be able to set the IP address associated with that domain address so the way the system works is fairly simple so you see a sort of Declaration at the top data domains with an owner basically a mapping between a domain name and a tuple containing an owner and an IP address and you have two functions one of them for registering a domain which basically says if this domain doesn't have an owner yet then sends the owner to or set the owner to whoever sends that particular transaction and you have another function set IP address which basically means if you own a domain then you set the IP address to what whatever you want to set it to so you know fa

very simple rule set this is you know in of course you know real life DNS systems have substantially more more features to them but sort of the the fundamental core basically stays the same and you know you could probably build a more complex DNS system on top of this just by turning the IP address into a hash and and throwing everything else on ipfs if you really wants to but point is very very simple logic you can write it in 10 line in 10 lines of serpent code and publish it to e the ethereum blockchain and there you have a system that's basically like namecoin so you know this is basically the meaning of you know anyone can create an application with any rules once once this application gets created then

anyone can interact with the application by sending transactions that specify the contract's address as their destination add address so you send a transaction to a contract and that's how you talk to the application that's how you call functions that's how you do you know whatever it is that you want to do so in Bitcoin the state is very is very simple it's basically just a mapping of you sort of a well technically speaking the sort of list of unspent transaction outputs but what it basically means in practice is it's the list of balances the list of sort of the amount of Bitcoins that every particular person has and if you want to do a currency system that's basically all you need in ethereum the state is kind of more

complex it's this sort of key value mapping going from addresses to account objects and there's two types of account objects one of them is the sort of simple accounts controlled by a private key in that case all you have is a nons balance the second kind is accounts that are controlled by code and in that case you have these extra two fields that get filled up one of them is the code hash so the hash the hash of the code of that particular account and the second is the storage tree route which in a sort of Fairly intricate way that I'll describe later basically represents the kind of the entire set of storage of that particular account so for example if I

create a contract with this code and it register a domain then the fact that one particular domain got registered to my and it got registered with me being the owner and then they said the IP address to some particular value that information all goes into the sort of persistent storage of this particular account so basically you know mapping from addresses to block blocks of this is the entire state of the ethereum system at any particular time so code execution every transaction specifies a two address that it's sending to so you know this is fair fairly simple it's the kind of logic that we all understand if you set the message then you generally

specify who you're sending it to and you want to specify the contents so what happens if you set a transaction to the two address well two options one of them is that the destination address is just a private key controlled account in that case all you're doing is moving ether around so if you do that then it's basically just being a cryptocurrency what if you're sending to an account that has code then what that's doing is it's basically activating the code and it's letting the C and it's actually letting the code run for for some number of steps so code in ethereum can do three things number one you can just send ether to other contracts number two you can read and write from the contract's own

storage so for example if you go in here then the way that I would call the register function is by sending a transaction with data formatted in a particular way and what that function is going to do is or rather what that transaction is going to do is it's going to start running the code that actually represents the register method and where it specifies self. domains address. owner equals MSG out sender that's setting a variable what that actually gets compiled to is setting a particular key in in the contract's own storage so then the third thing code can do is you can call or basically sort of create these sort of virtual transactions that go to other contracts so theoretically

you know I could create another contra contct that has some different set of rules and that contract itself can actually interact with this contract that contract itself can register domains they can set an IP address of domains it could you know theoretically do just about anything that an account controlled by an address can do so every full node on the blockchain processes every transaction and stores the entire State just like Bitcoin it's bold because it's important so this is sort of one of those one of those points that's importance to make because you know people often ask well if I publish transactions to ethereum you know who's actually well you know who so who's actually going to be enforcing the

contracts and who and and how many nodes are running them and so forth you know four theorum 1.0 the answer is very simple every node in the network runs every transaction and that Bas and the process of running a transaction in you know if that transaction goes to a particular contract it actually involves executing contract code and you know setting and sort of changing the state to whatever the state is after the contract gets executed so from that standpoint you can actually think of it as being a ra a rather fairly simple system you know you can think of the state as being a database and you can think of each of these contracts as as as being programs that are sitting on one computer except the computer is

massively globally distributed it's actually a highly secure network backed by tens of thousands of computers around the world so it's B because it's important so gas this is one of those sort of key mechanisms underlying the security of a theum so when you tell people that you have a Terin complete programming language and that you let untrusted parties send code that's that's executed in a ter complete programming language by every single computer in an entire network the first thing that they're going to ask is well what about the halting problem so there's this Pro there is this very widely known result in computer science which basically says that it is mathematically impossible to create an

algorithm which can tell whether or not any given program is just going to keep on running forever so how does ethereum prevent attackers from creating programs that just keep on running on everyone's computers forever and the answer is this concept of gas so what this basically means is that the ethereum network charges a fee for every computational step that a that a contract execution takes so the specific way that that's implemented is that every block in the ethereum blockchain has a gas limit so basically a limit on computational steps right now the gas limit is set to 31415 92 which is which is a nice number that we picked because we're totally not math Geeks and so the then you know if you're

creating a if you are a minor and you're creating a block then chances are you're going to want to fill it with the fill it with the transactions that give you the most profit so trans every transaction specifies a gas price basically an amount of ether that it's willing to pay per unit of gas and it specifies the maximum amount of gas that it's willing to take so a transaction might specify I am paying 50 shenon which is basically 50 billionth of an ether per unit gas and I'm willing to and I'm going to consume up to 500,000 gas minor see all these transactions coming in they select the ones that have the highest gas price and they include them in their block so when transaction specifies you know that it's willing to spend up to 200,000

gas then what ha what there's two things that could happen one of them is that execution takes up less than 200,000 gas in which case everything happens normally it's okay the the transaction pays only for the computational steps that it uses second case it goes over the limit if that happens then all the execution gets reverted and the reason why that has to be done is because if execution could stop halfway through then you might be able to sort of perform weird attacks on contracts we try to like set the G the execution to expire just at the time when like one half of the of the changes gets made but the other half doesn't get made and you know we kind of prefer developers to be able to just not worry about that and so

all the execution gets reverted but the transaction still has to pay for the gas anyway so don't run out of gas um there's also gas isn't just a sort of simple one one computational step one gas thing there are some operations for example operations that increase the amount of storage that every node has to store and those operations take up a very large amounts of gas so the gas system is also this sort of Fairly intricate set of incentives that are designed in order to encourage Bas you using the resources of the ethereum system responsibly SO gas limit is sort of the counterpart of the block size limit in Bitcoin how is the gas limit set fair fairly simple voting mechanism right now

if you mine a block then you can up or up Vote or down vote the guess limit by a factor maximum a factor of 1 over 1024 there is a default strategy that we currently sort of instruct all the clients to follow in the source code which is you have a minimum of 31415 92 and if the usage starts bubbling up and hitting close to the guas Limit then we target 150% of the long-term exponential moving average of the gas us so basically sort of Target it so that if it does get filled up the gas limit is going to be sort of just a bit higher than what than what what blocks normally contain so there are alternative strategies that we're exploring so one

of them is say for example Target a particular Uncle raate so what uncos are I'll explain a bit later but the general concept is that the more the blockchain gets clogged up then basically the more blocks and the slower blocks propagate and so the more blocks kind of get lost and don't end up being in the main chain and as it turns out thanks to a future of the ethereum protocol you can actually calculate how much this is happening and so you can kind of Target a particular ex amount to which it's happening and you can say if we're seeing more than let's say 20% waste from blocks being uh just falling off the network then the gas limit should go down so there are other strategies that we're considering so transactions contain basically seven sets of values

number one so what a non does is basically every every transaction has to have a has to have a unique incrementing nons so the first transaction you send from an account has to have a Nots of zero second transaction you send has to have a nons of one third has to have a Nots of two and so forth and the reason this is done is to prevent re replay attacks so if Alice sends ether 10 ether to Bob you don't want Bob to take that transaction and reinserted into the blockchain 10 times to give himself another 100 ether nonis fixed that gas price the amount of E ether per unit gas already already covered that start gas same thing to the address of the the account that you're sending the transaction to or you know the contract you're calling value the amount of ether

that you're sending data is just a field of data the transaction sender can fill the data field with whatever he wants it's just an array of bytes no limits on length but the more there the more data a transaction includes there is actually a fee for that in gas so basically you lose 68 you have to pay an extra 68 gas for every bite of transaction data that you include vrs there are values in an elliptic curve signature so the if you're familiar with cryptography you've probably heard of seen the r and the S the Val value is basically an extra flag needed to do public key recovery so basically instead of checking the signature against the public key you actually derive the public key from the

signature so there is this concept called of receipts so receipts are objects that kind of get hashed into the ethereum block into the ethereum blockchain and every transaction has a corresponding receipt so these receipts have a few pieces of data one of them is an intermediate State Route which once again I'll describe the mechanism later but it's a kind of hash that represents the entire state after the transaction gets executed number two cumulative gas Ed so the total amount of gas used in that particular block including that transaction so those first two things are kind of needed to do this sort of step-by-step accounting that that actually makes it possible to kind of check the validity

of trans of individual transactions very easily and the third thing is logs so logs are this special feature in ethereum that is is sort of it's kind of a different kind of storage so in general you know when you set variables that gets set in account storage and that can be read or written by contract s logs are basically aend only so if you have if you have a transaction in one particular block and it creates some logs those logs get hashed into that particular block but they are not visible they cannot be accessed by contracts and they appear just in that block they're not in any kind of persistent state so nodes if they want to for example they can just not store

logs and they can prun and they can prune them so logs are logs are also 10 times cheaper in terms of gas consumption than storage and the purpose of logs is to allow sort of efficient light client access to event records so one example of where you want to use a log is if you go back to the DNS example you might want to add a single line to create a log in the register method and in let's say the set IP method and the reason why you want to do that is basically in order to allow light clients to very quickly scan through all the logs and sort of see all the instances at which domain you know the domains got registered IP address got

changed and so forth so there's lots of applications that wants to be able to sort of have a history of things that happened and logs are a convenient way of do of of supporting that so logs have up to have a data field and they have up to four topics and there is a protocol that uses Bloom filters to make it very easy for light clients to kind of search through logs so basically if you create a log that has a particular topic then you can very easily search through the entire blockchain and sort of identify all of the logs that's that match that particular topic so ethereum virtual machine so once again as I mentioned it's this sort of combination of Stack based architectures

traditional virtual machines St so inside of the virtual machine you have the stack basically you know 32 32 by Fields up to a maximum of 1,24 of them memory just it's an infinitely expanding bite array but the more you expand the bite array the more gas you have to pay so most of the limits in ethereum they aren't sort of static they economic and that's just a standard thing you're going to see again and again storage permanent contract storage you can read and write to it environment variables so for example the virtual machine can access the block number the time the mining difficulty previous blog hash and a whole bunch of other data so one one possible use case of that is let's say if you have a some

kind of financial derivative that can only be processed or you know called struck or whatever after some particular time then you can you can say in the contract you can check the current the the current block time stamp against you know some particular limit that you set um logs so the virtual machine can create these logs and there is an OP code by which the virtual machine can itself call other contracts so this is basically the environment within which this this contract code gets executed now you don't have to write in virtual machine bite code yourself there's highle languages so and one of these highle languages is called serpent so this is this is one of

them there's also one there's one called solidity which is being you being used much more right now there's one called L which is kind of much more low level if you wants to like really go into low low level code and be as efficient as possible you might want to use lll although you know in my own opinion you you should probably consider serpent as well because serpent lets you go as level as LOL does and it has high level features as well so you know basically you write code in these languages then you have compilers and you compile the contracts into bite code and you can publish the you you can publish the B bite code in a transaction going to the blockchain the ABI so when you send a transaction let's say that I

wants to call the register method of that particular DNS contract that we keep on using as an example function calls get compiled into transaction data so in this case the function call gets the way that it gets encoded is the four but the first four bytes are function ID the next 32 bytes are the first the first argument so this was the um the domain that you that I wanted to change the IP address of and the second the second argument is another 32 by which is the IP address that you want to set it to so every time you call a function of a contract in ethereum it actually gets compiled into a transaction which is encoded in so in using this algorithm which you know in simple cases looks

like this just four byes 32 byes keep on going with 32 bytes but there are some special cases involving sort of variable length lists and so forth everything in ethereum relies on this one serialization algorithm called rlp recursive length prefix and coding so it's very simple so basically you know if you you know it's recursive length prefix encoding so you take the thing so in this case if you take the word dog and you just hex encode it you get 646 f67 and you before it you pref you prefix it with the length so in this case the length is three and you you can see a three here over here you know it's an empty string and so you have a length of zero over here this is a list and so

it still has a length of zero but it has a different sort of start flag because it's meant to be a list and not a string this is the uh for people who who are set theory Geeks this is a set theoretic representation of the number two it gets compiled to this or encoded to this rather if you have a string then you know gets compal it gets serialized into this so there are like five implementations of this there's a much more detailed description on the Wei but basically this algorithm is used for blocks transactions Network protocol literally sort of everything in ethereum land um ethereum also has a cool new mining algorithm so the goal of this so This is called eash so the purpose of

this is to be GPU friendly and Asic hard so to Target gpus as much as much as possible and basically prevent the the problem in Bitcoin where you have these sort of spe these uh specialized miners that have with extremely large mining Farms taking up huge percentages of the network so the algorithm is uses memory hardness to accomplish this goal so basically instead of traditional proof of work algorithms like Bitcoin where the bottleneck is always how much computation you can do here the bottleneck is memory access and the theory behind this is that the more that memory is something which is highly optimized in which people spend billions of dollars optimizing in existing

computers already and so the opportunity to make further opt optimiz ation istically smaller than it is for computation so the way that it does this is by using this sort of clever multi dead construction so basically the idea is that you start off with a seed then from the seed you use this memory hard function create a cache then from the cache you create a data set and the data set can be created in parallel so each individual piece of the data set actually gets created by hashing together 256 parents that come from the cache and if you want to just actually run the algorithm once then what you need to do is you take the data that you're mining on turn that into a seed then you grab a

random 64 nodes from the pieces of the data set and you get your result so what's the point of this the point of this is that mining with this algorithm actually is memory hard because if you wants to mine and if he wants to mine at any reasonable speed at all then you basically have to pregenerate this entire 1 Gab data set so that every time you pick a different non you can just sort of sample from it and that happens fairly quickly if you just wants to verify blocks then you don't actually need the gigabyte all you need is the cache and the reason why is basically that if when you get to this stage you don't need to create this entire thing you just figure out what 64 indices you want then from each of those 64 indices you just computed from the

cach now this does require you to make 16,384 hash calcul calulations but or as it turns out that's actually not that bad if you're only doing it once so because verification only needs to happen once this kind of low memory slow verification process actually works so but if he wants to mine then you need to do it with a lot of speed and so he needs to have the gigabyte so this is kind of the basic principle behind sort of why mining was designed in the particular way that it was an ethereum so uncles so in general Bitcoin has a 10-minute block time ethereum has a 17sec block time why is the block time not something even

shorter like 1 second the problem with fast block times is basically steale rates due to network latency so if the block time is extremely long like 10 minutes you can think of network latency as being negligible by comparison you can think of it as basically being instant and so you basically don't have problems once you start pushing block times down then your problem is that you have this risk that let's say over here we have one block you know 2059 then at some point someone creates another block 2060 that block takes time to propagate through the network and so someone else might actually make another block before hearing about this block and so we have two blocks both on top of the same block 2059 and they're kind of both competing with each other so this is called a fork

and the problem is that one of the blocks has to eventually win and the block that doesn't win just kind of gets discarded and the person who mind it doesn't get a reward so so this leads to centralization risks so basically the issue is that the more powerful you are as a miner the the better you are at getting high network connectivity and if you created the last block then you don't have the you actually don't have this risk because you can just immediately start making the next block so because of the more these network connectivity issues become important the more problematic these centralization issues become and so you have this effect where large miners end up being

more effective than smaller miners so our solution is this mechanism of uncles which is basically that if this happens then sure one of these one of these blocks is going to win But whichever whichever block loses it can kind of get reincluded in the blockchain and it still receives most of the reward so the point of this is it basically Cuts these centralization incentives by about 85% and it kind of allows the ethereum blockchain to continue to be to be relatively safe under somewhat harsher circumstances both in terms of fast block time and in terms of high load Merkel trees so how many people have seen The Economist cover from life

master saying it's it's all about the blockchain well my opinion is it's all about the marel tree so and particularly unlike life Master's dogs I like does more so the point of Merkel trees is that there this construction that allows for compact efficiently verifiable proofs that a particular transaction was included in a particular block so the the general principle is that you know in this is actually exist in Bitcoin if let's say every Blog has a few hundred transactions then if he wants to verify that one particular

transaction wasn't included in a block then you don't have to download the entire block you all you need to do instead is you can kind of download just one sort of branch of this tree and the tree is this kind of hash tree where every single sort of node in the tree is actually the hash of the two nodes below it and so all you need to do is just download this kind of Branch just download the specific part of the tree that's going down to your particular transaction and you check the hashes that are just going up the branch you don't check all the hashes just the hashes that are relevant to your particular branch and if all of those hashes check out fine then you know that your transaction that particular transaction actually is in that particular tree and still actually is in

that particular block so the reason why this mechanism exists is basically for white clients so what a light client basically is is a client that accesses a blockchain without processing every single block without downloading every single transaction why do you want this well it's pretty obvious you know the the Bitcoin blockchain right now is 30 GB is 30 GB and you know about a megabyte gets downloaded every 10 minutes it might be good enough for a high powerered laptop it's probably not good enough for this and it's probably you know and 30 gabes is probably too much for the devices that slock.it is going to be creating so instead you use these clever like you use these sort of clover tree protocols in order to allow these devices to only verify the

absolute minimum of information that they have to that's relevance to their particular transactions so in ethereum the Merkel tree mechanism is actually some quite a lot more intricate than it is in Bitcoin so the idea with ethereum is that it's not just transactions that get stuck into Merle trees it's also the state so if you hear about this concept of a state root what that basically is is this sort of is This root hash and from this root hash you have another tree and the tree goes and the tree goes down contains hashes of hashes of hashes of hashes of data representing the state of of basically every single account in ethereum so you have these account objects at somewhere

at the bottom of the tree you know you have nons balance code hash and storage and storage is yet another tree so the point here is that you know if one of these one of these white clients whether it's my phone or a SW walk.it device or whatever else wants to know what the status of one particular contract is it doesn't have to process the entire blockchain it doesn't have to trust anyone all that it needs to do is just ask for a mirle proof ask for this sort of branch of the tree and verify just that one particular Branch so this is basically the sort of concept of the ethereum white client um where result for foldy is actually going to be presenting quite a bit more about this I believe after lunch

today so every block header actually contains three trees transaction State and receipts so transactions are transactions we know what those are are the state is every you know the entire state of every account balance storage every all the sort of static information that that you need to know and receipts are they consist of this sort of extra auditing information and they have all the logs so if your light client wants to know what wants to know you know what particular logs are let say associated with a particular topic then the light client is going to ask for meral proofs of logs so future directions we talked quite a bit

about this on Monday but the general principles the general categories are number one proof of stake so where proof of stake is going to fit in is basically just at the top so it's just a change to the consensus algorithm so instead of blog instead of users having to verif having to verify proof of work they're going to verify validator signatures instead and the way that you would join the valid validator set is you know as what described by submit a secur submitting a security Bond so that entire mechanism is going to either be part of a protocol or it's going to be sort of specified in one particular contract not fully decided yet and from a user perspective all that's going to

change is that instead of validating a proof of work NS you're going to have a list of signatures that you validate instead scale blockchain rent so the idea with blockchain rent is basically that right now you can access storage key you can your code can set storage key ke and those storage keys will remain there forever but potentially this might present very present sort of serious long-term risks because the amount of storage could that the ethereum blockchain stores could just keep on going up and up and there's not enough incentive in some people's views to to kind of to kind of delete storage there is a bit of an incentive because if you kind of delete stor if you delete storage then you kind of get a refund but even still there's sort of no

incentive not to kind of use storage as a bank so blockchain rent is the idea that every contract is going to have to pay some amount per month for every storage key that it has so instead of storage being permanent you know I could imagine it lasting one year two years five years you know if this is going to happen it's going it's going to be as reasonable as possible but that's the kind of basic principle virtual machine upgrades so that's just going to not going to change any of the other semantics it's going to swap out the virtual machine that the code runs in so in practice for developers is going to mean very little because theoretically you could still program in the same languages we'll just have a different compiler that compiles

a different bite code but if he wants to do things faster then you could potentially I mean ideally even program ethereum code and things like C++ and Java scalability talks about that on Monday so you know the idea that of kind of having not every node process every transaction but I mean right now as far as the ethereum protocol go all of this is basically it thank you