Consensus Protocol Testing [EN] [Ethereum Protocol Day]

Transcript

Hello everyone. Hello. Um, thank you for for inviting me. Thank you for the team that organized this amazing event and that we're having here in Shenzhen. I remember the last time I was in this same building. I think was like the anniversary of the app learning. Do you remember the we did what you also presented here the anniversary the three years of the learning I think or how many years last time? Last time here >> three years. three years of dab learning we did the presentation here. So, I'm very happy to to be here again. As I said, I I am um I'm Leo. I I work at Steel and I'm going to talk a bit about my job. Basically, very informal talk.

Interrupt me if you have any questions. It's all right. Uh I work in Ethereum Foundation in in in a team called Steel that is specifications and testing for every Ethereum layer. Before every was execution but now we have every Ethereum layer and then the first on the consensus layer and what we do is to kind of um let me work on the coordinating all the clients on all the specifications and all the testing between the clients of the Ethereum uh protocol.

So before talk a bit about no before in the proof work era we have before the before the merge uh yeah I'm going to talk a bit about the the contestious protocol before the merge we have like this one block that have was mostly the transaction and a small part was the proof of work know that made work that was very simple was basically h the same than bitcoin and it was very simple but has like certain disadvantages that now we have solved with the proof of proof of estake that we have since the merge I think almost four five years already of proof of estake no and now in the proof of estake

we have this block have like a block inside it that is the execution layer uh block with the transactions and then this contained by other block that is the consensus layer uh proof of estate fake transaction. No. And this is the the execution layer have the transactions what people normally kind of think about Ethereum and interface with Ethereum and the consensus layer have this all these utility to build the blockchain using proof of a stake. No, like it's about validators at the station for choice.

You know here the the actors that build the chain and and decide about the chains are called validators that are nodes that have a stake. No, they have put like a treasury like a bond to uh contribute to the network. So they have like a financial interest to make it work correctly and have like financial incentives to do that. But that's those financial incentives are not like in the proof of a stake. The financial incentives were based on energy cost and hardware cost. Now this these incentives are based on actually holding Ethereum.

So it's kind of create better incentives we believe. No. And these validators kind of mainly uh have two roles to play. They play both roles. Sometimes they are testers and sometimes they are proposers. And when these roles and choosing for the validator is based on a pseudo random algorithm called the the randow that is based on the all the previous blocks. So you cannot really uh tamper with it because you need to kind of like be able to uh produce a lot of blocks in uh in succession that that will be be financially impossible and very unlikely. Um uh for some reason sorry

I hope Yeah. So this is cool. Yeah, proof of stake. Uh yeah, everyone if you have any question let me know. So to run Ethereum node uh basically you need to run two software an Ethereum uh execution client and a consensus client. And there is something in Ethereum that also another advantage with respect with Bitcoin and other protocols. We don't have just one implementation of this client. we have what we call client diversity. So, so the execution client there are five main implementations all in different programming languages all done by different teams and the

consensus client the the consensus client has six implementations all done by different teams. Some languages I think repeated like uh I think grand and and and lighthouse use the same but programming languages. So I think five programming languages by six different teams they kind of implement the same protocol and that's why where my team still uh kind of uh comes they have a very cool logo a very cool cool name uh kind of comes into play to to organize this to kind of have like a implementation of the core of the protocol in Python that is not really efficient enough to be used, but it's

kind of the the reference that all of them use to implement them and then create from there a series of tests that all of them can use to test the with the same test uh test all these all these seven uh no sorry yeah all all these 11 softwares based on the same test uh yeah test vectors Cool. Yeah. So one one of the most important things that the that the consensus layer does is the fork choice is when there is like a a split in the in the network like in the

split in the chain like the the suddenly appear to chains uh decide in the end like make that all of them agree which is the the main the main chain. No. So for example like how this this could happen. I have here the the the like uh the the the chain is divided in a slots that they are they are uh currently uh 12 seconds each. So the time is divided in a slots of 12 seconds you know and

and because of the randow that explain decide which of the validator is the block proposer for each of the slots. No. So, so validator one is the block proposer for this slot and then creates like um a block and then validator two is the block proposer and creates another block but then I don't know like like world war 3 starts or something and the the internet like different continents cannot really connect or the latency becomes very huge. So, so validator 3 uh creates a block. But this when validator 4 is the proposer for this slot, it's going to create a block

hasn't seen this because the network is split for example and then it will create a block with this parent because hasn't seen this uh soon enough. No, because it has an allotted time to create the block and then from there perhaps like two different uh chains start to be no no wouldn't be here because this slot is is cannot be readable by the validator but then two different chains start to propagate not to to to be created. Yeah.

>> Like a fork to me. >> Yes. Yes. So that's like a fork and and the the most important thing the more important role of the consensus layer is the for the fork choice. So it's kind of choose which one of them we have to choose like kind of like make in the end all the nodes agree on which of them should be chosen. Um yes this I'm sorry.

So yes, so the there is two algorithms that the consensus layer improve state uses. There is the the Casper FFG and the LMD ghost at the same time they are connected. The Casper LFG is kind of more high level. So it decides there is like what we call a finalized checkpoint and justified checkpoint. So the when we get to a note gets to a finalized checkpoint that is really definitive. It cannot be changed ever. So the all the four choice options from before can be completely not always and this is kind of complete finality for Ethereum the

finalist checkpoint. And then there is another justified set point that is after the feralization point always. And for the time being as long this is difficult to change but as long this doesn't change the the nodes are going to ignore the rest of the of the possible forks. Okay. And then from the justify chain point onwards the the nodes are applied the the LM LMD ghosts. So then kind of go in more like low-level detail and this make it efficient. No because like uh the LMD ghost has goes to count the stations in

each block etc. So by the fact of only doing it from the justify chain point onwards we we couldn't do it for the whole blockchain. No. So we just do it with the justified checkpoint always. So and then basically from the stations the LMD goes kind of give a weight. So see the like count the stations that are like the other validators that are not the proposes create the stations from different blocks and assign them and basically the the LMD ghost um the the LMD ghost uh kind of give a weight to each block depending on the

station and also depending depending on the on the balance in Ethereum that each of the the attestations come from an so each attestation come from a validator and the validator have a balance so if you have more balance you have more weight in your at the station there's basically a boat for which is the the the chain to choose no and then based on this LMD goal algorithm of taking account that the stations and the balance of the testers is ahead head is chosen and eventually all of them converge on this decision because they all start seeing the messages from the rest and yeah that's kind of like what the

most important thing that the consensus layer does. No, the consensus layer also kind of keep the accounting of who are the validators, how much balance they have etc. But all this is done in with the intention of being able to do this other thing that is kind of doing the for choice based on the of the validator the stations and the and their balances. Yeah. Is that okay? Yeah. Um so basically what is still does to know more about this? No, there are a lot of AIPs, no the Ethereum in improvement

proposals and that that really create the each version of the of the protocol. We need to think like Ethereum do huge upgrades like every every year and probably in the future twice per year like a very um complex upgrade to the protocol and the new features like we lately we had blobs that help like makes L2s cheaper. All these very big changes, they are codified in IIB. And what my team does kind of like codifies all these into uh what we call a specification in Python

that all these uh clients software are going to use to implement uh in their own language that is not Python, in Rust, in Go etc. But they are going to base on the same source that is Python. So that's kind of helps guarantee that in the end they are going to do all the same protocol. No, this is the main difficulty have like these six uh implementation exactly doing things exactly the same and also then we build these tests this kind of like it's not unit tests exactly but kind of like unit tests in Python as well for this spec. So that we we we see

that the spec pass this test but then from this test itself we extract the data the actual values that are being tested and we create like test vectors files etc that all of these clients consume. So they have like this kind of bank of tests that the all of them consume and all of them run on their own clients and that way they make sure that they are all following the the spec and they are all passing the same test and it's kind of converging completely. And they all like run we run deaf nets that are kind of like small blockchains that are running on all the different clients

at the same time. many times, many deathets around before a new version of the protocol is created and is launched and and make sure that all this work. And we see the problems here. And the problems that come here can revert into uh more tests here. And the the client teams also contribute to the to the tests. And then this goes like to all the clients again to run the tests or even kind of improve the IPS and the specifications. And we see like there is like a bug in the specification and the specification needs to be changed.

Um yes. Um so here I have an example. So for example this is the process at the station in the spec. So this is in Python. No. And is the actually the code of how the the client should process the station. So we write in Python and then all the clients are kind of in their own and more optimized because this is more implemented for readability uh etc in Python and it's not efficient but more like looking more into efficiency they implement we expect that they implement the same algorithm. Okay. And then we have tests like for example this one that uh test

this this method test this method like you do like in any normal unit testing you have a method you put different inputs different outputs and you you check that you are agreeing with the output that should has no and then ex more on that in this case like this will create like like three files this ter just test This it test this function in Python but also create these files with the with the pre state of the chain uh serialized before calling the method and the state of the chain after calling the method are actually the station that is the

input to the to the to the to the method. So that way thousands of these are shared with the client teams and all of them are going to uh test their own process at estation function uh with this same data. Um yeah and that's kind of the the story of what we do and if you have any questions let me know. Uh, are the tests not verified? So, it's more like uh you you guys find something that breaks

and then you write to the test. >> No, but we usually for any new feature and any EIP when we try that it comes with tests. So, it's not we don't expect like the dev nest to fail uh to write the tests. So we try like all of them have enough tests and try to preemptively write test that kind of test corner cases etc. But yes of course like there are bugs and then uh they fix it but they also want to add the test so it doesn't happen in the other clients.

No. >> Do you include any form of formal verification? >> No. I think there is uh some people working on formal verification of the Ethereum protocol but not in my team. >> Okay. >> Yeah. [Music] dev net. >> Uh there is that's another thing. There is some there is a team that is called if panda ops that run the dev nets.

There's like a devops team but yes like they have like many nodes of all the clients but I don't know the amount they they have. I I don't know how how many they they run but the defets usually are internal. So it's not like sepole and all that that anyone can participate but the dem are usually like the client teams run them like their clients I think with if panda ops that is another team in ef together they they they they run a cluster of of clients >> I see okay I have one more question uh I know the dev write the spec python script then You you write the test. Uh

yeah. >> Yes. >> How can the different clients uh change? You consume your test script? >> How they consume? They they grade their own consumption. So the test in them. So going >> which language you use? >> Python. Python. >> No, but all the clients use different languages. >> Yeah. So let's go to your question like we write the test like the so what we try like first like the AIP the IP griter we tried that them like send the the the version of the AIP as a Python spec with Python tests already. So they

send it like submitted to the repository in in that way with the yeah with the Python spec and the Python test. And then the second question kind of like so we generate files in like serialized uh serialized um uh objects like a state at the stations blocks in a folder. It's test like in a folder and usually there is a metaal meta a YAML YAML >> YAML file with more information and each of these folders is one test and each of the client team have to implement in their client a consumer for this format.

So each of the if the client teams have a a consumer for this format. Yeah. Cons consumer. Yes. for this one. Yeah. >> Different team need to different consumer. >> Yeah. So each team has its own consumer for the same format. Yeah. For the common format that is produced from the Python tests. Yeah. [Music] >> Hello. Uh so my question is the the beacon API stack is covered in a separate record, right? Is it also maintained by steel team or >> Yes. So we are with the I mean it's

something very collective no like is in the Ethereum organization and and is kind of there is one repo. So the the execution is in two repos there is one for the test and one for the specs. In the consensus we have one ripple that is called Ethereum/ consensus spec and and that will have the specs and the tests and it's really like there is the EF security team still like me from steel because I'm the only one in the consensus in steel steel because steel was born in execution and then also uh people from other client teams uh also uh kind of maintainers and and contributors. So

it's like it's not um it's a decentralized as much as it can. I mean like there is like I think six maintainers from different organizations. It's not Yeah. It's not we try at Ethereum we try that everything is is very decentralized and kind of like not Yeah. >> So my question about the beacon API spec is it only provide the open API schema. But it doesn't provide any test or test data like >> I I think you talk about the RPC >> the RPC API end points.

>> Yes. Yes. But that that's where we we we handle the the communication between the nodes but that's kind of like the the main part that actually the the rich consensus the nodes between themselves no more than like a client software communicating with the node. You know what I mean? Yeah. >> Yeah. >> I don't know the test the how to reach consensus more like internal functions. >> Yeah. But that that doesn't go Yeah. That doesn't go through the RPC that's go with PertoP messages and all that.

Yeah. But we actually are not well other teams test the network but we kind of like test kind of the core of the logic without the networking. you know like like Yeah. >> Yeah. Yeah. What I'm trying to do I'm trying to to uh test the beacon API endpoints for new plan called uh ring. >> Yes. >> Is the problem is that there's no like like uh universal beacon API >> integration test like is there any plan to create such things?

>> I don't know. Yeah. I don't know. Yeah. because it's claring I guess they have the the RPC I think for the execution for the execution layer there is a common RPC no that the the client use but for the consensus layer perhaps there is not >> yeah but I find one for consensus >> yeah perhaps there is not for >> consensus team I mean they create some like test suit for for the econ [Music] it's I thing is like from the point of view of the wallets. No, if you're building a wallet, you only need to talk with the execution layer. So I guess they haven't thought about which type of

of applications want to talk to the bon node. Uh so I mean what you can do is to run your own node and choose uh Nimbus for example and use that one. Yeah. And use only the Nimbus one. Yeah. >> You mean uh maybe adapt the test API test from another client. >> Yeah. Like choose one client and use that. No, because like anyway because the clients are all communicating um through the through the B2B. So you you run your own client of the consensus layer. You communicate with that one and that's kind of like should give you the vision of Yeah. the the network. Yeah.

>> Cool. Thank you. >> I guess it depends on the application. Yeah. [Music] [Music] [Music] Thank you.

[Applause] [Music] Very [Music] good.