Deep dive into Devp2p and how you can implement it

Transcript

And is it working? Yes? Okay. Hello everyone. Good morning. I hope you're feeling well. And well, welcome to this workshop about DevP2P. Okay, who is familiar with DevP2P already? All right, one person, two? Okay, nice. So, maybe you are not going to learn new things, but I hope you are going to enjoy the next session. Uh first, let me tell you who I am. Let me introduce myself. I'm Lola Rigollet-Zack. I'm a software engineer.

Uh I have been for the last 10 years. I've discovered Bitcoin and Ethereum back in 2013. I was studying cryptography at uni and someone started to talk about crypto. I was like, "Nice, what is this? Is it about cryptography?" And it was about Bitcoin and Ethereum. And I've since been kind of obsessed with the topic and every topic around uh peer-to-peer technology and protocols. So, things like BitTorrent, ZeroNet, Bitcoin, obviously, the original cash peer-to-peer, uh DevP2P, LibP2P. Also, you might have noticed I'm French.

I'm really sorry for the accent. I'm sorry to mispronounce some word. Uh and I really thank you to be here to this morning. All right, let's start. So, this workshop will be in two parts. First part, we are going to see theoretically uh peer-to-peer in Ethereum. And the second part is going to be a workshop. So, unfortunately, sorry, it's not going to be exactly how to implement this, but don't worry, I still have something for you guys. But, we are going to do something a little bit together and we are going to try to figure out what can we build with what we just learned. So, you will need to contribute.

All right. So, first part, peer-to-peer in Ethereum. Okay, little survey. Um How does peer-to-peer uh How does Ethereum nodes talk between them? When you think you hear the correct answer, please raise your hand. There is several answer and there might be several correct answer. First answer, using TCP. Okay two three four. Someone is not sure? Uh also, please, we are a small group, so if you have question, uh please raise your hand during the presentation. We can make it dynamically. It's more fun like this, I think.

All right, second answer, using JSON-RPC. Nobody is raising your hand? Okay, okay. Uh using DevP2P. Okay, almost everyone. Using LibP2P. Uh >> [laughter] >> Someone is not sure. Uh and well, the correct answer are actually A, C, and D. So, yeah, all of you, you didn't fall for the for the wrong answer, which was JSON-RPC.

All right. So, I've already talked twice about DevP2P, but what the hell is DevP2P? DevP2P is a set of uh Ethereum peer-to-peer protocol, but only used for the execution clients. So, it was a introduced uh by uh in Ethereum at the beginning, right? Uh And it allow you for it allow like to do different things. Uh like finding other nodes, thinking blocks. Uh it's also allowing light clients. At least, that's something that Ethereum is trying to do, is still trying to do.

Uh sharing uh state snapshots. Uh also, access to the mempool transaction. So, you know, when you have a new transaction, a node talk to them, they share it between them, and then it's added into the blockchain. Uh also, get transaction receipt within being the result of this transaction once it's being added to the blockchain. Uh is it uh successful or not? Do you have some logs? So, that kind of information that you can get through the DevP2P uh protocol. Also, some of them are encrypted and this is kind of important.

All right. So, what are those protocols? So, I took all the protocols that you can find in the documentation. You might have some more, but I don't think there is. Uh first one will be the RLPx, the transport protocol. This one must be supported by all nodes and they use RLP encoding. Is everyone familiar with RLP? It's It's a kind of a encoding for data for Ethereum. So, you have a specific field and then you make it into bytes, so you can send it over the network. Uh second protocol, there actually there is two, is Discv4 and Discv5. It's used for the peer discovery.

So, it's used to discover new peer, right? You connected to a node and you might find more node to connect to, so you can have a bigger network. Um it's Oh, yeah. So, this protocol use Kademlia like DHT to store information about the node. It's something that we can find actually in BitTorrent. So, when you want to find a file in BitTorrent, you will share a hash on using this algorithm, Kademlia, and you will find information about a peer who actually has this file and you can connect to it. So, they use something similar.

It's I'm not sure how we can justify to have this because, well, in Ethereum blockchain, everyone has the same data, right? But, I would be interested to know if any of you knows what this I see someone who is not entirely agreeing. And uh yeah, something interesting also, uh when you discover new peer, you actually don't know about which network they are from. And by that, I mean they might be mainnet, they might be testnet, but they might also be like other network entirely. As you know, lots of project have forked Bitcoin uh forked Ethereum, sorry.

And they they are also using this peer-to-peer uh discovery protocol and so you might have like Base, Polygon, uh Binance. So, you are not sure that the node you are going to have is the one that you want to connect to. It's part of your network, right? All right. Uh other pro- protocol, which is not really a protocol, it's an open format. So, when you It's called ENR, Ethereum Node Record. Sorry. [clears throat] So, when you have uh when you have requested a peer, you will have a message on the format of the Ethereum Node Record, which will give you information about the node like it's public key, uh IPv4 address, TCP and UDP

port, and IPv6 address. So, everything that you need to connect to it then after. And it's also RLP encoded. Okay, last one, uh which is a DNS-Disc for DNS peer, like to discover new DNS new peers using DNS. Because when you have when you start a node, this node is not aware of any peer, right? But, you still need something somewhere to start with. So, they hardcode a DNS, but because it's easier then to update to read from the DNS record and to update a list of peer than to do a new release of the client every time you want to restart or add new starting peers, right?

So, this is pretty useful. Okay. So, we are going to take talk now about the main protocol that we we want to understand, which is RLPx, the transport protocol. So, I'm going to read the description from the documentation. RLPx transport protocol, a TCP-based transport protocol is used in communication among Ethereum nodes. Okay. Now, that's why we are TCP because it's TCP-based. Uh this protocol uh carries encrypted message. All right. So, we will have encryption starting with this.

And uh it can belongs to one or more capabilities. So, capabilities is going to be a term that we are going to see a lot in the next in this presentation in the next slide. Uh it's basically subprotocol. All right. So, when using this transport protocol, you need to negotiate which subprotocol your node is talking and what is the other node you are trying to connect to is talking. So, it's not easy to find a match, right? Because let's say I speak French, you speak Spanish, we don't have any language in common, I cannot talk with you, right? I cannot exchange with you. It's going to be really difficult. It's kind of the same idea uh with capabilities. You need to have the same one.

And uh RLPx is named after the RLP serialization format because most of the message are RLP encoded and the name the name is not an acronym for anything. Okay. So, what kind of message can you send using RLPx? You have four message. Uh first message is the hello message, which is the most important one. You are going to uh exchange it automatically with all the peers and they are going to give you information about which version of RLPx they are using, the node ID, but also those capabilities, right? This is why this is the first step.

Uh second message. Okay, uh small thing. I have added a zero, it's because every message, the first byte of the message that you are going to receive is going to describe describe what kind of message it is. So, if you see zero, it's going to be to be a hello message. If you see one, it's going to be a disconnect message. All good? Okay. I see nodding. All good. Um so, okay. Disconnect message. Really important message because you are going to see it a lot of time. PR going to disconnect from you and you are going to disconnect from PR. So, for different reason, you actually don't share any capabilities. So, you cannot process to do anything more. Anything else.

Might be also you are not on the same network. So, the PR is going to disconnect because obviously you are no use to it. Uh could be also that you send a message which is wrongly encoded. So, it say, "Okay, I don't understand what you are talking about. I'm going to disconnect from you." Could be also because they already have too many PRs. So, they are going to say, "Sorry, dude. There is too many PRs that are talking to me. I'm going to let you go." Uh so, other reason for disconnect. Two last message are ping and pong. It's basically message to uh see if you are still alive. So, quite often node is going to send you a ping. You will need to answer a pong. If you

don't, what else is going to happen? Disconnect. Thank you. You are following. Thank you. Uh okay. I've put a link to the documentation where you can find all of this and even more. It's on the GitHub Ethereum devp2p repo. And there is a way more than than this slide and that I just say about. So, yeah. Go and find and take a look. All right. Capabilities. I'll talk about I mentioned already that they are sub protocol. Right? So, every capabilities, every sub protocol is designed for a specific feature functionality. Could be light client. Could be snapshots. Could be exchanging blocks.

And you can even create your own. Uh they all have version. It's kind of important because it will also enter into the phase where you negotiate between capabilities. You have a capability and its version. If you don't have the same version, well, same. Disconnect. Um and yeah, some uh some capabilities actually depend on other to work. So, you will need to have the both of them. Okay. Example of capabilities that we can find. First one is ETH.

It's used for exchanging uh blocks, new transaction, and network info. This is the most important one. Uh second one is LES. Used to be also in all the clients. It's for the light client Ethereum. Unfortunately, now it's been deprecated. So, it's being removed. You can only find it in the oldest node. Uh next one is SNAP. It's to facilitate the exchange of state snapshot between node. And this is an example of a capability that require another one because this one require ETH. And last one is an example of uh a company or foundation. I mean, Parity created their own uh capability that they are implementing in their own clients.

Uh it was also an attempt to do uh an Ethereum light client. And but now I think it's also deprecated and they have removed remove it. All right. Uh so, those four also you can find in the documentation, right? And uh yeah. I was um I was saying that we are going to talk about ETH which stands for Ethereum wire protocol which is the most important one. Because this is the one that you are going to use if you want to sync with uh so, get transaction and everything with other node.

Uh the current version of this protocol is ETH 69. So, in in um uh during its life on mainnet, we have seen like nine different version of this protocol. And something important is clients are not I mean, clients but I don't mean nodes. They are not going to support all of them, right? Most of them support the last one or the two or three last one. Uh which means that all this node cannot talk to the newest one. That kind of makes sense because lots of things have changed in the past 10 years. But yes. And the version before were like during the Ethereum development. So, we don't really care.

Okay. So, ETH. Uh what kind of message can you send over the network, right? And what kind of information can you actually gather? You you will have like kind of like two two kind of infor of things that you can do. First, you you can request for information. Uh you you will like see that the request to request an information, you have the prefix get. So, when you send a message which is get block headers, you will have an answer which is block headers. Okay? So, when you want to have some information, you send this and you will have the response right after.

You will also have like sometime have messages that you didn't ask for or subscribe for. It's because the network is relaying new information on the network and it's trying to give you information that it thinks you don't have. So, example, uh new block, new pull transaction, new block hash transaction. And there is one last message which is kind of particular. Uh this one is status. So, first thing that you do when you connect and over the this ETH sub protocol, is that you are going to exchange the status. And you have you have really really important information into it which is a network ID.

All right. We will see why it's really important. Uh again, I've put the link if you want to learn more about this and what kind of formats those messages are. Um okay. All right. Let's see an example. Uh use case. Uh let's say we have Alice. She has a node and she want to sync block, right? There is Bob. She want to connect to block to Bob to get the blocks. And uh first thing, TCP connection, obviously.

Uh second thing is going to be an ENR check. So, I mentioned that messages are encrypted. So, you are going to send what we call a auth message and they are going to you are going to receive a response which is called a ack message. It give you all the information you need to then start the encryption for the following protocols. And it's not Okay. I'm going to share my opinion on this. It's not really useful because all the message uh are blockchain. Blockchain is public. We all know what the message are, like all the information are.

But yes, it's a good standard. But yeah, it it has it does add complexity because then after that all the message you are going to encrypt them and decrypt them. So, you add like complexity to the clients. I see someone who is agreeing with me. All right. So, I'm not the only one to share this opinion. Uh I also like fun fact in comparison, Bitcoin doesn't have uh encryption in their message. All right. Uh second step is the RLPx. Uh it's part of the RLPx protocol. So, I mentioned you need to negotiate uh the capabilities. So, you need to send the hello message and you are going to receive the hello message to be sure that you have uh you can talk the same language,

right? Sub protocol. Um so, here I don't know if it's maybe too small. You can see. You can see like the version, client ID. You also have like clients, what kind of client they have. Uh capabilities for and things like this. >> [clears throat] >> All right. Next. Uh then it's Yeah. Then you we are part in the ETH capabilities sub protocol. So, you need to send the status and you need to have the status in response. And why this is really important?

Because you have the network ID. So, you might you might have done all of this and actually have to disconnect. Which is making like discovering new PR that are actually useful to you a bit more complex and taking a long time. I think it's not optimal and maybe there is other way to do. I would be interested in your opinion about that. Uh but also something really interesting that is new in ETH 69. That's why I put new. Uh is that you have the field earliest and lastest.

Do you guys know it why why they added that? Checkpoint? No. Not really. Yeah. Yeah, exactly. >> [laughter] >> Uh I will tell you more about about this after. Uh so, yeah. And then uh later lastest hash, the lastest block that you have received. All right. And then only then you can start syncing. So, you are going to start by requesting the block. And in answer, Bob is going to tell you give you the blocks.

All right. So, you when you request for block headers, you can start at any block. Uh also, you can batch. And this is really interesting, I think. Uh maximum being 1,024. Uh but that's already a lot because in one exchange, you can have 1,024 blocks header information, right? Uh you can also go either from Which is pretty neat, actually, this one, I think.

All right. And the answer you will get the uh but we are still missing information, right? We don't have the complete block. We only have the header. So, we are missing information like transaction, withdraws, and uh everything that you can find in block bodies. You also used to found something called uncles uncles block. But since we have uh moved to POS, it's not there anymore, right? Uncle blocks were part of the proof of work protocol. >> [snorts] >> All right. And then you keep doing those. So, beginning, it's a bit a lot of effort, I would say.

But then you can keep doing those and you can sync pretty fast, right? And this is end of the example. So so far so good? All right, thank you. Okay, so I mentioned this earliest and last is filled which are new fields that are appeared after the Spectra upgrade. Very sorry. And it's really important because so it's called EIP 444 which is a bond historical data in execution clients. I'm going to read the description.

Clients must stop serving historical headers, body, and received older than 1 year on the P2P layer. Clients may locally prune this historical data. That means that clients are not going to have the old blockchain. The kind of the concept of blockchain and be able to reproduce a step from the start to the end is going to disappear because since we've moved to pause, we actually don't need all this data, right? You can create consortiums on the layer using pause without having access to historical data. So it's good I understand the motivation because it's taking a lot of disk space.

So you usually need 1 TB SSD. Can be a bit expensive if you want to run it run your node locally like at your house or something like this. But I think that's kind of sad because that mean that you're going to lose all those data and you're going to take action to maybe start archiving those data, right? Let's take it early. All right, so I'm done talking almost. We are going to have a small workshop session, so I'm certainly unfortunately we're not going to do the implementation.

It would be long even if I already do it and we don't have tables. This is not kind of practical. But what we can do is that we can get together and think about what can we build now that we have learn all of that. Maybe you have some idea of application that you can do. But if you're interested in how to implement this P2P RLPx in Rust, I have a workshop project on GitHub. You can scan. It already has like basic function like to create messages like hello, status with the RLP encoding.

Each branch is a step of the slide that we have seen with the diagram. And the final the final branch is a full project. And yeah, so yeah, I haven't updated so it doesn't support ETH 69. It's only 68 and 67, but I will do it in the next day. All right, time for you to work. We have 3 minutes. I don't know what time is it, sorry? Mhm?

Oh, we have 10 minutes left. Okay, so if you want to take maybe 2 minutes to find proposition of what you want to build on this P2P or maybe you you already have some ideas. Okay, we have someone with an idea like you you can get the mic. Oh, question. So I'm just curious, what is the the minimal state that a node needs to implement or the minimal amount of commands that it needs to implement to be accepted by other nodes in there. So if you would, for example, want to monitor the mempool or block blocks without running an actual node and don't use that overhead to verify everything because you don't

care. Yeah, yeah, so good question. The minimal unit is the RLPx. So those four message, hello, disconnect, ping pong. You you won't get then anything for the mempool, for example, then you will need to have the ETH, but then you can ignore all the message, right? So you don't need to do request. You can just get interesting to for the message with the mempool. I read that nodes kick you out if you don't provide them, sorry. I I read that nodes are kicking you out if you don't provide them if data. So for example, if they ask them about block headers, etc. and you just don't respond or you just say I don't have that, they will kick you out and disconnect your peer connection so you lose kind of an insight and peers quite fast.

Yeah, so that's something I actually haven't monitored yet like look at really in deeply, but some clients won't. It depends, I guess, on the clients, but also like let's say you you start to index nodes. You don't have any block, right? So if you say they don't have any block, they cannot request anything for you and that would be unfair for them to kick you if you are just a node starting. There is no way to know if you are a bad node or a good node at this point. But you have an idea then. You you mentioned mempool. Do you have an idea?

Yeah, I just recently built an an MEV bot to to do background running, etc. And currently it's not profitable. I'm still off by a few cents to win some bits. But there was curious how or where do I get the mempool information and all the transactions out without running an actual full node because as a as an MEV bot, I don't care about if a block is valid or not. I just want to get the the transactions in the mempool and try to figure out something to make a profit. Okay. Yeah, that's a that's a good thing. Anyone else want to share something? All right.

Uh yeah, you mentioned Parity made their own sub protocol, their own capability. Is this if you meet nodes and you tell them, "Hey, I have capability XYZ" and they've never heard of it, they're not going to kick you for that. So you can just have you can create a sub protocol and have just you and a bunch of friends use it in their clients. And that works? Yeah, yeah, yeah, exactly. We don't care if you have if you have like sub protocol that we don't know about. So you can create your own propose your own for your own use case, convince other clients to audit or just write your own client now that you know about this.

Now there is way more like to do when you build a client, right? I'm getting very busy and all this stuff. Maybe that will be another talk. >> [gasps] >> All right. Okay, we are going another? I have one question. Yeah. If you if you were to rebuild the protocol, what would you keep? What would you take out? And why? Yeah, that's a good question. So I would change the peer discovery to a network ID so you don't have peer that are not useful to you.

I would remove the encryption actually just so you can get things faster. That's that's a choice. And and maybe I would make it compatible with Tor. Because here you have information about your IP address and IP address already give you a lot of information. I think this is more dangerous than knowing what kind of message you exchange. And it was a plan, right? I remember I read the blog article about having the encryption and moving toward Tor, but they never done it. But it was an idea back in 2013.

All right. Okay, so I I do have some application that I've built that I want to share with you. Those those are all like just for fun, right? You can contribute. It was just to see like what can we do and are we going to be kicked out of the network. So first things that I wanted to build when I learn all of that is a node to index blockchain because data I think data are interesting. So it also allow you to fetch directly like blocks from the network. You don't need to deploy a node. And it save you space, right? Also nodes use an RPC. So I've done projects for indexing where we had to use JSON RPC and I hate it. So this is also hate has

been a big motivation for me to learn about this P2P. So yeah, no need to deploy a node and wait for it to sync, right? You just start and save space and it's fast. It's fast now, but you can batch do batch call, right? You you there is a link to the GitHub repo. It's called prototype ETH prototype. I don't have a good name. If you have a proposition for name or if you want to also run it, it's working.

So I'm able to like I tried Hoodie like to sync Hoodie node and put everything in the in the database and create index and everything in Postgres. Took me like one morning to do that. Yeah, so if you are a Rust developer, fun stuff. All right, sorry. I'm checking time. I have 2 minutes left, I think. You can map the network. We talked about disc before disc before 5 and you can get a bunch of information also if you have ETH protocol.

So what kind of data do you get? You get pub key, client, what kind of client they use like geth, or things like this. Are they sync? Which network ID they are on? That's a bit something I'm obsessed with network ID. You will see why after. Fork ID and their IP address. So then you can analyze those data and see, "Okay, they are where are they based in the world?" And it's low in resources because if you remove EVM, if you remove the database, you don't need a beefy server to run this. It's really cheap and you can deploy a bunch of them. Actually, I have a bunch of them already on the network.

Eight. >> [laughter] >> And same, Uh to the project. Uh also, if you have a better name and if you if you want to run this and uh do the metric yourself, you're welcome to. Uh uh yeah, so I wanted to share like some of the findings that I have. So, I've been running this for a year. Uh here what the Postgres uh table looks like if you query the things. So, you have the IP, you have the ports, you have the network ID, uh the clients. Here it's boring.

Uh and I've also added the what kind of capabilities they have. All right, so you can start to do data analysis on this. And a little bit more of like numbers metrics. So, how many node I've contacted? Uh almost 25,000. All right. So, there is a lot of them on the network. And here uh yeah, here's the number of node associated to the network idea. So, you will see that Ethereum is one. It's not the first one.

You You have like more node for 137, which I don't know what it is. Someone knows? It's Ethereum test network? One the first one. Yeah, I think Sepolia is this one actually. I It's Polygon. Okay, it's Polygon. Uh 56 is actually Binance. So, they have a lot of nodes. But please note that uh because it's really difficult to map the network, it also uh depends on the time because nodes go up and disappear. This might not be right, right? This change a lot. But Ethereum I only have like almost 2,000 nodes.

But it might also have happened that I was not able to connect some node, right? They might have disconnected from me. But this is this is this give you a little bit of an idea. All right. Mempool. >> [laughter] >> So, you can get all the information from the mempool and same you get rid of the database, EVM, everything that you don't want and you can get the new transaction and connect to a lot of nodes, so you want so you can try to be the first one to get the information. Low in resources. It's really cheap to deploy nodes because they are like uh skinned from everything that you don't need.

And yeah, easy to deploy. And it's actually a small service that I've started with uh my friend Morad, who is here, uh that does that. It's called a puddle network. It's free to use. Please send us an email, contact us. Also, if you want to help and contribute and same, you can contact us. It's a fun project that we have done. And yeah, so last uh idea that I have, things that I want to do, is that building is fun, but breaking things also is fun. Uh there is something called uh an eclipse attack that you can do on Ethereum. I mean, theoretically, you can deploy enough nodes to isolate one node.

And we have seen that I've mentioned several time already that uh they are low resources nodes when you only keep the peer-to-peer stuff. So, you could theoretically deploy a bunch of them and be an So, if it's something that interests you, like to try in practical, maybe not to attack the Ethereum network, but let's try on a testnet, please contact me. And uh yeah, there is more peer-to-peer. Uh at the beginning of the presentation, I mentioned um I mentioned libp2p, which is actually used in the consensus layer. That's something that I've not yet uh been studying deeply, like doing a deep dive like I've done for this, but that's

something that would be interesting to do. Uh what I have I have a bit of beef though with libp2p. I'm not the only developers. People kind of hate it. I've used it for other project and I also kind of hate it. Uh it's really bloating, difficult to use. But maybe there is things that you can do that we can do like to improve Ethereum consensus layer peer-to-peer protocols. And that's all. Thank you. All right.

Do you guys have some more questions? Otherwise, we are good. All good? Well, thank you.