State of Decentralization and Getting Started with Ethereum Infrastructure

Transcript

[Music] okay hello everyone my name is philip blankens and i'm a software engineer at the ef javascript team and today i want to talk about the state of decentralization and getting started with ethereum infrastructure but before we talk about the state of decentralization let's talk about the state of centralization i bet you've all seen the attacks on twitter two weeks ago i think the one of the most interesting aspects of these attacks is that in order to get access to all the accounts that got compromised the attackers only had to get access to a single twitter employee account this means they did not even have to hack into the system maybe they just bribed someone or threatened or blackmailed someone we

also learned that twitter has this internal administration tool which allows to perform actions on all the accounts and we saw some interesting discussions about shadow banning and discussions if these attacks had the potential to start a world war these were some good reminders why we want to build decentralized systems and we want to eliminate single points of failure unfortunately decentralization is a complex complex term and there's definitely more than one meaning and type of decentralization client diversity is definitely one important aspect of decentralization and if you're following the core dev calls you've probably noticed that this topic has gained a lot of

attention recently for in order to track client diversity we have a couple of tools such as the ethereum node tracker on etherscan or the ethernodes website and these websites collect data about the clients on the network and the software that they're running the problem right now is that the majority of the clients on the network is is just running running the get client and for an in-depth analysis of the problem you can refer to the blog post applying the five wise to the client diversity problem but a frequent response to this to the client diversity problem is that we should just decentralize everything and everyone should run their own node

but unfortunately people are barely motivated to run their own node and it's also a very complex and difficult task we have some of the shelf solutions that people ideally should just plug in and then they're connected to the network however this helps this helps with the architectural decentralization of the network this means it creates more physical computers in the network but it does it does not help with a diversity problem because it's unlikely that these off-the-shelf solutions will run different client implementations solution might be that we run our own nodes and we configure them ourselves but this is not an easy task and it can

be helpful to have a good conceptual model of the infrastructure and how we want to work with it so one abstraction model is to see if the ethereum network as a nodes that are connected with links and that form this peer-to-peer network another view on the network is that these are machines running client software and that these clients have different roles on the network so they are all configured in different ways and need to perform cert different actions this model can also help us with a lot of the misconceptions that we have about ethereum infrastructure and the decentralization of the system such as that connecting to the network

means there are more nodes and this makes the network stronger or that the number of nodes correlates with the strength or that just simply running a node improves decentralization that everyone should run their own full node everyone should have a copy of the blockchain which will just decrease the amount of nodes that can participate that running these nodes requires terabyte terabytes of data that mining is not feasible for individuals and it's unnecessary to run a node anyways okay if we are beyond this stage and we

have a good good feeling for the network and its infrastructure the next step is usually that we jump directly into the configuration of a client and this can be a very frustrating experience we are confronted with a lot of different options and concepts and it helps to first learn about a few very important topics such as the the different types of nodes that we have and configurations that we can do on these clients the different consensus algorithms all the different clients that exist the different test nets all the different sync algorithms the different deployment modes and if you

if you've been to one of the release pages you've noticed that different binaries are compiled for different architectures and that this can determine our targeted hardware so if we run it on our own private laptop on a cloud computer on some dedicated server on a raspberry pi on our smartphone and this is how you get started with ifum infrastructure and this is usually also where it ends this is where the handholding stops and you are left alone with this information and now you have to deal with it and make the best out of it but let's try to be a little bit more specific here but before we start choosing a client and working for through each of these phases um let's make two things clear the first one is if one is not dead

and the second one is and we don't need to be scared of running alternative clients okay let's try this if we want to be really close to what we have on mainnet of course geth is a great choice as a client and um it is always a good first start but if we want to help with the diversity of the system then we should definitely not use geth and we can for example use bezu if we are developers there are development nodes such as ganache and if we love love microsoft then nethermind is probably our choice if we really can't wait and we're too excited and want to get started with eth2

and skip maybe eth1 even prism is a great client and if we want to learn more about the systems and um maybe also peek into the source code then trinity might be a great option it's written in python and very approachable a lot of reference implementations are happening in python first we can also check out the ethereum js projects and modules i think most of the people understand javascript and it's very approachable the next step is choosing a test net and test nets can also be very confusing but i would definitely always try out gurley because it's a multi-client test net and this is exactly what we want and for ev2 there are there is in-depth documentation on github

for the sync algorithm or the the sync protocols um it's obvious that everyone hates to sync their own node it takes forever and we need to buy we need to buy hard drives for this or ssds um and there's no silver bullet to this problem sync in general is a is a very complex and difficult problem but it's worth it to check out beams beamsync and snapsync and learn more about why syncing in general is so hard and the steps that are taken at the moment to improve the situation okay if we want to try try out all the clients and we want to build test networks and test scenarios and we hate the garbage that these binaries leave behind on our systems

and we might want to change the machines or the operating systems in the process i would recommend always to go for docker most of the clients available have docker images and you can find them on docker hub and it's also for someone that is quite new to this topic it's worth to invest and and go this extra step and learn about docker because it makes all of the these scenarios where you have to move your your configurations much easier it's also nice because it isolates yeah your private data and your machine configuration from the actual environment where the client is running and you don't have to worry about less set up steps okay the last step hardware which hardware should i use

and in this case i would always recommend dedicated hardware for eth2 it's obvious because we need we have staking um that we need to guarantee the uptime of our valley datas if we use our um our work laptop and we take it on the train the valley data goes down and we're losing money the problem here is we cannot rely on something like a low cost device like a raspberry pi so this is not really something for hobbyists and i would recommend to learn more about this and before setting up any infrastructure because you don't want to put a 7 seven thousand dollar value data on an unreliable system but also also for other scenarios i would always recommend

um dedicated hardware and i can give you an example why this is this could be important so a couple of days ago i was working on an ec2 instance and i was setting up a couple of docker machines and opened the docker port to to access it when i logged back into the system a couple hours later i saw that there was an instance that i definitely did not create and when i inspected the instance i saw it was downloading a script from an unknown ip address and starting it as a chrome job so whenever you do these kind of configurations you're you're running into the risk that you're opening up your system to attacks and you actually find out that most docker servers are usually infected with cryptocurrency mining malware

a fun fact is that before the malware actually is downloaded before it downloads any binaries um i check the script and it will go through a lot of processes and try to find all the other potential malware that is already on your system and try to kill it kill it to free up resources so this gets very very complex very fast and suddenly we need to know about certificate authorities and open ssl tls and then there's a lot a lot more requirements coming like uptime and availability constraints the monitoring of our services the recovery the load balancing the boot nodes the miners genesis configuration so it can be very helpful to know about

the tools that we have and the first tool i want to start with is puppet so puppet helps us to set up this kind of infrastructure for us it helps us to create private or public test nets and why do we want this because usually a scenario scenarios where we develop some kind of application but we need a shared state for the team so everyone should have the should look at the same state contract the mainnet is too expensive and obviously we don't test in production so we don't deploy all our our code to pro um to mainnet uh as as alternative options we're left with these with the available public test nets but sometimes um the conditions of these public test

nets are not suited for our needs or we don't have access to the amount of test ether that we want so it can be really worth it to set up our own infrastructure and puppet makes it very easy it's a cli wizard tool it will ask us a couple of questions about the machines that we have and about the software that we want to run there or the net network we want to configure and then it will use ssh to connect to these machines and docker compose to set up and configure everything there's a great presentation and live demo from peter and that he gave at defcon 3 that is definitely worth watching an alternative to puppet is ansible an ansible is an open source and enterprise tool that is used also

for cluster management but it can be used for these kind of scenarios where you want to create test nets as well it's very similar in the functioning to to puppet because it defines all the machines in a host file in a host inventory file and the code and the configuration you want to run in the in so-called playbooks and then it uses ssh to connect to these machines and install and configure all the necessary software ansible is used um in the in the ethereum ecosystem for example to manage boot nodes but it's also heavily used in eth2 to set up networks and make and create test scenarios another tool or set of tools is prometheus and grafana

and we use this for the data collection and visualization of the data and geth has a really nice grafana dashboard integrated into the client unfortunately launching it and finding the correct flex for the data collection can be difficult and is a little bit hidden in the documentation and if we are interested in eth2 we will also find on github some very exciting dashboards grafana dashboards that are created for example for prism the next tool on the list is ethereum grid and ethereum grid is an ef javascript project and it's it's a it was created as a user-friendly way to run a node on your own machine it's a desktop application that lives in

your menu bar and it gives you a selection of clients that you can easily start with one click and if you want to make adjustments to the settings you can launch a user a full user interface and you will see all the configuration options and how it translate into flags it can also be used to monitor the zooming state of your client and you will see the terminal output it also comes with an application launcher so if you're just interested in the graphql features of geth you can launch the graphql interface and grid will make sure that your client is configured the right way the next tool is ethereum binaries and ethereum binaries is also an ef javascript project

and it's basically the binary manage from the grid desktop application wrapped in its own node.js module and you can use it as the name suggests to manage your ethereum binaries and interact with them this also makes it a great choice for creating user interfaces with web technologies ethereum binaries helps you with a lot of different things such as package extraction binary verification runtime detection and it also has docker support so you can orchestrate containers but you can also work on the raw binaries it also has listeners for special life cycle events such as ipc available this makes it a great choice in combination with ethers to wire things up and then interact with your client over the rpc interface ethereum binaries has support for four

different binaries at the moment which are geth prism puppet and zocrates it also comes with a cli that you can use to download and interact with different with these different clients that i can show you the next option that we have are pre-pre-built images and pre-built images are very exciting because they give us a zero config setup for raspberry pi's so pre-built images are basically full operating system images that we can download and then we have to flash them on a micro sd card and we can insert these sd cards into our raspberry pi and we get a full working machine pre-configured with all the necessary tools libraries

configurations and user interfaces the problem with these with this approach is that we we have a single source of of trust and we need to make sure that they are not infected with any malware when we get them another option is that node and that that node is a is a great project that provides us with a user friendly client and client setup and management system for remote machines it comes with this nice looking web user interface and you can not only check the the state of your machines that you're managing such as the disk usage or the cpo cpu load um but it also gives you information about the ethereum specific internals of your clients such as the the swing state

it has a package management system which makes it really easy to install new software on your machines or do management and update tasks one of my colleagues has used that note successfully to turn a server from ebay into a machine that swings all the public test nets and the mainnet and it's a very exciting and cool project okay ganache for ganache is known as the development node for deps and the cool thing about ganache is that you can download and launch it and it not only comes with this pre-configured development client and but it also has a user interface that gives you all the necessary information that you need during development the elephant in the room is ideally we want to develop on something that is as

close to mainly as possible but without the fees and without all the other people and ganache has has a great option for this because we can fork mainnet and we can um we can mimic the state of the mainnet and of its contracts that we want to interact with in our development environment and one of my colleagues ryan has written a great article about how you fork mainnet and how you can mint your own die the last tool on the list is eth2 launchpad and e3launchpad is a great way to get started with e2 and becoming a validator so eth2 has a nice website and a web application that works as a wizard and guides us through all the necessary steps and it helps us to actually make the

configurations create the keys do the deposit transactions and then step step by step makes us a validator on these two okay so this is it i hope this was a good rough overview of all the different tools that are being used to manage and configure ethereum infrastructure and why it's an important topic for the ecosystem um we know it's very complex it's very difficult and we're trying to provide the tools to the community to make this easier and get more people onboarding onboard it so that that they can actually run their own note um yeah i hope this was motivating for you and now you can you have a much better picture and you want to run your own note now thank you