Filip Rezabek - How can you assess if your blockchain scales?

Transcript

great hello everyone uh my name is philli and I'll be today presenting to you our latest research regarding how can you assess if your bu scales so firstly briefly a few things about me uh as I was introduced I'm a PhD student at the chair of network architectures and Services overall the chair expertise is going for a couple of decades already where the main focus is on the aspects of protocol design active passive Network experiments and network security uh with today's focus on of the talk I am looking into the aspects of of distributed ledgers Technologies and blockchain mainly focusing on application performance and evaluation and also want to touch upon some aspects when it comes to the design of privacy preserving Network systems so without further Ado let's start a bit with the

motivation so uh as you all know there's currently for a lot of discussions regarding increasing the scalability of ethereum but of course it comes with a lot of additional research that we have heard throughout the run of the weekend uh touching on many many aspects from the perspective of our research we are interested in the parts on okay we see that the block size is increasing uh but of course how does this Behavior affect the validators because suddenly they have much more data they have to process they have to evaluate uh much more data within the same period of time because the consensus itself will not be really changing when it comes to the slot finality are the harder specification of corresponding peers still sufficient or

do we actually have to increase the hardware specification how does it also affect the peer-to-peer Network because of course uh as as we all know if we have to send around more data it means that all of the peers in the network you have to have better connectivity do we maybe even have to optimize uh the P2P Network and other aspects when it comes to network throughput and many many others of course as you all know there is a lot of discussions regarding all of these aspects but what we want to do and look into is finding a way on how to actually assess these individual Parts in a reproducible fashion which I'll be talking about a bit more in detail so in addition to that not only focusing on ethereum ecosystem but General layer one and Layer Two Solutions there is a lot of noise going around with respect to

marketing on how performant is your system with respect to TPS lighten finality and here you can see different values taking at who knows when uh for whatever period of time that later on can be sometimes summarized and put into a table tweeted put on a Twitter and basically the information that is such uh table provid cannot be very easily verified and be fully trusted in addition we of course see a lot of new use cases we we covered a lot of interesting aspects when it comes to encrypting M Pools private compute and many many others uh that I think is definitely something that we always have to keep in mind that the blockchain never sleeps which basically brings me to this third point there are many

parameters when it comes to designing your corresponding blockchain and operating it and there are also regular updates that you have to take into consideration so always you have to have a way on actually how to assess if you introduce certain change into your system is the system actually going to perform the way you expect and how can you be able to to verify it and for that we are introducing our structured approach uh that I will be covering in the second half of the talk where we want to have a way on how to actually assess the capabilities of various blockchain Solutions uh look into the individual layers of the stack in a reproducible fashion what does it mean reproducible so let's say if I run the experiments today but also in a week or in a month I should ideally be able to run the

experiment run the experiment meod with the same outcomes and not only us in our controlled environment but also ideally the people in the community should be able to verify that okay if we run it under the same conditions we actually able to achieve the same findings as the previous uh groups with respect to it we also want to compare different architectures right because as we know there is for example a lot of layer two solutions but they vary so so the aspect is actually how to find a good uh and stable way to actually be able to also compare them in a fair way in the sense that you actually know uh when it comes to assessing such U different systems how these architectures Compares and most likely collected insights why is it like that and can the corresponding

architecture be also improved in the future and find overall the common ground when it comes to having the the base basically which you can later on work with and try to increase the state of the art when it comes to the future Technologies and for that we are mainly focusing on the aspects of deployant VAR systems in our local environment where we can um emulate realistic behavior that we can see from the actual real systems that are running out there we can collect all of the data and be able to collect uh interesting insights when it comes to empirical studies of such systems as we all know blockchains are not coming out of th air they're building on top of of course the internet infrastructure because all of

the IND idual peers have to be able to communicate to each other uh this is basically the corresponding blockchain St as I believe many of you are familiar with uh we have the aspects of the peer-to-peer Network where of course all of the peers have to talk to each other in order to disseminate data and collect interesting insights we know that of course part of this communication on the P2P layer is the main focus is to share the transactions of course later on the blocks share the votes and other aspects so of course this is something that you have to be taken into consideration on how scalable is your corresponding system later of course on the content to layer you are agreeing okay this is the corresponding view we have and we are gaining confidence into the corresponding State over time and of

course for most of the interesting aspects you actually want to be able to deploy your application into the system you end up usually paying some fees for that and also of course the users interacting with the system also end up paying certain fees and you want to know maybe in advance actually how efficient is your deploy smart contract uh how efficient is not only the from the side of the user perspective but also from the side of the core developer actually how efficient is my virtual machine and that is actually executing the smart contract and is there something that I can do to improve uh the performance of the virtual machine itself itself or again in the in the aspect of okay if I want to actually add additional features uh to the system what does it actually

mean uh for the corresponding validators that at the end are later on running the virtual machine where all of the stuff have to be verified and validated needed and of course at the end we have the user clients that are basically relying on all of these aspects of the blockchain stck as an infrastructure just a brief refresher when it comes to scalability in general the main two separations is between horizontal scaling and vertical scaling so with horizontal scaling we are basically just adding more nodes and in ideal case we would of course like to see that the performance of the system gets better uh as we all know it's not usually the case it's rather actually we are currently happy that the system be performance doesn't get worse but this is I think uh something that is also

coming down to the individual interaction of the individual uh layers on the blockchain stack and now there's a lot of discussion with respect to Vertical scaling when it comes to to zups right that basically if you add more resources it's possible to create proofs faster and you can actually ensure that the system can be scaling in this direction for that basically something that we always have to keep in mind when we are evaluating various systems uh if we are focusing on vertical or horizontal scaling but of course ideally we want to collect insights from both of these aspects to at least see okay if we add 10 additional nodes 100 additional nodes or 1,000 additional nodes the system performance still doesn't deteriorate and stays as expected so considering a simple scenario uh that we want to briefly look

into uh we have to of course we have our PE top Network you can have a look at it here um in this case we have interconnected peers uh that basically are interacting with the with the corresponding chain of course some of them are either having the full uh are basically operating as a full node they have the whole copy of the blockchain some of them only uh maybe are relying on third party that basically is having uh having the the full copy of blockchain to in order to interact with it uh we of course when we are interacting with it we can send the transactions the blocks and in general of course we expect that our messages will be propagating through the system we of course for that have to accommodate for the part that each that actually sees the message and does some

activity uh and forwards the message farther has to actually ensure that has the right Hardware specification in order to be able to do so for that uh as you know there are always uh usually provided by the corresponding Foundation the minimal requirements in order to run a note but sometimes it's a little bit unclear uh how did we actually get to these Hardware specification is there a clear structure is there a way on actually um making making sure that if I run my note it will actually be working properly does it apply only to the CPU assumptions the RAM amount that I have or do we also have to consider some aspects of for example ssds with respect to IO and all of these parts basically have to be verified and of course more

importantly considering that the system always keeps evolving we have to keep in mind that uh maybe in the future this Hardware specification will be different there's also a lot of activities on the peer-to-peer Network right because of course we want to make sure that we are not just having let's say some naive start topology where we uh exchange all of the information over a single hope that later on basically can posst as a single point of failer so we want to have a robust peer-to-peer Network where all of the peers can exchange the data uh in such a fashion while using some goap protocol or some other form of uh unstructured or structured Network algorithms and of course we have to consider different type of transaction right of course someone want to maybe

just send a typical easy simple as transaction but but if we consider some aspects of more complex smart contract deployments we have to be able to handle those as well because of course it's again something that has to fit into the block has to be verified and validated by the corresponding peers and we have to be able to see if it actually is possible and of course a lot of discussion goes into the not only keeping let's say the typical uh flow of execution and the life cycle of the transaction but in addition we are adding new features when it comes to implementation and usage of secular multi party computation threshold cryptography Zer knowledge proofs and many many others that have to be taken into account and other aspect is the

infrastructure layer where of course certain types of peers in the system can also provide additional types of security guarantees uh using for example some form of Hardware acceleration that of course can overall improve uh the system performance itself so as you can see the scope of actually being able to assess if your blockchain scale is quite large because it's not uh only a matter of a single U single note a single Pier of course you can still collect some insight from that but you have to consider uh the whole deployment and all of the individual protocols that are in the system and most important most importantly try to collect the information about how these individual layers interact with each other and what type of uh guarantees each of this

corresponding layer can provide touching a bit on our evaluation methodology so basically what we want to do is we want to bring uh these the system on the left where let's say talking about ethereum we have around 6 to maybe 10,000 uh physical peers running in the system and we want to somehow map it to our local environment in order to be able to actually collect comparable insights in order to do so here on the right is just like a simplified topology of our local uh test bet environment uh where basically we want to emulate the globally distributed blockchain networks uh we want to have the options to configure various configuration

parameters for the corresponding blockchain and of course we have to consider that the system might be able to interact with other uh other systems as well so for example in case we want to deploy L2 solution we of course have to also deploy the L1 solution in order to basically have the full-fledged flow of interactions and guarantees offered by the corresponding L1 and L2 in order to collect good insights and see the the overall performance of the system in such a scenario as well when it comes to the decision we of course could consider deploying uh our experiments uh also in Cloud so this is actually something that we are looking into and I want to integrate it into our

framework uh but so far we are mainly focusing on the local deployment where of course when it comes to local deployment in comparison to Cloud uh or even using something like testnet you actually have much more sophisticated level of control over your system you can play around with various parameters you can modify the hardware specification of your corresponding peers so let's say going back to the figure a single note here maybe as as it as it is by itself uh maybe has around 64 or 128 gab uh of ram can be even more than that it can maybe have around 64 virtual CPUs uh it can also have very good connectivity so as you can see we have maybe 25 gbit links uh to connect

all of the individual peers or even 100 GB links which is of course something that we don't see early in the main deployments right so we want to have a granular control on actually how to lower the specification uh of the individual Hardware nodes uh using maybe some form of FL virtualization and also have a granular control over the corresponding Hardware stack to collect and um reproduce and emulate the right State as we can see on the Main and of course when in combination with Cloud this could provide also interesting insights coming I would say closer to the realism aspect because of course many of the notes running out there are actually running in cloud deployments and it's also interesting to see how uh

the cloud itself behaves and what type of individual noise it can introduce when it comes to the experiment design of course uh this is something that is important to spend a lot of time on because if you just deploy the stuff without knowing what is the Target and what type of insights you want to collect uh it's a little bit challenging to to be able to do so so usually you want to think of some theoretical assessment what does it mean theoretical assessment it can go uh very deep depending on how much time you have you can of course try to understand how the individual building blocks behave so for for example talking about ethereum you want to understand how the consensus behaves you want to understand what are the gas limits when it comes to the block sizes or actually what are the

corresponding block sizes themselves um you want to later be able to model and understand let's say the peer-to-peer Network okay if I send a transaction uh on etherum Main net to how many peers the message gets sent to and understand all of these individual Integrity details of the corresponding protocol uh in order to actually design the experiments properly and later on collect valuable insights so General separation we can of course also split between micro benchmarking let's say from the view of a single peer or micro benchmarking considering basically the full-fledged uh deployed uh L1 in your controlled environment and you can Al of course try to play around uh in the form of blackbox or white white box testing

where the main difference is that considering that we actually have the access to the code because most of the code is open source which is really great to collect insights we can also modify the code for our particular experiments but ideally due to the complexity and the way how often new new versions are introduced we rather want to take it like a blackbox without any need to modify uh the code itself and just observe the performance using external metrics and parameters and of course touching on that we can collect many many metrics from the system that we can measure we can of course consider throughput like when it comes to transactions per second various types of latencies that can affect the user's experience in the corresponding system it can also be aspect of finality

actually when the corresponding block is final uh and cannot be changed later on we can of course try to play around with various Hardware specific ation of the individual peers to see and collect the Insight uh about about that and of course when it comes to the parameters we can be adding higher number of peers to the system we can also modify the block sizes to for example assess okay in case ethereum in the future will have BX size of 30 megabytes how does it actually affect the performance and try to also emulate some FL injection because of course we want to always verify uh if the guarante is offered by the consensus when it comes to the bantine F tolerance is also present uh in the in the form of our environment

and how the system behaves under such conditions as well so for that uh briefly touching on the design I will just skip through it briefly we of course have the input where in general the idea is that you define your experiments that you later on deploy you have your system under the test so again following the examples that will be let's say uh your L1 ethereum with maybe 300 deployed nodes that will have different uh amount of Hardware specific Hardware allocated to the individual validator uh basically going also in the heterogeneous direction of how much uh different providers can be out there right and you basically start to issue the transactions to the system and

you're measuring the the individual insights when it comes to the corresponding metrics that you have defined as a part of your experiment briefly touching here on some of the subset of supported features we actually have in our framework and our testb so we are quite heavily looking into trusted execution environments we also have quite a several threshold cryptography schemes that are currently supported and that can be used and to interact with um the with the system we are also looking a bit together with my with my colleagues from the technical univers of Munich uh into me uh so for that we of course want to have the full flow of ethereum deployment uh in our local testb and while collecting also

insights from the mainnet actually try to emulate some form of bidding wars and see how actually the latency for example can affect uh such uh bidding wars in on the main net we of course want to measure different types of delays uh for for ethereum when it comes to uh to the aspects of uh interaction with the individual clients and we also want to for example modify the hardware specs so overall uh this set of subset of the feature was also important for us to actually validate that our appr approach to design such a framework is viable and it can be done even though it takes a lot of time to actually do so and on board new new Solutions into the framework it's possible to do it and we now I think have quite a decent way on

how to assess if your blockchain scales uh touching on the takeaways uh already basically the main aspect is always onboarding new systems takes quite a lot of effort because uh each system is slightly different but I think this is the part that is also fun because you keep learning along the way on how each system performs and behaves uh we would like to in the future find a possible unification and evaluation evaluation guidelines that the community can use and possibly extend in the future uh to have basically a fixed Baseline maybe also help with definition of the workloads and overall we believe that this is a good stepping stone towards collecting Hol holistic overview on the interaction of the individual layers that are part of such complex

systems like blockchains uh in case you interested we also have it available as an open source uh we of course have some of the latest updates still available only for us so usually we update our public repository once we have also uh academic Publications available but in case you're interested to find out more I think it already provides good insights and references to our Publications and follow as always happy to connect thank you very much phip thank you very interesting presentation um unfortunately we don't have time for a q a but you can ask him any questions during the coffee break um and the next session will start um at 225 mhm thank you