Monitoring Ethereum’s Beacon Chain: A DEFCON Alert System for Safety and Liveness - Pavel Bautista

Transcript

[Music] Hi everyone. Yeah. So, uh my name is Pavl and I'm here to present the work I did with Migalaps which is uh monitoring Etherum's beacon chain at Defcon system for safety and livveness. So why I'm here today I'm actually work as a geographer from 2019 until last year. Uh last year I decided to do a data science and a boot camp and at the end of last year Miguelabs gave me the opportunity of do an internship with them which I did and I finished two months ago. Uh at the end of this internship I published

two blog posts on the research that you can find with the cure codes and uh well I'm here today presenting this work and I'll be next month in Berlin. So why a defconor system for the beacon chain? Well, what is a defcon system? Uh defcon is defense readiness condition that is something used by the US military. Uh that has different uh levels meaning level one like the end of the world and level five meaning everything is just going right. So don't worry.

Just as a quick reminder, uh the beacon chain is composed by epochs. Each epoch has 32 slots and each slot should have a block proposed to a block to be proposed. We have we need a proposer and we need all active validators making at the stations. So we have three type of atestations as you surely know. We have source, target and head at the stations. Source and target at the stations are in charge of block finalization. And for this to happen we need twothirds of validators agree agreeing and for head at the stations we need 50% of validators agreeing. So in this figure uh we have

um in the xaxis we have the timeline and in the y- axis we have the source correctness percentage and the yellow or orange lines are epoch and blue we have the 30-day average and the red dotted line is the 99%. So here we have over 340,000 epochs and uh what we wanted to analyze is uh the correctness percentage. The correctness percentage is uh uh the percentage of validators agreeing making a test station correctly and in time. So when we are below 66% uh we cannot ensure block finalization.

So here what happened is that we can see that sometimes between hard fork transitions we have a a deep incorrectness that actually is kind of expected to happen because we have a lot of nodes that are not updated at this moment. At one moment in May 2023, we have a a really bad dip in correctness percentage uh which means that fin block finalization was not guaranteed and then we have some important divs and you can see in the purple arrows that actually we don't know what happened at this moment and actually we don't even know if people

was aware that did happen that this happened at this moment. So we think we thought that we need to have a larger system that can give most give us information like something bad is going on right now. So we did we try to detect these anomalies and we first we think we thought that we need uh to identify key metrics and then we needed to set thresholds uh to different levels of alert.

So how do we did this? Well, as you know, the beacon chain is a distributed system and as a distributed system, we can analyze it using two key notions, safety and livveness. Safety meaning nothing wrong should happen. Uh liveness meaning uh good things should happen. So for safety we we use source and target at the stations and for livveness we decided to go with head at the stations and meet slots. For this we analyzed over 340,000 epochs and we decided well actually we tested different uh time frames we decided to go with the 11 to 12 epoch interval which is more or less half a

day. Here for the safety uh defcon alert system we found four key metrics. The minimum source correctness percentage recorded within this period. We have the same for target correctness percentage and then we have the source and target coefficient of variation. So the coefficient of variation is just an equation that will analyze the variability within this period. So when uh actually the weekend chain performs really well 99 99% of the time we have correctness percentages that go above uh 19 95%. So if you have if we have a lot of variability it means that we have a lot of dips on this uh correctness. So

something that we don't want to happen and here we have four levels. So notify like the US military but it's okay for us. So level one meaning that some bad things are going are happening right now and level four meaning everything is just right. Here we plotted our results. So for the both figures each dot is an 112 epoch period and uh the color of the dot is the defcon alert uh the defcon level. So one is red and green four is green. Here on the left figure we have the source coefficient of variation in the x-axis and in the yaxis we have the target coefficient of variation and we

want these values to be as low as possible. for the figure on my left. So you're right, we have the source minimum correctness percentage in the xaxis and the target minimum correctness percentage in the uh y-axis. So just as a reminder, we want this to be as high as possible. And here we can see that actually our classific class classification work pretty well because all uh the same all same color dots are together and we have the red dots that are actually what happened in May 2023 for the liveness results. Uh we have three metrics. So the minimum head

correctness um recorded within the period. We have the head coefficient of variation and we have uh the average missed the lot recorded within this period. Uh we have also four levels and the values inside the table are this the threshold we set uh for every level. So we decided to plot the results and actually what we have here is that again each dot is a half day interval. H the size of the dot is the average mis load.

So the bigger the dot is uh the bigger uh the average was and in the x-axis we have the head minimum correctness percentage which should be as high as possible and the head coefficient operation which we need to be as low as possible. So this we here we have the live livveness classification and we see that it worked pretty well. And to conclude we actually thought that think that this classification was uh this work was pretty useful because it allows us to detect different kind of anomalies. For example, the the problem that happened in 2023 was really shortlived anomaly with with really bad apex going on, but

most of Apex was doing performing just well. So it can it helped us to detect this and at the same time we can we we could uh detect prolonged instability. Uh it allows us to detect a bad performing apoch but still doing their job meaning they could uh finalize block the blocks and actually we think this can be a valuable tool uh that will help the community to uh analyze uh bad performing epochs and uh give you more document or information about this. And actually uh just to finish we think we can improve this uh tool by supporting

multiple time frames like using half day, one hour or even shorter period of times and operate with real time time data. So yeah, thank you for your attention. If you want to me contact me, you have my information just here. Thank you. Thank you so much Pavle. It was interesting talk. [Music]