ZKPodcast: Testing PoW Consensus Algorithm Security with Ren Zhang from Nervos

Transcript

[Music] welcome to xero knowledge a podcast where we talk about the latest in zero knowledge research and the decentralized web the show is hosted by me anna and me frederick in this week's episode we chat with ren zhang from nervous about his work on consensus algorithms we revisit the topic of consensus chat about an earlier work he did on evaluating proof-of-work consensus protocol security and explore his more recent work on nc-max now before we start in i

first want to let you know about the zk summit the event is happening november 23rd and 24th this is a bi-annual event that i put on it's made for researchers cryptographers practitioners founders and developers working on zero knowledge topics as well as students and zk enthusiasts if you count yourself among these people i think you will enjoy the programming that we have planned for you it's a two-day event tickets are available now so i'll add the link in the show notes and i do recommend you buy your tickets as soon as possible because we actually have some discounted tickets available this week until november 15th so yeah hope to see you there now before we start in i also want to thank this

week's sponsor leased authority leased authority is a security consulting company known for their dedication to pushing the limits on how to build privacy respecting solutions their name least authority derives from the principle of least authority or the principle of least privilege the security best practice requiring system components to only have the privilege necessary to complete their intended function and not more this aids in limiting the potential attack surface and minimizing the extent that vulnerabilities can impact a system now given that privacy is at the core of their work and mission leased authority recognizes the importance of privacy in defy and other decentralizing technologies and the role it plays in preserving the balance of power

and they support this through a growing list of security reviews building distributed systems and regular contributions to open source software projects to find out more about leased authority's work visit leasedauthority.com there you can also check out their security audit reports and if you want to talk to them about the security of your d5 project do get in touch with them at contact us at leastauthority.com i'll add the links in the show notes so thank you again least authority now here is our interview with ren so in today's episode we will be revisiting the topic of consensus and we have ren zhang who's a researcher at nervos

so welcome to the show ren thank you so much for having me welcome it's my honor to be here so quick recap on how this episode actually came about we had an episode with etai abraham back in april on the topic of consensus and i think it was right after that episode alan who was also on the show before contacted me to say hey i have a colleague at nervous who you should really talk to if this is a kind of a topic of inquiry that you're exploring and that was you wren so it's very it's kind of a long time coming that you're supposed to join us on the show and i'm really happy to to get to dig in thank you i think as a precursor like it might it might not hurt for people to listen to that episode with

the tai if they hadn't heard it before so i'll add the link to that in the show notes so we have some really interesting topics lined up for today digging deep into consensus stuff but before we dig into that let's talk a little bit about you maybe sure what's your journey into into this space and and what gets you excited about this research i was doing research in peer-to-peer network in my master's study and after i got to kozak where alan and i get our phds i started to like look around to try to find an interesting topic for myself the journey it takes around two years and i have to say i failed

so i prepared several topics and asked some senior members of the group like which one do you think is the most interesting and they all recommended the same thing they recommended me to do research on bitcoin it said like the best time to study cryptocurrency has already passed according to them oh but maybe you you will still get a phd out of it that was 2014. oh my gosh no way yeah that was the time i started doing research in bitcoin and cryptocurrencies and it turns out the field lasts more than they expected they thought it was when they said that was were they actually thinking not that like the most exciting part of it had passed but that the entire trend was going to

go away is that what they were sort of hinting at or yeah i think at that time it seems that the entire trend was going away the price of bitcoin was dropping every day and there are fewer and fewer people about bitcoin in the field but who knows what will happen lately later wow it kind of makes sense because bitcoin is actually kind of simple like there isn't it's a very simple concept it wouldn't feel like there's decades of research to be done on it but here we are a decade later and people are still going and publishing papers so especially for the bitcoin network because at that time the bitcoin network was relatively small there are like 600 nodes

oh wow like the bigger the network the more interesting research questions you can find in it so at that time people believe bitcoin network is not very exciting okay but you decided to go with this anyway yeah because it's there's a kyle's whenever there's a chaos um there's opportunities cool i guess that leads pretty well into the first paper that we wanted to talk about today which is this paper that you co-authored of sort of um it's called lay down the common metrics evaluating for consensus protocol security yeah i think um and i i you know disclosure to the readers i haven't read the paper but i read the abstract and it sounded

pretty interesting and i what what caught my eye especially is that you talk about how people analyze consensus algorithms and how sometimes those analyses are are wrong so can you talk a little bit you know generally what is the paper about what but what did you discover in it yeah there was a time i was obsessed with selfish mining attack so i was trying to find a solution to that attack to find a consensus protocol that is immune to that attack and i i've tried to design a dozen of protocols and try to evaluate their security and all of them are not perfect i wasn't satisfied with any of these solutions

so i didn't publish any of them years later i discovered that these ideas that i believe are fraud are published by other people and they claim that their protocol is secured if they are better than nakamoto consensus better than bitcoin's consensus protocol so that makes me feel like this is wrong people need to be notified that these ideas are flawed they open new attack vectors that the designers are not aware of but these these other folks who actually published it they didn't know your work wasn't that they saw your work and then took it it was like they came to these conclusions on their own right right because all these ideas are

relatively simple and straightforward intuitively all of them make sense but if you dig deeper if you truly analyze their security you will discover the problems got it in particular what angers me is that many of these papers analyze their security via simulation against a specific strategy there's this paper in 2013 called majority is not enough it is the one that proposes the selfish mining attack and these papers that claims to be better than nakamoto consensus they analyzed the security of their protocol against this specific strategy i show that you see this strategy doesn't work in our design however i argue that this is not a

security proof maybe that your system your new design would inspire new attacks that specifically targeting your your design makes it more vulnerable to a new kind of attack so in order to argue the security you need to find a more rigorous way like either you prove that no matter what the attacker's strategy is it cannot gain more profit or cause more damage or that you manage to cover all the attacker's strategies saying that no matter what the attacker do like within this strategy space and the strategy space is complete it cannot um cause more damage that's the correct way to do a security

analysis from a perspective yeah so i mean this is a common problem where people don't honestly and straightforwardly talk about their trade-offs right but they say like oh we invented this new thing and it's secure against this thing and then they you know forget to mention that actually it opens up a bunch of other attacks but they are they did fix that one thing right so i assemble a group of these kind of protocols and analyze their security via markov decision process and wrote this paper that's basically the idea do any of the protocols that you touched on in the paper would we know them actually like i mean i guess it's written up so

if they are we we could talk about it yeah there is a protocol called uniform tie breaking protocol that is implemented in ethereum okay but i think all the ethereum developers and researchers they are aware of the vulnerabilities of this protocol so that is not a very big problem i see and also there's a product called rootstock that implemented a protocol very similar to the reward splitting protocol uh mentioned in my paper another relatively famous protocol is fruit chains what are they called fruit chains fruit chains i don't know if i know it designed by elaine shu fructans claims to be incentive compatible they are resistant to selfish mining

attack however my paper shows that they are they only have stronger resistance than a commodore consensus against selfish mining attack if each block is confirmed after like 100 blocks if you use six block confirmation same as nakamoto consensus in bitcoin then it's actually more vulnerable to selfish mining so the problem with nakamoto consensus as frederick asked is that it doesn't have the perfect chain quality which means that an attacker possesses less than 50 percent of mining power can securely invalidate the last few blocks in the blockchain that's why nagamoto consensus only provide probabilistic security guarantee so for this

new designs who claim to be more secure than ecomoto consensus we believe they need to satisfy one of the two requirements either they achieve better chain quality or they resist better against three kinds of attacks which basically summarizes the existing attacks we are aware of against the consistent protocol if you claim to be more secure than a commodore consensus you either achieve better change quality or you resist better against three attacks the first attack is selfish mining attack the second is double spending attack the third is feather forking attack or we call it censorship attack and accordingly we divide all these

protocols into two groups the first group they claim to achieve better chain quality and the second group they claim to achieve stronger attack resistance and analyze their security accordingly and our framework is mock of decision process which enables us to model the behavior of a strategic player in a partly stochastic environment basically a strategic player means the attacker and the partly stochastic environment means that the environment is relatively fixed it respond to the attacker in a known way there is some uncertainty that if the attacker do this with this probability this thing will happen

with this probability that thing will happen so we try to explore like if the attacker is allowed to do anything he want like um he can choose which chain to mine on or when to publish his blogs how many blogs to publish what's the best the attacker can do in terms of attacking chain quality or these three attacks i mentioned but in your model you are actually analyzing both because you said that you had two categories what was it again the first one the first one is called better chain quality protocols better chain quality and the other one is these particular attack resistant vectors right so it's either the chain quality or these attack vectors specifically but

in your model are you doing combinations of these things are you always like focusing in on one like there are two different criteria you can think of it okay as as long as you beat nakamura consensus you're one of these criteria you win i see but you might have a less secure chain in the end like there might be some new attack vector that's not clear if you are better than nakamoto consensus in chain quality then you naturally have better attack resistance oh you do okay yeah because the fundamental problem with the computer consensus is that its chain quality is not perfect however if you cannot achieve chain quality some protocols claim that they can still be better than naga motor

consensus because they are they have higher resistance against known attacks got it so we split the protocols into two groups the first groups claim that we are better than a global consensus in ching quality so we don't need to analyze the attacks the problem is solved yeah they actually are potentially better than right now they're superior in every way so okay the story ends who falls in that one well there are there are some protocols claimed that they are superior but um they don't they don't claim that they are superior like in all aspects but for convenience we just put them in that group but the second type of protocols they

say that we share the same chain quality with nakamoto consensus but it doesn't matter because we have a novel incentive mechanism and it's those ones that you wanted to check if their chain quality is in fact equal still no basically they use the same bitcoin backbone protocol oh they do if they use the same backbone protocol then the chain quality is the same okay but they claim that hey although we use the same backbone protocol we can resist better against all these attacks so i would like to check like whether you are truly better but is in in this case are you analyzing whether their chain quality is the same or whether

they actually can defend against these attacks because i understood it that like if they try to improve in this one direction they're opening themselves up to new attacks but that's within the attack resistant protocols so there are two groups the first group is better chain quality protocol the second group is attack resistant protocols but better tranquility protocols they modify they they modify the bitcoin backbone let me put it like this there are two simple rules if you look at the essence of nakamoto consensus you can see two very simple rules the first is longest chain the second rule says that if you receive two competing chains that have the same length then you always mind on the first one you received

so the bitcoin backbone protocol can be summarized in the simplest version into these two rules if you follow these two rules then you have the same chain quality with nakamoto consensus okay there's no need to do any further analysis so in terms of chain quality if you modify these two rules then i will put you in the better chain quality protocols group if you don't modify these two rules instead you modify the rewarding reward mechanism you use a new incentive mechanism to encourage people to not do selfish mining to not do censorship attack then i believe you're an attack resistant protocol understood i think to to make it more concrete

what are examples of protocols that that do either right like so either try to modify the the chain selection rules or the ones that like examples of those that try to change the incentive mechanism sure let's give one example embellishing quality protocols i think the most interesting example some for some people it's the most counter-intuitive example is that it's a protocol we call it um smallest hash tie breaking protocol smallest hash tie breaking protocol yes i give it that name because i don't want to name and it's got it okay project in this smallest hash tie breaking protocol i call it shtv protocol or simplicity the designers believe that if we always

mine on the chain whose tip has the smallest hash then all honest miners will mine on the same chain they naturally converge so the honest mining power will never be splitted whenever there's a fork they always mine on the same chain so that the protocol becomes more secure that's their intuition so this is a better quality protocol because it modifies the first receive policy in nakamoto consensus it always requires miners to mine on the chain that they first received but in shtb they always mine on the on the chain tip whose hash is the smallest and that intuition makes perfect sense to me because as you like anytime you talk about time in

programming you know you're going to be wrong right like if you're talking about first received like that will obviously be different for everyone so it's essentially just a random bias among what people might not because there's network latency and a bunch of other stuff involved in what first received means so is there any problem with the smallest hash tiebreaker again your analysis have you uncovered anything or is the intuition actually just right the integration is not right so in some extreme cases it performs slightly better than nakamoto consensus however for most of the parameter set we analyze it actually performs worse than a couple consensus

because it enables two new attacks the first one we call it catching up from behind for example if you are an attacker you have a secret chain your chain has one block and the honest team has two blocks in a komodo consensus you usually just give up because the honest chain is longer and there's no way you can catch up however in shtb if your chain is smaller and the longer chain's tip has a very large hash then you can keep mining because you know that if you find the next block in your secret chain and you release both blocks despite the fact that you release later

than the honest chain with very high probability that your hash is smaller than the honest chain tip so that all honest miners will switch back to your chain they will give up their previous chain and mine on your chain so this shtb enables this new attack behavior and the attacker can choose when to adopt this behavior it's like putting a new tool in the attacker's toolbox which is why it weakens the protocol security the second attack is that assuming that you have a secret block you know the hash of the block is very small and you're a very small miner the hash of the block is very small so that you can be certain that

with 99 probability the next onis blocks hash will be larger than your hash then you can say okay it is now safe for me to withhold the block i will mine secretly on top of my block however on the other hand if you're a small miner and you you discover a block whose hash is relatively large so that you know with very high probability if you withhold the block you will lose the block raise then you may choose to release the block because in sshb you can accurately estimate the success rate of a selfish mining attack it gives the attacker more information to choose when to withhold the block

and when to release the block so this also put a new tool in the attacker's because the attacker is equipped with several new tools the attacker can attack the protocol with higher confidence thus weakens the security of nakamoto consensus so that was one example i guess that that's an example of a better chance better chain quality protocol but i'm still i'm a bit confused because didn't you say before that if there was a better chain quality then it was secure like you were like i think i'm i'm sort of going back to that earlier thing you you had these two categories those that you didn't analyze right but you kept saying that if they had the better chain quality then you didn't

analyze them i'm confused yes if they have better chain quality they don't analyze their incentive mechanism okay but in this case what does it have it doesn't have better chain quality or it does have better chain quality this protocol claims to have better chain quality i see but it failed okay got it i understand that because because it analyzes its security via simulation i see that's the main message you shouldn't you should never analyze your security via a simulation against one specific attacker strategy okay okay because your protocol may inspire new strategies targeting your protocol so if we go back to the examples what's an example of the second category where

you change incentive mechanisms the second i can give several examples um i will talk about subjects this is a protocol published by peter reason dr peter reason on a journal called the ledger in this protocol like miners are allowed to publish blocks with smaller hashes there are two hash targets there's a bigger target there's a smaller target in bitcoin you are only allowed to publish your blocks if the block is smaller than the predefined threshold the smaller target but in sub chains if you didn't find a block but the hash result of your block candidate is smaller than

the larger larger target you can also publish this block candidate let's call it weak block so that they hope that the blockchain is composed of a chain including blocks and weak blocks the benefit of this is that they hope that transactions can be confirmed faster before it is included in blocks it is included in weak blocks so that the block interval is actually smaller the blocks get all the block rewards the weak blocks they do not get anything it's just a tool used to synchronize transactions however it participate in the consensus protocol in terms that whenever there are two competing chains the fork is resolved by comparing the

length of these two chains both blocks and weak blocks are counted in the length in hormonal consensus there are two chains of blocks in subchains there are two chains of blocks and weak blocks the only difference is the weak blocks don't get any block reward this protocol belongs to a subgroup of protocols we call reward lucky protocols so they don't care which mining product is honest which mining product is malicious all they care is which mining product is lucky lucky in terms of its hash result is smaller they only reward these blocks which hash result is smaller and weak blocks don't get any rewards but

they still participate in the consensus protocol so the problem with this subgroup of protocols the problem with all reward lucky protocols is that lucky block candidates are not necessarily the good ones so this kind of discrimination in terms of reward issuing allows a new kind of attack it works like this whenever the attacker discovers a weak block it will withhold the block hoping that the honest miner may find a block so that i can use my weak block to invalidate the block mined by the honest miners because the weak block doesn't get any reward anyway there's no risk of withholding this weak

block it's like i've got a new tool to attack the system and this tool has no cost i can do it anyway i don't have anything to lose and whenever the attacker discovers a block if the attacker doesn't want any economic risk that i can publish the block as an honest miner so again this protocol enables an attacker to calibrate the success rate and choose a more rational more accurate strategy to attack the system it's interesting how in both of these examples as you put you you put another tool in the attacker's toolbox yeah and people are doing it sort of

accidentally by introducing more complexity they're like oh i want to simplify this or i want to make this path more optimal or i want to introduce this other concept that makes x y or z simpler but then actually through introductions through the introduction of that new thing that's actually another thing that an attacker can analyze it's a thing that they can withhold it's a thing that they can you know get more information from so that their attack is potentially much more successful yeah i totally agree i think complexity is the enemy of security so if you're on the more secure protocol usually the first thing you should do is

to try to simplify the protocol to see which things you can cut which is why nc max is based on like a motor consensus we try to start from the very beginning without introducing any new attack vectors were there any other kind of takeaways or like ideas that you gleaned from your kind of creation of this new model to evaluate consensus protocol security so there are several insights we would like to stress with this study uh the first group is about how to analyze the security you should never analyze the security of the protocol via simulation of a single attacker strategy because that is always dangerous the other thing is that you should not

design a protocol that is too complicated that even you the designer cannot analyze it this is usually dangerous because complexity is the enemy of security and regarding our results on the incentive mechanisms we discovered that there is no incentive mechanism that discourages all three kinds of attack if you reward all the block candidates if you reward all the mining products regardless of whether they are honest or malicious then it actually makes double spending risk free for an attacker as i will always get the reward anyway why not try to double spend because even if i failed i can still get rewarded

if you forfeit rewards from all suspicious mining products like whenever there are competing chains none of them get any block rewards you are actually introducing a tool for the attacker to censor the network the attacker can say that if you include these transactions in your blocks i will try to invalidate your block even if i failed as long as i find something i can forfeit some reward from you to force the rational miners to join the censorship attack in order to avoid economic loss if you reward only the lucky blocks the lucky blocks are not necessarily the honest blocks so that it gives the attacker opportunities

to only perform honestly when they are lucky and perform this honestly when they are not lucky to sum up we discover um trade-off we call it rewarding the bad or punishing the good okay so you just gave us two examples but you obviously have analyzed probably many others and in the paper can you see it or do you have other work where you actually show your analysis using this model on other chains or other consensus protocols i should say there are seven protocols covered in this common metrics paper but this line of work continues after my work recently there's a paper called squirrel it extends the method based on markov

decision process the problem with markov decision process is that it can only model protocols with a small number of states if the protocol becomes more and more complicated then it is out of the computational capacity of an mdp okay so these researchers they adopt the deep reinforcement learning to model more complicated protocol difference for a deep reinforcement learning is a technique used in alphago the famous gold game it enables us to model far more complicated protocols and they get some interesting results has anyone i mean i i realized that it's such a different model from pos

but i'm wondering if any of the tools or findings that you found could be used for any of the like proof-of-stake models or is it so unrelated that we can't we can't actually put those in the same category i think it is related so in general i don't see any difficulties in applying markov decision process or deep reinforcement learning in analyzing proof-of-stake protocols okay but the thing with analyzing proof of stake protocols is that it doesn't have this simple result where you can just assemble a group of protocols and let them race to to know who is the first with the second you can put them in the same constraint profile protocols they usually have different security

assumptions yeah which means each protocol is um player game in its own rule so you cannot just um unify the rule and force them in the same field so there are some complexities in that aspect but in general i think it's interesting to apply a multiple decision process or deep reinforcement learning in analyzing proof of stake protocols do you in your own research do you actually explore that perhaps later but not now um on the horizon got it right i'd be keen to dig into more of the markov decision process and and like the deep learning aspect of it too but then we probably need a whole other hour

uh and so yeah we should probably move on to to talk about what you teased earlier of nc max so this is consensus algorithm that you and some other people designed right um that's the second paper that we're talking about today um and um give us the high level what is what is nc max what does it try to do sure perhaps i should start from the story behind sure we all love nakamoto consensus we love nakamoto consensus for three reasons first is that it has a minimum set of security assumptions unlike provo state protocols which usually have different security assumptions like motor consensus has a very

small number of assumptions the benefit is that the protocol is well understood there are dozens of papers analyzing the commodore consensus so that we we are perfectly aware of its advantages and disadvantages the second advantage of nanomotor consensus is that it minimizes the communication overhead in a motor consensus specifically in bitcoin if you discover a new block you can propagate the block as a compact block of around 13 kilobytes and that's it one message to confirm a group of transactions this is the theoretical bound you cannot get any better than this one message to confirm a group of transactions all other protocols are more expensive

in terms of communication the third thing we love about nakamoto consensus is that it is team based which means all the transactions has a global order which doesn't have duplicate packing problems as in dac protocols if you're a dac protocol then very likely simultaneous blocks will contain a lot of duplicate transactions and it takes either some computational power or some communication cost to remove these duplicates the problem with nakamoto consensus is that its throughput is very low which is why many protocol designers abandon nakamoto consensus or even abandoned proof of work in order to achieve higher throughput is this like scaling

is the speed of being able to like actually just do process transactions that's what you're saying when you say throughput there are two scaling metrics the first is throughput namely how many transactions can be confirmed within a second yeah the second is latency namely that and what's the time difference between the time i broadcast the transaction and the time i consider the transaction settled got it so the main problem with nakamura consensus is throughput i see so from my perspective like before you give up on the commodore consensus check the network let's examine what's the bottleneck of this low throughput can we do something on this bottleneck rather than abandoning the chain based

structure totally so i think the the main difference between me and other consensus protocol designer is that they don't have a background in computer networks when i did my internship at blockstream with peter welle and gregory maxwell we discussed a lot about how bitcoin propagates blocks and transactions which eventually lead to this research so in general from a very high level when we look at a node's bandwidth we can split it into three different parts the first part is used to synchronize transactions which is actually the part we want we want to maximize this part the more transactions we can synchronize per second the higher the throughput the second part is the bandwidth consumed by the

consensus protocol itself we would like to minimize this part which is already achieved by necomotor consensus the third part is idle there are some time that we're not transferring any transactions neither are we transferring any blocks ah you said the word you just used is idle so like when nothing is happening got it right the problem with nakamoto consensus is that in order to maintain its security the orphan rate must be low if we want a very low orbit rate then because all the block intervals follow exponential distribution then we have to have a very long average

block interval the long average block interval leads to long idle time so despite the fact that the network can actually transfer like 2500 transactions per second most of the time the network is idle because if we reduce the block interval then there will be more opens then the protocol will be less secure i think we've we've talked about this on the podcast before but in the context of ethereum but i mean just a very recent episode we talked about how the mempool works and like how transactions are distributed there and when we look at actual mining nodes of like what does their mempo look like they're actually pushing through

you know on the networking side they're pushing through thousands of transactions per second as you say but they're not all being included in a block because they're like competing transactions and people out bidding each other on taxes and whatnot so it's it's it's not a networking problem in like getting the transactions around it's including them in a block and and giving miners enough time to process that block and like continue to build on the next one that's the problem right in ethereum the bottleneck is also a computational one which is why nervous choose to use the utxo model we'll cover that later our solution to this um bitcoins long idle time problem is that

we want to find the bottleneck that leads to the high orphan rate we want to accelerate block propagation and lower the block interval and increase the block size note that we don't have to increase the block size or lower the block interval indefinitely as long as we are managed to find the sweet spot that the block interval is low enough and the block size is large enough that all the nodes bandwidth is exhausted then that's it we've accomplished the job if we manage to exhaust the node bandwidth then we don't have to introduce stack protocols what is the bottleneck of block propagation latency we discovered that the bottleneck of

block propagation is actually fresh transactions in the blocks fresh transactions are defined as the transactions that haven't finished their propagation before they are embedded in blocks a transaction usually propagates to the entire network in 15 seconds so assuming the block interval is 10 minutes for all the transactions that are broadcast in the first 9 minutes and 45 seconds all of them finish their propagation before they are embedded in the block however there if there are some transactions that are first brokers in the last 15 seconds and they are embedded in the next block then it is possible that some of these transactions have not been synchronized to the

network they have to be propagated along with the block the problem with this is that if i'm a node when i receive the compact block the compact block contains all the transactions or the truncated transaction hashes you can think of it as a compressed version of other transactions when you receive a compact block if there's no fresh transaction you can forward this compact block directly to your downstream peers however if there are some fresh transactions you'll have to first query these transactions from your upstream peers and then only after you receive their reply can you forward this compact block to your downstream peers so this extra round trip of

synchronizing these fresh transactions actually becomes the main obstacle in accelerating block propagation wow was this super well known or was this part of the research you were doing to like figure out what this was it is known by the bitcoin core developers okay but this is not a big problem in bitcoin because in bitcoin most of the blocks contain no fresh transactions so in bitcoin whenever a block is first broadcast it can be synchronized to 90 of nodes in the network within 600 milliseconds which is great however if if the block size is larger or the block interval is shorter then there are more fraction of fresh

transactions in the blocks so how do we remove how do we eliminate all these fresh transactions our solution is first we decouple transaction synchronization from transaction confirmation like um in each block there is a field of we call it transaction proposal zone all transactions must first appear in the in this transaction proposal zone as its truncated hash so the previous transaction zone we call it transaction commitment zoom a transaction must first appear in the transaction proposal zone only after two blocks can it be embedded in the transaction commitment zone so the benefit of that is that whenever you as a node

receives a new block all the committed transactions are already synchronized you can forward the compact block directly to your downstream peers only afterwards you query the fresh transactions in the transaction proposal zone and after you receive these fresh transactions you'll forward these transactions to your downstream peers so that the extra round trip of synchronizing these fresh transactions is removed from the critical path of the block propagation and the transaction validity in this transaction proposal zone does not affect the block validity there can be missing transactions there can be malformed transactions or even double spending transactions

all of this doesn't matter because a later miner can just skip these transactions and only commit the transactions that are valid and that they have already received by having this two-step confirmation mechanism we manage to remove the fresh transactions and we can make sure that all the blocks are synchronized in the network within 600 milliseconds so this enables us to reduce the average block interval to two or three seconds we'll also maintain a very low orphan rate it sounds pretty interesting so just to recap i guess the fundamental or the simplified version of it is

if i think about it that as you said like let's say a bitcoin block is 10 minutes 9 minutes 45 seconds of that you can submit those transactions and you know it will be distributed to the entire network and if if your transaction came in that period and was included then that's not a fresh transaction to anyone because everyone has already seen it by the time the block comes out right so essentially what you're doing is restricting the set of transactions that a miner is allowed to pick from to this zone where it will have been distributed to the entire network right you know before it's in a block so it's essentially like you're only in the bitcoin example a bitcoin miner is only allowed to

include transactions that were received within the first 9 minutes 45 seconds right you can only commit transactions that are proposed two blocks before uh so it goes even further back than that i see and the other thing that that had me thinking about this is it sounds very similar to pbft's phases so in peop in practical byzantine fault tolerance you have pre-prepare prepare and commit and you have so you have these three phases it sounds like you're you have a very similar structure here now that's why i always say it's you can think of it as a proof of work version of whole stuff oh wow postdoc is a it's a more obvious pipelining protocol there are actually

two techniques you can use to parallel the processing of transactions in any consensus protocol the first is called pipelining that is used in smx and similarly host staff when you're committing a set of transactions you're actually synchronizing transactions two blocks afterwards the transaction has different cycles and all of these cycles can overlap with each other we call that pipelining i think we actually talked about that in that episode with the thai talking about right yeah the second type is called concurrency the benefit of pipelining versus concurrency is that the security analysis is a lot simpler because you don't have so many simultaneously mined blocks you

can just analyze the security with the previous old techniques and everything remains the same and it has stronger functionality because as global order of transactions is still maintained i kind of wanted to ask the synchronous versus asynchronous question but like i don't know when to ask it is it in the concurrency part or what is it before in general all proof of work protocols are synchronous protocols ah okay that's actually good to hear because i i was i was just think like listening back to the old episode we talked a lot about that that spectrum and the partial synchronous but in this case it's synchronous so but nc max would you also consider that synchronous then given that you've added this pipelining

concurrency kind of updates like that comes to the part actually i love the most about ncmax is that there are other protocols that decouple transaction synchronization from transaction confirmation like prism prism is another example that decoupled these two things but the benefit of ncmx is that the transaction commitment zone and the transaction proposal zone is recoupled in the same block structure which greatly simplifies the security analysis because this is the mx still follows bitcoin backbone protocol it uses the longest chain plus first receive rule so all the previous security analysis on the commodore consensus

can be directly applicable all the previous analysis on the commodore consensus are directly applicable to zmax so that we don't need to prove any extra things like all the security proofs hold we have an extra security improvement is that in ncmx selfish mining is not profitable okay this is a tiny trick but that requires a relatively long security proof if you're interested you can check the paper i actually was about to ask you like you know given that your previous paper was all about sort of finding the issues with these like the new threats were you able to find any new threats that nc max actually introduces actually

there there is a threat there is a threat that is overlooked by previous work that ncmax is able to mitigate it's called transaction withholding attack the attack works like this if you're a miner and you want to perform selfish mining but you don't want any anyone to discover that what you can do is that you make a block that all the transactions are fresh so that your blocks actually propagate slower than all the other blocks which gives you some time to perform selfish mining but nobody can accuse you because you said okay i release the block immediately after it is mined it's just the transactions are synchronized slower than other blocks in ac max that attack is more difficult

to launch because if you want to launch that attack you have to first propose these fresh transactions and then hope that two blocks later you can find a block to commit these transactions you cannot launch the attack immediately you have to launch the attack only when you have two blocks mined at a specific distance that's why this attack is more difficult to launch in cmx than in a motor consensus but i think my question was more is there new attacks that you can't mitigate because [Laughter] i don't think so because because ncmax follows the same bitcoin backbone protocol it uses the same longest term plus first received rule

and relatively the same block structure it means all the security assumptions of daca motor consensus got it so we just covered this like relatively recent work of nc max now i'm wondering you work at the company nervous that's a network i i'm really curious like how does this work relate with the nervous system we have had alan on in the past maybe you can tell us a little bit more about how this work is incorporated into the network it's very simple uh nervous ckb use nc max as its consistent mechanism okay wow and also allen's work the the relation between alice work and nervous kpbs is also very simple allen designs the hash function equals

which is used as the hash cache puzzle of the proof of work algorithm okay so that's our relation with the new project but i think the consensus protocol is the most boring part in nervous ckb it has a lot it has some more exciting features in it okay yeah tell us a little bit about nervous if if you want since we have never actually talked about it on the show sure there are three things i think worth mentioning from my perspective worth mentioning about nervous design the first is that we use utxo model utxo model is easier for user defined tokens in ethereum one of the source of many security vulnerabilities is that all the user defined tokens

are stored in an erc20 contract like all the account balances are stored in the same contract so that if you crack this contract you can basically arbitrarily modify the user's balances so an utxo model allows us to give the access control back to the user your asset is stored in your own utxo and you can decide what signature algorithm you use to unlock these assets so there is no single point of failure in terms of smart contract so wait nervous is smart like you actually can run smart contracts on nervous but it uses utxo instead i actually like are there other projects that do that i always obviously because i've learned mostly

about ethereum first but like i always assumed you needed to have an account model to do the smart contract stuff but i guess not no no you don't need an account model there are other projects that do that i think we are one of the earliest i see i'm kind of trying to picture what that actually looks like like what how does a smart contract run on a utxo i will come to that later like user can define their own log script so in levels scripts are split into two kinds of script the first is called log script which you can think of it as the lock of your apartment the second is called typescript you can think of it as the furniture in your apartment the actual code that you run to execute the smart contract

the benefit of separating log script and typescript is that it removes a lot of vulnerabilities in ethereum basically who can modify the smart contract who can modify the variables is mixed with the code of how to execute the function in novel ckb these two kind of scripts are separated so that it's difficult to attack and you users can define their own lock script you can say that i prefer signature the other can say that i prefer like ecdsc signature these are all flexible this is the utexl model of nervous the second new thing about nervous is that

it's it's economic model the native currency is the storage space in ethereum you pay a one-time fee for storage and that's it no matter how big the storage is it can be stored on chain without any fee forever in novel ckb we use a different economic model that each utxo has some number of storage space and all space has interest if you use that storage then the interest goes to the miner if you don't use that storage then you got to keep the interest oh so there's no there's no fee to the to the user but like the miner will make more there

is fee there is transaction fee no no but i mean like the interest you're describing like it's not that like it's not that the user's paying for storage or something like that yeah exactly like this is state trends but it's not like to keep my contract alive i have to keep sending the contract money right like you're paying by being inflated right yeah and there are transaction fees and what's interesting is that you can pay transactions fee with any any kind of asset you want there is no gas mechanism we decouple the transaction fee estimation from the consensus mechanism as you may know many attacks of ethereum comes from the tight bond between the v estimation and the consensus mechanism

and we think that is not necessary let's remove it and see how it works the third thing about nervous is that it use risk 5 as the virtual machine of its smart contract risk 5 is a hardware instruction set which means it's very very powerful you can program with any programming languages you want including solidity and you can implement any cryptographic primitive you want which facilitate cross-chain operations and algorithm update because in many cross-chain operations different chains use different signature algorithms but in nervous that is not a problem because you can always refer to the library is there a an efficient way to compile risk 5 to like

x86 or does this mean that your miners essentially have to buy risk 5 cpus i'm not an expert in this so i've been told it's uh currently slightly slower than evm but they are optimizing it okay i assume that's in in an emulated mode then because if you're running live on like actual hardware it would be faster right the back of envelope estimation is that it is 50 times slower than running on hardware and i've been told that it's simpler to implement than web assembly the risk 5 instruction set and that's it so maybe to wrap up i'm curious about what kind of research you work on right now

like do you have anything in the pipeline coming up what do you what are you excited about there's one big thing i really want to do is to write a paper called lay down the common metrics 2.0 oh and what would you cover in that there are many new protocols coming out in recent years and some of them still make the same mistake of using simulations as a security analysis i would like to analyze their security and discover new attacks in these protocols cool all right very cool thank you very much for coming on the show it was a great pleasure talking through these different both attack vectors and interesting models as well as nc max so thank you very much thank you

and to our listeners thanks for listening thanks for listening [Music] you