All Core Devs - Consensus #65

Transcript

[Music] [Music] [Music] all right

stream is transitioning over hi all right uh stream transitioning over let me go ahead and get started this issue 220 call 65 i believe all right um we'll go through client updates then we'll discuss altair uh there is a breaking change being discussed

which i'd like to get a little bit input on um then we will switch over to talk about an ox o2 withdrawal credentials proposal and then do the regular uh call for research updates spec updates and other things let's go ahead and get started we can start today with uh lodestar hello everyone so we're

Cayman Nava: passing the alpha 5 spec tests working on getting to alpha 6 but we want to get the preset config split stuff finished first so we're also working on that

other than that we're working on alter interop by syncing to the teku devnets and that's been really helpful we fixed a bunch of altair performance issues and we're currently sinking to the head of the recent one

thank you let's do prism hey guys raul from prismatic labs here um yeah we we have a few things a few uh quite a

Raul Jordan: quite a lot of work done on altair so a lot of networking um transition work is being done uh significant updates on that front

um and for us it seems like end of uh end of this month will be good for multi-client test nets we're still ironing out a lot of things we're passing spec tests and aside from that our team has also been looking into sharding work trying to really just uh try to understand kind of where we can be most impactful on that front um and yeah aside from that still working on aligning to the e2 api's effort that's been one of the biggest engineering challenges for us it's been spanning many months for now uh but we should we're close to the finish line

on that front and uh yeah just chugging along thanks guys great thank you yep hey everyone um so

we finished

implementing the altera rest apis and we now have remote validation working on altair um all the outstanding e2 rest api pr submerged um we updated teku to ingest the new config file format for backwards compatibility we have the get spec right api endpoint including the constants that have been removed from the new config we're just manually injecting this for now we removed our java bls library is too slow to be useful at this point um we have been working on updating our eth1 tracking to fix an issue discovered through rayonism we had been using a static follow

distance and instead need to calculate follow distance based on timestamp um that fix is implemented but it's toggled off for now as we need to do some more real-world testing and uh finally devnets we launched a couple of devnets to test the altair transition i won't try and pronounce them um through that we found a few issues in tekku we're not publishing updated enrs correctly and we had a bug in our transition logic which caused us to produce an invalid chain on the first devnet the transition bug is fixed and we're still looking into the issue with e and r publishing on the current devnet we're seeing some issues with sync committee gossip

that need to be tracked down we'll likely keep that current devnet up for a while so we can get to the bottom of networking issues if any other teams would like to join that devnet and uh you need validator keys you can reach out to adrian

and he'll get you set up um and that's it for us

Danny Ryan: great yeah thanks for putting those dab nuts up i think it's a great way to get started nimbus

all right uh am i on all right uh we started work on out there a bit

Zahary Karadjov: late because we were focused on the march recently um but now our focus and entire leona

out there and just like prism we hope to be ready for the first test net at the end of this month in other news our rest api is probably will be coming out of beta soon we've addressed a lot of feedback from end users for small smaller income but inco compatibilities we are finally introducing uh the use of multi-cores in nimbus and so far we are we will be focusing only on bost verifications but since the majority of cpu usage in a client is the verification of the station signatures

we hope that this would be enough to take advantage of of course and without changing the architecture

of the animals in your other way

Danny Ryan: very nice uh remind me are you doing any sort of batching on the wire when attestations are coming in

for bachelor education

Zahary Karadjov: yes we've introduced a special form of batching which we bundle the incoming attestation into groups which could mean either 16 attestations arrive in a short interval of time which is 10 milliseconds or this 10 milliseconds elapse and we verify

the number of attestations that were received during the time but usually these punches are group we receive many more attestations than 16 in this time frame so that's the granularity that we do right now if any of the stations obviously the problem here is that if any of the signatures is invalid you have to verify each individual station again which is the potential cost if somebody's attacking the network but we believe that the

trade officer we're doing

Danny Ryan: great thank you and lighthouse

Paul Hauner: hello everyone um so we have

uh got our alter implementation sinking and following the chain with seku and perhaps load style i'm not sure um so once again big thanks to tekku for doing the legwork on setting that test net up we're presently on spec alpha 6 and we're passing the new config format and we have backwards compatibility to the old format loosely speaking we have all the features required for lt implemented but now we're working on updating some component style for six namely our sync committee caches and we also need to get everything reviewed and merged into our primary branch presently we're doing interop with a branch that's merges together if you work in progress prs so we're doing interop on the branch

with work in progress prs no doubt as well we've got some lte issues that we don't know about yet that we'll have to deal with we've done today a pre-release the first time so we've done a bunch of outgoing columns to users to tell them not to run on mainnet and explain what we're doing with it if there's any power users on this call that are running lighthouse i'd suggest giving it a run on testnets there's some good memory improvements and we'd like to see how they run on lots of different hardware and finally we're expecting to publish a week next a release next week um assuming that the pre-release goes well so this is not the full feature set we'd announced previously but it is early than we'd initially announced for the for that release so we're just going to split up the features into two

um two releases and we're expecting to have the rest of the features including doppelganger and altair uh in a release later this month

and that's it from us

Danny Ryan: great thank you all right moving on to altair discussion um it sounds like engineering progress is moving along uh thanks again for setting up those devnets i think this is a great way to start just it's been rough in place um we did patch the test vectors yesterday for alpha six there were a couple that were incorrect and then i accidentally uh there were a couple that were missing uh but the tars on that release

should be good and complete and correct at this point um we are targeting an alpha seven tomorrow pending uh discussion of this feature so i'd like to shift discussion into that um i don't think potas joined us but photos identified that essentially how the sync rewards worked um it created a pretty asymmetricness and rewards that could be unfair based off of how you win the lottery or not as in if you get selected for the sync committee um you know you see something like a 15

boost in your rewards and if you don't which could happen to a not insane amount over the course of like a year year and a half um you fork out those rewards so there are two pr's up i only link to one my apologies one has essentially shifts the entirety of the sync rewards to be a penalty so essentially by default you do get all these rewards um and if then when you are in that scene committee you don't perform well you get a penalty uh reducing the lottery effect or eliminating the lottery effect in the positive direction um and then vitalik has an alternative which is 2453 which modifies some of the logic reduces

the penalties and reduces the magnitude of the effect of this reward and has not only a award but also a penalty for non-participation um in effect reducing the magnitude of the effect of the lottery to something like three percent rather than 15 um i think it also vitalik noted in this pr that the sync committee being as valuable in the long run as proposing uh was probably not a good call anyway so both of these should probably reduce the magnitude of the committee award um there's two questions here um one would be uh the fact that this is a breaking change

and it would need to be introduced very soon um it's very small test factors would be re-run and get coverage on this thing but uh it is breaking nonetheless and the other is the balance between these two um i think some have signaled that they're into vitaliks uh but there's also the argument for protest um vitalik can you make the argument for your pr

[Music]

okay um are people are you hearing me okay now yes um okay so

Vitalik Buterin: basically the way to think about it is like status quo plus one if you participate zero if you don't participate potential proposal zero if you participate minus one if you don't participate and my proposal is plus 0.5 if you participate minus 0.5 if you don't participate uh so the two uh biggest or uh arguments um in favor of mine where one is that if you do pure penalties and no rewards then this introduces the first proposer reward that is not directly computed as a multiple of uh

rewards for things that it includes and so that breaks the invariant that we currently have about how proposers or proposal awards are calculated which it's like it's not fatal but it does just like make will make it continually harder to make sure that incentives are that rewards add up to one in the future and the second one is also that we've historically had a goal of uh not me of not wanting innocent people to lose a lot of um a lot of money uh and if there is during situations like validator leaks and if there isn't in a then like base if you do the kind of

plus zero minus one thing then the uh like basically if half the validators are offline then half the uh purple the the next salad proposals will be missing and that means that like basically you will if you are part of the committee at that time you will lose quite a bit um and then the second but then if you have plus 0.5 and minus zero and minus 0.5 uh then you like being in a sync committee only becomes a net negative if more than 50 percent of participants are missing and even then it's so much smaller net negative um oh i saw poachers replied in the discord for that

like proposers get rewarded by what they include i guess just like clarifying what i mean is that like if you do the plus zero minus like right now right if you include like proposer's entire rewards comes from the idea that like if someone else if you include a message that causes someone else to get plus x then proposers would get plus x over um x over seven right whereas if you do the plus zero minus one thing then like you would have to give proposers like a some reward but that reward would not be any kind of plus x by um over seven it would be like plus some uh plus something

else over seven uh so

Danny Ryan: question on the uh the penalty mechanism here this is only these

rewards are only run in processing committee which are only run if there is a block and so in the event that there is no block no one gets a reward no one gets penalized in both proposals correct

um oh good that's a good point um i think you know the the challenge the the challenge is like what if they're um um oh i see it actually i don't think they even thought about that considering the exact consideration that makes things up even like that makes things even weirder to like for the uh is especially for the

more like a penalty-based approach because that basically means that like you can gain by knocking other by knocking proposers offline which is generally the sort of thing that we try to prevent uh so i guess that would even be a a different argument in favor of keeping the penalties milder it's right right you

Danny Ryan: you can either do your job or you

can do some posters um but your job is also entangled with getting blocks online without decisions so the the incentive toss is probably actually extremely minimal

but no because actually coaches were saying in the discord they're just saying the penalty computation is not correct then i guess like i know just want to reply again i guess i'm a bit a bit confused like it's my characterization i think thank you adrian you just send me the link uh i can so the way

Potuz: the way that pr works is you

get the reward and you don't really get a penalty you just lose that reward uh if you if you just don't participate in that particular slot so the worst thing that can happen to a validator that is not participating on on my pr is exactly what would happen now as it is in the specs

so it's not really getting away but then hold up but then i'm confused that how does that address the the lottery issue because

Potuz: everyone gets by default the sync reward up front and if you're not participating it you just take it back from it so it's not really a zero minus one in your pr actually it is an actual penalty

in your pr if you oh i see it wait so you're saying oh so you're saying hold on that in your oh i guess so you made a you made a new approach then that's like okay uh so you're saying in this in this new approach that um the sig committee reward is just always given by defaults given to everyone and it's even given to people who are not part of the same committee so it doesn't get like modified by a probability factor and then if you i see it okay and then the proposal okay

Potuz: the proposer does get penalized if he doesn't include a sync reward

so right i didn't see him it still included that issue

Vitalik Buterin: but then does this not make the uh incentive to participate in state

committees and the incentives to include security signatures fairly tiny

no oh well so the proposal gets uh a reward according to how much he includes so one over seven is still part of the proposal so okay again but it's one over seven but it's one over seven of something much smaller right like if there's like basically like let's say for example there's 10 000 validators and there's 100 committee members then what we're saying like well the the status quo right is that if you're in a sync committee and you um get and you get included then you get plus 100

and if you don't get included you get zero and then everyone else gets zero but your proposal would be saying if you're not in the same committee you get plus one and if you're in the same committee and you propose you get plus one and if you're in this or if you're in distinctivity you participate get past what if you're saying humidity in this incubating you don't participate you get plus uh plus zero right am i understanding that correct so the incentive does

Vitalik Buterin: so the incentive to participate in the sig committee does go down from being

plus 100 to being plus one yes that it is true that's true so for participants if

Potuz: the incentive goes down uh you can right however you can make the penalty independent in that vr so you can actually increase the penalty quite a bit so that it actually becomes negative

without changing the proposal mechanism

i see um okay but then the proposal reward in your proposal is it like is it plus one times one over seven or is it like it's exactly as it is now it's exactly exactly the same thing the same numbers as it is now i think okay okay um well so i could but like

Vitalik Buterin: for example let's say 80 percent of people in the sig committee who get a reward right or participate then like there would be 80 people that get plus one and then there would be well there will be 9 900 other people that get plus one and then 20 people that get plus zero would the proposal reward be ninety nine point eight percent of the maximum or

would it would it be eighty percent of the maximum

it would be uh eighty percent of the maximum okay i see him right so that like that that's still i guess

Vitalik Buterin: a bit decoupled from uh rewards but i guess it's decoupled in a less harmful way um hmm yeah i know i i see the proposal i guess uh yeah my my main concern is just that like if you do like you are cutting down the reward by a factor of 100 and like cutting down the reward by by the factor of uh or cutting down the incentive by a factor of two seems reasonable by

cutting down the incentive by a factor of a hundred or a realistic way it's much more than a hundred is uh feels a bit on the risky side like as a client developer for example or as for example right like if i make a custom client and i just like never bother to write sync committee code then i would still get like over 99 of the for 99.9 plus percent of the rewards which so you could fix that with penalties but then we're back to having penalties

but just a second

Potuz: on the incentive parts if you're if you're a client if you are already say that you already are in the lottery and you are in the committee and then given that conditional probability you you're now

evaluating whether or not you should include or not uh your signature so given that conditional uh probability that you're already in the committee i think uh my proposal doesn't change anything right because you you lose the exact same that you would gain uh by

by participating well like status quo is that condition on being on the committee it's like it's plus 100 versus plus zero but here you're saying it's plus one versus plus zero no i think so currently you if you

Potuz: so currently if you're not participating one slot you do not get that reward for that slot and it's exactly the same report that you would get for

the penalty so in my in my situation is exactly the same if you do not participate in that slot you just get a penalty for for what you would have gone gotten for that

war so it's exactly the same in that sense i don't see what else but like it's still like it's like

Dankrad Feist: the question is that penalty is the same magnitude as the reward is now

Potuz: yes so the so currently okay it's set up that way but it's independent so you can just change that parameter and make it harsher

if you wish and then the incentive decision so that means that means actually your proposal is something like

um like plus plus one for participation at minus 99 for uh as a penalty well so so you only get penalized except that one also applies when you're not even in the same committee right

Potuz: that's correct but i don't see why the minus 99 analogy so so this only the penalty only applies when you don't participate in that slot

exactly that's we understand that yeah i think if it likes questions

Dankrad Feist: i think vitalik understood it as the penalty like you get this very small reward every slot like that's a hundred times smaller than it is now because right now like we have this huge reward when you are in the string committee

but none when you're outside so you're making that much smaller and you give it always and i think vitalik understood that the penalty is just the negative of this tiny reward but i understand you know saying that the penalty is actually much much larger like it's it's actually the same magnitude as the reward they're slot that's correct hmm okay so then basically 100 is our our number for what the reward is

Vitalik Buterin: right but then in that case like like so

if like if if it is if the proposal is plus one minus 99 then the problem that like to a single committee during an inactivity week you're going to lose a lot

still oblige right

Potuz: i think so but i think you will you'll lose what you already gained so you lose less than in your pr in your pr if you do not if you do not uh participate you actually have a negative income

in mind yeah yeah yeah but it's like you just you're just proposing basically

your

Dankrad Feist: proposal is equivalent to saying we have a universal basic income for all validators no matter what they do and then we add a large penalty if you don't participate in the certain committee which is exactly equivalent to taking on average to taking that universal basic income away

that's correct yeah right okay but if it's but if the penalty is just taken away then it is plus one plus one plus zero and not plus one minus ninety-nine right so that that still means that the incentive is small no

Dankrad Feist: no no the incentive is large in this proposal no but like probably you have the inverse lottery now like basically before you had the lottery that oh you

can be lucky you can gain lots and now we have like a death lottery like you can't be unlucky and be selected exactly when

like that's that

Potuz: i agree that it's it's an inverse lottery but that inverse lottery in general would only apply for a very tiny set of genera of validators so it's not only that you have

only 512 it's so it's still kind of offline and that particular slot

Dankrad Feist: exactly but you can't just be unlucky and that can feel pretty horrible

i mean that's that's just i think that's true that this is about like the psychology of it is very important no but

Potuz: the other way everyone is unlucky that's the point so the the way that on italics now which which

actually italics pr reduces a lot the variances but even competing with those numbers you see that you get 17 almost 18 of validators will be over four times in a committee in two years and only to eleven percent of validators will not be in any committee in two years so that's a fifteen percent of the rewards difference between those two

hmm [Music]

Dankrad Feist: the other possibility is to just make the committees larger overall like just even if we don't see don't currently foresee anyone using it

we could just make them

10 times larger and say like well um yeah just we have a lot more validators in there and that reduces the variance right

Vitalik Buterin: but does that mean that we're bringing back the aggregate public he doesn't mean we have to like change things um yeah yeah yeah like that if yeah that feels like a starter um so i guess it does feel to me like there's this kind of fundamental trade-off that you know either there's a lottery or there's an anti-lottery and if just the the lottery

or if you traded down to zero then like during a weekly um in the ncaa lottery becomes pretty severe and like that's at least a violation of a property we've been trying to uphold

whereas but then on the other hand yeah i mean maybe also like the

Dankrad Feist: the other thing is um after the merge we probably have a much bigger issue than this lottery with the mav lottery so it's not clear that we're gaining that much by over optimizing this now right like i mean i guess like it's good to like the this problem is tiny to what we're going to inherit in a bit

and like this kind of feels a bit yeah

Vitalik Buterin: right right so the merge water the the posterior would be like you get elected as a proposer at the same time that there just happens to be like an unusually large market movement

yeah and that's probably the sort of thing that you get over of course easily bigger than gonna say committee um actually we could quickly you're breaking up pretty sorry sorry i'm saying like

Vitalik Buterin: we can calculate the size of the sync committee lottery like it's something like

once a year for a day uh or once um you gain the or you gain the equivalent of what was it something like one and a half weeks of uh revenue and one and a half weeks of revenue for a single validator is gonna be what like something like two is multiplied by three percent so something like zero point zero six maybe zero point one each so i i can easily see that being much smaller than

maybe water is

Dankrad Feist: yeah one block can probably net you like yeah more than an easy [Music]

i mean i guess like yeah you know we need to fix those as well obviously in some way

but i agree but like

Vitalik Buterin: i feel like with mbv we're not we're never going to fix 100 percent like there's always going to be a very unusual market crunches

Paul Hauner: are happy to implement uh either of those two in terms of um complexity it's fine not a problem for us

yup sim here so how do we yeah i'd agree with that i guess the

Adrian Sutton: the one concern i have is that if we're struggling to come to a decision

um so i guess to take a step back the the other question to ask is if we don't fix this before altair how how much of a problem is it and how soon so if it waited till next four can we pick one of these two solutions and implemented it is that a problem or it probably is two forks away given the

merger's name

Danny Ryan: right i mean i think probably for one the constant is too high to begin with uh that's an easy fix um and then i think it would be better to put one of these in than not but not

catastrophic if we let it go

Adrian Sutton: yeah i mean i guess the other variant of that is to say well let's just pick one practically randomly we don't have to fully agree on it just pick either put it in and then we can continue the argument and change the other if we need to there's a reason

Dankrad Feist: to me after this discussion it feels like the plus point five minus point five variance is the best because it minimizes variant variance across all the ones it's only a 1.5 reduction variance but that's better than nothing so

i would just go for that one

Danny Ryan: does anyone else feel strongly one way or the other

okay so is that what go going for the plus point five minus point five or

Danny Ryan: um i would like to do that because i think it's reasonable and it's simple

and it generally achieves our goals okay uh we're gonna do a final review on that pr and likely mergers today and get out in alpha

seven altair planning

Danny Ryan: um generally it sounds like people are willing and able to target uh something more substantial than these short-lived devnets end of june

maybe first week of july um so i think proto xiaowei and myself will put uh some work into um planning something getting some configuration parameters and some timing in place to plan something a little bit more coordinated than the short-lived test nets but i would encourage uh maybe adrian to stand up in alpha 7 version of what he did uh these this past week i think that was really valuable in the meantime to kind of sanity check things and get things in place um and then from there let's see in the next two weeks we will pick

maybe a a date and a set of parameters for an initial um more coordinated test set cool assuming that goes well assuming we get some fuzzing action going and assuming uh we continue to enhance the tests uh that would be a positive signal to move towards picking um upgrade dates on our existing two test nets um probably doing another test net in the meantime and then looking towards mainnet um i think the the big unknown it sounds like with client teams right now is sure the bones and things are in place the tests are passing but we want to get some some more time

to sit with it more time to do testing more time to iron out the bugs on these test sets so security is certainly a priority and we will make informed decisions over the next few weeks

Mehdi Zerouali: quick question um is alpha 7 expected to be the final spec change at least in terms of consensus code

Danny Ryan: yes alpha 6 was also the expected final spec change before potos identified that kind of asymmetric issue with rewards um so yes it is the intention unless something serious with security is identified it will be stable

Mehdi Zerouali: sweet thanks

uh what about the state truth change

Danny Ryan: what's the state of that conversation

uh i think there's weak there's

Jacek Sieka: kind of agreement that's useful um there is one contention point which is the backfilling which is uh considered uh complex for some clients and the proposal is to make it into a two-phase operation so that on this heart on this fork

we make the change to the beacon state and on the next fork we backfill existing values as sort of a one one-off operation

Danny Ryan: so this is the alt the alternative that meets the same need is essentially on the network side serving batches of blocks along with a stateroot list root right and so defining a network format that gives you the extra little piece of data which is 32 bytes to be able to prove against the current items in state right

Jacek Sieka: um sorry can you ask again i didn't understand

yeah so

Danny Ryan: right now what we want to do is be able to serve batches of blocks against uh the state the block root list right now that is obfuscated in historical roots by combining those two lists what that looks like from an ssc proof if i wanted to serve you the block roots is i can give you along with the block roots a 32 by root as as a single proof that 32 byte root is the root of the state root list for that batch historical batch um so if i define a network format that allow

that allows me to pass essentially a small proof which is their two byte root and the block roots then i don't have to change the state right

Jacek Sieka: um i think it's the other way around i think it's that once you do have a state you have to you have the ability to verify uh any batch of blocks any 8k batch of blocks that that you happen to have for the whole history of these right and then so the idea is here there are two use cases for this um the first one is weak subjectivity thing

so basically the assumption here is that you're given a state that you already trust and then with this change you can quickly prove that any batch of blocks is part of that particular state right whether you receive that from the network or disk or wherever else right uh the other i think the other nice feature about it is that it gives us this like natural identifier for uh a day's worth of blocks and and we can use that later on for arcade notes archive purposes and very easy to coordinate around um these chunks of blocks so

again for weak subjectivity think it becomes very easy to do sort of out-of-band transfers of these chunks of blocks and then the final reason is obviously that if you want to create an archive node that serves um or really when you want to dig into the history of voting for example you can very easily prove that a particular block was part of um out of a particular state that you're trying to work with

Danny Ryan: right i guess i'm saying for a number of those use cases say for weak sensitivity sync where somebody's giving me batches of blocks they can give me the batch of blocks along with a single 32 byte route and i can prove it against what is currently

in the state so yeah it goes from batch of blocks um the so i'm just saying if you didn't if no change happened you could define like a batch block format which essentially a batch of blocks and a small proof um and where in many of those cases you could just pass those along rather than batches of blocks i'm not saying that that's optimal um i'm just saying that if this didn't go in that would be the alternative um i what is the state of the pr with

respect to testing and okay so the current state has historic batch summary um yeah i i mean at this point we could introduce a breaking change because alpha seven is going to introduce the breaking change i'm want to hear what other engineering teams think

Paul Hauner: to be completely honest from our end we

haven't been particularly across this one i think we're pretty full with bandwidth and just this one got left out but we can we can look into it

yep same here i haven't really studied up here as well so i'll probably have to do caddy again and then um and then uh get back to you guys

Danny Ryan: okay so if we delay alpha seven until wednesday then we can make a call on this alpha 7 looks a lot like alpha 6 as of now so it's not adding like too much issue and complexity but um is that reasonable can i get people to take a look at this by end of monday

Paul Hauner: yeah totally i like what um the pr is delivering i just can't make any calls on implementation at the moment

Adrian Sutton: yeah i guess my only concern is is the um the backfill stuff if we did it in two forks i think it's simple to to drop in

um yeah

Danny Ryan: so if we don't do backfill this is still is this still useful

Adrian Sutton: well i think so the thing for me is that i think we probably wind up with a two-fork approach anyway because it's the simplest way to do this it's just kind of it's kind of where each one landed on these kind of

approaches a while back um so if we put if we start building the new state now then next fork we can do the backfill and and we're kind of where we are if we punt this and argue about how to backfill it and then wind up doing two forks or even if we sort out backfill it's gonna it's gonna push it into the the next fork anyway so i kind of think if we get the usefulness next fork pretty much regardless of how we do this and there's a risk that it maybe becomes two forks away if we can't sort out the backfield before next

four

Danny Ryan: i see is there a technical is there a reason that you can't

pre-set the backfill so that you do the backfill at the same fork or is it just kind of a compromise at this point because it's not ready

it's

Adrian Sutton: it's probably doable uh you'd have to get client releases out that that add to the storage and kind of keep stuff around that we don't currently keep um but that that kind of increases the the engineering scope quite significantly over the change um we could probably do it but it's a big change to be putting in lake at that point

right oh

Danny Ryan: i wasn't saying necessarily

doing it for this fork i'm saying if the backfill was going to happen a fork a later from now and this isn't necessarily entirely useful without the backfill then if you can batch both together and do it in next fork then you you've gotten the usefulness at the same point in time i need to i need to study the pr as well

Adrian Sutton: yeah uh i i think backfill is is likely to be workable if we have the time to do it well um that is probably preferable to shoving something in especially if it doesn't have used if

it's not particularly usable um but i'm i'm happy to be told if it is used like if if building the state now starts to be usable sooner then i think we could fit it in

i'll defer to yes on that

Jacek Sieka: um it is usable in the sense that we can compute we can actually compute the backfill a day before the fork and if you think about it we can actually ship that as a constant incline way before the next fork and that means that we could start using this feature

right after the fork if we had it in there by shipping that backfill together with the client you know in the next client update after the force uh yeah that's a good one like the contention point here is that or the difficulty of doing a backfill is that [Music] you can only create a constant for up to like there's only a day of data in the state right and then it's lost so um but that doesn't mean that you can't ship a new version with the backfill already because that's by that time

historical information so um by doing the fork now all the clients start recording this data in the states if we do not do the fork we will have to record the data out of state until the next fork and then do the backfill which you know um for us it doesn't really matter that much we can easily attach stuff to beacon state in our storage but it does complicate um implementation in general is the feedback so i would say that it's useful to get it in now in this state and it

becomes useful immediately when clients make a release

basically

Adrian Sutton: yeah that's a good point i hadn't thought of that but i think that's a strong argument for me in favor of doing a two-fork release regardless because you can get the benefit very quickly um and it's just so much simpler and easier to test you keep the property of one state and plus a block and that's what you need and i think for the transition

Jacek Sieka: yeah i mean it's a good compromise i agree

Danny Ryan: so

let's study up on the pr um i definitely am worried about scope creep on altair just i i know that we're making a breaking change to the way scene committee rewards work in the next couple of days uh but that's like a few lines and most of the testing remains very stable whereas this is kind of a larger unknown item at this point and i just given that we are delayed on altair and given that one of the reasons for altair is just to do a warm-up fork um you know i just wanna caution digging deep into

another

feature but i'd like yeah i'm sorry you're good uh yeah just don't let the line number scar you it's mostly renames

Jacek Sieka: as proto mentioned in his in his comments um the actual changes is one line two lines uh one in the beacon state and one in the uh historical batch calculation function so we already calculate all this data in the current version of the client it's just that instead of storing one root we'll be storing two

right i understand no not trying to make and

Paul Hauner: not trying to poop with this idea but it is it does add work in terms of um for us in terms of storing this stuff in our database like we have to spread that thing out uh in our frozen database and stuff like that i don't think we can just copy the existing code we have but there is just some changes there that that are more than just the lines in the spec

Protolambda: i think so even though i like the way we can minimize the spec changes that we should really take the backfill series i'm i'm also worried about

scop grip

Paul Hauner: do you mean take it seriously as in

don't do it in a two fork solution

Protolambda: i mean we all have to test the two fork solution we have to document the two fork solution and it all has to happen in parallel to the already delayed fork

Danny Ryan: meaning don't put this feature in unless you've figured out what the second fork look like looks like which takes more time and effort at this point okay um we are

not quite running late but let's continue um continue conversation on altair in the next few days on that pr um oxo2 credentials uh the author of that is here would you like to give us a quick update on that or a quick tldr on that

sure you guys hear me okay yeah all right uh so i'm joe clappis i'm

Joe Clapis: joined by darren langley we're from the rocketpool team uh we've been paying attention to the merge pretty closely as you might expect and we're really excited to see nocturne working but we didn't notice a problem with it so i don't want to assume everybody has read that pull request so i'm just going to summarize it really quickly here discuss our proposed solution in that pr and

then we'll we'll go from there once that context has been set so right now there exist cases where a validator isn't owned entirely by one party or it's not owned by the party that's running the consensus client or the execution client and in those cases we still submit that the owners of the validator should be entitled to their fair share of all of the rewards that are generated by validation duties now the oxo1 withdrawal prefix solves this problem for block proposals and for attestation rewards by sending them to an address on eth1 during a withdrawal and that address can be a smart contract that handles the distribution logic appropriately but

with the current merge approach priority fees for transactions that are included in blocks are sent to the block's coinbase address which is set by the execution client owner as a command line argument it's not controlled by the validator owners so there's a decoupling there but this is a problem because the execution client owner can take all of those priority fees for themselves by simply setting that coin base to an address they control and not distribute those rewards to the validator owners this is a problem because the execution client owner they they take all of those fees and the validator owners don't get a fair share

of those rewards that's kind of the issue here so in a decentralized staking context we can't force a node operator to share with the rest of the pool for example by setting that coinbase to a splitter contract that handles the delegation appropriately if those node operators behave selfishly and they pocket those fees it means the overall return is diminished for the stakers that invested in that validator but that don't own the execution client which means decentralized staking solutions based on this model don't really have an equal footing compared to the other platforms they're less viable so that is in a nutshell that's the tldr of the problem

now a solution to that problem has to fix four criteria that we've come up with number one it has to allow for fair distribution of the priority fees to everybody sort of invested in that validator number two it has to scale to arbitrary balances and this is one of the sticking points with us because if you go via a punishment model for example where you withhold the initial investment into a validator that the node operator made you can only punish them up to that investment you can't punish them beyond it whereas somebody that abuses this mechanism could accumulate rewards beyond that so that punishment model doesn't really scale we need something that would number three we need something that

doesn't interfere with existing validators on mainnet today that use the oxo or the ox1 prefixes and uh based on some discussion we had in the r d discord it turns out there's actually 3200 validators on mainnet right now that are using ox1 as a prefix and they didn't sign up for any of the changes that need to be made to support a solution to this problem and we don't want to enforce those on them so we need something that doesn't interfere with all of the other validators today and then finally number four is we want something that doesn't really add any appreciable burden to the merge i mean as we've heard today it's already gone through some delays and we don't

want to further exacerbate that problem so with those four criteria in context we started looking for a solution internally with the rocketpool protocol itself and we couldn't find one that fits all four of those criteria the problem really ends up being scaling those being able to steal arbitrary rewards in perpetuity this this needs to be addressed at the consensus protocol level that's kind of the decision that we came down to so we workshop this with a couple of the core devs in the r d discord and we came up with a potential solution uh which is where oxo2 the withdrawal credential prefix comes into play so this is a new prefix it is an

extension of ox01 in that it also specifies an eth1 address to withdraw to when time comes in the validator exits but it comes with an additional condition which is that blocks proposed by oxo2 validators are only valid if the coinbase for that block is equal to the withdrawal address set in the withdraw credentials now mikhail wrote some sample code in our pr for that it looks promising it's actually a pretty light change uh and you think that this this issue would have some implications for coinbase logic for the execution clients as well especially in the case of a like client

provider like infire for example but his code addresses that problem too so this this proposal has benefits not just for rocketpool but also for other users beyond rockypool for example block staking could provisionally use this so they don't need to internally manage a map of the coinbase rewards that they earn to the withdrawal addresses for all of their clients and that's something that they've specified they would like to investigate so at the end of the day uh that's not the only solution it is a promising one but it's not the only one it's just the current mechanics disincentivize decentralized staking in ethereum

and we are simply looking for some solution that levels the playing field for all the platforms

so in a nutshell that's what the pr is about and what it tries to solve right

Danny Ryan: thank you so for one i i don't think we're going to come to a solution on this call but i do think this is good uh educational context for everybody um personally i'm not i i do see the problem um i think the naive solution of like fraud proofs or something that uh could burn somebody's capital uh does as you pointed out have a limitation on um how much can be burned or taken um and especially if the withdrawal

credentials cannot initiate exits say from smart contracts uh then you know they can kind of hold on to that capital uh the validation capital forever so there are problems there um oxo one was put in place as a compromise um it because it required no consensus changes to support and just a future promise um the oxo2 for it to actually be um enforced at the point of merge would require some line changes to the merge and so does uh kind of encumber that process a bit more and so that's

i mean ultimately it ends up being like a feature request on top of trying to do the merge at the same time which in general we've been attempting to avoid um there are two potential paths there one would be to avoid that would be to not define oxo2 until after the merge which i'm sure people that want to use it um would not necessarily like and then the other alternative is the that there is a time between the merge and uh when oxo2 is actually forked in where you can be using oxo2 but you don't actually get the guarantee of um the enforcement and the enforcement could be forked in later

um again i don't given given that this call is primarily focused on altair and shipping that i don't think we're going to get too far into this but um i'd be curious if

you had any response that or if anybody else has any any input why why and you why

Dankrad Feist: why not just fork this in for the 4x01 addresses that seems to make more sense to me

Danny Ryan: because there's like there's they're fun yeah i because it's it's an it's an unknown implication for the logic change for people that are using oxo one

uh but i suppose the setting of coinbase was already an unknown introduction to that so you could argue both ways probably

Dankrad Feist: i don't know i mean i would disagree i mean my assumption is that the withdrawal address is owner of the funds and whatever you get as a return on block rewards is actually part of those funds like i actually think like i mean it's even a question why yeah shouldn't the coinbase be signed by the withdrawal key in case it's a bls key as well

i don't know i mean there are yeah questions about that because this kind of goes against the spirit of the purpose of the keys if you don't do any kind of

um right so you're saying the issue doesn't enforce any kind of alignment then the issue exists with oxo already as well yeah

Dankrad Feist: i think so because like i mean i might have given someone like uh like i might have contract with someone like stake for me and i keep the withdrawal keys and now suddenly changes and they can like before the only way they could extract value was kind of like um extortion and bracket but now now they can just pre post blocks for their own eth1 address

Adrian Sutton: i guess the other way of looking at that is um that

it would have been perfectly reasonable to pay the inclusion fees to the validator balance which would then put it under the control of the withdrawal key

um so it's not not over there

Joe Clapis: that was one of the solutions we looked at originally and we decided to not go down that route because it adds an extra synchronization portion between the execution layer and the consensus layer

in this case oh yes the withdrawal it doesn't end up mattering here

Adrian Sutton: yeah i don't mean we would actually do it i'm just logically if if if it were paid there that would seem quite reasonable and that has the same net effect as

requiring the coinbase to be the same as

your ox1 address right it does okay i mean i now i'm gonna say exact pretty much

Dankrad Feist: the exact opposite of what i just said i best realized that it doesn't make sense because there's like the only thing that the coinbase address receives after one five f9 is the tip and actually a tip can also easily be paid out of that so this is like this this kind of i mean i i agree that it should be going to the to the validator owner but there's actually no ways to force it like if we if we make this kind of change then you can still easily like get someone to pay you out of

bands for the

transaction just keep that money that's right

Joe Clapis: this doesn't solve side channel problems and it's not intended to solve those

and

Danny Ryan: it doesn't solve muv as well like that can go to any uh any address

so it doesn't it's not tip is part of map

Joe Clapis: it's not intended to solve mav although if i understand right the way flashbots works right now for example is they use the coinbase to do that distribution so it might provisionally help there as long as you play along but again this isn't intended to solve the side channel problem and i would submit that ignoring it because psi channel exists isn't very fair because this problem still exists for honest node operators

Danny Ryan: but they're not necessarily acting honest if they put a different coinbase in right

if that's part of your agreement yeah

Joe Clapis: yeah and there's also people who don't know how to run mev but can still leverage this mechanism

Micah Zoltu: so i worry that by having a making it financially more profitable to accept transactions over a side channel makes the situation worse because basically you're saying hey i'm about to produce a block if you pay me over the side channel i'm willing to work for less because i get 100 of that and if you pay me if i have to pull a transaction out of gossip that ends up going to someone else so i'm just going to ignore gossip transactions because that could distribute to 200 other people

um and so then we're kind of creating an incentive where we're saying hey it's cheaper to send transactions over the side channel because miners or block producers rather um get to keep the whole thing like whoever actually creates that block running that execution client gets all of the side channel stuff and they get a very small percentage of the gossip tip transactions and so i worry that we're actually disincentivizing good behavior of transaction submission and block producers in other ways by kind of encouraging side channel work

Joe Clapis: it's definitely a related part of the problem yeah and and perhaps an entire solution needs to account for both the coinbase issue we've highlighted and side channels

i mean that would be great i i don't we don't have any solution to the side channels right now that we just have a solution at the the claim-based problem

yeah

Micah Zoltu: my worry is that i think this i think the side channel problem is insurmountable um because you can always do site channel payments like as long as you're not required to pay the miner i guess entirely optional like the tip for one five nine is then you can pay them in whatever way you want like the ultimately it's the block producer that gets to decide the transaction order and what's included and that's like getting out of that is a really hard problem and i'm concerned that solving half of it just makes it worse

mikko

Mikhail Kalinin: i just wanted to add if we enforce this logic for 0x01

then we are removing the option of changing the coinbase so it will be the same address because we don't have a way to change the withdrawal credentials um on the beacon chain so that's to consider if we go this road uh yeah and also if we don't and we don't want to like put extra complexity to the merge to the point of merge and we will likely enable this logic after the merge if we decide to so uh there will be a period of time where a coin base will be configured as it is now at the main net and then we change the behavior over already that the users are used to

have for 0x01 and that's probably the issue for them for users so that's why the new prefix makes sense

Danny Ryan: okay um this pr is up for continued discussion i think that at least everyone has a good introduction to the problem at hand um i do want to reiterate can you link that yeah we'll do the um i do tend to think that half solving

side channel problem actually does exacerbate the side channel problem or half solving the coin based problem um okay um i i don't think that we need to make decision at this point but i do think that um if you are interested in chiming in please jump into 24 54 and i also believe their people are conversing about this in the withdrawal credentials chat on the authority discord thank you joe

Joe Clapis: thank you

Danny Ryan: okay uh longer meeting than usual let's wrap things up any research updates today

great and uh any other spec discussion beyond what we've been discussing in the altair context additionally if you weren't on the merch call this morning there is a um merge pr up for transition total difficulty and dynamically calculating that at the point of fork check that out okay any other discussion points or closing remarks great okay we will close for today

i

appreciate it we'll jump on a call in two weeks um plenty of i'll tear progress and fun between now and then thanks everyone [Music] [Music]

you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.