Next Generation Quadratic Funding with Joel Miller
27 Apr 2023 · 38:35
Joel Miller, a computer science PhD candidate at the University of Illinois Chicago, joins host Kevin Owocki to discuss his research on pluralistic quadratic funding, including a paper he co-authored on leveraging social information to resist collusion in funding and voting. He walks through two proposed mechanisms, how they perform at scale, and what pluralism means for the next generation of quadratic funding used by platforms like Gitcoin.
Speakers
Topics
Transcript
foreign ation on the Pod today we've got Joel Miller who along with his co-authors Glenn Weil and Leon Erickson published a paper in December of 2022 entitled Beyond collusion resistance leveraging social information for plural funding and boating basically this is a quadratic funding episode we are talking on this episode about how to make quadratic funding more civil resistant and more collusion resistant so if you're a mechanism nerd like myself and you're into quadratic funding this episode is for you um if you're not into mechanism design then maybe this is an episode that you want to skip but um basically I'm excited about this problem space because
this paper explores pluralistic qf mechanic mechanisms that recognize the importance of social connections and incentivizing cooperation across social differences so basically this idea is that if you have a plurality of people who are diverse that like a public good then that public good is more likely to be good and we're exploring the inverse of that is collusion resistance so basically if people are colluding around a public good how do you build a mechanism that is uh Not Fragile against that so in this in this paper Joel and his co-authors propose a new mechanism of quadratic funding that is collusion resistant that is civil resistant and is pro plurality and it's that mechanism mechanism that we talked to talk about
on this episode I thought the paper was really meaningful and impactful and I wanted to have Joel and greenpelt to talk about that paper Joel is a CS uh PhD student at the University of Illinois Chicago and does research at the intersection of computer science economics and social sciences and has been working with Glenn Weil the author of the quadratic funding paper um well actually Glenn and Zoe and vitalik are the authors of the qf paper but uh Joel has been working directly with Glenn for the last couple years and um this is a just an exciting episode for me as a qf mechanism nerd and someone who's into into plurality so I think that if you are into qf if you're into plurality if you're into collusion
resistance and fraud resistance then this episode is for you without further Ado I give you Joe Miller hair Dow is here to Route Around the middlemen in biomedical research for hair loss hair Dow is an open source investment and operating organization using the coordination tools of ethereum to begin exploring the frontier of biomedical research that bureaucracy and Middleman have previously prevented did you know that 66 percent of men and 50 of women will experience hair loss in their lifetimes hair Dow has already done some crazy cool science stuff like their first research study on the impact of synthetic T3 and T4 thyroid hormones on the human scalp and is also the second doubt ever to Mint an IP nft so if you want to explore the frontier of the decentralized science movement and maybe also You've Lost Your Hair along the way
check out hairdow at hairdow.xyz hey Joel what's up how you doing I'm doing great uh thank you for having me yeah really excited to have you since I read the paper that you all y'all just published you and Glenn Weil and Leon Erickson published a paper in December 2022 entitled Beyond collusion resistance leveraging social information for plural funding and voting so take us down the rabbit hole why did you write this paper excuse me sure yeah so I wrote this paper because I've been really interested in quadratic funding and quadratic voting and ever since I read uh Glenn's book radical markets a few years ago
um and I became to uh get really interested in the web through space and especially this idea of plurality and essentially with Glenn working with him last summer we realized that there was a kind of big well I shouldn't say we many people were involved but I think I was a small part of this push to kind of realize that there was a way to push quadratic funding in a kind of more pluralistic and more democratic Direction the other thing that I got really excited about is you know I'm in a computer science PhD program and I do a lot of work at the kind of intersection of economics and computer science and
one economic assumption that keeps popping up that I'm not sure I fully agree with in all situations is this uh conception of economic agents as these kind of you know almost like Hobbs Yen very selfish types of people right and so the other thing that was really awesome about this work on quadratic funding is that to really see uh how to make quadratic funding better you have to kind of get rid of that assumption and understand that in real life people have friends people collaborate people look out for each other when they make decisions and it's through understanding that that you can kind of build a better
quadratic funding mechanism okay amazing well um I just want to bring the audience up to a common level of understanding so let's do some Base building before we can get into what the paper is so um most of the audience probably knows this already but what is quadratic funding sure so quadratic funding is an algorithm that takes in donations to some kind of public good so this could be donations to build a park it could be donations for an open source software project of course it could be donations to fund uh you know health research that everybody can benefit from so this
mechanism is going to look at all these individual donations and then supplement them with some money from a matching pool and then that amount of money will go to the project that's the kind of uh nuts and bolts of quadratic funding and the nice thing about it uh compared to other ways of giving money to public projects is that under some assumptions it's essentially going to really nail the kind of optimal funding level right it's going to fund a project to the kind of exact level that society as a whole ought to be happy with right yeah I think the thing that get coins used in its Marketing in the past is that uh it's the mathematically optimal way to democratically fund
public goods but uh maybe adding the bit about the assumptions is would be would be a fair preamble to add could you enumerate what those assumptions are totally yeah so so first off I should say that in general for any of these public goods there's always going to be some limit to how much you should fund it this is an assumption that we make in the paper um but I think it's really true in general right if you're you know a community building a park you want to you know get the grass laid down you want to put in some benches right but as you keep putting more and more money into the park eventually it's like okay we've got you know 50 rows of flowers
and we're going to add another you know 50 like at some point it's just not worth the extra money right so I want to be clear first off that you can kind of undershoot the funding level and you can overshoot the funding level right what quadratic funding is going to do is it's going to assume that the world is made up of these kind of very standard um you know rational selfish economic agents right it's going to assume that they are kind of donating in line with their own individual rationality and individual selfishness right and then the great thing is that what the mechanism is
going to do with all of that donation information is is kind of say the following it's going to say okay you guys are all selfish and isolated and you don't know each other and you all donated these little amounts but because I have all this extra information I actually know that if you guys all knew each other super well and we're all collaborating then you wouldn't you wouldn't actually donate these tiny amounts you would actually donate much more and so the extra money that the mechanism gives is exactly that difference it's exactly what is needed to kind of fill up the funding amount to the level that all of these individual agents would give if they could all perfectly collaborate with each other
so this is really the main assumption asides from some kind of nitty-gritty assumptions about the way utility functions work which we really don't need to get into the kind of main one is the this uh this uh this kind of isolation of the economic agents right so I mean I I think as someone who's worked on quadratic funding during my time at get coin of course I'm disaffiliated from leadership I get coins I'm no longer there but I have some domain Knowledge from that I know that uh civil attacks which is basically uh one person pretending to be many people um because the mechanism optimizes for the preferences of the poor and the many if you can make up identities you can game the mechanism is one layer of attack and then collusion resistance
which is basically if these economic agents are colluding with each other uh whether socially or um by bribing each other to use their identity then those are sort of attack vectors in the mechanism and um I I don't know that I'm deep enough on the technical language here to know if what I'm saying is exactly the same as what you're saying but it feels like collusion resistance and measuring the social distance between uh these actors is one way to sort of mitigate the attacks and so when you when when you talk about if these agents could perfectly uh communicate with each other to me it feels like we're talking about collusion resistance am I am I close at all oh you're very close yeah you're exact you're actually kind of right on
the money and I think I'm just using a different kind of language to express what I mean so if it's all right let me tell you a little bit more about um about these assumptions that this mechanism is making and how uh this idea of a civil attack or collusion can kind of challenge that so uh but but I won't use those words I'm going to try to use uh different terminology Maybe so let's say again we've got a community deciding to build a park right everybody in the community is isolated and so they donate right the mechanism says all right I'll fill in the funding level to what you really ought to give all right so now what happens if actually we mentioned that this these people in the community
um they all know they partially know each other right some people are neighbors here and there they have some trust ties right so what's actually going to happen is when it comes to time to donate these folks aren't going to act like uh totally isolated people they're actually going to partially fill in some of that extra funding right that the mechanism is going to fill in the first place right they're going to kind of donate in this pro-social collaborative way and then when the mechanism goes to look at everything remember the mechanism is still thinking of everybody as selfish and isolated so what happens now is the mechanism is actually going to kind of overshoot the optimal funding level because there's some of that kind of
collaboration money quote unquote that was already in there from the donations that the mechanism didn't realize so how does this have to do with civil attacks or um collusion well I mean it's just a matter of kind of how you think about the moral reality of what's happening right in a sense a civil agent uh who is you know just to be super clear for all the listeners a civil agent is a soft puppet account that's just controlled by somebody else uh you know there's a certain sense in which that's not so different from say uh a couple who are married right so a symbolic agent and its owner are going to do the exact same
thing and we might expect you know a married couple or a very tight family unit or a very tight you know a pair of people in a friendship to actually behave in like a similar way right and so I think for me one of the great things about this paper was the opportunity to kind of try to frame um collusion resistance and civil resistance within this more General framework of just thinking about how people coordinate of course civil resistance and collusion resistance are incredibly important for a web 3 you know based application like get coin right of course you want to
have a layer of just getting rid of the symbols as much as you can sniffing out collusion as much as you can but I think on another level the kind of more fundamental problem is just that the quadratic funding mechanism as it is is unable to recognize when people are coordinating and unable to Discount against that right um so uh yeah yeah I feel like what you said right there is like the nut graph right there is like the mechanism can itself can not figure out if you're colluding and um therefore discount it and so um that sort of leads me into my next
question which is uh what is this paper that you wrote what is cluster mapping what is the augmentation to the mechanism that y'all proposed in this paper sure so the first thing that we had to do actually when we wrote the paper was really nailed down a solid definition of exactly what we meant by collusion resistance right so there's been a bunch of work on quadratic funding before a bunch of examples of certain situations where colluding actors or civil actors can totally blow up the mechanism but we wanted a kind of more General way of talking about it that could capture all of those cases right so the first thing that we did was to
actually really try to specify that right um the next step for us was to look at some of the previous papers that had been written so there was a quite popular paper called decentralized Society finding web3 soul that came out I suppose around this time last year or maybe you know sometime in the spring of last year um so the appendix of that paper had some suggestions for collusion resistant quadratic funding so our first step was to just analyze those mechanisms under the uh kind of definition that we had made for collusion resistance and it turned out that uh none of those mechanisms worked but our job after that was easy because it
actually turns out that you could kind of combine two of the mechanisms that were in the appendix of that paper and actually get something that does work um and so that also hopefully will make it a little bit easier to explain uh our mechanism if that's uh in on the cards for today yeah let's let's pull that thread I'd love to hear uh about this mechanism sure so like I just said I'll try to explain it by first um explaining the two kind of simpler ones and then maybe it'll be easy to see how they combine so in the first mechanism the idea was to just try to compute some kind of measure of social distance between two people right
um let's say you know if people are you know close friends and there's a bunch of transactions between them or if we're running quadratic funding in a company and we know they're in the same Department you know there are plenty of different signals that you could try to use in different scenarios to understand social distance um so it turns out that you can actually look at the funding amount that quadratic funding gives and kind of break it up into little chunks and each chunk actually corresponds to like a pair of people right and so what you can do if you have the social distance measure is say okay I'll get my quadratic funding kind of amount I'll break it up into the chunks and
then for each chunk I look at those two people the associated two people I see what their social distance is and if their social distance is really high I'm actually going to take that specific chunk of money and make it smaller right so I get the funding amount I break it into chunks for each chunk I look at the associated pair and if the pair is socially close I make it smaller and then I take all those chunks with the areas or the funding amounts that I've adjusted and I add it back up and get my total funding amount right so that's mechanism one yeah so basically sorry I was just going to summarize for the listeners so just like measuring the social distance um between people is like a great way of seeing if they're coordinated and you
know the example that I think Glenn used when he explained to this to me is that if we're going to run quadratic funding for uh Colorado then um if me and my wife and uh our friends in Boulder fund a public good in Boulder then you know maybe that's a different signal to the mechanism than if a public good in Denver is so good that we're funding it all the way from Boulder so I don't know social distance for me is this abstract concept but it's easier to ground it in Geographic distance for me in it makes it more intuitive what it means to the mechanism by thinking about it in that way absolutely yeah yeah so uh let's go on to the second mechanism sure so the second mechanism was based on a kind of similar idea
I would say that it the second mechanism is uh instead of trying to understand social distance between each individual pair of people it tries to understand uh the common groups that people are in right so pretty much what the mecca what this second mechanism is going to do is it's going to have some list of social groups and a list of who is in each group right then what it's going to do is essentially take individual donations and it'll look at an individual and say all right if you know you are in group a and group b or let's say if you're in
you know the Denver uh you know uh soccer league and you're also in the Denver um you know homeowners association or something like that I'm actually what I'm actually going to do is I'm going to kind of funnel half of your money to a kind of proxy account that I've set up for the homeowners association and have to a proxy account that I've set up for the soccer league right and I'll do that over all the people and of course other folks will also sometimes be in the soccer league or the homeowners association or plenty of other groups and so as I do this I'm going to have these proxy kind of accounts for each of the different social groups that exist in the world
they're going to get filled up with donations from the people in those groups so once I've done all that I have this funny situation where now I have a set of proxy accounts It's associated with like a cumulative donation and now the idea finally is that I run quadratic funding but treating the social groups as the entities that are donating instead of the agents right so in the end it's quote unquote like the Denver soccer league donated a hundred bucks if like there were a bunch of people in the soccer league that each donated some money and then part of those donations filled up that hundred bucks for the soccer league and the great thing about
that is that it's another way of trying to um kind of discount for pre-existing coordination in the sense that we ought to expect that everybody in the soccer league is more or less coordinated because they all go play soccer games together they talk about the shared projects that they might like right and so when it's time to donate they are going to be somewhat coordinated and this mechanism by funneling all of that money into one or two or three places is going to try to accommodate for that and understand that yeah so I what I think is elegant here is that you're sort of measuring the social distance between the participants and if the social distance is close
enough then you almost collapse them into one contributor so we're we're um in a way collapsing what collusion resistance and civil resistance is to make the mechanism just anti-fragile against both but because if people are close enough social distance wise to each other then they're just treated as the same person from the benefit of the mechanism It Feels Like An Elegant way to um not make you know not make the system completely civil inclusion resistance because you can still obvious skate your social distance from each other but um to make it much more taxing on an attacker to build up a complicated uh social distance obfuscation layer whenever they want to uh to collude
um I'm trying to translate this is into layman term layman's terms but not B2 reductionist did I come close at all oh absolutely yeah and I know that in reality civil resistance is far more complicated than this but to your point you could imagine that if I spin up a civil account and I give it some amount of you know ethereum to donate then this mechanism might say okay well they're these two agents and the only thing that the second agent has ever done is received some ethereum from this other guy so actually in a sense maybe they're in the exact same group right and so like you said they get collapsed into just one agent at the end of the day yeah
well that feels really elegant to me because you know so one of the principles that I hold dearly in crypto economic systems is that they have to be cheaper to defend than to attack right this is the whole thing that's so elegant to me about proof of stake is that I have to stake 32 eth and I make incremental gains if I do something if I if I secure the protocol but if I attack the protocol I can get slashed within one block I can lose half my so it's cheaper to defend the system than it is to attack it and I feel like what you're doing here with cluster mapping within quadratic funding is that you're making it much cheaper to defend because the mechanism itself is less fragile to civil inclusion attacks but you also make it cheaper to attack the mechanism because if I'm a civil attacker or a
collusion attacker not only do I have to make up all these new identities but now I have to have them contribute to other grants to make it look like they're socially distanced from each other which um now I'm funding public goods as the cost of doing my collusion attack and um you know there's just sort of like a tipping where it just becomes not worth it to even attack the the mechanism anymore and so uh I think that it's it's elegant in in that way and it's probably even more powerful when when it's paired with um a privacy preserving civil resistance identity layer like passport Bitcoin passport in order to uh to do that so uh I I think that it's elegant game theoretically absolutely thank you yeah you know as
somebody who just keeps my nose and you know math books for most of the time I'm always uh I always try to be very humble about trying to understand the real world impact that uh any of this stuff could have and obviously there's some Frontline kind of really basic Betty or I shouldn't say basic Betty because I'm sure it's quite complicated but there's some kind of Frontline civil defense stuff that should probably always be in play things like Bitcoin passport are always going to be awesome and I think that or my hope is that you know this mechanism could just be an extra kind of layer uh making the system work even better yeah absolutely so um there's a link in the show notes to the paper if anyone wants to go read the
paper that you authored with Glenn Weil and Leon Erickson on this so you can read the paper is there anywhere that I can actually play with this um this yet and are there any plans to build it out into a actual quadratic funding ecosystem soon sure yeah so actually first off before we get to that I would love to explain uh the mechanism that we kind of created because it's remember that it is like a combination of those other two right right let's go for it um sure so and this will only take five seconds and I can get to your other question um so essentially our only idea was to use this idea that the second mechanism
had of funneling everybody's donations into proxy accounts associated with groups but then to also leverage this idea of calculating social distance but now understanding social distance as something between groups rather than between agents so really just kind of took those two ideas put them together and it turns out that it works now if you want to play around with this stuff um there's a few places you can first off um I do have a uh a GitHub um project for it which is hopefully we can put in the show notes um you can also play around with it at radical exchange it is now online as the radical exchange plural voting tool so you can run it with some friends or run
it on your own and see uh what it'll do um and last of all I'm actually about to write a git Cohen Community proposal or rather submit a Bitcoin Community proposal um to see about uh implementing some of this stuff in the actual Bitcoin ecosystem or rather at least kind of exploring how it could be implemented um I've been talking to uh Leon Erickson from Bitcoin who is a collaborator on the paper and um Alex who was previously on the fraud detection and defense team as well as Kevin Olsen about putting this proposal together and um since all of the theoretical work is already done
um yeah I think there might be an opportunity to uh essentially just fine tune it and make sure that we can adapt the mechanism to get coins kind of unique needs and circumstances right you know like when you're doing this math you kind of have to be General because you don't know the situation the math is going to be used done right but in practice uh um there's a lot of fine-tuning and optimization that you can do and that's something else that I'm really excited to see about getting done yeah very cool so it sounds like this is a mechanism that if your proposal passes uh you'll be able to play with soon um on bitcoin and then it's also available on quadratic vote.redick
radexchange.org so to people for people to play with right now I think that's right yeah yeah um well I am really excited about this this technique you know um when I explain to people the magic of quadratic funding they um they fall into one to two one or two camps they're like oh wow that's really elegant and then the adversarial thinkers will fall into this Camp of wait what about civil attacks and I've always kind of felt like civil attacks inclusion attacks are the Achilles heel of quadratic funding back when I was at Bitcoin I spent a lot of time trying to come up with small brain identity mechanisms to uh to prevent civil attacks and now we have this one big brain mechanism that takes uh that takes in and makes it
anti-fragile against um uh civil inclusion attack so I think it's it's really cool and um I really can't praise you enough for publishing this paper but I gotta ask as someone who's uh written a quadratic funding matching formula how do you make this algorithm efficient to actually run because get coin grants uh was getting 500 000 contributions per quarter and when you have an anti-collusion mechanism that scales O N squared that's just algorithmic geek talk by the way for the non-technical listeners to mean that the amount of computational Cycles you have to spend scales uh exponentially uh based off of the number of of like data points that you have like is is there going to be a way to to build cluster mapping into something
that has Bitcoin level scale and um you know are we gonna use gpus or parallel Computing have you have you started to think a little bit about how you would implement this mechanism at scale yet sure uh I so I'm hoping that in collaboration with Folks at Bitcoin we configure something out um it is true that yeah if you have n people donating you need to do around N squared steps right to compute all the stuff and if n is Big N squared is even bigger right uh so I think that it depends the scalability will depend on the use case so I mean if you don't need to calculate the donation amount instantly I think that sure maybe it's okay to have a cluster of gpus
spending some time working on the problem and by the way of course you know at the end of the day N squared isn't like 2 to the N or some of these other you know functions that that mathematicians like myself get really scared of right right so I think that yeah like it will be a while before you know your smart watch can do quadratic funding in you know a millisecond but I am still hopeful that for processes where uh where a really quick result isn't isn't Paramount uh there will be a way to make it work yeah very cool well that was a niche question only for the five people in the audience that have ever implemented quadratic funding and had performance
issues with it but I figured we had to get it out there just because uh of my domain knowledge with uh with qf and I also think that um you know if we want to scale qf out to the world then finding an algorithmically efficiently way of doing it and having it still be collusion resistance will be uh will be really important so um well we will keep it uh we'll keep a an eye out for that gcp um and then the audience can go to quadraticode.radexchange.org and uh that seems like that's how they can play with with cluster mapping um is there anything I didn't ask about the paper that you want to tell us um I think we pretty much covered it
um I would just uh encourage everybody the next time they read an economics paper well not that people are going and doing that every day but the next time you hear people talk about economics maybe uh just remember that the assumptions that people make about selfishness and quote-unquote individual rationality there might not always uh they might not always hold up in every case and um at least for quadratic funding understanding exactly where and when they applied was really the key to kind of breaking through to a better mechanism so you know I would just encourage everybody to be a little bit uh uh cautious about some of the more dogmatic stuff that economists say
yeah really a great Point um I'm wondering if you can maybe maybe sing me a song about about pluralism I know that we talked about um creating more plural funding and voting and uh I think it'd be a really sweet place to end the episode talking about why pluralism what pluralism means to you and you know why you sort of fell in love with this with this value set so um why plural qf why why should I want more plural qf sure yeah uh I think that in general it's awesome when we can support things that diverse groups of people like that's really the long and short of it
to tell you about why I got into plurality in the first place um it's because I used to spend a lot of time volunteering with mutual Aid networks Mutual Aid networks for people who haven't heard or maybe don't know about them are essentially like you know plural organizations in Meet space right so they're decentralized they really prioritize collaboration across differences and I saw how incredible those types of mechanisms I guess now by mechanism I mean like real world you know activities uh could be so you know any kind of uh opportunity to
make the digital space more pluralistic I think is really awesome um at the end of the day the fact that a diverse group of people like a project it's really a signal about the potential utility that that project could bring to the world right it's not just some kind of uh you know oh let's all walk hand in hand you know down the Boulevard or something like that type of speak there really is like a a kind of an economic reason to want those types of projects to succeed um and so uh yeah I'm just happy to be a part of the story of bringing that that
type of stuff into the digital world right so this is all about making pluralism practical and and it feels like the value set is that there's an economic reason that if a diverse group of actors like a public good that in that public good must be good whereas if it this public good only caters to a monolithic block of people who who are closely clustered in Social distance then maybe it's closer to a private good or a common good or something than something that's broadly um appreciated by people across social distance would be my restating of of what you said there definitely yeah yeah um yeah it's it's interesting to see this meme of plurality come out of the radical exchange group and
um you know I got a sticker at I think it was Defcon Bogota I know a shelling Point Bogota and uh the sticker said infinite diversity and infinite combinations which I think is originally a Star Trek thing but uh I thought it was cool to see the meme back with rad exchange was it was the first video yeah so for our video uh audio listeners uh Joel's hanging up uh or showing a sticker that says infinite diversity and infinite combinations um yeah it's it's it's sort of neat to see radical exchange as an institution put forward this meme of plurality and now there's the plurality uh in action uh Network and then the the plurality conference and all that kind of stuff so um supporting the meme and the value set
of plurality with mechanisms that brings that plurality in in into practice it feels like it's becoming more of a movement um now and and and I just think that that's cool I agree I think it's awesome yeah well um is there anything I didn't ask that that you want to say just in general um go out and uh go and talk to people I think that's that's the message I have I uh I think for me actually the the the the really the biggest benefit of the kind of plurality mindset as a whole uh aside from you know the whatever proof you can write about the quadratic
funding amount and how that might maximize some welfare function or you know whatever that's all you know details the the on the whole the really great thing about plurality to me is that it fundamentally kind of encourages people to break through uh what I see is a lot of kind of alienation that capitalism and other factors of the modern world have more or less imposed on us right to me plurality is about saying hey if we create these really kind of uh strong Links of trust uh and we all have the ability to cooperate and communicate then the space of problems that we can solve
is going to drastically open up right um and so part of that could be uh you know participating in funding rounds for public goods but part of that kind of same plurality mindset could also just be trying to uh you know um to form new links with people in your real world communities right yeah and so look I'm a computer scientist I know how scary it is to talk to people right of course you know I'm you know don't worry I know I've been there but trust me you know and maybe you don't need me to tell you this but it can be really really rewarding to form those new links and I think if
that's happening in real life or in the digital world in the digital world either way it's awesome so yeah for sure well you know I I think it's a good argument to get away from the routine and to push your boundaries because if you can um commune with people who are unlike you then I think that you're kind of getting that cooperation across social distance not in a mechanism deeply on the internet but in your social life if you can introduce some entropy and just by talking to you to to people then um there's a lot to be gained there so I think it's wise words thank you yeah well uh this has been a fun episode I think cluster mapping is an incredibly powerful mechanism as someone who's written qf uh before I'm curious to see
how the next people who implement it make it scale but I'm also excited about just mechanisms that are easier to defend than to attack and so um thank you for your contribution to the public good of creating better public goods mechanisms and um and there's a link in the show notes for the listeners who want to read the paper so thanks so much for coming on Joel awesome this was a blast thank you so much for having me you bet peace and love [Music]