Government... By Lottery? (When Experts Can't Be Trusted!)

Transcript

and I think asking scientists to make the final decision alone it asks them to make a value judgment that is a really tricky position to put them in right and I know that sortition is only one example of a way to create an explicit division of labor but to me that is almost exactly the purpose is to get a bunch of people together whose interests are truly affected in different ways and have them learn from external information and then exclusively make value judgments [Music] welcome to web 3 with a16z I'm Robert hackit and today we're dusting off an ancient practice that has become trendy once again the old but new idea of

sortition or selecting Representatives by Lottery sortition was used in ancient Athenian democracy to elect public officials it's also been lately revived by tech companies like meta and AI startups like open Ai and anthropic to tackle some of their thorniest policymaking challenges our guests today are experts on sortition including Bailey Flanigan a post-doctoral fellow at Harvard who is joining MIT as an assistant professor next year and who has helped develop selection algorithms for sortition that are in use today also joining is Andy Hall Stanford University polyi Professor adviser to meta and consultant to a16z crypto

research in this episode we discuss why not not to rely exclusively on Expert Authority how the process of deliberation changes people's minds and how sortition can apply everywhere from the governance of countries to the governance of crypto projects and more as a reminder none of the following should be taken as tax business legal or investment advice please see a16z docomo for more important information including a link to a list of our investments so okay I want to start off and ask you Bailey what is

sortition yeah so sortition as a concept came about in ancient Athens as a way of selecting everyday people to participate in government and there was this randomized process that was kind of like a uniform Lottery although I think historically it was a little bit more complicated but in principle you can think of it as just a uniform Lottery over the population and you select some people via this method and they come together as like citizen representatives of everyday people and they suggest policy maybe they deliberate um weigh in on existing policies it depends on the application of all of the governance methods that are out there why would you want a bunch of randos to be deciding things for you literal randos yeah

literal actual R from a technical perspective they're randos I personally like it because because I think that it lends legitimacy to government when you can say that you know these are everyday people who have no political motives or incentives they're not trying to get reelected or anything and they don't even really have any political training or priors and they come into the situation and they learn from ideally a balanced set of experts and information I think people are really capable of making really good reasonable suggestions under really good conditions in a way that I think it's pretty hard for politicians to do in some cases because they're motivated by so many things other than just what people want um I also think maybe the

public in the end if it's well executed May trust the outcome more because they can it's just a transparent or like more transparent process I'd add a few things to that one is that if you don't randomly select people and you allow there to be this process by which some people choose to engage in government and others don't you could have a problem where the people who stand up and want to be in involved are precisely the people you don't want involved so like a very great saying about this comes from Douglas Adams one of his other books not The Hitchhiker guide I'm going to paraphrase it but he basically says the people who want to rule you are ipof facto unqualified to rule you by the very fact that they want to be in

charge they shouldn't be in charge this also goes back to ancient Greece I forget one of the philosophers I think Socrates maybe like he knows he's not wise and that makes him wise whatever it's like a paradoxical kind of thing Socrates I think a big part of his view is that you needed experts to rule and he has draws these extended analogies to like you wouldn't ask a random person to be a carpenter or to navigate a ship he has a very famous analogy to the captain of a ship but others thought this could be a good part of the system for these reasons that you don't want to allow only the people who want to do it to manipulate everyone else and I'll just add it doesn't need to be and all be like you can use it for some things and

not others and I think most times in places where it's been proposed including in ancient Greece the idea was not like all decisions all the time would be made through this mechanism it was a way to select certain offices for certain decisions particularly ones where you don't think it requires the specialized expertise of full-time government officials yeah and just I mean add to that too there's just a lot of people in the standard representative electoral government representative I'll put that in quotes because there's just a lot of people who will never have access to the credentials or the money required to gain influence and that to me is a pretty significant problem and this is one way to at least allow a much less filtered group of people to weigh

in on policy there are different kinds of you know organizations you could be running or decisions you could be making I imagine back in ancient gree like you wouldn't want random people designing and constructing like trre warships like that probably was a specialized task that you needed somebody to know how to like you know build a hole or whatever what are the kinds of things that this is good for here's like I guess the framework I've been thinking about now I want to say outright I'm not sure it's good for anything like we don't know fighting words uh but other options don't seem all that good either so so I think we don't really know is the first part of the answer they're really not used a lot

there's no precedent for using them for like a high stakes decision that Society is very concerned about where if you decide it one way or the other major things would change you can't really point to any example yet where that's happened and personally I doubt we'll ever get there but although on a personal level like probably place where people would experience this most is in jury duty is that right yeah yeah yeah that's fair is consequential for people's lives absolutely absolutely that you know governing National policy yes yeah I think we could think of jury duty as fitting into this model they're making a oneoff decision on a single piece of adjudication versus setting a policy that's going to affect the whole

country there's a wide Gap there so in terms of applications I mean what people usually point to I think the most natural space is sort of like value laad in judgments that are more about resolving fundamentally irreconcilable trade-offs that we can only appeal to values to decide on like where we should set the line on abortion policy or something science is not going to tell us what to do there it's a values decision you don't need a lot of expertise to decide that you need deep understanding of what Society feels about this issue that's the first element I think and if you look everything I just said is often used to

justify referendums or ballot initiatives some kind of pressure valve Outlet where you go around elected officials and let the voters speak directly we have a lot of reasons to think this don't always work very well a big reason why we think they don't work very well is because of low rates of participation and low rates of attention sortition is then a tool where you undo the low participation you're worried not very many people are going to vote on this referendum and the people who do are going to have very weird unusual views because these are the people who are motivated to actually and then a very logical thing you might say is instead of a referendum let's randomly sample a group of Voters we know as representative get them all to think

about this thing and then make a decision together and that might be better than a referendum and in particular one part we haven't talked about yet is it might be especially good if you think that at Baseline people don't have well-formed preferences or views on this issue and so when you just have a vote on it they don't vote in ways they're actually consistent with their underlying values they have to sit in a room and talk about it and learn before they can actually figure out what they think and that's like you have to deliberate before you can form views and so you can see it as a mechanism to just replace referendums did the founders like the American Founders the founding fathers did they have different ideas about representative democracy and the way to achieve it did they consider

sortition or government by Lottery at all for any purposes or was that just too mad my impression is that sentiments were more toward we need experts to govern yeah people really through you know the classical era up to the founding of America people looked back on Athenian democracy not as this unalloyed success that everyone should model the whole view was this thing was a show there's a really cool idea underlying it that we want to bring back and it enshrined some values that we like but it did not work at all and it led Athens to make truly catastrophic decisions particularly this decision to invade Sicily and they looked instead

much more to Rome which had built this more hybrid system that had a a lot of mechanisms we would recognize today as suppressing the views of many people and being undemocratic but that also tried to make the system function much more effectively and it did for a much longer period of time Rome had its own set of issues though out of more than a few uh but when you look back at that history and you look at the founding like the debates over how to design it it was much more like we are not building some wackadoodle utopian society where everyone decides on everything we're like building a system that's going to work and it was not motivated by things like sortition that would have been a a

small small part of the discussion so why use sortition today why are we dusting off this old concept and trying to use it I'll use that question as an opportunity to kind of go back to what kinds of questions is sortition good for the example you gave about building a warship that's like an example of something where you pretty much need to be guided exclusively by the scientific method and then there's questions that are like purely value judgments like you've got two people's welfare and you just have to choose how to trade them off you can't satisfy everyone and so you need to make a really difficult trade-off but there's also like a lot of issues that kind of fall in the middle of those like they have both components regulating AI algorith Ms have technical

underpinnings but they also have human implications and trade-offs how do you tease those apart to allow a group of everyday people who maybe doesn't understand a lot of the technical aspects to still weigh in on the value judgments and then use those value judgments to inform the technical components and maybe vice versa as well I think there's a huge design space actually here in designing the deliberative process to allow the outsourc ing of the scientific method but still distill those value questions I'm actually working on a project on this right now and it's super difficult and I think a lot of people have tried to do it but they haven't necessarily thought about the the formal process of

distilling these two components and truly separating them because you don't want people to be implicitly guessing at the scientific method and you also have to deal with the fact that the scientific method leaves uncertainty and so then you have to ask the value question of as this assembly who do you want to Bear the risk of the uncertainty that the scientific method has left so you've got risks you've got values and you've got science and these are things are like very different and they you have to think about them in a really different ways so anyway this is something I'm thinking a lot about actually and I think hopefully one thing that will come out of it is an understanding of which issue should this be applied to and how do you apply it you said you're working

on project right now related to this in yeah in its infancy I mean I think one application called environmental justice metrics it's basically how you measure environmental risk of different communities based on like a lot of factors and people tend to kind of aggregate these factors into scores but they do so in a way that I think hasn't really been weighed in on by people it's usually kind of like a combination of maybe causal inference but um there are opportunities here for the scientific method there's certainly uncertainty and there's also value judgments about which factors are most important so that's one example maybe of the kind of like first application that would be interesting so I mean for this example

where you're talking about tradeoffs and getting people's opinions and stuff is it like maybe some people care less that they have a chemical plant in their backyard or something like is that the kind of input that you'd be getting out of this it could be I mean I think you're alluding to something like lived experience which is when you are the person who will bear the cost or bear the benefit of like whatever decision gets made here what does your lived experience say about what you think is best for you it would be an example yeah please ask I 100% agree with this and I think it's something Society is doing a very bad job on right now there's a huge separation between what we can use science to decide and what we need to decide via these value tradeoffs and just to jump to the most controversial

possible topic because that's what I like to do that's like co co policy had this problem where there's this attitude like we're just going to do whatever the science tells us to do science is doing a lot of work in thatly idiotic way to choose policy like the policies certainly need to be informed by science we need to know for example you know what are going to be the implications for societal Health if we close schools versus we open them that's a essential input to the decision but once we have the inputs from science the actual policy decision is a value tradeoff informed by science about how much we want to prioritize the health of different groups the education of our children Etc the idea that that's a

decision that could be made by science is an extraordinarily misguided and stupid belief that a large number of American Elites hold today that is completely crazy and I think Sardis is just a helpful example to show you why the two things need to be separated and why having a group of people not weigh in on the science but informed by the science weigh in on these difficult value tradeoffs is precisely the right kind of division of labor we would like to set up so instead of taking it on Fiat from some Elites that are saying like Okay your schools have to be shut down instead maybe like pulling the people who would be affected by this the kids the parents the administrators ask them what they want to do maybe they

want to keep the school open maybe there are good reasons for that exactly yeah and I think asking scientists to make the final decision alone it asks them to make a value judgment exactly and that is a really tricky position to put them in right because that totally it's the expectation and people I think they don't quite realize that there is really these two different steps as you very clearly laid out and I know that sortition is only one example of a way to tease those apart and really created an explicit division of labor but to me that is almost exactly the purpose of sortition is to get a bunch of people together whose interests are truly affected in different ways by the decision and have them learn from

external information to the greatest degree it exists and then exclusively make value judgments so how do you make sure that you get a representative group of people okay well it's really really really hard in the practice of sortition as it exists today the participation rate that I often hear quoted is about 2 to 5 % of people will respond to an invitation to participate in like a Citizens assembly that's quite low is the Outreach usually by email or something it depends sometimes it's by phone sometimes it's by letter usually like a uniformly random sample of the population that you get from speaking personally like I basically don't click links and emails and if I don't know the

phone number that's calling me I'm not answering it so yeah I can imagine why the rates are so low right and I mean it's possible that that will be somewhat addressed as Citizens if they get more institutionalized or publicly known maybe that will get better because you won't be like what is this why am I getting this random email thr away but the other problem is that the people who do say yes kind of seem to be very self- selected I would call it which is very very skewed on many demographic and belief related attributes this seems very related to the like weird issue in social sciences the Western educated industrialized Rich Democratic stuff like all these social psychology studies feature a particular sampling of the

population that's like college educated students who are willing to get paid some money to participate in a study and so it just skews the the research that it comes out of it yeah and so there's been a lot of work done recently and I think there's a lot more to do to try to say okay if you're given a pretty self- selected group what is the best way you can randomly choose the final assembly in a way that kind of restores some representation on observable characteristics but the unobservable characteristics are just such an enormous problem and you can't really do much about that until we understand what the filtration looks like from the population to those who actually agree to participate in the first place so yeah it's pretty thorny problem well I hadn't thought about this before but I

guess his putting together Robert's point from before with yours like one solution which would not work when it's a one-off project by a organization jury duty is mandatory and people do respond to those inv ations and the participation rate I've somehow dodged it uh to date I've never to serve on a jury and I never get on it I would like to mad I always get kicked off I've never even gotten in the room I've never even gotten the room to get kicked off you know I get the letter I check online and then I'm always told like you don't even have to come in we don't need so frustrating same here but anyways if you really in Shrine the system within a broader governmental system then you could get people to do it because you

could require require that they do it I do think that it's true that it would probably be a lot better but I do think there's still like self- selection in who goes to jury duty right because there's a lot of exemption criteria that you can meet and I was just actually reading about this because I was trying to understand whether mandatory jury duty is a good case study for kind of what would happen to self- selection bias if you mandated participation in sortition and there are still issues related to certain groups basically not appearing on juries like pretty much ever or just extremely low rates and that there's actually concern that that affects the quality of the decisions particularly about certain groups so it's not a perfect system I still think you need to think about how do you really ensure representation when

you select the final assembly even if it were mandatory but I do think it would go a long way yeah there are countries you know that have compulsory voting which is not something I based on my philosophical principles I am uncomfortable with but it works surprisingly well like conscientious objector to voting as you know Robert we talked about that at dinner last night but it works remark I I say this grudgingly as someone who doesn't like the idea of the government forcing me to do something that I might not want to do um but when you look at if you voted you could you know elect somebody who's going to not force you to maybe do that to undo Undo You Got have your voice represent if you look there's really great research on Australia and Brazil and and there are a few other

countries that have these laws and people in the countries tend to like it compliance rates are quite high if you really want to get out of it you can but vast vast vast majority of people do V countries that enforce mandatory voting yeah yeah so it's it's a pretty it's interesting yeah so it actually goes over pretty well but you just are not a fan there's something about America and the American government telling me I have to vote that would bother me yeah bother me quite deeply do you think that people who are required to vote really think about the decision what do you mean well I mean if if I'm like forced to vote on every issue that comes up I mean am I really going to do the research on every issue and think about what I really believe like I worry that

people will just respond maybe the turnout is really good but like people maybe they aren't thinking about the decision that much I don't know what do you think I had a friend who uh during standardized tests in high school would just do Guitar Hero in the multiple choice like you know he'd like fill out AC like for one column and then like whatever was whatever song he was playing at that time so yeah I think you could get some un voting if you tried to do that yeah I'm wondering how widespread you think that is it's definitely concerned a few things I would say one the compulsory voting is typically the way it's implemented it's only for these you know High salience national elections that don't occur very frequently so they're not being asked to shoulder that much of a burden it's in

the context of vigorous campaigning and while you are required to vote again you can opt out in Australia you can pay a relatively small fine most people choose not to I think the turnout rate's around 97% or something you're completely welcome to just cast a blank balot and so if you really feel like you don't know you don't have to put something that you don't and so I think that actually helps because Mo there aren't that many people who are going to like straight up troll if they're really uncomfortable and they feel like they don't know very much they can leave it blank the reality is in the context of a vigorous national campaign it's not that hard to like figure out which party you think you should vote for and so most people do vote Australia also has rank Choice voting but that's a whole other

rabbit hole we probably shouldn't go down next podcast yeah yeah yeah um random tangential question we refer to secret ballots as Australian voting isn't that like a thing yeah do you know the history there there are people who know better than me but basically there was this key time in the 19th late 19th century where a bunch of democratic countries you know Australia the UK the us became quite concerned we would today we would call this clientalism and you recognize it today in in many Democratic countries still where voting is organized around networks of patronage and there was the sense that it was verifiable who you voted for and

that then facilitated systematic vote buying where a party could observe you voting for them and then reward you for doing so in small Ways by like buying you a beer at the polling place or in big big Ways by giving you jobs when they were in office not everyone considered this a bad thing and in today not everyone considered a bad thing like there are context in which that might be the way you think politics should be organized but the sort of different use of the word Progressive than we would use it today these like Progressive ideals were growing that sort of like we need a more scientific and business-like approach to politics it was the era where we were building these stories about everyone gets informed and then we make the best decision and it's not

about things like getting jobs in the administration and stuff like that and the Australian ballot was implemented in Australia as I recall I think the main motivation was to shortcircuit this process of vote buying and the US implemented it there's some evidence that it reduced the power of these local parties because they weren't able to reward people for voting for them anymore I would guess too that it would lower participation rates but that's just a yes that's a major claim that's made yeah participation rates in the US first of all are generally higher than people say I always start by saying that because it really bothers me like I tell them participation in Dallas is low and they're like well it's low in the US too and I'm like it's actually pretty high

in the US it's like 70% um is it that high it's high that's actually surprising yeah I I find it maddening when people claim it's low I understand why they say it and you can certainly point to elections where it's lower than 70% but our presidential turnout rates are pretty darn high in the 19th century they were super high and a retrospective claim on one claim on that is that it's about it was an era in which highly organized very professionalized partisan organizations were systematically getting people to vote in a ways that we don't see today there's a lot of other factors so we're not like confident that's the explanation but it seems like it's def part of it okay that was a that was a tangent something we didn't talk

about which I know Bailey has thought a lot about and been involved in is sort of like using this in Tech and I think the appeal to using it there is that we don't have access to these other mechanisms I was involved in an effort to design one for meta which is now being used for AI related issues and the motivation we had was this is a very valuating decision the first one that meta ran was on how should you Define climate misinformation as I said I always like to go straight to the most controversial possible topics and and the view was this is a great one because it's super important it's super valuating it's super controversial and then unlike for government a tech company just doesn't

seem like the right entity to make a decision on something like that and so you have this real need to go out and perhaps get your users the people living in your online community to tell you what they think the rule should be and then if you accept that premise that that's how we should do this and that's better than having a CEO decide this then you have to start asking these practical questions questions about like well how would you get the users to decide and you could ask the users to vote that's like the first idea people had like Mark Zuckerberg ran a referendum on Facebook in the late uh SS uh 2008 or so and it just didn't work because you couldn't get enough people to vote and so now thinking about this you know Circa 2020 we were like okay

you want to give this decision out to the people the users you can't ask them to vote The Logical conclusion is should use sortition and so I think it's a cool area for this because with a government you can construct Arguments for why your elected officials and ballot initiatives and referendums and jury duty are already kind of occupying the space I'm a little bit more of a skeptic on like where the room is for sortition there maybe there is some but for a tech company making a decision it seems like it's the obvious right way to do it because they don't have access to these other options another question in the tech company that I think would be super interesting from a research perspective you know I'm from computer science a lot

of people there think a lot about recommendation algorithms and what they're doing and what are their impacts and I tried to work on this once and where I got stuck was the question of what constitutes harm that's a very subjective question right because you know some people would say that facilitating a riot if it aligns with their political beliefs for example is not harm but other people would say that that is and so if you want to like measure and study recommendation algorithms you kind of need a determined notion of what constitutes a harm and a benefit and that would be another example of something that I actually think people should weigh in on because a lot of the harms and benefits that people think about are inherently political and very personal and reflect

tradeoffs in people's well-being meta thought a lot about this with respect to the oversight board which is you need the science in this case to tell you if you rank content this way versus that way what actually happens because it turns out our intuitions about that are completely wrong maybe just like the age-old thing like everyone thinks that ranking content chronologically is the solution to all their problems and then anytime you actually give people chronologically ranked feeds they hate them and so it's like our intuitions about what to do with ranking are completely useless you have to study it and then given those facts you're then making highly valuating decisions and I really like the example you use on riots it's a perfect one where people form

these views that there's just a uniform principle you can apply and it just doesn't work because and this happened after 2016 it became accepted by a large body of Elites in America that the thing you needed to do was like anytime quote unquote violence was called for on social media you must remove it exactly and the problem is there's lots of context in the world where the government is horrible and it's not good to say I'm sorry you're not allowed to talk about your government being terrible you're not allowed to coordinate to meet somewhere to protest your government that's dangerous that's obviously not what people in America had in mind when they were saying we should remove this kind of content but it was a huge unforeseen though obvious to some

of us problem that came out of that and so it's again there are no easy answers there are really hard trade-offs to make how much do you want to prioritize protecting the status quo versus encouraging disruption which might be dangerous that's an unbelievably difficult trade-off and not an obvious one to resolve yeah this is exactly the conversation that we had where we got stuck and kind of there are there are good riots and there are bad riots and everybody who is in the riot thinks their Riot was a good riot in is perfectly moral Justified precisely there's other harms too I mean there's of course a whole Free Speech violent speech thing that's going on and I think there's real arguments on both sides I

am always surprised when you look at these kinds of questions about speech and content and all that and you just look at the survey attitudes of Americans it is remarkable how wise and reasonable and moderate their views are relative to the right-wing and leftwing Elite conversation about these issues and I think had the average American had more say in the 2016 to 2024 period over the conversation about social media content moderation we would be in a far better place than we are that's my totally agree very biased opinion my survey of my Uber drivers reflects that the average person is extremely reasonable like that's like my is this a formal survey or an informal Sur

absolutely not it is under representative um the other thing that I I'm kind of interested in potentially using assemblies for is government accountability but I'm not exactly sure what the right application of this would be but I think there's a lot of cases where there's institutions in the government that you kind of want to audit what are you doing how are you doing it like is is this really in line with the values that we think our government should have and I don't think that there's a lot of accountability in some of those cases and so I think it would be cool to see assemblies there maybe ranging from like involved with the FDA to like actually involved with governmental committees just because I think there's a lot of distrust of these

institutions among people and part of the reason is because you can't really see what's going on inside I love that idea that's fascinating yeah randomized audits yeah exactly because I think people are pretty good at looking at something and being kind of like it doesn't pass the sniff test like you know what I mean yeah for sure for sure I've got a question about the citizens assemblies do you just want to get the snapshot view of all these random people from hopefully disperate backgrounds or is there like an education process too maybe like you give people more information and they change their viws is that measured at what point does it become like re-education and you're sort of programming people's minds differently through the process I don't

know my personal view is that there needs to be a lot more research on what kinds of opinion change is caused by deliberation because right now there isn't really even that many repeatable protocols for deliberation like they're all organized by different people they're done in different ways there's a lot of anecdotal evidence there's some empirical evidence I mean I think Jim fishkin has written at least one paper about opinion ch that resulted from a deliberative poll which maybe you know more about there but uh I mean I think of like the famous I think it was a play first the book 12 Angry Men where you know yes I love that movie yeah right like you know 12 people go in 11 people think a certain way and by the end they all have swung in the other direction well this raises a different issue about whether like one

person can be way too persuasive but in I mean in that case it was a good thing but it may not always be but I think there's like a lot of ideas also about not just the information changing people's opin iions but the idea of kind of developing like a collective identity the idea that when you communicate with people who are different from you from all walks of life in your Society you develop this collectivist attitude when you start to care about the interest of your entire Society because you kind of identify as a part of it I do think there needs to be more research on that as well I have attempted some but it's pretty difficult to measure it's interesting so I have like just a couple things on this so one The Meta Community forums had had embedded in them some extremely interesting data that you can

view it's publicly available and what they did was basically survey the participants beforehand on a bunch of stuff like their view on the particular issue being discussed as well as their views on other things as well as like their views on whether this thing was going to be crazy or whether it was going to work or not whether they were worried about getting in arguments with people during it and so forth and then they measured a bunch of that stuff again afterwards and they have in their report these really interesting graphs and tables of how people's views changed before to after I think some of the learnings that I took out of it at least were that people's opinions changed quite a bit in the US part of it was pretty bipartisan like people on both

sides were learning when people showed up and were in this social situation they were much less adversarial much more open-minded than I think I would have expected and they came out saying they liked the process itself too that's pretty compelling to me you can go look at the data for yourself I just want to raise there is a broader downside to this process which I think is why it would never ever make sense to like use it for everything which no one's advocating for to be clear but like uh which is that we think a huge part of running a government and making Collective decisions for society is the accumulation of expertise and knowledge an experience over time and that can be

a bad thing certainly in America you can conjure up these I think reasonable concerns that politicians become insulated they become quote unquote career politicians and so forth but it's also undeniably essential for getting stuff done figuring out what needs to be done who wants what and these citizens assemblies are a very bad tool for that because they meet once and then the people disappear they don't gain that experience you're sacrificing that in change for something else of value which is this representativeness and then two like elected politicians they want to win re-election this is like the formal definition of accountability in political science is like they're now worried about winning reelection and that's the engine that drives them to go

figure out what needs to be done the people participating in a Citizens assembly do not have that incentive they're already there they've already gotten recruited or whatever other than their own sense of civic duty or the social norms of being in this room which I think are quite effective there is no long run I want to do the best job I possibly can so I can win and there may be lots of settings where that's actually quite problematic where the fact that in some sense they have no skin in the game it might be great when they just need to weigh in on an issue of value but if it's a super difficult issue where you have to go way into the weeds and listen to briefings from a bunch of boring scientists and stuff the fact that there's really nothing on the

line for you is a huge concern the ideal situation in my mind is you have this group of people and they have this wonderful property that they are not politically motivated yeah and it also has a significant downside but suppose you could just put a bunch of scientists in the room and some policy makers in the room and the people can say here this is what we want these are the values that we have you know what the form of the output is I think is really important and if you could design the process so that their output could then directly say okay here's what we need to know from science in order to inform what we would recommend and then you talk to the policy maker and they say you know what this is not going to get passed for these kind of complicated

reasons what do you think about this it sounds like a messy process but I think you can think of it in a pretty organized way of kind of querying experts and getting feedback in computer science it's kind of like online learning if you want to think about it that way I just think that there might be a way to get everything you want that's interesting should there be incentives for people to participate to help motivate people to make good decisions as well one of these like eye sortition stick yeah something like that maybe uh they'd have to pick a different word other people knew what it meant be awesome I mean I think you're getting at like a broader question of public trust in these processes which

maybe sounds unrelated but I think something that I've also been thinking a lot about lately is what information needs to be be provided to the public so that even if they disagree with the outcome they still trust that the process was done in a way that represents their entire society and they would be either compliant if that's relevant with the chosen policy or maybe they'll support it in a referendum even if it wouldn't have been their first choice especially if it's put to referendum after the assembly which sometimes happens that's happened in Canada it happened in Ireland and when you think about what you need to release to the public you have to start to ask yourself what do you need to ask the participants of the process to report and that's starting to get like is

sortition and kind of reflecting on maybe like what was your experience in the process did you listen to other people did other people listen to you was it truly deliberative as how a political scientists would ask that question and so I guess like injecting mechanisms of accountability and public transparency into the process might build in some of those incentives because what happened in the process will to some extent be public of course preserving anonymity right I was gonna say yeah the challenging trade-off and Bailey now we're talking about this yesterday is there's also downsides to identifying the participants a definite concern is that if you were to empower this body on a highly controversial and high stakes issue then all kinds of

powerful actors would have incentives to threaten cajo bribe the participants and then having their identities public would be potentially bad yet at the same time letting them be anonymous leads to doubts about whether the assembly was really even run at all was it run fairly as well as then you lose this social value to the participant of getting to talk about it after I mean jury duty is these are exactly the trade-offs yeah this is like when you're trying to maybe charge a mob boss or something and the people in the jury are afraid for their safety or something yes and unfortunately that recently there's been quite a few of these types of issues in the US which

has been pretty worrying but I'd like to pause at this moment to put in a plug for the TV show jurry duty if you uh seen it oh my God seen this no I'm not oh my God the premise of the show it's absolutely incredible is it better than Judge Judy no well that's a hard I can answer that no the premise of the show the show is it's essentially a reality TV show I'm not spoiling anything by telling you this in which a real member of the American public was recruited for jury duty and the only thing that he wasn't told was that the entire thing is fake and all the other people are actors this is just Truman showed him they go and they go through they go through a whole trial poor guy they're very

careful about the ethics of it they did a fantastic job and the trial just keeps getting increasingly absurd this this is going to destroy people's trust in sortition you could all be one giant catfishing sche it's a concern it's a concern but you have to watch the show it's amazing it's amazing speaking of TV when we were all at dinner last night I'm not sure if you noticed but Benedict cumber bash was at the table next to us are you kidding me do have you confirmed that as definitely him yeah 100% how did you confirm I looked at him again with my eyes fair enough I trust you deduction the power of deduction Watson it's very exciting I'm so mad that that did not make it my into the table unreal

yeah D anyway you never know you might serve on a jury with Benedict how you know you're getting punked yeah I want to take things back to the tech sector with these new technologies that are coming online between Ai and web 3 how might a governance system like this apply what experiments are being run with sortition with respect to these there are a bunch of experiments as I mentioned and I like to point out that it was the first one meta ran a community Forum on having to do with AI guard rails and it was very much a pilot so like the company received in public input from the process but the assembled people did not have binding authority to make any

decisions so it's in that sense a very much a pilot though that's true of most citizens assemblies in the real world too and I'm very pleased to observe that other AI companies have learned from that and so both open Ai and anthropic have used whether it's sortition or not is not exactly clear clearly inspired by meta's Community Forum on AI and the way they work is different from one another so in open ai's case they framed it as sort of getting Democratic inputs again for guard rails and it was a little bit I would say closer to the community form in the sense that it was about recruiting a representative group of I think it was just Americans and that is is pretty interesting because it was across a number of countries which required solving thorny linguistical

issues and stuff which was pretty cool and so the opening was similar in that sense but it was a little bit closer to like uh there's like a fine line that you can't really actually tell the difference of sometimes between like sortition which sounds fancy and like a focus group which sounds less fancy the anthropic one looks quite different in the sense that it was sort of like integrated into their technology in a kind of fascinating way where essentially this group of Representative people again I think theirs was only Americans was asked to help brainstorm a list of principles a very long list like 50 principles so one would be like make sure when you answer questions keep in mind that some

people might have different abilities and that you shouldn't answer assuming that a person you know can walk everywhere or something like that it's a list of like 50 of those and then those are actually fed into the system prompt essentially they're just like copy pasted into the like system prompt for Claud and like the metan it's a very early pilot it's not actually that amazing to like end up adding a few sentences to into system prompt in the same way that uh it's not that amazing to have a focus group make some recommendations and then the company can do what they want but like these are early experiments and it's cool to see them they're sharing some properties in terms of randomly recruiting represented

people and then they're taking the outputs and inputting them into the systems in different ways I think something to add here is one huge challenge that we just have not solved is that deliberation takes place and plain text and then the output is currently plain text but in the end even if the algorithm takes in plain text like Claude when we go to actually measure if that intervention worked what you're going to get is a bunch of metrics you have to distill things into numbers and functions and statistics and as soon as you have to do that you start to have to make tradeoffs that were not even considered by the people in the process and so the question is how do you get the process to produce an output that is so close to the algorithm that

like there's just not much Gap that has to be filled in by technical people because any gap between where the assembly leaves off and the actual metrics that are going to be used to evaluate the system will have to be filled in undemocratically I think this is a huge challenge when we have people weighing in on Technical Systems yeah this is a huge and is an ongoing area of interesting Evolution with the oversight board so the oversight board is not a sortition mechanism but it shares a lot of these features that it's a group of people they actually have binding power over meta which I think makes them especially interesting relative to these Pilots so they have the illegal authority to overrule the company on pieces of content and so they might

order a piece of content to be removed and meta is legally obligated to do that but then there's these insanely complicated technical issues about like well what about this other thousands of pieces of content that are a lot like this one but a little bit different it's the same video but it's been tweaked it's a similar image but it's a little bit different and it's been a big area of push and pull with the board is sort of like do we have access to the technical knowhow and data that we need to be able to judge whether the company has complied with our order or not I think we're learning a lot from that give and take that will teach us for AI and for these citizens assemblies how you might go about doing that so one

idea that's come out of that is like the oversight board should have access to its own team of Engineers and data scientists who can actually check and the thing is they'll work for the oversight board not for the company and so those people will not have economic incentives to like say whatever the company wants them to say is the claim so you could imagine replicating some of those features for a Citizens assembly in AI or on any topic I mean having those experts in the room to kind of go back and forth and say okay this is how I'm translating your words into the technical inputs here's the output that happens when you do that are you satisfied with this or what's wrong with it and kind of iterate in that process I think that would be useful for many many

different topics not just AI definitely the devil is always in the details I mean even if you look at our political process everybody generally wants the same thing right like Prosperity fairness Liberty but Republicans and Democrats have a different way of going about that it's all in the implementation details there's a huge debate that I don't think anyone has brought in I don't know how you would study it it needs to be studied but there is a very deep question in online governance which applies to AI to social media to everything else you know Airbnb Spotify PayPal any platform that's governing itself which is if you're making decisions that are too consequential to Society for you as a company to even want to make yourself

because it's actually in your interest to get out of the Trap of having to make these really difficult decisions is it more important with the goal of getting Society to trust that this process is fair is it more important to give out the power to set the policies which is pretty much how most citizens assemblies are headed or is it actually more important to give out the power to enforce to decide whether the policy has been enforced correctly or not and if you talk to different people in Tech who were in the people deep in the weeds you will find people with super strong views on both sides so just to take meta as an example it's writing meta's Community standards and content policies that's where the power lies

that's where you decide what counts as what and then other people equally Deep In The Weeds will be like no it's all about deciding how are you're going to enforce it at scale the rules are not the key thing it's the enforcement at scale and it's a huge deep question which of those is more important to give out and I actually don't think I haven't seen any academic work really show us the path and like how would you even go about figuring out which of those is more important and it may be different for every platform and every problem H my sense personally is that you maybe want to give both because part of specifying the rules that you think should be followed is actually making sure that they're followed in a way that reflects

your values that you are trying to communicate usually in text I mean what is the point of specifying what you want if you don't then get to hold the company accountable for enforcing it I think the challenge is that people will say writing these policies requires insanely deep knowledge and expertise and you think they're easy to write and they're unbelievably hard to write which is clearly true same way like if you don't understand these ml systems you can't possibly develop enforcement at scale in a way that's going to make any sense and that so it's those information asymmetries are I think what what dogs the the disc discussion of what to to give out yeah I think my optimistic view

is that we just need to find new ways to work together which is super hard right I mean talking to someone as a really technical person who is in the weeds trying to communicate with someone who's non-technical I mean this happens in all spheres like working with practitioners as computer scientist you're bringing such different things to the table and part of that is learning to Value the contributions of the other group really seriously even if they're different from your own and they don't speak your language and you know maybe sometimes it's harder you have to do like more work and more math but it's just so fruitful and I think it's just an investment to learn to communicate across and maybe we haven't really figured out how to do that yet because the incentives haven't existed but I don't think it's impossible no I don't

think it's impossible either it's just hard to figure out where to start if you're running like a trillion dollar online platform it's a little bit risky to like give these things out if there's some risk that uh it's going to go catastrophically wrong because people don't understand how the systems work that's the problem but maybe it's possible to nail the process in a low stakes test case right I mean because the process is not entirely separate but somewhat separate from the stakes of the decision yeah I totally agree is this an opportunity to run experiments in web 3 I mean surely there are lots of platforms that are securing lots and lots of value but um they're also you know a million experiments blossoming that are doing all sorts of crazy things and just see what works I think that's totally right no I think and I think

we're already learning a ton from those I think there's a whole deep conversation we should have with bayy on how you would Implement sortition for web 3 because it seems very very interesting and not obvious how you would like the cibil problem which means that you can't sample people all that easily obviously one area which people are already working on is like creating Notions of identity that get you around the Civil problem at which point we could just apply normal sortition tools perhaps the other question is like could you design Cil resistant sortition in a way that would be useful I would be really interested in trying to understand are there mechanisms that allow you to at least learn something

about who is who even if you don't have a perfect understanding of you know this wallet corresponds to this person or something maybe you would have a little bit more information and what are the tradeoffs between getting that information and what you can do with it later because I think 's a lot of benefits to representation and power distribution if you know things about people but in order to elicit that information in incentive compatible way you may need to actually distort power in ways that you don't want to say more what do you mean by distort power if you want to incentivize people to reveal information about themselves people in this system are represented just by coins or wallets or these things that are not really tied to an entity and there's this problem where you don't

want people to be incentivized to spread out all of their assets across many different wallets and pretend to be many people because then it makes them entitled to more power and you also don't want to incentivize other kinds of Dynamics like that and so one way of maybe doing this would be to actually incentivize people to consolidate their assets into one place which would require maybe creating like really steep rewards for having more assets like steeper than what I'll call linear and so that is like really distorting power toward people who have a lot of assets doesn't sound like it would go over quite well probably kind of bad I personally am pretty allergic to that

idea but at the same time you can learn something about who people are because they've Consolidated their assets at least to some degree so now you can see entities a little bit better and so you gain information and with that information can you now in an incentive compatible way rebalance power and maybe even do better than you could have done with no at all I like that it's sort of a precursor before we think about sortition let's think about whether we can create the conditions for incentivizing the development of these kind of like stable identities via these incentives to collect your stuff under one address right you can think about it in this strange way of almost well at like an auction or something if you want

to create an auction for the opportunity to participate if you structure the auction in a way that puts people in the position that they want to consolidate their assets you can then learn about who is in the auction and the nice thing about sortition and then the subsequent process of decision making in web 3 is that in web 3 it's totally viable to reweight votes it's already being done I mean the default is you have the power commensurate to the amount of wealth that you have because of the Civil problem and so you now have two levers to tune right you can tune the probability or the opportunity to participate and once people are in the room you can tune the voting power that people have so you have two levers now

the chance to participate and the weight that your vote will count with and both of those are just ways to toggle power and maybe there's some way to use both of those in order to do something better than just randomly sampling uniformly the coins or the wallets which will just allocate power proportionately to to the money that you have so it's unclear to me whether it will work but that's really inter this also sounds like it could potentially be a perfect sort of area where ZK zero knowledge cryptography could come into play where you could prove that you were the owner of a number of addresses but without revealing that necessarily to the broader public about that and so you could maybe prove to the system that you

are who you say you are you own these addresses but you don't have to reveal yourself that would be even better if it were possible to just get information like that not for free but with like very clever cryptography in algorithms I mean the more information you can get for free the better but if you can't get any how do you incentivize its Revelation I mean maybe we have to do another podcast yeah we have a lot more to disc okay well thank you both for joining thank you yeah thank you all right [Music] [Music]