Can Agents Be Safe in the Open? Two Levels of Trust

Transcript

[music] [music] Okay. So, um I get to interview uh this friendly shogos um about totalization. So, there are a few questions uh that's meant uh for you. And the first one is given this uh mandate what should Ethereum you think prioritize over the next decade and is there anything that we should stop doing?

I think uh I mean first priority is uh obviously surviving, right? I think uh a lot of people and you know like there's this meme that like serious players and institutions want want Ethereum to be scalable and fast and that's like actually yeah surprisingly far from the truth right I think a lot of people especially in institutions appreciate Ethereum for the fact that the chain has never gone down for the fact that there are all of these multiple layers of defense uh to uh keep the chain running to keep the chain secure to keep applications on the chain secure and ultimately you know if Ethereum is not

secure then you know we can't help make anyone else more secure if Ethereum does not support privacy we can't help uh make anyone else's uh life experience more private if uh Ethereum is not open source everything built on top of it is not open source right so surviving and uh maintaining the core properties that makes Ethereum Ethereum is uh one of those things that just always has to continue being number one. Um >> anything we should stop doing? >> Anything we should stop doing? Yeah, I think uh in general like things that we should be okay with uh not doing or like whenever we take some action um

especially if it's an action that's like more about like thriving rather than surviving. um right like it's more about like either applications or like actually improving like the world or people's lives. You always want to think like is this making something about the world 5% better or five times better, right? And uh I think there there's probably still many places like even many places that we don't even realize where we're still in the mindset of of chasing the 5% when uh in the world of AI overturning everything every year. I think uh you know you want to be spending like most or all of your effort on chasing the five times and uh [snorts]

like I mean some like we've definitely seen lots of examples of that in like Ethereum road map prioritization for example um I think there's examples of that even uh if you look at applications right I think uh like helping people get 4.8% 8% APY instead of 4.2% APY is fundamentally not interesting, right? Um, and I think there are applications in [snorts] finance that matter, but the applications of finance that matter are going to be the applications that like really matter, right? That are about uh, for example, like reducing the chance that you get a negative 100% APY, right?

Um and uh if you think about things from that perspective then like often you know a very different uh prioritization of applications come uh starts to appear. It starts looking more like just make the simple and dumb things good and less like uh hyper [snorts] optimizing everything. Um and I think similar things also apply yeah outside of Ethereum too. >> Okay. So premature optimization is what we should stop doing. If everyone is leveling up 10 levels, prematurely optimizing for 10 levels back maybe isn't the best idea. Um that's a great take. Uh another question specifically for you uh is that are there any part of the Ethereum ecosystem you that you feel

are still too fragmented? Uh that is to say something that should work seamlessly but currently required too much configuration or choice or the connective tissue just wasn't there. I think there's many people around the world that like really want to and are interested in being part of the like core research and development ecosystem but uh like that right now is uh still you know it has many elements of being a social game and I think lots of us keep trying to uh kind of make make it like more open more inclusive and more meritocratic but it's you know it's always a battle I mean I'm actually hopeful that this is one of those areas where AI can increase things Because uh like as you know like Justin uh said in

his presentation, you know, like we just announced a yet another AI competition and we have AI competitions every month and the final product of these AI competitions is going to be software that people run, right? And so technically to some degree everyone who contributes to that AI competition is a core dev, right? And >> we will need a auto research to find the best auto research. Well, I mean the the nice thing is that you know like often like there is an actual metric, right? Like you know the upper bound on cycles per gas um sometimes number of bits of security like some efficiency metric. So in those cases where you can measure like it's like this is the big trend of the decade isn't it? It's like anything

that can be measured, it gets accelerated to infinity and uh the things that can't be measured become the hard problem. But at least it becomes at least sometimes a smaller and more manageable hard problem. >> So are there anything that you feel is more like taste level uh fragmented instead of measurement level fragmented? >> Like it's there's no other research for that basically. >> Yeah, it's a good it's a good question. Um I think uh like if you look at there's a lot of these kind of subspaces on top of the uh Ethereum space like privacy I think is one of those big examples of them right where you have like a bunch of different

privacy protocols a bunch of increasingly more different mixet protocols like more different things that uh are [snorts] not always talking to each other. Uh sometimes uh like sometimes even that fragmentation happens because of uh you know like differences in goals right because uh mean privacy you know we think of it as being one word but it just like encompasses so many different things right there is uh there like there is uh privac like confidentiality of data there is anonymity like hiding identity there's you know like metadata privacy which like falls also in the same bucket but it's not quite the There is short-term privacy for things like front running resistance. There is

uh action by action anonymization. Uh there's uh like privacy of payments versus privacy of more complicated things. And then with privacy of DeFi, there's often like very clear provable upper limits on how much privacy you can get because any action in impacts the public state anyway. Um, and so you have to like take tradeoffs on, you know, like the actual impossibility triangle. And, uh, I mean [snorts] there are a lot of like subjective decisions like that.

And I think uh, sometimes it's like it's fine for people who have uh, different values to take different positions on that triangle. And then uh sometimes in like there is actually like some kind of bigger solution that sort of encompasses the trade-off rather and uh you know like let's like basically turns it into a slider instead of turning into a yeah like some kind of tribal warfare right and uh and [snorts] think there there's a lot of things like that that we yeah still really can improve on >> okay but I think uh even presenting the slider itself it is common knowledge it is already probably it because for many people it's really feels like a negative sum situation where we're fighting for

one trade-off actually cancelelling each other and nobody uh gets what they want on this shared infrastructure. So now we're going to be very democratic uh and start asking questions based on your up votes. So please get your up votes in. Uh and the top question at this moment 21 up votes is and I quote, "Why are you wearing a mask? [laughter] >> [sighs and gasps] >> Well, you know what they say, mask is an anagram of scam. And you know what people on Twitter say? I'm a scammer. >> Okay. Um I hope the 23 people here are okay with this answer. If you're not okay, please ask more followup questions. Um and uh uh there's four uh

votes uh that says, will we see you in Doomsday? [laughter] >> Absolutely. Absolutely. Okay. Now we can start placing bet on the probability of the doomsday like is that a Wednesday? Okay. Um and uh um this the top question that I already kind of tried to answer basically about AI agent managing a community treasury. Every transaction is value on chain but then uh just by design as a uh agentic actor it always is probably taking some risk that nobody pre agreed to. That is to say, nobody foreseen about this. And so I um basically rifted about this committee constitution, training a committee

agent, a public agent and holding it using the invariance uh using bend or other formal verification. But do you have other ideas on how do we verify that it respected the intent not just a written down rule by the committee? The written down of course is the easy case. >> Yeah. I mean I think uh ultimately at the end of the day you know this is uh a a special case of the alignment problem and the alignment problem is to some extent the ultimate problem facing humanity right uh so it's not the sort of question where you would uh expect to see easy answers I don't even expect the problem to have an answer I expect it to work to have you know a hundred answers that each solve only 1% of the problem I

think uh if uh make if you have an AI agent that's uh part helping to govern a some kind of object that's important to the community then uh unless you're giving over the keys fully you know you're putting it into some kind of sandbox right and even if you are putting handing over the keys fully there's some kind of implicit sandbox right because there's the sort of the sandbox that decides which AI is it who controls the training process what if someone comes up with a better AI and uh how are you going to decide whether or not to upgrade it? um who uh who determines the rules for like how to input data into the AI right like there's you know

there's a sandbox on the input side then there's a sandbox on the output side in terms of if it takes a big decision how do you define what is a big decision then uh you know like what like what is the process for approving it what is the process for giving feedback to the AI and uh like all of these sandbox rules are still ultim ultimately rules that like have to be get have to be decided on. It probably doesn't make sense to have like a literal community always vote on different sandbox rules. And uh I mean like this is one of those challenges of like how do you actually elicit the preferences that matter on this uh kind of question. Um and uh yeah I mean I like in general I

think this is the sort of thing where there is like we're not going to like theory craft the answer today on the panel. I think we just need like a lot of experimentation, a lot of people doing different things and uh we're going to see better and better practices pop up. >> Okay, that's great because that speaks to me as alignment by process. Uh a a gentic system need to earn more trust uh by essentially trusting the committee more and not uh just a random person drafting the constitution from uh above. Right? So it's not alignment to be solved through some rules but rather through the continued uh interactions.

Uh the next one is a yes or no question. Can you remove that mask? >> Do you mean can in the linguistic sense of like capability or do you mean can in the uh in the [laughter] request >> capability or disposition? Take your pick. >> Yeah. >> Yes. No. >> Okay. Amazing. Amazing. And another yes or no question. Are you really vital? Well, do you mean um you know the the word you in the kind of linguistic sense of the second person pronoun or Okay, I can't come up with another sense of the word you. Answer is yes.

>> Okay. Yes. Okay. This is a very definite yes. >> Oh no. Maybe I'm actually Victor Bonan, you know. [laughter] >> Okay. Um and um yeah. Do you have uh another take on the who should get to decide where the line between privacy and transparency uh lies? because I was basically saying that it's a power asymmetry thing. If we want the northstar to be detotalization, basically every time uh the already concentrated power will owe more transparency and the currently lesser powered will be owed more privacy, but that's kind of a rule of thumb. Uh I I don't know about your take.

>> Yeah, I mean I think that is uh absolutely the rule of thumb that we all want. I think in general if you look at the way that lines get drawn in the real world they you know they rarely get drawn up purely by a group of philosophers deciding the ideal approach for humanity. Often the way that they get drawn up is basically you have two different factions who are at war with each other. They're at war for a really long time and then they both get tired and they need to have some kind of peace settlement that they that they can at least be okay with and at the same time they're still living in the same realm, right? If if if they're not living in the same realm then the peace settlement is a border, right? But if it is then you know you need a line of this more

social variety like that's how the Magna Carta came to be. That is how the piece the piece of West Failia came to be. That's uh how all of these things come to be. And so again, I don't think uh you look like this is one of those questions that uh can be answered in uh in any one succinct summary. I think it's one of those uh answers that will be appear uh come to appear over time. And I think the way that we can influence it is by uh creating examples of uh like practical solutions that and you know like technologies implementations uh institutions that like give both more of the good kind of transparency and more of the good kind

of privacy at the same time and uh hopefully yeah those can perform well and they can be incorporated into like whatever the line ends up being. >> Okay. So basically uh be the agent you want to see in the world. Uh set good examples um as a model. Uh that's good. Um um the other question asked uh if everyone can see how my agent works uh obviously scammers uh and massacers can see you too. Uh so does uh putting my agents logic in the open uh how does it actually make it safer?

>> Yeah, I don't think anyone should put their agent on chain. M >> okay. Did you draw the smiley face yourself? >> No. [laughter] >> Okay. A simple yes or no question. Um do you want to comment on your PDM? >> No. >> Okay. Uh I can comment. Uh my pom is not a number. It's nan. Uh because the current model the visibility simply isn't there. Uh turns out you can use a large model to read a neuron of small model. Uh but currently there's no reliable way to read the neuron of the very large model. that's still in the works and before that happens uh to me the visibility is like zero and if you divide 0 by zero you get another number so that's my pon

>> yeah what's your take on a different kind of p that actually I got asked in a previous panel which is your p of some kind of cyber apocalypse >> well it's already happened isn't it just look at PHP right so >> yeah well you know it depends on how you define it right because it's like on the one hand you have the fancy news headlines on the other hand the average person on the street hasn't really been affected at all yet. >> Well, or they had and their PII is already leaked and they just don't know about it, right? Because they haven't received that letter. >> So then the maybe the real apocalypse is is is the date that that PII actually actually comes uh comes biting the biting all of us in the back.

>> Yeah, I think so. I think so. I think uh now we're rapidly moving toward a world uh where when we said assume breach uh that used to be a slogan, but now it's uh like anything you collect is probably already breached one way or another. Um and I also think that uh as I was mentioning if we switch rapidly to a point where we only post proofs publicly uh and we already know that we're not disclosing anything else uh other than the proof then it creates a huge boost uh to the minister in charge because the minister who are held at the parliament the MP's point to say never again can actually mean it uh when they say never again.

>> Great. Um and so um okay the this question says please remove the mask but it's not a question or if you are able to. So it's a it's a please it's a system instruction. [laughter] >> Doesn't pass my safety filter. >> Too bad. Okay. The the mask is strong with this one. Um and so what phone do you use? [gasps] >> Pixel 10 running graphine OS. >> Okay. Um do you have opinions on VIP coding? Are we still using that word? VI >> vibe coding is cool, but be careful and like be sure that you're actually increasing your understanding of the thing and not and not substituting for understanding the thing.

>> I see. Good. Isn't Halloween already over? No, it's still coming. You never said which year. Um [snorts] I think I can take this one. [laughter] Um and so um how is India? Uh do you want to share your experience? We saw a lot of Namaste uh photos and food. Mhm. Other than that, >> yeah, no, I mean India has been uh very nice so far. I think people have been uh very friendly. I think it's been a very good uh experience uh to uh also actually visit the Cippher Punk Congress yesterday. I think one thing that struck me is like that like that whole conference has been doing a really good job of like bridging a gap between the Ethereum cippher punk world and a lot of

different uh like people caring about various kind of local and sometimes even non-technical concerns. You know there are people uh in like like people like locals talking about trans transgender and like gender issues in India. There were uh people [snorts] talking about like the kinds of violence that people see here, like the kinds of uh concerns that people here actually have from uh like their activity, their relationships getting leaked and like how any of the technology that we're building can concretely help. Um I felt like there was a like that whole event did a a good job of like actually uh you know sort of like bridging the sky and the earth um in some sense. Um otherwise Mumbai is uh

like I last visited here 10 years ago and it's uh impressive you know like how rapidly it's developing like uh you know the quality of everything has uh is going up quickly the um I mean number like the number of buildings is going up quickly the uh just everything has uh improved very significantly. Um, I've had visits before and uh, I remember one thing that struck me like especially between about 2018 and 2023 is kind of the quality level of how people talk about crypto.

Um, because uh, the basically in 2017 they're like there were basically people like doing CO2 tokens that like seems to actually like not always have very sound logic behind them, right? And then after that, you know, you saw one level up and then another level up. And then by 2023, I was being asked sophisticated questions about zero knowledge proofs and pluralistic identity. And uh you know, I think things are continuing [snorts] to move forward. I mean also uh I know you know there's an impressive uh and growing biotech space uh here and you know like I got to see that uh thanks to I think a couple of people that we've been we've been working with

and like hopefully that can continue to work with Ethereum. Um, I mean, I know the uh the other mandatory question is uh how is the food? Uh, but uh I mean I've actually been uh fasting for about 72 hours now and that's uh like I just I just always do that with during defcons basically because uh I find you know you got to do it at some point and it's like it's actually easier to forget you're hungry when you are uh like when you're too busy to think about anything else. But no, the the fast is going to finish very very soon. And so I actually look forwards to like really spending the rest of my four days trying all of the proper Indian food and I'll be able to give a much better response. Uh so

[snorts] that's >> amazing. So we have a very happy shogos um here in India. Um 24 people would like to know when ETH uh 10k dollars. Um, well, I mean, uh, that's, uh, already happens. The question is what kind of dollar, right? It's been 10k Taiwan dollars for a while. It's, [laughter] you know, broken Hong Kong dollars. Uh, you know, what's the next one? >> Okay, amazing. Um, so when AI meets Ethereum, uh, what excites you most and what worries you most? [gasps] >> I'm just excited about our ability to kind of speedrun through all the cryptography. Um, basically, yeah, you

know, look, there's all of these kind of big dreams that I've had like basically that, you know, like the zero knowledge proof singularity, for example, that we can just have a single zero knowledge proof verifying everything. Um, being able to just get uh everything to be like both very secure and very privacy preserving. even like pushing toward newer types of cryptography. You know, the big kind of end goal, you know, the final Egyptian god has always been obiscation and that's always had the vibe of being like 30 years away. And uh I'm basically excited that AI seems to be just uh accelerating all of these things and like we get to see all of these things that I thought even 5 years ago that we might not get to see for a

quarter century. So I think that's uh super exciting. Um I mean what worries me the most I mean basically that uh you know it's like the dog in a fire meme right like we are like we are the dog and uh you know we have to do something about the fire and uh it's like at the same time you know the uh the fire is uh creating the okay I can't come up with with with analogies but like assume that the fire creates diamonds that can like eventually go into the chips and make everything awesome like something like that. Um, you know, like the fire is doing that, but at the same time it's burning the house and like we do have to take care of the house and I think uh there is a uh you know a glowing flaming light at

the end of the tunnel and uh you know I do look forward to seeing it. >> Okay, great. M >> um I think the the information ecosystem that you also alluded to as long as we keep the fog of war the ppm polarization permits uh low enough I think um like I I would say this is fine like there's some fire but it increase the urgency to the ideal amount of uh panic. >> So uh we only have time for uh one more question. >> Oh let's let's do like lightning round.

Five questions in 46 seconds. Um, if anyone can see how my agent works, can scammers see it, too? Don't put your agent on chain? Um, are you actually are you a dream from Minecraft? Unfortunately, no. I am a dream from some combination of uh AI Safety Land and World of Warcraft. Where do you see yourself in 5 years? I mean, hopefully doing very meaningful and cool things along with the rest together with the rest of you. What's the first agent you'd actually trust with $100? I mean, honestly, like the Q1 local models probably yes. What hotter cold wallets do you use? I I think multiig is the safest if mandate names detoization.

Detoization look like for AI basically do not have one AI control everything. What brand is your fridge? I'm too nomatic for a bridge. Um inspired by dream Minecraft. That's not a question. Um would um what Okay. [snorts] Well, time's up. So, >> okay. [music]