When operational autonomy of blockchains meets decisional autonomy of AI: legal challenges

Transcript

Ah, great. Okay, we're gonna start. Uh, so I'm de Philippe. Um, you can recognize me on the far and right left. Um, I'm a legal scholar. I'm a researcher at the center in Paris and uh I'm a faculty associate at uh the Burkeman Klein Center at Harvard and uh most of my research is about uh investigating the legal challenges of uh emerging technology in particular blockchain technology and more recently also AI. Um and then Fat.

Hi everyone, I'm Fatime Fatiz, a Swiss qualified lawyer work with crypto for the past couple of years. Um I work mainly now with privacy projects and other form of um more cypher punkish um tools out there. And um yeah, we're going to talk to you today about autonomy. Um autonomy with a big A and all the other letters here are big too. um and AI and blockchain and how both together create this big autonomous system. Um decisional autonomy.

AI is basically the innovation and the tool that gives us access to um having decisional autonomy. AI would reason make judgments on its own um through kind of reasoning to which we don't necessarily have access the path to which it uh uses and there is no human in between that tells the LLM which kind of um output to give us right like it makes its own decision on the output and that's what we call here decisional autonomy so now we have a system that can um basically make uh decisions But decisional autonomy is not enough

for a full autonomous system. We also need operational autonomy for that, right? Because I mean, as you saw on the previous uh slide, um an LLM that can be plugged off, um that can be unplugged, um that a human decides whether or not it uh can operate is not autonomous in that sense. But we do have systems out there that are operationally autonomous.

Um and that would be blockchain and smart contracts and like um decentralized uh permissionless autonomous technology and both of them together would build a more of a comprehensive autonomous system. It's a spectrum and you have decisional autonomy on the left, operational autonomy at the bottom and um there are systems that have neither right centralized platforms you you decide whether or not they operate uh but you decide also um you know what output it gives you make decisions for the platform like for the system itself. So everything is dependent on outside kind of um interference or inference which

would be usually human. Um and then you have LLMs which have high decisional autonomy but um right now mostly they are not operationally autonomous. Then you have Dow, smart contracts, blockchain as we I I mentioned before, oper operationally autonomous um not decisionally autonomous and then you have a mix of both that would be the autonomous system. So the the fully autonomous LLM um plus blockchain uh system and all of these have different kind of interactions with the current legal

system that we have. Um so centralized platforms I'm not going to get into details on this one but um the legal system as it's built right now in the world is meant to address centralized platforms like it's meant for centralized systems it regulates human behaviors it regulates corporate behaviors um it it kind of regulates it all successfully because there is uh people and corporation uh to to make accountable for the outcomes, the decisions, the ways they are using.

But for um LLM systems with decisional autonomy, uh the current legal system that we have doesn't necessarily grasp um the legal challenges that arises out of these systems. And here we listed a bunch of them. Um it's definitely not a comprehensive uh list but these are kind of areas of um of legal problem that may arise because of the autonomous nature of the decision making of the LLM. So it could be on uh the governance and accountability of the model itself. Um accountability would mean that if it gives an output that then leads to some form of liability. So someone relies on

that output and then there is some form of damages arising out of it. Who is responsible for that and that is not necessarily very clear or or really battle tested right now. Um and who is accountable there's like this gap in liability transparency fairness and fundamental rights. So there are regulations usually that would um say that decisions have to be made following certain certain sets of principles like it has to account for some form of equality uh remove biases. These are more of the how the government and public sector approaches decision making. Um and this is not really transparent within LLM systems. So you

don't necessarily know what path they use, what are the biases embedded in um in then like that leads to the output of of uh by the LLM and that can can lead to issues data privacy and IP. So the IP one is I think the most discussed uh since kind of the mainstreamization of um of AI but it's um does it violate IP of third parties by like using data that was written or produced by other people that is maybe copyrighted to then give you an output or modify it um for the user's benefit. Um and then privacy because there are again rules and

regulations around how our data must be kept like private how they have to be archived how they have to be treated and uh the LLMs don't necessarily internalize these rules and may kind of leak data on individuals that uh shouldn't be discussed or out there safety security and product liability. So that's more if um the LLM for instance or the the prompts are kind of fiddled with by third parties like prompt injection or any form of kind of misuse um of the LLM to uh harm um the user for instance. And then the last one which would be just more specific

regulation that exist for instance the financial sector um or some some other very highly regulated industries um that need to follow some specific decision- making path uh within the corporations and so on that um that the LLM cannot kind of comply with. So these are all different areas where like depending on the usage or on the tool that is being built um the the builder the user the adopter must pay attention to be liable on on on these fronts to be sorry um responsible on these fronts and then there are safeguards that um can be put in place. So this could be for instance terms and conditions um that clearly

allocates the risks the liabilities um who is supposed to to do what within that system so that you know where if if if there is a failure you know whom to blame or to look after um and uh and so on um so yeah a lot of testing so this is more when you're building the LLM like you need to put pay attention to kind of failure modes and how to preempt them. Um, insurance is also an interesting uh one to to have in mind and discuss that could basically um cover damages in case there is something that goes wrong.

And the other um autonomous system that has very specific uh legal risks because of its autonomy is blockchain and and Dows and these I think you're all pretty familiar with because we've been discussing it for a long time and you've been also encountering them in your use and and building of the of the tools. Um so I'm not going to go into details really but it's on the governance and accountability side. you know if there is um a smart contract that is autonomous who is liable for the usage of it for instance um a good example would be I think here tornado cash um who is liable for the misuse of the tool as well who is liable for uh compliance with some sets of regulation what sets

of regulations are those it's also an open question um if there is an exploit or a hack within a tool that you build who is liable again for This is uh is code law or not? Like should you have um paid attention to some things before? Like can you be insured? Um have you had enough audits? These are more industry standards than legal standards right now. But definitely things to pay high attention to. Um legal validity enforcement. So there are um again regulated sectors um whether like you're buying and selling certain goods that need for instance like handwritten signature that you cannot necessarily

have online or you can do things that are done differently than in the meat world and u that can lead to issues when you have to bridge with the regulated system that data and privacy is pretty much uh similar to the LLM in the sense that um there may be data uh onchain that is they're hacked forever and that shouldn't be there and and then what happens with that and who is liable or uh how is the harm kind of repaired on that front are also open questions. Um and now we're going to get to the most interesting part of this talk which is the autonomous systems uh and what they look like and what kind of problem spaces emerge out of that.

Thank you Fat. Um yeah so we're kind of reacquainted with everything that is on those sides. Uh fully autonomous system is something that is more emergent. Uh we are seeing like some glimmers of them but uh it's still in the making. So um basically what we're trying to look for is what are the systems that are integrating both decisional autonomy and operational autonomy. one prototypical version for the people that know the blockchain based life from Plantoid. Uh it's an attempt at actually creating uh autonomous system um that does indeed integrate some degree of decisional autonomy, some degree of operational

autonomy. However, the Plantoid system is imperfect in its autonomy for two reasons. The first reason is that so planttoid is a blockchain based life form. you can feed it with cryptocurrencies and then it's going to generate uh using generative AI is going to generate an artwork mint this artwork as an NFT and give the NFT as the digital seed back to the human pollinator. The problem is that the smart contract is not capable of processing the generative AI and therefore it does rely on a centralized platform an external platform that is doing the computation and then reminting taking that and remitting it. So there

is like a separation between the smart contract the centralized platform and then the decisional autonomy of the AI and the operational autonomy of the blockchain. So this is it has both but they are not connected to one another. Another problem of the planttoid is that while it does operate as a DAO, it does rely on human governance. So everyone that has collected the digital seeds participate into the governance of the planttoid in order to choose the way in which the planttoid will evolve. But this is obviously slightly problematic when we talk about autonomy because the autonomy of the planttoid is subservient to the participation of human beings.

And if there is voter apathy as in many dows then the planttoid cannot actually evolve. So the question the very important question and of course planttoid is just one example but a lot of blockchain based life form and dows as as a subset of blockchain based life form have actually the same problem. They they have some degree perhaps of operational autonomy. They have a smart contract of a DAO. They have maybe some degree of a autonomous decision making but they are not integrated with one another. And so how can Plantoid actually become a fully autonomous blockchain based life for by integrating decisional and operational autonomy? So there is two ways in which we can

actually think about integrating AI and blockchain. Uh the first one is basically having a DAO that is governed by AI agents. Uh it is very interesting but it's actually not very novel and not very like groundbreaking. uh because it has the same problem. It is a DAO. So it is operational autonomous and then you can have humans or you can have LLM or whatever type of autonomous agent on the side on the borders but there's still a disconnection. It is actually not the DAO that is autonomous. It is the DAO that is governed by autonomous entities and those entities might not have operational autonomy. So a more interesting model is the DAO LLM which

is a DAO that is governed by an LLM and that LLM is itself governed by the DAO and so the LLM and the DAO become one single systems uh which therefore integrate both decisional and operational autonomy. Now how to actually implement it? It's uh still uh a slightly open question. there is a lot of possible path towards this integration. Uh perhaps the most uh the most elegant of it will be like using some kind of like ZK VM and you having a proper uh ZK proof. The the main challenge here is you want to make sure that when the DAO is requiring a decentralized network to make an inference the inference is actually the

one that was requested. And because the DAO cannot verify what is being done, it needs to have a computational mechanism for verification. So you can use like ZKVM, you can use like T at test station, but therefore you also need some kind of trusted third parties or you can move into this kind of multiple validator system with like optimistic machine learning whenever it's uh going to be actually practically usable. Um and then AVS uh layer uh for those who don't know it the WAV system which is extremely cool. Uh so those are all possible ways to actually integrate this decisional and operational autonomy. Um there is not many uh blockchain based

life forms that are actually integrating them yet but there is definitely like some good hope uh along the way. Now there's some interesting design problem with all this which is that in order to actually allow this verification then we need to have some kind of like deterministic AI inference uh which kind of makes it inherently subject to the main problem of like adversarial prompt injection. So if I have a DAO that uh is requesting a decentralized network to compute uh a particular type of inference, if I know exactly what is the context in which the LLM will be executed, what is the LLM to be executed and what is the prompt that will be executed, then I can also brute force

the system in order to get the proposed outcome so that I make a proposal and I know this proposal will actually be accepted by the DA or I can do some more malicious usages and actually try and break the system. And so what this means is that while we can integrate operational and decisional autonomy because of the problem of verifiability and therefore of deterministic computation, um there is always a need of nonetheless having some humans except that humans are no longer at the center of the decision. But the humans become this kind of safety valve that exist at the outside which basically have the veto power which can say well actually this proposal was not really good. it was actually a malicious one, we flag it. Uh we can have a system of like

stake based system with slashing etc. So humans are still part of the picture but they are just there to protect the system as opposed to operating the system. Um now legal questions. Um so we've seen just decisional autonomy with LLM and so forth. They have their own legal questions. Just operational autonomy also have its own legal question. But the real interesting legal question emerge when we actually have a system that is operationally and decisionally autonomous. And so one important problem is the question of unstoppable system.

So when we when we integrate those two system of autonomy, it actually becomes autonomous. therefore at least in theory unstoppable. Uh which means that uh one of the one of the reason it becomes unstoppable is like one single actor if one actor is actually executing that computation and if that computation is validated uh by the by the operational system then it will be able to operate. And so you just need one person to be willing to create like something that maybe is an illicit uh activity and then it's going to be done. then you cannot stop the system itself and you need to stop everyone in order to make sure that no one will actually do it. Uh and then maybe more critically even if you know who's that actor that has actually done that uh that execution um how do you

actually enforce it? So if you if you don't identify the person the system will still execute and there is no plug that can be plugged off. There is no shutdown button. uh the only possible enforcement will actually go to go deep deep deep into the meool and actually censor the transaction towards those autonomous systems. Uh and another set of legal challenge in the question of responsibility which is well we cannot stop it. Okay. But then who are we going to punish if something is done? Uh there is the question of is should we claim strict liability for developers or for the deployers? uh should we impose some kind of liability on the trainers on the LLM which has led to a bad decision uh shall we shall we hold users to be responsible because

they have interacted with a DAO LLM which is actually doing illicit activities uh we have encountered similar question with tornado cash of course um so the big question is uh uh are we going towards um not a very beautiful future uh question mark uh the answer is of course not necessarily uh because there are legal solutions or legal approaches as well. Uh from the most simple one to the most complex one. The first one is well just to admit actually this is for nature in the same way as we're not trying to regulate a tornado. We just accept the damages of a tornado. A LLM is just something that we cannot control and we just have to cope

with it. It's easy solution but maybe not so interesting. Uh second solution is to actually uh accept that it is possible that those things exist and maybe we cannot fully control them and stop them. But in order to deploy them, we need to have an insurance scheme. They need to come with their own insurance system. In the same way as we cannot drive a car without an insurance because of the potential liability, then you cannot deploy a DAO LLM if the DA LLM itself doesn't comes with its own insurance system. A little bit more interesting and sophisticated. Uh third possible solution is actually maybe we need to recognize that the existing system of accountability and legal liability do no longer work in a world in which we're not related to human

autonomy but we're relating to algorithmical autonomy and so maybe we need to devise new forms of algorithmical accountability. uh we have of course Claros which is working really much on this and so perhaps those DO LLM could integrate some kind of like dispute resolution system which are themselves algorithmically and onchain systems and then finally perhaps the most interesting one is the question of oops uh is the question of like the law of the horse um which is basically going back instead of looking into the future and trying to reinvent again once again the wheel because the world has changed and therefore we need a new legal system. Uh we can actually go back into the past and we can actually look at how did we actually deal with autonomy and

liability at a time in which autonomous system already existed. They were just not mechanical and technically they were actually animals. And so for a long time we were actually driving not cars but horses and maybe horses and cars. Um and actually if we look at the history of the legal system, there is a lot of common law settings that are actually providing very interesting references about how do we deal with autonomy? When do we see strict liability versus start liability and so we can actually learn from the pre-technological legal system in order to actually address the new challenges that come with this new novel algorithmical

system. And just to conclude uh all this is to say that we should actually perhaps stop looking at those DAO LLM and those technical system as software just software and actually recognize that maybe they are like this new type of creature. Maybe they are symbants and so we should stop treating symbants like software and actually recognize that they are autonomous and maybe they also have rights just like humans. Thank you.

[Applause] [Music]