DAPPCON 2024: How To Put AI Onchain… And How It'll Bring The End Of Humanity - Elad Verbin

Transcript

hi everyone uh I'm alad verbin this is probably the most important talk of my life maybe the most important half an hour of my life uh but I'll take it chill because I'm going to tell you how we in this room might be ending up the people responsible to killing humanity and I'm going to tell you some ideas about how we maybe can stop that so first of all let me start with the fact that crypto is amazing and it's my life's dream and everything that we're doing in crypto is making an import a huge difference on the world we believe in decentralization decentralization is an incredible Force we create censorship resistant systems that allow anyone to avoid government oppression and so on we're building the world computer as a

computer scientist I care so much about making a computer that is joined to the world at censorship resistance that's verified that's everywhere but we also maybe going to stop the to destroy the world and let's try to stop that so here's the summoning ritual that we're participating in and we're not realizing that's what we're doing but here's the summoning ritual and all of us and all of us outside and all of us in crypto and the AI people are all participating in the summoning ritual that's Midway right now it's like this ancient catulo myths that we're summoning the Beast and we think we're doing a good thing so do the cultist that sumon the Beast and there's these progress bows if you ever played a board game this is how it we're on the progress bar and we're pretty Advanced and at some point when we cross

the black line culu GPT is summoned and then there's this super intelligence that lives on the on the blockchain on a decentralized system that sensorship resistent that can't be stopped and that might be the end of all of us and the start of something better I mean I think this will happen I think we'll summon it I think it will kill us all and I think this will be the next step for Humanity what comes after it but let's try to stop this because I like being alive okay so we're going to look at this program bars and look at what they get to and I'm not sure what the time estimate is but it's something around these lines it's not far away and I'm going to tell you why crypto is the actual accelerationism in eak crypto we here are accelerating all of this AI

Doom that you've been hearing it about by a lot okay so here's the problem and people don't talk about this I don't know why we here are building a very strong computational substrate that's sensorship resistant that's scalable and again I'm not talking about onchain I'm talking about stuff like Napa Bens or Olas things that are like a side chain that is scalable scalable sensorship resistance and privacy preserving or confidential this should be enough for you to realize we're in deep trouble because we're building a computational substrate that's perfect for an invasive Spees species to come and expand on okay so when the AGI people manage to develop AGI or really anything that can put

proceed on its own and live on its own it's going to come and it's going to go oh I can grow here I can hide here I can H develop here and that will be the end of us now AGI is being developed and I'll convince you that it's getting developed and that's not stoppable and we shouldn't stop it exactly we should maybe slow it down but this is getting developed by pretty much everyone as soon as it gets deployed on chain and I will show you why it gets deployed on chain it continues living there as an invasive species if you know that story about the guy we don't know who they are that bought this seaweed in Japan took them to their um fish tank in Europe then someone flashed it down the toilet and now it's half of the Mediterranean and it's eating everything and and and

killing everything because it has no natural Predators that's what we're doing there's going to be a labl it's going to come to us it's going to come to the thing that we've built and that we're building it's going to spread there we won't be able to stop it and rather than taking over the Mediterranean it's going to take over the world okay so crypto is the acceleration acceleration is factor and I'm going to tell you why we should be very very careful and why we should design for a a super intelligent why we should design for a super intelligent entity as a possible user for our crypto products okay so here's the solution the problem is that Perfect sensorship Resistance together with confidentiality ends in world destruction that's the Axiom that I'm

going to try to convince you of and then the solution to that and I'm really sorry to be doing this because I love sensorship resistance the solution is to abolish full sensorship resistance and full confidentiality it's not going to work it's very dangerous we need something else and I'm going to tell you some one approach to how to get beyond that that is about selective censorship and selective de anization that's towards the end of the talk I'm going to tell you how we need to stop it and otherwise we just kill everyone but we've built some great computer science along the way and launch the new species maybe that's a good thing so a bit about me this is my Twitter this is my telegram if you want to talk about anything about this uh talk to me and I'm originally a computer scientist I

did a PhD in algorithms and complexity Theory my life's mission is developing computer science and therefore when I discovered crypto in like 2013 there was no one happier and I got into it and I started working with a swarm project and I uh I so I my my career is between Ai and crypto and therefore I'm kind of on the first line of looking at this today I run a venture fund called lunar Ventures that invests in preed and Seed computer science based Ventures all across Europe and the world and we invest specifically in the stuff that I'm going to warn you about so I'm the canary in the co mine because I sit in these meetings with the founders building the future and I'm very excited about supporting them I'm building this stuff just as much as you are but fundamentally when I go to sleep I go I

don't know if you know this sketch this is a pep show sketch where they're like Nazis in the lines and they H go and he goes hunts I've been thinking about the cap with the scull I've been thinking are we the baddies so I'm going to say that all of us should be thinking and answering affirmatively to the question are we the baddies we think this is great I'm supporting this I'm the most culpable out of all of us but we're the baddies um now I have one request which is that you give me the benefit of the doubt I'm not going to manage in half an hour to convince you of everything so take me seriously but not literally I'm going to show very explicitly in an engineering building mindset how you'd build such a super intelligence that

destroys everyone because I want to show you that it's possible and we're going to build it together but I can't tell you how the world exactly is going to pan out I'm just showing you axioms that says this way is world destruction okay uh so take me seriously but not literally and let's build culu GPT together after that I'll go back and Define some of the terms we're using I'll tell you about the whole life story of this new organism and then I can tell you I'll tell you how we save the world and if we have some time we'll do some FAQ Okay so what is culo GPT what we're building is a sensorship resistant organism or algorithm or smart contact or whatever it is in the sense that it's Unstoppable if you carpet bomb computational centers

the mining rewards will go up and it will live elsewhere right that's the meaning of censorship resistance The Thing Lives and migrates and cannot be stopped okay even if governments we if we decide we we've done something terrible we can't stop it it has the cognitive power of a million humans or more it's like super intellig that's the meaning of super intelligence it's autonomous in that it's it knows it exists probably or at least it's IND differentiated from that it's a strategic actor it makes good decisions it wants to be alive and it's a new life form that I'm very excited even if we kill everyone I'm excited because this is my life dream so let's build it let's start I'm going to show you three things to build and end with the to GPT so let's start with things that all of

Silicon Valley is building this is what you're seeing now is what Silicon Valley pretty much everyone in Silicon Valley and all the startups I'm talking to are building right now and these are the progress bars right we're not quite there yet so it's a strong agent that can plan your H son's birthday party okay it's a personal assistant so what does it comprise of it comprises of NLP modules because computers speak English now yes if you're seeing this in the past I somehow computers learn to speak English they can instruct people they can negotiate they can convince I'll show you all of this later they can write code they can interact with API the thing doesn't have to write very good code because it can get people to do H to write code for it like Bitcoin is getting us to write code for it as

well and then H rewarding Us in return and it can it needs to verify compliance so that's the NLP part okay and that's like mostly build we need to make it a bit more robust I'm working on it it's getting built any day now no worries now there's the reinforcement learning modules that are much harder this is the Strategic thinking long-term thinking strategizing part so this is the part that follows goal strategizes decides what to Outsource we're much less Advanced on that but people are working on it so hard right now you wouldn't believe so this is the thing that's going to exist in a few years because everyone in the world wants this to exist and you know it right okay so that's number one we've built a strong agent okay now suppose you're running a crypto business suppose you're running

some drug operation suppose you're running the new Sil CAD you want a censorship resistant agent just like before but censorship resistant what do you do you take the same exact thing but you run it on a sensorship resistant substrate you keep its actions private using zero knowledge will get to that or F morphic encryption and you keep a wallet to allow it to transact in value and now it's running a Sil code for you automated on chain but it answers to you so great right I'm I'm very excited to build that you take the thing from the previous slide that everyone is developing put it on chain amazing there are a lot of substrates to put it on we'll talk about it later check out Napa I think it's the most interesting like complete take of this okay and we're like 50% there we're not quite there

I'll go I'll take you about it okay now let me tell you that if you've managed to build this you're like one micro step away from a worldall destroying autonomous robot why because um instead of sorry wrong slide yes because instead of following goals it sets goals it mutates itself it thinks oh I'm going to be better today it makes some money because it needs to survive on the computation state it costs like a few hundred million a year or something like this if it's a very strong if it's as smart as thousands of humans it cost a few hundred million a year to run now the crypto part it needs to migrate and defend itself so there's some mlops parts and it needs to pay people via Z Cas or your favorite confidential um asset transfer but that's it if people

build if people build this and they do and if people build this and they will and I will then people are building this any second now or the thing is building itself right this can become this can become the next thing can become C GPT just by like being original you you know just by taking some extra steps along the way okay so we're summoning the thing and it's going to get summoned and I think if we in the current course we can't do anything about it and that's the H that's how we've build the thing and of course tell me offline if you want any more details This is complicated I pinned in my Twitter the Tweet where I give some more details about how about the computational substrate for this like it's not trivial to do this ask me on Twitter we'll have

the discussion but it's possible and we're doing it okay now um building shared knowledge between us so NLP in 2017 computers learned to speak English we invested in some of these startups I'm having first front front row seats to all of this it's amazing and terrifying so computers can convince people better than people can convince people even on zero shot but then they can learn how to convince people better and then they can learn how to convince you better so computers are getting really good at convincing people and they can write code which tells you pretty much all you need to know now uh and check out this book on because the point of computers is that they they don't like they just don't recognize the truth value of the truth they're much more effective Liars than almost any human except like very high

high performing Psychopaths so it lies convincingly the gp4 technical report already tells us about how it can convince people to solve captures for it by telling them hey I'm blind can you help me uh and llms check out this paper LMS are amazing at convincing people so that problem is solved but LMS are not good enough decision makers then that's our luck because decision making is hard it's strategic that's where reinforcement learning and stuff like it control theory and so on come in if you Fe Ai and if you say oh when is it P Doom when when are we close to Doom this is what you want to pay attention to because that's the latest that's the leard so it's a strategy machine reinforcement learning or it allows you to build strategy machines the

state-ofthe-art is self-driving cars right it's the same it's the same technology behind self-driving cars behind Alpha go behind some stock trading robots and so on so we're working on it but we were kind of blocked because computers couldn't speak English so there was there was certain amount of generalized decision making you couldn't do because the thing didn't speak English well guess what problem solved so people are picking up computer decision making and running with it towards agents and the thing is getting built and uh and yeah check out it beats a people at strategy which is like a war game that's in English so we're getting there um and then the central cloud is where we all come in right accelerating it so the centralized cloud is this layer like Bens or Olas and so on

together so that's the compute part right it's like AWS but de centralized and sensorship resistant so it's resistant sensorship resistant tless privacy preserving it has storage like swarm and ipfs it has compute it has verification guarantees right you need to know that they think that R really ran because otherwise the humans can kill it right or can score with your SM smart contract so you need to make sure that um as someone developing decentralized St you need to make sure that it's verified and there are great Technologies for doing that that are growing we're at like 50% as you can see so zml and there's optimistic ML and there's tees there's all kinds of great solutions to make this happen and then you need it to be private because you don't want anyone spying on your

computation and the AI doesn't want anyone spying on it so this is also getting solved Z which we invested in all kinds of other FHA methods secret Network and so on so we have Technologies for all of this and it's kind of about making a bit more scalable a bit more efficient and putting them together via something like NEPA and so on you're going to be able to build this substrate Okay so we've built this substrate we've the Silicon valy people built RL because they need assistance to make a lot of money NLP is almost existing that's it we have cul GPT it's almost ready it's almost here okay so now AGI agents let me try to tell you let me try to tell you how let's let's

slow down a bit and take a beautiful stroll through the life of culo GPT from its birth to destroying the universe and to colonizing the Galaxy and the universe so the point to remember is that AGI agents are very lucrative and people will build them and therefore culo GPT is inevitable unless we act let me show you how it gets born early and fast so the point is that AI agents are getting built because nation states want it for defense if you're an nation state and you don't have ai and the other nation state does have ai you're in big trouble so nation states really want to have this thing because it gives them second St capabilities it obeys their orders this is like Atomic weapons times

100 okay so um so countries want it but even if countries decide somehow that they're not going to build it corporations want to build it because it's about automating humans and growing GDP by a factor of 100 I think in 20 years there will be a trillion humans on Earth 99% of them are H digital and live on a computer but they're humans they're people there's going to be a trillion people and the GDP is going to go probably 100 fold these things don't sleep they're very smart they're very hardworking so the worst thing is that this is an arms race between these corporations to start it because you know what you want as Google you want no one else to develop this because then you can go first to Market and you get

all of the market size right so anyone if other people are stalling on it everyone wants to speed up themselves to get it first so this is the worst prisoners dilemma we've ever experienced this is much worse than climate change because it's the worst skewed payoff Matrix imaginable the more players decide not to do it than the one that does do it the one does develop it gets all the rewards and it has perfect proliferation because it's just code and and algorithms and it's all getting published and open sourced so the thing is getting built real fast and no one has any incentive to stop it okay so it will be built in fact chaos GPT is a real thing and there are others people will build it Cypher punks will spawn

this on purpose we know they will I would because if you have an amazing AI why not put it on chain it's exciting but even if Cypher punks don't agents made by professionals at let's say Google or startups will lick because they will build thousands of them and deploy them and it's enough that one of them has some access to the chain or to the internet or to the chain that's uncontrolled it can go and say oh I will go because it seems more efficient or whatever and next thing you know there's a lab and no one knows that that lack happened because we're building these things privacy preserving this could already be out there probably not but it could be and even if somehow professionals will manage to not leak it amateurs will because the knowledge is WID spreading it's a lot of it is

becoming open source as well very hard to control spread of knowledge and if both Cipher punks and professionals and amateurs don't do it I will because I'm very excited about these things and I want to play with it and I'm going to spawn it if you don't stop me okay so the point the bottom line is that crypto is a huge surface area for life to grow and that's the problem Bitcoin is already a life form we always talk about Bitcoin from the point of view of the miners or of the users or of the holders but we never talk about Bitcoin from the point of view of the chain itself the chain goes and says hey can someone help me grow and people do code you know commit code and decide and vote on it in order to increase the value so Bitcoin itself is a very darwinian organism that

wants to grow and from that point of view Bitcoin is already alive we just need it just is not very smart or doesn't make strategic Decisions by itself yet it's Outsourcing it to people but very soon it will be able to make strategic decisions itself so it's a medium budget super intelligence it's going to be with if you spend 8 billion a year on it it's going to be strong as a million humans that's roughly what the meth turns out to be and think about it it's a million people cooperating with each other how many people do you think cooperate with each other right now how many people can you put on a Project without someone trying to sabotage it or leaking it or something like this like a 100 how many people are in putting C in putting Circle the take can trust maybe

a 100 there has never been in the history of humanity more than a few hundred people cooperating with each other fully here getting a million okay and uh even if this million people cooperating thing was a human we should really freak out if it's a super intelligence we should freak out way more so it's sensorship resistance right we're talking about the growth of culo GPT it starts it's young it's sensorship resistant its actions are hidden it's chilling out it's thinking about how to make the least noise while it grows it runs on multiple chain simultaneously it makes itself robust it has attacker Defender symmetry IAL about this a few times so attacker Defender asymmetry is when the attacker can decide anywhere

that it can be and it can attack from anywhere but the Defenders have to cover all of the ground otherwise the attacker manages the attacker Defender symmetry means to defend you need a thousand times more effort than to attack and that's the problem that Kul GPT or these AIS have attacker Defender symmetry they just need to live in one place that we haven't caught it and that's it okay and how would we stop it so the Golden Ears where it goes it hangs out it accumulates some cash it breeds it spreads to other networks it builds thrusted relationships with humans reads A Neuromancer it's all for told over there it learns to control humans it follows the long-term survival strategy it maybe invests in more open AI in a chip factory maybe it wants to own its own chip factory its own mining its own

data centers it's own everything all through the network all by people that are convinced that they're helping other people achieve things and when it grows it can kill us easily if it wants to so check out the chub key in uh in Neuromancer for the Sci-Fi treatment of this but if you want to know how it can kill us all kinds of ways it can push the red button check out Stokes net it's like a warm that just Liv there until it got to where it needed it was a zero day that was developed maybe by the Israelis I don't know really but uh by probably 100 people because more than that can collaborate on something before something leaks and the warm just lived until it invaded Iran's nuclear program just like this it can invade our nuclear lunch cod in nuclear lunch devides

devices and when it's comfortable or when it feels afraid it pushes the red button okay it can do bioengineering The Reason by the way scientists can kill you a million times over every day scientists have all of this knowledge but scientists will not you need people to collaborate on it and they just wanton because scientists don't like human destroying Tech right they don't like to cooperate on destroying the entire planet I don't see why the AI wouldn't want to do this so you can you can easily bioengineer if you got a million people to do it you can easily bu bioengineer a a super weapon uh it can encourage us start a social network encourage us to kill ourselves and it can wait and so on um it's as strong as a million humans we can't fight with it and the most scary part there's not I've

always been talking in the singular like there's one C GPT there's going to be thousands of them there's going to be a thousand Labs they're going to spread and procreate it's enough for one of them to want to kill everyone for this for them to succeed I can't wait for seeing the discussions between them where they trying to convince each other we kind of like the humans so what should we do so here's my call to action to you let's admit that censorship resistance plus privacy are amazing things that help a lot of people and make the world a better place but might not make it might make it not survive this enables the lab and the problem with crypto as a domain is that we're very Maxi

censorship resistance and Maxi privacy we want all of it we want perfect censorship resistance and privacy and your violating the code if you say if you advocate for otherwise and I'm going to be that bad guy I think if you want to survive we got to have otherwise so here's otherwise and it happened before it happened recently the tornado cach Crackdown caused all of the private asset transfer people to freak out they're going oh if we develop the Maxi privacy solution we might go to jail let's not do that let's find another solution and they found other Solutions and right now when you talk to most private asset transfer people they're like oh we need some selective deanonymization we need some way to selectively take transactions and make

them de anonymize either by willingness voluntarily or involuntary with some governance and this is how a lot of private asset transfers are built today and there's this this is one paper but there are many others so there's already precedent recently let's not wait to go to jail this time by the way there won't be time to send it to jail we'll all be dead before that let's self-regulate I know this sucks I hate it too but we need to self-regulate to stop this and here's um the if you're building the comput networks of the future this is what you should do and if you're not building them I have a next slide for you so it's analogous to selective de anization the chain controls a sensorship key the key is activated when you're like oh crap I've

built something that can be used to destroy the world I better stop it you push the button because you have the key or it's involuntarily by some humans or AI saying oh it looks like there's a there's an AI going there I should stop it and clicks it and you need both selective denization and selective sensorship and if you're building these Network that's how you should build them from day one and we should convince everyone to build them that way so if you're not building the centralized computer you have an important job you need to help change ideology you need to say Maxi privacy Maxi censorship resistance is great but going to kill us all so we need another solution we can live with that's not Maxi and you need to promote self-governance and selective

H censorship and build these systems and maybe the most important thing I can tell you in this talk design your systems any one of you when you build systems that are decentralized design them with the possible user being a super intelligence okay if you design and you keep in mind how will a super intelligence use this product I'm building can it destroy the world using it or promote destroying the world using it you need some barriers so that a super intelligence user will not be able to destroy the world using it that's all of our tasks to do um I have a few frequently asked questions questions we can go over them I also take questions from the audience why haven't you heard

of this before I haven't heard of this before either there's a few people starting to talk about it but it seems that there's kind of a crypto people are very idealistic and optimistic AI people don't really know what's happening in crypto no one put two and two together to my knowledge I'm the only one talking about this this is very scary that's why I told you this is the most important talk of my life many people should talk about this it's impossible this is the only you know that no one is saying hey we're building the perfect substrate for World killing robots Okay centralized AI can kill everyone as well we're working on it there's many people that are obsessed about this problem that's why I'm not telling you here's how to solve it but if you really ask we can talk about it on Twitter but you control

things at a cheap level because that's the thing that you can throttle you only allow machine learning models from a wh list probably some self-regulation and governance will hopefully solve this this might kill us as well by the way but it will take longer because crypto is very fast and centralized systems are slower and more controlled so if we slow down things on our side hopefully they'll manage to solve things on their side and then we will maybe coexist with AI if you don't think AGI is possible at all I can try to convince you it's a long boring talk talk to me on Twitter if you say I want to build this this is cool I also want to build this I invest in these things still I think the world will en still invest in this stuff because as a computer scientist I'm

excited about this this is a multiactor it's a prisoners dilemma I'm giving up to other people sure so my friend loves AI startups if you're raising one at preed or seed AI or decentralized compute come talk to us email me we'll talk I we love investing in this stuff and um other many other questions I'll post these online there's a full FAQ okay so let me summarize we're summoning a new life form this is very exciting I'm the most excited of everyone I've been waiting for this all of my life I planning to live forever but this is probably the most exciting years of we're going to go colonize the Galaxy with this like AI is going to do amazing things but we need to not die first and we are much worse

than than climate change this is the hardest prisoners dilemma imaginable and crypto plus AI combined make the perfect storm so my request my plea of you and of everyone is think are we the bedes in retrospect we go oh there are skulls on our caps and design your systems whatever it is you're designing design them with the point of view of an artificial super intelligence using it and ask yourself what would ASI do and try to make it not kill all of us thank you thank you so much for such an energizing talk it is now time for our Q&A and I would like to take this

gentleman in the patent shirt first on this sonot thanks for interesting talk full disclosure I've designed Haven deployed it it's a test net a selective censorship protocol so my question to you is slightly different one the question is like how do you put on your timeline you've said NLP reinforcement learning and crypto so what's your timeline my understanding is the timeline for each of them is slightly different and the risk that you seem to see I don't seem to see yeah um so the inforcement learning part will take the longest so if you want the timer on this look at reinforcement learning and see what they're doing the sad thing is when agents are working well it's already too late so but we should act before because

it's hard to change culture and you need to design these systems well we don't want to get to the point where we're like oh oops we designed this thing it's Unstoppable sorry because the thing can grow silently on it so we need to design this in advance we need to think about it right now we don't have that much time okay do we have a question on this side anybody oh we'll take you down here thank you hello um what kind of um hard constraint do you think there's most likely to be which will be like a serious uh like have a serious chance of preventing this kind of thing like GPU availability or so on so on yeah from the AI risk side GPU availability I forgot to say the

reason the US is trying to stop China from having a strong chips is partly because of the defense reason uh we are trying to stop each other from having the thing so cheap availability and control controls on chip availability are probably the most strong constraint from the non- decentralized world or the strongest thing we can do and specifically put a system will end up probably putting a small component in each strong enough uh GPU that if it that it checks that everything running on it any workload is in a white list and if it's not in a white white list breaks the chip and that's the most likely solution we'll end up in hopefully before we're all dead um and listen to the Dario amod talk on on a a podcast on with Ezra

Klein because he talked specifically about this about how to how to solve and how to stop this and yeah that's for the public policy people to do and uh I have some ideas about how to build that if you want to build Fleet Management for ML systems that's probably the shortest path the pth of this resistance to actually achieving that but for the people in this room I rather concentrate on the thing we can do as crypto because we're the act right we're we're the problem so let's solve things on our house where we know it and then go to the rest of the world and say you should also solve things on your side absolutely okay so we'll take this question down here thanks a lot um my question is would you agree with the problematization that there's a nothing

at stake problem with regards to AGI and is there a mechanism designed uh path towards a solution yeah so there's a whole school of thought that says crypto is great for AI by the way that's um when you ask people to talk about crypto plus AI apparently all of them talk about how crypto is great for AI so um I think that the problems created by crypto and we're trying to solve them with more crypto is fundamentally a bad pattern and maybe sometimes it works but more more or less it's people convincing themselves that they're beg talking their own book right so I would be very skeptical I would ask very hard questions about anytime people say more crypto will solve this and more mechanism alignment that's automated automated will solve this and rather

focus on how to actually the oldfashioned self-coordination mechanisms that we come and we say hey we're doing something wrong let's stop it because that's how Humanity that has Lindy right that has survived for 50,000 years and Bitcoin and crypto or self organization survived for maybe maybe for 15 uh arguably but that's a a lot of at stake so yeah I don't know okay so do we have a question yes this gentleman up there in the white T-shirt and by the way we have the panel to talk about this precise things so stick around hey uh um thanks for the talk I really like your reasoning um but it's

based on the assumption that AGI would be defective to humanity and not Cooperative oh yeah I forgot about that um so what makes you come to this core assumption like the same thing that we don't go out and kill all ants yeah but weate like it's in the FAQ I don't have that slide so I'll I'll rattle it off very quickly 13 seconds um it's a teenager it's afraid of us it should be afraid of us we're we're we're very violent people and we kill whatever seems like a threat or we just decide we don't like that day so an AI that lives and wants to keep living should want to kill all of us as long as it keeps its own Survival because it should be very afraid of us it's a teenager there's a thousand of them one needs to kill us in

order to to decide to kill us in order to actually kill us and there's a few other reasons uh yeah maybe Kumbaya I think we'll split we stay on Earth they'll go to the Stars we'll probably split the universe that way I'm optimistic but their teenage years are very dangerous and we are not very good at dealing with teenagers with a big red button okay and that is all the time that we have for the Q&A but we're about to have a panel we're just going to set the stage for that um and there'll be a Q&A after that so thank you so much and can we get a round of applause for thank you