ZKPodcast: P2P Messaging & Scuttlebutt with Dominic Tarr

Transcript

[Music] welcome to zero knowledge a podcast where we explore the latest in blockchain technology and the decentralized web the show is hosted by me Anna and me Fredrik [Music] today we'll sit down with Dominic Tarr to talk about p2p networking and secure scuttlebutt [Music] so today we're sitting with Dominic Tarr

who's a security auditor at least Authority and also a protocol designer he works on scuttlebutt which is a secure gossip protocol so welcome to the show Dominic Hana we're also here with Fredrik hello so we've started to ask guests who come to the show a little bit about like why they're working on what they're working on so I'm gonna throw that your way why do you work on what you work on well once upon a time this may be this is something like about 10 years ago I was living in in the forest I was like living in a hammock being happy basically and after like a year of that I decided this was too easy I needed to do something like challenging and I had

two ideas and one was to build a boat and sail it to Southeast Asia and then see how that wind and the other idea was to do this like computer stuff and I had some vague ideas and of like something I could build and I decided that by the time I was 35 now I'm 36 if I did the boat thing I'd look back and I'd wonder if those computer ideas would have worked but the other way around I was like I could definitely still go sailing if I wanted to and the computer thing was obviously harder so I I decided to do that how old were you when that happen 25 or 26 or something like that so we want to sit down and talk today about p2p networking it's something that we've talked a lot about on this show before

but in passing almost like it's a an assumption that we have that people just know what peer-to-peer networking is and how it works and I don't think that's necessarily true so I want to explain some of the basics but I also want to dig into some of the tricky topics of this area and you have obviously some some peer-to-peer network background with Scuttlebutt's so maybe just as a brief starter like what is scuttle but what is scuttle bots and you know what are these different projects and sort of how do they do networking well scuttlebutt is the protocol a scuttle butt is it's just named that's

like that's the server that runs the protocol so it's just it's just like that isn't really the name of the project that's just name of like one particular good report it's called that for like a historical big historical accident so don't worry that's gotta botsko part that's the important thing but I think for developers it's still like if you want to learn like a lot of people learn a protocol by reading through the code yeah so if you want to do that and skull bot is the right place to look or is there better place it's now called EC speed - server all right but this we have a lot of code it's all on the github /s s PC that sends for secure scuttlebutt consortium going back to that the where to scuttle what come from where does it start yes sir scuttlebutt

is a nautical term and for for gossip um a scuttle means open or opened and but is a barrel so it's the opened barrel of today's drinking water so it's like a water cooler but on earth you know pirate ship and then what what is the history of this girl but protocol well I so I actually got the term wrong with the word scuttlebutt from an Amazon paper this is sort of what led me to working on peer-to-peer stuff so I read about I read this paper called dynamodb which is a system Amazon built to build the shopping cart and stuff like there DynamoDB is basically it's a

peer-to-peer protocol inside of their data center so the architecture is quite analogous to how BitTorrent works they have cluster of neurons that are part of the network and they all gossip with each other about just to keep track of the information of one of the nodes and what are their IP addresses that are in the network and then each node it takes responsibility for part of the data set it's pretty much exactly like a DHT yeah I would highly recommend to read the dynamo paper um it's very clearly written and covers a whole range of concepts so that was the front that was a paper that you've stumbled upon we'll add that to the show no it was recommended to me oh it was recommended to you and then what happened from there

so basically scuttlebutt the protocol and Amazon was just like I implement it I implemented a thing based on the ideas like there and then I let mr. round with a whole bunch of other stuff I got to the point where had learned enough about data replication and data bases and a bit of cryptography and stuff that I wanted to build some kind of generic application platform like just saw there's some sort of system that you could build wide range of applications on on top of that were all decentralized and I was originally thinking like a Tova target use cases sort of more like Wikipedia because you know Wikipedia feels like it's really like one of the most positive things on the Internet I searched was trying to explore ways of

doing like generic replication and I sort of came to the conclusion that it was much more you could much more efficiently replicate like structured data then just arbitrary data and the simplest structure I knew was an append-only log one day I realized then um it's social networks basically mapped to this and people like using social networks even they probably liked it too much but that gives you like a story about like this is healthy you know this is you could build this it will grow it because people are addicted to using this kind of thing you know like the whole decade before blockchain became a thing like every application was just a

social media application so um you could build all of them using this approach so you scrapped the Wikipedia yeah I did I was just like put that back and put that back in the pile I'll just I'll just focus on like the simple thing that still is 85% interesting so like you actually had a use case right off the bat like or do the use case come later um well no it was like the the breakthrough was realizing that you could go go social media thing on it because one of the things I was like I was thinking it's like if you have this distant fly system you need some way of discovering the stuff it's on it I want to get back to scuttlebutt a little bit later and dig into some more

specifics there but let's take a step back and look a little bit at networking and explain some concepts so you already mentioned some of these things let's start at a DHT what is a DHT what does it stand for DHT sends for distributed hash table so it you can add EHT works by you get all the nodes in the network form into a circle and then content of the network is distributed around the circle evenly so if you keep track of a couple of notes randomly scattered around the circle when you want to look for something you go and ask the node that's nearest piece of content you're looking for and then they tell you either they give it to you if they have it or they

know about another that's nearer to it so they tell you about that node they say go ask that guy and this is like this has been around for a long time this is back when like you were doing kind of peer-to-peer downloading yeah in that time fence the I'm not sure exactly so disclaimer I am a DHT skeptic but I think they sort of came around and like around 2000s yeah the there's a bunch of problems which I'd be careful with dhts that I'd be happy to explain but I think it's like BitTorrent came out which was like 2005 or something if I recall didn't use DHT at the start but got ditched eased a little well there was something called cord I think was the first one that went up in the early

2000s and then condemned Leah was that improvement on net which is analysis was there do you know if like standard Gnutella used it I believe didn't until it didn't have a dad date okay sounds sort of pre yeah THD so you you mentioned Kadima lee out there orchid Emily I depending on I've never heard what the correct pronunciation is I always say Kadeem Leah myself but might be condemned alia I have no idea but Kadhim Leah I would associate with a discovery protocol mm-hm and DTS and discovery is very tightly linked but they're not necessarily the same like you can use a discovery protocol just to find one pair and pain and you know do something if you want to do that yet obviously they're kind of

usually used to build the DHD or you you could use AG DHT for discovery but you could use it for other things as well so what would you how do you explain a discovery protocol what's the purpose well in this case we're just looking for you mean discoveries to find other peers right yeah yeah so any system this usually appear is like let's let's start by defining what appears there's a good seller in a peer-to-peer protocol basically you've got a system a bunch of there's a bunch of computers in it and the thing that makes the difference between a peer-to-peer protocol and a server client protocol is that in the server client protocol is like a specialized role and one computer has a

and it's the server and then the other ones the clients but connect to that and then the server does some one important thing and the plants to some other less important thing or something like this and a peer-to-peer am system all the clients are roughly equal they more or less have the same capabilities and can replace each other so there's no central hub it's not like what was it called hub-and-spoke I think it's the term right when there's like a hub and then it the client and server Joe if the server is the hub and client service folks in this case it's more like a mesh yes there's different shapes that a network can be in okay for example you can have like a like there's a ring check network like at the edge T you could have like a lattice all that grid

techne works so one example of a of a lattice seven it workers actually cell phone tellers I arranged in a hexagonal lattice they have like directed like you know the antennas like to shoot some frequencies that way okay I'm just depth I'm just doing like a triangular shape and then they arranged so they don't overlap so they don't collide with each other and hexagonal lattice means that circular radiations of radio waves have minimal overlap it doesn't really make sense to have um like a lattice sort of network when it's just like computer connections over TCP but another really good structure is actually just random las' like if peers are just connected randomly to each other this is like it's

really good design because PS don't need to be clever first of all it's just like really easy to just connect randomly if you have a new design that works with random properties then it'll probably go no is probably gonna work like you don't really need so it's just easy to ensure it works and if noted slight drop out crash and that sort of stuff it'll still work very very resilient so but in that scenario like if I'm just connected to random Pierce and I want to send a message and I'm not a random person how do I find her like how does my message make it and they're well you've you've you're assuming that you want to build some kind of routing thing so there's

these various ways that people have designed systems to to do this kind of thing yeah I have avoided routing designs because I don't think that the best basis for the for the first layer because you'd need to basically you can't route to directly to entity so if you could if he knew that route and it was online right now and you knew how to contact her you could connect through you could just make a connection except in there's other things that throw spanis in the works like there aren't enough IP IP addresses left so there's a shortage of there's only four billion IP addresses and there's more devices than there now

it basically means they're like imagine if there weren't enough telephone numbers and some people had the same telephone number it would mean they're calling you can't really call those people reliably and so imagine a world where your two types of tufa telephones more expensive telephones that could receive phone calls and then cheap telephones that couldn't receive calls but could dial calls and so businesses would buy the ones that could receive calls and retail people would buy the ones that couldn't receive calls and this is held this is the situation with the internet you can't really call your friends except by calling a service for like introducing your friends that could be fixed with

ipv6 um however it's still been like 20 years and we're still waiting for ipv6 to be fully rolled out so I'm just assuming that it's not going to happen but let's go back to this idea because I think we you sort of said send something to me and I think we went in the path of routing yeah I'm actually curious or like I want to dig into two aspects of that question one is is the discovery part and what what what are we trying to discover in the discovery protocol are we trying to discover random pairs for anyone at all to connect to or are we trying to discover any specific pair well I mean I'm trying to answer why why I sir had why there wasn't just a simple answer because if there was I would

exist because they 24 one wants yeah but the other thing in you mentioned this already as well and we've actually talked we've explained this briefly on the show before which is gossiping so the dumb ants are a kind of how I get my message and eyes I send it to everyone yes and they send it to everyone yeah and then hopefully gets there oh that would definitely work this is a very high chance that they will work but then there'd be a lot of gossip out there yes and I just have to go through all the gossip to find out what yes actually say yeah me yeah on if there's some sort of ways of like limiting health things spread this would totally work very simple very resilient there are protocols that are for routing but because of this shortage of phone

numbers you need to have a server to introduce you they're making a fully peer-to-peer connection nowadays is more like being set up on a date than just calling someone that you need a third party involved ok so then is it still really peer to peer well it depends on how central that third party is in many cases that third party is just your router so like I can connect directly to your router but that assumes that your ISP isn't like interfering at some point where in many countries the ISP controls the IP yeah and then you're getting some subnets well there can be multiple layers of net before you get to your thing so it can be more difficult list

of cool it depends and if you are at the same like plight if you're on the same like Wi-Fi you can just meet directly because you're on you can just see each other's IP addresses basically back to what you just said it was sort of like it's sort of if if there's a third party sort of depends on how kind of decentralized you need it to be because that third party could be one of ten or one of 100 or one of a thousand or it's like up here as well who has a different role well I think it just depends on um who has the power to run those third parties and what their motivation is so for example if you know your application has one of these introduces that the developers of it run that's not very decentralized but if

anyone who's enthusiastic about the protocol can run one of these introduces now let's say that is decentralized it's just you have some nodes that do a little bit more some protocols everything called a super node and we'll you have special nodes there do extra stuff it's just you know it's this picture makes a little bit less centralized but you know it doesn't it's not a deal breaker I think in theirs when you're speaking about routing there's an additional problem other than Nats which is routing in itself as an np-hard problem just finding a path to someone is like the Traveling Salesman problem how do you find the shortest path from where you are to where you want to go through a set of other nodes

when it's not mapped for you before even if you have the map it's it's why is so hard well that's finding the optimal path you're finding like an adequate path as nice as Israel yeah okay so yeah exactly and what is an adequate path and and all of these issues are pretty tricky to deal with and like figuring out their prime like the correct parameters and like this is something that like lightning Network has been struggling with a lot of like how do I find where I'm sending my money and they have to do routing because the other person the receiver actually has to sign off on something and send it back and so they're kind of going with this hublin spoke model I'm just assuming there will be some kind of centralized entity that

it doesn't have any power other than the ability to censor you but that's still quite a lot of power and then you have like super node structures where there's like maybe 10 super notes and not just one and then you can kind of gradually build up other maps but yeah so I think the the smart thing to do here isn't try sir um you know squeeze a routing protocol out of a stone but just to avoid the hard problems just like accepts that there are some things that there's an easy solution to try to design systems that just avoid that and so for example one problem that you haven't mentioned yet but as one of the things that is like

the blocker that enables a bunch of bunch of stuff is how do you map a public key to appear this is the thing it's like if you know what public key and represents a particular person then you can establish your keys secure communication with them but if you don't know then you really have no idea if that you're talking to the right person so an encrypted channel is kind of meaningless unless you know that this is the key you need to be using this is one of the problems that's like once we solve this it was suddenly become easy to do things like routing I mean in that what you just described then it's also like identity right it's like I'm actually an identity to some number yep and then I mean there's lots of projects that are working on that but what's the

real challenge of that like what is the what's is it just because there's kind of no way to do without like some centralized database I know I disagree I think that so I have my own horse in this race okay so I think the the challenges really just make a system that people use and that people you know use it in the way they like that they don't really think about it just like I'm now I'm talking to this person that's who I want to talk to but how do you make it like the question there is like how do how do you you just sort of said this is like the big problem that still hasn't been solved yeah so so once upon a time there was this thing called PGP pretty good privacy and it used this idea called the web of

trust so this was like back in the early nineties it was it was a big deal back then now everyone pretty much agrees including its created that it's like horribly unusable and you would go to this as actually people actually do this they'd go to these key signing parties where they go and they would each other be like yes you are really the ski and then they would sign each other's keys and then that would go on a record that this person that not was that person and then you could be like if you trusted this person that well they said these people were I think so you could kind of figure out who was map to her it was like very decentralized and very time-intensive yeah and I've been led to believe that some of the key signing parties were attached to actual parties and some may not have been I'm

not really sure I've never been to a key signing party there's one step away from like wizard hats yeah yeah I think so I mean not one step away at all but anyway it doesn't to me I like that idea seemed like it was basically sound but doesn't really sound very fun to me and when I camped the to this space it was really like post we have 2.0 and so we've trooper and we figured out a whole bunch like a playbook basically of how you get people to use your application and the first thing really is you make it fun and you don't ask people to think about it you just like give people buttons to push to click on and get the feedback from other users and stuff like this and

you go for like viral growth and that sort of stuff but the thing with web 2.0 is you can interpret social networks and stuff created by web 2.0 you can interpret them as a we go trust the users don't think about don't explicitly think about it we're not using it but if you follow three hundred people on Twitter and then there's like 300 million people that you don't follow so you've said very explicitly like these are the people I trust all those other people I'm not saying anything about them maybe I explicitly just trust them but it's like you over like if I say I'm willing to receive whatever these people post you can interpret this as we as a

weber trust and interestingly it's more about lots of small interactions in one big key signature thing so you can also look at it look at the sort of thing is like you know we spend a lot of time nowadays interacting with entities with other things that as messages over the internet like I got invited this podcast by email from people I've never met um I read the email and then decided ah this is a real human we are introduced yeah we were introduced exactly so that's a wave of trust it was easy because I knew that list wasn't just in a long con yeah it's like now you're gonna harvest my kidneys [Laughter]

yeah that's interesting yeah so like that idea of like the social network as already kind of having this web of trust and the friendship stuff like that was all that's so far has been built over like centralized systems databases which is held by single entities I really like key bases approach to this which is they build a web of trust and but like you go on it's still centralized but you go on to there a platform and you look at a user and then you can see this user has verified like with a cryptographic signature that this is my github accounts this is my Twitter accounts this is my Bitcoin address this is whatever and you can add a bunch of

different of these like identities to your accounts and so I can go and check someone's like this person hasn't verified anything anywhere I think they're trying to scam me or like this person has verified their get up in Twitter a night and night and know them there so I know this is the same person yeah and therefore I can like follow them and trust them on this platform as well so you kind of you can merge that trust from other platforms onto key base yeah but here what you're also describing is like then taking that the next step in saying like well thus off the sort of ideas are there for like a truly kind of decentralized peer-to-peer thing I'm relying in a way on this is it relying on a web of trust not really well you just need to know that this agency is this does kay is this idea is

this identity deliver trust is just one way of doing it there is just kind of very close to what humans already do and that's why I sort of make sense then that's why you would then choose something like a social network you know where their needs or people are trusting each and then create that thing underneath yeah so you're kind of yes so you're basically taking an idea this is interesting because like it comes really from web 2 and then you're like bring it back to the decentralized direction yeah because the the they like user experience I'm using Twitter comm is decentralized like from that you look at the thing and you only see the people that you've followed and the idiots that retweeted and stuff like that um not

everyone doesn't see the same twitter.com they obviously different ones that it's centralized as an implementation detail that causes a lot of problems and I'm sure that when the way of employing to this in a peer-to-peer way is like all the details are really ironed out then people will be like why did he even don't even do it that way like if I have a whole data center and a big building in San Francisco and millions of dollars on like then again I mean there have been attempts Matt what is it master Don yeah but it sounds like federated yeah Master Don only gets off with you okay but I think the problem of it isn't that it's federated the problem is that is it's

not adopted as much and there's some issues of like replication of identity that's like your Federation issue so in a completely decentralized way in your controlling your own identity I think there's less issues of that actually but to your point it's still about adoption and network effects Twitter is popular because Twitter is popular there's no other reason yeah and you know how do you win over network effects and no one really knows like there's research that says that no network effects suddenly died for no apparent reason yeah so that's that's like yeah exactly yeah yeah I mean I think it doesn't really it's not something we have to like so

one goes it doesn't come out of no way it's not just like one day your friend decided to use Facebook it sort of built up from a smaller sad when a nation slowly expanded but you just have to find like who really really wants this and then who can you again who can expand out so it doesn't have to like nothing goes 100% from zero I want to go back a little bit too you mentioned a lot of problems with D H DS and I want to dig into that yeah but so the unlike peer discovery in particular especially in this sort of untrusted setup dhts and peer discovery remain an issue like DHT czar attackable yeah you

can fill a DHT with garbage data and it becomes impossible to find wrote like valid nodes has a bunch of these problems you know the way that block chains like all block chains that I know of solvents is just have boot nodes so B's yeah maybe we can start by explaining what boot nodes are and then like what are these problems how do we work around them just does SSP do anything around this that helps you just have a bootstrap nerd you just have a list of neurons those are built into the software that's like when you want to start the knee where you connect to one of these nodes and there will always be a real node yeah this is basically like verified account yeah yeah you could like if someone compromises those notes you can you could like reconfigure

different nodes or something like this it's just really to help things get started it is a like mild point of centralization but doesn't ruin the whole thing and then they'll tell you about some other peers and then you connect to those peers and yeah but the the problem the problems with deities I think stem from the fact that all of the members and the network all of the peers in the network interact in a very uniform way and they don't have any way of deciding who not interact with they just like they just have to execute the same protocol anyone who can next to them and it means they have no way of like avoiding malicious peers or even identifying or telling if appear as

malicious so that means so you know someone wants to come in and scan the network and see if it is holding what or something I can easily do that if someone wants to make a peer that is like one this is called an eclipse attack you you generate a peer it looks like it it might take a little bit of effort but it ends up standing in front of the peer that's actually meant to be holding that content and then you block they can turn or something it's not like a hugely effective attack it's just like an annoyance or sort of a denial of service sort of thing but well so once you get into blockchains it can be a pretty significant attack yeah because now suddenly you have one pair that's holding money or expecting to see some behavior or if it's a mining note

and you eclipse it then they're not there blocks aren't being propagated or whatever it may be so you can actually cause some real damage yeah yeah and and blockchain made this made security computer security really interesting because now there's like an easy to reason about financial value on like everything now there's like a dollar value for like how much hashing power it takes to like reverse a password or something like that like you can say that well if you could get you know these secondhand Bitcoin miners and use that to break passwords then you can actually say exactly how much their costs attacks that you know hacking into someone's machine or something that would have been pointless like you could have done it but already in pointless now if the

answer is are well you just steal the Bitcoin it's like suddenly all of the some of these hacks become monetizable so it's created a whole new set of opportunities so just to go back to what you said dhts you explain some of the issues around it but do boot nodes fix for that or they just sort of like that's that's a reset button no I'm boot nards just provide a way to get started and the network to join the network you normally need to know some other peer and the network and that they know you perhaps so but notice is a way where it's just like here's a fixed list of things if it you could have another

thing where it's like your friend invites you or you have to go to something or something depending on your use case there may also be a fallback for if you find a DHT that's awfully polluted and you can't find a real node in it you can fall back to the boot node and say okay I'm just gonna sing from you and not talk to these other people but then you're like if you're dc's that polluted you're running into a bunch of other problems if there are examples of DHT that are still used today oh yeah not know like plus mice is going to theorem and that would be considered a DHT but bottarga I think the beasts but I mean theorem has a ditch taper I don't really I don't really think it needs one like

the first blockchains have a thing where there's one blockchain and everyone has copy of it so any full node as as good as any other I think you could get by with just gossip protocol I know a theorem has a DHT last I checked the coin didn't have one oh maybe it doesn't I'm not sure the theorem for sure does yeah that's gonna be finding on doing other stuff with it like I want to get to this a little bit but like announcing like in the DHT you can store data you know and so they want to store like peer information in the DHT itself so I speak this protocol or I'm willing to sort of like clients and yeah there's things like that where like where does the DHT live in the kind of stack it's a very

low level networking like before you're really talking about blockchain stuff just in how you deal with peers so like under the EVM yeah yeah yeah okay I can picture what it looks like it's but it's nothing to do with like the block headers and the hashes around that I okay so each peer has a location and in a ring taking it to a couple of years that are near them a few more peers that are a little distance from them and some more peers that are like a long way from them so that if someone asks them oh do you know where this thing is then you can direct them to either like close or like at least moderately close or like

kind of like and if they're like far away you can say like go somewhere like way the hell over there and then they can continue asking and eventually they find the place so does SSB have @hd or what do you do I did SSB does not have a DHT I always explicitly intended to avoid dhts and SSB because of the aforementioned problems and I felt like everyone sort of like puts the ditch teams there protocol is like the hammer that solves every problem and I wanted to prove that you can build a totally viable protocol without DHT zurna so skuttlebutt uses a different kind of structure

like organize the content and organize the peers and stuff like that and that's just that given social network and this is also like a globally connected graph we're well pretty much close enough for globally connected it has some entry it has some interesting statistical properties we're like you know some people have lots of friends or last celebrities and so there that branches like long distances like you have friends and friends of friends and most of the people that you want to talk to are actually your friends or friends or your friends really like when is the last time you meet someone it wasn't a friend of a friend like sometimes you meet people purely randomly and that's like when you bump into them in person s

or the server or invade them to your podcast but this was actually a friend of a friend invitation so complete strangers it yes sometimes I mean sometimes you do bump into completely random people but you also walk past hundreds of people every day on the street and you don't talk to them when when those are completely random meeting that's like a special you know it's some kind of like cosmic theist or anything like it's not that's not the typical case so it's a bookcase is like a friend of a friend so I'm curious yeah that that's interesting so instead of having this kind of constructed network topology the network topology and in a

sense is determined by our solar social rough but that means this social graph would have to be public right yeah so far I saw I have I have ideas but I like higher to this better but so far I mean we're experimenting um we don't really but now we've learned quite a bit but we're trying a totally new thing I could become another podcast and talk about all the ways you could you could - you could make this more private and you know I think a big way that history technology and stuff progresses as one person does a thing and then someone later comes along and is like well that's stupid and makes it even makes it what makes a way better version of that

so I'm hoping someone will come along and say that scuttlebutt was stupid and they make something much better they'll be great because I won't have to maintain it zooming it is public right now I it doesn't feel like like you say you know where to really talk to a stranger so if you if you hid like you just had your own social graph and kind of cut it at some arbitrary limits and then to beat go beyond that limits you need some sort of introduction yes yeah yeah that'd be fine yeah does SSP use boot nodes then do you have some sort of like early list that you start from kind of so we have a thing called a pub server which is it's the

same code as an ordinary peer its pub essential for public but also pub isn't like an English let public house like a bar where it's like a place you can go there is always the same place and you can come like any time of the day and you've even your friends might be there pubs are basically like anyone can just run one and it's exactly the same code as an ordinary yeah it just you run it on with a public address so people can always connect to it and anyone can run one of these pubs and if they run a pub they can give out invite cards and and then people can use those to come into the network basically they're the British Reptoids but we don't have an official list of dim and built into the

protocol it's like you have to go and get one from somewhere and you can be imported from another peer using the protocol or from one of these so it's a little bit a little bit more democratized than just like built into the thing but basically the same idea what is what does somebody do in a pub or at a pub is not really and alas it's just like another peer just replicates me guys so when a pub follows you they rip start replicating your messages and your friends messages and so if you connect to a pub that follows your friend then you can get their messages from them so that was helped work so far we've got a new redesign there and because some people were a bit confused by what the pubs did and so the new idea is to change it so the pub is just

represented as like another device that human ions so you know my head of a laptop and a cell phone people familiar with people to having both the laptop and cellphone but and some people also have a server and that's just another computer that they aren't uh more control and if you have a computer if you have the server then basically what would happen is you would just copy all of the stuff that you have and all of the friends and everybody who's yeah yeah that becomes then like a it's good but in this case are you talking about data you actually talked about like replication of data or you're still talking about replication of peer addresses replication of data okay because scuttlebutt uses gossip it doesn't actually need routing as long as

you connect to a mutual friend Mike if it doesn't need to connect directly to you you need to get my messages but you can get them from me someone else and they could bump around like any number of mutual friends until and that's what you do make data like it's it's there it's sort of you send some data and it sits on the friend and then yeah I can access it if I want to yeah I'm curious if you've looked into gossip sub at all the new sort of protocol where yes II think so i mean they're they're working on it but i don't know if they actually like invented it but basically where instead of gossiping everything to everyone someone can declare and like i want to hear gossip votes yes this person yeah and then they only receive gossip about

that yeah um i have i'm differently in communication with the IP vs people and we meet at conferences and things like that and discuss protocol designs i've been trying to influence the direction of gossip sub and there's some elements that i figured out insecure scuttlebutt that be sort of using similar things this is thing called i'm tree it was influenced both of us the main difference we have so far is dead it's got about users log so with IPS you broadcast like the latest thing and if you're not online when something is broadcast you might not get it if you come back on and what scuttlebutt if someone as it's just a pendant so if you

come back online after being off lightly you get everything that was sinned so it's gotta but doesn't work very well for things that i like only the latest thing matters like you always get the history new but it works really well for that interesting i guess it the way you just scribed this and the social graph and sharing this message like there's nothing private in your system then it's not know um that's not great okay sitter is so correctly the basic thing is you have this log of messages that you've created and by default the messages are public I kind of like on Twitter but also like on Twitter you have a private message as well in a private message is just a public message

except it's encrypted so it's encrypted and only the recipients can read it if they have the key if they have the key and the neat thing about it is that everyone who sees it tries to decrypt it and only the people that it's for or successfully decrypt it and because everyone tries to decrypt it it means that no one can actually know who it's really for except for the person that's yeah exactly so think of it like instead of sending like a letter like if you post a letter to your friend your friend can only your friend will read the letter but the post office will know who you seen deliver so it's cuttlebug it's more like you put a note on your door and you just have it

to your front door and everyone can come by and see that you've written a note but only the person that it's actually for can read the note so no one actually knows who it's for um except to the person who receives it so it's for it has predicted metadata and I think that's really important nowadays it's mostly more important to predict to me the data than to protect the content because the content is quite hard to analyze automatically but the Mediterra is really easy so if you know who someone is talking to you can get all these people are influencing each other so if this one has trouble these people that we should watch them as well but if you could see the content of what people

are saying like it's hard to say to say if you know just from particular words and the content whether or not that it's sarcasm or a joke or whatever like it you actually need like a human who understands the context to look at that yeah computer still can't do there very well I like that I like that idea I have for some reason as you're describing it like putting some message in a public place physically and then having someone who has like special I don't know psychedelic glasses that can actually like read it because they have the key the special glasses and actually everyone else can walk right by it and have no idea like you look at it but they'll have no idea what's in it and they don't know who it's for yeah even though there's this one person with these really weird glasses like getting

really excited about the note yeah cool so often there's a term pub/sub that comes up has is that related to anything that we've already talked about we talked a little bit about gossip sub which is sort of the decentralized equivalent in some way pub/sub assumes that you have publishers and subscribers rights and that's what it sort of stands for and that's true in a decentralized setting as well like but but for the publishers to be able to reach the subscribers directly you need to solve the routing problem you need to route directly and like they need to have direct or like even if not direct connections than some routed connections to each other so that I can say I'm

going to publish this measure message for these subscribers and you can have one to one or many to many or one to many channels and all these kind of things but if you run into all of these other problems if you want to have a pub sub system so I think the more interesting thing and like what block chains like polka dot anything them to and all these other systems that have multiple actors they are now looking into how can we modify our gossip protocols so that we're not gossiping transactions and blocks to everyone because it's not relevant and everyone anymore I just want to gossip these blocks or transactions to this particular shard because that's the only one they're relevant for but they still

want to use gossip because they don't know who the validators or who the full nodes are for this chart so they don't want to just gossip everything out but they won't want to be able to say like only these people get it otherwise you overload the whole map in a way can you then use that for like defining roles like if you have different entities with different roles they would have special keys it would allow them to see the seeker messages in a pool of gossip yeah that's the idea but there's another crux of that matter and and that is how you do peer discovery in a topic-based way so we were certain mentioned briefly like anything one we're talking about adding data to the DHT to say like I am willing

to serve like clients but ideally I am able to find someone who serves like clients without having to go through this DHT because it's has problems but that's as far as I know an unsolved problem I just remembered I'm one of the other problems with th days to find a thing you may have to get to a lot of a logarithmic number of nodes which can take a while with you have a bad nation and lockups can be quite slow no that's a good point and I think actually in the blockchain world were actually a little bit spoiled by this and we're kind of saved by the fact that Bakhtin's aren't popular and aren't used by anyone

because there are so few notes that we can pretty easily find the whole network yeah and look through all the nodes in a reasonable amount of time but if you look at like BitTorrent that has yeah probably millions of entries in its DHT it's a completely different story yeah well you probably be alienated DHT in that case probably not yeah so we've covered a couple of the challenges of sort of decentralized networking but like with what's been created now and specifically what you've been working on what's what's still a challenge there I think it's a matter of like layers like once you have solved the identity mapping problem then things like messaging becomes much much more simple

it's just in the things that were the obvious next steps and a centralized we had were different to what is actually that simplest to build of most obvious things in a decentralized web so you know first and the traditional web you head-but websites then you had let Geocities and then it took quite a while before people discovered thought of blogs and then social networks but I think in the centralize world it takes easier to go back the other way so a social peer-to-peer thing was actually like so this is actually the best starting point and then we can use that to get to these other things you'll probably go other places cool well thank

you so much for coming on the show and talking to us about decentralized networking and what you've been working on thank you for having me thank you very much and to our listeners thanks for listening thanks for listening [Music]