P2P Apps Deployed P2P

Transcript

Uh oh, there we go. Okay, nice. Hey, come on in. Uh, my name's Dave. Uh, DMC. We've made something really exciting. I'd like to show you. Uh, and we can play with it a bit as well if you'd like to engage or I can also just demo it. We've got an hour and a half. Um, but I wanted to take a little bit at the top to to break down what this is because it requires a a paradigm shift. Um, it requires groing that this is a different way to to build. This is a different way to

solve. Um the headline is no servers, no cloud, no masters, build peer-to-peer apps. So four years ago we started working on uh a concept which is can we build peer-to-peer applications and systems that require no server infrastructure.

And it turns out that you can. We've built an app called Keat on top of on top of uh pair. Pair is the thing that allows you to build peer-to-peer apps. Keat has a million new users per month. It scales to the planet and the infrastructure cost is essentially zero. Compare that to Telegram with a cost infrastructure cost of 700 million a year. And you can see why this is a good idea just on practical grounds. We can build applications that currently exist.

We can build new types of applications that aren't even possible with a centralized approach that are real time uh that work almost identically to their equivalent apps but with no infrastructure cost. Imagine you're saving 700 million a year on infrastructure cost. Where are you going to put that? Right? It goes it goes to things like business strategy, marketing, compliance checks, all that kind of stuff.

So [clears throat] it also eliminates um a huge amount of communication because when you have infrastructure that requires people to oversee that infrastructure and that requires many layers of communication around that. You eliminate all of that. You eliminate a lot of noise with within uh an application's development and deployment. You eliminate that and you can with say four friends make an app that can

uh do something uh much high at much higher scale than you could have before. If you can build a UI and you can think a little bit in the peer-to-peer paradigm, you you can build things that don't have any infrastructure cost. If you don't have any infrastructure cost, you don't need to exploit your users by their data because you're basically taking a fire hose of user data exploitation and then you're giving that you're sending that fire hose straight through to pay cloud providers. So with this, you don't have to do that. And just on a basic human level, that feels right.

Here's how it works. Use a module, deploy peer-to-peer. No cloud involved. That's it. We have a module. You take it, you put it into your into your project. It enables peer-to-peer over-the-air updates on desktop, on terminal, and coming very soon on mobile as well. So that means that you can deliver your application via the normal channels, but then any updates to that application come through as peer-to-peer updates. So you can deploy an application to billions of people around the world with no servers and have peer-to-peer overthe-air

updates today. This is open source and available and it's how we build. This is ke. Um the beginning of this year we had an uptick in usage and uh whenever there's uh difficulties and challenges in a particular country uh and they start locking down communication and locking down internet access and different things we see a big rise in that country uh of usage of ke. We've not been marketing it. We've not really been advertising it, but as I said, we have a million new users a month currently and

growing. >> Sorry, what are the >> Oh, sorry about the confusion. Yeah, that one's velocity, I think. But look at the uh green line. I need to get a better graph. So, pairs.com peer-to-peer embedded application releases stack. Uh, if you'd like, you can scan that. It might still work. If not, I can sort you out later, but we can connect on Keat and you can come into the pair development room. You might need Ke on your phone first. I don't think I've updated that QR code. It'll be on the bottom left of the screen. You might not be able to scan it, but don't worry.

There's not that many of us in here. We can we can figure it out between us. The stack, we have a library called pair runtime. Uh, you mimm install that. This is a JavaScript thing right now, but the paradigms and the protocols apply to anything. The command line interface is pair. You use that to deploy your application. You can also use it to install applications peer-to-peer on desktop. Then bear. Yes, we have pair and we have bear. Uh the logo for bear is a bear, but it's not spelled that way. It's spelled in terms of bare metal. Uh, and the reason it's called that is it's a

NodeJS like JavaScript runtime with no core library as a very specific design choice. If you have no core library, then you're not granting any capabilities from the outset. If you have no core library, then it turns out getting mobile support is much easier. So we we run uh anything that you that you need uh nativewise uh with bear you just install it as a dependency. So if you need to use the file system in node you use the core fs module with bear you use the bear fs module when you install it. Um with that uh it works on mobile on desktop terminal. We can uh deploy a single peer-to-peer backend

uh on bear on mobile, desktop, terminal, and then the uh whatever view layer. So it says React Native on mobile and electron on desktop that can talk to the same uh code that runs in the bare back end on that machine. So this is really nice. we found is an architectural choice because most people are used to the front end backend paradigm. So we just say okay well there is a backend but it's local on device. You talk to it over RPC and then that local backend does all of the peer-to-peer magic.

Then there's the bare module ecosystem and then there's the hyper module ecosystem which cont which is where we get all of the peer-to-peer capabilities. There's about probably like 2,000 modules on the whole punch GitHub repo that that this all extends down into. This all comes under the stack. I've said some of this already. I can skip it. Uh the pair runtime module. The only thing I'll say is it has a run method and that's for encouraging this architectural approach I'm talking about. You can uh mpm install uh pair runtime and then use uh pair run within an electron app or a mobile app and it will start a bare worker which is your

peer-to-peer back end. This is the workflow overall. This is I'm just giving you an overall image of this and then we're actually going to uh go through a little workshop and and play with the tooling. But this is the full picture. uh you use pair touch to create a a pair link that's that's basically a deployment target. You use pair seed to expose hypercourse to peers. So the way that it works is it's um it's next generation bit torrent called kademlia uh and that's seeded over a DHT uh by and then a topic the the pair link is actually a topic which is then discovered via the DHT. So when an application connects to the seeded

hyper core uh it's connecting by the pair link key which is a topic on the DHT which allows peers to define each other. Uh there's pair build which creates a multiarchc deployment folder. So you've got you know Linux, Windows, Mac, you put those all in one multiarchc folder and then that's what you actually deploy. pair stage synchronizes the uh files on your disk into a hyperc core. Uh that's that's the equivalent of a staging environment.

So when you stage, you've got you're actually at the point we're in the workshop, we're only going to take it as far as stage because that just shows the functionality. So, but when you stage, you're actually at the point of doing peer-to-peer deployment because by synchronizing from disk into hypercore uh and seeding that to the network, uh you're exposing uh uh uh the the builds, your user installs the build, it connects to that uh to you as a peer, and when you do when you stage the next time and you do an update, the length of the hypercore increases.

And then that's that is uh replicated to the peers. The peers receive that as an update event and then they can click a button and it applies the update does a swap on disk. That's how it works. Pair provision, sorry, I wasn't sure if it was a question, but it was a yawn. It's all good. Pair provision is a pre-release step and it it basically strips down the uh the hyper core because every time you uh stage it if if it's doing a deletion then that's actually an append. So when you're staging to a hyper core it kind of gets bigger and bigger and bigger.

But when you do a provision uh it's it strips out all the intermediate steps and it's basically like a snapshot block sync right? So that that becomes a pre-production drive that that link that pair link uh you you you you give to like QA teams and uh uh that that's that's where you're getting really really serious like we're about to go into production. This is how we this is how we make ke. And then there's pair multisig. And that's the production step with pair multisig. You you have uh this this pre-release key. This pre-release pair link that that you've that you've been qaing and all of that. And then you have stakeholders who are responsible for what's going out. So say you've got

like five stakeholders and then three out of five have to sign in order for that to actually hit production. So all of your users are using the multisig production link in their application. Uh and the the three out of the five say in the if it's three out of five quorum who sign each one of those people who sign are saying I'm putting my career on the line to say there are no supply chain attacks. The diffs look good. I don't see any bugs. I don't see any vulnerabilities. Right? It's a very thorough process that is then signed off cryptographically with your public key and people know that that's your public

key. We've developed that process over the years because it doing a peer-to-peer deployment, it's really really important that it's done right and then it's done seriously. And the pair multisig command really uh hones the focus. So that's that's the full flow of how you deploy uh a pair app. It works like this. You have this draw distribution. Uh so you do your builds like you normally would. You build your mobile app, you build your desktop app, whatever. You put it on a website, you put it in the app store. The user gets it, they start running it, and then it connects peer-to-peer. From there, it's peer-to-peer updates.

We've talked about Bear, so I'm going to skip this. This is the Bear ecosystem. I I don't know if you guys have used Node or whatever, but the things in green are basically the equivalents of of Node Core. And then there's a bunch of stuff beyond that as well, like bare f, which we use in KE for um certain things. Then there's the hyper ecosystem. Uh you have the bedrock primitives. I've already mentioned hyper core. Uh you also have hyper DHT, which is peer discovery. And then you have hyperswarm which is peer discovery via topic. Then you've got the data structures uh hyperbe, hyperdrive and autobase.

Hyperbe is a key value store. Hyperdrive is a peer-to-peer file system and autobase allows you to do multi-writer. The really cool thing about autobase and this is what we use in ke for conflict resolution of messages is it has this apply function. So everyone in if you're on ke and you're talking to someone else, you and that person both have an appendon log. I write a message, it appends to my append only log and then uh you receive that message, it appends to your append only log, right? Uh but it goes through this this linearization process via autobase. So the autobase runs that through an apply function and the apply function determines whether it's written and how it's written and

when. Uh and with that what you actually have is tokenless smart contracts. [snorts] Just something to think about. Um storage and sync core storage is for managing multiple hyper cores. Local drive is the equivalent of hyperdrive but for disk. That's how we do a stage. We just we just have a local drive that mirrors to a hyperdrive. Uh which is what we use mirror drive for. So like I said, you have these deployment layers. You have this build, you have this build flow, you have a deployment directory, which is your multiarchc folder, and then you have uh the the staging uh layer which you can

just pair stage to. You could have a shared staging key. You could also just touch a new link, stage to that, give it to a colleague and say, "Hey, can you just check this over or like I'm struggling with this or whatever." that's really really easy to to create variations internally that no one else ever looks at even internally just between you and someone else. But then you can also have like these are the official links between us. So we have uh we have a development stage key, we have a staging stage key, and we have a release candidate stage key. And it goes we that's that's an additional process that we have where we're going through each uh stage until uh once we get to the RC key, it it stays on there for couple of weeks and then once we're like

happy with it, we we do a provision and we we create the pre-release key. The pre-release key is like, like I said, a very very serious moment. Like at that point, we have QA check the RC, but then QA check the pre-release key as well. If there's any problems at that, there shouldn't be any problems at that point because what you're saying with the pre-release key is like if we' have gone to production today, right now, this would be it, right? And then uh once that's been verified again, then you do the multisig and you go into production. Uh this is just a release cycle. every time you uh we're going to look at some uh the hello parallelon boiler plate. Uh but the way but basically every time you version uh every time you bump the

version that's uh to create a new build and the updates are based on the version increasing. So you version it, you make the distributables, you build the deployment directory, you stage, you provision, you prepare the request, you sign, you verify. Once you've got the signatures, you commit, you're in production. So, uh, we've got different quick start guides. The one we're going to be looking at is the Electron boiler plate. Uh, don't worry, we'll come back to this. We also have a, um, bare boiler plate, which is for creating, uh, peer-to-peer uh, terminal applications.

So CLIs, ripples, dammons, services, uh uh uh hooks, um whatever whatever you can make in the terminal, you can now make as a uh application that has peer-to-peer overthe-air updates. Pair Pass is a product that we've made which is a password manager. You can't leak clear text passwords if the thing doesn't have a server. like you eliminate all of that and you just have it between your devices. It's about personal data custody, right?

QAC is a uh uh local LLM uh project. There's a lot of uh optimization for running LLMs locally. This has a really nice parity with uh our peer-to-peer stack. Uh one of the things that QVAC can do is remote inference. So if you have a if you have a nice beefy machine with some LLMs on it, you can use QAC with our stack and use it on phone for like the easy stuff and then offload compute for the remote inference and of course that's all completely private and it's all completely your machines your data and then of course ke and like I said we've we've discussed that uh if anyone wants to uh connect with me on ke uh we can uh later on uh same for that uh

there's a news uh pairs.com news. Check that out. Uh docs.pairs.com to learn how to build and careers.te.io because we are hiring like actually uh a lot. Um we need people of all kinds, frontenders, backenders, cryptographers, whatever you are, we're probably hiring. So, uh have a chat with me if you're interested in that. Um I certainly would be interested in that. So, um, any questions before we get into the workshop?

Yo, >> thanks. I really like what you guys have been doing since the beginning. >> There is one thing that might >> Yes. Thank you. Uh there's one thing that might be uh a stupid question uh uh based on what you just described, but um I'm building an app that has quite a few users at the moment, but in order to make it as accessible as possible to the general public, it's a web app.

>> Uh it's a web app. We try to keep everything local on your browser and when you need you sync with end to end encryption, etc. Are you thinking of introducing anything in uh in your tech that could be applied to web apps like mine that try to maintain the best security and privacy as they can? >> Yeah. Uh so that's that question has been answer asked many times. Um and the answer is unfortunately always the same. Browsers are controlled by specs.

Specs are controlled by organizations who have a vested interest in remaining centralized. In order to do the kind of peer-to-peer that we do, you need UDP. Browsers don't have UDP. I don't expect they will ever have UDP. It's not a technical problem. It's a political problem. So our response to that is to go 100% on apps. However, you can just like like I said, you can create like a a bear a bare process with like OTAA updates or whatever. So you could

create a server that's a portal into a peer-to-peer landscape and then progressively enhance from there. So you could have a web app that gives your users as much as you can give them and then after a certain point when you have the scaling need or whatever whatever that feature is then you onboard them to the app which is a pretty normal flow. Um and that's we haven't decided to do that for us because Ke is a messaging app and it it it doesn't necessarily make quite as much sense as as a web app then. Um, but I I do think that there's other uh other types of apps that you might build where you you you're going to want a

portal probably. >> Super. Thank you. Uh, and last question. So, we're indeed also preparing some desktop and mobile apps. Uh, so slowly taking the users into something that's much much higher friction. Um, but for us it's uh it's document collaboration, spreadsheet collaboration. So, you have 100 people editing the same stuff. you need CRDTS etc. Uh do you think that at the moment uh uh what you've built is already at a good stage to handle applications that require I don't know thousand edits per 10 seconds by uh so many sources >> 100%.

uh not that it's easy uh but uh the the key is that every edit needs to be uh you what you you don't store the latest state you just you store every edit that's made and if you do that then you've got undo and redo and all that built in uh but you're taking advantage of the append only append only log uh and then the autobase apply you would need autobase for this for sure and then the autobase apply function function would then be deciding who wins or it might even have you can you can do it any way you want. You can even do it for certain situations where it sends it back to the user and makes the user

decide who wins, right? But there's many ways you you can do that part. Uh as long as as long as you're not trying to store latest state, as long as you're actually storing every single edit, then 100%. >> Thanks. >> Sure. Any other questions? Yeah. I mean, >> yeah, >> thank you. >> Thank you. >> So, yeah, super interesting. First of all, um, who's running the DHT server nodes? Um, maybe.

>> So, uh, every pier can be a bootstrap node, >> but, uh, you need the IP addresses of the bootstrap nodes. So the key app for instance has a couple of bootstrap nodes hardcoded >> and and we run those right but if you're making your own app you can set up your own bootstrap nodes and and run them or you could make it so that your users could supply >> would these networks then or these DHDs overlap is it like one giant DHD where all the apps would register themselves or is it does each app have their own >> by default the first but if you'd like to customize the second?

>> Right. Okay. Got it. Um, and how's the resource usage of a DHT client on on mobile? So, this has you >> how do you mean? >> So, for example, there's so back in the day, I'm not sure if they're still around Bertie, the they used to run a DHT client on mobile devices and what they were suffering is like they this was just draining the battery because you had to keep the routing table up. Yeah. Um, so, so, uh, just I'm going to answer this in a roundabout way. >> Uh, because it hopefully it makes sense at the end. So, um, if you're not online >> and I send you a message, you can't receive that message because it's peer-to-peer. But if we have blind peers, then those blind peers can

receive the messages and forward it on to you encrypted, which is how we solve the availability problem. But, um, mobiles will never be blind peers. So, they're not not every peer has to be the same. mobile will probably always stay as client peers >> and by doing that you reduce a lot of the battery and energy cost and all of that but it's not you know it's not a foregone conclusion it's it's still there's still definitely some trade-off there but I think the well I know like because I use keys all the time the capacity is sufficient that it works >> yeah I thought I thought so that it's it's fine I I I guess um what else do I have here ah um yeah what whole so I

P2P networking is hard. So you're probably applying some hole punching techniques. I guess >> when we first started doing this, we called the company hole punch >> because our secret source is the hole punching technique, >> right? >> Um I can't speak to that so much because that's the bit that I that's where I came in once once that was built, >> right? Um but as far as I understand it, it's a statistically guaranteed method to opening the uh the gates on the net routters and uh from what we can see it it it it works very well. Uh which you can see in the data because a lot of those million new users a month are actually coming from Russia. uh we have some conversations with uh people

inside China with having conversations outside of China. So um there's uh it's effective >> and then but it's not 100% effective for the like 1% that it's not then it goes through uh a blind pier actually but as a relay. >> Exactly. Yeah. This is exactly what I wanted to know. So what the fallback mechanism looks like. >> Yeah. It's just a message. So it works over a blind pier as well. >> Yeah. Sure. Um you me so you mentioned UDP earlier um is so does pair support like TCP transport or plus quick or is it are these just raw UDP datagramgrams that you're sending around?

>> It's our own flavor. It's a it's called UTP I think which it's just UDP plus. So it's it's it's TCP like but it's not TCP because it's it's optimized for what we're doing. >> Okay. But Bear supports TCP. I don't know if it supports quick, but you could write a native add-on just like easily, right? >> Is this all written in in uh JavaScript or is it just a shim or like a >> JavaScript and C? >> Sorry, >> JavaScript and C. >> So, uh we find JavaScript's really good at um rapid prototyping, rapid iteration. Um, we use it. We obviously we have practices and and have a pretty strict subset, but uh it's fantastic for for getting there

quickly. It's fantastic for on on boarding people. Uh it's such a commoditized language. Uh but all but and then with pairing that with C as well. So C is for like the native add-ons in Bear and then you have this thin, you know, JavaScript layer on top. Uh >> yeah, to me it just sounded like an unintuitive unintuitive choice of yeah using JavaScript for like s such a system low-level system um problem to solve. Well, but you mentioned C as >> Yes, there's C. But I I think for something like this, JavaScript as an evented language is excellent for as a med for the mediator pattern and in peer-to-peer it's all mediator patterns, you know.

>> Um so, so it I think it works really well particularly for the mental model. Um but we are uh as we move forward like more and more of the stack is being moved into C >> right >> because of efficiencies and and uh cross cross compat and things like that. >> Cool. Okay. Uh maybe one last question. So is there um some so you may have already touched on this but in other P2P libraries there's next to DHT which is used for peer discovery also some gossiping protocol.

Um, is there anything like one of a module that supports any kind of gossip? >> I a little bit out my area, but I believe Cademia is responsible for the gossiping. The Cademia, it's a it's a next generation Bit Torrent algorithm that we use right down deep in the stack. You can find that under hyperdh. >> Okay, so the the DHT is just a key value lookup thing. So if you want to disseminate a certain message across within the entire network, there are like probably other protocols. I will I will look it up if there's anything. We have something just on that just as a side note. We have something really really exciting coming that's not been released yet. It's still in the works but um search via DHT as well.

>> Interesting. Okay. Okay. >> So that's that's a global search engine at that point. >> Good. Okay. Anyone >> any any other questions before we get to the actual workshop portion? >> Okay. I I would have yet another one then. >> Okay. >> What did you want? >> We can take one more and then one more if you want. >> Right. Okay. If yeah um the hyperdrive that you mentioned is this in any way content addressed is content addressing take is playing a role in all in in this whole stack that you mentioned. >> Uh when you say content addressing you're talking about content hashing of the content. So we so pair links are um a standard.

>> Mhm. >> And it's uh the pair protocol and then uh a key string but it can also be the pair protocol fork.length length keystring and then it can also be the pair protocol fork.length keystring uh hash content hash or something like that. Right? >> So we have we have all of the forms and if you if you want to use the full content hashed version you can. But with the fork and the length uh you you you're actually you're pointing to a very very specific thing in a hypercore at that point. So there I think there are probably like if you're ultra paranoid you can use the full content hash but mostly we just use for length.

>> Okay, cool. Good. Thank you. >> Yeah, sorry I I missed the beginning but uh what I I've heard is that you're making a a distributed hasht kind of storage >> sort of but more much more than that. It's a it's a way to build applications at planet scale without servers. >> But how does that differ from from something like swarm or >> swarm as in DHT Swarm?

>> Ethereum swarm the >> right. Yeah. Uh not not too keed up on that. We have uh I don't know can't compare them. Uh but there's no tokens involved. Maybe that's the the key point. >> Thanks. uh stick around for the workshop and we'll get into it. Okay, cool. So, if you'd like to follow along, uh take a look at github.comholepunch2.

wait, Where's it gone? >> Pair workshop. Let me put this in um bigger letters so it's easier to see. Everyone got that? Anyone need more time? Yeah. Cool.

Okay. So to do this workshop you need to have uh node installed if you because we use npm. We don't use node on with pair actually bear is everything but we use node and npm because those are the things that people use. We we don't use node, sorry, we just use npm, but you need node for npm because that's what people use to install dependencies. Um, so if you don't have it, you can get it uh on Mac with nvm.sh or and Linux. And on Windows, you can get it with NVS. See it in the workshop. Uh oh, good. to install pair um you just

run shoot hold on let me sort myself out of it but you run mpx pair now if I run that it's going to do something different I think you know what I'm going to remove pair from my system just to show you how to do So this is this what what this is doing is it's uh executing uh an MPM module that downloads pair from peers

uh and installs it onto your system. The pair itself is actually a bear program. Uh, and the next the next version that we're going to release is also going to be using the same um PP OTA updates with bear approach that we uh published recently. Once you have it, you should be able to uh you're going to you're going to need to export you're going to need to modify your path. Um you can do it by running this command here. I don't need to because I have already done it. And then if I run pair, I should see some help output.

Come on. There we go. Um, that's a Mac thing. That's not a pair thing, by the way. When you put a new binary on Mac, it actually does a call back to Apple, I think. Um, so this is this is the uh the pair uh help. You can see some of the commands I talked about there. Touch, seed, stage, build, provision. There's some other cool stuff. Info dump is really cool because you can just you can dump whatever you staged. So it's it's synchronizing from your hypercore back onto disk. Uh change log is really cool, really fun. Um all right, anyone need help getting pair

installed because it's, you know, not as easy as it it as it could be? Everyone good? All right. Don't forget to do this part. If you don't set up your path, you can't you can't run it because it's pointing at a custom bin folder. In future, that's going to change. So, once you're set up, uh you can go to the first exercise. Uh, and what we're going to be doing is we're going to take the boiler plate.

How much longer do I have, by the way? Like, uh, 45 minutes or so. 50 minutes. >> Well, officially the workshop end. [clears throat] >> Great. Perfect. Lovely. Thanks. Okay, so we've got this boiler plate. Hello, Parel Electron. And we're just going to get clone that into a chat app. So I'm going to go get clone. Oh, did it twice.

Get cone. Hello paralleer into chat folder. Let me make that a bit bigger. And then uh we're going to do a uh run this command. And what that does is just sets the package JSON name uh the package JSON product name and the package JSON version. Um the product the product name is what's used when we actually do a build. So typically it has an uppercase letter in it and the package name is has to be lowerase. That's why we do that. And then I'm going to mpm install.

Uh while that's happening let me talk you through the boiler plate. So the boiler plate is uh the the read me of this boiler plate has the full explanation of how to do this end to end. Um talks about the architecture, talks about how the updata works, uh talks about where the storage lives, how you can set your own storage flag, which is important when you're developing peer-to-peer applications because if you it's all about where it where something's stored. So if you run two applications locally with different storage targets, then you're you're simulating a peer-to-peer interaction locally that they actually go they actually go and connect outside uh and then whole punch back into yourself. Uh when you when you that's why that's

important. Uh the workers which is the backend stuff um explain it it talks through the deployment stuff I've already been talking to you about. You can see some of these diagrams are quite similar. So, uh, this is this is something you can work through like end to end to get to a full production deployment of, uh, a par app. Uh, now that's installed, I'm going to do mpm start. Oh, my bad. I missed a step. So, the package JSON has an upgrade field and it says your key here.

So, uh, we need to we need to give it, uh, a key that we're going to stage to later to do peer-to-peer updates. So, I'm going to run pair touch. And that's given me a key, which I'm now going to take, and I'm going to set on the upgrade field like this. Uh, and now MPM start should work.

Great. So, that's launched uh an Electron app and it just says what the version of the app is. I hope someone hasn't died. That would be terrible if they have after saying that. Um, so that's that exercise two. Now we're going to make the actual chat app. So um, what we have here is uh a little diagram of the flow between the pieces of the Electron app and a

remote pier. So the Electron app, it has a renderer app.js. It has a preload.js which is actually part of renderer. Uh but it goes through this context bridge thing. It's Electron stuff, don't worry about it. And then there's Electron main.js which talks to the renderer. And then there's workers main.js which is bare runtime. That's what's running the actual peer-to-peer code, right? That's all talking over uh IPC RPC. Then from that it's doing a write to the peers uh via hyperswarm. Every time hyperswarm uh detects that a new pier has joined you get a peer object and

then you can listen for data on it and you can write to that object and that writes to the pier. So uh so we're getting the data flow is going from the front end UI the web view over to electron main over over to the bear worker over to the remote pier back from the remote pier and all the way back into the renderer. So to do this we're just going to install some dependencies. We need hyperswarm for the DHT uh hypercore crypto and uh uh B4A which is just a a buffer manipulation uh dependency.

So I'm installing those now and then uh copy paste. So we're going to take workers main and we're put going to put this code in it. You can see here that we have a hyperswarm. We set it up con swarm equals new hyperswarm. When the swarm gets a connection, it gives us a peer object and then we're listening for data on that pier and then we're calling our uh send function that we've made with uh this object. The send function is here. Uh and all it does is it writes to uh this pipe

um the which is a uh a frame stream over a socket essentially bear.c is the socket when you start bear as a worker bear.c PC is the is the the socket to the parent. Uh and then uh when data is coming in, we take the message uh and then uh depending on the message type, we either join a swarm uh with random bytes or we join a swarm with provided bytes. So that means like that's the same exact flow from there for creating a chat room or joining a chat room. And then uh if it's a type message, then we loop through every pair

that we're connected to and we just write the message text to them. Uh and that's that's it. That's pretty much that's that's the whole thing uh in terms of all of the peer-to-peer logic for a ephemeral chat app. There's no persistence here, but that's the whole that's that's everything. So I'm going to put that into uh workers main.js JS like so. Oh, wait a second.

Someone's updated this. I'm going to keep the um what we also have in workers main right now and it's a good thing but this needs updating in the workshop um is the the updater is running in the worker. The updator used to be running in electron main. So um I'm going to copy paste that code but it's going to go between these lines here. Hopefully this works. I'm not sure if it's going to work because it's a bit have to debug it if it doesn't.

That's fine. And then uh got some HTML here uh renderer index.html HTML which we're going to uh just replace verbatim. All it you'll see it all it all it is is uh some chat stuff uh for joining and creating rooms and then uh renderer.app.js will give us the um the glue code for receiving uh data from the worker and and vice versa. We're making that work with the UI. So that is

renderer.app.js not update. Okay. I'm not going to approve updates for the chat app. That's just not going to happen. Uh [sighs] right now that's done. npm start. Then we've got a chat app. And then if I create another process, mpm start with a storage flag.

Oh, that's the wrong wrong um now I have two instances of the chat app uh with two separate stoages. So it you could also if you've if anyone's got it running on your you can also you can also join but uh let me know if you have if you have it there. So if I create if I create a room here. >> Yeah. So when you press create it it joins the room with uh the random bites.

Uh so we're creating a topic on the DHT. Join the room. That should be happening faster. But I've just realized I'm on my uh uh I need to get on. Let me try on Dcon guest. Uh, and then the idea is that you then join that topic on the other one. I don't know why that's being weird right now. Let me try creating one on this one instead. Cool. Just checking. Yeah. So, the the boiler plates been updated and the there's a little mismatch here. I'm gonna have to debug

this real quick. >> Does it >> fantastic? >> Yeah, you didn't do the little thing that I did. Let me just do that and see if that works. Okay, cool. So, I've got this topic and then I join that topic

and now I have um are you on ke at all? >> No. >> So, I'd share the link with you on Ke. That's the thing. You have to share the topic out of band some way or another. Um but if I gave you this topic, it it would work. A lot of the time we do this uh uh as a remote workshop, so it's really easy then because you're like in a Zoom call or something, you know, but trust me, it it works. Um any any questions about that about that build?

>> What the bearip part? So I have I have suspicion that maybe >> Okay, thank you. Yeah, good. How does the bearapc IPC thing work? I said I have a suspicion how it works, but maybe you can explain it again. >> Um >> um so uh electron main.js, right? Mhm. >> Uh we have pair runtime run and that what it's actually doing is it's running uh bare sidecar.

Bear sidecar is just a duplex stream uh that spawns uh a uh a bear that's packaged in as a dependency, but it's just a native it's still just a native binary uh per OS you just install it in. And then uh the IPC is uh sdddio3. So you've got, you know, zero for input and then one for out and two for error. We do FD3, right? And that's just the that's just the pipe then. Okay.

>> Yeah. >> Nice. Anyone? >> I was just wondering if I don't know if if this is a Unix domain socket or how you connect all of these pieces together, but okay. This is different from what I expected, but it's cool. >> Mhm. This is after like four years of iterations and it's where we sort of landed for uh you know uh it's not just about like the technology side of it. It's about what makes sense with lots of people working on something at the same time, you know, um and and and you know, helping people uh framing things in a way where it just makes sense to like onboarding people quickly because that's what we've been

doing a lot, you know. Um let me also show you. Sorry, I'm just going to go off piece here and just go do whatever. So, I want to um I'm going to clone the the boiler plate again and just straight into Hello Pareleron um because then I can show you the update flow because of that little wrinkle there. So, uh, npm install we're going to do pair touch again. Create a link.

I'm also going to set the version to zero just to confirm it and then mpm start. Great. So, uh, now what I'm going to do is I'm going to do, uh, mpm run make make a build. So, what I'm doing is I'm going to go I'm going to go through the productionizing flow and then uh do a peer-to-peer update. Uh, and you'll see the version hopefully change if uh if I don't mess it up. So, I've got some artifacts here. Um, great that works. So now that so once

once you have once you have uh a build of an app, you can uh use the pair build command to piece it together. It's a really simple command. All it's doing is copying, but it's about structure, right? Um the point is that you want to be able to build on Mac, on Windows, on Linux, and then put them all into one folder. Um, at some point we'll extend the functionality of pair CLI to make that easier because we could totally leverage our peer-to-peer stuff uh for making like a build farm or something.

Um, but but at the moment it's like manual on purpose uh because there's less that can go wrong. Um, so uh this was I'm building on a Mac M1. So this is a Darwin ARM 64 app. So I'm going to do pair buildarm 64 app out hello pair blah blah blah hello pair app uh and I'm going to set a target folder also for out uh out build let's call it and uh I also need to provide a uh path to the package JSON uh and that's really important because what a what a build folder is is

the multiple architectures and then a package. JSON at the top that has that upgrade field on it. And that's that's what it all comes down to is that package J at the top with the upgrade field on it. So uh if I run this now in the out folder I should have uh build uh yeah so this is this is it. You got the package JSON at the top. Uh, and then you've got the uh the bark folder just with Darwin ARM 64 in it right now. But you you you know, you want to have like for a full production app, you're going to have everything.

So with that, what I can do is I can cd into the build folder or actually I'll do it from the outside. Hold on. So out build is the is the deployment folder right now. Uh so I'm going to pair stage to the uh upgrade link which is this. And the pair stage command looks like this. Pair stage flags link dur. So I'm going to say pair stage with the link and then point it to uh well sorry I'm

going to do a dry run first always dry run first uh and then that's out build that's my that's my deployment folder right so once I've run that you can see that it's it's printed out this diff um pretty big because it's an electron app and it's the first stage but the point of the diff is to to verify um that every change that's happening is intended. Um, and that's why I dry ran it because it's not actually written. It's just shown me what changes it would make. So now I'm going to run it again without the dry run.

Cool. Uh, and now uh I should be able to uh see this link. And that's now exposing that link uh to the DHT. So if I there's something else really cool that I just want to show you that's on the side of this. Um we don't have this out yet, but there's come there's that will fail. There's a there's a pair install command coming.

But you can preview the command with mpx pair-install and then a link. So if I run that, it will install the build that I just made from myself. Uh you can see the there's a join on the seed there. That's that's this. That's going to install it into my applications folder. Uh so it it will install UI applications and it will install uh uh command line applications. So there's a side advantage here that the same update drive that you're using everything actually can be a peer-to-peer install source as well.

So if I open applications hello pair now don't be like this. That's not the one. Try again. Great. Okay. Maybe that wasn't one. Now when I opened it, there was a join that that came that popped up on the seed. If you would if I was to give you this app and you opened it, you would also join in it, right? So now I know that the this this this pier this app is that's now a pier is ready

to receive updates. So what I'm going to do is I'm going to make a change. Uh, and all I'm going to do because it prints the version, I'm just going to bump the version, right? So, I'm going to do mm version patch. Oh, the git directory thing. I'll just uh do it this way. Um, yeah, I'll just do it manually. So, I'm going to bump the version and then we go through the build flow again. So, it's mpm run make.

And then once we've made it, we build it again uh for the uh uh the multiarch folder. And then we stage it again. And then once we staged it, the update should go out. So pair build again. Uh everything's exactly the same this time. And now uh P stage going to dry run it first. And we can see this time the diff is much smaller, right? because we've only made a small change. Uh the things that have been updated are updated by the Electron make flow, uh some P list stuff, some that Windows stuff, the package JSON itself, uh inside the app, and then the top level package JSON,

they're the same package JSON. So that all makes sense to me. So now I'm going to run it without the dry run. Uh but before I do, let me make sure to have this app visible and the seed visible. So, I'm going to hit return in this one. So, that's staged. Yes. Good. So, the uh that should come through, but it's not yet. This is always the bit where I'm like, "Ah god, it's not going to work, is it?" But uh I might have to restart it to get it to work. Um, I'm not sure why that is, but it it

generally happens when I'm on the same machine, but if I'm using two machines, it doesn't happen, you know. Um, so you see that the length increased here from 1263 to 1273. That's that's the thing. That's the version. That's the the the peer-to-peer version, right? Uh if I do pair info of the link, then it should tell us, okay, it's 1273. So, we know that it's been updated. Uh but the app has not received the event.

Of course, it's not. Uh so, we got any errors? Like I say, this was updated recently, so it might have been broken. Don't see errors. Let me try shutting it and starting it again. Sorry, I'm just trying to think about

why this might be. My bad. Um, you could just trust me that it works. [laughter] It's how we It's how we do updates and keep all the time. We know it works, but I think uh we have a lot of guys that are that are contributing all the time and and improving things. And it's possible that there's there's a bug that's come in here between the last time I checked and now. That's the pier join. Oh, okay. That's

weird. I restarted the seed and then it started working. So, I'm going to take that back to the team and and figure out what the hell's going on there. But you can see that it's actually picked up the update this time and it says updating. And you can also see that there's a upload from the seed happening because it's uploading to this this pier over here and says the update is ready. Still downloading there. That's weird. Or uploading. And then if I press apply update, it restarts with a new version. Uh that intermediate bit, I need to research exactly what went on there. Um but when like I say, when it's on the same machine, it's sometimes a little diceier than on different machines. but

I'm not sure if that's something to do with it. And with that you have peer-to-peer of the air updates. >> so this >> Yes, it went out and back in. Yeah. Yeah. Yeah. Any other questions? We've got a cool snake like on the workshop. You can keep going through and then we've got like this cool game that we've made um called um Snake. I if you've heard of Snake before, but um this works on exactly the same concept as the chat app um but instead of it being chat messages, it's structured data uh that determines the position of

a snake. But it's the same architecture. It's the same uh same exact thing all the way up and down here. You've got like the frame stream. Uh you've got the updater. Uh you got join game instead of join chat, right? Um but it's all the same thing. You're just joining a swarm with a topic. The two P connect and then they can play snake with each other as well. There's a whole part two that you can

take a look at here uh probably in your own time, but if you if you work through uh part two of this workshop, you can go through the same deploy flow that I just showed you without but you can go you can take it further and go to uh provision and multisig as well. Um I could show we got a few minutes I could show the provision stuff. So let's say that um we're at a point with uh this link that we're like okay good uh version 0.01 that's the production version right. Uh so the next step then is to go okay well it's time to it's time to um

create a provision drive. So I run pair provision dry run uh and uh it takes a versioned source link a target link and a versioned production link. When you're when you're creating your first pre-release uh key your first provision key there is no production link yet. So for you just use a a zero versioned link and it but what it does is it it synchronizes everything from the version production link onto the target link and then on top of that it overlays everything from the version source link onto the target

link and that way you have a replica of production first so it's always it's always correct and then you have your changes on top. So, uh, I'm going to do I need to create a new a new link for this. So, I'm going to do pair touch. And I actually need to create a new version now because and and this is sort of uh C a problem for CI or whatever when it comes down to like a proper deployment. Um, but I now need to update the upgrade link. I think actually we've got a thing where you can do I'm not going to do it here but you can do like develop uh like stage and then

production and then have a link per um but the raw way is to uh update the update the link. Uh so I'm going to update that but then I need to also stage that because we're going to be syncing from the stage link as a source. So I'm going to go pair stage dry run. Oh wait, sorry. I need to do a make first. So npm run make and then I need to build and then I need to then I stage dry run and then it's got the the provision link that we've not provisioned to yet.

So, uh now build and now stage dry run. Yep, that looks correct. And then without the dry run. And now uh I can do the provision. So my stage drive is is uh aligned with the future provision drive. So now I go pair provision.

The source link is the versioned stage link. So you can actually find the version stage link in the stage output kind of discreetly here. And now the target is what we've just set the upgrade field to and that doesn't need to be versioned. And then finally, uh there's a production link which we don't have yet. Uh so I'm going to pass in the um the source link at 00. So it syncs nothing to start off with. And oh, don't

forget to dry run first. Uh I think because this is the first one, there should be a diff, but uh because it's the first one, there's not. Uh so this this is saying uh yeah, good to go. Now provision. So this this is getting more serious and it says like are you sure you want to do this? I'm waiting 10 seconds for you to bail. And then uh when it's done it it should then have synced and we can see that.

And what I'm going to do just to make sure that it's worked is I'm going to remove hello pair from my system. Oh, because it's uh running. And now I'm going to try to do that install again with the provision key instead of the stage key. See the join. See the upload.

We didn't need to bump the version for that build. Generally, you always bump the version, but because we're up we're changing we're swapping the upgrade key out, you don't need to bump the version because you're starting with a new key, a new a new link. Now it's been installed to the applications folder. Uh open it. Great. Cool. So this is this is now a pre-release link. The step after this would be uh pair multistick. Um and that this is where it gets a little bit heavier. We're not going to do this. Uh we don't have time. But the flow is this. you uh you create signing keys. So uh each

signer needs to generate their own key. So we would say like myself, yourself, maybe you, right? Uh and then yourself, and then yourself. And that's that's five of us, right? And then uh we're going to say, okay, all of each person is going to to generate a key uh with a password. Uh and that's uh then that that writes that out. It's sort of like SSH, right? Uh to this pair folder in your your home. So that's your that's your public key for signing. And then what you do is you create a pair.json file in the project route.

Oh, it's already there with uh those those public keys. So, um I've just lost that pair multi-IG get. Cool. Yeah, it it creates it if you don't if you don't have it. Uh otherwise, get will just, you know, give you your your keys. So, that would be the first key. And then, uh maybe I take two more keys and we just do a two out of three cororum, whatever. Um once once you have that you then run pair multic link.

Oh I need to Hold on. Let me just fudge this. I'm going to do a one out of two quorum and hopefully it lets me use the same key. Huh?

See if pet works. My link works this God damn it. Cool. So once you've set up your pair.json with your quorum, you use pair multisig link because uh instead of pair touch because it's determined by your multisig configuration. If you change the signers, you you you need a new

link. If you change the quorum, you need a new link because it's it's basically a contract at that point and the link represents the contract. Um and then that would be then set that pair multisig link would then go on the upgrade field, right? And then you'd go through that whole process again where you'd like stage the new multisig link and you'd provision the new multisig link because this is now about production. And then uh then you'd do a then you'd go through the multisig flow uh write it to that link and then at that point you've actually reached the production piece where you can actually give this to your users. Um so uh you would version you'd make the distributables you build the deployment trajectory you'd stage. You've seen me

do that a few times already, right? And then you provision uh and then you go and write uh pair multic request, right? So, um I wonder if this will give me a little preview if I do it like this. Pair multic request. Going to version it as 00 and see what happens. It's going to freeze. No, it needs to be seated. But yeah, so you you you you you I've not written to it yet, so I can't do this part, but uh with what what pair multisig request does is it it creates a um a signing request that you then give out to your

signers and then each signer runs pair multisig sign and paste that sign in request in and it signs it with their keys and they give their uh uh responses back to whoever's whoever's actually running the release, right? So, say it's me. I I I get my response from you. I get my response from you. We put them together and then I run pair multisig verify uh with the source link, the the multisig link, the signing request that I sent out, and then all of the responses. Uh that's basically a dry run, but we we've made it a separate command because it's that important a step that it should have its own command. Uh once that's once you know

that that's all good then you can run pair multisig commit with the source link signing request and responses. Uh and at that point it actually writes to the link and then any users that are using your app will receive uh an update and get the update and go through that flow that we have already seen with stage. But that that takes us to the that takes us to the conclusion that takes us to the full end to end flow of creating production and deploying and installing uh production peer-to-peer apps.

So uh questions before we wrap or um how we feeling doesn't have to be a question. It can be a a take if you like. >> Pretty cool. Ah, that's very nice of you. Anyone else got any compliments for me? [laughter] All right, cool. Um, does anyone if anyone wants to connect on Keat? We can uh just do it. Uh, we can wrap now. We can finish this and then as soon as I figure out where I've put my phone, uh, we can connect on Ke. Here it is.

Uh, I'll wrap it. Thanks very much everyone. Appreciate it. [applause]