Hands-On with Waku: Building Private, Decentralized Ecosystems

Transcript

All right. Good morning. So before we begin, um, since it's just us, feel free to interrupt me at any time, ask questions along the way. Let's try to make this useful for you. By show of hands, have you heard of Vaku? Awesome. Logos graph the mic. All right.

All right. So, we know what Waku is. Um, do we know what Logos is? If All right. We'll what we'll do is we'll focus on Waku but we'll also focus at the ecosystem around Waku and today's route and this is 35 minutes. Um we'll go over why Waku um a primer to what Waku is about and the current projects and the integrations that Waku has built around it.

We'll build a simple app. It's going to be a step-wise app that we're going to build and then we'll wrap up. And the goal for today's workshop or rather presentation is to leave with a Waku client runin. Just for somebody who does not know what Waku is, it's a lip p2P based gossip sub message bus. What it does is it's a gossip sub network. So each node in the network gossip the message further and then the idea is that from the moment it leaves the client it gossips around in the network and then it is received at the other end and this ensures a few

um uh this this ensures a few characteristics that we'll build upon. How it works is it's a simple pub based architecture. So the concept of content topics is derived or implemented and these content topics can be seen as application level content topics. It runs everywhere. So there's a light client implementation for um resource restricted devices. There's also a full node implementation for um devices that might not have enough resource or that have sufficient resources.

And in a line is a secure bandwidth aware and chain agnostic messaging transport. The protocol stack looks a bit like this where we have relay which is the real-time gossip network. This is also the full node implementation of the network. Then we have the store protocol which helps with history retrieval. Um there are some constraints around the history retrieval. Waku is not primarily um a storage layer. It is an ephemeral messaging bus. So history is restricted to a month. Then we have filter and light push. These two protocols go hand

in hand. Um filter is the subscription part of the pub sub where um you subscribe to a particular content topic and this is primarily this primarily helps with resource restricted devices. So think of filter and light push as a substitute for relay. For example, if you want to run vaku on a device that does not have enough resources um maybe internet bandwidth is limited perhaps um you want to save some battery because relay is expensive to run. So you would opt in for filter and light push which is filter is a replacement for the subscription part and light push is the pushing part the publishing part. Then another thing we have here is RLN. RLN stands for rate limiting nullifiers

and it essentially helps a decentralized network like WAKU with spam prevention. In an anonymous network, spam can be a hard problem to solve. And thus, um, RLN is essentially attaching a proof of your identity along with the message and then sending that entire message which also composes of not only the actual message but also the proof um of your identity um into the network.

All right, the why. The why is um and this is derived from just general uh decentralized architectures as well. With centralized architectures we have a single point of failure of course and um here we have since we have distributed the trust into multiple actors there's a permissionless mesh then we have um there's also IP linked metadata that uh is attached with centralized APIs scaling is with centralized APIs you want to go vertical but with you already have a network that's horizontally scaled And it is censorship resistant by

design. All right. So what is it? Just a recap is waku is a drop in transport. So any sort of JS TypeScript um app that infrastructure that you might want to build that requires communications between any sort of actors be it client to client, node to node, client to node. Um you can just use waku as a transport layer. It is a message bus. It works in any TypeScript stack. We also have uh more implementations. We have uh bindings available in C. We have um uh bindings available in Rust. We have uh a NIM um SDK. I'm not sure how many of you have heard of NIM at this point. Um and

we have RLN V2 already deployed on test net. RLN V2 is essentially our way of um uh introducing rate limiting nullifiers into the network and um all messages that are sent into the production um network for Vaku has a proof attached. So there is spam prevention and then the spam prevention works in a way where you're allowed every identity is allowed a certain number of messages per epoch and epoch is a way to say that okay x number of messages per n minutes is how we define that um uh that rate limiting and then we have works incentivization um uh for the for a year now and it's

all open source waku is open source all discussions are open source So feel free to if you're interested um it's on the forum. It's on forumwack.org if I'm not mistaken but I'm sure a quick Google search can find you the right forums. All right so building upon where it actually runs today and I could only talk about so many projects. So status um has have you all heard of status? Status. So for people who don't know, status is uh status started a few years ago about 2017 and it's uh today it is a private communities like messenger. It's a social but also it encapsulates a larger part of um what we want a full

stack social product to look like. So it's a it's got an inbuilt um wallet. It's got a browser. It's got communities discord like communities. And we'll talk more about how status and what status is uh in the next slide. Then we have portrait and uh open internet protocol. Uh portrait is an open internet protocol are is a self-s sovereign reputation layer for um which is decentralized. Uh rail gun um have you heard of rail gun? Rail gun. So, Rail Gun is a ZKbased uh private DeFi that essentially masks the transactions that are being sent uh into the network. So, not everybody gets to know what you're sending, who you're

sending it to. And Graphcast is a project by the Graph team. um it is their uh off-chain indexer uh network that is used by the graph protocol for indexing and vaku is used um for this for the communication the coordination that these offchain indexers need to do. So for status as we talked briefly status is a private messenger it's a wallet uh and a web3 browser and it supports both light implementation and the full node implementation.

Um and the mission is that communications should be censorship resistant and the metadata should be minimally leaked but to other recipients or to the network and this is the infrastructure that this is built upon. One of the infrastructure key infrastructures is Vaku and then there's a storage layer which is codeex and uh this is all incubated by uh logos the parent or what does it look like? So for the node types we have both full nodes and a light node implementation. So um even within mobile you can choose to run a full node if you so want to but it is not recommended. It is recommended that

you run a lighter implementation so as to save bandwidth and battery. Um there are some key topics used. uh key topics are essentially the content topics that the application uses and um X3DH is used uh the encryption scheme is used for negotiating 1 is to1 um secure channels. The protocols that it uses are so it uses relay for um the full node implementations, filter and light push for the light node implementations. As as we prior discussed, filter and light push is is essentially a substitute for relay uh in a bandwidth restricted environment. And then store is just for

some um querying of historical messages. And then X3DH is used for negotiating encryptions. And then for spam protection, we use RLN relay. RLN relay is um a fork of the relay protocol along with RLN rate limiting nullifiers which are uh we discussed this. RLN is attaching a message uh is attaching a proof of your identity along with the message into the network so as to realize that okay how many messages have has this identity set uh sent in the last x number of um uh in the x uh last n minutes for example.

Yeah, the network diagram is simplified but uh what it looks like is on the left is mobile or any device that you have. For example, it could be a full node desktop and there's a network of WKU nodes that help gossip the message that is sent by the device and then it so the message is sent into one node and then it further gossip it to another node and so on and then it is sent back to whoever is the recipient of that said message.

Now the developer takeaways here is it is a browser mobile full node whatever device it's a friendly out of the box solution for a for a drop in transport. Uh status has um it it is analogous to how we see communities or servers in discord but completely decentralized and it is in beta. It is out for download so you can go try it. Um and you can self-host your node. So any full node that you run with status is itself a node. You're a node yourself. So you're helping um protect the network at the same time. For portrait and a OIP, open internet protocol. Portraits the

reputation layer and the way it works is it uses signed JSON objects and it sends those over vaku. um they have for they use smart contracts they use um the blockchain for storing these CIDs for these JSON objects and CIDs are content addressable um identifiable something uh CIDs are essentially a hash for where your uh what your uh data stands for and open internet protocol is the umbrella specification that uh portrait is built upon. So Waku here is the default networking transport for all of portrait and of course for OIP it is

only a recommended practice because OIP is a spec. Um and OIP decouples the messaging from the uh storage and from the onchain logic. So portraits the reputation layer. OIP is the general stack for the scope. Um, portrait's got profiles and nodes, but OIP has decoupled the messaging from the storage from um the uh chain and Waku's role in portrait is it it is being used as a full transport but for OIP is a recommended baseline and the takeaways here are similar to statuses where it's a drop-in transport.

Um it it's easy to use it as um a caching layer if you don't have high reli if you don't require high reliability on storing your messages and then of course it's verifiable. So they have the CIDs on chain and then it is verifiable with uh signatures that are sent into the Waku network for rail gun. Uh rail gun is a zero knowledge based u layer for shielded ERC20 transaction. So any tokens that you might want to send that want to be shielded that you don't want to be public.

Um here the community broadcasters pay gas. Um and the wallet SDK is used to not only discover fees but also for the transactions that are sent into the network. They use um the light nodes but also use the full node implementation which is nvaku as the broadcasters which essentially pay the fees. They use all the protocols and store here the store protocol here is optin and to send a private transaction in the client is essentially just 10 lines of code.

The entire user experience is gasless and it horizontally scales. So you you can keep adding broadcasters into the network. For graphcast it's an off-chain gossip sub for um the graph's indexers and they graph has a concept of radio that embeds waku. U there's a registry contract on chains that is used for identity and then that identity is used to send all the radios which is u used for the communications between two indexers. So it's radio is a message that used that is used to send uh between indexers.

It's an abstraction over Waku and the um and the blockchain's uh RPC calls and radios are app specific. the concept of radios um the registries the registry contract is an identity contract and it is real time with vaku and um they haven't migrated to the waku network which is the rl which also has rln rate limiting nullifiers which helps with spam prevention this is yet to be upgraded uh by the graph now building a radio building a indexer to indexer communication is less than 30 lines of code and all of these projects

are open source. Uh all of the projects that are mentioned in this presentation so far and the network is chain agnostic. All right. So I'll do a quick walk through of how um this looks like in practice. There's a repository that already has um the entire workshop that we'll do bifurcated into nine steps. Um the repository is um this if you're interested it's github.com/danishor09/js waku hyphen stepwise could have made it simpler but uh bear with me and the road map looks like such so we'll have um

it's it's got nine folders the repository the first is the setup so here we'll just set up a boilerplate react plus wheat um app. Then on the second folder, we'll set up a Waku node and so on. At the end, we'll have a full-fledged um chat app that also has two sorts of encryptions along with being able to persist your peer ID and whatnot. So, let's see what that looks like.

So we already discussed um so we'll what we'll do is we'll use JS Waku which is the TypeScript implementation of the protocol and our approach is going to be to step by step building an on a complete messaging app and the end result will be a fully functional decentralized messaging application. So for first um for the first step we're going to just set up um react and uh react with weed and um there's I'll probably have um the GitHub repository mentioned somewhere.

So this is what it looks like. So the first is setup. And this just sets up a React application. And this is what all the folders will look like. If we were I'm going to try to also open it. So yeah, this is what the step one looks like. Not surprised.

For the second step, we're going to set up the Waku node. And what that looks like is just installing uh at waku/ SDK and then having a TypeScript file that imports two two things. It imports the create light node function and waiting for remote peer. Waiting for remote peer is essentially you want to if you want to wait for your app to um make sure that it has connected peers before you start implementing executing the application level logic. So you call create light node and you pass default bootstrap if you want to connect to the Waku network or you specify your own nodes here. If you do not want to

connect to the Waku network and you want to self-host your nodes, you start the node and then you wait for the remote peer. what that looks like. Yeah. So, we see here it's trying to connect to the Waku network. So, some of these nodes are actually going to be unreachable and this is normal. We have a little warning at the beginning that says ignore these like don't think there's a problem with

websocket connections failing. So it will try to reach out to as many peers at as it can discover. So there's a bootstrap list and then those boot your node will contact that node and then it will ask for its own peers. Um so far we've been able to connect to two peers which is good. So we know that this works. The second is uh so every node that is started is started with a an undeterministic peer ID. Every time you start a node there's a new peer ID attached to your node but you can use a deterministic per ID and what how you can do that is by generating your own public private key

pair and just passing that private key as you're creating the node. If you're familiar with lip2p, this is similar to how lipop nodes work. Let's try let's see how that looks like. Yeah. So now we have two buttons here. one is just using the default behavior which is indeterministic peer ids and then or using a custom seed which should ensure that every time the node starts it has the same peer ID.

So yeah so we used 1 2 3 4 as the seed and this is the generated peer ID. It ends with W D. We can quickly verify if this is deterministic indeed. Ends with WD. Yeah. So it is the same PR ID. So you can choose to use your own um seed to make the peer ID deterministic and it is an optional step which is why it's 2.5.

Now comes the interesting part where we start defining what data structures and what message types look like when you're send when you're using Waku. So with protocol buffers, you can also use JSON. Protocol buffers are just more performant. Um but since we're building a chat app and this could literally be anything and we've seen that for example graph introduces the concept of radios which of course the data types look very different. Um for communities the data types look very different. For u the chat app, we'll have three things. We'll have a time stamp, we'll have a sender, and we'll have the message itself.

And since it's a um open uh with no recipient in mind, uh it's an open uh messaging app. There will be no recipient. But if you're building a one-on-one uh chat app, you might want to have recipient and you might want to introduce some encryption schemes uh to ensure there's a secure channel between the sender and the recipient. Then you def you then define a content topic uh that you want to use. It's similar it's analogous to uh pubsub content topics. Uh then you create an encoder and a decoder with that content topic.

For the next step, we'll build out some basic UI components and um let's see how that looks like with the UI components. But this is not um something that you want to focus on at least for this part. Yeah. So what we have here so far is initializing the Baku node with a random seed to make your peer ID indeterministic or using a specific seed.

It starts the Waku nodes uh node starts to connect to different peers. It discovers it has two available peers. This is the per ID and then this is what the end app will look like. Um we haven't implement we haven't gone to the part where we start implementing sending and receiving messages but the data types have been defined. And moving on we'll do exactly that. The first thing we'll do is we'll see how we can send messages into the network. And note this is a light client. So this is relying on light push and filter and not relay.

And it's as easy as constructing this message that comes from the protobuffers um definition. So we define a type stamp, a sender and the message. We serialize this message with the protobuffers. And this is an optional step. Um so you use the serialized message and you call the light push. API you pass it the encoder that you had created and the payload which is the serialized proto buffer and that's it. So this is how many lines probably less than 20 for the entire thing.

Seeing how that looks like. Let's actually do it um with filter so that we can actually see also receiving messages because sure we'll be able to send messages into the network but we can't see uh how they're received. Now to receive messages with filter we use the subscribe API on the same node object and we pass it the decoder that we had created. So three steps ago we created an encoder and a decoder along for the content topic. So we use this decoder. We pass it to the subscribe function and then there's a callback method and the callback returns a message. You decode this message because it's an it's a

serialized protobuffer um message. You deserialize it and you construct it into your own data type and then you of course um store it in your UI. At this step you can now send messages and receive messages over Waku. So subscri the subscription and the call back can be as long as you want or as short as you want. to see how that looks like. All right, we start the node

and I've enabled some debugging logs here so you'll see what's actually happening. All right, we have two connected peers and we have let's send let's try sending a message. Yeah, we receive the message. We send the message with light push. We receive it with filter right here. And this is all done without any servers in place. I think we missed the part where we start

to see this um as obvious, but there is no servers that are responsible for um communicating your message. It is all done in a peer-to-peer way. The next step is introducing querying historical messages. Let's say I have a peer here that isn't online, but I still want to see what messages were sent when I was offline because these messages are only received with filter when the node is online. So, Let's see if we can try to retrieve the

message that we just sent that was um hey from HCC. So we'll go to the step that implements store and implementing store is also a couple lines of code. It's similar to how filter is uh filter as an API is used. you use um there are a couple of methods available. Um so there's a callback based approach, there's a promise based approach. For this we'll use a call back based approach. So we'll use query with

ordered call back. We'll pass the decoder and then the call back itself. And along with that you can specify some options. You can have a time filter, you can have pagenation. And this is essentially again um 10 15 lines of code. Let's see if we can get the node to retrieve our messages. I also sent some earlier this morning. So maybe we can retrieve that. So now the UI has a little button that initializes store on request. So we'll click that button and it'll try to load historical messages. All right, we'll try that again.

You now have three beers or two again. All right. Classic demo where something breaks. But ideally what what happens here is um you retrieve all the messages that were sent out in the into the content topic and you see u those messages here and all peer-to-peer networks are inherently unreliable and there's a lot of work being done to make these systems reliable but they're not perfect.

For the next step, we'll assume that store works. We'll add encryption. And encryption in Waku is um we're a bit short on time. So we'll run through it. Um encryption in Waku is application layer. So while you while Vaku suggests some u encryption schemes that you can use, it is up to the application what encryption schemes uh they want to use. For the um use case of this demo, we have implemented can you see that? Yeah. All right. Um bigger. All right.

So we've we've written two encryption schemes. Uh we've implemented two encryption schemes. one's ess. So with the way asymmetric works is you generate your key pairs, you copy this key pair and then you use this key pair in your recipients um uh tab and this forms a secure channel so that only the messages that are sent with that this particular private key are can be decrypted by that recipient.

Similarly for symmetric which is more so um used in um uh which is a different way of um implementing um the encryption scheme. Let's actually just quickly see how that looks like. So we So we have one pier that exports these keys. We we'll go here and we'll we'll use the peers public key that that we exported here and then we'll paste it

here and then that's not it. Yeah. So this is the hex for that pier's public key and this exports and then similarly we'll export this guy's public key and imported here. Now this forms a secure channel. So both of these messages that will be sent um will be encrypted by sees which is the asymmetric form. And you do the same. You just send a message like it's nothing. And no other recipient except this recipient will be able to understand what the message was. And you can use any encryption scheme here. But uh we have some that you can use out of

the box uh within the library. So it works in a similar way. You generate a private key. Um then to get a public key you call this function and you create an encoder and decoder respectively with these private keys and public keys. Similarly for the symmetric encryption and the last step is just enabling and disabling waku logs. Um you can do that by setting a key value pair in local storage which says debug and the value can be waku star. This enables all logs in your system.

and also using custom nodes uh if you do not want to rely on an existing fleet of nodes. So the next exercise if you're still interested is can be adding a group chat. Uh so we already have a group chat. So you can perhaps segregate that into rooms where um the same application can now have the concept of rooms instead of just all messages being sent and received by the same person. Perhaps reactions or make it a PWA. There are some resources here uh that you can refer to. Waku.org is the go-to website.

And that's it for me. Thank you. I don't know if we have time for maybe two minutes. Maybe anybody has any questions. Yes. Or is it sorry is this on mainet? It is on test net. Uh it is the RLN part is on cipoleia uh test net but at the same time Waku is completely offchain. If you do not want to use the RLN which if you do not want to use spam prevention if you're certain that your end consumers will not be inducing spam you can use u the network you can self host your network or be completely chain agnostic so if I self host my network can I have

a closed source of peers that is not connected to the rest of the work theoretically you can yes so that's exactly what rail gun does rail gun has its own private uh network um if so if you're certain that you want to maintain the entire infrastructure. You can self-host your network. Everything's open source at the end of the day. Okay. Thank you very much. Of course. Thank you for the question. All right. Thank you. Thank you.