scaling decentralised real-time communication infra at huddle01

Transcript

[Music] Um, what am I going to talk about today? Scaling realtime communication. Wow, sounds complicated. Yes. So I work at a company called Huddle01 and what we do is we build decentralized edge networks for high bandwidth workloads which basically means we build um software infrastructure for folks to build realtime applications which would include pubsub messaging um video conferencing live streaming anything low latency live media or data transfer across continents um that's what we do in a decentralized fashion and hoping to

brief you on how the network was designed and the decisions we kind of made to make this work. Um so the core components are uh there's the media node uh registry nodes and orchestrators. Media nodes are basically worker nodes which route and manage WebRTC traffic. So these are what the clients would connect to and uh media nodes kind of take care of are like worker nodes which kind of route WebRTC traffic. Registry nodes are uh kind of gateway or bootstrap nodes for the network to which all media nodes onboard to and um they use lip p2p to like uh maintain a

distributed ledger of all nodes. So um that's where the fun stuff happens and they're also tasked with not only onboarding the media nodes to the network but also ensuring quality. We have stuff like proof of bandwidth and quality service asurances which we will get into later. And orchestrators they coordinate connections between nodes and clients. So there is a fairly complicated diagram on the right which kind of talks about how everything kind of works. We use uh um protocol schema proto schemas and gRPC to maintain um clear definitions of how each implementation should work. So people are free like free to make their own

implementations. I'll go into a bit of depth of what each component does. So uh a media node is basically a wrapper around a WebRTC SFU which is something which allows you to scale WebRTC connections across multiple clients at once. Um it also has a few modules like an API RPC QoS module. uh this is what our community and infrastructure partners run um on high throughput high bandwidth networks and this is what kind of powers the network. So very high specs like 1Gbps up and down link or minimcs for you to run it and they have

to be publicly addressable which means either a static IP address or a dynamically changing one but they have to be publicly addressable. uh this is what kind of powers whole of the network and yeah moving on to orchestrators. So uh media nodes are like worker nodes. They have no logic in them. They're their sole goal is to take packets, duplicate and route them. So because we wanted people to be able to define on depending on their use case on how they want to use this architecture on the left you'll see uh the orchestrator design is where uh you're trying to do a live stream. So

what you could do is you could arrange your media nodes in a fanout manner where the streamer is connected to one SFU and the SFU or the media node is cascading the whole thing out to other viewers. So this is more of a fan out approach. On the right you would see a meshed topology approach where if you're doing something like video conferencing, you would ideally want the client to connect to the closest node and then um hope for the network to take care of the communication because uh once you're in the Hudlo one network, we try and optimize and use private networks which means no public network hops and reliable packet transmission. So that

would kind of lead you to a mesh type configuration here. So orchestrators are userdefined. So people are free like are free to design however system architecture depending on their use case which kind of allows for a fairly usable and dynamic way to approach um media transmission or data transmission u through the best possible path. Um okay so we're talking about networking. Networking is pretty complicated. You play around with uh BGPs or border gateway protocols. Uh internet is not perfect. So you have to kind of take care of hair pinning and back haul. So what is hair pinning? Um I have a very um fun diagram which I was just making

there. Um so this is Canada. So imagine you have a friend friend in Canada and friend in the US. When you're trying to send packets over the public internet, it's going through public internet hops where you see you see a very convoluted path the internet might take and you have actually zero control over it. But when you want to do low latency stuff, what web two or the cloud does is they have their own data centers with their own private lines and private IPs through which they control their own BGPs. So the whole point of Arlo one network is if people are hosting their nodes on multiple clouds or bare metals, we learn all of that data and we try and

ensure that we are using those same private lines to allow for low latency communication. So this was this would be kind of the scenario where obviously this is theoretical there are ocean cable fibers running there but you have the lowest latency possible um which you would get on cloud but is hyperdemocratized because it's decentralized and being run by the community and infrastructure providers um so yeah moving on to registry not so this is where uh I have a few P2B folks here this is where the fun magic happens So media nodes don't gossip to each other. Like I said, they are worker nodes. So all of the consensus and ledger maintenance needs to happen

happens on a smaller set of nodes called registry nodes. These are what take care of the health of the network by doing quality checks on the media nodes through u quality assurance and bandwidth is on the next slide. Um and they maintain a DHT of nodes because the number of media nodes are in thousands. uh you need to maintain a ledger where the orchestrators can go and query which country or which continent has which nodes and you get the information dial the nodes directly and connect to them.

Um they're also responsible on um maintaining the quality scores and hence u also responsible for delivering incentives and rewards. What is quality of service? Um this is a fairly interesting challenge we had to face is because we depend on highly reliable high bandwidth nodes and they have to constantly give out the same throughput and performance. We have to run randomized uh extensive tests um between nodes to check if they're and kind of push them through congestion to check if they're actually supporting their claims which they're making on the

bandwidth availability and the latency availability they have. And think of this as an over complicated way to do a speed test on your browser. But the issue is I don't uh the network does not have access to UCLA or speed test because that would be like a centralized provider. So what you do is you make media nodes fight against each other and uh kind of let the best man win but then you aggregate all of these scores and uh build like a reputation system and use that to allocate work and rewards. Um yeah uh this is about the demand side.

So today you will see a lot of storage providers provide SDKs for folks to build but I think we've been product first since the day one. We've already had SDKs in Unity, React, Flutter, JavaScript, um even C I think. Yeah. Um and we also have a few products built on top of it which has a few thousand users now and a few million minutes of usage. Yes. Um what's the goal of the network is to allow for anything realtime communication realtime AI is the new thing u decentralized inference etc. But at the same time, if you're trying to do

something like open AAI where you're talking to the model live, the models are deployed in the US, but they are being delivered through web RTC to you. So having a low latency communication fiber for all of these delivery use cases is pretty important. Um, like I said, because we were very demand first, we had the SDKs ready. And you can also run your own nodes and run a private network to do the same thing. And these are some images of programs and applications engineers have built on our community um future work. So this is what I gave a talk on yesterday at L P2P where WebRTC is dying and you need a new protocol. Um the advantages for uh low

latency communication are multiffold like you have multiath multihoming uh streams and datagramgrams and you can prioritize your streams and now that you have web transport available um it unlocks a huge level of possibilities which were kind of bound to web RTC but UDP for the win. Thank you. This is all uh I'm up for any questions. Oh yeah yeah yeah forgot. So I wanted to show you all. Can we have a camera on guys? Turn on a camera. I just wanted to show you all the actual product running on decentralized infrastructure. This is our team and I think Daran is from Mumbai. We're in

France right now. And if I go to network stats that's 27 milliseconds. So there's a node being run in Europe in France I think maybe in Paris. And that's 27 milliseconds. Uh Dashan would have a very good latency because we have node in Mumbai and there are so many people. I hope they turn their cameras on. Guys, turn your cameras on. You're live. We only have like 30 seconds left. Oh, let's go. So yeah, that's that. Any questions?

I think everyone's just happy looking at faces. Okay, then. Thank you. [Applause] Okay. Nice. Thank you. Bye. [Music]