Decentralized video streaming: architecture, performance, and trade-offs

Transcript

[music] >> Thank you. And hello, everyone. My name is Levente Kis. I'm an engineer, builder, and creative thinker. And I'm a tech lead at Solarpunk, technical partner of Swarm. And in this talk, I would like to give you how traditional streaming stack works, how Swarm comes into the picture, and what we built for this event. So let's start. What is Swarm? In a nutshell. So at its core, Swarm is a peer-to-peer network of B nodes working together to provide a decentralized storage and and messaging.

There are three types of nodes, ultra light, light, and full nodes. Ultra light and light nodes providing read and write access to the network, meanwhile all the full nodes are providing the storage and the messaging protocols behind the scene. When you upload something to Swarm, Swarm chops your file into 4 kilobyte chunks of 4 kilobyte of chunks and spread them across the network. Each chunk is addressed by its hash, and nodes whose addresses are closest to the chunks stores these chunks.

Nodes forms neighborhoods and nodes in these neighborhoods holding copies of the same data. It's one way of Swarm to ensure higher availability and redundancy in the system. Every upload is requires a postage stamp, and as long as the stamp is funded by the utility token of Swarm BZZ, the network keeps your data alive. And node operators get rewarded for holding this data. Once a neighborhood proves it stores its chunks, the network pays them off. To download, you only need one thing, a client that can read from the network

and a hash of your files. Uh chunks sit in uh a tree underneath, a marker tree, so the network can locate each piece and assemble your files back to your original file. Routing chunks uses bandwidth, so nodes pay each other with signed checks from a checkbook. Swarm is not a blockchain, it's a peer-to-peer network. Just certain operations require a chain, stank checkbook operations and rewards payments. So now that we know like what is swarm in a nutshell, let's step to a typical streaming pipeline. Netflix, Twitch, YouTube, they all run the basic pipeline. Of course, their architecture

is more complex, but this is the foundation here. It starts with a capture. For example, like a tool like OBS can record video from a camera or a screen, an audio from microphone, but what we capture is a raw a raw feed, and this feed is too large to to send over the network or to do further processing. So we encode it with a codec, something like AV1 or H 264. But one quality usually doesn't fit every viewer, so the server transcodes the stream into several renditions. Full HD, HD, low HD, SD. It's When you open the gear icon in YouTube, you can see these qualities.

Uh next is each rendition is is packaged, cut into small segments, a couple of segments of each wrapped into containers like impact impact TS or fragmented MP4 and listed in a playlist. And uh because these segments are just small static files, a CDN can catch them and deliver from edge servers close to the viewer. And finally, we have the player, client pulls the segments, buffers a few segments, adapts the quality on the flight, and of course decodes our stream back to picture and sound so you can enjoy the stream.

So, what what changes if Swarm handles the storage and and the delivery instead? This is event stage streaming, the system for we built for for Devcon. On the left we see streaming actor, stream operator, infra operator and the broadcasters encoder and the middle the streaming stacked, the full streaming pipeline, the viewer enjoying the streams and the Swarm actor responsible for the storage and the messaging and let me walk through you on different scenarios where I'm going to describe you each component and their and their connections.

So the first thing is is control host and here operators, stream operators and infra infra operators interacting with our management side of things. This this is Web 2. We have an application, a single page application, a Node.js server, a DB and and here managers can interact with for example with the infrastructure, check on alerting, monitor the resources, monitor the components that we deployed, manage stamp stamp Swarm related configurations and parameters like the stamp, the funding of the stamp, the funding of the checkbooks and

of course all the required parameters for for the engine itself that does the the processing and ingestion part of the stream. Similarly we have the stream operator in a similar application, single page and a Node.js server behind the scene can manage the stream content. Similarly how you can imagine in YouTube you have the stream list and the stream items. You can create the stream, placeholder stream title, description text, schedule start and and the thumbnail and when when you're ready to to publish your stream then by pushing the publish button you can you can make alive on swarm. And for this reason, the stream

manager component it's connected to to a node. We call this the the catalog node. When [snorts] when you write this data to swarm, the stream list, the data is chunked and then stamped and and delivered on the network on these farmers chunks. And a really important swarm term, the feed, it's in a nutshell, it's a it's an address that comes from an owner and and a topic. An owner signs every every update and you get this type non-changing address that always points to the newest version of your content.

That's how the stream list that keeps changing can live on an immutable storage. And on the client side, you just want address that you have to keep poll for the newest version of your of your update. And this feed, in our case, the stream list stream list feed is where viewers find the stream when they know your public address and and a chosen name. It's like it's like your channel name. So so we have now our so we published a stream list and we have this on the client, so it's time to go live. [snorts] A broadcaster sends one stream to the stage host over SRT or RTMP protocol using a stream engine's access URL, stream ID, and stream key. A stream

management operator can provide this provide this key to us. Basically, this is our access point to our streaming server and [snorts] and authorization for the stream key and to identify the stream and start the stream ID. In our case, this is an open source project called SRS, but a full pipeline here is modular. If you do thing that you want to do ingestion, the encoding, transcoding, and packaging in a different way, then you can customize it however you want.

In our case here, SRS does this four thing, and in the end we have our transcoded stream for qualities in the form of HLS segmented streams. And with a with another service service stream uploader we can register webhooks to our engine and keep pushing these four qualities segments to to these webhooks. Stream uploaders kind of acts like a router. For every segment the uploader sends the file to to to that qualities level of B node. So we have four B nodes here, each responsible for one quality per level. It

this reduces latency and increases some availability here and these nodes upload the segments to Swarm and return their root hashes. From those hashes the uploader builds four standard HLS playlist where each playlist contains metadata for the player about how to interpret the stream and also we have all the segments in order with their root hashes so we know where to access them on Swarm. Each playlist or four qualities leaves leaves on its own Swarm feed which can be shared even before the stream starts as as feeds can exist as empty feeds.

So we don't have to update the stream list when a stream starts. The feeds are already on the client side and then this when we start broadcasting the stream just switches to live. Note that the stream is live on Swarm. Let's look how people actually watch this stream. A viewer opens a streaming client in in their browser, another single page application uploaded to Swarm and and served under an ENS. You can grab the the root hash, the hash of your bundle that you that you uploaded to Swarm and can add this content hash under ENS and any ENS aware gateway can display your site in your

browser. In our case, we use Ethereum on gateways e.link e.limo that supports swarm. And this is this is the real client cybersun congress e.limo. We were official streamers of the event. We worked together with their AV team. I was the stream manager. I made the stream. You can see how it looks like when a stream item is published. I shared them the SRT connection string, the stream key, and stream ID, and we managed to broadcast full HD stream for the whole event 8 hours without any issues.

So when when you open the site that by default the videos comes to our our own event gateway, a B node. It's basically it's a B node behind an NGINX server. And for playback we use the open source library HLS JS with our own loaders. This player reads its playlist straight from swarm, straight from the swarm feed, and it keeps everything standard that HLS JS does. It switches qualities as the bandwidth changes and keeps a safe buffer behind the live edge. This is default way to watch, but this is not the only way because the site lives on swarm and all data lives on swarm that the site displays.

So you can read the exact same stream through your own B node. In the client you can open my swarm an experimental feature that we are still exploring how this will look like in production in the future. You can add your own gateway, your own B node, or another new shiny tech an in-browser node. It's called it's a project called VIP 3. Basically a node runs in your own browser. We're trying to reduce the complexity to learn and and tunnel your own node into the browser and also leave behind even gateways because it's comes with sensorization and give you network and read and write access

straight in your browser. You can even pick different sources for each for for each features the video, the stream is the thumbnails and and the chat history and here what you get is independence if you want to take control. Viewers who connect of course is don't depend on on the gateways so if it anything goes down on our side you can keep watching the stream by design but watching is just half of the experience people also want to talk and for this reason we made our own chat implementation.

Chat runs through swarm two the client chat runs through swarm two and we are here we are relying on a feature called G sock. The client signs your message and sends it as a as a G sock chunk. It's a type of chunk that any viewer can write into and and just very in a nutshell you can mine an address with a topic that make sure it falls into a specific neighborhood on the network. In that neighborhood we we run a node that subscribes to to this address so it receives every message as it arrives.

Then we can run an aggregator service connected to our road which can pick up each message and and write it in into its own feed a feed called the chat feed that this is the the feed that every client can read. So this is how individuals writing separately on their own feed or can sending messages on their own can be aggregated together into into one single feed on the client. Because every message is cryptographically signed and even can verify that it wasn't tempered with but the only bad thing the only malicious thing that aggregator could do is drop messages or or change orders. So that's where identity management comes into the

place. And here we're experimenting with another thing, a swarm ID. By by default, uh you chat as a guest. The client creates a key, a public a private key public key pair in your browser and just simply stores in your local storage. This is a temporary identity. And uh and the message is going through our own gateway using our own stamp. So you are not the own owner of your own data. And here we started to develop and experiencing a new way of thinking about identity. You can swarm it. You can sign in with swarm ID, which manages your identity and your and and your stamp so you can own your own data with no account on our servers. Your messages

carry your key and are stored on your own stamp. Both just for your for your information about the swarm access and the swarm ID are are experimental and this is not official use case. We just wanted to give you sneak peek how portable identity and owning your own data across swarm depths might look like in the future. This is a direction that we are heavily working on. And the whole architecture in in in one picture. So compared to earlier versions, uh here here is what we shipped, bunch of features, management services, pluggable stream engine, monitoring alerting system, CICD, all of these things.

Um maybe that's worth to highlight it out is a crash resistant logic. Uh there are a lot of logic behind the scene to make it possible if something goes wrong, the internet connection is lagging, the uploader goes down, one of the quality nodes go down, we can automatically recover from from many many of these cases. And the reason why I'm highlighting this because this project is a huge milestone for us. Not too long ago, a couple of years ago, I was standing on a stage on swarm summit and the question was, can we even run media streaming? And we managed to run the very first piece of media through Swarm and here we are today we are officially streaming Web3 Events and this is a huge milestone for us.

And we did contests of tests measurements which came with some interesting numbers um a on our test tech in case of a full HD stream 1080p 8,000 bit per second two second long segments 30 fps one sec one segment is around two megabytes one dot eight six it's about 460 chunks and from the camera when it's available in your client it's an average to be precise 95 percent of time arrives less than two seconds which is kind of kind of insane to think about uh

and the latency is between 10 15 seconds seconds glass to glass average is 15 seconds and the in-browser node we are experimenting gives us some really really really promising results most of the time segments arriving less than half a second. The storage when when you stream 4 quality levels at full HD and you can see the levels 8,000 to to that eight one dot two zero dot seven then on eight hour long stream produces close to 50 gigabytes 46.5 11 million chunks and this costs you 26 BZZ on on bandwidth and for one day of

storage 11 BZZ this price is changing um I'm going to share the slides and and you can check it out yourself. And uh we are working on more features we are trying to bring more features that modern platforms has live transcripts kept on swarm audio only streams with transcoding clips you can share, search and discovery, and modern UX and and UI that that feels as familiar as using one of your favorite websites nowadays. And beyond our own stack, there is a bigger vision that we are considering to to move towards to open up the stack and and create a

new ecosystem owned by its users. Any broadcaster can publish to Swarm and one client plays them all. There are incentives for creators, viewers follow channels directly with no platform in between and nothing block them in. And and of course a proper comment system that that you can use to talk with your friends or your favorite streamers. Your channel and and and your audience and your data stays with you. Basically decentralized YouTube, decentralized Twitch early days and there is no date on it, but this is one direction that we are heading to.

And you can find all the resources here, documentations, earlier versions, current versions, guides, everything is open source and projects that are close to Swarm. One of them is like Freedom Browser. I highly suggesting checking them out. They're doing some really really cool things, running node on your phone, streaming straight from your phone, and they have their own native browser that includes a B node. Thank you very much.

>> Okay, what a great talk should I say. Let's move straight to the center of the stage because this is the place where the questions should be answered. >> Perfect. >> [laughter] >> Okay. We definitely have like first of all so many people in the audience, you are absolutely amazing and you, Lavant, you are amazing attracting so many people here. We have the most votes for the question number one over the whole segment today. And the question is, fetching data from decentralized storage protocols like Swarm or IPFS takes quite a while.

How does the video streaming work seamlessly without any latency? And by the way, the questions are already here, so you can look at them. >> I mean, there is latency. As I mentioned, it's between 10 and 20 seconds, on average 15 seconds. And I mean, it's done by Swarm. As you can see, we made the application side of things, and the Swarm protocol itself handles everything. How it's done under 15 seconds? Good coding, I would say, and good architecture of Swarm.

I'm I'm not sure if I'm capable of in detail explaining how the protocols does this this seamlessly. I highly suggest you that you're interesting on the mathematics and the proofs checking out the book of Swarm, which contains everything. It's basically the the white paper, a little bit more than a white paper of of Swarm, or talking with Victor sitting right there, the founder of Swarm. >> Okay. Thank you for that. And moving to the next question, which is also pretty upvoted.

How did you align HLS video segment sizes with Swarm's native content addressed chunk limits to minimize re retrieval re Jesus Christ, I will not pronounce that retrieval overhead? How videos work without any latency? So everyone cares about latency. >> Yes. I'm I'm not sure what why why the question is around without latency because I mentioned that it comes comes with limits. It's come with latency. I mean, maybe you are thinking about why the video can play continuously. And so the network is peer-to-peer, so the response time is not deterministic. What matters that

95% of the times the chunk the the segments are are coming in in time. And the last of the 5% at time, the player just handles everything seamlessly. It's buffered. Also also it's I guess it's worth to mention again the the checkbook. By default on Swarm, you have free allowance, which is really good for everyday usage. You can you can access your files, you can browse, but if you need premium bandwidth, which opens up the bandwidth for use cases like decentralized streaming, then you have to pay XBZZ and you're paying for the full node operators giving you the the premium bandwidth.

>> All right. Good. And moving up next. And well, I mean, I also want to ask this question. What would what would it take for a traditional Web2 streamer or viewer to switch to Swarm stream? >> At the moment, I would say it's kind of technical. You have to understand the open source repository and run the full stack on your own. If you are an engineer with agents, it's kind of easy nowadays. I would say that if you read the documentation, under a couple of hours, you can understand everything and spin up the infrastructure. If you're someone who doesn't want to hustle or you're not an engineer, then you can ask anyone who can do this for you and they can become

the provider. This is kind of the part where I guess the opening up the ecosystem makes sense because there we are I'm imagining a system where multiple providers can exist in the ecosystem. Some of you want to spin up your own stack and stream into the side, but maybe some of you trust your friends who can do the same thing and they can become your own provider. And maybe bigger providers can emerge emerge, which can give you the the the providing side for some fees.

Basically a full incentive ecosystem around providing and and viewing streams on through these decentralized There are many many use cases. It's under under heavy research what what the needs in the business and yeah follow us and you will see the journey that we are on. >> [snorts] >> Okay, staying on the topic of business we move on to the next question. Will there be any revenue for streamers? >> Yes, on the opening side I would say yes. I I definitely want to add features where we where we give support for subscription, donation and even more.

I mean I'm more of a technical side of I'm I'm more better on the engineering side of things and on on business with we have our business dev team that coming up with these questions and and also you as well uh when we introduce a technology or a solution like this and opening up the ecosystem people will come with with needs and demands and we are listening on feedbacks. So if you have ideas let us know and we you might see it. >> Okay great. I see that questions are coming coming and coming. Unfortunately we are over time already. So guys feel free to catch our speaker right after the call and let's give the huge