I'm bad at having plans, so I'll write everything down here I guess.
So uuuh building a live streaming platform, seems like a cool idea to me. I just have to find a cool stack, some servers and we're ready to go.
Okay but which languages for the stack? How do I scale the thing? I need a mockup for the front-end...Where the fuck do I even begin?
Okay, so far I was pretty lost. Right now I'm still a bit lost, but I've come up with a plan :
- First, design the server that will handle the incoming video streams and redistribute them. Some details later
- Second, design an API (login/viewers/followers/etc)
- Third, design a chat system
- Fourth, design the front end.
Seems like a simple and "easy" plan, but it'll take some time. I might get people to work with me on this project. I still need to figure out a lot of things.
Anyway, let's talk about the streaming (RTMP) server.
Inputs, outputs, too many people and how to avoid a general collapse
I don't think there will be many people when the platform will launch. But we still need to consider an important thing : nginx with the rtmp module can consume a lot of cpu, even with only two streamers on the same instance.
I mean, yeah of course it depends on the CPU of your instance, but I still have to do estimations on the costs.
So...the idea would be to have multiple instances of nginx-rtmp? The latter being easy to deploy through Docker.
My current idea would be using a load balancer for high traffic stuff, and/or a Kubernetes clusters with nginx-rtmp instances deployed.
I stumbled upon this really interesting article by Chen Fisher : "Customizing Kubernetes DNS using Consul to scale Nginx RTMP", which explains the Kubernetes cluster technique.
There's also this very good talk "Building an Open-Source based audio streaming platform" by Maxime Bugeia from Radio France, presented at FOSDEM.
I had to get something out of my head visually, so here's something :
W I P