At Livepeer, we developed Livepeer Pipelines. Livepeer Pipelines does for livestreams what OpenArt Workflows has done for image generation. It takes ComfyUI Workflow and applies it in real-time to your video stream. You can seamlessly customize and extend AI-powered transformations in a user-friendly way, while Livepeer Pipelines takes care of the rest. Think of it as a cloud service for real-time video transformations but here’s the twist: Livepeer isn’t just cloud-based; it’s open-source and decentralized, though it's material for a separate blog post. For now, let's dive into why you would even apply AI to your livestream?

Use Case 1: Change Yourself

Want to create a new identity online? You can stream your webcam and transform yourself into anyone you want, like Bob Dylan, Donald Trump, or even Mona Lisa! But that’s not all - you can even enhance your own look to create a unique virtual avatar that you can use for streaming services like Twitch, YouTube Live, Twitter Live, and more. We’ve made it easy to use models like LivePortrait and StreamDiffusion, along with LoRA datasets, and you can customize your avatar by creating a ComfyUI workflow.

Sample Pipeline: Live Portrait

Use Case 2: Trace Anything

Imagine this: you’re watching a football match on your screen, and you want to follow a specific player or the ball. You just type in your natural language, like “Track the ball” or “Follow the player with the number 5,” and voila! The output stream automatically highlights or zooms in on the object you’re interested in. And guess what? You can already do this with Livepeer Pipelines by combining the power of Florence-2 and SAM 2 models. But wait, there’s more! We’re also working on adding LLM models, which will open up even more possibilities. For example, you could use them to create automatic comments during sports events or summarize livestreams in real-time. It’s going to be amazing!

Sample Pipeline: Segment Anything + Florence 2

Use Case 3: Create Art

By combining StreamDiffusion models with the right input, prompt, and LoRA optimizations, we can create animated art. Take a look at this example: we used WebGL Fluid Simulation and prompted “cat” into the StreamDiffusion model.

Sample Pipeline: Stream Diffusion

Use Case X: Push the Envelope

There are no limits! We want to make Livepeer Pipelines the go-to tool and repository for creators to share their video-related ComfyUI pipelines.

You can open any pipeline at Livepeer: Explore Pipelines and experiment with its parameters. Or, if you want to tweak ComfyUI nodes, you can import the entire ComfyUI JSON Workflow into your local Hidden Switch ComfyUI and play with it like any other ComfyUI pipeline. Imagine creating a workflow for a frame and turning it into a video!