by Doug Petkanics

Livepeer’s mission has always been to build the world’s open video infrastructure. By pairing the best free to use open source video software, with an optimally cost effective, maximally reliable, and infinitely scalable infrastructure network, Livepeer has the opportunity to be the backbone for the future of video over the internet. The path to achieving this ambitious mission is a worthwhile challenge, and being that the project is 7 years in the making, it is worth checking in on the progress and next steps to move closer to this vision.

The Livepeer ecosystem has evolved tremendously since the last time the project’s technical roadmap was revisited two years ago. At the time, the priorities could be categorized as:

1. Driving Demand through wrapping Livepeer within robust products that the market accepted to meet their video streaming needs.

2. Expanding the Core Technology through adding additional capabilities to the Livepeer Network.

3. Ushering in Decentralization, through layer 2 scalability, governance, and more.

Focus areas one and three saw incredible progress through the evolution and growth of the Livepeer Studio hosted gateway and the many applications built on top, as well as the Confluence upgrade which migrated Livepeer to L2, onchain decentralized protocol governance, and the Delta upgrade introducing a community governed onchain treasury. Capability expansion was achieved through the Livepeer integrated software like MistServer, and hosted products like Livepeer Studio and MuseMe, which added these types of capabilities - however attempts to add this natively into the Livepeer network remained in R&D mode.

Now that it’s been a couple of years, and the project has reached its next level of maturity through a growing Studio gateway product, and sustainable community funding available through Delta, it’s time to revisit the key technical opportunities that the community can consider taking on with urgency in the coming months. There are three key themes that are critical for Livepeer to achieve its vision.

1. Increasing the pace of innovation by making permissionless development on top of the Livepeer network easier for developers.

2. Adding media processing capabilities to the network, enabling node operators to earn economically and play a key role in the future of AI-based video compute, content ingest, delivery, and additional job types.

3. Continue strengthening the network’s resilience through decentralization, achieving full trustless protocol status.

Accelerating innovation

‘Catalyst’ Broadcasting Node: A modern media server on the Livepeer Network

Perhaps the most important item on the project roadmap is shipping a well documented, easy to use, Livepeer Broadcast Node that can serve as an open media server connecting to Livepeer’s decentralized video infrastructure. This should allow video developers to build their own custom video workflows and experiences, build and scale permissionlessly against the Livepeer network infrastructure taking advantage of its superpower capabilities, run in a self-hosted environment, and have no reliance on any trusted third party or hosted service.

The original Livepeer Broadcast node, go-livepeer started in broadcast mode, is very limited in its interface and video capabilities. It interacts with the Livepeer network for transcoding video segments, but doesn’t do much else. More mature pieces of software built on top, either bundled many video capabilities behind hosted services (like the very useful Livepeer Studio), or integrated with Studio itself for transcoding (requiring an account and trust in a service provider). A better path would be for a great open source media server to integrate directly with the Livepeer network, enabling a full video development suite, in an open and trustless environment.

There has been an early attempt at this project via the Livepeer Catalyst project, which combines MistServer with go-livepeer to serve as a permissionless gateway to the Livepeer transcoding network. Today, Catalyst forms the building blocks for the Livepeer Studio API, however as a released node replacement, it remains undocumented. The rapid pace of Catalyst development has meant quickly moving API targets which are challenging to rely on in a production application. One consumer app, Xenon, did host and built directly against Catalyst software despite the challenges of it being a moving target.

It’s time to get a first version of the Catalyst API packaged up, documented, and shipped to the community in an easy-to-use way. This should lower the barrier to innovation in any self-sovereign use case, and lead to far more stakeholders building directly on the network. Catalyst’s development is being stewarded by the Livepeer core engineering team and is seeking community contributors in these early stages of development to provide feedback and testing for a 1.0 release.

For this purpose, we have launched a technical working group that meets bi-weekly called the Livepeer Catalyst Hackers Group. The CHG held its inaugural meeting on Friday, October 27th. The session included a live demo on how to run your own fully-equipped broadcast node locally on your laptop, guidance on how to hack on the node itself and how to add new features that span all the way from low-level video processing to front-end development on the dashboard. Catch the replay here. The next CHG session is on Friday November 10, where will continue to discuss the future roadmap with the goal of Livepeer community collaboration on a full-featured broadcast node for the next generation of decentralized video.

Building the future of onchain media

There are undoubtedly new primitives that are unlocked in video when you combine media with blockchains. These touch on the areas of video verifiability, provable content ownership, and collaborative media creation, especially when paired with the impacts of generative AI. Unlike traditional centralized video infrastructure, Livepeer’s blockchain roots, open innovation environment and aligned web3 partners and community make our ecosystem poised to lead the next wave of disruptive onchain media. For example, Livepeer is leading R&D on Verifiable Video to explore blockchain based solutions for content authenticity and provenance. Through this work we are actively participating actively in industry working groups such as the Content Authenticity Initiative to ensure that we’re at the forefront of the collaborative experiments and standards emerging in this new world.

Prior innovations in blockchain applications have emerged from simple examples that inspire a wave of new developers and entrepreneurs to adopt breakthrough concepts (think of Cryptokitties inspiring the NFT wave or Synthetix inspiring DeFi incentives through farming). The Livepeer ecosystem should be a thriving hub of these types of experiments as it relates to the future of media. Here are just a few ideas:

• A 24/7 onchain generative video “channel”, co-owned, with content derived from those coordinating onchain inputs.

• Onchain coordinated post-production and distribution incentives. Video creators should be leveraging and giving co-ownership to their community members who do the work of clipping highlights and sharing them out on social, driving awareness back to the main content and growing the communities.

• Community growth video CMS tools. Countless decentralized communities want to use daily live streams and archived video content libraries to engage with and grow their communities. The traditional CMS’ aren’t cutting it in a world of wallet-based identities and user owned content.

The Livepeer Grants Program is also committed to accelerating and funding innovative experiments that push the boundaries of what’s now possible, and to inspire other builders to incorporate these themes into full fledged products. There are a range of financial supports available to committed teams and individuals looking to develop and showcase inspiring projects. Have an idea for a grant that you’d like feedback on? The #grants and #developer-lounge channels of the Livepeer Discord are a great place to start.

Expanding the Capabilities of the Network - The Road to AI Video Compute

Since its origin, the Livepeer Network’s bread and butter has been performing various flavors of video transcoding. Despite all the capabilities of the surrounding software for video development, the node operators are primarily performing one function - receiving a video segment in, and transcoding it into different outputs. It’s long been dreamed that the network can perform additional job types, and in R&D it has even been used to do various forms of AI based video compute - scene classification, content type detection, closed caption generation, object detection in video, and more. Recent capabilities in AI, and the explosion of demand for GPU based AI compute, mean that Livepeer could be the leading video compute network for things like generative video, deep fake detection/generation, realistic avatar generation, and more.

In addition to those compute capabilities, the Livepeer software is used in other elements of the video pipeline, from ingest, to origin services, to content delivery networks, and in interfacing with storage providers and indexing services.

It’s time to take the concrete steps to bring some of these capabilities to the node operators and network itself. This will open up more revenue opportunities for Orchestrators, more products built on top of the network, and more independence from third party services. The two areas required to push forward on the roadmap to get there are job type abstraction, and security+verification research.

Job Type Abstraction

As mentioned above, currently the Livepeer protocol and client software supports transcoding. Node operators register on chain, advertise a price they’ll charge for transcoding offchain, and understand the security + verification process for confirming a transcode job was done correctly.

Job Type Abstraction is about removing the assumption of transcoding into the Livepeer protocol and software, and instead abstracting it out so that nodes can register on chain for, and perform, multiple job types. Imagine that onchain nodes are advertising a [job type, verification function] pair that defines the type of job and verification function that they’ll subject themselves to for that job to enforce slashing in the case that they cheat. The verification function is necessary to give confidence and security to the user in a trustless environment. Offchain, they can give an independent price for the different job types that a Broadcaster requests. And the client software, for both Orchestrators and Broadcasters knows how to pass job inputs/outputs and micropayment tickets around for these additional job types.

The protocol goes from being just a transcoding resource allocation protocol to an abstract compute and services protocol, where users can coordinate and leverage any service provided by O’s on the network, as long as they are both using clients that support that capability. It is strongly suggested that the ecosystem retain focus on video use cases initially, because the productization chain of going from capability on the network all the way up to usable product and go to market is long. A video-focus is mission aligned, and will attract a lot more help from existing developers and design partners. In the long term these job types can even expand beyond video compute and services.

A recommended first step here is identifying and building a spike of one end to end additional job type that fits nicely with existing transcoding requests, such as closed caption generation for videos. It would be easy to embed the open source Whisper model into a client spike to support this, and one could quickly hack the protocol and payments changes to demonstrate this capability. We will learn a lot by building, seeing, and using the spike, and we can move towards actual protocol LIPs and production client support from there.

There will be an open contributor opportunity for a small team or individual to lead this work. Please join the #developer-lounge channel to introduce yourself and discuss if you’re interested.

Security + Verification Research

Full verification on video transcoding so that it can be trusted in a decentralized and permissionless environment has been an ongoing area of research since project inception. A Truebit style verification game works for deterministic CPU transcoding, but is not directly possible in a non-deterministic GPU environment. As such the fast+full verification framework is in place to give users a high degree of confidence that their segments of video were transcoded correctly, but the “full” half of the equation still needs more research to be deployed in a slashing-enabled environment.

Going from transcoding to additional job types will require even more research and solutions when it comes to verification. And the bounds of what “verification” means likely needs to move beyond predictable verified compute, to instead take into account things like slashable stake deposits, reputation, and governance based mechanisms like judiciary panels to weigh in on disputes in the case of clearly presented evidence of misdeeds.

The existing body of research and potential paths gets a bit technical for this post, but needless to say, that this track pairs hand in hand with job type abstraction, and working groups should form and be expanded to push this forward in the near term.

Trustless Protocol Status Through Decentralization

Livepeer has long been one of the most decentralized DePIN and resource allocation protocols, and has made significant progress on its Path To Decentralization since launch. Given its decentralized governance, widely distributed stake, fully decentralized node operator network, and community governed treasury, it has all the building blocks in place to take the final steps towards reducing dependence and single points of failure on any small groups or entities.

1. Binding on chain execution of protocol upgrades and parameter updates via governance.

   These types of upgrades are governed by the community through the LIP process, but require the security committee to actually execute the upgrades. With the introduction of the Governor framework for the Delta treasury management, the similarities in governance mechanism over the treasury and LIP process, and the capability of that framework to execute any onchain action it has permission to, there exists a possible straightforward path to this framework being used to execute binding protocol upgrades on chain, removing the role of the security committee in this process.

   Of course the actual governance mechanisms should be debated and potentially modified should the community wish to evolve governance itself as well, but the scope of governance mechanisms is a bit beyond the technical scope of automating execution here.

2. Introducing direct community governed control over the security committee process itself. Once again, the committee’s power is ratified via LIP-19, and it operates according to a social contract commitment. This includes a commitment to modify its own size or processes via the LIP process. However it would be more trustless and decentralized if the governance process itself was binding on the security committee - including adding or removing keyholders, modifying signing thresholds, or removing the committee altogether. The community could also consider a formal election process, like recently seen in the Arbitrum ecosystem, should it determine that to be a valuable step in decentralized control over the committee and protocol.

Engaging in the processes towards decentralization is open to all through Livepeer’s LIP process, and the #governance channel in Discord is a great place to get involved in the discussion.

A Hotbed of Video Innovation

Some items on this list of themes are high priority and instantly actionable, such as shipping Catalyst as an open media server Broadcast node. Others will be ongoing and neverending aspirational objectives, like perfect verification on infinite job types on the network. Ultimately, they each play a key role in helping the ecosystem surrounding the world’s open video infrastructure become the hotbed of video innovation that it has the potential to be.

There’s absolutely no reason that developers shouldn’t choose to use the best free and open source video software, run on the cost effective, scalable, and reliable video network that is Livepeer, should it meet their product and infrastructure needs. If you’re interested in getting involved and helping to bring this vision to reality join us in our community Discord today and introduce yourself!