Part of Eyevinn Open Media Supply Chain Solution
A solution for content preparation for Video On Demand (VOD) streaming fully based on open web services in Eyevinn Open Source Cloud. An open web service is based on open source giving you the option to host this entire solution in your own premises or cloud infrastructure. This solution features:
- Automatic generation of subtitles.
- Transcoding and creating VOD package for streaming in HLS and MPEG-DASH.
- Automated process triggered when a file is uploaded to a bucket and result stored in another bucket.
This repository contains the media supply chain orchestrator that will drive this process.
- An Eyevinn Open Source Cloud account on Professional plan.
- NodeJS 20+ installed.
- Minio CLI installed.
- An OpenAI account and OpenAI API access (for automatic subtitling).
We will setup and use the following open web services. Make sure you have activated these services or have remaining services available on your subscription plan:
For creation of the instances we will use the Open Source Cloud CLI in this example. If you don't want to use the CLI you can use the Open Source Cloud web console instead.
For deployment of this orchestrator in Open Source Cloud we need additional three services:
- Web Runner
- Applicaton Config Service
- Valkey key value store
Obtain your OSC Access Token (your personal access token) from the web console in Settings/API page. Copy this value to the clipboard and save it in an environment variable called OSC_ACCESS_TOKEN.
% export OSC_ACCESS_TOKEN=<personal-access-token>Navigate to the MinIO service in Open Source Cloud web console and create a service secret called rootpassword storing the password for the MinIO root user. It needs to be a mix of lowercase and uppercase and contain a number.
Create a MinIO server for the storage buckets we will use.
% npx -y @osaas/cli create minio-minio mediasupply \
-o RootUser=root \
-o RootPassword="{{secrets.rootpassword}}"
Instance created:
{
name: 'mediasupply',
url: '<minio-server-url>',
...
}The URL to the MinIO server (<minio-server-url>) is the S3 Endpoint that will be referred to later in this guide. Create an alias for convience using the MinIO client.
% mc alias set mediasupply <minio-server-url> root <minio-root-password>Now we can create three buckets we will need.
% mc mb mediasupply/input
% mc mb mediasupply/abrsubs
% mc mb mediasupply/originNavigate to the Subtitle Generator service in Open Source Cloud web console and create the following secrets:
openaikey- Your OpenAI API keyminiopwd-<minio-root-password>
Create a Subtitle Generator instance using the Open Source Cloud CLI (or the web console).
% npx -y @osaas/cli create eyevinn-auto-subtitles mediasupply \
-o openaikey="{{secrets.openaikey}}" \
-o awsAccessKeyId=root \
-o awsSecretAccessKey="{{secrets.miniopwd}}" \
-o s3Endpoint="<minio-server-url>"
Instance created:
{
name: 'mediasupply',
url: '<subtitle-generator-url>',
...
}Navigate to the SVT Encore service in Open Source web console and create the following secret:
miniopwd-<minio-root-password>
Create an SVT Encore transcoding queue.
% npx -y @osaas/cli create encore mediasupply \
-o s3AccessKeyId=root \
-o s3SecretAccessKey="{{secrets.miniopwd}}" \
-o s3Endpoint="<minio-server-url>"
Instance created:
{
name: 'mediasupply',
url: '<svtencore-url>',
...
}Navigate to the Shaka Packager service in web console and create the following secret:
miniopwd-<minio-root-password>
Now we can deploy this Media Supply Chain orchestrator that will automate this supply chain process.
Create an Application Config Service for storing the orchestrator configuration. Start by creating a Valkey instance to be used by the config service.
% npx -y @osaas/cli db create valkey mediasupply
redis://<valkey-ip>:<valkey-port>Then create the config service instance
% npx -y @osaas/cli create eyevinn-app-config-svc mediasupply -o RedisUrl="redis://<valkey-ip>:<valkey-port>"
Instance created:
{
name: 'mediasupply',
url: '<app-config-url>',
...
}Navigate to the <app-config-url> and create the following configuration variables.
S3_ACCESS_KEY_ID=root
S3_SECRET_ACCESS_KEY=<minio-root-password>
S3_ENDPOINT_URL=<minio-server-url>
ENCORE_URL=<svtencore-url>
SUBTITLE_GENERATOR_URL=<subtitle-generator-url>
To ensure that the orchestrator maintains state during a restart you can provide a URL to a Redis or Valkey store. Create the Valkey for this purpose first.
% npx -y @osaas/cli db create valkey orchestrator
redis://<orchestrator-valkey-ip>:<orchestrator-valkey-port>Then add the following to the configuration:
REDIS_URL=redis://<orchestrator-valkey-ip>:<orchestrator-valkey-port>
If you want to trigger a custom workflow you can provide a URL to a workflow definition file. Add the following to the configuration.
WORKFLOW_DEFINITION_URL=https://raw.githubusercontent.com/Eyevinn/media-supply-orchestrator/refs/heads/main/workflows/vod-transcribe.yml
For deployment of this repository you need to first create a GitHub personal access token. Follow the instructions in this guide to create it.
Navigate to the Web Runner service in Open Source Cloud web console and create a service secret ghtoken to store the GitHub personal access token and another service secret osctoken for your OSC personal access token.
% npx -y @osaas/cli create eyevinn-web-runner mediasupply \
-o GitHubUrl=https://github.com/Eyevinn/media-supply-orchestrator \
-o GitHubToken="{{secrets.ghtoken}}" \
-o OscAccessToken="{{secrets.osctoken}}" \
-o ConfigService=mediasupply
Instance created:
{
name: 'mediasupply',
url: 'https://eyevinnlab-mediasupply.eyevinn-web-runner.auto.prod.osaas.io',
...
}Now the media supply chain orchestrator is running and listening on events on the input bucket.
Download the example file VINN.mp4 and upload it to the input bucket to try this out.
% mc put VINN.mp4 mediasupply/input/Setup the open web services as described above (except for the orchestrator) and create the following environment variables pointing to the Open Source Cloud resources created.
% export OSC_ACCESS_TOKEN=<personal-access-token>
% export S3_ACCESS_KEY_ID=root
% export S3_SECRET_ACCESS_KEY=<minio-root-password>
% export S3_ENDPOINT_URL=<minio-server-url>
% export ENCORE_URL=<svtencore-url>
% export SUBTITLE_GENERATOR_URL=<subtitle-generator-url>For local development you need to use a tool like ngrok to create a tunnel to your local computer for the callbacks.
Then install NodeJS dependencies.
% npm installCreate a tunnel using ngrok.
% ngrok http http://localhost:8000Copy the forward URL returned by ngrok and set that as PUBLIC_BASE_URL environment variable.
% export PUBLIC_BASE_URL=<ngrok-url>Then start the orchestrator.
% npm startSee CONTRIBUTING
This project is licensed under the MIT License, see LICENSE.
Join our community on Slack where you can post any questions regarding any of our open source projects. Eyevinn's consulting business can also offer you:
- Further development of this component
- Customization and integration of this component into your platform
- Support and maintenance agreement
Contact [email protected] if you are interested.
Eyevinn Technology help companies in the TV, media, and entertainment sectors optimize costs and boost profitability through enhanced media solutions. We are independent in a way that we are not commercially tied to any platform or technology vendor. As our way to innovate and push the industry forward, we develop proof-of-concepts and tools. We share things we have learn and code as open-source.
With Eyevinn Open Source Cloud we enable to build solutions and applications based on Open Web Services and avoid being locked in with a single web service vendor. Our open-source solutions offer full flexibility with a revenue share model that supports the creators.
Read our blogs and articles here:
Want to know more about Eyevinn, contact us at [email protected]!