Video Support

Native Media Decoder #

Prysm comes with its own Media Decoder. You can read more details about the native decoder here.

Unreal Engine Bink Media Plugin #

As of UE 4.27, the engine comes with the Bink Media plugin by default. Prysm and Bink can be utilized together in a fairly straightforward fashion with minimal code.

Implementation #

The easiest way to add a Bink video playback to your UI, is by using a Live View. The general idea is to:

• Have a Render Target
• Create a <video> element in your HTML page, with an Unreal asset source path to your Render Target
  • For example "TextureRenderTarget2D'/Game/MapAssets/BinkMediaPlayer/BinkVideoRT.BinkVideoRT'"
• Create a UBinkMediaPlayer object and configure its settings
• Call UBinkMediaPlayer::Draw on each Tick

Here is an example code we use for our BinkVideoMap sample:

1. Subscribe to the ReadyForBindings event

    ACoherentSampleBinkVideoPlayerHUD::ACoherentSampleBinkVideoPlayerHUD()
       : currenttime(0.f)
       , duration(0.f)
       , volume(1.f)
       , paused(false)
       , muted(false)
       , loop(false)
       , View(nullptr)
       , MediaPlayer(nullptr)
       , BinkTextureRT(nullptr)
       , ModelSyncNeeded(false)
       , VideoEnded(false)
   {
       check(CohtmlHUD);
       CohtmlHUD->ReadyForBindings.AddDynamic(this, &ACoherentSampleBinkVideoPlayerHUD::BindUI);
   }
   

2. Create and initialize a UBinkMediaPlayer object when your ReadyForBindings delegate gets executed:

   void ACoherentSampleBinkVideoPlayerHUD::BindUI()
   {
       MediaPlayer = NewObject<UBinkMediaPlayer>();
       MediaPlayer->URL = TEXT("./Movies/SampleVideo_1280x720_vp8.bk2");
       MediaPlayer->BinkSoundTrack = BMASM_Bink_Sound_Simple;
       MediaPlayer->Looping = false;
   
       BinkTextureRT = LoadObject<UTextureRenderTarget2D>(nullptr, TEXT("/Game/MapAssets/BinkMediaPlayer/BinkVideoRT.  BinkVideoRT"));
       BinkTextureRT->RenderTargetFormat = RTF_RGBA8;
       BinkTextureRT->SRGB = false;
       BinkTextureRT->bForceLinearGamma = false;
       BinkTextureRT->ClearColor = FLinearColor::Black;
   
       // By this point there will be a valid Viewport even in a packaged game
       FVector2D ViewportSize;
       GEngine->GameViewport->GetViewportSize(ViewportSize);
       BinkTextureRT->SizeX = ViewportSize.X;
       BinkTextureRT->SizeY = ViewportSize.Y;
       BinkTextureRT->UpdateResource();
   
       MediaPlayer->InitializePlayer();
       duration = MediaPlayer->GetDuration().GetTotalSeconds();
   
       View = CohtmlHUD->GetView();
       check(View);
       View->CreateModel("BinkVideoPlayerHUDModel", this);
       View->SynchronizeModels();
   
       UE_LOG(LogCoherentSample, Log, TEXT("UI is bound!"));
   }
   

3. Call UBinkMediaPlayer::Draw each Tick

   void ACoherentSampleBinkVideoPlayerHUD::Tick(float DeltaTime)
   {
       if (!View)
       {
           return;
       }
   
       FVector2D ViewportSize;
       GEngine->GameViewport->GetViewportSize(ViewportSize);
       if (BinkTextureRT->SizeX != ViewportSize.X || BinkTextureRT->SizeY != ViewportSize.Y)
       {
           BinkTextureRT->SizeX = ViewportSize.X;
           BinkTextureRT->SizeY = ViewportSize.Y;
       }
   
       if (!BinkTextureRT->HasPendingRenderResourceInitialization())
       {
           MediaPlayer->Draw(BinkTextureRT);
       }
       currenttime = MediaPlayer->GetTime().GetTotalSeconds();
   
       if (ModelSyncNeeded)
       {
           View->SynchronizeModels();
       }
   
       // UBinkMediaPlayer's OnMediaReachedEnd event seems to be buggy,
       // as it gets fired even when calling UBinkMediaPlayer::Seek at
       // a time before the end of a given video, so we handle this ourselves.
       if (MediaPlayer->GetTime() == MediaPlayer->GetDuration() && !VideoEnded)
       {
           OnMediaReachedEnd();
           VideoEnded = true;
       }
   }
   

With this approach, if you have an in-world View, you will be able to play Bink videos in-world as well.

Video Controls and BinkVideoMap sample #

If you want to create UI video controls such as volume, seeking, sound, playback, etc., you can expose the UBinkMediaPlayer APIs to Prysm.

We have a sample map demonstrating this implementation under CoherentSample/Content/Maps/BinkVideoMap.umap within the CoherentSample project. The relevant source code is located in the CoherentSample/Source/CoherentSample/BinkVideoPlayer directory.