User Images

User images is a feature of Prysm that allows developers to manually load and decode textures (which Cohtml would normally do). User images also allows you to specify a content rectangle where the actual image is. This can be used if the texture has padding or is part of a texture atlas.

This lets you achieve the following:

  • Improve the loading time of pages by preparing the textures in advance instead of when Cohtml asks for them.
  • Reduce memory footprint when the engine and the UI share textures, by having the texture stored in GPU memory only once, instead of twice (once for the engine and once for Cohtml).
  • Use custom image formats.

Using user images for preloading textures

  1. Override cohtml::IAsyncResourceHandler::OnResourceRequest in the resource handler.
virtual void MyResponseHandler::OnResourceRequest(const cohtml::IAsyncResourceRequest* request, cohtml::IAsyncResourceResponse* response) override
{
    std::string url = request->GetURL();
    if (MyApp::IsPreloaded(url))
    {
       auto texture = MyApp::GetPreloadedTexture(url);
       // Provide texture information to Cohtml
       cohtml::IAsyncResourceResponse::UserImageData data;
       data.Width = texture->Width;
       data.Height = texture->Height;
       data.ContentRectWidth = texture->Width;
       data.ContentRectHeight = texture->Height;
       data.Format = texture->Format;
       // Pointer to a user-defined user data of the texture.
       // User images sharing the same `Texture` pointer will leave only a single image's memory footprint inside the GPU Memory Tracker.
       // Draws using different user images that share the same `Texture` pointer can be batched together in a single draw call.
       data.Texture = texture->Resource;
       // Pointer to a user-defined identifier that will be used to identify textures belonging to the same atlas.
       // User images sharing the same TextureBatchingHint will leave only a single image's memory footprint inside the GPU Memory Tracker.
       // Draws using different user images that share the same TextureBatchingHint can be batched together in a single draw call even when the Texture pointer is different.
       data.TextureBatchingHint = texture->Id;
       response->ReceiveUserImage(data);
       response->Finish(cohtml::IAsyncResourceResponse::Status::Success);
    }
    else
    {
        HandleNonPreloadedResourceRequest(request, response);
    }
}
  1. Modify the rendering backends to keep track of user textures. When WrapUserTexture is called, you should mark the texture as a user texture.
void MyBackend::WrapUserTexture(void* userObject,
    const Texture2D& description,
    Texture2DObject object)
{
    // userObject contains whatever you have passed to
    // cohtml::IAsyncResourceResponse::UserImageData::Texture

    WrappedTexture t;
    t.Texture = userObject;
    t.IsUserTexture = true;
    m_Textures.insert(object, t);
}

When DestroyTexture is called, you should notify your engine that the texture is not used anymore and clear the reference.

void MyBackend::DestroyTexture(Texture2DObject texture)
{
    auto t = m_Textures.find(texture);
    if (t != m_Textures.end())
    {
        if (t->second.IsUserTexture)
        {
            MyApp->TextureReleased(t->second.Texture);
        }
        else
        {
            DestroyTextureImpl(t->second.Texture);
        }
        m_Textures.erase(t);
    }
}
  1. Attach to the system callback OnUserImageDropped and notify your engine about textures that were dropped even before being used. For such textures the DestroyTexture method won’t be called.
cohtml::SystemSettings CreateSystemSettings()
{
    cohtml::SystemSettings systemSettings;
    systemSettings.OnUserImageDropped = [](void* userData, void* userImage)
    {
        // userImage contains whatever you have passed to
        // cohtml::IAsyncResourceResponse::UserImageData::Texture
        static_cast<MyApp*>(userData)->TextureReleased(userImage);
    };
    systemSettings.UserData = this;
    return systemSettings;
}

Lifetime of user image resources

Reference flow when using user image resources as background-image:

  1. Create an element with a style which has user image resource for a background-image.
  2. When the element’s style is evaluated, the style will add a reference to the resource.
  3. The reference will be removed when:
    • The background-image in the element’s style is changed.
    • The style of the element is removed or the whole element is removed from the DOM tree.

Lifetime flow in <img>:

  1. Create <img> element with a user image resource specified by the src attribute.
  2. The element will add a reference to the resource.
  3. The reference will be removed when:
    • The src attribute is changed.
    • The Garbage Collector destroys the element.

User images' memory footprint inside the GPU Memory Tracking

The cohtml::IAsyncResourceResponse::UserImageData struct contains a TextureBatchingHint that is used as a hint to the batching algorithm and the GPU Memory Tracking to indicate that the user image belongs to the same texture when the Texture pointer is different. For example, you might have 2 user images that share the same texture from an atlas, but you also need to pass additional data via the Texture pointer that will be different between the 2 images. In this case, you should hint to Prysm that it is the same texture by passing an identical hint identifier to allow for better batching and correct tracking of user resources.

When the TextureBatchingHint is not provided, Prysm will fall back to using the Texture pointer for batching and GPU resource tracking.

← Preloaded Resources
Single frame loading →