Input

Overview #

Input propagation in Prysm is managed in a different way, depending on how you’ve set up your UI. There are two main cases:

1. You can set up a Prysm HUD class from the GameMode for the player viewport, then use Prysm Actor Components for in-world Views. All UI input is managed through the SCohtmlInputForward Widget when following this approach.

2. You can add a Prysm Widget to the game viewport and use it as a HUD, then use Prysm UMG Widgets as Actor Components for your in-world Views. When using this approach, propagating input through different Views in the same map relies solely on Unreal’s UMG framework. Therefore, using the SCohtmlInputForward is neither needed, nor supported when using UMG.

The SCohtmlInputForward is an empty Widget that covers the whole screen and forwards input to a Prysm View. For simplicity we have created the ACohtmlInputActor Actor, which encapsulates the logic behind the Widget and is accessible through both C++ and Blueprints. All you have to do is spawn the Actor and call its SetupInput method.

Keyboard, Mouse and Gamepad focus in Unreal Engine #

Prysm will process input events only when the Input Forward Widget is focused. It is generally on the top of the viewport so it can receive input events first and then decide how to propagate them.

The order in which this propagation will happen per event is the following:

1. If the Widget does not have input focus, or the Widget isn’t set to always accept mouse input events, the event is left unhandled and will be received by the Engine. By default, the Widget does not allow mouse events, so you need to specify this by using the AlwaysAcceptMouseInput method.
2. If a Widget has focus, the events will be propagated to any HUD Views that can receive input for them to decide if they should handle said events.
3. If none of the HUD Views handled a given event, a raycast through the cursor position will be made. Depending on the Widget settings it can trace either multiple objects, or just the first object the ray hits. For any and all objects hit that have a Prysm Component, they will have the opportunity to handle the event in the order closest to the player (i.e. closest to the viewport).
4. If neither a HUD View, nor an in-world View have handled a given event, it will be left unhandled and get propagated to the Engine.

For each of the above steps, the Views will “decide” if they handle an event or not, based on the type of the event, or which part of the HTML was pressed. Some events might be propagated by default, while others will not. More details on this can be found here.

Gamepad input works by default when the viewport Widget has focus. Keyboard focus implies gamepad focus, which means that if the Prysm Widget has keyboard focus, the game will stop processing gamepad input. If that’s not intended, the Input Actor has a method for keeping gamepad focus to the viewport after setting keyboard focus.

Each View Component has a property (Receive Input) that controls whether a specific View takes part in the mouse event forwarding.

Input (Blueprints) #

To add input in a Blueprint-only game, you need to spawn an Actor that will set up the SCoherentInputForward Widget. This is an empty Widget that covers the whole screen and forwards input to a Prysm View.

The Widget should be added in the front of the viewport so it can process mouse and keyboard messages first. Due to Unreal Engine’s architecture, the Widget can process keyboard messages only if it’s focused. Check the ExampleMap’s SetupInput method in the ExampleMapHUD asset’s (located under Content/MapAssets/ExampleMap) Blueprint for an example:

The Input Actor setup part shows how to spawn and initialize the Input Actor, which in turn adds the SCohtmlInputForward Widget to the viewport.

The Game to UI Focus part is for managing input focus - when you press Tab, the character input will stop and the UI will take it. This would seem to be enough, but currently Unreal stops firing key events when the focus is not on the viewport so we need other means to get focus back to the game. This is shown in the UI to game focus part. The Input Actor provides events for key/mouse up/down - in the sample we’re using the key down event. When it’s fired there’s a check for the key enum to see if it’s the Tab key, and if it is, the focus is toggled.

The Toggle Input Focus function provides means for toggling input without tracking it. If you want to, there are also functions for querying and setting the focus explicitly.

Input (C++) #

The input in Prysm is managed through a Slate Widget (SCohtmlInputForward). This Widget is an empty Widget that covers the whole screen and forwards input to a Prysm View.

The Widget should be added in the front of the viewport so it can process mouse and keyboard messages first. Due to Unreal Engine’s architecture, the Widget can process keyboard messages only if it’s focused. To focus the Widget, you can use the FSlateApplication::Get().SetKeyboardFocus(Widget) method. Focus management must be implemented by the client.

You can also show the system mouse cursor using FSlateApplication::Get().ResetToDefaultInputSettings(). If you want to set focus back to the game, FSlateApplication::Get().SetFocusToGameViewport() will do that.

If the SCohtmlInputForward Widget has focus, then keyboard events are sent to the focused Prysm View. To focus a View, you should click anywhere on it.

Mouse events are sent only to the Prysm View that is under the cursor.

To setup the Slate Input Forward Widget, you only need to spawn an instance of ACohtmlInputActor and call ACohtmlInputActor::SetupInput.

Of course, you can also use the cohtml::View API to create your own input events and send them to Prysm. The View provides the MouseEvent, KeyEvent and TouchEvent methods that send input to it. Each input event has to be populated with properties that can be trivially translated from the Unreal Engine events.

Cases where Views would handle events #

All Views decide individually if a given input event “was meant” for them and either handle it themselves or propagate it to the next relevant View down the line. If no such is found, the event will be left for the Engine to handle.

For each View this would be determined:

• Based on the event

  • Keyboard and joystick events will always be handled by the currently focused Widget. If you want these events to reach the Engine as well, you can control this using the InputPropagationBehaviour option
  • Mouse and touch events will be handled/unhandled based on their target.
    • Mouse Move and Touch Move events will always be handled, so they will never get propagated
    • Mouse Wheel will always be handled by default and not get propagated, unless the Always Handle Mouse Wheel Events option is disabled, in which case it will be based on the target, same as any other mouse/touch event, for example depending on the HandleInputOnHTMLBody and InputTransparentCssClasses options.
  • Which events will be handled (or reversely propagated) depends on the options described in the notes.

• Based on the target

  • As specified above, for some events Prysm will decide whether or not to handle them, based on their target. The target of a given event is the topmost HTML element under the cursor.

• Based on the HTML element

  • If the target element is the HTML <body> (meaning the user has clicked on an empty part of the UI), the event will be left unhandled.
  • If the target element is a CSS transparent class or a child of a CSS transparent class (depending on which option is used) the event will be left unhandled. By default, this happens for HTML elements with the hidden CSS attribute and their children, however, you can expand this to include additional class attributes, using the Input Transparent CSS classes options lists as shown in the screenshot below.

Toggling input focus between the UI and the Game #

Toggling input focus can be done in two ways:

• Clicking with the mouse on a Prysm View gives focus to the View since the Unreal’s Slate system will focus the Widget that handles the click. If you click back somewhere in the game viewport, the game will take focus.
• The “click to focus” method works, but in a more realistic use case, you’d want to be able to change the focus programmatically. You can do that using the Input Actor’s ToggleInputFocus method. The Actor keeps track of the current focus internally so it can toggle between game and UI at any time. If you want to set it explicitly, you can use SetInputFocus, or query the state with IsFocused.

After the Input Actor has focused the input forwarding Widget, you need to set which View should receive the keyboard input. This can be done with the SetViewFocus method. An alternative is to simply let the user click on the input field that she wants to type in.

Propagating keyboard and gamepad events when the UI is focused #

When the UI is focused, keyboard events are consumed by the Input Forward Widget by default. You can change that using the SetInputPropagationBehaviour method. The available options are to forward keyboard events, gamepad events, both, or none to other Widgets.

This can be useful if you need to have your action handlers (e.g. “Jump”, “Fire”, etc.) executed even if the focused Widget isn’t the game viewport.

Note that gamepad forwarding is only taken into account when using the Input Actor’s methods for setting (or toggling) focus.

Forwarding mouse input events to Prysm regardless of the focused Widget #

If you want your mouse input to be forwarded to Prysm first for hover events, but still receive keyboard events in your game, then you can use the AlwaysAcceptMouseInput method. When set to true, mouse events will be received by Prysm, regardless of whether the Input Forward Widget is focused or not.

Including or removing specific Prysm components in the input forwarding #

To control whether a Prysm View receives mouse events you can change the View Component’s Receive Input property. Only when this property is set to true will the Component be considered for receiving input events.

Input on multiple Prysm components in the 3D world #

When you have multiple Views on 3D objects and they are overlapping when viewed with the current camera, the front one will receive the mouse input events, and the back one will get nothing. If, for example, you have disabled input on the front one, the back one will still get nothing, because 3D objects under the cursor are gathered with a raycast from Unreal, which is independent of the Input Forward Widget’s logic.

For performance reasons, the raycast returns only the first object hit by default. In a scenario like the one described above, where you have two overlapping objects and you want to forward input to the one in the back, the raycast type must be changed to return multiple objects. This is needed even if the object in the front is marked to not receive input.

Virtual Keyboard Input #

On a variety of platforms, text input can be achieved only by using a virtual keyboard. Prysm supports this feature out of the box. When focusing an HTML input field or textarea, Prysm will try to spawn virtual keyboard on screen, using FSlateApplication::ShowVirtualKeybord and supplying FCohtmlVirtualKeyboardEntry as an argument.

Virtual Keyboard layout #

Specialized virtual keyboard layout will be used for the listed HTML input field types:

• Input field of type password will set the virtual keyboard layout to EKeyboardType::Keyboard_Password
• Input field of type number will set the virtual keyboard layout to EKeyboardType::Keyboard_Number
• Input field of type url will set the virtual keyboard layout to EKeyboardType::Keyboard_Web
• Input field of type email will set the virtual keyboard layout to EKeyboardType::Keyboard_Email
• Input field of type alpha will set the virtual keyboard layout to EKeyboardType::Keyboard_AlphaNumeric
• Input field of type all will set the virtual keyboard layout to EKeyboardType::Keyboard_Default

Default text virtual keyboard layout will be used for HTML textarea fields and HTML input field of all other types not listed above. Depending on the value in UCohtmlSettings::SetDefaultVKTextLayoutToAlphaNumeric, the default text layout can either be EKeyboardType::Keyboard_Default or EKeyboardType::Keyboard_AlphaNumeric.

HTML textarea fields will also set the multiline entry modifier for the virtual keyboard layout.

Overriding Prysm Virtual Keyboard behavior #

The default behavior described above can be changed by binding an override function to the ICohtmlPlugin::OnShowVirtualKeyboard delegate.

Inside that function, you have to create an instance of FCohtmlVirtualKeyboardEntry, or a derived class. You can use FVirtualKeyboardOptions as an optional parameter. Lastly, you need to show the virtual keyboard either through FSlateApplication::ShowVirtualKeyboard, or your own implementation. You can also completely deactivate this feature for a given platform, by leaving the implementation of the bound function empty.

void ShowVirtualKeyboardOverride(TSharedPtr<FCohtmlVirtualKeyboardEntry>& CohtmlVirtualKeyboardEntry, int UserIndex, cohtml::View* OwningView)
{
#if !PLATFORM_WINDOWS
    // fill in the CohtmlVirtualKeyboardEntry shared ptr and show the virtual keyboard
    CohtmlVirtualKeyboardEntry = MakeShareable(
        new FCohtmlVirtualKeyboardEntry(OwningView, FVirtualKeyboardOptions())
    );
    FSlateApplication::Get().ShowVirtualKeyboard(
        true, UserIndex, CohtmlVirtualKeyboardEntry);
#else
    // leave empty to disable for a given platform
#endif
}