On-screen sticks/buttons, touches, and gestures.

Virtual & touch input

Phones and tablets have no keyboard. Virtual input is how a HUD control — an on-screen stick or button — feeds the same actions a keyboard would. Touch input is the raw finger data underneath: positions, drags, and gestures. Together they let one Skyforge gameplay script serve desktop and mobile unchanged.

The virtual pipeline end to end

Three pieces connect a fingertip to a gameplay action:

1. A HUD node of type virtual_button or virtual_stick is drawn on screen and names a control.
2. That name becomes the binding target virtual:<control> in the project’s action bindings.
3. When the player touches the control, the action’s queries return exactly as they would for a physical device.

-- HUD (hud.json) — the on-screen stick
{ "id": "move_stick", "type": "virtual_stick", "control": "move" }

-- Project (demi.project.json) — the action binding
{ "input": "virtual:move", "normalize": true }

-- Lua — unchanged gameplay code
local x, y = Input.vector("move")

Field meanings: control is the name the stick publishes; "input": "virtual:move" is the binding that listens for it; Input.vector("move") returns the same x, y pair a left stick would produce. Nothing in the script knows a finger was involved.

Authoring virtual_button

{
  "id": "jump_btn",
  "type": "virtual_button",
  "control": "jump",
  "anchor_min": [1, 1],
  "anchor_max": [1, 1],
  "at": [-180, -180],
  "size": [140, 140],
  "accessibility_label": "Jump"
}

• type — virtual_button. Renders a pressable on-screen button.
• control — name published as virtual:jump while the button is held. Required.
• dock, at, size — placement in canvas units, like any other node (see Layout).
• accessibility_label — spoken name for screen readers.

While a finger is down on the button, bound actions read as pressed. On lift, they read as released — the same timing as a keyboard key.

Authoring virtual_stick

{
  "id": "move_stick",
  "type": "virtual_stick",
  "control": "move",
  "anchor_min": [0, 1],
  "anchor_max": [0, 1],
  "at": [40, -260],
  "size": [220, 220]
}

• control — publishes virtual:move as a two-axis value from the knob’s deflection.
• Binding virtual:move feeds a vector2 action. Binding virtual:move:x or virtual:move:y feeds one axis of the stick into an axis1d action.
• Deflection is relative to the stick’s size; a full drag to the rim reports about length 1 before any normalize.

"move": {
  "type": "vector2",
  "bindings": [
    { "input": "key:w", "vector": [0, -1] },
    { "input": "key:s", "vector": [0, 1] },
    { "input": "key:a", "vector": [-1, 0] },
    { "input": "key:d", "vector": [1, 0] },
    { "input": "gamepad:stick:left", "deadzone": 0.18 },
    { "input": "virtual:move", "normalize": true }
  ]
}

This move action is filled by WASD, the left stick, and the on-screen stick at once — active contributions are summed, and Input.vector clamps the combined result to unit length.

Reading touches

Touches are the raw fingers on the screen. Use them for world panning or custom gestures when an action is not the right shape.

local Input = require("demi.input")

function World:on_update(dt)
    for _, t in ipairs(Input.touches()) do
        print(t.id, t.phase, t.x, t.y)
    end
    print(Input.touch_count())
end

• Input.touches() returns an array of touch tables for this frame (empty when nothing is touching).
• Input.touch_count() returns how many touches are active right now, as a number.

Each touch table has these fields:

A one-finger drag reads as one began, several moved rows with the same id, then one ended. A second finger gets a different id so you can track both at once.

Reading gestures

Gestures are recognized multi-touch shapes. They arrive as a list per frame, separate from touches, so you can react to a tap without writing a recognizer.

for _, g in ipairs(Input.gestures()) do
    if g.type == "tap" then
        print("tapped at", g.x, g.y)
    elseif g.type == "pinch" then
        print("pinch scale", g.value)
    end
end

• Input.gestures() returns an array of gesture tables (empty when no gesture completed this frame).

Every gesture table carries the same fields:

Use tap for select, drag for panning, pinch for zoom, and rotate for camera roll in Skyforge’s map screen.

Pointer helpers around touch and mouse

local x, y = Input.mouse_position()
local dx, dy = Input.mouse_delta()
local down = Input.mouse_down("left")
local wx, wy = Input.mouse_world_position()
local vw, vh = Input.viewport_size()
local captured = Input.ui_pointer_captured()

• Input.mouse_position() returns x, y of the mouse or the primary touch in canvas units.
• Input.mouse_delta() returns dx, dy since last frame.
• Input.mouse_down(button) returns a boolean for a mouse button name.
• Input.mouse_world_position() returns x, y projected into the scene, for clicking world objects.
• Input.viewport_size() returns width, height of the render surface in canvas units.
• Input.ui_pointer_captured() returns true when a HUD control currently owns the pointer — a virtual stick being dragged, a slider, a scroll area. Pass an optional pointer id to check one specific finger. Check it before treating a press as a world click.

Text input

On-screen keyboards appear only when you ask for them:

Input.set_text_input_active(true)   -- show the keyboard for this field
local typed = Input.text_entered()  -- string appended this frame, or ""
-- later, when the player leaves the field:
Input.set_text_input_active(false)  -- hide the keyboard

• Input.set_text_input_active(flag) takes a boolean and raises or lowers the platform text input. Pair it with focusing a text_input node.
• Input.text_entered() returns the characters typed this frame as a string (empty string when nothing arrived).

Mobile tips for Skyforge

• Keep virtual controls at least 120 canvas units across so thumbs hit them reliably.
• Place sticks and buttons inside the safe area; see Layout for when to disable it.
• Always offer a gamepad path too: virtual:* bindings sit beside gamepad:* bindings on the same action, so a Bluetooth pad works with no extra code.
• Use Input.ui_pointer_captured() so dragging the move stick never also pans the camera.
• Prefer actions and virtual controls over raw touches for anything players remap. Touches are for bespoke gestures like the map screen.

Next

• HUD nodes — every node type including virtual controls.
• Input bindings — binding fields and presets.
• Actions explained — queries and action types.

Found an issue or missing topic? The full engine sources and markdown notes live in the DemiEngine repository.