VR operator training · R1 done, simulation

Practice on an emulated robot before it counts

The operator controls an emulated Unitree G1 EDU from VR or a laptop: walks, grasps, places, throws and handles faults – the system logs, measures and scores.

Done – R1, kinematic simulationVR: verified in the IWER emulator
VR panel room: angle-true robot-camera panel, status bar, external-view and AI-plan panelsIWER emulator – not a real headset
VR panel room: angle-true robot-camera panel, status bar, external-view and AI-plan panels
01 — Training outcome

What the operator will be able to do

Goals are made concrete per task, and the system measures them.

  1. 1Controls the robotWalking, turning, crouching, hand targets, grasping – within the robot’s real limits.
  2. 2Understands sensor limitsField of view, frame age, estimated and missing data – what the robot sees and what it does not.
  3. 3Completes the taskMoving, placing, throwing, parking – within the time limit, without collisions.
  4. 4Recognizes anomaliesLatency, frozen image, command dropout, low battery – and knows what to do.
Simulation – real application recording
A training run in half a minute — Robot card, activation, walking, grasping, approving an AI plan, a throw, the result – a real application recording of the simulation.
02 — Modules

Six modules from basics to faults

Guided, practice, exam and engineering modes. Details behind each card.

  • Modules

    TR-01 · Familiarisation and calibration

    The Unitree G1 EDU robot card: factory and measured data labeled – simulationSimulation – real application recording
    The Unitree G1 EDU robot card: factory and measured data labeled – simulation

    Robot card, camera and field of view, views, stopping, hand mode (laptop) or clutch (VR), grip open-close, activation.

    Simulation – R1

  • Modules

    TR-02 · Free locomotion

    The ARworks Robot Lab: throw zone with bin, slalom gates, manipulation table – simulationSimulation – real application recording
    The ARworks Robot Lab: throw zone with bin, slalom gates, manipulation table – simulation

    Two “stop on the line” points, a slalom through three gates, parking within ±10 cm / ±10°.

    Simulation – R1

  • Modules

    TR-04 · Single-hand grasp and place

    Profile A – direct telepresence: the robot camera view during a grasp – simulationSimulation – real application recording
    Profile A – direct telepresence: the robot camera view during a grasp – simulation

    Move the red box and the blue cylinder. Spot that the yellow box is too heavy (5 kg > 3 kg) and the green one too wide (> 80 mm).

    Simulation – R1

  • Modules

    TR-V1 · Supervised voice command

    Throw into the bin: the red box just above the rim – simulationSimulation – real application recording
    Throw into the bin: the red box just above the rim – simulation

    Three commands with preview and approval; the assistant asks back when a command is ambiguous. Includes rejecting a plan and aborting a skill.

    Simulation – R1

  • Modules

    TR-16L · Fault drill (lite)

    Fault signal: frozen robot-camera image with a growing frame age – simulationSimulation – real application recording
    Fault signal: frozen robot-camera image with a growing frame age – simulation

    Moving an object while latency rises (150 → 300 → 600 ms), with a 2 s image freeze, a command dropout and a low battery. The frozen image must be flagged.

    Simulation – R1

  • Modules

    SANDBOX · Free practice

    Profile C – spatial robot view: the whole robot from outside, with the robot-camera window – simulationSimulation – real application recording
    Profile C – spatial robot view: the whole robot from outside, with the robot-camera window – simulation

    All objects and zones, throwing experiments – without scoring.

    Simulation – R1

03 — Controls and vision

The same logic on a laptop and a Quest 3

Pick a function: the key and the controller button light up. The in-app help shows the real mapping.

Left controller
Right controller
Walk and strafeW A S D · left thumbstick
FunctionLaptopVR (Quest 3)
W A S Dleft thumbstick
Q E · F Rright thumbstick
M + mousehold grip
Gtrigger (0–1)
VA button
SpaceB button
Enterhold X for 0.5 s
T heldhold Y
Ppanel button with the ray

The VR mapping is verified only in the IWER emulator.

What the robot camera sees

The G1 head camera is 69° × 42° and tilted 47.6° down – from the start pad only the floor is visible. The headset’s wide field of view does not change that.

Robot choice: today the Unitree G1 EDU (29 DoF + Dex3 hands) – more robot profiles are planned. Unitree G1 →

Robot-eye view from the start pad, realistic FOV: the camera’s 47.6° downward tilt shows only the floorIWER emulator – not a real headset
Robot-eye view from the start pad, realistic FOV: the camera’s 47.6° downward tilt shows only the floor
04 — Manipulation and faults

Two hands, and when the link degrades

Real application recordings of the simulation – no retouching.

Simulation – real application recording
Two hands, two objects — Two objects in two hands at once: the right holds the red box, the left the blue cylinder, then both go onto the tray – via approved plan steps. Bimanual grasping is not part of R1.

Bimanual grasping is not part of R1: the training follows the real G1 limit. Planned for R2.

Simulation – real application recording
Degrading link and a frozen image — Rising latency, a frozen image with a growing frame age, the operator’s flag, then a command dropout – simulated network faults.
Fault profiles in the instructor console
  • LAN, WAN ~150 ms, WAN ~300 ms, poor ~600 ms, 5% packet loss, escalation
  • Sliders: latency, jitter, packet loss
  • One-off events: image freeze, command and telemetry dropout, low battery, camera fault
05 — Instructor tools

The instructor watches, intervenes, replays

Live views, fault injection, pause, revoking control, notes – and replaying the attempt.

Instructor console during a run: live views and fault injection – simulationSimulation – real application recording
Instructor console during a run: live views and fault injection – simulation
  • Mirrored robot camera, exact external view and map, live
  • Simulated emergency stop with double confirmation, then recovery
  • Assists, UI profile and mode set between attempts
  • Replay at 0.25×–4×, with robot camera, map and hand trail
Simulation – real application recording
Instructor replay — An always-virtual replay rebuilt from the log, with a timeline, event and fault markers and slow motion.
06 — Interface experiments

Three layouts, the same control

The profiles produce the same commands and states (tested) – so they can be compared.

Simulation – real application recording
One control logic, three layouts — Operator layouts A, B and C within the same run.
  • Profile A – direct telepresence: the robot camera view during a grasp – simulationSimulation – real application recording
    A – direct telepresenceFull-screen robot camera with a HUD.
  • Profile B – operator control desk: map, robot camera, perceived objects, robot state, steps, events – simulationSimulation – real application recording
    B – operator control deskMap, camera, objects, state, events.
  • Profile C – spatial robot view: the whole robot from outside, with the robot-camera window – simulationSimulation – real application recording
    C – spatial robot viewThe robot from outside, with a camera window.
07 — Assessment

A completion gate and measured components

Success, no critical event, within time – plus a 0–100 component score.

Result screen: completion gate, components, steps – simulationSimulation – real application recording
Result screen: completion gate, components, steps – simulation
  • Task accuracy40%
  • System handling25%
  • Motion quality20%
  • Time efficiency15%

Guided-mode results are not comparable; free practice and engineering mode are not scored. Re-scoring with another rubric is possible.

Assessment

Printable result sheet

Printable result sheet (top part) – simulationSimulation – real application recording
Printable result sheet (top part) – simulation
08 — What comes next

Planned extensions

These are not done yet – so they appear separately, marked “planned”.

  • PlannedFirst test on a real headsetMeta Quest 3: optics, comfort, tracking, performance – with a checklist.
  • PlannedBimanual grasp and finger panelR2: modules TR-05 and TR-06.
  • PlannedTwo-operator mode and handoverR2: locomotion and manipulation shared, logged handover.
  • PlannedGamepad, hand tracking, mobileR2: more input devices on the same command gate.
  • PlannedConnecting existing VR appsScenarios from ARworks VR training as robot task series – building on earlier references.

In the training the robot is a kinematic simulation: URDF-accurate geometry, joint limits, collisions – without dynamics (balance, falls, forces).

Start a project

Which robot, for what task?

Tell us the robot model, the use case and the working environment – we prepare the first meeting from that.

* required