Real-world teleoperation · development service

Operator systems for humanoid robots

We develop robot-agnostic, primarily VR-based teleoperation systems: know-how, technology, interface and UX – with integration verified robot by robot.

Development serviceShared operator layer: done in simulationReal-robot integration: planned
Profile A – direct telepresence: the robot camera view during a grasp – simulationSimulation – real application recording
Profile A – direct telepresence: the robot camera view during a grasp – simulation
01 — Task and users

A human steers, the robot protects itself locally

Needed where the task is remote, hazardous or not yet automatable – but needs human judgement.

01

Operator organization

Industrial, institutional or rescue work with a robot and a trained operator.

02

Robotics integrator

A shared operator layer for several robot platforms, with adapters and testing.

03

Training lead

The same interface for practice on an emulated robot, then in the field.

Reusable
  • Operator workflow and interface
  • Session and permission handling
  • Training framework
  • Logging and replay
Built per robot
  • Robot adapter and command translation
  • Motion model and retargeting
  • Safety profile, local limits
  • Firmware and SDK compatibility
Task and users

What do we check for a new robot?

  • Accessible SDK or API, documentation, license
  • Controllable DoF and supported commands
  • Cameras and sensors, aspect ratio, field of view, frame rate
  • Hardware limits, usage rights
  • Real testability: robot, site, supervision

The same hand gesture can map to different joint motion on different robots – so every integration is verified robot by robot.

02 — Operator station and robot

Six layers, one command path

Every input takes the same validated path: keyboard, VR, voice, AI plan and instructor.

  1. 1
    Operator clientVR or PC: video, state, intervention
    Done – R1 (laptop; VR: IWER emulator)
    Operator station and robot

    Operator client

    Device setup, hand and head tracking, video display, task display, handover and replay. Desktop and VR share one session data model.

    Done – R1 (laptop; VR: IWER emulator)

  2. 2
    Teleoperation corePermissions, command gate, IK, state machine
    Done – R1 (gateway)
    Operator station and robot

    Teleoperation core

    Input normalization, coordinate transforms, motion mapping, command checks, control lease and state machine. Retargeting matches human and robot proportions.

    Done – R1 (gateway)

  3. 3
    Robot adapter and local gatewayVendor SDK, watchdog, local limits
    Simulated endpoint done · Unitree adapter planned
    Operator station and robot

    Robot adapter and local gateway

    Vendor SDK link, command translation, sensor polling, watchdog, local limits and reconnection. Separate adapters can serve the body, the hands and an external sensor pack.

    Simulated endpoint done · Unitree adapter planned

  4. 4
    Local controllerWalking, balance, actuators
    On a real robot: the vendor’s · emulated in simulation
    Operator station and robot

    Local controller

    The vendor’s motion controller or separately validated software handles walking, balance and actuators. A high-level API and low-level motor commands are different integration paths.

    On a real robot: the vendor’s · emulated in simulation

  5. 5
    Media transportTimestamped video and audio, frame age
    Simulated media path done · real stream planned
    Operator station and robot

    Media transport

    Bandwidth handling, stream state, frame age and dropped frames. Sensor and video alignment has to be measured.

    Simulated media path done · real stream planned

  6. 6
    Session and analysisLog, result, replay
    Done – R1
    Operator station and robot

    Session and analysis

    Events, configuration versions, telemetry, result sheet and export. A reporting or AI outage cannot stop control.

    Done – R1

03 — Control modes and feedback

The operator sees how fresh the picture is

Measured, estimated and missing data are labeled apart. Click the numbers.

Profile B – operator control desk: map, robot camera, perceived objects, robot state, steps, events – simulationSimulation – real application recording

Profile B – operator control desk: map, robot camera, perceived objects, robot state, steps, events – simulation

Inputs
  • Laptop: keyboard and mouse
  • VR: Meta Quest 3, two controllers (verified in the IWER emulator)
  • Voice or text → AI plan → approval
  • Gamepad, hand tracking, mobile – planned (R2)
Simulation – real application recording
One control logic, three layouts — Operator layouts A, B and C within the same run.
04 — Integration status

Function by function, honestly

Being in the catalog implies no integration level by itself.

FunctionUnitree G1 EDUOther catalog robots
Kinematic modelSimulatedR1 training: URDF-accurate geometry, joint limits, speed limits, self- and environment collision; main dimensions within 1% of factory values. No dynamics (balance, falls, forces).Catalog data
3D model3D modelCode-built, fully articulated model using the official URDF kinematics – our own work, not a vendor mesh. No 3D viewer on the website yet.Catalog data
Walking (velocity request)SimulatedEmulated gait within the walking limits. Known deviation: at start-up the leg joint speed can exceed the URDF limit (measured 39.7 rad/s vs 20 rad/s).Catalog data
Arm control (IK)SimulatedMouse and keyboard on a laptop, clutch in VR. An unreachable target causes no jump or joint-limit violation (tested). VR verified only in the IWER emulator.Catalog data
Hand and grasp (Dex3, single hand)SimulatedGeometric grasp without force simulation, within the training profile limits (80 mm hand opening, 3 kg per arm). No bimanual grasp – planned (R2).Catalog data
Head camera (D435i)SimulatedClient-side render following the camera profile: 69° × 42° field of view, 47.6° downward tilt. The image is not a real camera stream.Catalog data
Fault handling (latency, freeze, dropout)SimulatedSimulated network profiles and one-off events; stale late commands are discarded by the core, a frozen image is clearly flagged (tested).Catalog data
Unitree SDK adapterPlannedR4: the command mapping to the unitree_sdk2 G1 interface exists as a plan (locomotion, arm, Dex3, state); to be verified on firmware and SDK version.Catalog data
Test on a real robotPlannedNot done yet. It needs the robot, firmware, peripherals, on-site supervision and a per-robot acceptance test.Catalog data

Catalog data – integration after a per-robot assessment · Robot Catalogue →

05 — A complete session

From preparation to close

Every step has a precondition and a log; a failed calibration blocks the start.

  1. The Unitree G1 EDU robot card: factory and measured data labeled – simulationSimulation – real application recording
    1Preparation

    Robot, profile and capabilities: a robot card with factory and measured data.

    Done in simulation
  2. VR panel room: angle-true robot-camera panel, status bar, external-view and AI-plan panelsIWER emulator – not a real headset
    2Calibration and pre-check

    Device and direction test, then activation by holding for 1 second.

    Done in simulation
  3. Profile C – spatial robot view: the whole robot from outside, with the robot-camera window – simulationSimulation – real application recording
    3Work

    Walking, manipulation, commands – the control lease belongs to one source.

    Done in simulation
  4. PlannedR2
    4Handover

    Logged, jerk-free, explicitly accepted operator change.

    Planned
  5. Result screen: completion gate, components, steps – simulationSimulation – real application recording
    5Close

    Result, log, export – the attempt can be replayed.

    Done in simulation
06 — Latency and link loss

Only intent travels over the network

Balance and motor control run locally. Drag the slider: what changes, and what does the local guard protect?

Operator
intent →
← video + stateNetwork
Robot
Local loop: balance, motorsruns on the robot, does not wait for the network
Round-trip time
150 ms
On-screen reaction approx.
200 ms
→
Noticeable delay: slow down for fine manipulation.

Illustrative profiles from the training’s fault injector – not a measured real network.

Simulation – real application recording
Degrading link and a frozen image — Rising latency, a frozen image with a growing frame age, the operator’s flag, then a command dropout – simulated network faults.
Local protections in the training (simulation)
  • The core’s 300 ms guard stops walking when no fresh command arrives
  • The robot-side command expires after 1 s and the robot slows to a stop
  • Stale late commands are discarded by the core
  • Frozen image: growing frame age and a visible alert

On a real robot the local procedure must be validated per robot.

07 — Logging, testing, replay

Every command can be traced

Command, feedback, fault and configuration on one timeline – replay is always virtual.

  • Commands, acks and endpoint commands with timestamps (30 Hz state)
  • Result sheet as machine-readable, printable and CSV; re-scoring with another rubric
  • Replay at 0.25×–4× with event and fault markers
  • A fault can be reconstructed from the log by command, state and event (tested)
Simulation – real application recording
Instructor replay — An always-virtual replay rebuilt from the log, with a timeline, event and fault markers and slow motion.
08 — AI assistance and human supervision

The AI suggests, the human decides

The AI sends no motor commands: it can only request approved, parameter-limited skills, with a preview.

Command preview with a continuation: “Go to the table, pick up the box and throw it away” – simulationSimulation – real application recording
1Command and plan

“Go to the table, pick up the box and throw it away.” The box is not yet visible, so the plan starts with walking there.

1 / 3
Second preview at the table: it picks the red box – the yellow one is too heavy, the green one too wide – simulationSimulation – real application recording
2Continuation with fresh perception

At the table a new preview: it picks the red box, because the yellow one is too heavy and the green one too wide.

2 / 3
Ambiguous command: at the table the assistant asks back – “Which object?” – simulationSimulation – real application recording
3Asking back

For an ambiguous command it does not guess: it asks which object.

3 / 3

The plans in the images were written by a local rule-based mock assistant. The real Claude interpretation measured 2.7–5.1 s after warm-up and about 10 s cold. Manual control always remains.

09 — Deployment and development process

From the lab to the site, robot by robot

Five deployment modes; the public website and demo always stay outside the system boundary.

  • Local labIntegration and testingOperator and robot on one controlled network
  • On-site systemIndustrial or institutional useLocal gateway, local log and permissions
  • Remote operationSeparate operator siteAuthenticated, encrypted link to the gateway
  • Air-gapped networkClosed customer environmentNo cloud dependency, local AI optional
  • Developer simulationWork without a robotSame client, simulated endpoint
  1. 1Goal and robotTask, environment, operators, robot model.
  2. 2Integration assessmentSDK, DoF, cameras, limits, rights.
  3. 3Simulation profileEmulated robot and operator training with the same client.
  4. 4Adapter and lab testVendor interface, local limits, test record.
  5. 5On-site acceptancePer-robot acceptance test, supervised.
Deployment and development process

Planned Unitree G1 mapping

Endpoint commandSimulated G1 (R1)Unitree G1 (planned)
loco.move {vx, vy, wz}Velocity request, emulated gaitLocoClient.Move(vx, vy, vyaw)
loco.stopSlowed stopLocoClient.StopMove
loco.height {h}Crouch within the allowed rangeLocoClient body height (firmware-dependent)
arm.q {q}Arm and waist targets with collision checksrt/arm_sdk LowCmd
hand.q {side, q[7]}Simplified closure valuert/dex3/{left,right}/cmd
estopPosture freezesLocoClient.Damp – may cause a fall, vendor procedure only

Based on the unitree_sdk2 G1 examples; to be verified on firmware and SDK version. Not yet run on a real robot.

Deployment and development process

Status of the teleoperation functions (TEL-01…14)

TEL-01 Robot and adapter identificationPartlyModel, profile, URDF revision, camera; serial and firmware only on a real robot
TEL-02 Capability queryDone in simulationUnsupported capabilities (head, jump, bimanual grasp) cannot start
TEL-03 Pre-checkDone in simulationA failed calibration blocks the start
TEL-04 Exclusive control leaseDone in simulationOf two sources only the valid lease is accepted (tested)
TEL-05 Motion intent and manipulationDone in simulationLimited targets; an unreachable target causes no jump
TEL-06 Video, audio and telemetryPartlySource, data age, frame age and link state visible; no robot audio
TEL-07 Local fault handlingDone in simulationIn simulation: guard, command expiry, stale-command drop
TEL-08 Operator handoverPlannedR2
TEL-09 Two-operator modePlannedR2
TEL-10 Test and operation logDone in simulationCommand, feedback, fault and configuration can be linked (tested)
TEL-11 Three UI profilesDone in simulationA, B, C – identical commands and states (tested)
TEL-12 Simulator run with the same clientDone in simulationEmulator principle: the endpoint is swappable
TEL-13 AI assistanceDone in simulationSuggestion and execution separated, with human approval
TEL-14 Service and updatesPartlyProtocol version check and versioned log; rollback and compatibility gate planned

“Done” means done in the training, in simulation – none is validated on a real robot.

Start a project

Which robot, for what task?

Tell us the robot model, the use case and the working environment – we prepare the first meeting from that.

* required