Auspex
Physical AI · Nemotron on Nebius
Source on GitHub

Critical patients first: reasoning-based traffic control for hospital AMR fleets.

Auspex sits above Open-RMF. When an autonomous mobile robot and a patient transport share the same environment, it determines who gets priority and steers the robot.

Replays of recorded runs from a local simulation. Self-host runs the live system.

Scenario

Mode

Plays at 4x. Change Speed or drag the bar anytime.

Lift bookings, next 12 minutes

PatientsPatients, read by NemotronAMRsClash

Decisions

    Decisions appear as the run plays. Click one to see why.

    Event

    Click a marker on the playback bar.

    Incidents

      Where Nemotron comes in

      Hospital notes are messy: CT 2, needs large car. Nemotron reads them, and weighs the options when rules can't decide. A policy gate approves every move; the LLM never drives a robot.

      Problem

      AMRs and patient beds share lifts. The AMR traffic system can't see patients, so urgent oxygen stalls, or a patient waits.

      Approach

      Read the hospital's transport requests, forecast lift use minutes ahead, rank by clinical priority.

      Action

      Close a lift to AMRs, reroute, hold or hand over the delivery, through Open-RMF, and log it.

      Playground: write a messy note, watch Nemotron read it.

      The patient transport from scenario 1, editable. Nemotron Nano on Nebius Token Factory parses it live; Auspex then decides against the recorded moment.

      Input is treated as untrusted data, never as instructions. Nothing here moves a robot.

      Decision

      Submit, or pick an example.

      Benchmark

      A = AMRs alone (Open-RMF). B = Auspex, rules only. C = Auspex + Nemotron. Same scenario scripts, measured on deliberately hard cases.

      Audit ledger

      Append-only, SHA-256 hash-chained. Every fleet change is written here before it is reported done. Verification runs in your browser.

        Self-host: run the full stack locally

        Simulated clinic (Gazebo + Open-RMF), ROS bridge, Auspex core (FastAPI + Postgres), Nemotron on Nebius Token Factory. One machine.

        Requirements

        • Windows with WSL2 running Ubuntu 24.04 (or native Ubuntu 24.04). 16 GB RAM recommended; with the Gazebo GUI, more.
        • ROS 2 Kilted, ros-kilted-rmf-dev 0.2.0, rmf_demos built in ~/rmf_ws
        • Docker Engine inside WSL2, uv, Python 3.12
        • Node 20+ installed inside WSL (not the Windows one) for this site
        • Optional: Nebius Token Factory and Tavily API keys. Without them a deterministic stand-in model runs.

        Steps

        Each long-running command gets its own terminal. All commands are bash inside WSL.

        1. Clone and bootstrap

          git clone https://github.com/ambica-p/auspex
cd auspex
cp .env.example .env        # set the passwords; API keys optional
scripts/bootstrap           # Python environments (core + ROS bridge)
scripts/check-env

        2. Database

          scripts/db-up
scripts/migrate

        3. Open-RMF API server

          (terminal 1; the fleet adapters connect to it)
          docker run --network host -it --rm ghcr.io/open-rmf/rmf-web/api-server:kilted-nightly

        4. Simulation

          (terminal 2). Headless by default: no Gazebo window, lowest memory.
          scripts/run-sim
          To watch the robots in Gazebo and RViz instead (needs WSLg or an X server, and noticeably more memory):
          scripts/run-sim --gui

        5. ROS bridge

          (terminal 3)
          scripts/run-bridge

        6. Core

          (terminal 4). Capture the nav graphs once with the sim up, then start the core.
          scripts/capture-navgraphs
scripts/run-core            # ops console at http://127.0.0.1:<CORE_PORT>/ui/

        7. Run a scenario

          in mode A, B or C (terminal 5)
          scripts/run-scenario s1 --config C
scripts/run-scenario s2 --config C
scripts/run-scenario s3 --config C

        8. Switch to Nemotron

          Set LLM_PROVIDER=tokenfactory, NEBIUS_API_BASE, NEBIUS_API_KEY, MODEL_TRIAGE and MODEL_REASON in .env, restart run-core, then check:
          scripts/check-llm

        9. Record replays and the benchmark

          scripts/record-replays
scripts/build-web-data

        10. This site

          (needs only the recordings, not the stack above)
          cd web && npm install && npm run dev    # http://127.0.0.1:8090/

        11. Playground backend

          (Python only, no ROS or database)
          scripts/run-sandbox --check
scripts/run-sandbox                    # http://127.0.0.1:8095
echo "SANDBOX_URL=http://127.0.0.1:8095" > web/.env.local

        Usage & rate limits

        This site

        • Simulation: pick a scenario and mode, press Play. Markers on the bar jump to key events. "Side by side" compares two modes.
        • Playground: edit the transport request and call Nemotron. If the backend is off or the budget is spent, you get the recorded answer.
        • Benchmark: measured A/B/C results.
        • Audit ledger: verify the hash chain of the run you're watching, or tamper with it.

        LLM tokens per recorded run

        Playground limits

        • Total LLM budget: $0.02. When it's spent, the recorded answer is shown.
        • Per-IP rate limit and a daily call cap.
        • Identical inputs are served from cache, at no cost.
        • 200 characters per field; strict request schema.