Research — Yanheng Zhu

Research index

Four studies

Selected research

Monocular multi-goal navigation

GEMA Streaming Geometry Model Is an Episodic Memory: Monocular Multi-Goal Navigation Without a Pose Oracle

Yanheng ZhuLead author

Submitted to ICRA 20272026

GEM streaming geometry state, episodic archive, verified goal readout, and frozen-controller adapters — Method overview

A robot may need to return to a place after its view has left the controller’s short context. GEM uses the persistent state of a frozen streaming geometry model as episodic memory, paired with an indexed RGB history. It retrieves and geometrically verifies a historical view that connects the goal to the current camera. The recovered goal direction or supporting image then guides a frozen navigation policy. If verification fails, the policy continues with its native goal request.

My contribution:I lead the research and manuscript, from the memory method and simulation experiments to Go2 deployment and real-world evaluation design.

Method & evaluation

Method

Frozen LingBot-Map state and indexed RGB observations persist across successive goals, without additional training or an external pose oracle.
A verified historical view bridges goal-to-history matching and history-to-current geometry to recover a scale-free goal bearing.
NavDP receives a bearing cue and monocular depth; ViNT and NoMaD use the verified historical image through their native goal interfaces.

Evaluation

Paired HM3D and MP3D evaluations compare supported revisits and novel goals under NavDP, ViNT, and NoMaD. On a Unitree Go2, both real-world arms use frozen NavDP with predicted monocular depth; sensor depth is reserved for local safety.

Go2 baseline: 4/30
Go2 + GEM: 25/30
Test settings: 8 indoor + 2 outdoor

The real-world study uses 30 trials per method, paired by start pose, goal image, and prior survey, across eight indoor and two outdoor settings. Recall needs verifiable historical overlap; pose drift and obstacle detours can limit a bearing cue. Archived memory still grows with episode length.

Cooperative ObjectNav

TopoFocusCombating Visual Context Overload in Vision-Language Models for Cooperative Embodied Navigation

Yanheng ZhuReal-robot systems and evaluation

Research project2026

TopoFocus system architecture from calibration and semantic mapping to multi-robot coordination and safe execution — System architecture

As a robot team explores, its shared map becomes visually denser even though the next decision depends on only part of it. TopoFocus addresses this focus dilution by organizing the map into persistent room units, preserving recent changes and nearby room context. After visual encoding, room summaries and a bounded set of active patches guide the vision-language planner toward the next search region.

My contribution:I built the real-robot system: cross-robot calibration, shared mapping, coordinated navigation, and safety checks on a Go2 and a wheeled robot.

Method & evaluation

Method

Geodesic room decomposition and temporal IoU tracking keep room identities consistent across planning steps.
Geometric patch selection retains recent changes, the previous frontier’s room, and its one-hop neighbors.
Training-free selection after ViT encoding supplies room-pooled tokens plus up to 256 active patch tokens to the Decision-VLM.

Evaluation

The manuscript evaluates HM3D_v0.2 and MP3D, team sizes from one to five robots, and a heterogeneous Go2–wheeled-robot deployment. A shared hub handles mapping and goal coordination; each robot retains local planning and safety control.

HM3D SR / SPL: 0.767 / 0.445
MP3D SR / SPL: 0.619 / 0.384
Chair-search trials: 5/5
Plant-search trials: 3/5

These are the manuscript’s benchmark results and ten physical target-search trials. The physical study validates deployment and does not compare against a full-map control. All 1,600 patches still pass through ViT; the 256-patch cap applies after encoding, with room summaries added to the LLM context.

Vision–language–action

G³VLAGeometric Inductive Bias for Vision-Language-Action Models

Yue Peng, Yongzhe Zhao, Artur Habuda, Khuyen Pham, Yanheng Zhu, Tran Nguyen Le, Fares Abu-Dakka, Li Guo

Preprint2026

G3VLA camera-aware geometric module and two-stage geometry distillation pipeline — Method architecture

Multi-camera robot policies often process calibrated views as independent 2D images. G³VLA gives visual tokens their viewing directions and lets them exchange information across geometrically related camera views before action prediction. Geometric supervision, including teacher distillation, trains this pathway while preserving the pretrained backbone, action space, and imitation objective.

My contribution:I contributed to camera-aware features, multi-view fusion, geometry distillation, and evaluation.

Method & evaluation

Method

Intrinsic-conditioned ray embeddings derived from K⁻¹
PRoPE and bidirectional cross-view fusion
Two-stage training with ground-truth or confidence-gated π³X point maps

Evaluation

π₀, π₀.₅, and GR00T 1.5 across LIBERO, RoboCasa24, RoboTwin2.0, plus a bimanual UR5 camera-shift benchmark.

π₀ LIBERO baseline: 84.6%
π₀ LIBERO · GT geometry: 88.1%
π₀ LIBERO · π³X teacher: 87.0%

Gains depend on supervision and the task. On RoboTwin2.0 handover_block, success is 44.0% for the baseline, 49.0% with ground-truth geometry, and 41.0% with π³X supervision. GR00T 1.5 results are mixed; stronger geometric transfer through direct action pathways remains a hypothesis.

Multi-robot planning

Bounded-Context PlanningBounded-Context Planning for Efficient Multi-Robot Object-Goal Navigation

Yanheng ZhuResearch contributor

AAAI 2027Phase 2 review

Bounded-Context Planning architecture with semantic-spatial grouping and goal-conditioned team assignment — Planning architecture

A growing map is useful memory, but repeatedly sending all of it to a planner makes robot-team decisions costly. This framework separates persistent spatial memory from the context used for each decision. Incremental semantic-spatial groups yield target-relevant summaries for team selection; a local language model then ranks a bounded set of executable waypoints. This keeps coordination focused on useful, complementary search regions within edge-compute and communication constraints.

My contribution:I contributed to map summaries, robot selection, waypoint ranking, and evaluation on Jetson hardware.

Manuscript in review; no public paper link yet.

Method & evaluation

Method

Incremental Semantic-Spatial Grouping combines semantic compatibility with geodesic connectivity to separate disconnected regions.
Target-conditioned summaries support team selection using relevance, travel cost, exploration support, and cross-robot support overlap.
LLaMA-3.1-8B ranks bounded owner-local waypoint candidates; detailed maps and graph structure remain local.

Evaluation

Two-robot ObjectNav is evaluated on HM3D and MP3D, with team scaling to five robots on HM3D. The controlled context comparison uses the same LLaMA-3.1-8B and Jetson AGX Orin 64GB. Latency covers summary construction through waypoint decisions and excludes path execution.

HM3D SR / SPL (%): 72.8 / 40.2
MP3D SR / SPL (%): 57.6 / 34.4
Median input tokens: 5.8k → 0.92k
Planning p50: 2.30 → 0.49 s
Planning p95: 8.57 → 0.61 s
Planning payload p95: 110.0 → 4.1 kB

Planning payloads exclude perception, localization, and simulator-synchronization traffic. The bounded interface limits decision context, while the underlying map and memory may still grow. The current owner-local design does not dynamically reassign regions between robots.