Robot Navigation Training Data for AMRs, Delivery and Legged Robots (2026)

Robot navigation training data is the footage and sensor logs a mobile robot learns to move from: where it can drive, what counts as an obstacle, how pedestrians behave around it, and which command a competent operator would give next. For autonomous mobile robots (AMRs) in warehouses, delivery robots on sidewalks and legged robots on mixed terrain, the model is only as good as the range of places and situations the data covers.

Navigation programs often get scoped as if they were ordinary video collection, and most of the later problems start there. Manipulation data is usually recorded at a fixed workstation where the scene is controlled. Navigation data is recorded while the camera moves through spaces the program does not own, such as public footpaths or a live warehouse aisle, and a frame is only useful if it can be matched to the robot's pose and speed at that instant and to the command it was following. A clean 4K video with no pose or command stream is a perception asset at best, and of little use for policy learning.

For anyone scoping a mobile robot navigation dataset, the expensive mistakes tend to happen in the specification, well before the first route is driven. A camera mounted at the wrong height, or a command stream nobody thought to log, cannot be fixed in post-processing.

Most navigation programs serve two internal customers. Perception teams need diverse, well-exposed imagery they can label for detection, segmentation and traversability. Policy teams, whether they train end-to-end models or learned local planners, need action-labelled trajectories: synchronized sensor observations paired with the velocity or joystick commands that produced the motion. A dataset specified for one team rarely satisfies the other. Agree the split between them before anyone records a route.

Public datasets show the pattern. SCAND, from the University of Texas at Austin, recorded 8.7 hours and 138 trajectories of socially compliant navigation on a Boston Dynamics Spot and a Clearpath Jackal, logging lidar, cameras, odometry, inertial data and the joystick commands of four human operators. The commands are what make it a demonstration dataset rather than a video archive.

Viewpoint is the second decision. On-board cameras, mounted at the robot's actual height and angle with the production lens, see what the deployed model will see, so they carry most of the training value. A chase camera on a follower or at a fixed point behind or beside the robot shows how people react to the robot, and it records near misses and failures in context. Most programs give it a smaller share of hours and use it for evaluation and safety review, plus some interaction labelling.

On a navigation rig, timing matters more than camera resolution. Cameras, IMU, wheel encoders, lidar and GPS run on separate clocks at different rates. If the camera stream drifts 80 milliseconds against odometry, a robot moving at 1.5 metres per second carries 12 centimetres of disagreement between where the image says it is and where the pose says it is. You will not catch that in a video review. It tends to show up much later, as a policy that brakes late.

Production rigs use hardware triggering or a shared time source, such as PTP on the sensor network or GPS-disciplined time outdoors, and log every stream against a common clock. Calibration covers camera intrinsics plus camera-to-IMU and camera-to-lidar extrinsics, and it gets re-checked every time someone touches a mount. The UT Campus Object Dataset (CODa) is a good reference: hardware-synchronized lidar, stereo RGB, RGB-D and 9-DoF IMU, captured over repeated traversals of the same campus routes in different weather and at different times of day.

There are three ways to carry the rig. Collecting on the client's own robot gives the most faithful data but depends on shipping or leasing hardware to the collection site. A proxy platform - a push cart, wheeled base or backpack rig matched to the robot's camera height and field of view - is cheaper and faster to scale, but it produces observations without the target robot's actions. Teleoperating the real robot through the target environments produces the action-labelled data policy teams want. Many programs mix them: proxy rigs for perception breadth, the real robot for a smaller volume of policy demonstrations.

Environment type changes almost every capture decision, from sensor choice to how much chase-camera footage is worth paying for. The table summarizes the three environment families most navigation programs cover, plus the transition zones between them that are easy to forget.

Warehouses look structured. The aisles repeat and the floors are flat and evenly lit, which is exactly why teams under-collect them. The failures come from everything else: a forklift reversing out of a cross-aisle, shrink-wrap trailing across the floor, a picker kneeling below the lidar plane, reflective floor tape that confuses depth. AMR training data needs to capture those interruptions at the rate they occur on a busy shift, not the rate they occur at 6am on a Sunday.

Plan collection across shift patterns and seasonal peaks, and log which zones were active. Racking and staging areas move, so re-record the same routes whenever the layout changes. Chase-camera footage is worth less indoors than outdoors; a few fixed overhead cameras at intersections usually give better interaction coverage per hour spent. The facility sets the real schedule: site access and safety induction take time, as does working around live forklifts, so agree all of it before the program starts.

Pedestrian interaction data matters most on sidewalks. Delivery robots share footpaths with people who stop suddenly or walk in groups, some with prams or dogs, and the way they step around a robot is something simulators predict poorly. The capture plan needs deliberate coverage of each crowd density band, from empty footpaths to congested ones, plus the moments where the robot has to yield, pass or wait.

Geography matters more than many teams expect. nuScenes recorded its 1,000 driving scenes in Boston and Singapore, so one dataset contains both right-hand and left-hand traffic. Sidewalk behaviour varies between cities in much the same way. Footpaths in Bangkok, Hanoi or Kuala Lumpur carry parked motorbikes, street vendors and uneven kerbs that a dataset recorded in San Francisco will not contain. If the robot will deploy in APAC cities, collect in them.

Outdoor programs also carry a privacy burden that indoor programs mostly avoid. Faces and licence plates appear in almost every frame. Before anyone films, the program needs a documented position on signage and local permits, plus written rules on blurring before delivery and on retention. Leaving it until delivery is too late.

Legged robots and all-terrain platforms need to learn traversability, which vision alone cannot label. Tall grass and a hedge look similar in RGB; one can be walked through and one cannot. Useful datasets pair camera and lidar with proprioceptive signals such as IMU, joint or leg odometry, wheel slip and shock travel, so a model learns what terrain feels like as well as how it looks. TartanDrive, from Carnegie Mellon University, took this approach for off-road driving, logging roughly 200,000 interactions on a modified ATV with seven sensing modalities.

GND, from the University of Maryland and George Mason University, works at campus scale. It covers over 11 hours across ten university campuses, recorded with lidar, IMU and GPS plus RGB and 360-degree cameras, and maps the ground into five traversability categories (pedestrian walkways, vehicle roadways, stairs, off-road terrain and obstacles). A commercial program can borrow a similar category scheme and add terrain classes specific to its deployment site.

Transitions deserve their own quota. The moments where a robot passes through a door, enters a lift, climbs a ramp or crosses from shade into full sun are short, so they end up badly under-represented if you only log continuous routes. Script them instead: a fixed number of passes per doorway type and lighting condition, recorded on purpose.

Hour count is a poor measure of a navigation dataset. Two hundred hours of the same loop in fine weather adds less than twenty hours designed against a coverage matrix. The matrix should list environment, route, time-of-day band (day, dusk, night, artificial light), weather (dry, wet, heavy rain where relevant), crowd density band and surface type, with a target number of minutes or passes for each cell.

Edge cases need explicit quotas too, or the rare ones barely appear. On the people side: someone stepping out from behind an occlusion, a group that does not part, children or pets, cyclists or e-scooter riders on the footpath, wheelchairs and prams. On the environment side: temporary signage or roadworks, glass and mirrors, puddles with wet reflections, objects moving toward the robot. Some can be staged with consenting participants on private sites. Others only appear in real traffic and need longer capture windows to sample.

Teleoperated segments carry a different risk, which is that the policy learns each operator's habits. Operators should drive the way the deployed robot is meant to behave (yielding distance, passing side, speed near people) rather than however they would drive on their own. SCAND used four operators; commercial programs usually want more, with a written driving guideline and periodic review of each operator's traces. Simulation can fill part of the gap for rare events, but pedestrian behaviour is one of the areas where synthetic data transfers worst.

Most scoping delays trace back to the questions in the checklist below. Two cause the most rework when left vague: the time-sync tolerance between streams and the delivery format. MCAP is a sensible default for new programs. It is a serialization-agnostic container for timestamped multi-channel logs, and it is also the default log format in ROS 2, which means it opens in standard tooling without custom converters. Policy teams often want a second copy converted and segmented into episodes.

A managed vendor turns that specification into a capture protocol and proves it in a pilot run at production standards. At DataX Power, mobile navigation video is one of four capture modes. It covers on-board and chase-camera footage across warehouses, outdoor terrain and mixed environments, and it includes pedestrian interaction and obstacle avoidance as well as edge cases. We usually return a pilot specification within five business days and onboard programs in about two weeks. We handle hardware and participants, consent, QA and delivery end to end. Each batch is checked by QA engineers trained on robotics video for temporal consistency and frame quality as well as task coverage. Programs scale from 100-hour pilots to 50,000-hour production on the same contract, drawing on participant networks in Vietnam, Thailand, Singapore and Malaysia.

Robotics and data leads raise these most often when scoping a navigation collection program.