Household Robot Training Data: Collecting Task Video in Real Homes (2026)

Generalization tracks the number of distinct homes far more closely than the hours recorded inside each one. Fifty hours from five homes teaches a model less than fifty hours from fifty homes. Before recruiting, define the variables the dataset must span and a target count for each.

Cultural variation is a real axis in APAC. Vietnamese home kitchens often centre on a gas hob and a rice cooker, with prep at floor level or on a low counter. A Thai kitchen may rely on a mortar and pestle and a wok. Singapore apartments tend to have a compact galley with an induction hob and the washing machine close by. Dishwashers are much less common in many Southeast Asian homes than in North America or Western Europe; handwashing and drying racks dominate instead. A home robot trained on Western kitchen data will first meet chopsticks, woks and wet-market produce in deployment unless the dataset includes them.

Recruiting homes is harder than recruiting people. The whole household has to accept cameras in a private space and a visit schedule, plus rules for other members who may appear or need to stay out of frame. Participants need to be comfortable wearing a head-mounted rig for sessions of 30-90 minutes and following a light protocol without performing for the camera.

Managed programs recruit through existing participant networks and screen households against the diversity matrix. Onboarding is short: a rig fitting and a practice session, plus a briefing on what to do when someone who has not consented walks in. Setup speed also matters, since a field team that needs hours to calibrate cameras in each home caps the program at a handful of homes per week.

Two collection methods apply. Onsite collection sends a trained field team into each home with the full multi-view rig, which suits programs that need synchronized third-person cameras and calibrated geometry. Crowdsourced collection ships simpler wearable kits to vetted participants who record in their own homes to a protocol, which grows home count faster at the cost of fewer views. Many programs use the first for a core dataset and the second for breadth.

A private home is the most sensitive place a collection program can operate. Egocentric cameras capture whatever the participant looks at: mail, screens, family photos. They also capture other household members, children included. Consent has to be layered: the participant, every adult in the home who may appear, and guardians for any minors, with a clear rule for what happens when someone who has not consented enters the frame.

Before recording, a room sweep removes or covers documents and screens, and the rig has a pause control. Faces and visible text are blurred automatically before data leaves the collection environment. Storage is access-controlled, with written retention and deletion terms. For collection in Singapore, Malaysia and Thailand, the local PDPA rules set the baseline, and programs serving EU clients typically add GDPR-grade transfer and processing terms. All of this belongs in the contract and the capture protocol from the start.

Household sessions sit between fully scripted and fully natural. A fully scripted session (pick up the red cup, place it in the sink) gives clean episodes but unnatural behaviour. Leave people entirely to themselves and the behaviour is realistic, but the footage is hard to segment and balance. Most programs give participants a task list with a goal and success condition for each item, such as "wash and rack the lunch dishes", and let them work their own way.

Segmentation turns a 60-minute session into training units. Each episode needs start and end timestamps, a task and sub-task label, a natural-language instruction, a success or failure flag and, where useful, a note on recovery behaviour. For VLA training the language instruction is the conditioning signal, so it cannot be an afterthought. Failed and recovered attempts are worth keeping and labelling, because policies trained only on clean successes handle mistakes badly.

QA at this stage checks that views stay synchronized across each episode and that frames stay sharp during fast hand motion. Task coverage per home is checked against the matrix too.

Fix the delivery format during the pilot. Imitation learning teams commonly want episodes in HDF5, RLDS or LeRobot format with synchronized camera streams, timestamps, calibration files and a language annotation per episode. Open X-Embodiment, which pooled 60 existing datasets from 34 labs into more than one million trajectories across 22 robot embodiments, is a good argument for standardized formats: data built to a common schema can be combined and reused across robots and models.

Human video without robot actions is still useful. Teams pre-train visual representations on it and use it to learn task structure and sub-goal order. It also supplies hand trajectories for retargeting. If the robot has to learn actions directly, go back to the same homes with the same task list and record teleoperated or handheld-gripper demonstrations. Human and robot data then share one taxonomy.

DataX Power runs household programs with the capture modes described above: egocentric rigs including Meta Aria and smart glasses at up to 4K/60fps, third-person arrays with synchronized timestamps for multi-angle 3D pose, and wrist-mounted cameras synchronized with force/torque logging where a task needs it. Collection runs onsite or crowdsourced across participant networks in Vietnam, Thailand, Singapore and Malaysia. Pilot specifications are usually ready within five business days, and programs scale from 100-hour pilots to 50,000-hour production on one contract.