# How to Import LeRobot Datasets into FiftyOne Using Agent Skills

![FiftyOne App showing a LeRobot dataset of robot arm episodes, with the sidebar filtering 800 samples down to 100 by task.](https://image.mux.com/IHIqY927o8I4OTQAbtBWblk5qu02ANUuVWWhhhVbBA4k/animated.gif?width=640)

*FiftyOne App showing a LeRobot dataset of robot arm episodes, with the sidebar filtering 800 samples down to 100 by task.*

*A standardized dataset format solved robot learning’s data fragmentation problem. Here’s how to get one of those datasets into FiftyOne without writing a parser, and what happens when the version doesn’t match what you expect.*

Robot learning needs [demonstration data](https://voxel51.com/blog/libero-dataset-curation-fiftyone) the way language models need text, but until recently every lab and vendor recorded it differently: proprietary formats per robot, inconsistent schemas, no shared way to index or search across datasets. Hugging Face built `LeRobotDataset` to fix that: one standardized format (Parquet for state and action, MP4 for camera feeds) across radically different [embodiments](https://voxel51.com/embodiment), from SO-100 arms to bimanual ALOHA rigs to industrial Kuka arms.

The standardization paid off measurably. SmolVLA, a 450M-parameter [policy](https://voxel51.com/policy), was pretrained only on community-uploaded `lerobot`-tagged datasets on the Hub, and that alone lifted SO-100 task success from 51.7% to 78.3% ([source](https://huggingface.co/blog/lerobot-datasets-v3)). The format itself had to evolve under that success: the original v2.x layout stored one Parquet file and one MP4 per [episode](https://voxel51.com/episode), which started hitting filesystem limits once [community datasets](https://voxel51.com/blog/open-x-embodiment-fiftyone-data-curation) reached millions of episodes. [`lerobot`](https://voxel51.com/lerobot) v0.4.0 shipped v3.0 in response, packing many episodes into shared chunked files and supporting streaming without a full download first.

FiftyOne ships `fo.types.LeRobotDataset` as a first-class import path, built around one idea: a sample is an episode, not a file. The `fiftyone-lerobot-import` [agent skill](https://voxel51.com/blog/fiftyone-agent-skills-mcp-tools) wraps that native support with the parts a script usually skips: detecting whether a Hub dataset is still on v2.x, running the documented conversion before import, and refusing to create a dataset or install a package without asking first.

## Key takeaways

- The fiftyone-lerobot-import agent skill imports LeRobotDataset v3.0 recordings into FiftyOne, where one sample is one episode rather than one file.
- The skill auto-detects LeRobot's older v2.x layout (`meta/episodes.jsonl`, `meta/tasks.jsonl`) and runs the documented conversion to v3.0 before importing, instead of failing on an unsupported-version error.
- Two explicit stop gates hold the run, one before installing any package (pyarrow, huggingface_hub, lerobot) and one before creating the FiftyOne dataset.
- Tested end to end against `lerobot/aloha_sim_insertion_human`, a real v2.1-tagged Hub dataset, which converted to v3.0 and imported as 50 episodes of a bimanual ALOHA arm with zero episodes skipped.
- The skill is a thin layer over the open-source FiftyOne SDK (`fo.Dataset.from_dir`, `dataset.add_dir`), so the same import runs identically from a chat prompt or a checked-in script.

## Part one: what is an agent skill, actually?

Strip away the buzzwords and a skill is a folder. Inside it is a markdown file that tells an AI coding agent, step by step, how an expert would do a specific job: which files to check first, which command to run, which traps to avoid, and how to tell success from failure. The agent reads that folder like a new hire reading a training guide, then does the work itself in your terminal, on your data.

Voxel51 packages this kind of runbook for FiftyOne into an [open-source repository](https://voxel51.com/blog/antigravity-cli-fiftyone-skills), installed with one line:

```
curl -sL skil.sh | sh -s -- voxel51/fiftyone-skills
```

The installer clones the repo, discovers every skill inside it, and lets you pick which ones your agent should have access to going forward.

This post is about `fiftyone-lerobot-import`. Point it at a local LeRobot directory or a Hugging Face Hub repo id, and it detects the version, converts if needed, and builds the FiftyOne dataset. That’s a different problem from the one [`fiftyone-dataset-import`](https://voxel51.com/blog/mcap-import-fiftyone-agent-skills)’s [MCAP](https://voxel51.com/mcap) path solves. MCAP’s question is a schema scan: what kind of sensor data is this? LeRobot’s question is about versioning: what version of this format is this, and does it need conversion first?

## Part Two: A Real LeRobot Dataset, End to End

A skill only proves itself on data that actually exercises its hardest path. A dataset already sitting on v3.0 would import cleanly and tell you nothing about the conversion logic, so we looked for a dataset still tagged v2.1 on the Hub.

### The dataset: lerobot/aloha_sim_insertion_human

`lerobot/aloha_sim_insertion_human` is one of LeRobot’s official example repos: a bimanual ALOHA arm running a simulated peg-insertion task, one task label (“Insert the peg into the socket.”), 50 episodes, 50 fps, 25,000 total frames.

| Detail | Value |
| --- | --- |
| Repo | lerobot/aloha_sim_insertion_human |
| Revision pulled | v2.1 |
| Episodes | 50 |
| FPS | 50 |
| robot_type | aloha |
| Camera | observation.images.top (video, av1 codec, 480×640) |
| observation.state / action dims | 14 / 14 (bimanual: waist, shoulder, elbow, forearm roll, wrist angle, wrist rotate, gripper, same layout on both arms) |
| Task | “Insert the peg into the socket.” |

*What lerobot/aloha_sim_insertion_human contains, pulled at the v2.1 tag.*

We pulled the Hub’s `v2.1` tag specifically, not `main` (already v3.0), so the conversion path would get exercised for real instead of simulated.

### The prompt and the version-detection moment

The prompt to the agent was one sentence: import `lerobot/aloha_sim_insertion_human` into FiftyOne.

The skill’s inspection step reads `meta/info.json` and finds `codebase_version`: v2.1, along with layout markers like `meta/episodes.jsonl` and `meta/tasks.jsonl`. Instead of failing on unsupported versions, the skill detects the older format, hits the packages stop gate, and runs the conversion script (python-m lerobot. scripts. convert_dataset_v21_to_v30) to update the dataset to v3.0, ensuring reliable import even with legacy data.

1. Hits the packages stop gate and asks before installing `lerobot` (needed only for the conversion).
1. Runs the conversion: `python -m lerobot.scripts.convert_dataset_v21_to_v30 --repo-id lerobot/aloha_sim_insertion_human --root <path> --push-to-hub=false`.
1. **Keep it honest.** The installed lerobot release (v0.6.2) didn’t expose the converter at the documented `lerobot.datasets.v30.*` module path; it was still at the older `lerobot.scripts.*` path. The skill’s own troubleshooting notes already anticipate this exact fallback, and it’s what happened on the first real run, not a hypothetical edge case.
1. Re-checks `meta/info.json`, confirms `codebase_version: v3.0`, and the new layout: `meta/episodes/chunk-000`, `meta/tasks.parquet`, `meta/stats.json`.
1. Hits the creation stop gate and presents the full import plan before touching anything:

```
LeRobot Import Plan for lerobot/aloha_sim_insertion_human:

Source:
  - codebase_version: v3.0 (converted from v2.1: yes)
  - 50 episodes, 50 fps, robot_type: aloha
  - camera: observation.images.top (video, av1)
  - observation.state: 14 dims, action: 14 dims
  - tasks: 1 ("Insert the peg into the socket.")

Import:
  - Dataset name: lerobot-aloha-sim-insertion-human
  - Episodes: all
  - Persistent: yes

Proceed with import? (yes/no)
```

### The payoff: 50 episodes, zero skipped

`dataset.media_type` comes back `"multimodal"`. All 50 episodes import; `dataset.info["lerobot"]["skipped_episodes"]` is an empty list: a clean import, worth stating plainly rather than manufacturing a gap that isn’t there. Every episode runs exactly 10 seconds (`dataset.bounds("duration")` returns `(10.0, 10.0)`), which checks out at 50 fps × 500 frames/episode.

Opening episode 0 in the App shows the `observation.images.top` tile: two robot arms working over a blue and red object, with the episode header reading “Insert the peg into the socket. · 10s.” That’s what “one sample equals one episode” looks like in the viewer, not an abstraction: a single click surfaces the whole 10-second demonstration, not one video frame.

## Part Three: The Same LeRobot Import, in Code

A chat prompt is a good way to explore a dataset you haven’t seen before, but a pipeline that runs on every new upload needs code, not a conversation. Everything the skill did above sits on top of the open-source FiftyOne SDK. Here’s the full pipeline, runnable from a fresh environment against the same Hub dataset used in Part Two.

### Step 1: Install the dependencies

```
pip install "pyarrow>=10.0.0" huggingface_hub lerobot
```

`lerobot` is only needed for the v2.1 → v3.0 conversion below. A dataset already on `codebase_version: v3.0` only needs `pyarrow` and `huggingface_hub`.

### Step 2: Download from the Hub

```
fo.Dataset.from_dir reads from a local directory, not directly from the Hub, so pull the files down first:
from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="lerobot/aloha_sim_insertion_human",
    repo_type="dataset",
    revision="v2.1",
    local_dir="/abs/path/lerobot/aloha_sim_insertion_human",
)
```

### Step 3: Convert v2.1 to v3.0

```
python -m lerobot.scripts.convert_dataset_v21_to_v30 \
    --repo-id lerobot/aloha_sim_insertion_human \
    --root /abs/path/lerobot/aloha_sim_insertion_human \
    --push-to-hub=false
```

Skip this step for any dataset already tagged `v3.0` on the Hub; for example, pull `revision="v3.0"` in Step 2 instead.

### Step 4: Import into FiftyOne

```
import fiftyone as fo

dataset = fo.Dataset.from_dir(
    dataset_dir=local_dir,
    dataset_type=fo.types.LeRobotDataset,
    name="lerobot-aloha-sim-insertion-human",
    persistent=True,
)

assert dataset.media_type == "multimodal"
fo.launch_app(dataset)
```

Adding a second recording session to the same dataset is the same four steps repeated against the new source directory, then `dataset.add_dir(dataset_dir=..., dataset_type=fo.types.LeRobotDataset)` instead of `from_dir`.

No Parquet reader, no video decode loop, no custom viewer. The importer reads `meta/info.json` and `meta/episodes/*.parquet`, nothing in the video files, so it stays fast even on large sources. The skill’s stop gates are a conversation; the SDK calls underneath are the same ones either way.

## Where This Leaves You

Every team re-solves “is my data in the format my tools expect” by hand, one script at a time, and LeRobot is just the latest format to need that check. `fiftyone-lerobot-import` turns that check into something an agent already knows how to run correctly against your files, every time.

- Dataset: [`lerobot/aloha_sim_insertion_human`](https://huggingface.co/datasets/lerobot/aloha_sim_insertion_human), part of LeRobot’s official example datasets, Hugging Face
- Skill repo: [`voxel51/fiftyone-skills`](https://github.com/voxel51/fiftyone-skills), installed the same one-line way as Part One

## Frequently Asked Questions

### How do you import a LeRobot dataset into FiftyOne?

Two ways: ask an agent with the `fiftyone-lerobot-import` skill, or call `fo.Dataset.from_dir(..., dataset_type=fo.types.LeRobotDataset)` directly. Both require the dataset to be on `codebase_version: v3.0`.

### What’s the difference between LeRobot v2.x and v3.0?

v2.x stores one Parquet file and one MP4 per episode. v3.0 packs many episodes into shared chunked files and supports streaming. `fo.types.LeRobotDataset` only supports v3.0. v2.x sources need conversion first.

### Do I need the lerobot package to import into FiftyOne?

Only for converting a v2.x source to v3.0. A straight v3.0 import needs only `pyarrow>=10.0.0`.

### How does FiftyOne represent a LeRobot dataset?

One sample per episode, not per file, with `media_type == "multimodal"` and fields like `episode_index`, `task`, `duration`, and `fps`.

### What happens if some episodes in the source are malformed?

They’re reported under `dataset.info["lerobot"]["skipped_episodes"]` instead of being silently dropped or silently accepted.

---

*This post first appeared as [the original article](https://voxel51.com/blog/lerobot-import-fiftyone-agent-skills). I'm Harpreet Sahota, Hacker-in-Residence at [The Employable Data Scientist](https://employabledatascientist.com), where we help AI and developer-tool teams ship prototypes, evaluations, and technical content. I also host the [Artists of Intelligence](https://employabledatascientist.com/podcast/) podcast. For new field notes by email, [subscribe on Substack](https://harpreetsahota.substack.com).*