Chapter 5: The Simplest Learned Policies — From Behavior Cloning to ACT

Overview

Chapter 4 preserved human intent, mapper transforms, safety projection, and the action actually sent to the controller on one episode timeline. We can now train the simplest learned policy from that record. The first candidate is behavior cloning (BC). It turns imitation into supervised learning: provide an observation and predict the expert action. This absorbs visual variation that strains a classical rule set without first defining a reward or allowing online exploration.

A low validation loss does not show that a robot can recover from states it creates. A small deployed error changes the next observation, which may move farther from the demonstration distribution. Action chunking predicts several future actions together and can create short-term coherence, but a longer chunk may also delay the use of a new observation. This chapter does not present ACT as merely a larger model. It approves complexity in the sequence plain BC → deterministic chunked BC → ACT, while holding the dataset, action semantics, controller, and evaluation protocol as constant as possible.

Evidence boundary: Learner-state labeling in DAgger is supported by the original paper ^[2]. The conditional-VAE Transformer, action chunks, and aggregation of overlapping predictions are described from the original ACT/ALOHA paper ^[7]. The Capuano tutorial supplies a teaching sequence and a LeRobot implementation map, not sole evidence for ACT results or current APIs ^[13]. The timing budget, stale-action rejection policy, and promotion gates below are S12 engineering synthesis, not universal safe settings.

After reading this chapter... - You can distinguish the supervised BC objective from closed-loop covariate shift. - You can explain which states and targets DAgger, perturbation, and human intervention add. - You can set chunk length, observation horizon, execution horizon, and temporal aggregation separately. - You can compare a conditional VAE with a deterministic baseline without assuming that a latent variable automatically solves multimodality. - You can measure controller rate, inference latency, action age, and queue state and reject a stale chunk. - You can promote a checkpoint through offline, simulation, shadow, and bounded-hardware gates.

The experiment question is: When plain BC and ACT are trained on the Chapter 4 single-arm tabletop-assembly dataset, how do we separate an improvement near insertion into effects from the model architecture, chunking, temporal aggregation, and intervention data rather than attributing everything to “ACT”?

1. Make plain behavior cloning the first learned baseline

BC learns a policy \pi_\theta that maps observation o_t to action a_t from an expert dataset D=\{(o_t,a_t)\}. A minimal continuous-action baseline can use a regression loss with a validity mask m_t and per-dimension scale w:

$$

\mathcal{L}_{BC}(\theta)=

\frac{1}{\sum_t m_t}

\sum_t m_t\left\|w\odot\left(a_t-\pi_\theta(o_t)\right)\right\|_1.

$$

The observation may contain wrist and scene images, joint and gripper state, and a task identifier. The action is not an arbitrary choice among the Chapter 4 raw, mapped, and sent fields. First decide which interface the policy owns and fix the target semantics. If the policy proposes a bounded Cartesian command before safety projection, evaluate the projection delta as well. If sent actions are used directly, supervisor modifications can be mistaken for expert intent; rejected and clamped segments need masks or explicit labels.

Learning-from-demonstration research has long separated demonstration source, representation, policy learning, and execution rather than treating them as one operation ^[1]. RoboMimic similarly compares how data, observation modalities, architectures, and training choices interact in offline robot imitation ^[5]. The first run should therefore expose whether data and controller semantics agree. It is the smallest executable learned baseline, not a contest for a headline score.

1.1 Keep the first baseline deliberately small

Adding an encoder, temporal model, generative head, augmentation, and intervention data at once destroys failure attribution. Begin with a deterministic policy that consumes one observation or a short observation window. Combine image and proprioception encoders, regress one fixed action representation, and fit normalization on the training split only. Freeze seeds, optimizer, augmentation, frame selection, and transform versions in configuration.

Implicit Behavioral Cloning provides an energy-based alternative for multimodal actions ^[6], but it need not be the first baseline. Candidate generation, energy sampling, and an inference budget add moving parts that can conceal a unit, sign, frame, or split defect visible under plain regression. Chapter 6 compares generative policies under the same task and timing contract.

1.2 Offline loss is necessary, but it is not a promotion metric

Training and validation losses reveal whether the pipeline learns at all and help diagnose overfit or underfit. Averaged action error can nevertheless combine a small Cartesian error near contact with an equally sized, less consequential free-space error. If actions are noisy or two valid motions diverge from a visually similar observation, a mean prediction can lie between useful modes.

Stratify the offline report by task phase, object/scene/operator split, intervention state, projection state, and action dimension. In addition to one-step error, record chunk rollout error, endpoint error, velocity and acceleration discontinuity, gripper-transition timing, and disagreement among overlapping predictions. These metrics are diagnostic artifacts, not replacements for closed-loop evidence.

2. Covariate shift is created by the closed loop

An expert-only dataset primarily contains states from d_{\pi_E}. Once \pi_\theta controls the system, a small action error changes the next state and the policy acts under its own distribution d_{\pi_\theta}. If learner-induced states are absent from the demonstrations, a low one-step loss supplies no recovery target. Training and deployment distributions differ partly because the deployed policy changes them. BC can therefore accumulate error after entering learner-created states; the magnitude and failure pattern are task- and platform-dependent, and DAgger's theoretical reduction must not be converted into a cell-specific failure rate ^{[2] [1] [3]}.

Collecting more episodes does not necessarily resolve the problem. Repeating successful trajectories from the same initial condition can make the expert tube denser without covering departures from it. Coverage must include exogenous variation—object pose, lighting, and fixture tolerance—and recovery states near a slightly rotated grasp, a delayed approach, or another policy-induced deviation.

2.1 DAgger changes the label distribution, not the reward

DAgger rolls out the current learner, asks an expert to label states the learner visits, aggregates those examples, and retrains ^[2]. The key change is supervised labeling on the learner distribution instead of collecting only states along expert trajectories.

$$

D_{i+1}=D_i\cup\{(o,\pi_E(o)):o\sim d_{\pi_i}\},

\qquad

\pi_{i+1}=\operatorname{Train}(D_{i+1}).

$$

This expression does not authorize unrestricted learner control on hardware or deliberate creation of unsafe states. Start with simulation, recorded perturbations, shadow proposals, or reduced-speed bounded rollouts in which an expert can observe and label. DART perturbs supervisor actions to collect recovery-like states ^[3]. HG-DAgger lets a human expert choose when to intervene in an interactive variant ^[4]. Neither replaces a local risk assessment, measured takeover latency, or an independent stop path.

2.2 An intervention is not automatically a DAgger label

Reuse the Chapter 4 INTERVENTION_REQUESTED → HUMAN_RECOVERY → HUMAN_CORRECTION timeline. If a person needs time to recognize risk and acquire controller authority, the first accepted takeover action may not label the state at which intervention was requested. The state and camera frame have already changed. Current LeRobot HIL documentation provides an implementation pattern that distinguishes autonomous and correction segments ^[17], but the cell must validate the safety and semantic meaning of the authority transition.

Separate intervention records into three types:

• query label: an expert target for a frozen or recorded state without moving the robot;
• takeover action: the action sent after human authority is accepted by the controller;
• recovery trajectory: a sequence that returns to a safe state or task manifold and may not be a nominal target for the original subtask.

Do not merge all three into one expert_action. Give the policy proposal, intervention request, accepted takeover, and recovery completion their own timestamps and state snapshots. TRANSIC is a useful candidate showing how online correction can support recovery from simulation gaps ^[11], but its apparatus-specific result is not transferred to this cell. Interventions are biased toward failures people notice and do not cover unseen hazards.

2.3 Bound the correction loop

Perform the first correction round in simulation or replay. In the second, record policy proposals while a validated classical baseline moves the robot. The third may use a bounded rollout under reduced workspace, speed, and force limits, with an expert and independent stop authority. After every round, test whether new data expand learner-state coverage or only repeat easy recoveries.

Stop the round if takeover p95 exceeds the task response budget, stale policy actions continue after an authority transition, correction episodes leak test objects or scenes into training, recovery actions mixed with nominal targets create oscillation, expert fatigue reduces consistency, or safety events rise while task success rises. An improvement purchased by more hazardous exposure is not a promotion.

3. Action chunking makes time part of the prediction target

When plain BC predicts one action per step, observation noise and small regression errors may appear as command jitter. Action chunking predicts K future actions from time t:

$$

\hat A_t=\pi_\theta(O_{t-H_o+1:t})

=\left[\hat a_{t|t},\hat a_{t+1|t},\ldots,\hat a_{t+K-1|t}\right].

$$

H_o is the observation horizon and K is prediction length. The client does not need to execute all K actions. It can execute an H_e\le K prefix and replan from a new observation. Without this distinction, “chunk size 20” does not say whether 20 steps are predicted or run open loop.

In the reported bimanual system, ACT predicts action chunks with a conditional-VAE Transformer and aggregates overlapping predictions produced at different start times ^[7]. The mechanism can be tested on the single-arm assembly, but the original camera layout, action representation, task horizon, controller, and apparatus do not transfer silently.

3.1 Read the ACT objective by component

A simplified view of ACT training combines sequence reconstruction with latent regularization:

$$

\mathcal{L}_{ACT}

=\sum_{j=0}^{K-1}m_{t+j}

\left\|a_{t+j}-\hat a_{t+j|t}\right\|_1

+\beta D_{KL}\!\left(q_\phi(z\mid o_t,A_t)\,\|\,\mathcal{N}(0,I)\right).

$$

The first term aligns the action chunk with an expert sequence. The second regularizes the training-time latent posterior toward a prior. A missing padding mask m can teach repeated or zero-padded values as real actions near an episode boundary. An excessively strong KL term may make the latent irrelevant; a weak term may encourage memorization or unstable samples. Inspect reconstruction, KL, latent sensitivity, and rollout variation together.

A conditional VAE does not guarantee preservation of all modes whenever human demonstrations vary. First determine whether divergent labels come from camera ambiguity, operator style, hidden task phase, calibration drift, or genuinely different valid strategies. Record whether local inference reproduces the deterministic convention and training-latent behavior in the original ACT implementation. Do not expose latent samples directly as hardware action diversity. Evaluate multiple samples for task-phase consistency and feasibility in offline replay or simulation first.

The ACT paper's 80–90% success figures and roughly ten minutes of demonstrations apply to selected fine-grained bimanual tasks and its own apparatus ^[7]. They are not an expected range for this single-arm assembly and must not be ranked against unmatched studies. ALOHA Unleashed extends the dexterous-policy lineage, but this chapter does not import an unmatched result from it ^[10].

3.2 Temporal aggregation is both ensembling and a latency choice

If inference runs repeatedly, several chunks predict the action for real execution time u. Let \hat a_{u|s} denote the prediction made by a chunk starting at s\le u. A generic temporal aggregation is

$$

\bar a_u=

\frac{\sum_{s\in\mathcal{S}_u}w(u-s)\hat a_{u|s}}

{\sum_{s\in\mathcal{S}_u}w(u-s)},

\qquad w(\Delta)=\exp(-\lambda\Delta).

$$

A larger \lambda favors predictions conditioned on newer observations. A smaller value smooths more strongly across old chunks. An older prediction may suppress noise but become stale when contact state changes. Select aggregation weights using action age, phase transitions, force or collision gates, and observation divergence, not visual smoothness alone.

BAKU combines multimodal encoders, a Transformer trunk, detachable action heads, action chunking, and temporal smoothing, making it a useful counterpoint for separating a generative head from temporal design ^[9]. Its aggregate result does not transfer to this cell. Match the encoder, dataset, and compute budget across plain BC, a deterministic chunk head, and ACT wherever possible. Requiring a strong plain or chunked BC before a generative-policy or RL branch is an S12 method-selection rule, not a universal ranking proved by one benchmark ^{[9] [7] [8]}.

4. Choose chunk length with the controller

Treating chunk length only as a model hyperparameter omits the deployment contract. Let controller reference rate be f_c, predicted length be K, and the number executed without a new observation be H_e. Prediction and open-loop spans are K/f_c and H_e/f_c. Add observation age L_o, preprocessing L_p, inference L_i, transport L_n, and queue wait L_q:

$$

L_{decision}=L_o+L_p+L_i+L_n+L_q,

\qquad

age(a_u)=t^{exec}_u-t^{obs}_{condition}.

$$

“Model inference is 20 ms” does not establish freshness. The camera frame may already be 70 ms old, and the chunk may wait another 100 ms. LeRobot's asynchronous-inference documentation separates RobotClient execution from PolicyServer chunk prediction and overlaps them with a queue ^[18]. This is a throughput pattern, not a hard-real-time or safety guarantee. Chunk length, aggregation, control rate, inference delay, and stale-action behavior must be measured together ^{[7] [18]}. A2C2 is a candidate aimed at correcting long chunks with newer information, but the source packet lacks a sufficient inspected primary-body locator, so this chapter uses no performance number from it ^[14].

4.1 Worked timing example

The following values illustrate a local configuration; they are not hardware recommendations.

• A trajectory/controller interface consumes references at 50 Hz, one action every 20 ms.
• The policy requests a new observation window at 10 Hz and predicts K=20 actions, a 400 ms span.
• The client intends to execute only H_e=5 actions, at most 100 ms, before replacement.
• Measured p95 values are 24 ms observation age, 8 ms preprocessing, 54 ms inference, 6 ms transport, and 12 ms queue wait.

The p95 decision age is 104 ms, already longer than the intended 100 ms execution horizon. Inference may run continuously while the assumption “the next chunk always arrives in time” still fails. The team can issue requests earlier, reduce encoder/model tail latency, increase H_e while testing the added open-loop contact risk, or execute a validated hold, retreat, or classical fallback on a miss.

Free-space transport and insertion approach need not share H_e. A longer chunk can follow a collision-checked corridor in free space. Near contact onset, a force event or pose residual may invalidate the old chunk immediately. A phase-dependent horizon needs a versioned phase owner and transition condition.

4.2 Specify the stale-action policy

Attach policy_checkpoint, config_hash, chunk_id, sequence, observation event time and clock domain, inference start/end, intended action times, maximum age, task phase, controller mode, calibration epoch, state fingerprint, aggregation eligibility, replacement rule, and fallback to every chunk.

The client must not execute a chunk merely because it is newest. Reject it if its observation is too old, its sequence is reordered, the phase changed, or current state diverged too far from the conditioning state. Never hold the last action indefinitely on queue underrun. A safe gripper hold, zero velocity, trajectory pause, or retract depends on task phase and controller validation. The learned policy cannot own the independent watchdog or force limit.

5. Treat training through checkpoint promotion as one pipeline

The tutorial's dataset → ACT → optimized-inference sequence is useful implementation orientation ^[13]. Current calls and behavior return to official LeRobot documentation and a pinned version. Processor documentation distinguishes raw hardware values, model transitions, and robot actions ^[15]. Rollout and HIL documentation distinguish autonomous execution, recording, reset, and intervention workflows ^{[16] [17]}. An example command or duration is not a motion-safety envelope.

5.1 Reproducible training workflow

1. Freeze the dataset: record release ID, parent, task card, split manifest, calibration epochs, intervention masks, and schema hash.
2. Freeze processors: test observation/action order, units, train-only statistics, image transforms, and padding/missing masks with a golden batch.
3. Run plain BC: train the smallest deterministic head under several seeds and retain curves, phase-stratified errors, and checkpoint hashes.
4. Ablate chunking: retain encoder and data, add a deterministic K-action head, and record K, H_o, H_e, and aggregation on/off.
5. Run ACT: add the cVAE encoder, KL weight, latent dimension, Transformer, and padding masks. Plot reconstruction, KL, and latent sensitivity separately.
6. Replay sequences offline: replay chunk prediction, overlap, timestamps, and projection from stored observation windows.
7. Run simulation regression: vary initial-state bins, objects/scenes, delay/drop, and contact transitions under the same evaluator.
8. Run shadow mode: record proposals while the validated baseline moves the robot; measure age, rejection, and divergence.
9. Bound hardware: only an approved checkpoint enters a reduced envelope with independent stop authority and rollback.

A checkpoint is more than a weight file. Its immutable manifest includes dataset ID, code commit, dependency lock, processor configuration, action semantics, observation keys, normalization statistics, model config, seed, training curves, offline report, timing profile, evaluator version, and known limitations. A changed camera order or gripper scale creates a different policy artifact even if weights are identical.

5.2 Evaluation matrix

Separate unassisted from intervention-assisted success. Report uncertainty and failure taxonomy by initial-state bin, object instance, scene, operator, and calibration epoch. Rising success alongside more frequent safety projection does not by itself show a better policy; the supervisor may be doing more corrective work. Latency needs p50, p95, maximum, deadline misses, and clock provenance, not only a mean.

5.3 Keep evidence tier beside the result

• Original method papers support the DAgger and ACT mechanisms ^{[2] [7]}, bounded by their apparatus and evaluation.
• Benchmarks and follow-up work broaden alternatives and failure questions: RoboMimic, IBC, BAKU, ALOHA Unleashed, and TRANSIC ^{[5] [6] [9] [10] [11]}. Unmatched headline figures are not imported.
• Official software documentation establishes current processor, rollout, HIL, and async interfaces ^{[15] [16] [17] [18]}. It requires an access date and pinned release and is not application-safety evidence.
• Tutorial synthesis supplies a learning sequence and source map ^[13]. Claims about summarized methods return to originals.
• Local experiment evidence decides timing, intervention, safety events, and promotion for this cell. It is more direct for deployment than an external score but does not generalize automatically.

Source and license: Mandlekar et al., original PMLR 164 publication, Figure 1 “Tasks.” with its original caption. CC BY 4.0. Cropped only the figure and original caption from PDF page 2; no semantic edits.

6. Manufacturing Cell Checkpoint

Review the following artifacts together for the tabletop insertion cell.

Task and data

• Evaluator phases agree on acquire, grasp, transport, pre-contact, insert, release, and retreat.
• Train/validation/test are independent by object, scene, operator, and calibration epoch.
• The target is identified as proposal or sent action, with projection and rejection masks.
• Intervention request, accepted takeover, recovery, correction, and handback are separate authority labels.
• Failures and resets are preserved, with distinct training-admission rules.

Model and timing

• Plain BC, deterministic chunked BC, and ACT use the same dataset release and matched encoder conditions.
• Configuration records H_o, K, H_e, controller rate, request rate, and aggregation weight.
• Observation and end-to-end action age p50/p95/max retain clock-domain provenance.
• Stale, reorder, duplicate, underrun, and phase-transition tests trigger the intended rejection or fallback.
• cVAE KL, reconstruction, latent sensitivity, and seed variance have been reviewed.

Safety and ownership

• The policy owns only a bounded proposal; independent owners retain collision/force gates, watchdog, and E-stop.
• Queue purge and authority handoff pass intervention state-machine tests.
• A duration limit is not treated as a safety envelope.
• A rollback-capable approved checkpoint and classical fallback are available.
• Human workload, takeover latency, and reset labor are reported with success.

Promotion checklist

• [ ] Dataset/processor golden-batch and round-trip tests pass.
• [ ] Required claim, configuration, checkpoint, and evaluation artifacts have versions and hashes.
• [ ] The team reproduces plain-BC failure and names the failure chunking should fix.
• [ ] ACT's gain is separated from encoder, data, chunk-length, and aggregation changes.
• [ ] Delay/drop injection never executes an expired chunk.
• [ ] Simulation and shadow runs do not increase new safety-event classes.
• [ ] Stop authority and abort conditions are approved before a bounded hardware trial.
• [ ] Unassisted success, intervention, projection, deadline misses, and recovery are reported together.

7. A bounded Codex implementation prompt

Goal
- Build a reproducible plain-BC → deterministic chunked-BC → ACT comparison
  for the versioned tabletop-insertion dataset.

Context
- Read the task card, split manifest, action/observation schema, calibration epochs,
  controller contract, and approved simulation evaluator.
- Treat raw, mapped, proposed, and sent actions as different fields.

Constraints
- Do not move hardware, change controller/safety limits, or invent device settings.
- Pin dataset, code, dependencies, processor config, and every checkpoint.
- Keep object/scene/operator/calibration test groups out of training.
- Implement plain BC first; add chunking without changing the encoder/data;
  then add ACT with explicit padding, KL, latent, and aggregation configs.
- Log event/arrival clocks, observation age, inference p50/p95/max, queue state,
  chunk sequence, stale/reorder rejection, projection delta, and authority state.
- Never let the policy own collision/force gates, watchdog, E-stop, or torque authority.

Done when
- Golden-batch processor tests and checkpoint reload tests pass.
- All three baselines have matched offline, sequence-replay, and simulation reports.
- Delay/drop tests prove expired chunks are rejected and fallback is deterministic.
- A shadow-run report separates proposal, projected, sent, and baseline actions.
- The output includes configs, hashes, plots, failure taxonomy, rollback command,
  and a promotion recommendation; no hardware command is executed.

Safety
- Stop at simulation and shadow artifacts. Generate a separate human-reviewed
  hardware trial card with reduced envelope, independent stop authority,
  abort thresholds, and rollback; do not execute it.

This prompt does not ask Codex to run an ACT repository unchanged. It first turns data semantics and timing into tests, then bounds the artifacts introduced by each model change.

8. Remaining failure modes and open questions

There is no universal chunk length. Task phase, contact timescale, controller rate, model latency, and observation age move together. Transport and insertion may need different horizons even within one task.

Temporal aggregation trades responsiveness for smoothness, and an averaged action is not guaranteed feasible. Joint or Cartesian projection and the contact controller remain in the execution path.

Without finding the cause of demonstration multimodality, a generative latent may model noise. Separate operator style, hidden task variables, calibration drift, timestamp error, and genuinely valid alternative strategies. Even work seeking portable, embodiment-spanning collection such as UMI needs new action-alignment and deployment-interface validation ^[12].

Intervention data can teach recovery only inside the boundary of human attention. Late takeover, unseen hazards, fatigue, and recovery-only bias cannot become independent safety evidence.

Checkpoint reproducibility is not physical reproducibility. Reloading the same weights does not preserve camera exposure, network tails, controller firmware, or fixture friction. The deployment manifest must describe the cell state as well as the model.

Relation to Prior Surveys

The small BC exercise in #S11 introduced the observation–action–episode–checkpoint–evaluation path. This chapter does not copy its prose or figures. It extends that entry point on the new Chapter 4 dataset contract with fresh evidence for covariate shift, correction authority, chunk timing, ACT ablations, and deployment gates. A final public chapter link is deferred until #S11 is release-ready.

What to Learn Next

If matched plain or chunked BC and ACT are sufficient, there is no reason to add a more complex model. Move to Chapter 6 only when the same observation admits multiple valid action trajectories and deterministic chunks or the cVAE baseline continue to show mean actions, mode omission, or recovery failures. Diffusion Policy and flow matching are then tested under an explicit hypothesis about a multimodal action distribution, not because they are newer.

This chapter hands Chapter 6 a fixed interface: versioned dataset and splits, observation/action semantics, H_o/K/H_e, controller rate and deadline, stale rejection, constraint projection, checkpoint manifest, and matched evaluator. Pretraining, tokenization, and cross-embodiment comparison for VLA and generalist policies belong to #S13. No future upstream policy may bypass this timing, controller, and safety contract.

References

1. Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A Framework for Learning from Demonstration. Robotics and Autonomous Systems. DOI: 10.1016/j.robot.2008.10.024.
2. Ross, S., Gordon, G., & Bagnell, J. A. (2011). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS.
3. Laskey, M., Lee, J., Fox, R., Dragan, A., & Goldberg, K. (2017). DART: Noise Injection for Robust Imitation Learning. CoRL. arXiv:1703.09327.
4. Kelly, M., Sidrane, C., Driggs-Campbell, K., & Kochenderfer, M. J. (2019). HG-DAgger: Interactive Imitation Learning with Human Experts. ICRA. DOI: 10.1109/ICRA.2019.8793698.
5. Mandlekar, A., et al. (2021). RoboMimic: A Benchmark for Robot Learning from Demonstration. CoRL. arXiv:2108.03298.
6. Florence, P., et al. (2022). Implicit Behavioral Cloning. CoRL. arXiv:2109.00137.
7. Zhao, T. Z., Kumar, V., Levine, S., & Finn, C. (2023). Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. RSS. arXiv:2304.13705.
8. Chi, C., et al. (2023). Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. RSS. arXiv:2303.04137. Terry's English explainer.
9. Haldar, S., Peng, Z., & Pinto, L. (2024). BAKU: An Efficient Transformer for Multi-Task Policy Learning. arXiv:2406.07539.
10. Zhao, T. Z., et al. (2024). ALOHA Unleashed: A Simple Recipe for Robot Dexterity. CoRL. arXiv:2410.13126.
11. Jiang, Y., et al. (2024). TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction. CoRL.
12. Ha, H., et al. (2024). Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots. RSS. arXiv:2402.10329. Terry's English explainer.
13. Capuano, F., Pascal, C., Zouitine, A., Wolf, T., & Aractingi, M. (2025). Robot Learning: A Tutorial. arXiv:2510.12403v1. DOI: 10.48550/arXiv.2510.12403.
14. Sendai, K., Alvarez, M., Matsushima, T., Matsuo, Y., & Iwasawa, Y. (2025). Leave No Observation Behind: Real-time Correction for VLA Action Chunks. arXiv:2509.23224.
15. Hugging Face (2026a). LeRobot Robot and Teleoperator Processors. Official LeRobot documentation, accessed 2026-07-15.
16. Hugging Face (2026b). LeRobot Policy Deployment and Rollout. Official LeRobot documentation, accessed 2026-07-15.
17. Hugging Face (2026c). LeRobot Human-In-the-Loop Data Collection. Official LeRobot documentation, accessed 2026-07-15.
18. Hugging Face (2026d). LeRobot Asynchronous Inference. Official LeRobot documentation, accessed 2026-07-15.
19. Dean A. Pomerleau (1989). ALVINN: An Autonomous Land Vehicle in a Neural Network. NeurIPS 1988 annotated primary reading. canonical proceedings record. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Dataset coverage, action semantics, covariate shift, horizon, and recovery policy limit transfer.
20. Brian Armstrong-Hélouvry et al. (1994). A Survey of Models, Analysis Tools and Compensation Methods for the Control of Machines with Friction90209-7). Annotated primary reading. DOI: 10.1016/0005-1098(94)90209-7. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Secondary source and pre-modern actuators; each model must be revalidated on the actual joint/hand.
21. Carlos Canudas de Wit et al. (1995). A New Model for Control of Systems with Friction. Annotated primary reading. DOI: 10.1109/9.376053. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Parameters can be weakly identifiable under limited excitation and vary with temperature, wear, lubrication, and load.
22. Stefan Schaal (1999). Is Imitation Learning the Route to Humanoid Robots?01327-3). Annotated primary reading. DOI: 10.1016/s1364-6613(99)01327-3. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Dataset coverage, action semantics, covariate shift, horizon, and recovery policy limit transfer.
23. Levine, Sergey (2016). Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection. Annotated primary reading. canonical URL. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Dataset coverage, action semantics, covariate shift, horizon, and recovery policy limit transfer.
24. Lerrel Pinto & Abhinav Gupta (2016). Supersizing Self-Supervision: Learning to Grasp from 50K Tries and 700 Robot Hours. Annotated primary reading. canonical URL. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Dataset coverage, action semantics, covariate shift, horizon, and recovery policy limit transfer.
25. Ebert, Frederik (2021). Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets. Annotated primary reading. canonical URL. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Dataset coverage, action semantics, covariate shift, horizon, and recovery policy limit transfer.
26. Zipeng Fu et al. (2024). Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation. Annotated primary reading. arXiv:2401.02117. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Platform, task, dataset, and evaluation assumptions must be checked in the primary paper before transferring a result.
27. Ulf Kasolowsky and Berthold Bäuml (2024). Fine Manipulation Using a Tactile Skin: Learning in Simulation and Sim-to-Real Transfer. Annotated primary reading. canonical URL. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Simplified tactile representation discards analog richness
28. Miquel Oller et al. (2024). Tactile-Driven Non-Prehensile Object Manipulation via Extrinsic Contact Mode Control. Annotated primary reading. arXiv:2405.18214. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Platform, task, dataset, and evaluation assumptions must be checked in the primary paper before transferring a result.
29. Various (2024). DexH2R: Task-oriented Dexterous Manipulation from Human to Robots. Annotated primary reading. arXiv:2411.04428. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Platform, task, dataset, and evaluation assumptions must be checked in the primary paper before transferring a result.
30. Ying Yuan et al. (2024). Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing. Annotated primary reading. arXiv:2312.01853. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Platform, task, dataset, and evaluation assumptions must be checked in the primary paper before transferring a result.
31. Hao-Shu Fang et al. (2025). DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation. Annotated primary reading. arXiv:2509.04441. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Requires per-hand mechanical co-design
32. Zilin Si et al. (2025). ExoStart: Efficient Learning for Dexterous Manipulation with Sensorized Exoskeleton Demonstrations. Annotated primary reading. arXiv:2506.11775. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: 200 Hz simulation versus roughly 20 Hz real control is a mismatch
33. Di Zhang et al. (2025). KineDex: Learning Tactile-Informed Visuomotor Policies via Kinesthetic Teaching for Dexterous Manipulation. Annotated primary reading. arXiv:2505.01974. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Platform, task, dataset, and evaluation assumptions must be checked in the primary paper before transferring a result.
34. Hugging Face LeRobot Team (2026). LeRobot: An Open-Source Library for End-to-End Robot Learning. Annotated primary reading. arXiv:2602.22818. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Young API evolves quickly and requires version-pinned tutorial commands
35. Hung-Chieh Fang et al. (2026). DexDrummer: In-Hand, Contact-Rich, Long-Horizon Robot Drumming. Annotated primary reading. arXiv:2603.22263. — Role: primary reading for failure conditions in BC, intervention, and action horizons. Limitation: Workshop result with narrow musical task