Dataset aggregation

Train on states the learner encounters#

The DAgger algorithm alternates between executing a policy, obtaining the expert's preferred actions for visited states, and training on the combined dataset. Its name abbreviates Dataset Aggregation.

The original formulation can mix expert and learner behavior during data collection. The expert supplies action labels even when the learner has caused the system to enter a state outside the original demonstrations.

Correct the distribution mismatch#

Basic behavior cloning learns from states produced by expert behavior. Its own errors can later move it into different states. DAgger explicitly gathers training examples from the distributions induced during its iterative learning process.

For a reaching robot, this could mean asking for a corrective action after the learned controller approaches an object from an awkward pose. Adding another perfectly executed demonstration might never show that pose.

Expert access is a real requirement#

DAgger is more specific than simply combining robot datasets. Its defining feature is the loop between learner execution, expert labeling, aggregation, and retraining.

The original analysis gives performance guarantees under stated learning and reduction assumptions. It does not guarantee safe exploration on physical hardware. Deploying the collection loop requires an expert who can label encountered states and a suitable way to manage the learner's actions during those trials.