The question crosses three literatures: identifying unknown dynamics, expanding a repertoire of controllable behaviors, and choosing practice tasks. This guide connects 19 papers, including eight coauthored by Benjamin Eysenbach. The proposed research structure below is a synthesis, not a theorem attributed to one paper.
This order follows the base-skills question from planning structure to dynamics learning and curriculum selection. SLAP is especially close, but its main formulation already assumes a simulator and sufficient base skills.
Define the accessible experiments. Let Ω₀ be the current skill library and Π(Ω₀) the allowed ways to choose and compose it. If two candidate worlds generate exactly the same observation distributions under every accessible policy, these experiments cannot distinguish the worlds. This is an identifiability limit, even with unlimited repeated data. It is a basic statistical deduction; active-identification theory makes quantitative versions possible under explicit model assumptions. Active identification
Pθ(observations | π) = Pθ′(observations | π) for every π ∈ Π(Ω₀) ⇒ indistinguishable under Ω₀
Separate observation access from action access. If the agent sees only the endpoints and durations of opaque skills, its immediate learning target is an option-level transition model. Observing intermediate states and primitive actions supplies additional constraints. Identifying enough structure to plan skill sequences need not recover friction, stiffness, contact geometry, and every microscopic state variable separately. Skills to Symbols gives the planning perspective.
Separate recombination from new control. With frozen skills, a new high-level plan can improve sequencing and reachability over a longer horizon, but it remains in the policy class generated by those skills. Expanding that class requires an additional channel such as adjustable skill parameters, low-level actions, changing termination, or learning new controllers. SoRB illustrates composition; SLAP learns new low-level shortcut policies.
Define the task family and accuracy. Full dynamics recovery, predictions on reachable states, success probabilities of options, and accuracy sufficient for one task are different targets. Reward-free exploration and task-optimal exploration formalize two different versions of this choice.
Filter by the part of the problem you want to solve. Each entry states its mechanism, relevance, and assumptions.
ICLR 2026 · EYSENBACH COAUTHOR
Y. Isabel Liu; Bowen Li; Benjamin Eysenbach; Tom Silver
Method + simulated robot experiments
Builds an abstract planning graph from existing TAMP options, then learns new low-level policies that shortcut paths through that graph.
Why read it: The closest match to starting with programmed skills and deciding which additional skills could make plans better. Read Sections 3–4 and the limitations before transferring it to a new setting.
Scope: The main formulation assumes known deterministic dynamics, full observability, an abstraction, and base options sufficient for the task distribution. It does not establish which arbitrary new skills are learnable in an unknown world.
JAIR 61
George Konidaris; Leslie Pack Kaelbling; Tomas Lozano-Perez
Representation theory + robot experiments
Starts with high-level actions and derives the sets or distributions needed to evaluate their compositions. Initiation conditions and effects ground an abstract planning model.
Why read it: Use this to specify what an agent must learn about its existing skills before predicting which compositions can succeed.
Scope: Planning sufficiency is relative to the supplied action repertoire and model assumptions. It is not full identification of every physical parameter.
ICLR 2020
Archit Sharma; Shixiang Gu; Sergey Levine; Vikash Kumar; Karol Hausman
Method + continuous-control experiments
Learns diverse, predictable skills together with a skill-conditioned dynamics model, then plans in skill space for downstream goals.
Why read it: Use as the bridge between a skill library and a predictive model. Ask whether an added skill opens outcomes that existing skill compositions cannot reliably produce.
Scope: Predictability concerns transitions induced by the learned skills. Accurate predictions on this distribution do not identify all primitive-action dynamics or guarantee novel-task success.
ICML 2020
Ramanan Sekar; Oleh Rybkin; Kostas Daniilidis; Pieter Abbeel; Danijar Hafner; Deepak Pathak
Method + visual-control experiments
Learns a world model and plans exploratory behavior using ensemble disagreement about future predictions; later adapts to downstream rewards.
Why read it: Use for deciding where to collect informative dynamics data, including multi-step preparation before an informative interaction.
Scope: Disagreement is a practical uncertainty proxy. It can be miscalibrated and depends on model class, training data, and whether useful experiments are reachable.
ICML 2021
Andrew J. Wagenmaker; Max Simchowitz; Kevin Jamieson
Finite-time theory + experiments
Characterizes task-dependent exploration difficulty and designs experiments that collect the information required for the target decision problem.
Why read it: The strongest conceptual starting point for asking how much dynamics accuracy is enough for a particular skill, rather than trying to reconstruct the whole world.
Scope: The guarantees concern a specified class of linear dynamical systems and decision problems. They do not transfer unchanged to arbitrary nonlinear, partially observed robots.
CoRL 2019 · proceedings 2020
Rémy Portelas; Cédric Colas; Katja Hofmann; Pierre-Yves Oudeyer
Method + curriculum experiments
ALP-GMM samples parameterized environments according to absolute learning progress, adapting practice to the current learner and revisiting regressions.
Why read it: Use for choosing which skill variants to practice next when easy, useful, and unlearnable regions are initially unknown.
Scope: Requires a task/environment parameterization and measurable competence. Progress can be noisy; a learning-progress signal alone does not certify transfer to the final task.
COLT 2020
Andrew Wagenmaker; Kevin Jamieson
Identification rates + lower bounds
Adaptively chooses control inputs for parameter estimation and establishes finite-time estimation bounds with asymptotically matching rates up to constants.
Why read it: Use to formalize persistent excitation, estimation error, and the difference between purposeful experiments and merely injecting random action noise.
Scope: Assumes a linear-system setting and control of the excitation inputs. A frozen skill library restricts this experiment set, so the paper’s input-design guarantee is not automatic.
ICML 2020
Chi Jin; Akshay Krishnamurthy; Max Simchowitz; Tiancheng Yu
Sample-complexity theory
Explores a finite-horizon tabular MDP before rewards are specified, then supports near-optimal planning for subsequently supplied reward functions.
Why read it: Use as a rigorous version of learning dynamics once for many later tasks. Separate this requirement from learning just enough for one chosen task.
Scope: The guarantee depends on state/action/horizon size, accuracy, and the stated access model. Rarely reachable states and unrepresented actions remain important; a fixed skill library changes the action model.
NeurIPS 2016
Rein Houthooft; Xi Chen; Yan Duan; John Schulman; Filip De Turck; Pieter Abbeel
Approximate Bayesian method + experiments
Rewards information gain about uncertain dynamics parameters, approximating Bayesian posterior updates with a variational neural dynamics model.
Why read it: Use to distinguish curiosity about unknown mechanics from empowerment, which measures how much actions can influence outcomes.
Scope: The usable bonus is approximate and model-dependent. Prediction error by itself is not information gain; irreducible randomness need not teach anything.
ICML 2021
Akhil Bagaria; Jason K. Senthil; George Konidaris
Method + continuous-control experiments
Alternates discovering skills and planning with a graph whose nodes are subgoals and edges are skill policies, expanding the explored state space.
Why read it: Use for the iterative loop: deploy known skills to reach a frontier, learn a connecting skill, add it to the library, and explore farther.
Scope: An acquired graph covers the discovered region; it does not enumerate all physically possible skills or prove complete exploration in a general continuous environment.
JMLR 23(152)
Sébastien Forestier; Rémy Portelas; Yoan Mollard; Pierre-Yves Oudeyer
Framework + robot experiments
Formalizes agents that generate and select goals, reuse outcomes across goals, and organize practice through intrinsic signals. Modular goals can create stepping stones toward more complex behavior.
Why read it: Use for a broader curriculum architecture when the agent must choose both what to practice and how to exploit incidental outcomes.
Scope: Results depend on how goals, policy search, and competence are represented. Self-generated goals do not make every target reachable or learnable.
ICLR 2025 · EYSENBACH COAUTHOR
Grace Liu; Michael Tang; Benjamin Eysenbach
Empirical mechanism discovery
Conditions exploration on one target observation while learning contrastive goal-conditioned values. Intermediate behaviors emerge before the final goal is first achieved.
Why read it: An especially relevant Eysenbach baseline: test whether explicit subtask curricula outperform the implicit exploration already produced by single-goal contrastive RL.
Scope: The reported emergence is empirical, not a guarantee that arbitrary prerequisite skills or impossible goals will be discovered. The agent still receives a target observation.
ICLR 2026 · EYSENBACH COAUTHOR
Mahsa Bastankhah; Grace Liu; Dilip Arumugam; Thomas L. Griffiths; Benjamin Eysenbach
Objective analysis + controlled experiments
Analyzes how contrastive representations induce implicit rewards that favor exploration before goal discovery and exploitation afterward; low-rank representation structure is central to the investigated mechanism.
Why read it: Read immediately after SGCRL to ask which representation updates make a new task look learnable, and when an explicit teacher adds value.
Scope: The explanation is about this algorithm and its analyzed settings. It is not a universal theorem that every goal-conditioned learner constructs a useful curriculum.
ICLR 2026 · EYSENBACH COAUTHOR
Faisal Mohamed; Catherine Ji; Benjamin Eysenbach; Glen Berseth
Method + exploration experiments
Uses temporal contrastive representations to guide exploration toward states with unpredictable future outcomes, seeking complex behavior without extrinsic rewards.
Why read it: A recent Eysenbach connection between learning temporal structure and producing useful exploration, complementing explicit world-model disagreement.
Scope: Temporal unpredictability is an exploration signal, not a calibrated estimate of all unknown physical parameters. Coverage and downstream utility require separate evaluation.
ICLR 2026 · EYSENBACH COAUTHOR
Patrik Reizinger; Bálint Mucsányi; Siyuan Guo; Benjamin Eysenbach; Bernhard Schölkopf; Wieland Brendel
Conditional identifiability theory + experiments
Connects diverse skill-conditioned policies and contrastive successor features to recovery of underlying features up to a linear transformation.
Why read it: Use when asking whether the existing skill set is varied enough to reveal latent factors. Read the assumptions before treating representation recovery as a general world-model guarantee.
Scope: The theorem imposes specific data-generating distributions, sufficient skill variability, an injective observation map, critic structure, and ideal optimization. Identifying features is not identifying the entire transition law.
May 2026 preprint · EYSENBACH COAUTHOR
Alireza Modirshanechi; Benjamin Eysenbach; Peter Dayan; Eric Schulz
Theoretical correspondence · preprint
Separates three goal-reaching formulations and relates matching mutual-information skill objectives to formulation-specific downstream goal sensitivity.
Why read it: Use to choose a skill-learning objective after deciding whether the desired task means reaching, occupying, or otherwise controlling a goal-related outcome.
Scope: The formulations can prefer different policies. A correspondence or bound for one goal criterion is not a universal success guarantee for every downstream task.
NeurIPS 2019 · EYSENBACH COAUTHOR
Benjamin Eysenbach; Ruslan Salakhutdinov; Sergey Levine
Planning method + experiments
Uses a learned goal-conditioned value function to connect states in a replay-buffer graph and plan through intermediate waypoints.
Why read it: Use to test whether a supposedly new task is already solvable by composing available local behaviors, before spending trials on new skill learning.
Scope: Stitching depends on reliable local reachability estimates and useful replay coverage. Planning alone does not create a physically missing primitive capability.
NeurIPS 2022 · EYSENBACH COAUTHOR
Benjamin Eysenbach; Tianjun Zhang; Ruslan Salakhutdinov; Sergey Levine
Value/representation connection + experiments
Applies contrastive learning to action-labeled trajectories to obtain representations related to goal-conditioned values, then uses the critic to improve goal-reaching behavior.
Why read it: The technical foundation for the SGCRL line. Useful when a goal-reaching model can replace a more expensive full observation-prediction model.
Scope: A critic’s probabilistic interpretation depends on its sampling distribution and ideal fitting assumptions. It does not by itself certify exploration coverage or uncertainty.
NeurIPS 2017
André Barreto; Will Dabney; Rémi Munos; Jonathan J. Hunt; Tom Schaul; Hado van Hasselt; David Silver
Transfer bounds + experiments
Decouples expected future features from reward weights and uses generalized policy improvement to reuse a set of policies across tasks.
Why read it: Use to distinguish reusing existing skills under a new objective from acquiring new dynamics knowledge or a new motor ability.
Scope: The principal transfer setting keeps dynamics fixed and assumes an appropriate reward-feature representation. It is not a theorem for arbitrary changing mechanics or an unlimited task family.
Proposed synthesis, not an established guarantee. Maintain a library containing each skill’s initiation region, parameters, termination rule, success estimate, duration, and predicted effects. Alongside it, maintain a dynamics model with uncertainty estimates that can be checked on held-out trials.
Current skills→Reachable experiments→Dynamics evidence→New skill candidates→Practice and validate
Generate candidates from graph bottlenecks, useful shortcuts, model disagreements, and prerequisites for desired tasks. Rank them using estimated information gain, expected competence improvement, and the new tasks they could unlock, while accounting for execution and recovery cost. These estimates have different units; a weighted score needs calibration and ablations, not just a convincing name.
Evaluate each acquired skill on fresh starts and perturbations. Keep separate measurements for better dynamics predictions, a larger reliable skill initiation region, successful novel compositions, and final-task performance. A policy trace that talks about mechanics is weaker evidence than predictions made before acting and checked afterward.
This is an original experiment sketch for the earlier shoelace setting, not a result reported by the listed papers. It assumes the agent can parameterize or modify low-level controllers; a frozen API of opaque skills may not support these new behaviors.
No single successful pull identifies all rope mechanics. Observation quality, parameter confounding, accessible forces, and the simulator’s state interface determine which hypotheses these experiments can actually separate.