Hydra: Elegant Configuration Management for ML and Python Applications

Most machine learning projects start with a single training script and a handful of hyperparameters passed through argparse. A few months later that same script has thirty flags, hardcoded paths scattered across functions, and nobody can reproduce the run that produced last quarter's best model. Hydra, an open-source framework from Meta built on top of OmegaConf, is designed to solve exactly this class of problem. This tutorial walks through Hydra from first principles using a realistic model training workflow as the running example.

What Hydra Is and What It Solves

Hydra is a Python framework for composing and overriding hierarchical configuration. Instead of one giant config file or a wall of command-line flags, you describe your configuration as a tree of small YAML files that Hydra composes at runtime. You can then override any value from the command line, swap entire config groups (for example switch from a ResNet to a Vision Transformer), and run sweeps across combinations of parameters.

The concrete problems it addresses in ML work:

• Scattered argparse definitions. Flags accumulate across files and become hard to discover or document.
• Hardcoded hyperparameters. Learning rates and paths buried in source code make experiments brittle.
• Hard-to-reproduce runs. Without a saved snapshot of the exact configuration, reproducing a result months later is guesswork.
• Combinatorial experiment configs. Trying three models against four datasets with two optimizers is twenty-four runs; managing those by hand is error-prone.

Hydra sits on OmegaConf, which provides the underlying typed configuration object (a DictConfig), variable interpolation, and merging logic. Hydra adds composition, command-line overrides, automatic output directories, and multirun.

Installation

Hydra is published as hydra-core on PyPI. Install it into your project environment:

pip install hydra-core

Optional plugins for sweepers and launchers are separate packages, installed when you need them:

pip install hydra-optuna-sweeper
pip install hydra-joblib-launcher
pip install hydra-submitit-launcher

Verify the installation:

python -c "import hydra; print(hydra.version)"

This tutorial targets Hydra 1.3.x, which is the current stable line.

The Basics: Entry Point, Config File, and cfg

A Hydra application has three pieces: a config directory, a YAML config file, and a decorated entry-point function. Create a project layout like this:

project/
├── train.py
└── conf/
    └── config.yaml

Start with a minimal conf/config.yaml:

epochs: 10
batchsize: 32

optimizer:
  name: adam
  lr: 0.001

import hydra
from omegaconf import DictConfig, OmegaConf


@hydra.main(versionbase=None, configpath="conf", configname="config")
def main(cfg: DictConfig) -> None:
    print(OmegaConf.toyaml(cfg))

    print(f"Training for {cfg.epochs} epochs")
    print(f"Optimizer: {cfg.optimizer.name} @ lr={cfg.optimizer.lr}")


if name == "main":
    main()

A few notes on the decorator. versionbase=None opts into the modern behavior and silences a compatibility warning; set it explicitly so your script behaves consistently across Hydra versions. configpath is relative to the file that defines main, and configname is the YAML filename without its extension.

Run it:

python train.py

The cfg object is an OmegaConf DictConfig. You access values with dot notation (cfg.optimizer.lr) or dictionary syntax (cfg["optimizer"]["lr"]). It behaves like a nested namespace, and OmegaConf preserves the types declared in YAML, so cfg.epochs is an int and cfg.optimizer.lr is a float.

Command-Line Overrides

The single most useful Hydra feature day to day is overriding configuration without touching files. Any leaf value can be set from the command line using dotted paths:

python train.py optimizer.lr=0.01 epochs=50

This produces a config where optimizer.lr is 0.01 and epochs is 50, leaving everything else untouched. Hydra parses the right-hand side into a typed value, so epochs=50 becomes an integer.

By default Hydra only lets you override keys that already exist; this catches typos. To add a key that is not in the config, prefix it with +:

python train.py +optimizer.weightdecay=0.0001

python train.py ~optimizer.lr

conf/
├── config.yaml
├── model/
│   ├── resnet.yaml
│   └── vit.yaml
├── dataset/
│   ├── cifar.yaml
│   └── imagenet.yaml
└── optimizer/
    ├── adam.yaml
    └── sgd.yaml

name: resnet50
numlayers: 50
pretrained: true

And conf/model/vit.yaml:

name: vitbase
patchsize: 16
pretrained: true

A dataset, conf/dataset/cifar.yaml:

name: cifar10
numclasses: 10

imagesize: 32
datadir: /data/cifar10

defaults:
model: resnet
dataset: cifar
optimizer: adam
self


epochs: 10
batchsize: 32

The defaults list is Hydra's composition instruction. Each entry says "from the model group, take resnet". The special self entry controls where this file's own values are merged relative to the groups; placing it last means top-level keys override anything pulled in from groups.

At runtime Hydra composes a single config from these pieces. The selected model lives under cfg.model, the dataset under cfg.dataset, and so on. You override the selection on the command line by naming the group:

python train.py model=vit dataset=imagenet optimizer=sgd

This is the heart of Hydra: each axis of your experiment is an independent, named, version-controlled file, and you compose them however you like at launch.

Variable Interpolation with OmegaConf

OmegaConf supports interpolation, letting one config value reference another so you avoid repeating yourself. Use the ${path.to.value} syntax:

paths:
  root: /workspace
  data: ${paths.root}/data
  checkpoints: ${paths.root}/checkpoints

runname: ${model.name}${dataset.name}

Here paths.data resolves to /workspace/data, and runname is built from the selected model and dataset names — so it automatically reflects whatever you composed.

apikey: ${oc.env:WANDBAPIKEY}
datadir: ${oc.env:DATADIR,/data}

The second form supplies a default (/data) when the variable is unset. Hydra registers a now resolver for timestamps, useful for naming runs:

runid: ${now:%Y-%m-%d%H-%M-%S}

You can register custom resolvers in code when you need computed values:

from omegaconf import OmegaConf

OmegaConf.registernewresolver("mul", lambda a, b: a * b)

Then effectivelr: ${mul:${optimizer.lr},${numgpus}} computes a value at access time.

Automatic Working Directory and Reproducibility

Every time you run a Hydra application, it creates a fresh output directory and changes into it. By default the layout is outputs/YYYY-MM-DD/HH-MM-SS/ for single runs. Inside that directory Hydra writes a .hydra/ folder containing three files:

• config.yaml — the fully composed configuration for this run.
• overrides.yaml — the exact command-line overrides you passed.
• hydra.yaml — Hydra's own resolved configuration.

This snapshot is what makes runs reproducible. Months later you can open .hydra/config.yaml and see the exact configuration that produced a result, or re-run with the recorded overrides. Logs and any files your script writes to the current directory also land here, keeping each run self-contained.

Because Hydra changes the working directory, save artifacts relative to the current directory and they will be organized per run automatically. If you need the original launch directory, use the runtime API:

import os
from hydra.core.hydraconfig import HydraConfig



@hydra.main(versionbase=None, configpath="conf", configname="config")
def main(cfg):
    outdir = HydraConfig.get().runtime.outputdir
    originalcwd = hydra.utils.getoriginalcwd()

    print(f"Writing artifacts to {os.getcwd()}")

hydra:
  run:
    dir: ./outputs/${runname}/${now:%Y-%m-%d%H-%M-%S}

  job:
    name: train
    chdir: true

python train.py -m model=resnet,vit lr=0.1,0.01

defaults:
override hydra/sweeper: optuna

hydra:
  sweeper:
    sampler:
      target: optuna.samplers.TPESampler

    direction: maximize
    ntrials: 30
    params:
      optimizer.lr: interval(1e-5, 1e-1)
      model: choice(resnet, vit)
      batchsize: choice(16, 32, 64)

@hydra.main(versionbase=None, configpath="conf", configname="config")
def main(cfg: DictConfig) -> float:
    valaccuracy = trainmodel(cfg)
    return valaccuracy

python train.py -m

python train.py -m hydra/launcher=joblib model=resnet,vit lr=0.1,0.01,0.001

from dataclasses import dataclass, field
from hydra.core.configstore import ConfigStore


@dataclass
class OptimizerConfig:
    name: str = "adam"
    lr: float = 0.001
    weightdecay: float = 0.0



@dataclass
class ModelConfig:
    name: str = "resnet50"
    numlayers: int = 50
    pretrained: bool = True


@dataclass
class TrainConfig:
    epochs: int = 10
    batchsize: int = 32

    optimizer: OptimizerConfig = field(defaultfactory=OptimizerConfig)
    model: ModelConfig = field(defaultfactory=ModelConfig)



cs = ConfigStore.instance()
cs.store(name="config", node=TrainConfig)

This is one of Hydra's most powerful patterns and worth adopting early. With hydra.utils.instantiate, a config node can describe a Python object to build — its fully qualified class path lives in a target key, and the remaining keys become constructor arguments. Your YAML stops being just data and becomes a recipe for objects.

target: torch.optim.Adam
lr: 0.001
weightdecay: 0.0
betas: [0.9, 0.999]

And a model, conf/model/resnet.yaml:

target: torchvision.models.resnet50
weights: IMAGENET1KV2

import hydra
from hydra.utils import instantiate


@hydra.main(versionbase=None, configpath="conf", configname="config")
def main(cfg):
    model = instantiate(cfg.model)
    optimizer = instantiate(cfg.optimizer, params=model.parameters())
    train(model, optimizer, cfg)

Note that you can pass extra keyword arguments to instantiate at call time (here params), which is how runtime objects like model parameters get injected into a config-described constructor. To swap optimizers you now change a single YAML file or a command-line override — no if/else ladder in your training code. This pushes the choice of model, dataset, and optimizer entirely into configuration, which is exactly what makes the whole experiment sweepable.

By default instantiate is recursive: nested target nodes are built first. Use partial: true in a config node when you want instantiate to return a functools.partial instead of calling the target immediately — useful for schedulers or losses you construct later.

Integrating with a Training Loop and Logging

Hydra configures Python's standard logging module for you. Just acquire a logger and use it; output goes to both the console and a log file in the run directory:

import logging
import hydra
from hydra.utils import instantiate
from omegaconf import DictConfig

log = logging.getLogger(name)


@hydra.main(versionbase=None, configpath="conf", configname="config")

def main(cfg: DictConfig) -> float:
    log.info("Starting run: %s", cfg.get("runname", "unnamed"))

    model = instantiate(cfg.model)
    optimizer = instantiate(cfg.optimizer, params=model.parameters())
    dataset = instantiate(cfg.dataset)

    bestacc = 0.0

    for epoch in range(cfg.epochs):
        trainloss = runepoch(model, optimizer, dataset)

        valacc = evaluate(model, dataset)
        log.info("epoch=%d loss=%.4f valacc=%.4f", epoch, trainloss, valacc)

        bestacc = max(bestacc, valacc)

    log.info("Best validation accuracy: %.4f", bestacc)

    return bestacc


if name == "main":
    main()

Returning bestacc lets the same function double as a sweep objective. You can customize log levels and formatting through the hydra/joblogging config group if the defaults do not fit your needs. Because the working directory is per-run, the log file sits alongside the .hydra/ snapshot, so each experiment carries its config and its logs together.

Using Hydra in Applications, Not Just Scripts

Hydra is not limited to top-level training scripts. For packaged applications, point configpath at a package using the pkg:// notation so configs ship inside your wheel, and expose a console entry point in pyproject.toml. When you need to compose a config programmatically — for example inside a web service or a notebook where the @hydra.main decorator is inconvenient — use the Compose API:

from hydra import initialize, compose

with initialize(versionbase=None, configpath="conf"):

    cfg = compose(configname="config", overrides=["model=vit", "epochs=5"])
    runinference(cfg)

• Set versionbase=None explicitly so behavior is stable across Hydra releases.
• Keep group files small and focused — one model, one dataset, one optimizer per file — so composition stays readable.
• Adopt target instantiation early. It removes branching from your code and makes every component swappable and sweepable.
• Use structured configs for anything long-lived. Schema validation catches mistakes at startup, before expensive training begins.
• Commit your conf/ directory to version control, and archive the per-run .hydra/ snapshot with your experiment results.
• Prefer interpolation over duplication. Derive paths and run names from base values so a single change propagates.
• Document your config groups in a short README so collaborators know which options exist.
• Validate sweep ranges on a tiny subset first to avoid launching hundreds of jobs with a broken config.

Conclusion and Key Takeaways

Hydra turns configuration from an afterthought into a first-class, composable part of your project. By describing each axis of an experiment as a small YAML file, overriding freely from the command line, and snapshotting every run, it removes most of the friction around reproducibility and combinatorial experimentation that plagues ML codebases.

Key takeaways:

• Hydra composes hierarchical config from small files and lets you override any value on the CLI.
• Config groups make model, dataset, and optimizer independently swappable; defaults selects them.
• Multirun plus sweeper plugins (Optuna, Ax) and launchers (joblib, submitit) cover everything from grid search to cluster sweeps.
• The automatic per-run directory and .hydra/ snapshot give you reproducibility for free.
• instantiate with target is the pattern that makes a training script truly configuration-driven.
• Structured configs add type safety, and the Compose API lets you use Hydra inside applications and services.

Start by moving one argparse script to a single config.yaml, then gradually pull model, dataset, and optimizer into config groups. The payoff is a training pipeline where every experiment is a one-line command and every result is reproducible.