Axolotl Skill: YAML LLM Fine-Tuning with LoRA, DPO, GRPO

Axolotl is a YAML-driven fine-tuning framework for large language models, supporting LoRA, QLoRA, and preference optimization methods like DPO, KTO, ORPO, and GRPO. This skill, installed on demand into Hermes Agent, turns the official Axolotl documentation into an interactive reference: you can ask it about Axolotl features, debug configuration errors, or get code patterns without leaving your terminal. Reach for it when your training job involves Axolotl YAML files, multimodal models, or distributed setups with DeepSpeed or FSDP.

What it does

The skill packages the official Axolotl documentation into a set of reference files and quick-reference patterns. It knows the common configuration blocks, the Python API for trainers and prompters, and the gotchas around sequence parallelism and compressed checkpoints. Instead of grepping through scattered docs, you ask the agent a question and it pulls the relevant pattern, explains what each key does, and shows you a working example. The skill also points to reference files for deeper dives: API documentation, dataset formats, and miscellaneous guides.

Before you start

This is an optional skill, so you install it on demand. It runs on Linux and macOS only; Windows is not supported. The skill itself is just documentation and code patterns, but to use what it describes you need a working Axolotl environment: Python, PyTorch, and ideally a GPU cluster with NCCL for multi-node training. The skill assumes you already have Axolotl installed and can run its CLI. You do not need to place custom integrations in a specific folder; any location works as long as the integration is installed as a package in your Python environment.

Working with the skill

Common patterns

The quick reference section holds the most frequently used configuration snippets. Each pattern addresses a specific concern, from network validation to checkpoint compression.

Pattern 1: Validate data transfer speeds with NCCL tests. Before launching a long training job, confirm that inter-GPU communication is fast enough. The NCCL all-reduce performance test pinpoints bottlenecks:

./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3

This runs a bandwidth test across three GPUs, from 8 bytes up to 128 megabytes. If the reported bandwidth is far below what your hardware should deliver, investigate your network fabric or driver settings before wasting GPU hours.

Pattern 2: Enable FSDP in your Axolotl YAML. Fully Sharded Data Parallelism reduces memory footprint by sharding model parameters, gradients, and optimizer states across GPUs. Add this block to your config:

fsdp_version: 2
fsdp_config:
  offload_params: true
  state_dict_type: FULL_STATE_DICT
  auto_wrap_policy: TRANSFORMER_BASED_WRAP
  transformer_layer_cls_to_wrap: LlamaDecoderLayer
  reshard_after_forward: true

offload_params moves parameters to CPU when not in use, which trades speed for memory. transformer_layer_cls_to_wrap tells FSDP which module type to wrap; change it to match your model, for example GPTNeoXLayer for GPT-NeoX.

Pattern 3: Context parallel size must divide the GPU count. When you use context parallelism, the sequence is split across a group of GPUs. The context_parallel_size value has to be a divisor of the total number of GPUs, otherwise the split will not be even and the job will fail.

context_parallel_size

This is a YAML key, not a value. Set it to a number like 2, 4, or 8 depending on your cluster.

Pattern 4: Understand how context parallelism changes your batch size. With 8 GPUs and no sequence parallelism, each step processes 8 different batches. If you set context_parallel_size=4, each batch is split across 4 GPUs, so you only process 2 different batches per step. If your per-GPU micro_batch_size is 2, the global batch size drops from 16 to 4. This is a common source of confusion when tuning throughput.

context_parallel_size=4

Pattern 5: Compress your checkpoints. Setting save_compressed: true saves models in a compressed format that uses about 40% less disk space. The compressed checkpoints remain compatible with vLLM for fast inference and with llmcompressor for further optimization like quantization.

save_compressed: true

Pattern 6: Custom integrations can live anywhere. You do not have to put your integration in the integrations folder. As long as it is installed as a package in your Python environment, Axolotl will find it. The official example repository is at https://github.com/axolotl-ai-cloud/diff-transformer.

integrations

Pattern 7: Handle variable-length sequences. The drop_long_seq utility trims or drops samples that exceed a maximum length. It handles both single examples and batched data: a single example has input_ids as a list of integers, while batched data has a list of lists. Use it to keep your training data within the model's context window.

utils.trainer.drop_long_seq(sample, sequence_len=2048, min_sequence_len=2)

Example code patterns

The skill also includes Python API examples for programmatic control. These are useful when you want to integrate Axolotl into a larger script or custom trainer.

Example 1: Initialize a Modal cloud instance.

cli.cloud.modal_.ModalCloud(config, app=None)

Example 2: Run a command on Modal.

cli.cloud.modal_.run_cmd(cmd, run_folder, volumes=None)

Example 3: Instantiate the base trainer.

core.trainers.base.AxolotlTrainer(
    *_args,
    bench_data_collator=None,
    eval_data_collator=None,
    dataset_tags=None,
    **kwargs,
)

Example 4: Log training metrics.

core.trainers.base.AxolotlTrainer.log(logs, start_time=None)

Example 5: Use a raw input-output prompter.

prompt_strategies.input_output.RawInputOutputPrompter()

Reference files

The skill ships with a references/ directory containing three files:

• api.md - API documentation
• dataset-formats.md - dataset format documentation
• other.md - miscellaneous documentation

Use the view command to read a specific file when you need details beyond the quick reference. For example, if you are unsure how to format a preference dataset for DPO, open dataset-formats.md.

When not to use it

This skill is specific to Axolotl. If you are fine-tuning with a different framework, such as Hugging Face Transformers directly or Unsloth, this skill will not help. It also assumes you are comfortable editing YAML and running training jobs; if you need a fully managed fine-tuning service, look elsewhere.

Limits and gotchas

• The skill is generated from official documentation, so it may lag behind the latest Axolotl release. Check the upstream repo for changes.
• The quick reference patterns are extracted from common usage, not exhaustive. For edge cases, consult the reference files or the official docs.
• Context parallelism changes your effective batch size, as shown in Pattern 4. Recalculate your learning rate and scheduler accordingly.
• Compressed checkpoints save space but require vLLM or llmcompressor for downstream use; make sure your inference stack supports them.
• The skill does not include helper scripts or templates; the scripts/ and assets/ directories are placeholders for you to add your own.

Related resources

This skill pairs well with other MLOps skills in the Hermes Agent optional skills collection, such as those for data preparation or evaluation. For a full list, browse the optional-skills/mlops directory. To update the skill when Axolotl changes, re-run the scraper with the same configuration and the skill will be rebuilt with the latest documentation.