Dromedary

Paid

IBM Dromedary: An open-source, self-aligned LLM for helpful, ethical, and reliable AI.

#open-source#large language model#self-aligned#LLaMA#LLaMA-2#SELF-ALIGN methods#SFT#RLAIF#SALMON#minimal human supervision#Hugging Face#QLoRA weights#synthetic datasets#multi-GPU model-parallel deployment#NeurIPS 2023

Inputs: textOutputs: text

Type

Saas

Company

IBM Research

Visit Website

About Dromedary

Dromedary is an open-source self-aligned large language model developed by IBM Research, designed to be helpful, ethical, and reliable with minimal human supervision. The project introduces a principle-driven self-alignment pipeline (SELF-ALIGN) using SFT and RLAIF (via SALMON) on LLaMA and LLaMA-2 base models. Dromedary-2 simplifies the process to two stages: diverse user prompts (ShareGPT, Dolly-15k, OpenAssistant, OpenOrca) with an improved one-shot exemplar from FastChat promoting a general-specific-general response style, and a RLAIF stage using the SALMON reward model pipeline. The model is accompanied by a NeurIPS 2023 Spotlight paper, released delta/QLoRA weights on Hugging Face, synthetic datasets, and full training/inference pipelines including model-parallel multi-GPU support.

Key Features

Open-source self-aligned LLM from IBM’s Self-Align team

Principle-driven alignment with minimal human supervision

SELF-ALIGN pipeline with Topic-Guided Red-Teaming Self-Instruct

Dromedary-2 SFT: simplified two-stage process with FastChat exemplar

Dromedary-2 RLAIF: SALMON reward-model pipeline on LLaMA-2-70B

Delta weights (LoRA/QLoRA) compatible with LLaMA and LLaMA-2

Synthetic datasets released on Hugging Face (65B and 70B SFT)

Custom llama_dromedary package for training and inference

Model-parallel, multi-GPU inference faster than HF pipeline-parallel

End-to-end training and inference guides with chatbot demo

Pros & Cons

Pros

Open-source with complete training and inference pipelines
NeurIPS 2023 Spotlight paper validates the self-alignment approach
Minimal human supervision required for alignment
Releases delta weights (LoRA/QLoRA) and synthetic datasets for reproducibility
Supports efficient multi-GPU model-parallel inference
Dromedary-2 simplifies and improves alignment pipeline

Cons

Requires base LLaMA/LLaMA-2 weights from Meta, which are not openly distributed
Substantial GPU resources needed for 70B parameter model inference and training
Setup complexity: merging delta weights and managing dependencies
Primarily research-focused; not a polished end-user product
Only supports LLaMA-based architectures

Best For

ML researchers: Reproduce and study principle-driven self-alignment (SFT and RLAIF/SALMON) on LLaMA/LLaMA-2 bases.RL/RLAIF practitioners: Experiment with SALMON reward modeling and reinforcement learning fine-tuning workflows.Data engineers: Generate and curate synthetic instruction–response datasets for alignment and SFT.Enterprise AI teams: Deploy multi-GPU, model-parallel inference for chatbots and internal assistants.AI safety researchers: Assess principle-driven alignment, red-teaming prompts, and ethical behavior outcomes.Educators: Teach modern alignment pipelines (Self-Instruct, LoRA/QLoRA, PEFT) with reproducible code.Open-source contributors: Extend training scripts, inference modules, or datasets to new domains or tasks.Benchmarking engineers: Evaluate helpfulness, ethics, and reliability across public benchmarks and custom suites.Startups and prototypers: Rapidly fine-tune via LoRA/QLoRA on domain data for product MVPs on LLaMA bases.MLOps teams: Integrate scalable inference across 1, 2, 4, 8, or 16 GPUs with deployment guides.