**Guidance**
Free
FreeFree tier
Inputs: textOutputs: text
About **Guidance**
Guidance is an efficient programming paradigm for steering large language models. It provides a Pythonic interface that allows developers to control output structure, interleave control flow (conditionals, loops, tool use) with generation, and constrain outputs using regular expressions or context-free grammars. This approach reduces latency and cost compared to conventional prompting or fine-tuning. Guidance supports multiple backends including Transformers (Hugging Face), llama.cpp, and OpenAI, and integrates seamlessly with Jupyter notebooks for an enhanced user experience.
Key Features
Pythonic interface for working with large language models
Constrained generation using regex, context-free grammars, and select()
Interleaving of control flow (conditionals, loops, tool use) and text generation
Support for multiple model backends: Transformers, llama.cpp, OpenAI, and others
Integration with Jupyter notebooks via an interactive widget
Ability to capture generated text for downstream use
Easy installation via PyPI (pip install guidance)
Reduced latency and cost compared to conventional prompting or fine-tuning
Pros & Cons
Pros
- Significantly reduces latency and cost versus conventional prompting or fine-tuning
- Guarantees output syntax with powerful constrained generation (regex, CFGs)
- Seamlessly integrates control flow and generation in a single Python script
- Familiar Pythonic interface lowers the barrier for developers
- Supports a variety of popular model backends out of the box
Cons
- Requires a Python environment and manual setup of model backends
- Learning curve for advanced features like context-free grammars and regular expressions
- Jupyter notebook widget may have limited support outside of notebooks
Best For
Ensuring structured output from LLMs (e.g., valid JSON, specific formats)Building conversational agents with controlled response generationReducing API costs by guiding generation more efficientlyImplementing tool use and reasoning chains with step-by-step generationCreating prompts that adhere to exact syntax or grammar constraints
FAQ
How do I install Guidance?
You can install Guidance via PyPI by running: pip install guidance.
What model backends does Guidance support?
Guidance supports Transformers (Hugging Face), llama.cpp, OpenAI, and other backends. You can choose the backend appropriate for your model.
Can I constrain the output of a generation to match a specific pattern?
Yes, Guidance allows you to constrain generation using regular expressions, context-free grammars, or a select() function for predefined lists.
Does Guidance work in Jupyter notebooks?
Yes, Guidance provides a widget that enhances the user experience in Jupyter notebooks, displaying prompts and model responses in a rich interface.
Is Guidance free to use?
Yes, Guidance is open-source and free to use. You only pay for the underlying model API costs if using a commercial backend like OpenAI.