Neura Market
LMQL logo

LMQL

Free

LMQL is a query language for large language models.

FreeFree tier
Inputs: textOutputs: text
Type
Open Source
Visit Website GitHub (40 stars)

About LMQL

LMQL is a programming language specifically designed for interacting with large language models (LLMs). It enables robust and modular prompting by incorporating types, templates, constraints, and an optimizing runtime. With LMQL, users can write expressive, multi-part prompts using Python control flow and string interpolation, enforce hard constraints on model output (such as length, stop tokens, and data types), and easily switch between multiple backends like llama.cpp, OpenAI, and Hugging Face Transformers. The language also supports advanced features like nested queries for modular instructions, tool augmentation, chatbot development, and meta prompting. LMQL is developed by the SRI Lab at ETH Zurich and is available as open source.

Key Features

Constrained LLMs with types, regex, and hard constraints via 'where' clauses
Multi-part prompts using Python control flow and string interpolation
Nested queries for modularized local instructions and re-use of prompt components
Automatic backend portability across llama.cpp, OpenAI, and Hugging Face Transformers
Support for tool augmentation, chatbots, meta prompting, and distribution measurement

Pros & Cons

Pros
  • Robust and modular prompting with type safety and constraints
  • Guaranteed output format through hard constraints
  • Deep Python integration for expressive programmatic prompts
  • Portable across multiple LLM backends with a single line of code change
  • Open source and free to use
Cons
  • Requires learning a new programming language/syntax specific to LMQL
  • May have a steeper learning curve compared to simpler prompting frameworks

Best For

Building complex multi-step LLM workflows with structured outputsDeveloping chatbots with constrained and reliable responsesCreating tool-augmented LLM applicationsPortable LLM code that runs across different backends without modificationInteractive prompting with precise control over output format

FAQ

What is LMQL?
LMQL is a programming language for LLM interaction that uses types, templates, constraints, and an optimizing runtime to make prompting robust and modular.
What backends does LMQL support?
LMQL supports llama.cpp, OpenAI, and Hugging Face Transformers. You can switch between them with a single line of code.
How does LMQL enforce constraints on model output?
Constraints are enforced using 'where' clauses that can limit output length, require specific stop tokens, or restrict generation to particular types such as integers or regex patterns.
Can LMQL be used for chatbots?
Yes, LMQL includes support for chatbot applications, as indicated by the chatbot feature in its documentation.