ReLLM
FreeRegular Expressions for Language Model Completions. 
FreeFree tier
Inputs: textOutputs: text
About ReLLM
ReLLM is an open-source Python library that filters language model token completions using regular expressions to enforce exact structure. It works by masking logits for tokens that do not match a predefined regex pattern, ensuring outputs conform to syntactic structures like JSON or XML, semantic structures like dates or numbers, or complete templates. Compatible with any Hugging Face transformer model, it improves output quality and parsability even for small models.
Key Features
Filters non-matching tokens pre-generation by masking logits against a regex pattern
Supports syntactic structures (JSON, XML) and semantic structures (dates, numbers, templates)
Works with any Hugging Face AutoModelForCausalLM or compatible transformer model
Lightweight Python library with simple `complete_re` function
Can improve output quality even for small models like GPT-2 (124M parameters)
Pros & Cons
Pros
- Ensures exact syntactic and semantic structure in LLM outputs
- Works with small models, improving their output quality
- Easy to integrate with existing Hugging Face pipelines
- Outputs are easily parseable programmatically
- Open-source and free to use
Cons
- Requires knowledge of regular expressions to define patterns
- May still produce incorrect completions if pattern is too broad (e.g., date example gave '00/00/0045')
- Not a hosted service; requires local setup and model loading
- Limited to patterns expressible in regular expressions (not full context-free grammars without additional tooling)
Best For
Generating structured JSON arrays from LLM completionsEnforcing date format patterns (e.g., mm/dd/yyyy)Filling sentence templates with constrained wordsGetting yes/no or multiple-choice answers in a specific formatParsing context-free grammars like JSON (via companion project parserllm)
FAQ
How does ReLLM work?
For each token, ReLLM tests every possible completion against a partial regex. It masks the logits of tokens that do not match the pattern, preventing the language model from generating them.
What kind of structures can ReLLM enforce?
ReLLM can enforce specific syntactic structures like JSON or XML, semantic structures like dates or numbers, or complete templates such as sentences with blanks to fill in.
Does ReLLM work with any language model?
ReLLM works with any Hugging Face transformer model that uses the AutoModelForCausalLM interface, such as GPT-2. It can be adapted to other models with compatible tokenization.
Is ReLLM a hosted service?
No, ReLLM is an open-source Python library. However, a hosted version is available as the Thiggle Regex Completion API at github.com/thiggle/api.
Does ReLLM improve output quality?
Yes, preliminary results show that constraining the token space with ReLLM can improve the quality of completions, even for small models.