OpenLM

Free

a drop-in OpenAI-compatible library that can call LLMs from any other hosted inference API. Also [Typescript](https://github.com/r2d4/llm.ts) ![GitHub Repo stars](https://img.shields.io/github/stars/r2d4/openlm?style=social)

FreeFree tier

Inputs: textOutputs: text

Type

Open Source

Visit Website GitHub (40 stars)

About OpenLM

OpenLM is a drop-in OpenAI-compatible Python library that allows developers to call language models from multiple providers—including HuggingFace, Cohere, and OpenAI—using the same interface as OpenAI's Completion API. It supports running multiple prompts across multiple models in a single request and has a very small footprint by calling inference APIs directly rather than through separate SDKs. The library is open-source and currently focuses on the Completion endpoint, with plans to support more standardized endpoints.

Key Features

Drop-in replacement for OpenAI's Completion API – same parameters and response format

Supports multiple model providers: HuggingFace, Cohere, OpenAI, and custom implementations

Run multiple prompts on multiple models in a single API call

Very small footprint – calls inference APIs directly without requiring provider SDKs

Open-source Python library (pip install openlm)

Pros & Cons

Pros

Seamless drop-in compatibility with OpenAI's Completion API
Multi-model and multi-prompt support in one request
Minimal dependencies – no need for multiple provider SDKs
Free and open-source

Cons

Currently only supports the Completion endpoint, not Chat or other endpoints
Requires separate API keys for each provider
Limited documentation and community support compared to larger frameworks

Best For

Switching between LLM providers without changing codeComparing model outputs from different providers with the same promptPrototyping applications that need multi-model inferenceBuilding lightweight agents that leverage multiple LLM backends