vllm

Free

A high-throughput and memory-efficient inference and serving engine for LLMs

Model APIsFreeFree tier

Type

Open Source

Visit Website GitHub (78.9k stars)

About vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Alternatives to vllm

AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.