FastChat
FreeA distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.
FreeFree tier
Inputs: textOutputs: text
About FastChat
FastChat is an open-source platform by LMSYS for training, serving, and evaluating large language model (LLM) based chatbots. It powers the Chatbot Arena on lmarena.ai, which has served over 10 million chat requests for 70+ LLMs and collected over 1.5 million human votes. The platform includes training and evaluation code for state-of-the-art models such as Vicuna, MT-Bench, and LongChat, a distributed multi-model serving system with a web UI and OpenAI-compatible RESTful APIs, and tools for fine-tuning and benchmarking. It is available under a permissive license and can be installed via pip or from source.
Key Features
Distributed multi-model serving system with web UI and OpenAI-compatible RESTful APIs
Training and evaluation code for state-of-the-art LLMs (e.g., Vicuna, MT-Bench, LongChat)
Powers Chatbot Arena with over 10 million chat requests and 70+ LLMs supported
Supports fine-tuning of models
Command-line interface and web GUI for inference
Comprehensive evaluation tools including MT-bench and Chatbot Arena battles
Pros & Cons
Pros
- Fully open-source and free to use
- Supports a wide range of popular LLMs out-of-the-box
- Scalable distributed serving architecture
- Includes training code for reproducing state-of-the-art models like Vicuna
- Active development and community support (Discord, X)
- Provides both web UI and API for easy integration
Cons
- Requires self-hosting and technical setup (not a managed service)
- Dependency on Hugging Face model weights and large GPU resources for serving
- Documentation can be sparse for advanced configuration
- Limited to text-based models; no native support for multimodal inputs
Best For
Serving multiple LLMs in production with a unified APITraining and fine-tuning custom chatbot modelsEvaluating chatbot performance using standardized benchmarks like MT-benchRunning side-by-side LLM comparisons (Chatbot Arena style)Researching long-context models (e.g., LongChat with 32K context)Building distributed chatbot systems with OpenAI-compatible endpoints
FAQ
How do I install FastChat?
FastChat can be installed via pip with `pip3 install fschat[model_worker,webui]` or from source by cloning the repository and running `pip3 install -e .[model_worker,webui]` in the FastChat directory.
What models are supported in FastChat?
FastChat supports a variety of LLMs including Vicuna, MT-Bench, LongChat, and many others available on Hugging Face. It is designed to work with any Hugging Face compatible model.
Is FastChat free?
Yes, FastChat is completely free and open-source. You can download, modify, and deploy it without any licensing fees. It is released under the Apache 2.0 license.
Does FastChat provide an API for integration?
Yes, FastChat includes OpenAI-compatible RESTful APIs, making it easy to integrate with existing tools and applications that use the OpenAI API format.
How can I evaluate chatbots using FastChat?
FastChat provides evaluation tools such as MT-bench (a multi-turn benchmark) and supports running Chatbot Arena battles to collect human preferences. The code for these evaluations is included in the repository.