LlamaIndex RouterQueryEngine

Free

Intelligent query routing for LLM applications

FreeFree tier

Inputs: textOutputs: text

Type

Open Source

Company

LlamaIndex

Visit Website GitHub (38 stars)

About LlamaIndex RouterQueryEngine

LlamaIndex RouterQueryEngine is a component of the LlamaIndex framework that intelligently routes user queries to the most appropriate data source or query engine. It acts as a selector module, taking a user query and a set of 'choices' defined by metadata, and returns one or more selected choices. The router leverages LLMs for decision-making, supporting both LLM selectors (text completion) and Pydantic selectors (function calling with Pydantic schemas). It can be used as a standalone selector, a query engine, or a retriever, and is composable on top of other query engines as tools. The RouterQueryEngine enables use cases such as selecting the right data source among diverse sources, deciding between summarization and semantic search, and multi-routing to combine results from multiple choices. Note: this feature is currently in beta.

Key Features

Takes a user query and a set of choices (metadata-defined) and returns one or more selected choices

Can be used as a standalone selector module or as a query engine/retriever

Supports LLM selectors (text completion) and Pydantic selectors (function calling with schemas)

Composable on top of other query engines as tools via QueryEngineTool

Enables multi-routing to try multiple choices and combine results

Integrates with LlamaIndex framework for data connectors, indexes, engines, agents, and observability

Pros & Cons

Pros

Leverages LLMs for intelligent and flexible routing decisions
Supports both LLM text completion and Pydantic function calling selectors for different use cases
Enables multi-routing for combining results from multiple sources
Composable with any existing query engine as a tool, making it easy to integrate into existing pipelines
Part of the comprehensive LlamaIndex framework, providing data connectors, indexes, and observability

Cons

Relies on LLMs for routing decisions, which can introduce latency and additional costs
Currently a beta feature, so may have instability or changes in future releases
Requires setting up and managing multiple query engine tools upfront
Routing accuracy depends on the quality of tool descriptions and LLM capability

Best For

Selecting the right data source among a diverse range of data sourcesDeciding whether to perform summarization (summary index query engine) or semantic search (vector index query engine)Deciding to try multiple query engines at once and combine results using multi-routingBuilding retrieval-augmented generation (RAG) applications that require dynamic query routingAgentic workflows that route queries to different tools or knowledge bases

FAQ

What is a LlamaIndex RouterQueryEngine?

It is a module that takes a user query and a set of choices (defined by metadata), and uses an LLM selector to return one or more selected choices. It can be used as a standalone selector or as a query engine/retriever.

How do I define a selector for the RouterQueryEngine?

You can define a selector using built-in classes like LLMSingleSelector, LLMMultiSelector, PydanticSingleSelector, or PydanticMultiSelector. Pydantic selectors use function calling with Pydantic schemas, while LLM selectors use text completion endpoints.

How do I use the RouterQueryEngine with multiple tools?

You wrap each query engine in a QueryEngineTool with a description, then pass the list of tools to the RouterQueryEngine along with a selector. The router automatically selects the appropriate tool based on the query.