What is actually sent to the LLM when invoke is called on messages?

When you call invoke on a list of messages that includes an AIMessage with tool_calls, the tool_calls field is sent to the LLM on every subsequent call. This includes the full arguments of each tool call, such as file contents in a write_file tool. However, if the AIMessage also has a tool_use block in its content list, that block is not sent again because LangChain deduplicates it. The result is that large tool call arguments can balloon token usage on every call unless you manually trim or summarize them.

The Full Answer

What gets sent to the LLM

When you invoke an LLM with a list of messages, the tool_calls field of any AIMessage is included in the prompt sent to the model. This is confirmed by both the question and the accepted answer on Stack Overflow. The user who asked the question tested this by comparing the input_tokens count before and after adding messages with tool calls versus without them. The token count jumped when tool calls were present, proving that the LLM receives that data.

An AIMessage object in LangChain has two key fields:

• content: a string or list of content blocks (e.g., text, tool_use blocks)
• tool_calls: a list of tool call objects, each containing an id, type, and args (the arguments passed to the tool)

The tool_calls field is sent to the LLM so that the model has context about which tools were invoked and with what arguments. This is necessary for the LLM to understand the conversation history and make informed decisions about subsequent tool calls or responses.

The tool_use block behavior

The user noticed a confusing behavior: when the content list of an AIMessage contains a block of type tool_use, the inputs in that block are not sent to the LLM again. The accepted answer explains that this is LangChain being smart about avoiding duplication. The tool_use block in the content list is essentially a duplicate of the information already present in the tool_calls field. LangChain recognizes this and does not send the tool_use block content again, because the same data is already being sent via tool_calls. This prevents redundant token usage, but it can be confusing if you are inspecting the raw message structure and expecting both fields to be transmitted.

Impact on token usage

For a use case like a write_file tool call that may have a very long argument (e.g., the entire content of a file to be written), this behavior can cause token usage to balloon quickly. Every time you invoke the LLM with the same message history, the full tool_calls arguments are sent again. If you have a conversation that involves multiple tool calls with large arguments, the token count can become very high, leading to increased costs and slower responses.

How to manage token usage

The accepted answer provides two practical approaches to manage this:

1. Strip old tool calls before calling the LLM. You can manually remove tool_calls from AIMessage objects that are no longer needed for context. For example, if you only need the last few tool calls for the LLM to understand the current state, you can filter out older ones.

2. Keep your message history short or replace bulky tool calls with summaries. Instead of sending the full file content in every tool call, you can replace the arguments with a summary or a reference (e.g., a file path or a short description). This reduces the token count while still providing enough context for the LLM.

Step-by-step: Trimming tool_calls manually

Here is a practical example of how to strip old tool calls from an AIMessage before invoking the LLM. This assumes you are using LangChain in Python.

from langchain_core.messages import AIMessage, HumanMessage

def trim_tool_calls(messages, keep_last_n=2):
    """
    Remove tool_calls from AIMessages older than the last N.
    """
    trimmed = []
    tool_call_count = 0
    for msg in reversed(messages):
        if isinstance(msg, AIMessage) and msg.tool_calls:
            if tool_call_count < keep_last_n:
                tool_call_count += 1
                trimmed.insert(0, msg)
            else:
                # Remove tool_calls but keep the message content
                trimmed.insert(0, AIMessage(content=msg.content, tool_calls=[]))
        else:
            trimmed.insert(0, msg)
    return trimmed

# Example usage
messages = [
    HumanMessage(content="Write a file with a long content"),
    AIMessage(content="", tool_calls=[{"id": "call1", "type": "function", "args": {"filename": "test.txt", "content": "A" * 10000}}]),
    HumanMessage(content="Now write another file"),
    AIMessage(content="", tool_calls=[{"id": "call2", "type": "function", "args": {"filename": "test2.txt", "content": "B" * 10000}}]),
]

trimmed_messages = trim_tool_calls(messages, keep_last_n=1)
# The first AIMessage's tool_calls are removed, but the second is kept

Step-by-step: Replacing bulky tool calls with summaries

Instead of sending the full file content, you can replace the args with a summary. For example:

from langchain_core.messages import AIMessage

def summarize_tool_call(tool_call):
    """
    Replace large arguments with a summary.
    """
    if "content" in tool_call["args"] and len(tool_call["args"]["content"]) > 100:
        original_length = len(tool_call["args"]["content"])
        tool_call["args"]["content"] = f"[File content truncated: {original_length} characters]"
    return tool_call

# Example
original_call = {"id": "call1", "type": "function", "args": {"filename": "test.txt", "content": "A" * 10000}}
summarized_call = summarize_tool_call(original_call)
# summarized_call["args"]["content"] is now "[File content truncated: 10000 characters]"