Fix: pip install anthropic Fails on Python 3.13 (tokenizers dependency)

Diagnosis

Running pip install anthropic on Python 3.13 currently fails because one of the SDK's dependencies, tokenizers, requires pyo3-ffi, which is not yet compiled for Python 3.13. The exact error you will see is a build failure for tokenizers, typically ending with something like error: can't find Rust compiler or a more cryptic ModuleNotFoundError for pyo3-ffi during the pip install process. The root cause is that the tokenizers package (a dependency of anthropic) has not yet published a wheel for Python 3.13, and its source build requires pyo3-ffi, which also lacks a Python 3.13 compatible release. This means that as of now, no environment running Python 3.13 can successfully pip install anthropic without a workaround.

What Causes This Error

1. The tokenizers dependency is not yet available for Python 3.13

This is the primary cause, confirmed in the GitHub issue discussion. The anthropic Python SDK lists tokenizers as a dependency. The tokenizers package is a Rust-based library that provides fast tokenization for natural language processing models. It relies on pyo3-ffi, a Rust-to-Python bridge, to compile its native extensions. As of the time of this writing, pyo3-ffi has not been updated to support Python 3.13. This creates a chain of failures:

• pip install anthropic pulls in tokenizers as a dependency.
• tokenizers attempts to build from source because no precompiled wheel exists for Python 3.13 on your platform.
• The build process requires pyo3-ffi, which fails because it doesn't recognize Python 3.13's internal API changes.
• The entire installation fails with a build error.

2. The tokenizers dependency may be unnecessary for most users

According to the GitHub issue reporter, the tokenizers dependency is only used for a "VERY old count tokens feature" that relies on the Claude 2 tokenizer. This feature is not actively useful for most modern use cases, as the Claude 2 tokenizer is outdated and the SDK's primary token counting mechanism (via the API) does not require tokenizers. The relevant code in the SDK is located in src/anthropic/_client.py around lines 270-286, where a local token counting function is defined. This function is a fallback and is not called in the normal API request flow. The dependency is therefore a legacy artifact that creates an unnecessary barrier for Python 3.13 users.

3. No official fix from Anthropic yet

At the time of the GitHub issue, the Anthropic team had not released a patch to remove or replace the tokenizers dependency. The issue remains open, and the workaround is community-driven. This means that simply waiting for a new release of the anthropic package may not resolve the problem immediately, as the fix requires either:

• The tokenizers package to release a Python 3.13 compatible wheel, or
• The anthropic SDK to drop or conditionalize the tokenizers dependency.

4. Environment-specific factors

• Operating system: Linux, macOS, and Windows all experience this issue because it's a Python version problem, not an OS-specific one. However, users on systems without a preinstalled Rust compiler may see a different error message (e.g., "error: can't find Rust compiler") before the pyo3-ffi failure.
• Python installation method: Whether you use the official Python.org installer, Homebrew, pyenv, or conda, the issue persists because it's tied to the Python 3.13 version identifier, not the distribution.
• Virtual environment: Using a virtual environment does not help because the dependency resolution happens at the package level, not the environment level.

How to Fix It

Solution 1: Install from a community-maintained fork (Recommended for immediate use)

This is the most reliable workaround as of the current state of the issue. Community member simonw created a fork of the anthropic-sdk-python repository that removes the tokenizers dependency. You can install directly from this fork using pip.

Steps:

1. Ensure you are using Python 3.13 and have pip installed.
2. Run the following command:

python -m pip install https://github.com/simonw/anthropic-sdk-python/archive/9c13bb441ee4eb88a100ed363fc431ec8fd30c43.zip

This command downloads a zip archive of the fork at a specific commit (9c13bb441ee4eb88a100ed363fc431ec8fd30c43) and installs it. The fork has the tokenizers dependency removed from setup.py or pyproject.toml, allowing the installation to complete on Python 3.13.

What to expect:

• The installation should succeed without errors.
• You can then import anthropic and use the SDK as normal.
• The only missing functionality is the local token counting feature, which is not used in typical API calls. The SDK will still count tokens via the API when you send messages, so this is transparent.
• If you need to update the SDK later, you will need to repeat this process with a newer commit from the same fork, or switch back to the official package once it's fixed.

Caveats:

• This is a community fork, not an official Anthropic release. Use it at your own risk in production environments. The fork may not receive updates as quickly as the official package.
• The fork may lag behind the official SDK in terms of features and bug fixes.
• If you are using a dependency manager like Poetry or Pipenv, you will need to specify this URL as the source for the anthropic package.

Solution 2: Downgrade to Python 3.12 or earlier

If you cannot use a community fork and need the official anthropic package, the most straightforward solution is to use a Python version that is supported by all dependencies. Python 3.12 and earlier versions work without issue.

Steps:

1. Install Python 3.12 (or 3.11, 3.10) using your preferred method:
   • pyenv: pyenv install 3.12.0 then pyenv local 3.12.0
   • Homebrew: brew install python@3.12
   • Official installer: Download from python.org
2. Create a new virtual environment with Python 3.12:

python3.12 -m venv myenv
source myenv/bin/activate  # On Windows: myenv\Scripts\activate

3. Install the official package:

pip install anthropic

What to expect:

• The installation will succeed because tokenizers has precompiled wheels for Python 3.12.
• All features of the SDK, including local token counting, will work.
• You lose the ability to use Python 3.13 specific features or performance improvements, but for most projects this is not a significant loss.

Caveats:

• This is a temporary solution. Once the dependency issue is resolved, you can migrate back to Python 3.13.
• If your project requires Python 3.13 for other reasons (e.g., new language features), this is not an option.

Solution 3: Install the tokenizers dependency from a pre-release or alternative source

If you want to keep Python 3.13 and still use the official anthropic package, you can try to install a pre-release version of tokenizers that may have Python 3.13 support. This is a more advanced solution and may not work immediately.

Steps:

1. First, try installing the latest pre-release of tokenizers:

pip install --pre tokenizers

This tells pip to include pre-release versions. If a Python 3.13 compatible version exists as a release candidate or alpha, this will install it.

2. If that fails, try installing from the tokenizers GitHub repository's main branch:

pip install git+https://github.com/huggingface/tokenizers.git

This builds from source. You will need a Rust compiler installed on your system. See the prerequisites below.

3. Once tokenizers is installed, install anthropic:

pip install anthropic

Prerequisites for building from source:

• Rust: Install Rust via rustup (https://rustup.rs/) or your system package manager. On macOS: brew install rust. On Ubuntu: sudo apt install rustc cargo.
• Build tools: On Linux, you may need python3-dev or python3-devel and build-essential. On macOS, Xcode command line tools are required (xcode-select --install).

What to expect:

• The build process may take several minutes.
• If successful, both tokenizers and anthropic will be installed.
• This solution is fragile because it depends on the tokenizers main branch being compatible with Python 3.13 at the time you run the command. It may break if the main branch introduces breaking changes.

Caveats:

• This is not guaranteed to work. The pyo3-ffi issue may still prevent compilation.
• You are now using a potentially unstable version of tokenizers.
• If the anthropic SDK updates its tokenizers version requirement, you may encounter version conflicts.

Solution 4: Patch the anthropic package locally

If you prefer not to use a fork and cannot downgrade Python, you can manually patch the installed package to remove the tokenizers dependency. This is a hacky solution but works in a pinch.

Steps:

1. Download the official anthropic package source:

pip download anthropic --no-binary :all: -d ./anthropic-src
cd ./anthropic-src
tar -xzf anthropic-*.tar.gz
cd anthropic-*/

2. Edit setup.py or pyproject.toml to remove the tokenizers line from the install_requires list. Look for a line like "tokenizers>=0.13.0" or similar and delete it.

3. Build and install the patched package:

pip install .

What to expect:

• The installation will succeed without tokenizers.
• The SDK will work, but the local token counting function will raise an ImportError if called. Since this function is not used in normal operation, you may never encounter this.
• If you need to update the package later, you will have to repeat this process.

Caveats:

• This is a manual process and error-prone.
• The patched package is not managed by pip for updates. You will need to track new releases manually.
• If the SDK's internal code changes in a future version to rely more heavily on tokenizers, this patch may break functionality.