Fix RuntimeError: Failed to import transformers 'NoneType' object has no attribute 'split' in Docker

The RuntimeError: Failed to import transformers.models.auto.modeling_auto because of the following error: 'NoneType' object has no attribute 'split' occurs when the transformers library (version 4.39.3) attempts to parse CUDA device environment variables that are missing or return None in a Docker container. The root cause is that the standard python:3.11-slim image lacks the NVIDIA CUDA runtime libraries that transformers expects, causing device queries to fail silently. The fix is to use an official NVIDIA CUDA base image and ensure proper environment variable mapping, as confirmed by the accepted Stack Overflow solution.

The Full Answer

This error appears during the import chain of sentence_transformers (version 2.7.0) when it tries to import transformers.models.auto.modeling_auto. The traceback shows the failure originates in transformers/utils/import_utils.py at line 1463, where getattr(module, name) is called. The underlying issue is that transformers scans for CUDA capabilities by reading environment variables like CUDA_VISIBLE_DEVICES, CUDA_DEVICE_ORDER, or internal device query results. In a Docker container without NVIDIA drivers, these queries return None instead of a string, and the code tries to call .split() on None, causing the crash.

The Environment

The user reported this error with:

• Python 3.11 (in Docker)
• TensorFlow 2.15.0
• tf-keras 2.15.1
• Transformers 4.39.3
• Sentence-Transformers 2.7.0
• Keras (standalone) uninstalled

The error occurs during FastAPI application startup, specifically when sentence_transformers.CrossEncoder is imported inside a RAG pipeline module. The full traceback shows the import chain: sentence_transformers.__init__ -> datasets -> ParallelSentencesDataset -> SentenceTransformer -> models.Transformer -> transformers.AutoModel -> modeling_auto.

Solution 1: Use NVIDIA CUDA Base Image with Multi-Stage Build (Recommended)

This is the accepted solution from the Stack Overflow answer. It addresses the root cause by providing the system-level CUDA libraries that transformers expects.

Step 1: Host Preparation

On Windows, ensure WSL2 is installed. In Docker Desktop, go to Settings -> Resources -> WSL Integration and enable it for your distribution (e.g., Ubuntu). This ensures Docker can access the NVIDIA GPU from WSL2.

Step 2: Create the Multi-Stage Dockerfile

Replace your existing Dockerfile with this multi-stage build that uses nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 as the base image.

# Stage 1: Builder
FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 as builder

ENV DEBIAN_FRONTEND=noninteractive

# Install Python 3.11 and build tools
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.11 \
    python3.11-dev \
    python3.11-venv \
    build-essential \
    gcc \
    wget \
    curl \
    git \
    libffi-dev \
    libssl-dev \
    && rm -rf /var/lib/apt/lists/*

# Setup Virtual Environment to isolate dependencies
RUN python3.11 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Stage 2: Runtime
FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 as runtime

ENV DEBIAN_FRONTEND=noninteractive
# Crucial: Map the venv path to the new container's environment
ENV PATH="/opt/venv/bin:$PATH"

# Copy only the compiled venv from builder
COPY --from=builder /opt/venv /opt/venv

WORKDIR /app
COPY . .

CMD ["python", "grpc_server.py"]

Step 3: Why This Fixes the Error

The nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 image includes the NVIDIA CUDA runtime libraries (.so files) that transformers uses to query GPU capabilities. In the standard python:3.11-slim image, these libraries are absent, so device queries return None. The multi-stage build also ensures that the virtual environment is built in a consistent environment and then copied to the runtime stage, avoiding conflicts with system-level Python packages.

Solution 2: Environment Variable Workarounds (Partial Fix)

The user who reported the error tried setting several environment variables, but these did not resolve the issue. However, they may help in some configurations. Documenting them here for completeness:

import os
os.environ["TRANSFORMERS_NO_TF"] = "1"
os.environ["TF_USE_LEGACY_KERAS"] = "1"
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"

• TRANSFORMERS_NO_TF=1: Tells transformers to skip TensorFlow-specific imports. This might avoid some code paths that trigger the device query.
• TF_USE_LEGACY_KERAS=1: Ensures TensorFlow uses the legacy Keras API (tf-keras) instead of the standalone Keras 3, which can cause import conflicts.
• CUDA_VISIBLE_DEVICES=-1: Forces TensorFlow to run on CPU only. However, the user reported that this did not fix the error because transformers parses the raw CUDA environment before TensorFlow's override takes effect.

These variables are not a complete fix but may reduce the likelihood of the error in some edge cases. The accepted solution (Solution 1) is the definitive fix.

What About Keras Conflicts?

The user initially suspected a Keras 2 vs Keras 3 conflict and uninstalled the standalone keras package, keeping only tf-keras==2.15.1. This is a good practice to avoid import ambiguity, but it did not resolve the 'NoneType' object has no attribute 'split' error. The error is not caused by Keras version conflicts; it is caused by missing CUDA runtime libraries.