OpenAI Embeddings for Semantic Search: A Practical Guide

Learn how to use OpenAI's text embedding models for semantic search, clustering, recommendations, and classification. This guide covers concepts, setup, API usage, dimension reduction, and practical tips from official documentation and community experience.

This guide covers how to use OpenAI's text embedding models to build semantic search, clustering, recommendation, and classification systems. It is for developers and data scientists who want to turn text into numerical vectors and use them for similarity-based retrieval and analysis. You will learn the concepts, setup, API usage, advanced techniques, and real-world tips from both official documentation and community experience.

What You Need

Before you start, make sure you have the following:

• An OpenAI API account: You need an API key from the OpenAI platform. This key must be set as the OPENAI_API_KEY environment variable or passed directly to the client library.
• Python 3.7+: The examples in this guide use Python. You will need the openai Python package installed. Install it with pip install openai.
• A vector database (optional but recommended): For production-scale semantic search, you will need a vector database to store and query embeddings efficiently. The official documentation recommends this for fast retrieval of K nearest neighbors.
• The tiktoken library (optional): To count tokens in your input strings before sending them to the API, you can use OpenAI's tokenizer. Install it with pip install tiktoken.
• Basic knowledge of Python and NumPy: The code examples use NumPy for vector operations and scikit-learn for machine learning tasks.

What Are Embeddings?

OpenAI's text embeddings measure the relatedness of text strings. An embedding is a vector (a list) of floating point numbers. The distance between two vectors measures their relatedness. Small distances suggest high relatedness, and large distances suggest low relatedness.

Embeddings are commonly used for:

• Search: Results are ranked by relevance to a query string.
• Clustering: Text strings are grouped by similarity.
• Recommendations: Items with related text strings are recommended.
• Anomaly detection: Outliers with little relatedness are identified.
• Diversity measurement: Similarity distributions are analyzed.
• Classification: Text strings are classified by their most similar label.

Embedding Models

OpenAI offers two powerful third-generation embedding models, denoted by -3 in the model ID. The official documentation states that these models feature lower costs, higher multilingual performance, and new parameters to control the overall size.

Model Comparison

• text-embedding-3-small is the most cost-effective model, offering the highest number of pages per dollar. It is suitable for most applications where budget is a concern.
• text-embedding-3-large provides the best performance on the MTEB (Massive Text Embedding Benchmark) evaluation, but at a higher cost per token.
• text-embedding-ada-002 is the previous generation model. It is still available but is outperformed by the newer models in both cost and accuracy.

By default, the length of the embedding vector is 1536 for text-embedding-3-small and 3072 for text-embedding-3-large.

Important Note on Knowledge Cutoff

The official documentation explicitly states that the text-embedding-3-large and text-embedding-3-small models lack knowledge of events that occurred after September 2021. This is generally not as much of a limitation as it would be for text generation models, but in certain edge cases it can reduce performance.

How to Get Embeddings

To get an embedding, send your text string to the embeddings API endpoint along with the embedding model name. The official documentation provides examples in Python, JavaScript, and cURL.

Python Example

from openai import OpenAI
client = OpenAI()

response = client.embeddings.create(
    input="Your text string goes here",
    model="text-embedding-3-small"
)

print(response.data[0].embedding)

JavaScript Example

import OpenAI from "openai";
const openai = new OpenAI();

const embedding = await openai.embeddings.create({
  model: "text-embedding-3-small",
  input: "Your text string goes here",
  encoding_format: "float",
});

console.log(embedding);

cURL Example

curl https://api.openai.com/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "input": "Your text string goes here",
    "model": "text-embedding-3-small"
  }'

Response Structure

The response contains the embedding vector along with some additional metadata. Here is an example response:

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [
        -0.006929283495992422, -0.005336422007530928, -4.547132266452536e-05,
        -0.024047505110502243
      ]
    }
  ],
  "model": "text-embedding-3-small",
  "usage": {
    "prompt_tokens": 5,
    "total_tokens": 5
  }
}

The embedding field contains the vector (list of floating point numbers). The usage field shows how many tokens were used for the input. You can extract the embedding vector, save it in a vector database, and use it for many different use cases.

Reducing Embedding Dimensions

Using larger embeddings generally costs more and consumes more compute, memory, and storage than using smaller embeddings. Both new embedding models were trained with a technique that allows developers to trade off performance and cost. Specifically, developers can shorten embeddings (remove some numbers from the end of the sequence) without the embedding losing its concept-representing properties by passing in the dimensions API parameter.

Using the dimensions Parameter

The official documentation suggests that using the dimensions parameter when creating the embedding is the suggested approach. For example, on the MTEB benchmark, a text-embedding-3-large embedding can be shortened to a size of 256 while still outperforming an unshortened text-embedding-ada-002 embedding with a size of 1536.

from openai import OpenAI
client = OpenAI()

response = client.embeddings.create(
    model="text-embedding-3-large",
    input="Your text string goes here",
    dimensions=256
)

print(response.data[0].embedding)

Manual Dimension Reduction and Normalization

In certain cases, you may need to change the embedding dimension after you generate it. When you change the dimension manually, you need to be sure to normalize the dimensions of the embedding. The official documentation provides a function for L2 normalization:

from openai import OpenAI
import numpy as np

client = OpenAI()

def normalize_l2(x):
    x = np.array(x)
    if x.ndim == 1:
        norm = np.linalg.norm(x)
        if norm == 0:
            return x
        return x / norm
    else:
        norm = np.linalg.norm(x, 2, axis=1, keepdims=True)
        return np.where(norm == 0, x, x / norm)

response = client.embeddings.create(
    model="text-embedding-3-small", input="Testing 123", encoding_format="float"
)

cut_dim = response.data[0].embedding[:256]
norm_dim = normalize_l2(cut_dim)

print(norm_dim)

This code first generates an embedding, then truncates it to the first 256 dimensions, and finally normalizes the truncated vector. Normalization is important because it ensures that cosine similarity and Euclidean distance produce identical rankings, as the official documentation notes that OpenAI embeddings are normalized to length 1.

Practical Use Case for Dimension Reduction

Dynamically changing the dimensions enables very flexible usage. For example, when using a vector data store that only supports embeddings up to 1024 dimensions long, developers can still use the best embedding model text-embedding-3-large and specify a value of 1024 for the dimensions API parameter. This will shorten the embedding down from 3072 dimensions, trading off some accuracy in exchange for the smaller vector size.

Use Cases

The official documentation provides several Jupyter notebooks that demonstrate representative use cases, using the Amazon fine-food reviews dataset. The dataset contains a total of 568,454 food reviews left by Amazon users up to October 2012. The examples use a subset of the 1000 most recent reviews for illustration purposes.

Obtaining the Embeddings from a Dataset

The following code shows how to get embeddings for a dataset and save them to a CSV file:

from openai import OpenAI
client = OpenAI()

def get_embedding(text, model="text-embedding-3-small"):
    text = text.replace("\n", " ")
    return client.embeddings.create(input=[text], model=model).data[0].embedding

df['ada_embedding'] = df.combined.apply(lambda x: get_embedding(x, model='text-embedding-3-small'))
df.to_csv('output/embedded_1k_reviews.csv', index=False)

To load the data from a saved file, you can run the following:

import pandas as pd

df = pd.read_csv('output/embedded_1k_reviews.csv')
df['ada_embedding'] = df.ada_embedding.apply(eval).apply(np.array)

Text Search Using Embeddings

To retrieve the most relevant documents, you use cosine similarity between the embedding vectors of the query and each document, and return the highest scored documents.

from openai.embeddings_utils import get_embedding, cosine_similarity

def search_reviews(df, product_description, n=3, pprint=True):
    embedding = get_embedding(product_description, model='text-embedding-3-small')
    df['similarities'] = df.ada_embedding.apply(lambda x: cosine_similarity(x, embedding))
    res = df.sort_values('similarities', ascending=False).head(n)
    return res

res = search_reviews(df, 'delicious beans', n=3)

This function takes a DataFrame with pre-computed embeddings, a query string, and the number of results to return. It computes the embedding for the query, calculates cosine similarity against all stored embeddings, sorts by similarity, and returns the top N results.

Code Search Using Embeddings

Code search works similarly to text search. You extract Python functions from all Python files in a repository, index each function using the text-embedding-3-small model, and then search by embedding the query in natural language.

from openai.embeddings_utils import get_embedding, cosine_similarity

df['code_embedding'] = df['code'].apply(lambda x: get_embedding(x, model='text-embedding-3-small'))

def search_functions(df, code_query, n=3, pprint=True, n_lines=7):
    embedding = get_embedding(code_query, model='text-embedding-3-small')
    df['similarities'] = df.code_embedding.apply(lambda x: cosine_similarity(x, embedding))
    res = df.sort_values('similarities', ascending=False).head(n)
    return res

res = search_functions(df, 'Completions API tests', n=3)

Recommendations Using Embeddings

Because shorter distances between embedding vectors represent greater similarity, embeddings can be useful for recommendation. The following function takes a list of strings and one source string, computes their embeddings, and returns a ranking from most similar to least similar.

def recommendations_from_strings(
    strings: List[str],
    index_of_source_string: int,
    model="text-embedding-3-small",
) -> List[int]:
    """Return nearest neighbors of a given string."""
    # get embeddings for all strings
    embeddings = [embedding_from_string(string, model=model) for string in strings]

    # get the embedding of the source string
    query_embedding = embeddings[index_of_source_string]

    # get distances between the source embedding and other embeddings
    distances = distances_from_embeddings(query_embedding, embeddings, distance_metric="cosine")

    # get indices of nearest neighbors
    indices_of_nearest_neighbors = indices_of_nearest_neighbors_from_distances(distances)
    return indices_of_nearest_neighbors

Data Visualization in 2D

The size of the embeddings varies with the complexity of the underlying model. To visualize this high-dimensional data, you can use the t-SNE algorithm to transform the data into two dimensions.

import pandas as pd
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
import matplotlib

df = pd.read_csv('output/embedded_1k_reviews.csv')
matrix = df.ada_embedding.apply(eval).to_list()

# Create a t-SNE model and transform the data
tsne = TSNE(n_components=2, perplexity=15, random_state=42, init='random', learning_rate=200)
vis_dims = tsne.fit_transform(matrix)

colors = ["red", "darkorange", "gold", "turquiose", "darkgreen"]
x = [x for x,y in vis_dims]
y = [y for x,y in vis_dims]
color_indices = df.Score.values - 1

colormap = matplotlib.colors.ListedColormap(colors)
plt.scatter(x, y, c=color_indices, cmap=colormap, alpha=0.3)
plt.title("Amazon ratings visualized in language using t-SNE")

This code visualizes the Amazon reviews, coloring each point based on the star rating: 1-star (red), 2-star (dark orange), 3-star (gold), 4-star (turquoise), and 5-star (dark green). The visualization tends to produce roughly three clusters, one of which has mostly negative reviews.

Embedding as a Text Feature Encoder for ML Algorithms

An embedding can be used as a general free-text feature encoder within a machine learning model. The official documentation notes that incorporating embeddings will improve the performance of any machine learning model if some of the relevant inputs are free text. An embedding can also be used as a categorical feature encoder, which adds most value if the names of categorical variables are meaningful and numerous, such as job titles. Similarity embeddings generally perform better than search embeddings for this task.

A key observation from the documentation is that the embedding representation is very rich and information dense. Reducing the dimensionality of the inputs using SVD or PCA, even by 10%, generally results in worse downstream performance on specific tasks.

Regression Using the Embedding Features

Embeddings present an elegant way of predicting a numerical value. In this example, the goal is to predict the reviewer's star rating based on the text of their review.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    list(df.ada_embedding.values),
    df.Score,
    test_size=0.2,
    random_state=42
)

from sklearn.ensemble import RandomForestRegressor

rfr = RandomForestRegressor(n_estimators=100)
rfr.fit(X_train, y_train)
preds = rfr.predict(X_test)

The ML algorithm minimizes the distance of the predicted value to the true score and achieves a mean absolute error of 0.39, meaning that on average the prediction is off by less than half a star.

Classification Using the Embedding Features

Instead of predicting a continuous value, you can classify the exact number of stars for a review into 5 buckets, ranging from 1 to 5 stars.

from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, accuracy_score

clf = RandomForestClassifier(n_estimators=100)
clf.fit(X_train, y_train)
preds = clf.predict(X_test)

After training, the model learns to predict 1 and 5-star reviews much better than the more nuanced reviews (2-4 stars), likely due to more extreme sentiment expression.

Zero-Shot Classification

You can use embeddings for zero-shot classification without any labeled training data. For each class, you embed the class name or a short description of the class. To classify some new text, you compare its embedding to all class embeddings and predict the class with the highest similarity.

from openai.embeddings_utils import cosine_similarity, get_embedding

df = df[df.Score != 3]
df['sentiment'] = df.Score.replace({1: 'negative', 2: 'negative', 4: 'positive', 5: 'positive'})

labels = ['negative', 'positive']
label_embeddings = [get_embedding(label, model=model) for label in labels]

def label_score(review_embedding, label_embeddings):
    return cosine_similarity(review_embedding, label_embeddings[1]) - cosine_similarity(review_embedding, label_embeddings[0])

prediction = 'positive' if label_score('Sample Review', label_embeddings) > 0 else 'negative'

This code filters out neutral reviews (score of 3), creates binary sentiment labels, embeds the label names, and then classifies a new review by comparing its embedding to the label embeddings.

Obtaining User and Product Embeddings for Cold-Start Recommendation

You can obtain a user embedding by averaging over all of their reviews. Similarly, you can obtain a product embedding by averaging over all the reviews about that product.

user_embeddings = df.groupby('UserId').ada_embedding.apply(np.mean)
prod_embeddings = df.groupby('ProductId').ada_embedding.apply(np.mean)

This approach can be used for cold-start recommendations, where even before a user receives a product, you can predict better than random whether they would like the product by comparing the similarity of the user and product embeddings.

Clustering

Clustering is one way of making sense of a large volume of textual data. Embeddings are useful for this task, as they provide semantically meaningful vector representations of each text.

import numpy as np
from sklearn.cluster import KMeans

matrix = np.vstack(df.ada_embedding.values)
n_clusters = 4

kmeans = KMeans(n_clusters=n_clusters, init='k-means++', random_state=42)
kmeans.fit(matrix)
df['Cluster'] = kmeans.labels_

This example uses K-Means clustering to discover four distinct clusters in the Amazon reviews dataset.

FAQ

How can I tell how many tokens a string has before I embed it?

In Python, you can split a string into tokens with OpenAI's tokenizer tiktoken. For third-generation embedding models like text-embedding-3-small, use the cl100k_base encoding.

import tiktoken

def num_tokens_from_string(string: str, encoding_name: str) -> int:
    """Returns the number of tokens in a text string."""
    encoding = tiktoken.get_encoding(encoding_name)
    num_tokens = len(encoding.encode(string))
    return num_tokens

num_tokens_from_string("tiktoken is great!", "cl100k_base")

How can I retrieve K nearest embedding vectors quickly?

For searching over many vectors quickly, the official documentation recommends using a vector database. You can find examples of working with vector databases and the OpenAI API in the OpenAI Cookbook on GitHub.

Which distance function should I use?

The official documentation recommends cosine similarity. The choice of distance function typically doesn't matter much. OpenAI embeddings are normalized to length 1, which means that:

• Cosine similarity can be computed slightly faster using just a dot product.
• Cosine similarity and Euclidean distance will result in the identical rankings.

Can I share my embeddings online?

Yes, customers own their input and output from our models, including in the case of embeddings. You are responsible for ensuring that the content you input to our API does not violate any applicable law or our Terms of Use.

Troubleshooting

Token Limit Issues

If you run into token limits, the official documentation suggests limiting the number of functions loaded up front, shortening descriptions where possible, or using tool search so deferred tools are loaded only when needed. For embedding models, the maximum input is 8192 tokens. You can use tiktoken to count tokens before sending a request to avoid errors.

Performance Degradation with Dimension Reduction

While reducing dimensions can save storage and compute, the official documentation warns that reducing the dimensionality of the inputs using SVD or PCA, even by 10%, generally results in worse downstream performance on specific tasks. The recommended approach is to use the dimensions API parameter when creating the embedding, which is designed to preserve concept-representing properties.

Knowledge Cutoff

The text-embedding-3-large and text-embedding-3-small models lack knowledge of events that occurred after September 2021. This is generally not a major limitation for semantic search, but it can reduce performance for queries about very recent events or topics.

Community-Reported Issues

Community members on various forums have reported that the openai.embeddings_utils module may not be available in newer versions of the openai Python package. If you encounter an import error, you can implement the cosine similarity function yourself:

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

Another common community tip is to batch your embedding requests to reduce API costs and improve throughput. The API accepts a list of strings in the input parameter, which can significantly reduce the number of API calls.

Going Further

To deepen your understanding and build more advanced systems, explore the following resources mentioned in the source material:

• Vector Databases: For production-scale semantic search, integrate with a vector database. The OpenAI Cookbook on GitHub provides examples of working with vector databases and the OpenAI API.
• OpenAI Cookbook: The official documentation references several Jupyter notebooks that provide complete, runnable examples for each use case. These are excellent resources for learning by doing.
• Embedding V3 Launch Blog Post: For more details on how changing the dimensions impacts performance, read the embeddings v3 launch blog post.
• Fine-Tuning for Function Calling: If you are combining embeddings with function calling, consider fine-tuning to increase function calling accuracy for large numbers of functions or difficult tasks. The OpenAI Cookbook has a guide on this.
• Tool Search: For applications with many functions, explore the tool_search feature (available with gpt-5.4 and later models) to defer loading rarely used tools.