Conference Paper
Large Language Models

Bias and Fairness in Large Language Models: A Survey

Isabel O. Gallegos(Stanford University), Ryan A. Rossi(Adobe Systems (United States)), Joe Barrow, Md Mehrab Tanjim(Adobe Systems (United States)), Sungchul Kim(Adobe Systems (United States)), Franck Dernoncourt(Adobe Systems (United States)), Tong Yu(Adobe Systems (United States)), Ruiyi Zhang(Adobe Systems (United States)), Nesreen K. Ahmed(Intel (United States))
January 1, 2024Computational Linguistics1,193 citations
arXivPDFDOI

1.2k

Citations

67

Influential Citations

Computational Linguistics

Venue

2024

Year

Abstract

Abstract Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this article, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely, metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.

Analysis

Why This Paper Matters

As large language models (LLMs) become increasingly integrated into social systems, the risk of perpetuating and amplifying harmful social biases grows. This survey addresses a critical gap by providing a structured, comprehensive overview of bias evaluation and mitigation techniques. It consolidates fragmented literature into clear taxonomies, making it an essential resource for researchers and practitioners aiming to build fairer AI systems. The paper's formalization of bias and fairness notions in NLP is particularly valuable, as it establishes a common language and set of desiderata that can guide future work.

Technical Contributions

The paper's main innovations are its three taxonomies:

  • Bias Evaluation Metrics Taxonomy: Organizes metrics by the level at which they operate in a model—embeddings, probabilities, and generated text—clarifying the relationship between metrics and evaluation datasets.
  • Bias Evaluation Datasets Taxonomy: Categorizes datasets by structure (counterfactual inputs or prompts) and identifies targeted harms and social groups, with a consolidated list of publicly available datasets.
  • Bias Mitigation Techniques Taxonomy: Classifies methods by intervention stage: pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that highlight research trends.

These taxonomies unify a diverse and rapidly growing field, enabling easier comparison and selection of appropriate methods.

Results

As a survey, the paper does not present new experimental results. Instead, it synthesizes findings from 1193 cited works, providing a qualitative overview of the landscape. The paper identifies that bias evaluation metrics span from embedding-based (e.g., WEAT) to probability-based (e.g., log-probability bias) and text-based (e.g., toxicity classifiers). Mitigation techniques range from data augmentation (pre-processing) to adversarial training (in-training) and decoding strategies (post-processing). No quantitative comparisons are made between methods.

Significance

This survey has broad implications for the AI field. By providing a structured guide, it lowers the barrier for practitioners to incorporate fairness considerations into LLM development. The formalization of bias and fairness desiderata can influence future evaluation standards and regulatory frameworks. The paper also highlights open challenges, such as intersectional bias and dynamic fairness, pointing to important directions for future research. Its impact is likely to be substantial given the high citation count (1193) and the timeliness of the topic.

large-language-modelsbiasfairnesssurveyevaluationmitigationnlp
Large Language ModelsPrompting

Related Papers