Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

Analysis

Why This Paper Matters

This paper addresses a critical challenge in AI fairness: locating and mitigating demographic bias in large language models without degrading their capabilities. While prior work often treats bias as a global property, this research shows that bias can be traced to specific neurons in GLU-MLP layers, offering a more granular and actionable target. The finding that zeroing as few as 40 neurons (0.031% of MLP width) preserves 99.49% of reasoning and knowledge is a striking demonstration of the dissociability of bias and capability circuits.

The methodological approach—using minimally contrastive prompt pairs and inference-time activation capture—provides a causal, rather than correlational, lens on bias. This is a significant step beyond observational bias metrics, as it directly tests the effect of neuron removal on model behavior. The paper's focus on GLU architectures (common in modern LLMs) and models up to 3B parameters makes it relevant to a wide range of deployed systems.

Technical Contributions

• Differential Activation Localization: The method identifies neurons that react differentially to demographic attributes by comparing activations from minimally contrastive prompt pairs, evaluated at the down_proj input of GLU layers.
• Causal Intervention via Zeroing: By zeroing the identified neurons, the method establishes a causal link between specific neurons and demographic bias, moving beyond correlation.
• Surgical Pruning: The intervention is extremely targeted, affecting less than 0.031% of total MLP width in Llama-3.2-1B, demonstrating high precision.
• Bidirectional Bias Destabilization: The paper reveals that because BiasScore is unsigned, the candidate neuron set includes both pro- and anti-stereotype neurons, leading to bidirectional effects on aggregate bias. This is an important nuance for future mitigation strategies.
• Capability Retention: The method preserves 99.49% of reasoning and general knowledge, showing that bias circuits are separable from capability circuits.

Results

The empirical evaluation on Llama-3.2 family and Salamandra-2B (up to 3B parameters) shows that zeroing the identified neurons alters responses to demographic variables. However, the net effect on aggregate bias is not uniformly reduced; instead, it destabilizes bias bidirectionally, depending on the dominant sign in the candidate set. The intervention's surgical nature is highlighted by the fact that zeroing at most 40 neurons in Llama-3.2-1B (less than 0.031% of MLP width) retains 99.49% of reasoning and general knowledge capabilities. These results confirm that demographic bias processing and model capabilities operate on dissociable circuits.

Significance

This work establishes a foundational methodology for causal bias localization in LLMs, which is essential for moving from blind zeroing to directional behavior modulation. The demonstration that bias and capabilities are dissociable opens the door to more nuanced fairness interventions that can selectively adjust model behavior without sacrificing performance. The bidirectional destabilization finding is a cautionary note for future work, emphasizing the need for signed bias metrics or more refined selection criteria. Overall, this paper contributes to the growing field of mechanistic interpretability for fairness, with potential applications in auditing and debiasing deployed LLMs.