stanfordnlp/GloVe
FreeSoftware in C and data files for the popular GloVe model for distributed word representations, a.k.a. word vectors or embeddings
About stanfordnlp/GloVe
GloVe (Global Vectors for Word Representation) is an unsupervised learning algorithm developed by the Stanford NLP group for obtaining distributed word representations (word vectors). The implementation is written in C and provides tools to train word vectors on new corpora as well as download pre-trained vectors from large-scale web datasets. Pre-trained vectors are available for corpora such as Common Crawl (42B and 840B tokens), Wikipedia 2014 + Gigaword 5, Twitter, and newly released 2024 vectors trained on Dolma and Wikipedia+Gigaword 5. The repository includes source code, a demo script for quick testing on a small Wikipedia sample, and an evaluation script for word analogy tasks. It is widely used in natural language processing for tasks like word similarity, analogy reasoning, and as feature inputs to downstream models.
Key Features
Pros & Cons
- Efficient C implementation with low memory footprint for training
- Widely adopted and benchmarked in NLP research
- Pre-trained vectors available off-the-shelf for many common domains
- Open source (Apache 2.0 license) with active community and clear documentation
- Supports training on arbitrary text corpora, enabling domain-specific embeddings
- Includes both training code and evaluation scripts in a single repository
- Produces static word embeddings (no context sensitivity like ELMo or BERT)
- Training requires significant RAM for large vocabularies and high-dimensional vectors
- No built-in GPU acceleration (CPU-only implementation in C)
- Requires compilation with GCC and GNU Make, which may be non-trivial for some users
- Pre-trained vectors may become outdated if not periodically updated (new 2024 vectors address this partially)