Neura Market
StarCoder-1|3|7B logo

StarCoder-1|3|7B

Free

Open-source code generation models trained on 80+ languages

FreeFree tier
Inputs: textOutputs: text
Type
Open Source
Company
BigCode
Visit Website GitHub (38 stars)

About StarCoder-1|3|7B

StarCoder is a family of open-source large language models for code generation, developed by the BigCode project. The models are based on the StarCoderBase architecture, a 15-billion-parameter decoder-only transformer trained on 1 trillion tokens of code from over 80 programming languages. StarCoder (16B parameters) is a fine-tuned variant with an additional 30 billion tokens of Python code to enhance Python-specific performance. StarCoderPlus (16B parameters) extends the base model with 600 billion tokens of natural text from RefinedWeb and Wikipedia, improving general language understanding. The collection includes smaller variants (1B, 8B) and the associated dataset, The Stack, which provides the fully processed training data. All models are freely available on Hugging Face for download and use.

Key Features

Multiple model sizes: 1B, 8B, 15B (StarCoderBase), 16B (StarCoder and StarCoderPlus)
Trained on over 80 programming languages from The Stack dataset
StarCoder fine-tuned on additional 30B tokens of Python for code-centric tasks
StarCoderPlus fine-tuned on 600B tokens of natural text for improved language understanding
Open-source and freely available on Hugging Face
Published in May 2023 with accompanying research paper (2305.06161)

Pros & Cons

Pros
  • Open-source with permissive licensing (Apache 2.0 inferred)
  • Large context window (8,192 tokens) and high parameter count for strong code understanding
  • Variety of model sizes to suit different computational budgets
  • Trained on high-quality, permissively licensed code (The Stack)
  • Backed by the BigCode scientific collaboration, ensuring transparency
Cons
  • Large models (15B+) require significant GPU memory and compute resources
  • May not match proprietary models on certain narrow benchmarks
  • Limited documentation on specific supported programming languages
  • No official API or hosted inference provided by the project

Best For

Code generation and completion across multiple programming languagesPython-specific code development and assistanceNatural language to code translation (especially with StarCoderPlus)Learning and exploring code patterns from diverse language datasetsResearch in large language models for code

FAQ

What is StarCoder?
StarCoder is a family of open-source large language models for code generation, developed by the BigCode project. It includes models of varying sizes (1B to 16B parameters) trained on code from over 80 programming languages.
How is StarCoder different from StarCoderBase?
StarCoderBase is the base 15B parameter model trained on 1 trillion tokens of code. StarCoder is a fine-tuned version of StarCoderBase with an additional 30 billion tokens of Python code, making it more proficient in Python.
What is StarCoderPlus?
StarCoderPlus is another variant fine-tuned from StarCoderBase with 600 billion tokens of natural text from RefinedWeb and Wikipedia, improving its ability to understand and generate natural language alongside code.
Is StarCoder free to use?
Yes, StarCoder models are open-source and available for free on Hugging Face. They are released under an open license (Apache 2.0) allowing for research and commercial use.
What dataset was used to train StarCoder?
StarCoder models were trained on The Stack, a fully processed dataset of permissively licensed source code from over 80 programming languages, compiled by the BigCode project.
How many parameters does StarCoder have?
The collection includes models with 1 billion, 8 billion, 15 billion (StarCoderBase), and 16 billion (StarCoder and StarCoderPlus) parameters.