PromptSource

Free

About Toolkit for creating, sharing and using natural language prompts. ![GitHub Repo stars](https://img.shields.io/github/stars/bigscience-workshop/promptsource?style=social)

FreeFree tier

Inputs: textOutputs: text

Type

Open Source

Visit Website GitHub (40 stars)

About PromptSource

PromptSource is an open-source toolkit developed by BigScience Workshop for creating, sharing, and using natural language prompts. It provides a structured way to define prompts as Jinja templates, mapping dataset examples to natural language inputs and target outputs. The project includes P3 (Public Pool of Prompts), a growing collection of over 2,000 English prompts for more than 170 English datasets as of January 2022. PromptSource integrates seamlessly with the Hugging Face Datasets library, allowing users to load and apply prompts to any dataset with a simple API. It was created to support research on zero-shot generalization in large language models, as seen in models like T0 and FLAN. The repository on GitHub fosters community contributions and sharing of prompts.

Key Features

Create, share, and use natural language prompts via a simple API

Includes P3: over 2,000 English prompts for 170+ English datasets

Prompts defined as Jinja templates in standalone structured files

Seamless integration with Hugging Face Datasets library

Supports zero-shot generalization research for large language models

Open-source with active community on GitHub (bigscience-workshop/promptsource)

Pros & Cons

Pros

Large collection of pre-built prompts covering many datasets
Simple Python API for loading and applying prompts
Open-source and community-driven with active contributions
Direct integration with Hugging Face Datasets
Facilitates reproducible prompt-based research
Lightweight installation via pip or local setup

Cons

As of early 2022, limited to English prompts and datasets
Requires Python 3.7 for stability when using the prompt creation interface
May require familiarity with Jinja templating language to create new prompts
Documentation could be more extensive (API docs available but sparse)
Not actively maintained after initial release (last commit may be old)

Best For

Evaluating zero-shot and few-shot performance of large language modelsCreating and sharing prompts for new NLP tasksFine-tuning models like T0 with multitask prompted datasetsStandardizing prompt formats across research teamsBuilding prompt-based applications with Hugging Face datasets

FAQ

What is PromptSource?

PromptSource is a toolkit for creating, sharing, and using natural language prompts. It contains a growing collection of prompts (P3) and provides tools to apply them to datasets from Hugging Face Datasets.

How do I install PromptSource?

You can install promptsource via pip by running 'pip install promptsource'. For creating new prompts, clone the repository and run 'pip install -e .' from the root directory. A Python 3.7 environment is recommended for stability.

How do I use prompts with Hugging Face datasets?

Load a dataset using datasets.load_dataset, then load prompts for that dataset using promptsource.templates.DatasetTemplates('dataset_name'). You can then apply a selected prompt to any example from the dataset.