Phi2-2.7B

Phi-2 is a 2.7 billion parameter Transformer language model developed by Microsoft. It was trained using a combination of synthetic NLP texts and carefully filtered websites, building upon the data sources of Phi-1.5. Despite its relatively small size, Phi-2 achieves state-of-the-art performance among models under 13 billion parameters on benchmarks testing common sense, language understanding, and logical reasoning. The model is open-source under the MIT license and is designed to facilitate research into safety challenges such as toxicity reduction, bias understanding, and controllability. It has not been fine-tuned with reinforcement learning from human feedback. Phi-2 is integrated in the Hugging Face Transformers library (version 4.37.0 or higher) and is best used with prompts in QA, chat, or code formats. Known issue: attention overflow with FP16, which can be mitigated by adjusting autocast settings.