Non finito

Non finito is a platform designed for evaluating multimodal models, with a focus on making it easy to run and share evaluations. It aims to provide tools specifically tailored for multimodal models, which are often overlooked by tools primarily focused on language models (LLMs). The platform emphasizes easy comparison of models and public sharing of evaluations.

How to Use

Users can sign up to create evaluations, compare different models, view public evaluations, and access examples to understand the platform's capabilities. The platform allows input and output comparisons for various tasks.

Non finito's

Key Features

• Multimodal model evaluation
• Model comparison
• Public sharing of evaluations
• Example evaluations for various tasks

Use Cases

• Evaluating entity tracking in language models
• Evaluating logical reasoning capabilities
• Real-world question answering
• Visual deductive reasoning
• Evaluating visual models by counting cards
• Graph comprehension and policy recommendation