AcruxCore

AcruxCore is an open-source LLMOps platform that provides a comprehensive solution for engineering teams developing features and agents based on large language models. It allows users to manage prompts in a versioned registry, enabling easy changes and rollbacks without the need to redeploy applications. The platform includes an OpenAI-compatible gateway that interfaces with multiple model providers, offering features such as budget management, rate limiting, and caching. Each request is meticulously traced, recording essential metrics like tokens used, cost, and latency, which helps teams monitor performance and optimize their LLM interactions.

The platform also features evaluation capabilities, allowing teams to score different prompt versions against datasets using an LLM judge. This functionality is crucial for ensuring high-quality outputs and continuous improvement of the prompts used. AcruxCore supports SDKs for both Python and TypeScript, making it accessible for developers across different programming environments. Users can choose to self-host the platform using Docker or utilize the hosted version available during the beta phase, which appears to be free to use.