Neura Market
Interview: About deployment, evaluation, and testing of agents with Sully Omar, the CEO of Cognosys AI logo

Interview: About deployment, evaluation, and testing of agents with Sully Omar, the CEO of Cognosys AI

Free

No-code AI agent platform for daily tasks

FreeFree tier
Type
Open Source
Company
Cognosys AI
Visit Website GitHub (40 stars)

About Interview: About deployment, evaluation, and testing of agents with Sully Omar, the CEO of Cognosys AI

Cognosys AI is a closed-source, web-based platform that enables non-technical users to create and run AI agents without coding. It operates in iterative loops, breaking high-level objectives into smaller tasks, calling an LLM, and repeating until completion. The platform specializes in a research agent that conducts internet searches, synthesizes information, and provides cited sources, similar to Perplexity AI. Cognosys AI uses Cloud Run for scalable deployment and faces challenges in observability and monitoring at scale, currently relying on its own UI and tools like Sentry and Langsmith. The platform aims to simplify agent use for daily tasks, focusing on a few key use cases to maximize user value.

Key Features

No-code web-based UI for creating AI agents
Iterative task loops that break down objectives and call LLMs until completion
Specialized research agent that searches the internet, synthesizes info, and provides links
Cloud Run deployment with unified API via API Gateway
Built-in UI for tracing and monitoring agent runs
Supports choice between GPT-3.5 and GPT-4 models

Pros & Cons

Pros
  • No coding required, accessible to non-technical users
  • First positive experience leads to high user retention
  • Focuses on a few specialized use cases for better user value
  • Scalable deployment using Cloud Run with unified API
Cons
  • Closed-source platform with limited transparency
  • Users may struggle to choose appropriate LLM models (e.g., GPT-3.5 vs GPT-4)
  • Observability at scale remains a challenge despite using multiple tools
  • Serverless architectures are unsuitable due to long LLM call times

Best For

Research tasks: conducting internet research and summarizing findings with citationsDaily task automation for non-technical users via natural language objectivesIterative task decomposition for complex goals

FAQ

What is Cognosys AI?
Cognosys AI is a closed-source web-based UI for creating AI agents. It works in loops by breaking high-level tasks into smaller ones, calling an LLM, and iterating until completion. It is designed for non-technical users to complete daily tasks without coding.
How does Cognosys AI deploy agents?
Cognosys AI started with Vercel edge functions but switched to Cloud Run for the main system, using an API Gateway for unified access. Serverless functions are used only for minor tasks like user profile updates, as LLM calls are too long for serverless architectures.
What observability tools does Cognosys AI use?
Cognosys AI uses its own UI for tracing agent runs, along with Sentry, Google Cloud Platform, and is exploring Langsmith for agent-agnostic observability.
Who is the founder of Cognosys AI?
The founder of Cognosys AI is Sully Omar, as mentioned in an interview on the E2B blog.