I Rebuilt My JavaScript Database From Scratch for the AI Agent Era

--- title: I Rebuilt My JavaScript Database From Scratch for the AI Agent Era published: true description: Why I rewrote Skalex v4 from the ground up with vector search, agent memory, and a one-line MCP server built into the core. tags: javascript, ai, database, webdev cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7pratpyuex5ih1y1ubu4.png --- Three years ago, I built Skalex - a simple, zero-dependency in-memory document database for JavaScript. It did what it said on the tin: store documents, query them, persist them to disk. People used it. I was happy. Then everything changed. ## The moment I knew I had to rewrite it AI agents became real. Not just chatbots - actual agents that remember things, reason about data, and take actions. And as I started building with them, I kept running into the same wall: The database layer wasn't designed for this. Every time I wanted an agent to recall something from a previous session, I had to bolt on a vector database. Every time I wanted natural language queries, I had to wire up an external service. Every time I wanted to expose data to Claude Desktop or Cursor via MCP, I had to build plumbing from scratch. I was spending more time on the infrastructure than on the actual agent logic. So I asked myself: what would a database look like if it was designed for AI agents from day one? That question became Skalex v4. ## The constraints I refused to compromise on Before writing a single line, I set three rules for myself: 1. Zero dependencies - no supply chain risk, no bloat, no `node_modules` hell 2. One package - everything the AI stack needs, no separate installs 3. Every JavaScript runtime - Node.js, Bun, Deno, browsers, edge workers These constraints made everything harder. And they made the result much better. ## What I built ### Vector search that just works The most requested feature from v3 users was semantic search. I wanted it to feel native, not bolted on. ```js const db = new Skalex({ ai: { provider: "openai", apiKey: process.env.OPENAI_API_KEY, embeddingModel: "text-embedding-3-small" } }); await db.connect(); const notes = db.createCollection("notes", { vectorField: "content" }); // Insert with automatic embedding generation await notes.insert({ content: "The cat sat on the mat" }); // Search semantically const results = await notes.search("feline on furniture", { limit: 5 }); ``` No separate vector database. No separate embedding pipeline. One package, one API. Want to use local models instead? Swap the adapter: ```js const db = new Skalex({ ai: { provider: "ollama", embeddingModel: "nomic-embed-text" } }); ``` Fully offline. Zero API costs. ### Agent memory that survives restarts The biggest pain point I kept seeing in AI agent code was memory that evaporated when the process died. Session memory is easy. Cross-session memory is the hard part. Skalex solves this with a dedicated memory API backed by semantic embeddings and persistent storage: ```js const memory = db.useMemory("agent-1"); // Remember something await memory.remember("User prefers dark mode and concise answers"); await memory.remember("User is building a SaaS product in Next.js"); // Recall semantically across sessions const context = await memory.recall("user preferences", { limit: 3 }); // Compress old memories to save space await memory.compress(); ``` When the process restarts, `db.connect()` reloads everything from the storage adapter. The agent picks up exactly where it left off. ### Natural language queries `db.ask()` translates plain English into structured filters via any LLM: ```js const results = await db.ask( "find all users who signed up this month and haven't logged in" ); ``` Works with OpenAI, Anthropic, and Ollama. The LLM generates the filter, and Skalex executes it. No prompt engineering required on your end. ### A one-line MCP server This is the feature I'm most excited about. Model Context Protocol lets you expose your database as a tool to Claude Desktop, Cursor, and any MCP client: ```js const db = new Skalex({ path: "./data" }); await db.connect(); db.mcp().listen(); ``` Three lines. Claude now has full read/write access to your database - find, insert, update, delete, search, ask questions in plain English. Add it to your Claude Desktop config and your AI assistant has a real persistent memory layer. ## The hardest part The hardest part wasn't the AI features. It was keeping everything in a single zero-dependency package while supporting six different runtimes. Node.js, Bun, and Deno all handle crypto, file I/O, and module resolution differently. Browsers don't have a filesystem. Edge workers have memory constraints. Every platform is a special case. The solution was pluggable storage adapters: ```js // Node.js const db = new Skalex({ adapter: new FsAdapter("./data") }); // Browser const db = new Skalex({ adapter: new LocalStorageAdapter() }); // Cloudflare Workers const db = new Skalex({ adapter: new D1Adapter(env.DB) }); // Bun const db = new Skalex({ adapter: new BunSQLiteAdapter("./db.sqlite") }); ``` Same API. Every platform. Zero code changes in your application layer. The test suite ended up at 787 tests across Node.js, Bun, Deno, Chrome ESM, Chrome UMD, and BunSQLite. Every commit runs all of them. ## What I learned from the rewrite **Constraints are a feature.** Zero dependencies forced me to implement crypto, compression, and vector math from scratch using only built-in APIs. The result is leaner and more auditable than any dependency chain. **The AI stack needs a database that understands it.** Tacking vector search onto a traditional document store feels wrong because it is wrong. When memory, search, and queries are all first-class citizens, the agent code becomes dramatically simpler. **Local-first is underrated.** Ollama support means you can run the entire AI stack - embeddings, LLM, database - on your laptop with no API keys, no costs, no data leaving your machine. For prototyping and privacy-sensitive workloads, that's a huge deal. ## What's next v4 is in alpha today. The API is largely stable, but may shift before the stable release. What's on the roadmap: - Hybrid BM25 + vector search with Reciprocal Rank Fusion - CRDT real-time collaboration - SQLite WASM adapter for browser-native persistence - Graph traversal queries - Framework adapters for React, Vue, Svelte, Solid, and Eleva If you're building AI agents, CLI tools, desktop apps, or edge workers where the dataset fits in memory - give it a try. ```shell npm install skalex@alpha ``` Feedback, bug reports, and contributions are what make the stable release good. - GitHub: https://github.com/TarekRaafat/skalex - Docs: https://tarekraafat.github.io/skalex