I Scanned 100 AI Codebases - Here's What I Found

I've been building VibeCheck for the past few months - it's a security scanner specifically for...

title: "I Scanned 100 AI Codebases - Here's What I Found" published: true tags: vibecoding, ai, security, webdev series: "Vibe Coding Lessons"

I've been building VibeCheck for the past few months - it's a security scanner specifically for AI-generated code. And after scanning over a hundred real codebases that people built with Cursor, Copilot, Claude, and various other AI tools, I have thoughts.

Not the "AI is dangerous" hot take. Something more specific than that.

The pattern that kept showing up

Almost every codebase had the same category of issue. Not SQL injection or XSS or anything that would show up in a classic OWASP checklist. The dominant problem was what I started calling trust misconfigurations - places where the code just... assumed everything was fine.

Open CORS policies. Service accounts with admin permissions because that was the fastest path to getting it working. API keys hardcoded in config files that weren't in .gitignore. Input that got passed straight into shell commands with no sanitization.

None of it was malicious. The AI wasn't trying to introduce vulnerabilities. It was just optimizing for "make it work" and had zero weight on "make it survivable in production."

The thing that surprised me most

I expected the biggest problems in the actual logic - like the AI misunderstanding authentication flows or getting crypto wrong. That exists too, but it's not the main thing.

The main thing is environmental. All these tiny decisions about permissions and access and trust that a senior dev would make automatically, almost subconsciously, because they've been burned before - the AI just doesn't make those decisions. It picks the path of least resistance every time.

One project had a DB connection string with full admin creds, no connection pooling limits, and a query that accepted raw user input. Technically functional. Completely fine for local dev. The kind of thing that gets quietly exploited six months after launch.

What actually helps

Scanning after the fact (what we do with VibeCheck) catches the obvious stuff. But the real fix is earlier in the loop.

The projects that had the least issues were the ones where the developer was actually paying attention during generation - not just accepting output wholesale but reading it, asking "wait, why does this need admin access?" That friction. Even a little bit of it makes a big difference.

Some people are building this into their prompts - explicitly telling the AI to follow least-privilege principles, to validate all inputs, to not hardcode credentials. Works okay. Feels like workarounds.

The better solution is probably tooling that runs in the background during vibe coding sessions and flags stuff in real time. Not a code review gate. Just... something watching.

The uncomfortable part

A lot of these codebases were shipped. Some had real users. A few were running in production environments with actual credentials and real data.

The developers weren't careless people. Most of them were genuinely excited about what they'd built - and most of what they built was genuinely cool. The security stuff just wasn't on their radar because it never came up during development. Nothing broke. Tests passed. It worked on their machine.

I keep thinking about that gap. Between "works fine in dev" and "safe to run with real users." AI coding tools are really good at closing the first gap - getting something functional fast. Nobody's really solved the second one yet.

That's the problem I'm trying to figure out. Not sure I have it yet. But the 100 codebases were pretty clarifying.

If you're using AI to build things and want to know what the scanner finds in your repo, VibeCheck is live. Free tier, no credit card. Takes about 2 minutes.