Yeah, this is a common problem, I have been thinking about it a fair bit.
I ran into a related problem at scale though: deterministic linters catch syntax and obvious anti-patterns, but they miss the harder stuff. Logic bugs that pass all the rules. Spec drift. Security issues hiding in otherwise "clean" code.
So I built Caliper to layer AI review on top of deterministic checks. It reads your coding conventions (CLAUDE.md, .cursor/rules, whatever you use) and compiles them into checks that run after each AI turn—free and instant. Then optionally an AI layer evaluates changes against your project's actual policy. Catches what linters structurally can't.
Very different approach from Vibecheck but complementary. Actively looking for alpha testers if you want to try it — https://getcaliper.dev
yeah that makes sense. regex catches the surface-level stuff fast, but logic bugs and spec drift need something smarter. the AI layer on top of deterministic checks is a good approach.
vibecheck is intentionally the dumb fast pass, sounds like caliper handles the deeper analysis. will check it out.
I'm a little wary of this being regex-based; seems like it'd be too brittle for general use? But I really would need a tool like this to hook up to my vibecoding sessions.
fair point on regex being brittle for edge cases. it works well for the obvious patterns (empty catches, bare excepts, hardcoded secrets) but yeah it wont catch everything.
for vibe coding sessions thats actually the main use case i had in mind. run it after a generation pass to catch the low-hanging stuff before it gets committed. zero config so you can just pipe it in.
Yeah, this is a common problem, I have been thinking about it a fair bit.
I ran into a related problem at scale though: deterministic linters catch syntax and obvious anti-patterns, but they miss the harder stuff. Logic bugs that pass all the rules. Spec drift. Security issues hiding in otherwise "clean" code.
So I built Caliper to layer AI review on top of deterministic checks. It reads your coding conventions (CLAUDE.md, .cursor/rules, whatever you use) and compiles them into checks that run after each AI turn—free and instant. Then optionally an AI layer evaluates changes against your project's actual policy. Catches what linters structurally can't.
Very different approach from Vibecheck but complementary. Actively looking for alpha testers if you want to try it — https://getcaliper.dev
yeah that makes sense. regex catches the surface-level stuff fast, but logic bugs and spec drift need something smarter. the AI layer on top of deterministic checks is a good approach.
vibecheck is intentionally the dumb fast pass, sounds like caliper handles the deeper analysis. will check it out.
I'm a little wary of this being regex-based; seems like it'd be too brittle for general use? But I really would need a tool like this to hook up to my vibecoding sessions.
fair point on regex being brittle for edge cases. it works well for the obvious patterns (empty catches, bare excepts, hardcoded secrets) but yeah it wont catch everything.
for vibe coding sessions thats actually the main use case i had in mind. run it after a generation pass to catch the low-hanging stuff before it gets committed. zero config so you can just pipe it in.
[dead]