I keep a text file called tools.md. It started as a wishlist. Now it's a graveyard.
I'm going to show it to you, because tracking every AI dev tool I tried for 90 days taught me something I didn't see coming. It wasn't about the money. The thing that actually bled me dry was the hunt itself — the hours spent just figuring out which tool to even try. One disclosure before the log: the specific counts and dollar figures below are a composite illustration of the pattern, not a line-by-line receipt.
Why I started the file
Back in March I got annoyed at myself. I'd open Cursor, hit a wall, see a tweet about some new agent thing, install it, forget I installed it, then trip over it three weeks later eating a background process. So I made a rule. Every AI dev tool I touch goes in the file: a date, a one-line reason, and a status. Alive or dead.
I figured I'd land around fifteen entries.
I hit thirty-one.
Thirty-one in ninety days. That's a new thing to evaluate every three days. Some were CLI agents. Some were MCP servers I bolted onto Claude Code. A handful of VS Code extensions, two "AI terminals," a few of those autonomous-PR bots. And an embarrassing number were just wrappers around a model I was already paying for.
Of the thirty-one, four are still alive.
The AI dev tool graveyard
Here's how the other twenty-seven died. The categories matter more than the names, but I'll name a few, because hiding behind "I'm not naming names" is how you know someone's making it up.
- Dead on arrival (9). Installed, opened once, never again. The demo video was a lie, or setup took longer than the problem it solved. One MCP server wanted four env vars and a Docker container to summarize a file. I already have a model that does that.
- Killed by a better one (7). Liked them, then found something cleaner two weeks later. The first tool wasn't bad. I just had no way of knowing the better one existed when I committed to the worse one.
- Died quietly (6). I actually used these. Then the maintainer ghosted, or an update broke it and nobody fixed it, or the free tier vanished overnight. I'd only notice when I went looking and it was gone. One of these was Smol Developer for me — great for a month, then it just fell out of my flow and I never reinstalled it.
- The hype tax (5). Tried these purely because my whole timeline was. None survived a real task. I genuinely can't remember what two of them did.
That's twenty-seven. And the names barely matter, because half of them have already pivoted or shut down. The list rots faster than I can keep it current. That's the whole point.
The cost nobody puts on an invoice
So what did the graveyard actually cost? I went back through the file with a coffee and tried to add it up honestly. Not a study. Just me and the log.
Money: somewhere around $180 in trials that auto-renewed before I cancelled, plus a couple of annual plans I bought in a moment of optimism. One of them charged me $47.12 the day after I'd decided I was done with it. Annoying. Recoverable.
Time is where it got ugly. The real cost per tool isn't the install. It's the read-the-docs, wire-it-up, test-it-on-a-real-task, decide-it's-not-worth-it loop. Call it fifteen to twenty-five minutes for the quick rejects. Easily two hours for the ones I gave a fair shot. Do that math across thirty-one and I counted maybe nine hours. A full workday, gone, just deciding what not to use.
And that's the part I can measure. The expensive part doesn't show up in the file at all.
I wasn't evaluating tools. I was evaluating rumors.
Every one of those evaluations started the same way. A tab. Then six tabs. A GitHub README that hadn't been touched since the last breaking change. A Reddit thread from eight months ago. Some reply guy swearing tool X is dead and tool Y is the future. A Product Hunt page with four hundred upvotes and zero signal on whether the thing still works today.
I was never really looking at the tool. I was looking at noise about the tool and trying to guess.
Here's the part that stung. Two of my "killed by a better one" deaths? The better tool had been out for months while I was grinding away on the worse one. I just had no trustworthy place to look that would've told me, in March, what the real ranking was on the day I needed it. So I paid the full discovery cost twice for the same job.
Is the tool moving fast, or is the conversation about the tool moving fast? Most days I couldn't tell. By the time I'd verified something was good, the verdict had aged out from under me.
So I asked around. The dozen-odd builders I trust most, people shipping real things — how do you pick? Not one had a system. The most common answer was "I saw it on my timeline." The second was "someone I respect mentioned it." We're senior people making infrastructure decisions on vibes and recency, we all know it, and we keep doing it because nothing better is sitting in front of us.
What I actually changed
I didn't fix discovery. I can't. One person can't out-research a category that ships a new agent every Tuesday.
What I fixed was the funnel. New rule: a tool gets fifteen minutes and one real task. If it can't beat what I already use on that one task, it's dead, logged, done. No "I'll circle back." I never circled back. The file proves it.
I also stopped trusting any single source. Timeline, README, one breathless blog post — doesn't matter. If I can't find the same verdict in two or three places where someone actually used the thing recently, I treat it as noise.
It helps. It's also just a bandage. Because the wound is still there: nobody has a ranked, current, trustworthy map of this space. So every one of us rebuilds the same map, alone, badly, and throws it out when it rots a month later.
Thirty-one tried. Four survived. And the four that lived weren't the ones with the best demos. They were the ones I happened to find on a good day, before the noise buried something better.
That's not a tooling problem. That's a map problem. And right now none of us have the map.