Hi Everyone,
AI comes up in almost every conversation we have with clients these days. How are you using it? What’s working? What isn’t? Is it actually changing how software gets built, or is it mostly hype?
We’ve been building AI-assisted software for our clients — and for ourselves — for the better part of a year now. We’ve had moments that felt like magic and moments that felt like debt. This newsletter is our attempt to share what we’re actually learning, past the marketing and the benchmarks.
First issue. Let’s get into it.
The question we stopped asking
Ten months ago, our team started a new internal project — a modest reporting tool that needed to pull data from a few sources, apply some business logic, and produce clean output. Nothing exotic.
With AI assistance, features that used to take six or seven weeks were shipping in three. The velocity was real. The output was clean. Tests were passing. We were delighted.
Then, about four months in, a routine enhancement came in. Add a new data source, adjust a calculation, update the output format. The kind of ticket any senior engineer should be able to close in a day.
It took two weeks.
Not because the problem was hard. Because no one on the team understood the AI-generated code well enough to modify it confidently. The logic was correct — it had always been correct — but it was a black box. The engineers who had “built” it had guided its creation without fully internalizing it. When the time came to change it, they were starting from scratch.
A new definition of “done”
We’ve updated what “done” means on our projects.
A feature is done when:
- It works as specified
- It’s tested
- Someone on the team can explain how it works
- That same person is confident they can change it
The first two criteria are the ones we’ve always had. The last two are new — and they’ve changed how we use AI tools in practice.
We no longer let AI generate logic that the team hasn’t reviewed line by line. We use AI to accelerate the work, not to replace the understanding. The difference sounds small. In practice, it changes almost every workflow.
Velocity without understanding isn’t speed. It’s debt. The bill always comes due the first time you need to change something.
The question worth asking
Before any AI-assisted feature ships, we now ask one question in review:
Six months from now, when this feature needs to change, who on your team will understand how it works?
If no one can answer that confidently, the feature isn’t done — regardless of what the tests say.
It’s a simple question. It tends to surface the right conversations.
Coming in the next issues
- How we structure AI-assisted code review — what we look for, what we flag, and why “it passed the tests” is never sufficient
- The prompting patterns that actually hold up in production — and a few that looked great in demos and fell apart in code
- Building RAG pipelines for regulated industries — the compliance constraints that change everything about how you architect retrieval
Naveen Sathiya Moorthi
Senior Director, Cloud & AI Platforms
AnaData Consulting Inc.