Whereas when you’re doing this in a private repo, a model has never seen that repo. And now the iteration loop for finding the outcome is much longer, right? If you have a semantic understanding of your entire private repo, you can ask a question, and you get to those outcomes much more quickly.

Ars: People are concerned about token efficiency. Does this semantic approach help with that?

Perneti: We’ve certainly seen it in certain situations. In fact, we published a blog post—we ran Terminal-Bench with Claude Code and Augment Code, same model. And we completed at similar accuracy, but we were 33 percent more efficient than Claude Code. So I do see this showing up in terms of making better use of tokens because you’re not spending as much time on the exploration side.

Ars: Anthropic told me that Claude Code does not see measurable eval gains from including more semantic code-navigation tools. But then I look at your blog, and I see you posting benchmarks and other claims that, actually, this really helps. Are you guys measuring different things when you say this, or what are their evals missing?

Perneti: Great questions. I don’t know what specific retrieval engine they used, and I think the other mistake that often people conflate is, not all retrieval systems are equal just like not all databases are equal, right? And by that, what I mean is the models and the system that are working together, which is the context engine, makes a big difference in terms of the quality of outcomes as well.

So, for example, at Augment, we spent about 18 months at the beginning of the company being founded in 2022—this is pre-ChatGPT—researching retrieval and embedding models predominantly for large code bases. So there’s a lot of research that has gone into, when you’re trying to achieve a particular outcome, what are the right pieces of code to get in this embedding space? All of that is encoded into our retrieval models.

And doing that in a very, very fast way is the system that we built around it. So when somebody says, ‘Hey, I tried Claude Code with a RAG implementation, but I’m not seeing benefits,’ that’s because the implementation and the context engine are very, very different, if that makes sense.

https://arstechnica.com/ai/2026/07/beyond-grep-the-case-for-a-context-rich-ai-coding-harness/