I'd happily demonstrate this kind of workflow on my day job if not for company trade-secrets.
That's as legacy as it gets, 20+ year old code base with several "strata" of different technologies and approaches.
Claude Opus handily navigates around it, and produces working bug fixes with minimal guidance.
I'm not going to claim it's 20x or 50x yet, there's still navigation and babysitting involved, but it's definitely capable of working on complex problems, I just don't trust it to run it in YOLO mode.
The key thing is, you need domain knowledge. You need to know where to correct it, and which direction to point it in.
It's not magic, and it will have bad ideas. The key picking out the good ideas from the bad.
Hi eterm, this is very relevant to me as I'm building a self-hosted open-source tool for legacy code comprehension (AI/ML final project).
You mentioned "navigation and babysitting", could you share what that looks like in practice? Do you have to spend time reconstructing context or correcting Claude's misunderstandings? Do you still need to interrupt colleagues for some tacit knowledge, or has that changed?
I don't know. There's lots of options. At the extreme ends it would be interesting to see these agents work on something like boost, or metamath/set.mm, to choose deliberately obtuse candidates. Perhaps a web browser.
I'd happily demonstrate this kind of workflow on my day job if not for company trade-secrets.
That's as legacy as it gets, 20+ year old code base with several "strata" of different technologies and approaches.
Claude Opus handily navigates around it, and produces working bug fixes with minimal guidance.
I'm not going to claim it's 20x or 50x yet, there's still navigation and babysitting involved, but it's definitely capable of working on complex problems, I just don't trust it to run it in YOLO mode.
The key thing is, you need domain knowledge. You need to know where to correct it, and which direction to point it in.
It's not magic, and it will have bad ideas. The key picking out the good ideas from the bad.