You are welcome to be skeptical and I will happily continue to use these tools for drafting code and writing every day. I usually provide documentation for the systems I want to use on input. Then I include source code of anything I'm hacking on. If none exists I build it up using "let's do this step by step" where the first step is outlining or architecture and subsequent steps fill out components. The same pattern works for long form prose. Having 100k tokens is almost enough. It means the model can attend to a ton of relevant information.
Also, this works well when the system gives you patches. I wouldn't say "rewrite this entire thing to make this one change" because if it's anything non trivial it will shift. Again I see the same patterns for code and text. Ask for small changes, contained updates. Guide the model to work in small increments, using abstraction to deal with high level broad topics.
You wouldn’t use it refactor because the implementation will drift, but you’d use it to generate a patch to apply to the code?
Hm.
Well, I don’t think that style is relevant or practical for large scale code generation as shown even in this ~600 line example.
…but, I’d love to see the details of how it could done, end to end, by someone who has done something like that successfully and is willing to prove it by sharing the prompts and outputs.
It's very hard to ask one of these systems to copy an entire piece of text and then change one or two things in it. They will often make small errors. So it's easier to ask for patches and apply them manually to edit the code or text that's being worked on. Happy to talk more how do I find you?
Also, this works well when the system gives you patches. I wouldn't say "rewrite this entire thing to make this one change" because if it's anything non trivial it will shift. Again I see the same patterns for code and text. Ask for small changes, contained updates. Guide the model to work in small increments, using abstraction to deal with high level broad topics.