Skip to content
Alex Lee
Go back

Developing with AI right now: 11 field rules from a veteran

Someone asked me what good approaches there are for developing with AI right now. I said it mainly depends on how far you are using AI, and whether it is a new project or maintaining an old project. If this were really expanded on, it would be a very large amount of content. But briefly, from my personal experience, there are a few points:

  1. Requirements and detailed design can be done by AI, but the architecture must be reviewed and controlled by you personally.

  2. There must be four core constraints. The first is to require documents and conversations to speak in plain human language. The second is that any creation or rewriting of documents must not compress any information, and must not be oriented toward saving tokens. The third is that code must have comments: any function, class, interface, and important logic must have comments. These three points, to put it plainly, solve the problem of whether you can understand your documents and whether you can understand your code. The last one is to reuse existing code whenever you can, and if there is a mature open-source project or third-party library, use it. Do not reinvent the wheel. This is also very important. Otherwise you will find that the AI has written a large amount of code, which in practice is not as useful as one mature library.

  3. The first batch of documents written by AI must be read carefully. Even if you feel very impatient reading them, read them carefully anyway, and try to solve as many problems as possible at the document stage. Otherwise, as the AI writes more and more code and more and more documents, you will become less and less willing to read documents and less willing to read code. A lot of drift and scope expansion comes from this point.

  4. In the requirements, architecture, and design stages, the interview style is very useful. You can have the agent use an interview-style approach to write together with you, or you can have the agent and sub-agents ask and answer questions among themselves as a substitute for you. But every step must be appended to a list. Do not trust that its context compression mechanism is good enough.

  5. Unit tests and quality gates are needed, especially as the project gets bigger. But the rules must be set properly: one is the timing for writing and using unit tests and gates, and the other is not to run the full suite every time. This can decide the development speed and the waste of tokens, while also avoiding falling into the premature optimization swamp too early.

  6. Whether one agent writes your project or multiple sub-agents develop in parallel, one very obvious problem appears: duplicate code and duplicate logic. This is currently hard to avoid, even if you have written constraints like in my point 2. The Ponytail skill library has a certain effect, but you cannot use the original version. It has to be modified, otherwise the effect will not be very good. Looking across the whole project, there will still be a large amount of code that could have been merged or abstracted, but in fact exists in multiple written versions. And if it is used improperly, it will cause problems with your code quality.

  7. Review is a must, no matter the size of each task, and the review must be independent and strict. What does independent mean? It means the developer does not review its own work. Hand it to other sub-agents, and if conditions allow, even hand it to a different large model. Reviews generally need to go through multiple rounds, depending on complexity. For something simple, one agent reviews it. For something complex, multiple agents review from different angles, or are assigned different modules to review, and then the main agent is responsible for deduplication, cross-verification, and the final investigation. This step is very important, because many issues require cross-verification to find the root cause. Otherwise it becomes patch-style development over and over: the fix only solves the symptom, not the cause. You have it fix something, then review again and new problems appear. After multiple rounds you have fixed a big pile of things, but the root cause might actually be only one. Another point is to watch whether the review is over-reviewing. You can write prompts for this, or write a skill yourself, but in the end it still relies on your own ability. No prompt or skill can guarantee that over-reviewing will not happen. Finally, do not just say “review this for me” and be done with it. Point out the aspects to focus on. For example: review whether there are memory problems such as out-of-bounds access, overflow, or leaks; whether there is over-design; whether thread ownership and the use of async threads are correct, and so on.

  8. On the use of skills, I do not recommend enabling strongly constraining skill packs like Superpowers on the current top-tier models. These do have an effect, but the benefit is not necessarily that large. Personally, the built-in skills plus a few I have modified myself are enough. For a team, one or two engineered skills are enough. Any more than that becomes a burden. To put it in plain words, even if you use TDD, DDD and all that stuff, the output is still reviewed by AI in practice. The bigger the project, the harder human review becomes.

  9. The most important thing is that even if your project runs tens of thousands of gates and passes them all, it does not mean your project can go live. This itself is a false proposition. Because no matter how many gates there are, and no matter how complete the unit test coverage is, it does not mean the project itself will not have problems. At most, these guarantee that under limited, already verified preconditions, the project is fine. As for how to do it, every company has its own way. For example, on one small project of ours, the full gate suite has over 100,000 checks and takes 30 minutes to run once. But we still had another agent rewrite a set of attack programs based on the design documents, and added it as the final stage of CI.

  10. There are some other techniques that many people online have already talked about. For example, AGENTS.md and CLAUDE.md should contain no more than 100 lines of core instructions, and everything else should only serve as an index. Internal documents should be stored in a structured directory, with an entry index in each directory. Tests should have a ledger, and every test run must be appended to the ledger. Document management should not rely on numbering, but on the git repository. And so on. I will not say more about these; they are everywhere online.

  11. Oh, I almost forgot, there is also the selection of large models. I do not know domestic (Chinese) models very well, and I have basically never used them. Just from Claude and GPT, they are suitable for different fields. Claude’s analytical ability is very strong. It performs very well in the architecture and design stages of a project, much better than GPT. Its only real downside is that its autonomy is too strong. If you do not constrain it well, it will do a lot of unnecessary things for you. GPT is indeed much worse than Claude at solving complex problems, but it is more obedient when it comes to doing the work, so it depends on how you use them. I generally give complex problems to Claude, and simple, local code writing to GPT. Also, a lot of the time I use Claude to direct GPT to do the work. This works quite well. In Claude Code, it supervises GPT doing the specific tasks, and corrects and adjusts in time. As for model switching, I do not recommend switching back and forth the way people online suggest, like using the strongest model for analysis first, and a second-tier model for doing the work. A continuous task is best not switched back and forth, unless it is inherently a main task and sub-tasks. This is not a context problem, it is a problem of the large model’s own cache hits.

There is more that I either cannot recall at the moment, or that needs proper expansion to explain. In short, the above is a brief summary of some of my experience. If it were really expanded, this would be a very large body of content, not something a few points can make clear. You happened to ask, so I briefly shared my experience. Since this was off the cuff, some of it may be incomplete or incorrect. For reference only.


Discuss on X→

Comments live on X — replies, quotes and discussion happen there.