Documentation is a skill. Like any skill it takes practice.
I have moved to a FAANG and their documentation is downright fucking awful. _everyone_ just writes code, with lots of "clever" bits, and doesn't bother to fucking comment.
Not only that because people don't even _comment_ their code, the wiki is a total shit show. Want to know how to use a Queue? tribal knowledge. want to know which DB is best for x? tribal knowledge. Want to know how to create a new endpoint? tribal knowledge.
Worse still, we had a class during induction where some ponytailed "10x" said "If you are messaging me asking questions, I can't help other people" I didn't have the bollocks at the time to ask why his documentation sucked arse. There seemed to be a weird pride in the fact that people needed to message him to figure his shit code.
The moral of the story is this:
fuck off with your clever code, spend that effortyou put into learning new languages, or trying a new techniques and put it into developing your writing skills. It takes empathy, organisation and skill. It'll make you a better programmer and a better person.
> Documentation is a skill. Like any skill it takes practice.
The type of documentation you described - code comments and wiki entries - they don't need practice. They only need care.
If you know how to write a function that takes Foo and transforms it into Bar under conditions X and Y, you also know how to just append this above it:
//! Transform Foo into Bar
//!
//! Foo must be an X-ing Quux adhering to Y.
//! Foo is not modified.
//! Returns a Bar that is Z.
No knowledge or skill needed. And will make everyone's day nicer.
I'm not sure what to do about lack of care in a team/company, other than trying to promote it by example.
These seem too me mostly useless comments. I need comments for things that are not apparently visible. Transforms foo into bar is either clear from params and return types, or should be from name. Likewise, in most cases it is easy to see whether argument changes.
The parts that are difficult to see and difficult to understand are the one that need documentation. They are the ones that happen to be difficult to explain.
What's missing IMO from those example comments (and most automatic documentation) is the "why". I can almost always look at the code to figure out that a function takes a Foo and returns a Bar. What's almost never obvious from the code itself is "why would I want to convert a Foo to a Bar" or "under what circumstances should I use this function instead of something else".
both copy a string from src to dst with a limit on the number of bytes copied. Good comments for would not simply explain what they do and what the arguments/returns represent, but under what circumstances to prefer each variant.
Going with the "6 W's":
"How" (does this code work) and "What" (does this code do) can mostly be explained by the code itself, although comments should be used to clarify anything non-obvious, for instance if you're depending on a side effect or something.
"Where" (should you use this code) and "why" (should you use this code) need to be covered by comments. It is extremely hard to figure those out from the code alone.
"Who" (wrote it) and "when" (was it written) should be in the version control system metadata. Putting those in comments is a good way to ensure the comments are out-of-date/wrong in any long-lived codebase.
Note how most of the text there is focused on the "How" and "What", because both functions have a bunch of requirements for their arguments that are not expressed in their respective signatures.
Some languages have better tools for expressing these requirements in code. But when they can't be expressed in a way that can be enforced by the compiler, IMO they absolutely need to be mentioned in an interface-level comment (i.e. above function signature), to give users a fighting chance of avoiding bugs.
Also worth noting that the particular constraints around strncpy() and strlcpy() will not be obvious in the implementation either - the programmer trying to make use of these functions would have to study the implementation to notice potential issues. A well-placed comment can save them an expensive context switch here.
It depends on the language. Perhaps Haskell will not need such comment at all.
In C++, at the very least the preconditions X and Y, as well as the invariant Z, will usually not be apparent from the function signature. Whether the argument is modified? That's usually clear from the use of const... except when it isn't, e.g. because the function is a universal-reference template, or some C compat thing that must use bare pointers because reasons.
In JavaScript, you won't even know Foo, Bar and Quux, unless someone puts it in the function name.
The primary benefit of such comments is to encode enough information that isn't obvious from the signature, that you don't need to read the actual implementation. It's particularly useful if you're using an editor or IDE that can pull signature comments and show them during auto-completion - it saves you from constantly jumping into other places in the codebase, just to verify if you're picking the correct function for the task.
> The type of documentation you described - code comments and wiki entries - they don't need practice.
Yes, they do.
> They only need care.
They need that, too, but “care” is what gets you to apply what you know of how to do it rather than neglecting it, but practice (and interactive practice with feedback, specifically) is how you develop the skill to make useful comments, and ideally only useful comments.
> If you know how to write a function that takes Foo and transforms it into Bar under conditions X and Y, you also know how to just append this above it:
Well, mechanically probably you know how to. But absent relevant practice you might not know that (assuming, for the sake of argument, that this is correct in context — and in many cases, IMV, it wouldn’t be as much of that seems to reiterate information that is contained in type signatures and thus violate the principle of single source of truth if placed in comments) you should prepend comments with that content, rather than none or some other content foe the function.
I just make sure I supply a detailed what, why and how comment with each commit and then copy and then just export that for documentation. Also spending time putting thought into a standard system for properly naming variables in your app goes a long way. A lot of people like to abbreviate things just to avoid typing. Modern IDEs allow autocomplete so there is little excuse for this. The only time that is acceptable is in a very small very self evident class, method or function that does something very simple and atomic. I try to avoid this as well since I like to be able to do named search and replaces later if needed. Too simple of a variable name will lead to collisions with sed/awk ect
I’m in games so the visual element often forces this, but I’ve moved our team to making a lot of video content, both for PRs and for much of our “documentation”.
I’ve always preferred reading, and the trend to everything being on YouTube has driven me nuts, but I’m a convert to this method for a few reasons.
First, it’s fast. I can sit down and make a deep dive video in 30 minutes and not have to sit around writing and polishing documentation for hours. When the code inevitably changes, I can throw away the old video and spend another thirty minutes making a new one.
Second, I can demonstrate what the code does while offering instruction around it. If I likened it to anything, it’s like having a series of lectures to your code base rather than a textbook.
And lastly, it’s proven GREAT for on boarding a new engineer in the more complex aspects of the project. Because they can go and view thousands of PRs, 90% of which have video demonstrations or explanations, they can do a lot of self directed learning and not require nearly as much over the shoulder time with other engineers. When your team is spread across 12 hours of time zones, this is very useful.
This method isn’t a panacea, and we still keep written documentation when we need to provide a concise set of “how to’s” to other teams. But for the dev team, it’s been great.
With respect, you save time making 30 minute videos, but everyone else then wastes time watching your video looking for info.
I prefer it if people just write it down so I can ctrl-f or find it in a web search and get what I need instead of sitting through your videos.
For general "welcome to Team X!" onboarding or training though I agree that videos have benefits. But for day to day knowledge and docs I couldn't think of anything worse, although it seems to work for you and it is popular on YouTube for some things (e.g. Unity game dev content seems be be pretty much 100% video based - if I just need to know how to set up something in the UI like character rigging or wheel physics etc, I often have to sit through 30min videos to find the 15 seconds where they show what buttons to click etc - if it was on a web page I could just skip right to it)
I just love YouTube's automatic transcripts for this. Open, copy it into a text document so you can read it in a bigger window, then ctrl-f. You can skim it quickly to find the key point you're after. Sometimes things will be transcribed incorrectly but the surrounding context helps.
Videos are also better at transfering tacit knowledge. Stuff like little habits and workarounds people have that are important but which they don't realize need to be communicated.
For example, creating a virtual environment for a python project instead of installing it globally. This is a crucial step for things to work well, but it's so common it might be left unstated.
Something the author didn't dive into - which is very important - is the audience of said documentation. In my 20+ years documentation is pretty much essential, but there is definitely a skill in documenting for a particular audience:
* Internal, non-technical users; what does the code do, why was it written, who currently owns it, when should it be run, how to run it, and what to do in case of failure. The more accessible, brief, and direct the better.
* Technical users; this is your README that usually doesn't need to be updated all that often. What is this code for? A quickstart for install/loading, dependencies, and the bare-bones set of functions, arguments, etc.
* Fellow implementers; the above README, but likely with a few additional notes of how the code is (high-level) broken up, and where to go for the most common things. This is where documented code is really good as well. I usually find the best code comments have always been A) NB: here's a drawn-out explanation of why the code does this, it's for a reason, do not question the code first unless you understand this and/or the above reason has changed, and B) in the future, you may want to do X, in which case you'll need to do J, K, and L and update M. Comments like those have saved me (from myself, even) so much grief and time.
* 3rd party users; this is where real documentation gets to be a monumental PITA and any company worth its salt will hire a technical writer or two. This is a very different skill than what's required for any of the above.
The biggest concern I see brought up again and again with documentation is that it becomes out of date quickly as soon as it's finished being written. This needn't be true. Internal documentation is just enough info for whoever is using it to know what it is, how/when to use it, what to do if it fails, and who to go to for more information or help.
Honestly though, the biggest problem with documentation has always been where it lives and how it's edited. Is it all repo READMEs and markdown? Are they google docs? A wiki/confluence? This is always where the breakdown happens. Programmers are comfortable with just being pointed to a repo and text files, while non-programmers want WYSIWYG docs somewhere central, which is the last place most programmers even think about. Wikis/confluence attempts to be a compromise that (IMO) no one likes. This - like task tracking - is a unicorn of software development.
> Programmers are comfortable with just being pointed to a repo and text files, while non-programmers want WYSIWYG docs somewhere central, which is the last place most programmers even think about.
Yup, I think if you are forced to use central WYSIWYG documentation, it makes sense to link to it from the readme in the repo. There's nothing worse than having documentation but being unable to find it or being unaware of it - it may as well not exist, and was a waste of time writing ;)
There are a remarkable number of commenters saying no, documentation isn't hard actually. One question for those commenters:
How do you test your documentation? Properly test it, ensuring that it answers many kinds of questions for those who are new to the topic, and who haven't already been working on this project for months?
If you're not testing it, then how can you consider it done? Would you do the same with your software?
The vast majority of documentation that I see these days is bad in multiple ways. Most common is lack of coverage for important topics or cases, and I'm not even talking about things like "why this product is built this way".
I'm talking about stuff like:
- An API providing all kinds of methods for manipulating "Fribblers", and when you look at the Fribbler class it just says "This is the primary class representing Fribblers." No idea what this means or how it fits into the wider API? Good luck!
- Parameters or properties appearing in lists without any explanation of what they mean
- ... and that's assuming those parameters are even listed at all, which (often) they're not
- ... especially when those parameters appear in other parts of the documentation. "This method will work asynchronously unless you've passed the "borgle" attribute to the Fribbler constructor." There's a borgle attribute?!
- And this is all just for basic usage of a product. Want to contribute, or run tests, or anything else? Nothing.
Documentation needs usability tests, same as anything else which is primarily designed to be used by humans who may not have seen it before. But it's hard enough getting proper UX testing for the product, let alone its documentation.
There's a common refrain that internal documentation never gets updated and so it's better to just look at the code. IMHO outdated docs are still better than no docs, you just have to approach them with an appropriate level of skepticism and archaeologist mindset. At the base level of course the code is the ultimate source of truth, and being able to navigate code history quickly (via something like the Fugitive vim plugin) is incredibly valuable, but higher levels of documentation such as code comments, commit messages, tech specs, product specs, and user documentation can give richer clues into the mindset of the team at the time. As such, I believe the single highest ROI thing you can do is to choose tools that automatically maintains history/edit timestamps for all docs (git obviously does this, but Google Docs and Quip also do quite well here).
I think the big difference is what type of information you're looking for. The code is the final arbiter on what the program does, but gives incredibly little information about why it is designed that way.
* Can be learned from the code: Function A accepts a C-style pointer to a data array.
* Can be inferred from the code: Function A is called in an inner loop, and so it probably accepts that data array to avoid doing any memory allocation.
* Cannot be known from the code: Even though the data array is initialized to 0 in the current code, function A should not assume that will always be the case. That data array is intended to be used as persistent storage in a future version.
The first one is kind of pointless to document beyond what doxygen already gives you. The second is useful, but not necessary. The third is absolutely essential, because nothing in the current code can possibly tell you about the future intents for changes in the code.
* Cannot be known from the code: The intent to use the array as persistent storage in the future is driven by the ongoing work around feature F.
* Cannot be known from the code: Feature F morphed into feature Q, and the function A doesn't really need that array anymore. Since nobody could tell why it was there in the first place, nobody removed it afterwards.
The next time someone has to make changes to function A, they'll be thankful for a comment or commit message that explains the first point above - as it'll let them complete the picture and realize it's no longer needed, so they don't have to worry whether their change is impacting anything else in the program through (mis)use of that array, but they can instead just go ahead and delete it.
> you just have to approach them with an appropriate level of skepticism and archaeologist mindset
The problem is most people don't have this mindset. They expect documentation to be right, and when it isn't, they become frustrated and learn to avoid documentation.
The single biggest ROI I've seen on getting documentation written is to provide a template for developers to fill out.
Blank wiki pages are incredibly intimidating and developers can't always anticipate what people will want. Having a "madlib" style outline with things like:
- Where does this app run?
- How do you start it?
- Where are the logs?
- How do you common items X,Y,Z?
Takes your odds of documentation being written from near zero (in my experience) to at least 60%.
This. Some of the best documentation is a simple "getting started" guide that explains: How to build (any dependencies?), how to hook a debugger or access debug logs and some pointers to where to start when looking at code (example: set a breakpoint here and trigger this action, you'll end-up in X component and should be able to figure out what's going on from there).
What if we don’t incentivize and it’s important? Further, what if we can improve it without incentivizing? My teams code is very well documented because we encourage documentation during code review. No one gets a bonus for this, but it doesn’t matter because we care about our softwares quality.
Put another way, good software bad doc is a lot harder and more costly than good software good doc.
That's just a negative incentive where you get punished if you don't write documentation.
It works. But it's hard to be consistent.
Let's say someone build a great feature with a lot of traction (e.g. money) but no doc.
Will you punish the team or celebrate the success?
"Sorry, your project make 20m, higher than any other projects in the company, but your doc is bad, so... you get below expectation rating this time".
There are tons of successful software with very little doc. So, my imaginary situation is very plausible. On the other hand, bad software with good doc rarely succeeds. But tbf bad software means unsuccessful software....
Writing documentation is painful, because it reveals inconsistencies and fuzzy areas in your design. If you're not paying attention, you may associate that pain with the act of writing documentation. In reality, the problem lies with your designs.
This. As a technical writer, this is one of the biggest pain points of the job. While writing the documentation, valid questions will come up regarding design decisions, but no one has the answers.
Most teams nowadays are agile teams, so there was never any real thought to why something was done a particular way other than it was the easiest way to complete the next ticket.
Touching an interesting point.. maybe doc is just an ad-hoc rewrite in a different language. Maybe writing your program in different languages can refine the program to the point everything is clear.
Yep. A different language, a different viewpoint, a different audience. It helps to keep things simple, consistent, and familiar.
If you're designing, say, a RESTful API, it's a good idea to write a couple of client programs in different languages. See what kind of hell you would be putting your users through.
The idea generalizes, as well. Marketing is, sort of, documentation for the value proposition of your business. If you can't make the marketing work, maybe your business fundamentals need to be adjusted instead of hiring a “better marketer”.
Everywhere I’ve been except for IBM in the 90s, documentation is not given priority (time) as would be needed to both write the first round adequately and then maintain it as realities change.
Tools are much less a problem.
I suspect also that modern “agile” approaches work against building and maintaining documentation because developers are hyper focused on ticket level changes in short sprints.
Same goes to a lesser degree in writing tests. And when it comes to tests, devoting an entire sprint to increasing coverage seems to compensate, so perhaps a documentation sprint every few months might work.
Actually, the agile sprint approach also works against clean code based and discourages refactoring. So refactor sprints need to happen periodically.
I think I see a pattern here: current agile methodologies trade one set of problems for a new set.
Because it's thankless. The bosses I've had generally seem to think that writing documentation beyond inline comments is a waste of time; something that shouldn't be done unless scheduled for. And of course it's always the first job to be postponed if the schedule is tight (it always is.)
At work, documentation gets written when the boss isn't looking. Corporate culture makes writing documentation taboo. The examples of great documentation I am most familiar with are written for open source projects where managers aren't around to hassle people for writing it. Particularly: mpv, ffmpeg, racket, emacs. In my career I have yet to encounter a commercial software project with documentation at this level.
> In the world of programming, where “it depends” is often the best answer and everything is based on trade-offs, writing becomes that much harder. It needs to set the context, justify the decisions, and then power the low-level thinking leading into the code.
Might be that the author is talking about a different kind of documentation, but I believe describing the behaviour of a program in sufficiently detailed fashion and justifying why the program behaves like it does are two very different things.
The former is the task of the programmer and the audience is likely other programmers while the latter is the task of a product manager and the audience is likely higher-ups.
As a programmer who needs to interface with a particular API, I will be very thankful for a documentation that tells me exactly how the data to pass to the endpoint has to look and what kind of responses I have to handle. If there are any particular quirks, constraints or special cases I have to be aware of when using the API, the documentation must explain those as well.
But to be able to use an API, it's not necessary to know the exact decisions and tradeoffs that explain why the quirks and special cases are as they are.
> the code is already a perfect description of what it does.
No it isn't.
Code is a series of steps that in the right environment will lead to a particular behaviour. Code doesn't (automatically) tell you what that behaviour is - even less if that behaviour is also the intended behaviour.
You can have "self-documenting code" to some extent, but even this code is often low-level, spread out over dozens of files and has to handle various cross-cutting concerns which are usually not of interest to you.
This is why command line tools have manpages, even if the audience are programmers and the code is open source, so a user could theoretically learn everything by looking through the code.
> A disorganized pile of classes and methods in code may work – a pile of work of words and paragraphs won’t work. Writing HAS to be clear if it is to be of any use. Code will be accepted (to some extent) as long as it does its job.
This suggests to me that the problem with writing documentation isn't that writing in itself is hard. It is that it is hard to write clearly about badly organised code. So the average programmer can get his disorganised pile to compile and pass the tests, but he can't clearly articulate, in speech or in writing, its organisation.
It's similar to the problem of naming things. If it is hard to find a clear and precise name for a class, it is usually because the purpose of the class isn't clear and precise.
So my suggestion is to write the documentation in advance. If the organisation of the code is very hard to express in plain English, then it is because the organisation of the code isn't very well thought through in the first place, and should therefore be worked on some more. And that is easier to do before a lot of code is written already.
I often see peers struggle with writing documentation because they prioritize it as a separate task. For example they write all the code in a giant PR and then try to go back and write the documentation for everything at once. This creates a much larger more insurmountable seeming goal than if they had broken up the task and written the docs for each piece instead.
Another critical mistake is expecting devs to cover writing the documentation that explains the deep contextual intricacies of the business logic and reasoning for said logic. Big mistake. Your docs should cover the code and how it can be operated. Anything else is why we have project briefs, strategies and other documents and meetings.
So writing documentation isn’t hard. It’s just more and more devs are coming from a willy-nilly-web-search-when-you-think-of-it background with no formal organizational skills or experience.
Most of what is written in doc-blocks above your function should be generated. Everything else is operational information.
This article is a bit of a non sequitur. I generally agree with the points raised but the real problem is whether documentation is given enough time within the development cycle. In my experience it isn’t.
The point of documentation is to communicate to others how to keep developing a code base - what is does, how it does it etc. What form the documentation takes can be fluid, a full fledged wiki or a single readme.md file can fulfill the same role just as well! Some documentation is better than nothing, so start small and then improve it over time.
Diataxis[1] (mentioned on here recently) and ADR[2] massively helped me to create clearer docs. The Diataxis framework really helps you create something that could bring a new developer on board quickly, and ADR helps ensure the thinking behind architecture decisions is easy to understand.
You know what's worse that writing documentation? Keeping it up to date with software changes! Even worse if it has a GUI and all of the images have to be refreshed for every tweak. I've tried a number of auto-document tools, Doxygen, NaturalDocs, ReadtheDocs., and they just add another layer of pain. IMHO. In 30+ years I have not found a satisfactory solution, except to budget time & resources for constantly, incrementally reviewing & updating docs.
The problem with documentation is twofold:
1. it's broadcast-only. so you have to try to anticipate in advance what the questions might be.
2. it's disconnected from the source. so someone looking at the source code has no idea whether there is good, bad, or any documentation about it.
Documentation is so important. The problem with documentation is that engineers don't seem to understand it's benefits. I think one reason is that so much of the documentation out there is so poor quality that engineers think documentation can't be good quality.
You are right! You said just what David Parnas keeps saying. People don't know what or how to document
because they weren't taught to do it. It's not common knowledge. One has to dig into reading books and papers just to get the idea that it is possible.
Done right, it’s the final abstraction to the code: The whys and the hows, and effectively and gently guides the developer from epic/functional/spec/user requirements to the underlying design & implementation.
The problem with documentation is that the infrastructure/tooling for it is terrible compared to the infrastructure for code. Documentation is often in external systems (Confluence, etc) which lack version control and are hard to keep in sync with the code, and those systems are often extremely slow (especially considering all it does is handle text) and the user experience is terrible (no Markdown support, mandatory "wysiwyg" input, etc).
In all the places I've been that's now how things were and there wasn't any interest in changing it, so instead they kept going with a shitty and outdated Confluence/etc.
Documentation is a task that is never trivial, but it can be made bloody hard, or not, if you optimise for that. If you ignore documentation as a priority you're optimising for everything else, which implicitly breaks documentation.
But it doesn't have to be impossible, if the trivial aspects as designed to be minimal and the impact on documentation is allowed to dominate other impacts.
https://rant.gulbrandsen.priv.no/udoc/trolltech-documentatio... is relevant, even if most documentation isn't developer documentation. (It's about the Qt developer documentation, which IMO is/was the best developer documentation ever written and maintained by a small team, and no big team has ever done much better either. The hard thinking and work on that was mine.)
The key is that if documentation has such a low status that any other consideration can override it, the result will suck. And if documentation has such a low status that the writing tools suck, so will the result.
your manager will never give you a raise because you wrote nice documentations, in fact, he/she might think you're wasting time for not doing bugfixes or features development.
as long as documentation of code becomes one factor to evaluate the developers, with rewards somehow, things will change immediately.
I never felt doc is a technical problem, it's purely management for this one, it has been neglected for too long.
I've seen an awful lot of discussion of recruiting and interviewing practices on this site over the years, and there have been very few mentions of ways to check that people employed as programmers are able to write documentation.
It just requires time, and I would be happy to spend that time (I love writing, whether docs or just thoughts). As long as there's no ticket for it (approved by a stakeholder and assigned to me by the micromanager), I can't clock hours on it, and if I'm not clocking hours on a ticket, I'll eventually get an angry call..
It's not just lack of incentives, it's disincentives.
Documentation is a feature, the rest of my comment is anecdotal but in many shops, it’s tracked as part of the task of coding. I’ve started making separate JIRA tasks to explicitly create or update the documentation and have told my team to do the same. So far, the results are good.
It's bloody hard, you say? Hm, maybe in part of the cases, but you know what else is even harder?
Getting onboarded in a new project and having zero clue why X is written like that, why is Y is where it is and why Z is using a 10-year old thread-pool scheduler that is grossly inefficient. And you have to deliver feature A and bugfix B and you might collapse the house of cards and of course, critically important pieces of institutional knowledge are missing.
Eventually you do find out everything you need since you're not dumb and are a bright programmer, but you've lost weeks or maybe even months. The business have lost money because they basically had to give you anywhere from 1 to 3 full salaries just so you can catch up. And it's not even your fault, it's the last person's.
So I can't sympathize with "it's hard" at all. So what, dude? It's part of your job. Do it well. Nobody hired you to only do the easy stuff.
But this does outline the somewhat introverted, almost autistic nature of many programmers. When it comes to writing good docs most of them give up because that requires good and clear articulation which they don't possess. Or they find it "boring".
But they'd still curse if they found an obscure GitHub repo that could help with their their niche problem and find out that it has zero explanations or code comments.
> critically important pieces of institutional knowledge are missing
And the guys who have it are the "10x devs" in that joint.
>Or they find it "boring".
Or, it doesn't count much towards your annual performance review so then why bother if you have enough stuff on your plate that does count towards your performance review.
When was the last time someone got promoted because they write really, really good internal documentation?
Let's face it, most web facing SW nowadays is like fast-food. Investing tons of time in writing quality documentation would by like a triple Michelin star chef writing a 20 page article about a doner kebab.
> Let's face it, most web facing SW nowadays is like fast-food. Investing tons of time in writing quality documentation would by like a triple Michelin star chef writing a 20 page article about a doner kebab.
A lot depends on the project size and expected lifetime (honest expected lifetime). But you can look at it from another perspective: the most basic internal documentation, like commit messages explaining why something was done, interface-level comments explaining what a function or class does, internal comments explaining the tricky bits of implementation - they all increase velocity of product development. So it's a good thing to do on team level.
(But it's kind of a "pay it forward" thing. You may not benefit much from your own comments, but you'll be thankful for the ones your co-workers leave.)
> Or, it doesn't count much towards your performance review
Oh I agree that most managers have no clue about this metric so doing it well will likely mark you as the slowpoke of the team.
The way I address this is that I budget the time for those "extra" activities beforehand. I just find it a professional courtesy to leave good docs for the next person after me -- or new hires while I am still there.
Not sure there's a way to actually incentivize programmers to write good docs.
It would help if people read them. I write good docs and I often wonder why, as I have to endless redirect inbound queries to the doc. It's ok for my team, they learn, but it's annoying that the whole org never reads anything.
I am one of the guys who appreciates docs but also wants pointers at the start.
Just giving me an URL with 50-100 pages of docs is not good enough. Give me something like "when you are just starting", "when you want to tackle a ticket involving X" or "when you need to edit the deployment script" etc.
In my last job the architect was always irritated with me because he wrote a bunch of docs but never organized them or just gave a proper small index -- seriously, just 15 lines of text with links so you know where to click at the start would have been enough! -- but then somehow the team was at fault for "not reading the docs".
So there's a balance. I don't appreciate being given a book and being told "figure it out", which is what happened in my last jobs. Sigh.
Having a small page with starting pointers I always found priceless and is what I do in my work and I've had people contacting me 5 years after I left the job to thank me for it.
my docs are a single page full of one liners for practical issues that have organically arisen several times over the years, with a table of contents at the front.
As I said, I write good docs, but no-one will read them until they have been caught out 3 times asking obvious questions had they gone to the doc first. If I am not around, I know the docs will be abandoned immediately despite it representing a tome of hard won knowledge that saves an incredible amount of aggregate time from never needing to figure out how to do the same task twice.
There should exist a need, for pouring labour into something, to make it worthwhile. Also, when the need is there, it is much clearer for who and what purposes you're providing knowledge.
This is harsh, but I largely agree. I think the issue is that you often weren't hired to write docs either... biz often only cares about new features.
It's not the writing docs is hard, but that it takes a lot of time. I can write great docs, but it easily takes 50% of my time relative to code. The quality falls off pretty hard too -- my half-effort docs are pretty bad, like maybe worthless?
Obviously, there's a baseline level of, like, just commenting code which some devs still complain about, and that's absurd. But I think we need to be better at getting other people to value docs if we want them to be written, because they take time and effort to produce, it's a tradeoff like anything else in building software.
IMO the only way to achieve that is to inform the business about it. If they are aware that knowledge is being lost and then has to be re-acquired by the next person then I think that a lot of business people immediately grok the idea that not writing docs is a liability and a risk and that it makes swapping people hugely expensive (indirectly, due to you being paid just to catch up for a while).
Businessmen get this stuff quite well. But I feel very often nobody explains them the situation.
Yeah, I agree that business people are often quite receptive when you lay it out in terms of economic impact. I agree that this is a skill all devs should work on -- even just being able to talk about orders of magnitude of $ in your estimates is incredibly helpful.
But I've often found other devs can be your worst enemy here -- they rely on esoteric knowledge to build defensible moats around their seniority. Harder to convince execs about firing their "star" 10xer. Ultimately, this is why I think we need compiler assistance so that stuff like docs can be enforced in CI unilaterally.
> they rely on esoteric knowledge to build defensible moats around their seniority
Sadly you are correct. Job security and thus gatekeeping are the higher priority.
So yeah, yet another case of perverse incentives. :(
> Ultimately, this is why I think we need compiler assistance so that stuff like docs can be enforced in CI unilaterally.
Completely agreed, plus declarative programming. At 41 I am already sick to my stomach about having to manually write boilerplate. Tooling helps only a little and is hugely overrated; so what if the tool generates skeleton controllers et. al.? 90% of what you know should be there is yours to do anyway.
I read docs from 10 years ago, still relevant, but just hard to get any value out of. You can see they try to tick all the boxes, but there's very little practical value, direction or any insight. They don't communicate or provide entry steps into a domain, just list stuff very structurally. So much labour seems to have gone into making too much structure vacant of content, following invisible guidelines while trying to please uncaring taskmasters, long gone.
I think we're going to go down a more personal road, start caring about who reads what, and just write what would help out, initially, and as reference. Lots of projects on Github do this right, the popular ones, as they convey meaning much better than their competitors. Such examples stand the test of time too.
> having zero clue why X is written like that, why is Y is where it is and why Z is using a 10-year old thread-pool scheduler that is grossly inefficient. And you have to deliver feature A and bugfix B and you might collapse the house of cards and of course, critically important pieces of institutional knowledge are missing.
I've already resigned to the fact that any changes in the codebases I tend to work on involves spelunking through past commits, trying to git-blame my way into knowing where some suspect feature came from (and thus who to ask to explain it to me).
But few things annoy me more than when, after an hour of poking around and following code being moved across files, I finally arrive at the commit that introduced the thing I'm after, only to discover that the entire commit message is "refactored $foo", or "fix $blah". Oh, and the commit touched 20 different files, implementing 3-5 different things, and the author left the company a year ago.
So if you aren't doing it already, then for the sake of everyone (including yourself few months from now): please, write descriptive commit messages, while everything is still fresh in your memory. By descriptive, I mean at least a bullet point list of everything that was changed, and why. Even if it means repeating some of the stuff from a ticket or a discussion somewhere - because none of these things will be available or easy to find few months later.
Something like:
Add feature Foo
This commit implements feature Foo, as per ticket #12345:
option $foo controls whether or not Flux blergs or blargs;
in the latter case, $this and $that will happen.
* Configuration files have been updated with the new paramter.
* Flux Controller no longer looks at unobtanium to determine
the type of blerging to perform.
* The above implies that checking Flux for the type of blerg
is now speculative, and should not be relied on in the future.
* A new utility Asdf has been added; it provides common functionality
for blarging.
...
Etc. You get the picture. It's quite easy to write a message like this when committing - it's essentially a polished brain dump. And its usefulness will be immense the next time someone has to work in this part of the codebase.
I think the solution is the land where documentation is part of the code, so that you have to write it. Ada workbenches tried to get there, as did UML workbenches, but failed.
Lisp and Smalltalk somewhat succeeded, but only for single user systems.
Two questions remain:
Firstly, does that land exist?
Secondly: if so, how do we get there from out local optimum that many, many programmers spend decades on to build higher and higher?
I guess indeed many find it bloody boring. A chore to be dodged if possible. And also when doing some documentation work, because people are so deeply into the subject matter, they feel many things are obvious and need no further elaboration.
I do both and it’s not hard, it’s work and people have gotten lazy and don’t do work anymore they just write and read BS on the net and call it true and vote people down for telling the truth.
I can agree that writing documentation is certainly more boring than writing code, and lazyness tends to play a role in the willingness to complete boring tasks.
But there are many legitimate reasons why one would clarify it as "harder" than programming:
- It requires a different skillset, namely writing.
- It is not uncommon for non-english shops to have a policy of documentation in english. That might be sensible, but complicates the task even further.
- In programming, it suffers from the same problems as math: Natural language is more often than not unsuited to express entirely abstract concepts, at least in concise and easily understandable ways.
- Conversely, natural language often lacks the necessary precision to talk about technical details.
- To alleviate all these problems with language somewhat, you might opt to use diagrams. Which requires yet another skillset.
- It requires time. And quite a lot of it actually. Usually more than you need for the actual programming task. This is why most managers care way less about documentation than they should: They know very well that it detracts time from actual programming tasks.
So sure, people tend to be lazy, but there are certainly good reasons why that happens more often in this area of our work than in others.
I think the answer is just to have more people in the loop. Why do I have to write code, tests, documentation, dealing with jira tickets etc... In the old days, people had secretaries to handle some tasks. I would gladly exchange lower pay for having more people to spread out these tasks.
If it was that easy, there would indeed be a silver bullet: Secretaries could write specs that would compile into magical code solving every problem. They'd also infer user tutorials and data flow diagrams from machine code, to help out fresh blood.
What people can do it help allievate the gaps, but sadly, they're not geeting much help from managers and business people this time 'round either.
So do I. I even write documentation for personal projects, as a manual to my future self. However, absolutely every team I've ever worked with held a majority view of documentation being unnecessary.
Some even mentioned "code as documentation". In the end, they just kept finding justifications for not writing documentation or even proper code comments.
Three months after some solution was written, they couldn't explain the thinking behind it or even all the deep dependencies and magic return values because the person who had written the solution had left the team. Even then they were unwilling to see how documentation would have helped.
This repeated with every freaking team I have ever worked with. The only documentation any of those projects had was the one I created while figuring out the code base.
I observed a fundamental laziness in written communication in many developers and to this day I find it puzzling. I sometimes write down a draft of some solution just by myself, like I would explain it to another developer. More often than not, this helps me in highlighting inconsistencies or errors in my thinking.
Lazy people do the minimum amount of work required of them, not the easiest work. Generally, documentation is useful, not required or business critical
As a backend programmer, I really thought I had this nailed. I'm good at writing clean, mostly-pure code that composes well. The idea is to get the code to the point where the "what" and the "how" is communicated (clearly; avoid cleverness whenever possible) by the code itself, and to save the comments for the "why". That way if the "why" gets out of date, you can revisit whether that code should be there at all. The exceptions should be the highly tuned code that is hard to read - documenting what/how is more appropriate there, and that code should rarely change anyway.
But all that started changing with react hooks on the frontend. With contexts and state going everywhere, it's really hard to document. How do you document a state machine that by definition is spread between multiple code locations? You can document a hook's purpose but it doesn't tell you anything about complex behavior. It starts to feel as hard as documenting code that has a bunch of mutable global variables.
First of all, if there is no documentation then it's not engineering.
If you don't agree, then imagine civil engineers which don't bother writing documentation, or mechanical engineers not bothering to do blueprints, or electrical engineers don't bothering to document schemata, properties and behaviour of components, etc. Would it be a work of an engineer?
The biggest problem with SW documentation is that people don't know how to do it properly and even why to do it at all. Mostly because people in universities also don't know how to do it properly and so it is not taught. As a result the documents are often a mix of useless prose with some incomplete and imprecise diagrams. That's why most people don't bother.
David Parnas makes it clear WHY and HOW to write documentation for SW.
If you wonder how good can documentation be then google "requirements document for A-7E aircraft". Barry Boehm has said that it's the best requirements document he has ever seen. I guess that it's still the best one!
My experience so far when I wrote documentation for some pieces that I though needed it:
- Nobody asked for it
- Nobody reads it
- Nobody reviews it
- Nobody updates it
- It takes me over a day to write so I feel quite guilty wasting the time
- It is often not clear were I should locate it (as code comments? , in markdown file in repo? In knowledgebase wiki? Some other pile of corporate docs?)
When I talked to my successor about some piece of code a year later he told me how hard it was to refactor and (jokingly) that they pissed of a client breaking a bunch of features while refactoring it.
I mentioned the documentation about it, but apparently he did not even notice it existed (seemed also not interested in it).
I guess my fault for not just saving it as markdown in our repo, but instead saving it in a document in our knowledge base (as was according to protocol).
I tend to not write documentation because with a massively understaffed team it is futile. If we had 10 devs working on this project we could all spend a month focusing on docs and significantly move the needle. Right now I'd have to slave away for 10 months to get the same amount done on the docs backlog. So it doesn't happen, because I want to write code and not docs all day long. When I look at the many person-years of backlog of documentation that needs to get done I just get demoralized and that makes me not care about any of it. Want more documentation? Hire more devs.
And putting barriers in the way so that I can't code without docs just pisses me off when I'm demoralized. Make it take twice as long to code anything and I'm definitely thinking about looking for other work.
I suspect that one big, big reason developers don't document is this: most developers work on making relatively small changes something that came before them and is poorly documented. There is no documentation for them to maintain in parallel with maintaining the code.
There are barriers, both psychological and real, against documenting changes that ride on an undocumented ball of mud.
The psychological barriers amount to, "what is the point". Why document some insignificant part of something that is documented, on the whole? For instance, imagine a windowing system that is mostly undocumented, except for the wonderful 15 page treatise on its scrollbars, because a compulsive documenter touched the scroll-bar code. Which was 13 years ago.
The real barriers stem from the problem that the documentation depends on other documentation, much like code depends on other code. If you document a small part of some undocumented whole, that documentation has nothing to refer to. There are no certainties it can rely on, provided by other documentation, no context. This attaches a barrier to the documentation: the first person to document anything at all in that system has to provide the context for that documentation, and start the process.
That barrier brings with it psychological barriers: the fear of the effort (how much time it will take to start the documentation effort) and the perception of futility: that nobody else will come on board, and so that will be the first and last piece of documentation ever written, effectively making it a waste of effort.
For these reasons, even programmers who write excellent, lengthy documentation for their side projects can become reluctant to document something on the job.
Developers working on an undocumented ball of mud might put their documenting effort into detailed summaries of their changes and the reasoning behind them, rather than into building a coherent description of the system: in other words, to write that documentation that is more likely to save them from any predicament caused by their changes, rather than to help someone else navigate the system as such.
> Writing is a tough, demanding task. It requires organizing our thoughts clearly, examining them critically, and expressing them clearly. While the expressing part can be simplified to some extent (depending on the quality of writing required), all three steps are taxing when done properly.
Wow, almost sounds like some other aspect of a job a software engineer is required to do...
> If a developer doesn’t write documentation, their work still gets done.
If documentation is part of their job, it literally does not.
> Not writing doesn’t block shipping (at least not right away).
> Wow, almost sounds like some other aspect of a job a software engineer is required to do...
Which I think is the real problem. Writing/modifying code can't be ignored; the resulting behavior of technical systems absolutely depends on doing that.
You can argue that the desired behavior of human system also depends on good writing and you're correct. But the connection of the input and output is much more opaque, the social conception of the role is focused on the behavior of technical systems, the incentive structures are therefore focused on the behavior of technical systems, so when there is more to do than can be done writing docs will not make the top of the priority queue.
the last is bs. if you dont ship you wont make money, the compaby dies. if you dont write docs, you might incur tech debt but the company keeps on living.
ofc if there is enough time and manpower: write docs, write unit tests, discuss architecture. no prob.
This was my attitude as well for much of my career, and it was a mistake. The "correct" decision depends on the state of the business and your resources at the time. And nothing else.
I don't know why you're getting downvoted. Tech debt is a tool and I think it's absolutely fair to put the needs of the business before documentation, or even code quality to a certain extent. As long as developers and management are aware that isn't free, and the cost must be repaid, I think it's the right decision in many cases.
Yes, I've worked mostly in money losing startups, but since I work in a $ billion-profit investment bank I saw the difference. It's better to waste money on technical debt crisis down the line than have no money to waste because of thousands of devs all being perfectionists locking each other.
Small startups seem to reach this stage eventually and in big companies, management is acutely aware of it, and breaks every attempt at correctness-over-business-rationality, under the teary cries of the autists, sometimes :D
I don't want to work for a company that puts its need above mine. That company isn't worth my energy.
The correct decision is to take an approach that involves producing quality code and documentation as you go.
If you don't have the time available to do that then you don't actually have the time to do that piece of work.
Doing so regardless, or becoming expeditors is the worst thing you can do for a company. It creates a race to the bottom which in turn creates a god awful place to work.
This feedback loop continues to worsen as you struggle to hire and struggle to retain.
If you're building an open source general purpose tool, or something else meant to be reusable and consumed by the general public then sure. But the vast majority of software we write has a very definite lifecycle of birth, maintenance, and death. For the most part, one team with continuous word of mouth knowledge transfer will be responsible for it. And by the time that team has moved on, the software itself will have outlived its' usefulness. In an agile environment like this, keeping any kind of documentation up to date to be meaningfully useful is almost impossible without a dedicated team member.
Documentation is not generally useless. Most code is used far longer than it's intended life, most code is read far more often than it's changed, and documentation can save literally hours and days of struggle. Word of mouth knowledge transfer is abysmal at keeping critical knowledge alive. Nobody ever knows anything about legacy code, and it's because no one wrote documentation. Please stop telling people documentation isn't important
I do agree that code is read more than it is written. At the same time, there's the adage: "treat your data as permanent, and your code as transitory."
Not really. Maybe in your very narrow environment, but all the multinational corporations I've worked in past 17 years on, situation is way more complex. Software often outlives people who created them, sometimes even whole teams originally responsible for it.
Suddenly you have a Pune team managing all environments including production, who have rather vague about yet another system thrown on them due to that smart idea called outsourcing. Sure they can change a thing or two, but corner cases can and often do bite hard. Code itself, while describing well what is happening, often doesn't contain much info about why. Or further effects of decisions. Full picture of whole integration involving 20 or 100 systems etc...
Another issue in huge companies spread across the globe is the ability to actually connect with relevant team, and their reluctance to share crucial info. A job security political game is not foreign to devs in some cases. I've had my request for source code of one of our internal security libs, the cornerstone of all of our inter-app authentication, refused with justification that its safer for the company to not share it even within company. Mind you, the .jar wasn't obfluscated at all so JAD got me to almost-compilable version so that effort wasn't even half-assed.
Man, I could spend whole evening telling stories how documentation can be great. Even incomplete, not completely up-to-date one can save your ass from time to time. And tons of time on top of that.
Writing is thinking and oftentimes just being able to explain in words what a thing does, should do, and should not do- has tremendous value as part of the design process before any code is written.
Under this kind of plausible argument, lie all sort of insects (bugs) in the dark. If people can make time to write tests, so should they write documentation. There is inordinate amount of frustration, productivity loss, and regression associated with having poor/outdated/no documentation.
And a good set of unit tests can also cover what's the most useful, like edge cases, complex sequence, particular client flows, bug that actually happened in prod etc
I don't understand 1) Why every company doesn't use Github/Gitlab to do documentation. and 2) Why Github/Gitlab don't offer a cheaper account type for employees that don't code or even write documentation to make internal docs available to everyone.
It's easier to maintain documentation when it's closer to the project. You can also automate a lot of its parts for your projects. You can create template projects and use tools like cookiecutter every time you create a new repo to have the basic structure in place. That's already half of the road.
The single best thing you can do for good documentation is to hire actual technical writer. Someone who already knows how to write in general, how to write and structure technical texts specifically and who wants to write.
It has three advantages:
- They are cheaper then programmers.
- They write faster then programmers.
- Unlike programmers, they produce well structured easy to read text.
And bonus: if you are lucky, they will be able to explain to programmers how to write better. They won't make them into James Joyces, but will make them learn a bit. It will take time and only some programmers will improve, but the technical writers has that effect.
The best documentation I ever saw in a product was written by actual technical writers that worked in collaboration with the developers. If you want fantastic documentation, make it someone’s full time job.
Writing documentation is one of the main ways to show you understand the system you wrote, and as a side effect, instruct the users. It's one of the highest exercise a programmer can do.
Documentation seems to follow two principles. It’s either (1) never complete, or (2) always outdated. As a general rule, you can always rely on it to never have the information you need, or to always have incorrect information.
Joking aside, documentation is great but it needs to be treated as importantly as the code itself. Treating documentation as a feature, rather than an afterthought or side effect, will allow it to get the funding and attention that it deserves.
I'm going to write an unpopular opinion. I really dislike when anyone in my team writes documentation...
First of all you now have 2 sources of truth, both rot and only one of them is the actual truth.
Besides that I expect us all to write code the same way so after you are onboarded and need to learn something you should just dig into the code. In fact your job is to dig into the code and iterate it.
Own that code even if you weren’t the last to touch it you might have to be the next.
I'm writing documentation now and it's not bad, just takes a long time and keeps me from coding more features. I try to add screenshots and relevant links to helpful blogs Luckily, my role on the team is more of a consultant and technical researcher, so this work provides a lot of value as we anticipate onboarding more junior engineers in the next phase of our project.
It's bloody hard to write documentation at some small companies, because management want you to put out fires and maintain buggy legacy codebases most of the time. The rest of your time is spent in meetings and actual development. At these companies writing documentation is brushed aside and developers are supposed to "figure it out".
I may be a minority here, but I really enjoy writing documentation.
It lets me wrap my head around the problem and really get into how people would use the software;
often times I find a lot of usability issues when writing the documentation, so it has become instrumental in almost every coding I do.
Although I do agree that it is vastly underappreciated :/.
I like writing documentation (for my mainly command-line tools), but the trick is to write the documentation first. It's a great way to prototype how you want the tool to work.
Now you have one more problem (several, in fact) - explaining the business and the software to the technical writer. This is so difficult, that in all my long career in software development and management, I never saw any company specifically hire a technical writer, though I have done a lot of technical writing myself, on the side.
The technical writer can be involved in the release management and ask for details that then he can use as a starting point.
They can help identify gaps in documentation, outdated documentation, etc. They can take care of the clarity, formatting and distribution. You can create a ticket about documentation and assign it to them.
Then, while many developers write excellent documentation... some other developers truly SUCK at documenting. Some people like to sound intelligent and their comments read like a choose-your-adventure monologue that cannot be read linearly. Or they use vague, ambiguous language, or overuse acronyms or abbreviations. Or they sign every piece of code they touch like a dog scent marking your entire code base. Or they use profanity, or they get offtopic or humorous... All of that is a distraction from doing my job.
The technical writer takes care of those problems for you. They can create guidelines for documentation so that people treat documentation with the respect it deserves.
This is a terrible solution. Writing is part of the developers job requirements. And writing doesn't just happen in documentation. It happens in emails, technical specs, presentations, code comments. Developers should maybe just get better at their job because documenting and writing is part of it.
Writing is part of developer job requirements, sure.
Is technical writing part of your interviews? do you hire, promote or fire people based on technical writing performance? Is documentation taken as seriously as other code deliverables?
Why paying an expensive senior developer to maintain documentation in a non-commited way when you can pay a technical writer to do it better and for cheaper than the engineer can?
And with real ownership and accountability over documentation, unlike the engineers.
Technical writers are cost efficient and pay themselves very quickly.
Writing docs isn't hard, but maintaining a well-documented ecosystem is quite difficult. The aim of docs should be to limit reverse engineering time, or at a higher level should provide direct impact on velocity of a given team member.
When I first started out I would keep big centralized documentation stores. If a project had a dedicated wiki then I'd use that, but if that wasn't present I'd throw together something on my internal note taking like Obsidian. The downside of centralized and detached documentation is that it's hard to check per pull request if it's been updated, so it relies on regularly fallible human processes. Second, code and architecture tend to drift, and it's difficult to stay on top of that drift with a centralized doc store.
I then gravitated towards in-repository documentation. I'd open up a /docs folder and either include a static site to be run locally or configured to run on the web. I haven't seen a ton of downsides for this approach other than that it will skew respiratory metrics if you keep them. Changes in the docs folder can be prompted and checked for in pull requests.
The only potential downside is when your software spans multiple repositories of different types. A deployment repo here, code repos there - have one store of your documentation in the application repository means you treat most (if not all) other repositories as generic and document them as such so they neatly fit into your software docs.
None of this even begins to touch on documenting inline, which is fairly key to maintaining good code. For all the churn and hand-wringing I see about when to document which is often phrased as when not to document, this is the chief barrier I see software engineers hit their head on. When you're working on a package / module / library it's easy to substantiate lots of implicit context that's easy to admit during this process. I follow this pretty loose framework:
- document inputs and outputs
- document error conditions
- document package purpose, intended use
This direct documentation should then be backed up by more implicit documentation like variable / function naming conventions and tests. Testing is all about maintaining contracts; not attaining "coverage". Though it is possible that tests are not only reinforcing trust for end users; I do generally trust tests in PRs but only so far as I know I won't break my users as opposed to being "bug free". Really thorough testing that instills my confidence involves other methods like fuzzing.
Documentation can be implicitly managed thorough writing clear, slightly verbose code with good comments, as well as clearly described test cases. It can also live in the project management tools if used effectively. With a good software development process you can have a historical record of the background, requirements, discussions around that feature/component, and how it was developed
I have moved to a FAANG and their documentation is downright fucking awful. _everyone_ just writes code, with lots of "clever" bits, and doesn't bother to fucking comment.
Not only that because people don't even _comment_ their code, the wiki is a total shit show. Want to know how to use a Queue? tribal knowledge. want to know which DB is best for x? tribal knowledge. Want to know how to create a new endpoint? tribal knowledge.
Worse still, we had a class during induction where some ponytailed "10x" said "If you are messaging me asking questions, I can't help other people" I didn't have the bollocks at the time to ask why his documentation sucked arse. There seemed to be a weird pride in the fact that people needed to message him to figure his shit code.
The moral of the story is this:
fuck off with your clever code, spend that effortyou put into learning new languages, or trying a new techniques and put it into developing your writing skills. It takes empathy, organisation and skill. It'll make you a better programmer and a better person.