IMO the main problem is in the very way organisations are structured - it is about power, and people (really meaning management) want to achieve good enough results, given that their control is maintained.
Agile as aesthetics is cool, but actual implementations move power away from management and we can't have that. So we will rebrand project managers to POs and whatever managers to scrum masters, and will follow the rituals as long as they don't get in the way of normal run of things.
Agile was grounded on solid principles, but is very ideologically naive, which allowed its easy cooption by consultants and management.
This is a very good summary. The universal formulation of this is Pournelle's Iron Law of Bureaucracy [0].
Pournelle's Iron Law of Bureaucracy states that in any bureaucratic organization there will be two kinds of people:
First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration.
Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc.
The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.
There is also what Stafford Beer writes about when discussing viable systems - the goal for most organisations, is not their stated goals, but survival and perpetuating the system. The stated goals are a byproduct of that, if and when achieved.i.e. in a lot of universities education is a byproduct, in a lot of hospitals, healthcare is a byproduct, hell in a lot of governments, governance and actually improving things is a byproduct, that will not happen unless there is an absolute need for it...
In a way it's like running a tricky multi-objective optimisation, and the AI will find a way to cheat the stated goal.
This whole trail through to the top comment is spot on.
It's also helped me work out a corollary in my mind that has puzzled me for a bit.
My sense, in the Marxist tradition, is that modern organisations are dependent on the extraction of value from highly capable technical resources and, especially outside the tech bubble, they largely resent this dependency.
Let's say developers are an example of a highly capable technical resource, though I am by no means limiting the scope.
This results in a series of mechanics that lead to developers being alienated:
- From the product of their development (ownership of IP and the resultant value of their code, distance from seeing the positive impacts of their work or talking to those it helps)
- From the act of developing itself (by its reduction to commercial use and control over how it is done, approval gates, arbitrary coding standards, ticket systems, scrum processes, project managers and product owners, Jira, timesheets etc.)
- From their fellow workers (stack ranking, power dynamics, labour competition, structural organisational tension)
- From their human nature and natural talent (by the reduction of their humanity and passion and capability to a mere "developer" or "engineer", use of stereotypes, reduction of humanity to output/LOC/story points delivered, corporate gaslighting at questioning this state of affairs etc.)
And most non-developer people that have worked in an average organisation and spent much time with developers have seen all of this at play, and heard how much developers hate it. Yet many still refuse, even in the face of self-interest (e.g. faster delivery of outcomes for a non-technical manager), to empathise and accept the reality of the experience enough to support better workplaces for developers. A tangible recent example is the insistence with all sorts of reasons on getting developers back into the office where they can be watched despite demonstrably lower productivity and engagement.
My sense of this is that what developers can bring to the modern world is the closest humanity has got to magic. And this dependency is resented. And this resentment leads to workplaces in which this resentment is externalised in the form of debasement (e.g. caricatures and other forms of ego compensation - "they're just the boffins, they don't have people skills!"), control ("the boffins can't really be trusted - better add some process and oversight, and given they don't have the people skills better make sure they aren't anywhere near management/clients!"), and dependency inversion ("sure the boffins can do their coding stuff, but they'd be nothing without us to help babysit and organise things, they don't get the way the world works, clearly it's they who actually need us!"). And this environment in turn leads developers to internalise this systemic resentment as a resentment of themselves, their capability, and their work, aka burnout.
But one question has been bubbling away for me for a while.
How do so many organisations arrive at a system in which it's almost a badge of honour to not be one of the doers? That those who can't do, should, as a moral claim, oversee, and manage, and lead? And we should keep adding more of those people until the doers can't possible do. Even when that produces lower tangible results.
Maybe at one point I internalised the Office Space / IT Crowd idea - the non-doers are "people people" who didn't spend decades at their PCs honing their craft but instead went to wild parties and focused on normal people stuff (the implication of course being that developers are lesser than normal people). Maybe the developers really can't be trusted. Maybe they do need to be managed and watched and distanced. Maybe the code monkey caricature (before being reclaimed by those it was used to demean) is right.
But now I wonder.
What if those people are empowered by the systems into positions of power over developers because in the first instance they affirm the original resentment: what if putting non-doers in charge safely perpetuates the idea that the developers belong at the bottom of the pyramid and affirms the extraction of their labour in support of the salaries and profits of those above relying on it? Or to put it another way, what if the code monkey caricature is effectively a justification of Marxist exploitation?
And what if the second order effect here is that some, let's call them the senior management class - leaders of the Second order of Pournelle's Iron Law of Bureaucracy in the example above - are more conscious of these dynamics and consciously perpetuating them. What if they are knowingly hiring more non-doers to help keep this balance and control. What if it's not accidental, or people skills, that means non-doers are in charge? What if the very reason they're there in the first place is to be in charge even in positions outside of formal leadership (or at least indirectly support the power of someone else who brought them in for that reason) - after all they're not there to do.
So to close out a long post, a corollary to Pournelle's Iron Law of Bureaucracy: in any bureaucratic organisation the ability to be able to contribute to the goals of the organisation will be an insurmountable barrier to having influence or control within it.
> How do so many organisations arrive at a system in which it's almost a badge of honour to not be one of the doers?
I believe this is a holdover from Western society's feudal roots*: the people who do things, especially who do things with their hands, are considered to be lesser beings than those who, by divine right, rule over them.
The ideas and values of feudalism are deeply embedded in our culture, and it's very hard to escape, for instance, the idea of the Good King, ruling benevolently because he is divinely granted better discernment to know what is good and right. (And also the Iron-Fisted Tyrant, which is a seductive fantasy to a different type of person.)
Some of these ideas come to us through the medium of things like just world theory and other Protestant notions, but at their core, they're still basically the same thing: the idea that the people who rule do so because they are Better People, and the people they rule (the ones who have to work for them) are in the position they're in because they are Lesser People.
Thus, rising to management becomes a proof that you, too, were a Better Person all along.
* Not saying other societies don't have feudal roots, just that I'm familiar with Western society
> What if they are knowingly hiring more non-doers to help keep this balance and control. What if it's not accidental, or people skills, that means non-doers are in charge?
You bet this is the case. Every manager is, consciously or not, empire building. Hiring more underlings so they become more important. This not necessarily mean that they are evil schemers - on average, people are pretty good at convincing themselves that what they do is good and necessary.
Note that Big Corps have much more in common with communism than with capitalism. The way people think about capitalism, where everything is hyper efficient, is incompatible with the huge amount of waste that goes on in big organisations (this is a gross simplification, but you get the point). Communism by its nature means a large centralized bureaucracy, with ironically even worse denial of personal agency and thus more alienation.
Valve tried the whole "let's get rid of managers" and "taking back power in dev's hands' thing. It always devolve into doers-turned-barons who screw everyone and everything with power struggles and political intrigues. And I say this as someone who finds most managers useless and incompetent.
This is the main problem with Marxist theory. It is a well contrived system of beliefs mixing strong emotions, down to earth realism and an overcomplicated and oververbose theorical development that justifies the existence of its own expertise. All of this form a strong opium for most intellectuals.
In the end it loses touch with reality and its scientific pretense very quickly as the predictions don't meaningfully materialize and it fails to deliver on its promises.
History does not follow a direction. The workers never formed a group united just by virtue of being workers. Humans are not malleable not do they have a good natural state. Reforms have always outperformed radical and violent revolutions. Marxist revolutions were never lead, instigated and started by the workers. It miserably fails at deciding on who actuallt gets the power ans make decisions.
A lot of people blame "capitalism" for societal problems but there are no other reasons given except that those are unique to and caused directly by capitalism; capitalism being whatever exists just by virtue of being outside a marxist/socialist society. This is made evident by the fact that marxist revolutionaries are always confronted by the same problems that those who they overthrown had; and so they become labeled as state-capitalists to excuse their faillings.
I think that some of this is true, but, in the Marxist tradition, it strives to define classes of people to hate, and ignores the key aspects of capital, risk and relationship in things succeeding.
True but not only management is to blame. Proper agile requires a certain amount of common sense and a willingness to spend some thought on improving the process.
There is a group of developers that are either not able or not willing to think about these things. This group just wants a set of rules that tells them exactly what to do in every situation so they can switch off their brains, don't have to carry any responsibility and just do what they are told.
> There is a group of developers that are either not able or not willing to think about these things.
In most cases because process improvement suggestions by non-managers (or in the worst cases, by anyone non-external) tend to get ignored by the upper chain or, worse, being stolen by someone along the chain.
Or, to put it shortly, why invest mental energy into improving processes when it's either being discarded anyway or used for the promotion of some manager without delivering any benefit to oneself?
And to make it worse, some organizations are actively set up that way to discourage internal improvement ideas so that the uppermost management is free to hire the infamous Big Four "consulting" agencies to mask what upper management wants as "external and independent" expertise that must now be followed. The Big Four (and their smaller fellows) are a fucking scam set up to enrich the top level at the expense of everyone below.
I'm not sure I see the point here. You improve the process because you're subject to it, and because the current process has some downsides to you.
If you're trying to improve a process just for the recognition, then you're by definition not subject to it, so you're playing the role of a manager. And you excluded that yourself in your first sentence.
> If you're trying to improve a process just for the recognition, then you're by definition not subject to it
Simple: when hired as a developer, I expect being compensated in one way or the other for work that goes beyond scope - especially when it is like in most private companies where you need some form of achievements to show to get a raise or a promotion. And do not forget: when you improve processes, usually your workload will be increased as a result, so in the worst case you'll end up with your manager getting the raise for your suggestion and double the workload as "thanks".
This falls squarely into stepping out of the developer role and taking on the role of a manager though. It's really not about the people who don't want to think about those things.
>True but not only management is to blame. Proper agile requires a certain amount of common sense and a willingness to spend some thought on improving the process. There is a group of developers that are either not able or not willing to think about these things.
You can fire or train those developers. You can't do that to the CEO.
Agile tranformation requires buy in right to the top. I've seen it work from the top down, never from bottom up and never from middle up/down either.
Oh, there is definitely something to that. There are people, not just managers, in general who for one reason or another _want_ someone to take charge, be responsible and tell everyone what to do. Which is not bad, per se, it gets bad when the insistence is that this be a boss assigned to you, rather than someone you choose, but that is broader than just agile and the topic in question.
The generic law for that is: any methodology that intends to take power away from the managers will be thoroughly corrupted and inverted after adoption
To me that sounds like socialism - a perfect system if it just weren't for those pesky people and their deeply ingrained selfishness and desire to gain more status, power and wealth.
Agile is a system that ignores how power and status plays work in companies which almost always makes it into a miserable system for everyone involved.
Well agile is rooted in anarchism's self-organization. It is meant to evolve in a collaborative shell. I think the main factor of trouble is that we want to implement Agile principles in competitive driven companies. They are not looking for customer's satisfaction, they are looking for customer's money.
Agile was grounded on solid principles, but is very ideologically naive, which allowed its easy cooption by consultants and management.