Like a lot of business methodologies, the principles are sound - but eventually they turn into cargo cults around a meta-process that only exists to feed itself.
The exact same thing is happening now with 'Agile' where big corporations are switching to sprints and hiring scrummasters, and renaming departments - mostly as a way to stay hip and cool. But at the end of the day, there is no business framework that will replace good judgement and discernment.
This is the managerialist dream. To replace employees' judgement and competence with a process and management methodology. That way everyone is just a replaceable cog in their beautiful machine.
this. I got into real trouble a few years ago by speaking my mind to my MD about the latest motivational posters which portrayed our business unit as a machine made of gears and cogs. I had two objections. One, people are not cogs. Two, the gearing system on the pictures wouldn't work. I got approbation because I was not "commercial enough" and also "not onboard" but the pictures stopped being made. So... a win.
The thing that strikes me is that engaged and experienced employees deal with rafts of problems silently, using their networks to create and co-ordinate responses without executive intervention. In fact executive intervention is actively avoided because there isn't enough time in the exec's diaries to communicate and debate the issues properly so inevitably such intervention will be via over simplified solutions that often make things worse. The problem with this is that execs do not see what is happening. If they did they would be impressed and pleased with what is going on, but uneasy about the risks and their irrelevance. What this then leads to are management culls where value is lost because the problems that are being solved are not acknowledged in the company before the cull. After the cull this all emerges as a huge and nasty surprise that the company now lacks the capability to deal with.
The lesson : not everything (in fact not much) about a business is on the balance sheet, and in the corporate world the dead have no voices.
> I got into real trouble a few years ago by speaking my mind to my MD about the latest motivational posters which portrayed our business unit as a machine made of gears and cogs.
Well you could say it never works optimally, but it depends on what you are optimizing for. A lot of people think that because business need to make money that they should optimize for making the most amount of money and therefore try to maximize productivity. They look at organizations and what they see is very much at odds with that.
In reality they don't try to maximize employee productivity, they try to maximize employee fungibility.
If all the company is trying to do is survive then I'd say it definitely works.
In the short term; the twin problems - things change (more so now than in the past) and much corporate activity is necessary but unacknowledged in the org chart and balance sheet. Ripping out "inefficiencies" leaves businesses without capability. Reacquiring corporate capabilities (or acquiring them) is a long and risky road. Business schools and investors need to think about diversity of capability and expertise and ability to respond and change as measures of long (even medium term) corporate value.
(1) what does it mean to optimize within limits of average managerial perception? while some management knows the domain well enough they can identify issues and talent that add additional levels of productivity... many can't. Or as some say, A players hire A players, B players hire C players, etc.
(2) predictable + control-able levels of productivity may well be more valuable in meeting managerial goals than exceptional levels of productivity. this is a generalization of the fungibility optimization; fungibility is one aspect of predictability and control.
I liken it to an organism. An organism is trying to stay in homeostasis. It's also either growing or decaying, though it mostly appears static.
It's exactly as you say. Many aren't capable of identifying opportunities for optimization. Re: the organism metaphor - an organism can be relatively unhealthy and still survive. Some people are ultra-fit. Most are not. Some people really are quite unhealthy. The bar for survival is fairly low. Project that on to companies you can imagine the ultra fit being those companies whose managers can work together to identify opportunities for all the things they are aiming to optimize for and are able to optimize for more than just staying alive.
so this would confirm system theory's point of view, that exactly states that just this can be the goal of a system, where companies are just an instance of precisely that.
> This is the managerialist dream. To replace employees' judgement and competence with a process and management methodology. That way everyone is just a replaceable cog in their beautiful machine.
It's ironic in the adoption of sold-as-Agile processes, since it is precisely the idea that Agile (as in the Manifesto and associated Principles) was a firm reaction against.
- they decide to apply it and see the "shortcomings"
- apply additional processes to where they feel it doesn't match
You end up with something not so far away from the original process, but slightly changed and more marketable. It can have noticeable good effects, but usually doesn't end up being any radical change or revolution.
I think a lot of new ideas also get neutralized by modifying them in a way that the power position of the people on the top doesn’t get threatened. I have seen that now several times. Something like agile gets introduced. Management embraces it but after a while when management would need to change, the enthusiasm stops and no further changes happen.
It seems that management only responds to top-down change but not bottom-up.
Pilots especially are heavily trained to use checklists for just about everything. A significant chunk of training to be a pilot is just reinforcing the behavior to always use the checklist.
Doctors, based on my limited understanding, are becoming more and more pragmatic about using checklists in surgeries. I am not a doctor, nor am I an expert, I have just read some interesting articles and heard from surgeons that they are trying to improve the process to reduce careless mistakes.
No clue about lawyers.
So processes can actually be incredibly useful, and oftentimes are designed after the fact. Agile has some neat ideas as well, but in practice I'd say 90% of the time it's pretty fucking terrible and the company would be better off without trying it entirely. Not because agile is bad, but because the people in charge of agile are usually bad.
At the end of the day it comes back to something I've learned painfully time and time again throughout my career. No matter how talented you are, you are powerless against a shitty manager.
> Pilots especially are heavily trained to use checklists for just about everything.
Yeah, but it didn't appear out of nowhere. NASA et al did a tonne of research [0, 1] that led up to the idea of using checklists. Likewise Cockpit/Crew Resource Management [2] and the psychology of getting people quickly out of aircraft that are on fire [3], all of which were prompted by one disaster or another.
Software engineering could use some of that rigour, but I'd anticipate a little resistance to publishing research which shows that the process of building some piece of software was a shitshow because the people running said show were bad at it.
As always: for most software, the consequences aren't that bad, so a company that was really rigorous and disciplined and as a consequence a bit slower might well be out-competed by a company that's run in a bit more 'fast and loose' way.
Yeah, I was going to say that - the difference between the rigorous research in aviation/medicine and most software projects can be measured in dead bodies.
There's plenty of software that really matters though, in the same way that using checklists and things makes sense in environments where you won't kill people. We could all learn something if someone researched using, for example, an Agile methodology for building the F-35's flight control software, for example!
Edit: Thinking about this some more, I suppose my point is how to measure this:
> a company that was really rigorous and disciplined and as a consequence a bit slower might well be out-competed by a company that's run in a bit more 'fast and loose' way
while correcting for other factors like marketing etc. You can measure how quickly a group of people will get off an aircraft and vary things like the dimensions of the exit row to test whether your change has improved things but you can't, e.g., re-run a project with TDD and test whether it was "better" (defining "better" in the first place is hard enough!)
For many financial services companies the consequences for bad software quality tend to be high, either from bad calculations at scale or downtime. It’s not life and death but in many companies the pain is real and one of the many reasons for non-agile highly planned and highly process oriented operations.
Yup, I agree. If there was a heavy penalty on the business/financial level(not just on the software level), businesses would take things like software testing, proper timeline management and rigorous specifications more seriously.
Unfortunately, that's not the case. So most business just bulldoze their way to their primary objective.
> Not because agile is bad, but because the people in charge of agile are usually bad.
That's the crux of it. Currently working at an org which is "Agile" and has Scrum masters, but management still insists on measuring work in man-hours and defining requirements in huge chunks then delivering them at arbitrary deadlines. So... it's waterfall with Scrum masters who make it look nice on Jira charts.
> I am not a doctor, nor am I an expert, I have just read some interesting articles and heard from surgeons that they are trying to improve the process to reduce careless mistakes.
Also not a doctor but I've built some checklist software for them. At good hospitals there's a nurse counting every screw, piece of gauze and scalpel passed around in surgery, counting them again on the way out, counting them when the go into the dishwasher and counting them again when they come out of the dishwasher and get preped for the next surgery. Apart from emergencies doctors will have a list of instruments needed for any given operation and there will be a special package of them ready.
This was heavily influence by the aviation industry.
Pilots especially are heavily trained to use checklists
But I bet there are no methodology merchants who never fly planes or operate aviation businesses selling expensive checklist consulting, wasting the time of busy pilots with actual work to do. Like who is the Thoughtworks of piloting? Or of surgery? They don’t exist. Only software is plagued with them.
Funny enough, the practical test standards are more or less published as the test checklist, which includes verification that the person getting their pilot's license used checklists proficiently. (The checklist checks checklist checks?)
Checklists are fantastic if used in the moment by a team.
Checklists used post hoc and imposed on individuals are (IMHO) worse than useless - they give a sheen of process and quality by forcing someone to randomly ink boxes.
I don't know much about Big Law, but I would be shocked if they don't have the same sort of thing. Hundreds of entry level folks with high turnover with a simple metric (billable hours) to track their progress; it sounds ripe for some equivalent to Agile.
No such thing. Each partner runs their own show for their own clients.
Firm-wide common resources such as a docketing departments will have checklists but they are mostly software driven/automated these days. For example, creating a new matter in the docketing system will automatically generate most of the critical dates and checkoffs for that matter.
It's because software development is always harder than you think it's going to be, takes longer than you think it's going to take, and results in more bugs than you think it should. So the temptation to seek simplistic magic bullets that "fix" software development is overwhelming.
A better idea would be to reexamine why we "think" software should be so much easier than it really is. Perhaps it's because when we watch it being done, it looks like typing.
Can't speak for lawyers or commercial pilots, but doctors absolutely have "methodologies" or processes. And processes that each individual hospital dept sets up last for years.
They're usually different across depts and hospitals though.
Oh man, our provincial health region recently went through a “Lean” transformation. Talking to people on the inside, it had some positive efficiencies come out of it, but it also resulted in some pretty terrible negative consequences. For example, medication shortages in rural areas, because they figured it would be better to send it to areas that needed it Just In Time. Except... some of those areas only get weekly deliveries, or worse.
Similar story with diagnostic medicine. Smaller laboratories shut down because they were redundant, and then lead times for blood work etc go up because of travel time. We used to have a provincial bus/courier system in place, but the govt shut that down because it was losing money, and the health care system suffered for it. Last I heard, rush samples between major centres are currently being shipped by 3-hour taxi cab rides.
You get into a funny position because management philosophies, Lean included, are basically defined as "Do things that work, don't do things that don't work. Only spend money on things that add value".
So basically anything that spectacularly doesn't work is usually not 'actual' [insert method here]. Management failures are rarely obscure even in foresight.
> What is it about software development that keeps on attracting these "methodologies"?
Here is one data point for you from personal experience. A few months after got started at a US company operating in about 30 countries (primary business is manufacturing in the Casino and gaming industry) the word came from the CIO that I implement Six Sigma as part of the enterprise architecture activity that I headed. I countered that neither software engineers nor the IT architects and admins worked in the Six Sigma way and wondered at the meeting where this whole idea came from. I was told this was what the Chief Operating Officer and/or VP of Finance wanted to implement "across the company". Such things happen in this world we live in.
Software is derived from the production line/engine room model. It's based almost entirely on metaphors that assume the things that matter happen in a sequence, and there are ways to make the sequence faster, more efficient, more reliable, and predictable.
So the same thing applies to software management. And also - in different ways - to management in general.
Law is derived from the competitive sport model. The sport is verbal rather than physical, but it's really a form of mental combat with the physical violence abstracted away.
So in practice it's a combination of business negotiation, corporate politics, national and international diplomacy, book research, a very specific kind of creative writing, acting, public speaking, and psychology of persuasion, with different emphases in different areas.
You could have a process for basic case management, and I would be surprised if practices didn't. But the sharp and pointy end - the part where all the elements are used to create a negotiated contract or a court performance - is very difficult to formalise.
Software is also a really immature field. Software development is not an industrial process, it's more like craftsmanship. Part of it comes from the fact the field is relatively young, but also because in software, it is extremely easy to modify to suit new needs and to fix introduced mistakes which results in an inherent instability of design and no strong patterns.
It's getting better, but it's still present.
For example, nearly each source tree I've seen has its own idiosyncrasies in term of organization and layout.
Are the tests in ./test/ or ./tests/? how do I run the tests? where do you put the headers? ./inc/? ./include? what are the options available at compile time? how are they exposed? what are the install targets? does it support the usual variables (looking at you DESTDIR)?
Granted more modern languages tend to be more normalized than the previous examples which are valid for C/C++ mainly. But these questions still applies. For example, the Go projects I've seen tend to vary wildly in term of compile tool chain, some have nothing, just run go build, other have a ./build.sh and the better ones have a basic Makefile which is generally not so great (no variable present to override the path to the go binary or set some compilation variables, no proper install target, very weird handling of the ./vendor/ directory in some cases).
It's not that immature. I'd measure the maturity of engineering, or any field really, as it's ability to create reproducible results. That result could be designing bridges that last a predicable amount of time, cars that run for a certain number kilometers before wearing out, or surgery that produces the outcomes predicted. This doesn't mean the surgery always works, or there isn't the odd car that fails in it's first 1000km: just that we know what the rates outcomes are and we can keep below a given level for a known cost. That level is never 0 because the cost for 0 rises to infinity.
For software it's a case of delivering something that does the job asked for and that operates at below a certain failure rate. This doesn't mean a zero failure rate or no bugs: just that we set a upper limit and can reliably meet it. It seems pretty clear there are many large teams of software engineers out there that can do precisely that. Google has even allowed their engineers to write & publish books about how they achieve that: https://landing.google.com/sre/workbook/toc/ It's not rocket science - just a method of doing things team of sufficient size can replicate.
> For example, nearly each source tree I've seen has its own idiosyncrasies in term of organization and layout.
I've got some news: that happens in every engineering discipline, even the mature ones. There are some things that don't effect the outcome overly. What you name you directories may be one of them. It may be it so unimportant they leave it to an engineers discretion, or it may be an organisation can settle on any name, provided they stick to it.
> But these questions still applies.
No, it doesn't. If you look at that site reliability engineering handbook, it's not about naming directories. It's mostly about measuring failure rates, and then applying well known techniques if it is getting too high. The well known techniques are mostly about change control, which happens to be a very boring subject most young software engineers almost universally hate. Trying to impose change control on a young software engineer is like trying to put a lead on a cat: it will do anything to avoid it including pulling in the reverse direction to where the lead is going, playing dead and refusing to acknowledge it's possible to move at all with the lead on, and attacking the person trying to put it on.
Also, there are big differences in the paradigm used. Sure OO is dominant in enterprise software, but there is also functional programming and many others that have a big impact on organization.
The fact that software development is not something most organizations understand how to do (well/at all) but still insist on trying to do anyway. Unfortunately, this means that a significant percentage of their projects will fail. Changing out methodologies/tech stacks/languages are great way to redirect blame when things go wrong.
aren't hospitals infamous for their broken processes? It doesn't seem to me like forcing doctors to work 24 hour shifts improves their judgement while on the job, or improves overall health outcomes, anyway.
> What is it about software development that keeps on attracting these "methodologies"?
Nothing. Management theories exist around managing every specific kind of work, as well as generic theories. Software development, HR, and other kinds of work that are done in (or contracted for by) almost every organization of non-trivial size probably get a bit more because more people are managing them. Many of the theories in software development are direct adaptations of process theories from other disciplines. Waterfall
—before it was named—was just application of practices from non-software engineering, Lean is from manufacturing, kanban is a specific practice from the manufacturing organization whose processes inspired Lean, etc.
“Agile” started in software but it's everywhere.
> Why don't doctors or lawyers or commercial pilots have people coming along with a methodologies for doing their jobs?
Note that the theories around software are often the theories of process improvement rather than the daily work in other domains (interestingly, this is not true or kanban), because software is just automated process so software development is essentially the exact same task as process development/improvement, where some portion of the process is executed on a computer.
This tends to be the NHS learning about something and implementing it themselves, rather than hiring a management consultant. That's because they want to protect public money and there has been a lot of bad experience of management consultants for this kind of work in NHS organisations.
Software development is an incredibly immature professional discipline, which has lead to people mostly making it up as they go. You also have the rate at which it changes, making it hard for any sort of standard practices to keep up with. Outside of some industries that regulate security and privacy, software development has also never attracted the attention of any regulator that may have an interest in actual quality of workmanship (which would be impossible in any case). Can you imagine the architecture equivalent of shipping terrible code? “This software has bugs” is acceptable, “this brand new building is falling down” isn’t. There’s a standard way to build a house, there’s a standard way to perform heart surgery, there’s nothing close to a standard way to build software, and if you can’t properly plan your work, how can you properly manage it?
That seems to be the problem these management fads are trying, and failing, to solve. People hated waterfall because you have to do so much planning up front, and in software development, that process rarely works. Project requirements change, or planning assumptions are proven wrong. So you need a management framework that works without that level of planning. That’s the core of agile in my opinion, only plan a little bit of work at a time. Really I think it just leaves you with the same problems, only now you have a lot of small failures rather than fewer large failures. If you look at the agile shops that succeed, they tend to have good leadership that simply has the competency to plan ahead well. Those teams would likely succeed under any management framework, only with agile you now also have the benefits of continuous delivery.
having done software for scheduling air crew onto planes, one of the biggest factors for them are the various aviation authorities rules / airport certifications etc. That shapes a lot of their lives, but airlines do have their own methods for trying to get the "best" out of their staff.
Methodologies are attractive for situations that are ambiguous.
Lawyers and pilots have a clear cut set of procedures. Pilots aren't inventing new technologies each time they fly, they're just driving the bus from City A to City B and worrying about occasional edge cases. Lawyers follow legal procedures and have their own built-in rules and processes; they're tested on these processes to pass the bar.
You see methodologies for things that vary wildly like sales, military leadership, and of course, engineering tasks. Each project, situation, or client could be extremely different, and it's not possible to build a prescriptive how-to for each scenario. So you create a methodology and framework so that managers and leaders can quantify uncertain situations and build (and alter) their own how-to's.
Doctors, lawyers, and commercial pilots are licensed professionals.
Doctors and lawyers are personally liable for their screw-ups. I don't know how liability works with pilots but they will lose their license if they screw up.
By operating within the medical standard of care, doctors diminish malpractice liability. So having well-defined systems that help prove that you follow the medical standard of care is important.
Lawyers don't have anything like that as a whole. Each lawyer handles their clients their own way. Even in the same firm different partners will have different practices. Some use checklists for their staff to confirm that the certain required forms have been filled out or provided.
In the actual work of lawyering, checklists are not a thing I have seen. Though there are "outline" documents to help with bigger projects.
They all have methodologies. The pilot's checklist that was already mentioned is just one. For doctors you have different paths of treatment that can be standardized for years, then undergo major changes when new information is discovered. With lawyers it depends on what they do. New approaches to cases, new discovery methods, everything is a process and everything can be improved.
Software development attracts these things because software development isn't as firmly entrenched in tradition as doctors or lawyers are. There are way more people that are open to implementing ways of accomplishing tasks that are more open and efficient than there ever could be in most other realms.
As the demand for computer automation increased, the market produced tools that less intelligent people could use, like Visual Basic vs C++. Management fads infect the dummies. How many lisp nerds do you see at an agile circle jerk meetup? Lawyers and doctors still have an intelligence test in the form of admissions criteria to universities, programmers don’t.
> Lawyers and doctors still have an intelligence test in the form of admissions criteria to universities, programmers don’t.
This is both incredibly rude and pretty much irrefutably true.
As a startup cofounder I've had the experience to hire like 3.5 programmers to do front-end work in the past year. I'm not at a specially "hot" location for developers, but my n=3.5 anecdata is that the few intelligent programmers you can find have terrible work ethics and continually try to spend more time with what they find interesting and avoid drudgery work. The hoi polloi (who might have made damn fine designers, copywriters, etc.) rely on stack overflow for whatever they can, and dither on the slightly more challenging tasks they didn't know existed.
Besides filtering on IQ, one thing a nerdy education teaches you is that challenging problems are often solvable if you will just [wo]man up and try.
Law as an industry is pretty dysfunctional. Plenty of data governance issues, losing track of documents, etc. A lot is at stake there, for sure, but that's actually the thing that process helps with - reliability and safety. And law is very resistant to incorporating that.
Lawyers feed on complexity and obscurity. If things made sense and the processes were logical and simple and easy to navigate, nobody would need their services.
They are also not asked to be innovative. If the software industry decided to stick to one stack for the next few decades we would also get much better methodology and maturity. Instead we have this constant churn of new stuff where we never learn how to use our tools properly.
Hmm... actually, process and outright automation (the apotheosis of process) work brilliantly for certain types of work.
Got something boring and repeatable? Process / Automate the daylights out of it. You'll be better off.
Anything that requires innovative thinking or customer focus? Requires fitting square pegs in round holes? Probably isn't going to go so well...
Speaking as someone who spends more time trying to manage the former (boring + repeatable), independent humans are generally the key point of failure in any process.
After being a manager, I'd say that is not a wrong goal. If your team's success depends on individual heroes, then it is a fragile setup. People can be absent for a lot of reasons - changing jobs, parental leaves, disabilities, ... A good manager should put in place appropriate processes such that even if someone leaves the team, the show goes on smoothly for everyone else.
Manager of what? Some things require skill, some things don't. Low skill trades can be transformed into paint-by-numbers processes. High skill trades cannot be so easily encoded into a process.
If thing explode and you can't do anything about it then you are clearly in a cargo cult, not under a management that understood the underlaying philosophy; it is a glorified flow chart and you have to come up with your own practices for each step unique to your own process(es).
It seems to me that if you're an IC in a large team, you are a fairly replaceable cog.
Of course a large percentage of the cogs of any machine fly off at the same time, you're in trouble. But I don't think one IC matters much in any large organisation.
There's some really sound ideas in six-sigma as well as the other practices that evolved from manufacturing like Lean, Kanban, TQM, etc.
These approaches work, for real.
The problems start when they become excessively dogmatic, or as some say cargo-cultish. These systems are intended to help people solve problems in an orderly way, they're not a replacement for common-sense judgement, and original thinking (which, <gasp>, is STILL NEEDED).
Sadly, the personality types that tend to go into supply-chain and manufacturing (and that's where I happen to work) tend to be frequented by rigid-thinkers who crave process and prescriptive "checklist" approaches. Whatever new thing these people take up, it's eventually ends up as a turgid mess of dogma.
I like to think of it like an evolving idea passing through people. The methodologies evolve to take advantage of certain human characteristics which aren't always positive and often revolve around controlling others and perfectionist order. People get sucked in by the enticing promise of better but only if we change the process here and here which leads to an enormous amount of time talking about how you're going to talk about how you're going to do your work.
At least "waterfall" never existed, it was invented by people to describe something somebody else was doing.
The design-build-test cycle necessary for physical products where the build and test phases can take days, weeks, or years was being used for software where build/test can take seconds or at most hours -- faster design cycles are beneficial and lots of folks used to engineering things got stuck with really long cycles. You don't need a management framework to accomplish those faster cycles.
In most cases what you need is management permission to move quicker. Everyone says they want to be agile, but doesn't want to invest in the tooling (unit and integration tests, build servers, etc) that would let you be agile, so you end up with agile overlaid over the waterfall methodology, which does absolutely nothing.
I would argue that management often doesn't truly understand "Agile" beyond "software engineers should operate this way." In my organization, the engineers are willing to be disciplined with Agile practices, but product management has no interest in real planning that would achieve a manageable backlog across releases. Instead, they change release scope numerous times, and nearly every release cycle runs long because we never finish what we start--by the final iteration we are working on entirely different stories than what we groomed in iteration #1. To them, "agile" means "do what we want whenever we want it."
Isn't that quite well in accordance of Agile Manifesto's "Responding to change over following a plan"?
Sprints and backlogs and grooming, all those sound they're straight from the Scrum Book. And while I understand that for many Scrum == Agile, actually it's not.
Note that I'm not saying anything about what's the best way to do software. Only that "responding to change" sounds very much agile.
This sort of thing happens all the time. The Japanese-style quality movement had many different fads (for example 5S) but most of the top denizens said that there are some basic universal facts about quality but every organization has to develop their own system that works. Consultants can help you do that but you can't follow a textbook and make a company. By contrast, the American quality movement has always been filled with charlatans and fads like "Zero Defects". Do Americans want religion and not mastery? Who knows.
I see Scrum a lot like I see Six Sigma. They take some of the processes you see from a successful team and they turn them into gospel for managing all teams. Sometimes they try to apply these processes to the rest of the company in places where they don't make sense. They create a bunch of certification levels so that they can claim you can't just learn it from a book. They make a bunch of money but leave their clients in a strange place.
That's my organization. Most of the actual work doesn't go through either process because the engineers know if they start discussing things in the sprint planning a manager will pick it up for a multi-year "investigation".
Fortunately with cloud and open source tooling, there isn't anything to buy. And management is already excited about cloud because they read it in CIO magazine and their buddies mention it on the golf course.
The only thing needed is training on proper usage of the tooling. But that can come about organically when companies accidently hire a few competent engineers.
That's what happens to everything that management gets their hands on. Agile is slightly different though, it was corrupted as a concept and commodified long before the big guys decided to embrace it.
I was frustrated at work about whether our "Agile" practices were really agile, and then the Rick Sanchez voice in my head spoke up: "Look, Morty, nobburpody gives a shit how 'agile' you are, or whether 'stories' actually meet the qualification for user stories, or whatever other horseshit you can think of. No one gives a shit about quality, either. All that really matters is that you make the ink on the bottom line black and make your boss look good. You could literally be shoveling out buggy shit with waterfall and no one will complain as long as you meet those criteria. Don't like it? Tough titty, motherfucker! Welcome to wage slavery! Now swallow your food pellet and get back to work, those giant capstans aren't going to turn themselves."
This is interesting to me, because I work in a "methodology" business -- except in our field, everyone understands that properly implementing Earned Value Analysis requires the combination of:
- Good people, who know what they're doing;
- Good processes, that support those people;
- Good tools, which enable the processes
There's been no effort (of which I'm aware, anyway) to turn this into a cargo cult. EVMS complex, and takes a lot of work to implement, but it has real & demonstrable benefits on large-scale projects. So it gets to stay a "real" thing and avoid the weird cargo-cult transformation that's plagued other ideas.
Certifications and systems and methodologies can make good leaders better, but I can't say I've seen an example where they've turned a non-leader into a leader.
Maybe an example of good leadership, but not an example of leadership. You don't need to earn leadership these days, you can purchase it or be born into it, especially at a large corporation.
The goal is to make sure you are gone before the title roll back around to the original titles. I worked in a aerospace firm and the took all the R&T (Research and technology because development was done by production) titles and changed every single one with innovation. Then about 8 years later, they rolled all of them back, killed all innovation titles and put R&T back into the titles. Net result, not much as most of the same people were in the same positions.
Actually it amazes me how many highly intelligent people do not realize benefit of visualization every little detail of work being done (because planning is hard), checklists (because people forget), communication and shared vocabulary (because projects can be huge and involve a lot of people).
Not to mention control of execution, because projects are late one day at a time.
Even Deming said that you can be focused on eliminating defects and fail to keep the market share, fail to delight with your product development. "The most important figures for management of any organization are unknown and unknowable."
A lot of times, management fads are a procrastinating activity for the organization. The real problem is they are lacking direction. Imagine Apple, circa 1998, going 6 Sigma instead of becoming product-focused. They would get more and more effective at producing a computer fewer people want to buy every month.
Look at what Juran writes about the development of the Ford Taurus, the car that really pulled Ford out of a slump in the early 80s. How much of it is focused on preventing defects? No, it's about designing quality into the car, defining quality as the things that make the car something people want to buy over another car. Doors fitting tightly by having low variation in size is just one aspect, important but not enough.
For Tesla, quality is a 200+ mile range, but also a bunch of other things, without which people wouldn't pay for the battery. I assume that Tesla has quality control but the stories of disorganization in the plant suggest that it's not running at a similar level to the Japanese plants, which also produce millions of cars people want to buy. Nevertheless, they are focused on what makes demand for the car.
Also price - with disruption (in Tesla's case) on sticker price vs. TCO.
The Taurus could have been every bit as good as a Merc or BMW, but if it had been at a non competitive price point Ford would have had a problem. Being able to deliver at a good price is every bit as important as delivering the quality and the image.
I worked at a company 8 or 9 years ago that kept hiring all of these Six Sigma Black Belts who sat around _doing nothing_, earning high salaries and making all of the wrong moves. Didn't stop our stock from sliding into the sub-dollar range.
Had the same thing happen at a company I was first hired at as a developer back in 2008.
They hired a director from outside the company because Six Sigma was still kind of a big deal and he was a black belt. It took them two years of consistent failure to figure out Agile was the way you did software development, not applying Six Sigma industrial manufacturing principles to software development.
The amount of analytics and metrics we were responsible for was insane under that director. I lasted two months before I tapped out.
from the link "it's an idea". I notice that evidence is mentioned further down, but evidence that this will work, has worked or might work in the future, well, I see none.
SUN used to be like that in its final phase - six sigmas everywhere, people teaching how to use "neurolinguistic programming" to achieve one's goals, cool zen-buddhist priests in management, and the bottom fell out...
Project management is hard. It will always be hard, because it involves working with humans. But nobody wants to admit that; we want easy answers. So we will chase fad solutions until they become popular and everyone realizes that project management is still hard. Then we'll move on to the next fad solution.
What I want to know is: when does agile go the way of six sigma, and what is the next PM fad to replace agile? There's lots of easy money to be made as a consultant training people on the next big fad.
Six sigma wasn't really a project management solution. It was an iterative process framework specific to manufacturing where the number of defects would be measured, the causes of common defects were identified, then resolved. The process would continue to iterations until the number of defective widgets produced was reduced below two-in-a-billion.
This remains a very effective approach in large scale manufacturing. Many engineers still learn about this in school; if not explicitly six sigma, then at least the statistical underpinnings and application of it.
Six sigma was a victim of its own success. Business leaders tried to expand and generalize this framework to apply to different areas of business. This doesn't work well because building a billion nearly identical widgets is nothing like managing a team of accountants.
I don't see this as a bad thing even though it's framed that way. As you said, project management is hard and the field isn't that that old, especially in tech. Processes and techniques will come and go as we learn from the old and build up the new. One might even say we ~iterate~ on them.
Six sigma is a manufacturing process that was shoehorned into business and software development. There were some great concepts that probably influenced later methodologies, that's far from being a useless fad. You wouldn't call the sledgehammer a fad because we have jackhammers now.
To tie the analogy together, these are just tools. We would both probably agree that having the best tools don't matter if you don't have good, well-trained people behind them.
I'm not saying methodologies are useless; having some structure is better than winging it. But they are absolutely fads. Nowhere is this more clear than in the terminology. Black belts? Sprints? Punting? Scrum masters? That's pet rock level fad stuff, and I say this as an "agile" PM... who was at one point was encouraged to get a six sigma black belt by management chasing a fad.
I've been on dozens of different teams at small and big companies and agile implementation has been different on every team I've been on. It has a good way way of just being a template and teams usually implement their own style to make things work for that unit. For that reason I think agile will be around for a while.
There is no magic bullet for PM. The truth is estimating how long X feature will take is hard. Its hard for product managers to for see everything and this leads to incomplete requirements. Which leads to inaccurate estimates and pushed out timelines. Even when requirements are fairly thought through, things always come up that no one saw. Shit happens. The key to being a great PM is making sure the team is always working on whats most important and then managing expectations with senior leaders. Takes some blunders to fine tune that skill though.
Project management isn't trivial, but it isn't outrageously difficult either. What makes it difficult is when you need to spend hundreds of hours a week making up numbers that bear no relationship with reality in order for someone above you to put a green box in their powerpoint presentation.
I think it should be noted that Six Sigma is mostly (originally?) aimed at processes which are not projects, such as factory operations. There's my point of view there's a big difference.
Well terms like extreme programing, sprint, scrum/scrum master
are also used unironically today. And claiming people over process equally hilarious where devs stand like chump each morning to provide banal status update and spend rest of the day as death by JIRA.
We need a system that doesn’t require project managers, at least not as a full-time position.
I don’t think they’re all that necessary. I think they’re a byproduct of considering developers as high-functioning autists not capable of managing themselves, and we’re in an emperor’s new clothes scenario.
All of the PMs I’ve met have either been wholly non-technical, or they were allegedly technical 20 years ago and they have no understanding of modern technologies. Useless either way.
I think by getting rid of project managers, and the managers of project managers (who are even more disconnected from what’s going on) companies would save themselves a lot of money.
Senior developers could take on a management role. But that’s the key, they should be developers, not people who have a PMP certificate and don’t know what a compiler is.
You have a massive misunderstanding of what PM's do, likely because you only ever see one half of their job when you're trying to explain how something works and they're not understanding why it's needed. Trust me, if you had to deal with the OTHER side of a PM's job (dealing with upper management and answering questions about why you're three weeks behind on deliverables without giving technical explanations to people that DEFINITELY have no clue) you'd very quickly decide that it's better to have a meat shield between your team that needs to get shit done and a corporate structure that wants nothing more than to shitcan all of you and outsource you all to India.
What you are saying is correct. But it also shows how dysfunctional a lot of management structures are. Not too long ago I had a few discussions with my VP and it was kind of shocking how little he knew about what’s going on at the ground level. The information he gets is quite distorted by all the levels of management in between.
Meat Shields work both ways -- it's also to shelter mgmt from the sausage making process and to keep clusterfucks from taking up the time of higher level resources and managers.
When you have a great project manager who is absolutely rocking it, its a dream. They are helpful inprojects where you lack face to face contact with lots of stakeholders.
If they're managing a technical project they better be technically inclined.
I'm sick of having PM's who have no tech chops calling shots for big ticket development items. An MBA doesn't mean you are capable of leading a team of engineers.
You can't manage a project if you have no idea about the effort and risk associated with each piece of work. A typical job for a PM is to negotiate solutions with a customer, in that case they need to estimate work items relatively accurately in real time to do a good job. Non technical PM's will either make up bullshit estimates and then pressure their developers to follow through or they will be very bad at fulfilling customer needs since they constantly need to verify everything with the devs.
> Deming’s methods became systematized into Six Sigma
I think Deming and anyone who has seriously studied his work would take exception to that. Most of the underlying mental models are missing from Six Sigma, and all that remains is a watered down version of Statistical Process Control 101.
Yeah, I think this is really missing the point of Deming. While statistical quality control was a big part of what he taught, he was almost more interested in the philosophy of management than process management. Everything you read about him talks about statistical quality control, but nothing ever mentions his Fourteen points, or Seven deadly sins. I read about him as a kid, but alot of it has stuck with me through multiple careers, and his Fourteen principles could easily be applied to most software and engineering teams and companies.
Things that are really key to his philosophy (and deeply impressed me) were:
* Cease dependence on mass inspections for quality control (Do you rely on manual testing for every release?)
* Break down barriers between departments (i.e. devops)
* Eliminate slogans and quotas, goals, targets etc. (Maybe try the release train process for shipping software instead?)
And my personal favorite:
* Eliminate barriers to pride in workmanship, for hourly workers, management and engineers (is part of your team embarrassed by bad UX or known bugs that they can't fix because they are busy with other work?)
Deming's philosophy basically wrote an entire ad campaign for Ford, "Where quality is job #1."
The way Six Sigma was presented to me by Mike Harry was that it is a tool for finding out where to apply your effort. I've not been involved in it for a long time and am sad to hear of his death. The great thing about it as he presented it was that it did not require a great deal of sophisticated tools or a lot of statistical knowledge just a spreadsheet and an appreciation of multi-way analysis of variance.
When I was working with engineers using Six Sigma in the late 80s, early 90s, Six Sigma was not really to do with day to day statistical process control. It was all about finding assignable causes of variation and how variability in each part's properties contributed to the variability of the whole. The connection with process control was that it helped you decide what tolerances one could allow. In several cases the Six Sigma investigation resulted in widening the tolerances instead of narrowing them as it was found that they were pointlessly tight and resulting in more costly processes. In many others it resulted in a change to the process, adding steps, replacing parts, etc., and not just an adjustment to the process' parameters.
There are some sound principles in Six Sigma. There are also some gross misunderstandings and optimizing towards meaningless metrics just because it makes things look good in the Six Sigma world.
I’ve been an engineering lead in an organization which worked by Six Sigma. It provided a nice, standardized framework and a focus on metrics, but I’ve seen people again and again take absolute shit and “optimize” it to meet the metrics instead of fixing the underlying issues. I’ve seen another person (VP level, former professor) get to very illogical conclusions because Six Sigma methodology said so.
So I’m all for adopting some Six Sigma principles as an underlying quality process, but please use sound judgement together with it.
"And as with all fashions, once Six Sigma was picked up by the masses, fashionable companies lost interest and moved on to the next big thing. “These things have a life cycle: They get popular and then people start looking for something else,” says Art Swersey, a professor emeritus of operations research at the Yale School of Management. “These things run their course, and it has run its course.”"
So says a friend of mine who has written two books on Six Sigma and for a few years was consulting full time in QC. He told me: "At its core, Six Sigma is a set of very old, well established tools encased in a slick marketing shell. There is nothing revolutionary. Inevitably, Six Sigma fades away and something new will take its place, using the same old tools."
"Lean" and "Lean Six Sigma" seem to be following the same path -- and to some degree "Agile" too.
Big companies like buzzwords and "processes" that'll fix all their woes... except for the elephant in the room that everyone at the coalface knows about.
Scott McNealy was a protegé of Jack Welch, and Scott applied the Six Sigma philosophy to Sun. There was a whole corporate effort to get people trained up as black belts, etc. During the mid-2000s when the company was struggling, it was quite telling that the first people laid off from any organization were the Six Sigma Black Belts.
Six sigma has nothing to do with GE's overall performance. They are ruined by their insurance exposure. Not just Markopolos thinks it's already bankrupt, Buffett sold his stake several years ago because he didn't understand what's going on with the books.
If this ship sinks some people will pick up the good engineering pieces going on sale. "Six sigma" may still be effective there. But it has little do to with GE stock.
> The issues outlined by Markopolos lie primarily in GE's troubled Capital unit, a financial services division often seen as a black hole in the company. The Capital unit holds commercial and personal loans, as well as insurance policies that include coverage of long-term care.
> In his report, Markopolos suggests that an accounting rule change for insurance liabilities and a significant lack of reserves to cover long-term care liabilities will push GE to take a $29 billion hit. While GE called the report meritless, an analyst said the allegations reflect "a GE culture that historically hid losses and deceived investors."
The goal to ensuring quality was to see your business as a repeatable process. That way you could apply statistical methods, measure and optimize.
The flaw with six sigma was that it is applicable to only very repeatable manufacturing processes where the defects can stem virtually only from random variation in environment. Eg. room's temperature changes, worker sneezing, etc. In this case you are expecting many small probability defects compounded together, yielding a normal distributuon by the central limit theorem. That way you can apply all the statistical methods around the gaussian distribution and try to eliminate 'six sigma' of defects.
The flaw is that people tried to apply this to highly explorative disciplines like software engineering, where the process is very far from being normally distributed. As Nassim Taleb likes to explain, most of the human processes follow an exponential distribution. In the case of software defects, you tend to have many small ones that nobody even logs anywhere and just fixes them (thus being invisible to anybody (PMs) trying to find a pattern). And the other category which are defects where you go down a deep rabbit hole to understand and the fix might require huge rearchitecting of the app. There is no normal distribution, averages on exponential distribution don't make any sense and delivery estimates don't either.
Part of the reason is why lean and agile are the only successful methodologies in software is because they treat it as a discovery process which eventually can be turned into highly repeatable process - by that point operated 90% by a well designed machine.
Many parts od the industry still got stuck with the flawed analogy between software development and construction business. They posit that both should have an architect to design the system and then an army of low paid workers to execute on the plan. The flaw is in thinking that the software developer is the 'worker' and not the architect. He is actually the designer and the worker in the analogy is the compiler, which performs its job so quickly, that project managers don't even see it to include it in their gantt charts.
It's also sad to observe how the certification business is turning the original lean and agile ideas into another black belt type cargo scrum cults.
It helps going back to the roots and listen to Deming, Juran, the guys around agile manifesto and especially system thinkers like Ackoff.
> Part of the reason is why lean and agile are the only successful methodologies in software...
I would highly disagree with a blanket assumption like this:
An discussion on HN [0] discussed the article Waterfall (
Everybody likes to laugh about the Waterfall methodology) [1] and I highly recommend both the article and a discussion as a deeper reflection of assumption of 'Agile vs Waterfall (or anything else)'.
BTW, I keep a small personal wiki of articles from around the web that I find interesting, just in case I might come across as personally invested in any topic; I find the discussion enlightening.
When I was a kid, my father worked at a place that was pushing the "Five Nines" methodology for manufacturing. I'm guessing the next management fad should be Seven Something.
I've had TQM and Lean inflicted on me, but this (quoted in the article from a satire) made me laugh out loud : ""embodies a pillar of the Six Sigma business philosophy: teamwork, insight, brutality, male enhancement, hand-shakefulness, and play-hard.”"
There I think we have the red queen challenges of modern western business culture laid out for all the see.
The idea of Six Sigma never made sense to me. I read a book on it, and it basically said that you should optimize a business so that mistakes are extremely rare (the six sigma refers to that point on a N(0,1) gaussian cdf).
The idea that you should try to avoid making mistakes seems like a good one, but also is rather obvious, and the right threshold is also obviously problem dependent.
I suppose the idea is kind of fluff, but of all the fluffy ideas out there, trying to be reliable and dependable is probably one of the better ones.
Don't feel bad. Green Belt offers a neat set of tools that can help you prove something to a manager in the future. Black Belt just continues to build on the same information.
Do popular methodologies suffer from popularity? Is the opportunity to turn the methodology into a brittle, monolithic and successful enterprise perhaps just too tempting? Before you know it, the high principles and ideals on which the methodology was founded are now helmed by corporate sponsors at HP, Lockheed and Capital One. Rust gathers, staleness sets in, since the point of the methodology is no longer substantial or revolutionary improvement, but simply preservation and perpetuation.
People started realizing the truth: Engineers who can do engineering, do engineering. Engineers who can't do engineering, become Six Sigma consultants.
> Silicon Valley’s culture of “move fast and break things” meant business leaders were less concerned with reliability and more focused on game-changing discoveries.
This too is a pendulum. And I for one am looking forward to the pendulum swinging back just a little closer to reliability. Especially in terms of security models and secure by default innovations.
i think we should be grateful to FB/Google/AMZN/etc. for not coming up with (nor publicly embracing/promoting) any such new madness - I mean giving the copycat "monkey see, monkey do" nature of the management we'd for sure get a replica of whatever these biggie ones would cough out like we got for example the "cool" open floor plans which the FB/Google use to dampen down into the manageable range the otherwise stellar performance of their elite workforce :)
After having survived and successfully recovered from the [pretty devastating in some cases] damage by the most recent process-first innovation hurricane that hit our industry several years ago - ie. the Agile/Lean/Scrum, the industry has been living in a kind of mini golden age since then - the mentioned big [super]successful companies have been producing technical-first innovations (from which some process innovations like DevOps do get derived), and thus management has been naturally copycatting the good things like embracing of cloud, Kubernetes, AI/ML, [micro]service oriented architectures (don't mistake it with the first monster of SOA of 15 years ago :), etc.
Making large org-level changes to milk efficiency out of existing processes is capital intensive, and companies shortchange their own efforts to save money. Changes through Six Sigma are expensive to implement.
"Agile" has promised cheaper improvements, and companies ate that shit up.
I used to like six sigma. I worked with an IT director who pushed it bigtime as he came from a large automotive supplier for the big 3.
So much of it made sense. The 5 whys or really just 'get to root cause' is great. Fix the actual problem is way more intelligent than Ticket #348584573 of the same thing.
The problem is that sometimes you break the scripting rule #1. https://xkcd.com/1205/ They'll spend a week analyzing something they don't have enough information about why it failed. It's the first time it ever happened and won't happen again for many years and the fix was literally just restarting a windows service on a server.
Devops in a large enterprise means you can finally get some money for your CI/CD tooling. And some new plush positions for DevOps facilitators/evangelists.
The exact same thing is happening now with 'Agile' where big corporations are switching to sprints and hiring scrummasters, and renaming departments - mostly as a way to stay hip and cool. But at the end of the day, there is no business framework that will replace good judgement and discernment.