I saw an interesting post on The Register about a study of agile projects. I reviewed the blog posts and I have a few thoughts. First, the study!
The study seems to have been done by surveying 600 software engineers in the UK and the US. The blog post didn't mention how many companies were covered by the survey. The Register's post was quick to point out that the EngPrax post seemed a little bit like a marketing post to boost a new methodology called "Impact Engineering". I'll include the link anyway for completeness. Out of curiosity, I searched for "number of software engineers in the us" and a result from Wikipedia indicated there are over 4 million. The survey group is so small that I cannot believe any numbers coming out of this study are representative of the whole.
The interesting part about the survey is that there seems to be supposition that projects failing is a bad thing that needs to be avoided. I couldn't help but wonder if the agile methodology was actually working as designed.
I've worked at a number of larger organizations. I've been on projects that should have failed. Those projects were driven onward at a high cost of money and morale. A person (a VP or C-suite person) was usually attached to the project and their reputation was at stake. They authorized the project and just kept pushing no matter what changed or what new challenges came up. For them, project failure might equal a career setback.
On the idealized agile side, you're trying to deliver value to the business quickly. It is true that many people believe that agile means "start work with no plan". I believe more people (myself included) don't equate "agile" to "no planning". Agile, in my opinion, has room for as much documentation at the project dictates. An insurance application, for example, has many regulatory requirements and extensive documentation may be needed. Other applications may be more intuitive or conceptually simple.
I have also worked on projects that seemed like a great idea that never saw the light of day. The market was up, the company was growing, and it was time to create a new product only to have a merger, pandemic, or some other event take the wind out of our sails. On those projects, we spent a lot of money and had nothing to show for it. The business never received any benefit from our work. Development teams in that position are obvious choices for being laid off. Those teams cost money and aren't providing value yet.
My current team has already experienced building a very useful project over the course of weeks that was shelved due to a lay off event. Not only did they lose friends and co-workers in the lay off, they were frustrated and disappointed that all their hard work was for nothing. Now, we do our best to deliver value to the business every two weeks. If something changes, we can immediately pivot to more valuable work and deliver more value every two weeks.
It is also worth noting that if a company is not staffed correctly, no methodology in the world will save your projects. If a company lays off their project managers, agile practice leaders, scrum masters, quality analysts, and other roles, you can't really "agile" right or "waterfall" right.
When you're trying to do more with less, it is more important than ever to build relationships around your organization, build useful software quickly, collaborate with your users, and respond to the changing needs of the organization.