How to Prevent Agile Initiatives from Failing

In my previous post I highlighted a number of patterns that lead to failures of agile initiatives. The post generated a lot of conversation and a few comments, so instead of responding to everyone individually I will address all of the questions and comments in this post. Let’s look at each fail pattern and discuss strategies to prevent agile from failing.

*** Disclaimer: This is not an exhaustive list of solutions. There are many ways to address these fail patterns and there are many great blogs and books that focus on “doing agile right”. The solutions below are simply solutions that I have experiences with.

The “IT Only” pattern

Problem:

In many cases, only the IT department is an active participant in the agile methodology. Sure, there is a “product manager” involved, but not with the full support of the business. This pattern often emerges when agile is a grassroots initiative from IT development teams that has garnered little support from the rest of the organization. The end result is that the business does not understand and buy into the MVP (minimum viable product) concept, and important tasks like creating roadmaps and grooming wind up looking like large, time-consuming meetings because the business is not educated on how to participate in an agile environment.

Solution:

My recommendation for any transformational change initiative is to start small, establish a core team of winners, pick a project that has a chance to succeed, deliver, and then evangelize. Let’s dissect this.

Start small

Starting small increases the chances of success. Instead of declaring that the entire company moves to agile all at once, start by focusing on one product and start the learning process. Getting the business to buy in is critical for the success of agile. Pick a product team whose business and product owners are most likely to get on board or pick a team that is in desperate need of drastic improvements. Sell the business on WIIFM – “What’s in it for me?”. The business may not understand terms like grooming, velocity, and sprints, but they do understand terms like improved quality, speed to market, and customer satisfaction. Sell the vision of the future state in business terms with no technical jargon.

Good vision statement:

• Our goal is to improve customer satisfaction by delivering software more quickly and by consistently meeting our delivery dates.

Bad vision statement:

• Our goal is to use time-boxed, fixed, scheduled iterations to improve our velocity and increase the number of story points we can deliver on

Pick a team of winners

Assemble a team of people who are positive and have a track record of delivering. Negative, pessimistic, and dominant personalities are not good selections for the pilot team. Pick team members who are enthusiastic, flexible, open-minded, and who welcome a better way to deliver software. If possible, choose members from different areas of the company to increase awareness and visibility. The team members will need to be trained appropriately. Sending one person to a class or having them read books is not enough. The team should attend training as a group and work through exercises as a team with a seasoned coach/trainer.

Pick a project that can succeed

Teaching organizations how to learn agile software development is similar to teaching your son or daughter how to drive a car. On day one, you don’t let your child drive down the interstate. Instead, you start small and safe and teach them in an empty school parking lot and focus on how to use the gas and brakes. As they catch on you start teaching them how to park, back up, signal, etc. Next, you teach them the rules of the road and get them on-road experience. Eventually they will be driving on the highway.

Try the same approach with agile. Start with a smaller project that has very few dependencies on other projects and products. It is a good idea that the first project is made up of people from the same location, since distributed teams add another level of complexity. New products are good candidates because they don’t have the added complexity of support and maintenance to deal with. In choosing the first project, make sure the project is relevant so that the business gets the benefits of the project’s success, but stay away from mission-critical projects to minimize risks.

Deliver

It takes time and practice to adapt to any new process. Before expanding the agile initiative to other teams, it is a good idea to get good at delivering first. Use retrospectives to gather data and learn how to get better. Retrospectives are used to discuss what went well, what did not go so well, and what can be improved. The key is to take action on what can be improved so that the next sprint is better. The learning process never ends.

Evangelize

Once the pilot team starts having success, they must be the evangelists for the rest of the company. The team can blog about their experiences, hold brown bag sessions, speak at team and company meetings, share their experience on internal social networks like Chatter, and use any means of communication to promote the further use of agile. Evangelizing is not just for IT team members. Product managers and business owners can have a great amount of influence by promoting the value of agile and sharing success stories.

 

Wagile pattern

Problem:

This is the most common pattern for agile failures. In most cases, the company is transitioning from a waterfall methodology to agile and just can’t break old habits. They use the agile terms like grooming, scrum, and daily stand-ups, but they still produce work in sequential order. First the business creates requirements (they call them user stories), then development writes code and throws it over the wall to QA. QA is rushed to certify the release because they are getting the build late in the “sprint”, and then they throw it over the wall to operations, who has to deploy the build in the final hours. Other than the terminology, there is nothing agile about this approach.

Solution:

Make sure that the team has some experience with agile. A team made up of people with only waterfall experience will likely struggle with agile. I highly recommend the scrum master and the product manager have at least some relevant experience with agile. If these two leaders are accustomed to waterfall, it will be a challenge to change old habits even if they have good intentions. If experienced agile product managers or scrum masters are not available, bring in an agile coach to work with the team through the first few sprints.

Unplanned work pattern

Problem:

Another challenge companies face is that key resources are often pulled off sprints to fix production issues, answer questions, or perform other tasks that cause them to miss their dates. To accommodate this, either the scrum master reduces the number of hours they can allocate the critical resource or the critical resources start padding their estimates to account for lost time. The end result is that projects always take longer than they should because it is hard to predict how much time these critical resources can dedicate to each sprint.

Solution:

One of the biggest challenges dealing with unplanned work is determining what unplanned work must be addressed immediately versus what can wait until the next sprint. It takes a disciplined product manager to deal effectively with unplanned work. The first step is evaluate each issue to determine what is a true emergency. An issue is only an emergency if it must be fixed in the current sprint. Over time, a team will be able to set a baseline for the amount of time required in each sprint to handle true unplanned work. If the amount of unplanned work continues to increase, the product manager should consider dedicating a sprint or sprints, or a part of each sprint to fixing the root cause of the issues. Another option is to increase the team size by a resource or two, but be careful—if the team gets too big, it could lose its effectiveness.

No architecture pattern

Problem:

This one is my biggest pet peeve of them all. Some teams think agile means “start coding”. They believe that architecture evolves over a series of sprints. Nothing could be further from the truth. When building something new, there needs to be a sprint “zero” which allocates enough time for architects to perform their due diligence and create the user stories for non-functional requirements that deal with things like performance, scalability, security, reuse, and more. Skipping sprint zero almost always results in high maintenance costs and unreliable systems down the road. Without architecture, the only thing that evolves from a series of sprints is fragile systems.

Solution:

One of the main causes of unplanned work is poor architecture. To reduce the amount of unplanned work it is highly recommended that an architecture sprint is performed first (the infamous “sprint 0”). In bigger organizations, this step is performed in conjunction with the portfolio management processes to determine feasibility and enforce standards. Once the project is accepted into the portfolio, an architecture review takes place. The architecture team assesses the business requirements and makes decisions such as build vs. buy, selects target architectures (on-premise, cloud, etc.), and creates non-functional requirements in an effort to govern the architecture. In smaller companies without an established architecture team, the architects and/or tech leads perform an architecture sprint where they sit with the product manager and business owners to ask relevant questions to uncover those non-functional requirements around security, scalability, reliability, SLAs, etc. From these requirements the team can then create user stories for the desired target architecture as opposed to simply sprinting and hoping an architecture evolves over time.

Measuring all the wrong things pattern

Problem:

This pattern is annoying to those of us that believe that the goal of agile is to deliver reliable software quickly. A common fail pattern I see is management focusing too much on metrics like velocity, defect counts, and story points and not enough on delivery and customer satisfaction. Don’t get me wrong; tracking velocity, defects, and story points can provide useful information, but they don’t tell us whether we are being successful or not. In addition, those metrics are unique to each team. I like to compare these metrics to blood pressure. My blood pressure and my wife’s blood pressure will always be different. Mine runs a little high and hers runs a little low. The fact that our numbers are different has no bearing on our health because our numbers are unique to our bodies. What is important is comparing my blood pressure to my previous numbers over time. That is exactly what velocity, defect rates, and story points should be used for. I have seen teams get beat up by management because Team A does not have the same numbers as Team B. That is a waste of time. The best way to compare teams is to measure how happy their customers are and how successful they are at actually meeting their commitments.

Solution:

It is important to separate the metrics that help the team measure their performance over time from those metrics that measure the impact that agile is having on the business. Metrics that measure performance over time like velocity, story points delivered, and defect rates can provide valuable insights to allow the team to detect trends and make the appropriate adjustments to be a more efficient team. But at the end of the day, it is the company’s KPIs (key performance indicators) that really matter. Everything that we do should align with the company’s goals and measures. What good is it if we have great velocity numbers if our customers don’t like our products and services? Sometimes metrics drive the wrong outcomes and teams prioritize internal metrics over customer needs because they are measured on the wrong metrics. My advice? Focus on the metrics that matter to the business.

Delusional management pattern

Problem:

Nod your head if you have seen this scenario. Someone in management reads a story about how company XYZ implemented agile, which resulted in their IT team delivering new features left and right. So management declares that the company is moving to agile, puts someone in charge who has little authority to make organizational changes, and tears down the cubes in favor of an open collaborative space. The executive kicks his or her feet up on the desk and declares, “I did my part, now do agile”. “Doing” agile requires an entire organization to change their business processes. Sales people need to stop promising products and features that are not on a roadmap, product managers need to minimize changes to the current sprints and be accountable for the delivery of the product, IT needs to stop working in silos and in a sequential manner, and everyone needs training and new incentives to enforce the proper behaviors. Simply declaring “agile” is not enough.

Solution:

This pattern can be a challenge. Realistic expectations must be set and the management team must be educated on what it takes for an organization to implement agile. Implementing agile requires a culture change for most organizations. It is not just an IT thing. It is critical that sales, product, IT, and management have all bought into the same principles and are all accountable in order for there to be success. If possible, bring in an expert speaker or coach to present to the management team. Invite members of management to a conference or meetup. Visit local companies or partners who have experience with agile. If management does not have the proper expectation, then it will be hard to succeed. It is your job to manage up and help management understand what it takes to be successful so they can provide what you need to get there.

Teflon roadmap pattern

Problem:

Companies that excel at agile do so because they break work down into small manageable chunks and are allowed to stay focused on that work. I often see companies struggle to keep sprints intact because priorities are forever changing and the business has no discipline. Whoever yells the loudest gets their item on the list and the list changes daily. Sprints are interrupted and even completely stopped. Developers and architects are pulled into countless meetings to discuss the next “high priority” item of the day. All of this noise leads to countless distractions that keep teams from focusing on the task at hand.

Solution:

One word. Discipline. The product manager must run a tight ship and have the support of management to say no when sales makes promises the company can’t keep. But let’s be fair to the sales people. In order for sales to play along, software delivery must be predictable. The product team must publish a roadmap that gives sales visibility to what features are going to be delivered in which timeframes and when the next available time slot is to fit in the next big thing that they are on the verge of selling. If sales can trust the roadmap, then they can be taught to not promise things that disrupt the roadmap or at least be taught to consult with product before making promises. If software is always late and they have no idea when they can get their new widget, then they will continue to make promises that require emergencies to get new stuff out the door.

Summary

Companies that have successfully implemented agile methodologies are often more productive, have better quality than before, get to market faster, have happier customers, and get a bigger bang for their IT investments. Unfortunately, many companies struggle with agile and don’t see these same returns on investments. My hope is that by recognizing the common fail patterns, companies can implement solutions like the ones I have recommended or others and have more success with their agile initiatives.