Rethinking Backlog Refinement

I’ve gotten a few questions recently about how I do Backlog Refinements with my teams. Backlog Refinement is a complementary practice to Scrum, meaning it is not actually a Scrum event or even part of The Scrum Guide, but it’s a tool a lot of Agile teams use to get Product Backlog Items ready for an upcoming sprint. Since there’s no technical rules around how (or even why) to do Backlog Refinement, it’s become a practice that teams can really mold into the form that provides them the most value. I’m going to tell you about how one of my teams has done this and what their Backlog Refinements look like today.

When I started at my current company 5 years ago, all squads were doing some form of Backlog Refinement, usually 1-2 times per 2 week sprint. As a new Scrum Master, I joined my teams and just observed what they were doing and how these meetings were helping them. What I saw was largely mixed results. There’s a number of reasons why some teams were having fun, laid back Refinements and pointing tons of stories, while other teams’ Refinements were cold, quiet, awkward, and ended with the team getting through maybe one or two stories. I think the biggest challenge my teams were facing around this had to do with a lack of Product Goals and overarching vision for our company. 

Each of my teams supports numerous stakeholders in many different departments of our company. Many of those stakeholders represent external clients as well. When I first started, our teams’ backlogs were largely a messy, disorganized list of requests from all these disparate stakeholders. A sprint often included a lot of one-off, unrelated stories that were quick to do but completely unrelated to each other. This made focus incredibly difficult for my teams, and made our sprints more of a timebox for running our sprint events than a focused period to deliver an increment of value. Our Product Owners were not empowered and did not own their products. They served more as order takers for stakeholders, and their practice of prioritizing the backlog was more about ranking tasks by perceived urgency than it was about ordering cohesive features by potential value to the customer. My Scrum Masters chapter (the group of 5 of us who work across these 8 teams) has done a ton of work over the past several years to help our Product Owners set Product Goals (we use OKRs for this) and practice saying “no” or “not right now” to their stakeholders. 

Today, our team Backlogs are definitely more organized and prioritized according to product work that our teams are planning to do to achieve their goals. There are still plenty of one-off stakeholder requests, but our POs have gotten better at deprioritizing the ones that don’t support the team’s current Product Goal. We are still far from perfect with this, however. 

One of my teams, the Blue Team, has recently gone through a bit of a reckoning regarding their Backlog Refinement meeting. They had a change in leadership (Scrum Teams should not have team leaders, but this is an anti-pattern my company suffers from and I can talk more about in a future blog post), which helped them take a look at their habits and routines and question if what they’d been doing was really working for them. One of the areas they noticed they wanted to change was Backlog Refinement. Once a sprint, we’d been spending 90 minutes together on a Zoom, trying to get stories ready to be pulled into an upcoming sprint. This typically looked like our Product Owner and UX Designer presenting a new feature they’d been working on to the team, describing the goal they were hoping to achieve with it, and then walking the team through the mocks the UX Designer had created for it. More often than not, the team would end up asking questions that the Product Owner and the UX Designer did not have the answers to. Our product is complicated, with many different user types and corresponding rules, and no one understands it better than the engineers who built it and work on it daily. Both our PO and UX Designer are relatively new to the company, and they would often spend a great deal of time building out a detailed plan for a new feature, only to present it to the engineers and learn that it violates a business rule, it isn’t possible for a technical reason, or it conflicts with something another team is currently testing within our product. 

This was a really difficult situation for me to observe as a Scrum Master, and honestly a pretty awkward and uncomfortable problem to try to address. The engineers were constantly frustrated that their Product Owner didn’t understand his product, and I kept feeling a bit resentful for the same reason. How could I help the team improve this, when what seemed to be the issue was a PO who just needed to put in the time to better learn his product? 

After a lot of conversations with my PO and Technical Lead, and a lot of idea-bouncing with my SM Chapter, I began to come up with some ideas to try out. Our SM Chapter has a book club, and we were reading Lean UX by Jeff Gothelf & Josh Seiden at the time. Our design process has always been far from lean, but it’s something we’re working on improving. This book helped me understand that part of the problem with the way we’d been working was that we were not involving our engineers in the ideation and design phases of planning our work. These activities were happening separately, and only between our Product Owner and our UX Designer. In the past, when we’d had a PO who had worked at the company for years and knew our product inside-out, this problem was masked by the fact that she always brought work to the team that was well-defined and made sense to pursue. This problem was revealed when we got a new PO, not because he was a weak PO, but because it made apparent this gap in our processes. We were leaving our engineers, the people who knew the most about our product, out of our ideation and design work! We were serving them orders of fully fleshed out ideas, instead of pulling them into the conversations to discuss how to solve the problems our users were facing. 

This was the first big change we made to our Backlog Refinement process. Every new product idea was assigned a Tech Sponsor, one of the team’s engineers, who would go to all the ideation and design meetings, offering their thoughts and inputs to the problem far before it would be brought to the team. Ideally, I’d love to get the squad into a place where the whole team is doing this process together, but this has been a good incremental first step for us. 

The second big change we made to our Backlog Refinements was to reexamine the purpose of these meetings. We’d always thought of them as “the time where we get stories ready to be worked on in a sprint.” This is why we’d end up deep in the weeds of debating copy on a button and find our whole 90 minutes used up on one story. We decided to try something different, and instead only use the time to estimate the level of effort for each story. My engineers can be a bit uptight about their Definition of Ready, and they want final mocks, final copy, and final requirements laid out in a story before they would even think about starting work on it. I love that they are so into their Definition of Ready and Definition of Done, but I’m trying to help them massage some flexibility into their DoR, so that more of the problem-solving and collaboration can happen during the sprint, rather than all being finalized before it begins. A lot of this stems from my company’s anti-pattern of wanting to do A LOT, really FAST. Teams are motivated to constantly raise their velocity, without a feedback loop to evaluate the effectiveness of what they’ve created. Our SM Chapter is trying to coach our teams and our leadership to SLOW DOWN and focus on value. I don’t care if my team completed 20 stories in a sprint if none of them added substantial value. I don’t care that my team is 100% occupied at all times if they didn’t end up releasing working software at the end of the sprint. I would much rather see idle engineers resting, helping each other, or learning something new if it means we released a focused increment of value by the end of the sprint. 

I’m trying to work with my teams to be comfortable pulling in fewer stories during Sprint Planning, and to collaborate with each other to design, plan, and complete those stories within the sprint. This will help us deliver more value because the entire team, including engineers, will be more involved in the solution we are building from inception to testing to release. This will mean that the stories we complete are more valuable to our users, so even if we get fewer of them done, we will deliver more value than if we’d gotten more stories finished, but those stories were not well-guided ideas.

So today, the Blue Team is not looking to the Refinement meeting to nail down every detail of an upcoming story and have it ready to build. Instead, we are looking at this meeting as a way to estimate our level of effort for each product backlog item, to help us break it down into increments we can complete over the course of a single sprint. We understand that the sprint is when a lot of in-depth conversation should happen about how to build the item and what it should look like. We like to use collaborative tools like Figma to work on our Design Files and whiteboard our ideas, both during the Refinement meeting and throughout the sprint. 

Typically in our Refinement Meeting, we’ll use the filter view we’ve created in Jira to look at the upcoming items on our Backlog that are not yet in the Ready to Start status, sorted in order of priority by our PO, and then work our way down the list one-by-one. For each story, the PO will describe the problem they’re trying to solve, the Tech Sponsor will discuss the high-level technical implementation they’ve envisioned, and the whole team will weigh in on the approach. The UX Designer will show off their mocks in Figma, and the team will make sure they understand the big picture. We’ll use Planning Poker to estimate the level of effort, and break down anything that’s deemed too big for a single sprint. We also have a code word that anyone on the team can call out or type in the chat, at any time, which means “we’re getting off track or too deep in the weeds.” When this happens, we pause, look at where we’re at, determine what is blocking us from pointing the item, and usually move onto another story. This has helped us from slipping back into our old habit of digging too deep into a detail and spending all of our time on a single story. Part of the goal of our new Refinement meetings is to figure out what still needs to be investigated or understood before we can pursue a piece of work. If there’s a lack of clarity around a PBI’s purpose, that will be something the PO will take away from the meeting and try to get figured out before the team jumps into solving it. 

We are definitely far from perfect in how we’re refining our backlog, and there is a lot of room for improvement here. But we’re also proud of how far we’ve come. One of my favorite parts of working with Agile is our constant openness to change and how we welcome opportunities to experiment. We are always looking at our processes and asking ourselves why we’re doing things and whether it’s providing the intended value. And often, it’s not. However, since we’re an Agile team, that’s not a bad thing. It’s an opportunity to ask ourselves how we could improve. We probably don’t know the answer, but we can propose and discuss some options, pick our favorite, and then test it. We’ll try a new practice for a couple sprints and then evaluate, adjusting as we go. These changes to our Backlog Refinement meetings are just the latest example of the Blue Team inspecting & adapting their processes. We'll be keeping an eye on how this goes over the next few sprints and continuing to look for ways to improve.