HILDA MYER-POST
Idea Triage
Case Study:
THE PROBLEM
Gathering feedback on your product is important. Ideally, however, someone would also be going through and dissecting that feedback as it comes in, right? Not so for us, once upon a time. After the departure of our Product Manager it came to light that 300+ idea submissions (feature requests) had been sitting in a queue dating back as far as two years previous with no outreach or resolution of any kind.
THE APPROACH
I led the effort to triage the existing idea submissions, making sure to include the relevant development teams in any decisions being made. Then, forming a joint UX, development, and product team, we shaped a protocol for idea submission intake and management. The process includes tagging incoming ideas with helpful keywords and linking them to cards in Azure Dev Ops (ADO). This keeps both product and development informed of progress made. We continue to iterate on this process to make it more efficient and better aligned with lean and agile methodologies.
Digging In
DIVVY IT UP
Initially, looking at the list of 300 ideas was overwhelming. I took some deep breaths and did some studying of our boards in ADO and how we categorized work items. And then I went to work. The development/design team on Engage was organized into four separate dev teams, three of which included a dedicated UX person. Each was responsible for a different area of the platform. So, I started there. I took the 300 ideas and bucketed them by which dev team the work would be relevant to.
After bucketing the ideas, and with the blessing of our development manager, I met with each of the teams once a week for six weeks until we worked our way through the list. I knew that we essentially needed to get every idea into a final state where it didn't need any more work - which either meant deciding to build the requested idea, or throwing it out. So, I crafted a protocol for us to use in our meetings.
Do we understand what they are asking for?
IF NO:
I would put the idea into a pile for me to research later in order to figure out exactly what was being asked for. We called these ideas "UX Review."
Would the thing they are asking for make sense in the context of our platform?
IF NO:
These ideas essentially went into a trash pile. We pulled them out of pending and marked them as "Will Not Implement."
Do we feel that we can make a decision on this without input from product?
IF NO:
The third pile was for me to discuss with product in order to get their opinion. This pile we called "Business Analysis."
If the answer to all of those things was “yes,” I would put a card in the backlog for it (or add the idea information to an existing card if it was already in the backlog). I was truly surprised at the sheer amount that we were able to add to the backlog. As a matter of fact, our backlog grew exponentially. I talked to our development manager about it, and his perspective was very useful – he said that the backlog was, in fact, already this size. We were just making it visible.
TRIAGE RESULTS
After the marathon month of going through the 300 ideas, we had added roughly 175 items to the backlog. Many of the ideas were also repeats (an excellent indicator that we should pay attention to those items) so in the end roughly 75% of the ideas ended up in the backlogs for the four teams. The developers did not enjoy going through them like this but understood that we needed to get ourselves to a blank slate in order to make the process better.
Intake Protocol
IDEAS TEAM
Of course, during the triage period, idea submissions continued to come in, so we needed to simultaneously be working on an intake process. We ended up forming an ideas team (me, our dev manager, and someone from product management) that met once a week and processed the week’s ideas, following basically the same question model I had designed from the triage meetings. The added challenge for this team was that we wanted to close the loop for all incoming ideas – reaching out to the requestors to both thank them and to let them know their ideas weren’t just going into a black hole. We documented our process as a flow chart for easy reference.
For each idea submitted, our goal was to get it through this process and down to one of the three final states at the bottom as quickly as possible while still responding thoughtfully. Each of the grey boxes in the center represents a question that we felt that the combination of me, product, and dev management could handle answering together. The yellow boxes to the left and right of the core process indicate points where additional research and/or conversations are needed. The vast majority of ideas could skip the yellow boxes and end up in one of the three final buckets.
PROGRESS
As we followed this process, we whittled the number of ideas down so far that we decided it was time to be more intentional about marketing the ideas form (the place for customers to submit ideas) – which, while not a secret previously, was not something that was well advertised. We started with our release notes where we made two major changes. 1) We thanked specific campuses when making releases related to ideas they had submitted, and 2) we added a link to the ideas form at the bottom of every release note.
We also were able to set up a connection between Aha (the platform that the product team uses to manage ideas) and ADO that updated the ideas in Aha when progress was made on the development of a story.
FEEDBACK FRIDAY
Once we got into our groove processing the feedback we identified two major problems with our process and crafted a solution accordingly.
PROBLEM 1:
We took the development teams out of the process.
PROBLEM 2:
We weren't getting to the root of the campus requests.
In response, Feedback Friday was born. In an effort by the ideas team (me, dev manager, product), through some trial and error, we made a rotating Friday meeting with each team to go through ideas relevant to their team. This added the developers back into the equation and gave them a say as to what was going into their backlog. Then, instead of processing the requested features on the surface level (i.e. build the thing), together we dug deeper - asking ourselves why those requests were being made - and wrote user stories (as a [user role] I can [do the thing] so that [reason]).
THE SOLUTION
Example Idea Flow
Let's say we have a new idea from Anya at Knowledge State University. Anya writes the following:
"I manage 200+ student organizations, and I need to be able to remind them when their registration deadline is coming up. I would like you to give me an export of just the email addresses of the leaders of the organizations so that I can email them."
STEP ONE: Which dev team?
Easy peasy - this one is about managing organizations, so it goes to the manage team.
STEP TWO: Follow the flow
It's Friday, and when it's time for the manage team's meeting, we get to Anya's idea and process it.
Do we understand why this is being requested?
YES!
Anya is struggling to help her student organizations to meet their deadline for registration.
Do we already have
this functionality?
NO!
Right now, Anya does not have an easy way to accomplish her goal of helping her orgs meet their deadline.
Is this for an existing system?
YES!
We already have a system for registering student organizations, so the concept is not new.
Is this reasonable to ask of the existing system?
YES!
Anya's problem seems relevant to a large amount of users, and solving it is in line with the goals of Engage.
Those answers = Add to Backlog!
STEP THREE: DIG A LITTLE DEEPER
Since Anya's idea is getting added to the backlog, we need to write a user story. And in order to do that, we need to probe past her feature request (give me an export of email addresses) to discover her problem. Together, we determine that Anya's problem is that student leaders do not know the due date for registering their organizations. That date is not surfaced to students anywhere in Engage - it is just set behind the scenes by people like Anya. We come to the conclusion that the problem isn't even really Anya's! It is one level deeper than that. So, we write the following user story:
As an organization leader, I need to know when my registration period ends so that I can comply with university policy.
STEP FOUR: ADMINISTRATIVE TASKS
During the meeting, I add the user story to the backlog and we collectively try not to come up with a solution yet (sometimes that's easier said than done). Then I update the idea in Aha and connect them. That whole process takes about a minute, and it doesn't even disrupt the flow of the meeting. And then we move on to the next!
Reflection
Overall, processing the ideas this way feels like closer to the right thing to do. But we aren't there yet. There are a couple of problems that we don't have solutions for yet. The first one being the size of the backlogs. They are unruly and enormous - and even if this is their true size, it's not great for workflow or morale.
​
The second one is the upkeep. This is a big time suck, especially for the ideas team that attends the meetings with each development team. We essentially lose a big chunk of our Fridays. And, the ideas continue to come in faster than we can deal with them. The number of ideas we were receiving per month doubled, tripled, and then quadrupled in the six months since our initial triage. So the intake process worked initially, but then we began to fall behind again. And, the longer we did Feedback Fridays, the more those meetings became at the bottom of everyone's priority list - they often get cancelled because no one can come.
​
We are due for a new iteration, and another triage. This time I will take what I learned and apply it, and attempt to create a more sustainable solution that will still allow us to keep up with the huge amounts of submissions. Here's hoping!
