Dashboard Confessional

Case Study:

THE PROJECT

In the spring of 2021, we were just getting into the groove of using Objectives and Key Results (OKRs) - and we knew we wanted to be very intentional for our next quarter. We formed a product discovery trio to better understand where our biggest needs were. Then, based on our discoveries, I designed and implemented something that Engage (our product, a student engagement platform used on college campuses) had never seen before - data and visualizations.

THE PLAN

Our product discovery trio consisted of me, a senior developer, and our product manager. Our plan was as follows:

Make several hypotheses about areas of improvement
Narrow down the list to one potential hypothesis to focus on this quarter
Confirm that hypothesis to be true or false

I led our trio through the OKR research phase, collaborated heavily with my colleagues to craft our objective followed by our key results, and came up with a design that fit the bill. Post-release, I monitored feedback and as a team we iterated in response.

Research & Artifacts

PERSONAS

Our division's leadership team had identified that a major goal of theirs was to improve relationships with users, so we started by identifying a list of our primary roles in Engage. I crafted several personas to help inform this step. I started by creating a list of (too many) possible personas for crafting, and then ran it by my colleagues in campus success. They helped me narrow down our list to five personas and refine their contents. Three of these personas are available for viewing below.

OPPORTUNITY SOLUTION TREE

After I crafted the personas, we spent some time mapping them to the different administrative roles that exist within Engage. The product discovery trio then used this mapping and our combined extensive knowledge of the platform to write our set of hypotheses.

We laid out our hypotheses as an opportunity solution tree. Product took the lead on assessing which branches of the chart made sense to pursue, with assistance from me and our senior dev colleague. We ultimately decided on a role for this quarter (Branch Admins), with the idea that we would continue to use this chart to evaluate other hypotheses in the future.

The blue rounded rectangle path represents the hypothesis that we chose to move forward with confirming based on our research, which consisted of reading idea submissions (feature requests), sifting through submitted feedback, talking with the support team, and past interviews/surveys that had been conducted. Because we had this information already, we were able to define this path quickly and set to work confirming our hypothesis - which was, simply, "Is this the right thing to do?"

If you follow our path through the tree, you can see that we start with the division-wide priority of bettering our relationships with users. Then, as previously stated, we selected Branch Admins as the users to prioritize. Through our research, we knew that there were two major gaps with Branch Admins - their lack of an ability to view their data in aggregate, and the rigidity of site permissioning. Again, based on the research, we chose to prioritize the data experience, and specifically the data experience surrounding student involvement data.

This path did not mean that we were absolutely going to build a better data experience for Branch Admins. What it did mean was that we were going to prioritize pursuing this line of inquiry. And, if it turned out that folks were uninterested, we would shift gears and work to confirm a different hypothesis. Being as flexible and agile (see what I did there) as possible was paramount in this process - as well as in product design in general.

Interviews & OKRs

INTERVIEW QUESTIONS

We knew that in order to craft a better data experience for Branch Admins, we needed to understand two things - what are they doing with data right now, and what do they wish they could do? I wrote the following interview protocol that we used to interview five Branch Admins (names supplied to us by our colleagues in customer success).

What data do you pay attention to right now, and what do you use it for?
- Who are the relevant/interested parties?
- Walk us through your process. Be sure to call attention to A) things that really help you and B) pain points.
What do you feel is missing for you right now in terms of data, and why?
What do you see as barriers between you and improvements to processes you would like to make?
We want to use data to empower you. What is important to you – improving efficiency of process? Risk management? Delivering data to other administrators?

RESULTS

The results from these interviews luckily confirmed our hypotheses, but also gave us some new things to think about.

DIGESTIBLE DATA

All of the users stressed the importance of only seeing the data that is relevant to them, and to have it in a format that it can be understood at a glance, without much additional thought.

MEANING MAKING

Just seeing data wasn't going to be enough - these admins are not data experts and need help making meaning. Helpful things might be recommendations for actions or lines of inquiry to follow. Or at the very least, an explanation of the data.

An icon of a person going through a door

INVOLVEMENT INFO

Everyone agreed that the most important data to look at was user involvement, primarily what events users are attending. Additionally helpful points would be event data and membership in organizations.

OBJECTIVE & KEY RESULTS

Based on the results of our interviews, we confirmed that the path we selected was the right path to follow (yay!) and wrote our objective, which was:

The team will empower Branch Administrators to be able to understand trends within the student experience in their Branch.

This objective isn't perfect - but we did our best to incorporate the results of the interviews and our hypotheses into a concise statement that we could use to inform our direction for the quarter. We then wrote our key results:

Increase Branch Admins that login weekly from 31% to 45%
Decrease the amount of Branch Admins with community level access by 10%
When surveyed, see a rate greater than 70% of Branch Admins agreeing that they are able to better understand trends within the student experience in their Branch
Maintain less than 10 Low Impact Defects
Maintain < 15% Change Failure Rate

Our first three KRs are all about behaviors that show we are accomplishing what we set out to do. We are adding value to the role (which will increase logins), giving the role increased access to their data (reducing the need for them to have additional access), and helping them make meaning (surveying them about understanding trends). The other two are balancing KRs, ensuring that we maintain quality as we add value.

Personas

Design & Release

BRANCH LANDING PAGE

After creating our KRs, it was time to figure out our first story to help us achieve what we had set. Lucky for us, we had a perfect spot in the site (the branch landing page) that was very underutilized (almost blank). It was a spot that Branch Admins had to travel to extremely frequently in order to access the tools to manage their branch. So, we decided that would be the place to start. And we selected statistics based on the information we had gathered in our interviews. I used Material UI components and icons for this design.

PROTOTYPE V1 (Whimsical)

This initial prototype allowed me to lay out my thoughts about a few things. I chose to provide the statistics in sentence form to quickly give a little more meaning rather than just a label. The visualization served to show trends over time. And the prominent feedbacker allowed us to quickly gather feedback and iterate on the design. I knew this design was not ideal, but it was my first stab at getting my thoughts onto paper.

After talking over the design with the developers and product, we made a few changes - mainly adding some important stats to the top, and altering some titles and labels.

V1 IN PRODUCTION

We received quite a bit of feedback on this page. Many people were very excited! Others, however, had very good pieces of feedback for us, which emerged as one major theme. This data had become the most prominent thing on the page, and it was harder for them to access their branch tools. We also used Microsoft Clarity to see what users were clicking on and discovered that they were clicking on the statistics at the top despite them not being clickable. So, we set back to work.

In the end, we elected to remove the sentence structure from the insights in the interest of saving space, and placed their tools on the left side along with the stats from the top. That way they could easily access them and still view their data. It was a sacrifice that we had to make, despite the fact that the sentence structure had been well received in feedback. We also took this opportunity to expand the date range from weekly to a date picker (which we hadn't done initially just in the interest of keeping the story smaller).

V2 IN PRODUCTION

EXPORTS

After getting the landing page in production, we focused on providing Branch Admins with the ability to export their data. We added exports to every major tool they had access to. Exports are not the ideal solution, but they will serve in the interim while we find ways to display more data to Branch Admins within the platform itself.

Reflection

Creating this feature offered me a wealth of learning experiences. I was so grateful to be able to collaborate on the goals for my team for the quarter as I do think that design provides a new perspective to goal setting. It was very interesting to get some insight into how product looks at prioritization and assesses business needs.

In Engage, I spend much of my time updating old functionality because even when we create something new in all likelihood it touches legacy apps and has quite a few limitations. While we certainly had some limitations on this, I loved having more freedom to create something completely new to help campuses. Plus, making data informed decisions is a real passion area of mine, so enabling people to make those decisions more effectively was really a joy.

This dashboard was a big hit internally, which was very exciting! Because we didn't have anything like this in Engage before, leadership in product was very excited to share what we were working on to help motivate other teams to use data to empower their users. I am very proud of the work I did on this, and I hope that I can continue to both shape our goals and empower our users with data.