When you’re a smaller team of makers, product management is little more than an ad hoc concept shouldered by whatever designer or developer on the product team is last to say “not it.”
That was the case at FullStory for what felt like a long time. We were scrappy. When it came to product decisions, we'd rely more on well-reasoned arguments and gut-feelings than research and analysis based on real user behavior.
Product management was an afterthought.
But times change.
While a scrappy, ad hoc approach is a great way to innovate and experiment, it’s a crummy way to validate user need and iterate toward ideal states. And when you have a product that has lots of savvy users and an ever-growing UI surface area, if you don’t turn to data to make decisions, you’re going to miss something critical.
Thankfully, as we’ve grown, FullStory has implemented some process and formality in the realm of product management. Now, when customers be clamoring and features be struggling, we have the brains to investigate problems and coordinate plans of attack.
And that’s just what we did when we reevaluated FullStory’s product solution for web analytics “maps:” we turned to research, rethought our product, solicited feedback, and released a new way to view engagement on your site called Engagement Heatmaps.
Sit back and get the story of how we got there.
You Want Heatmaps (No, Really!)
Rewind to early 2017 when we launched an amazing feature that we thought was the right thing for our users: Click Maps. We thought our Click Maps were the best, most accurate answer to the question so many of our customers were asking: where are your heatmaps?
Well, FullStory’s Click Maps were built to analyze precisely how your users, in aggregate, engage with a given part of your site. Meanwhile, by being element-centric, our Click Maps neatly sidestepped certain major challenges of the more common web heatmaps. Click Maps are great!
So why were our customers still asking us about heatmaps?
Our good friends in marketing brought this issue to the fore when they discovered people were searching for “FullStory heatmaps” a lot. Like "it’s a top 10 search" a lot.
Data that’s hard to debate.
While a smaller team might have freaked out and doubled efforts on improving Click Maps, our product management hugger helped us evaluate the challenge in a measurable way. Our objective was to understand if a different visualization of clicks and engagement could be of real use to customers.
So we got to work prototyping a heatmaps-inspired version of Click Maps. In short, we replaced the numbered markers of our Click Maps with a scaled color spectrum that correlated to user engagement on the page.
It was an inexpensive amount of effort for a dramatically different visual take.
Testing the Prototype
Once our “Engagement Heatmaps” prototype was ready, we took it to two groups of users. The first sample was a group of Click Maps “power users.” The second was a group of industry experts who rely on traditional heatmaps as an analytics tool.
We enabled our proto-heatmaps for these select users using their own data. We found this working prototype to be much more compelling and useful to these testers than static images or abstract visualizations. Having this dual setup also gave us the ability to demo Click Maps side-by-side with the Engagement Heatmaps prototype..
Then came the interviews. The hardest part of user interviews is scheduling, and we ended up with clumps of interviews at the beginning and end of our demoing period. This bug turned out to be a feature as it enabled us to refine our questions and improve the demo, itself. As the process went on, we were also able to formulate a scope of work to be done should we decide to make the prototype a full-fledged production feature.
The Interview Results
So what did we learn from the interviews? Let’s take a look.
1. Are Click Maps Confusing?
Some users find Click Maps initially confusing. Many users reported that they thought Click Maps portrayed a single users set of interactions with the page.
The confusion was centered on the ranked numbers in our visualization. It’s not a mental stretch to think that User X clicks the thing with the "1" on it first, "2" second ... and so on. And remember, these were our Click Maps power users we were talking to, not someone that saw that feature once and never came back.
This point helped us validate a visual mode that relied on colors and density over literal ranked numbering. As a side bonus, people know immediately when they see a heatmap that its an aggregation of data. Something that didn’t necessarily “Click” with our existing Maps.
2. What's the User's Job to Be Done?
Depending on the job the person was doing, there was clear preference for a given visual mode. For example, if the "job to be done" was in marketing and product design, more meaning and value came out of the visualizations. Alternatively, those concerned with product management or UI engineering preferred the “to the metal” ranked mode of Click Maps.
Some users even liked both views—depending on the business of the underlying page.
Let’s take an approach that allows us to preserve both visual modes!
3. Are Heatmaps Always Useful?
Heatmaps aren’t really a vital, mindblowing tool for anyone we spoke with. At best they’re nice to have as a complement to larger insights. Plus, colors!
And speaking of colors, we found that some users believed heatmaps were a distraction for whomever they were presenting them to (i.e. detailed presentations regarding user behavior inevitably devolved into questions regarding heatmap color choice). Also, traditional heat map colors were seen as overly connoted with weather and often inaccessible to those with certain color-blindnesses.
Maybe we don’t need to get too crazy over-developing the perfect heatmap feature. Functionally speaking, we liked that heatmaps are a quick and easy tool for centering focus on most clicked things. There’s no need for unpacking the visualization when you have a good color scale and quickly discern the most clicked things.
Research Complete—What’d We Do?
Our user interviews and demos gave us confidence that the adaptation of our product—to bring Engagement Heatmaps as an alternative to “Click Maps Classic”—we decided the visualization was compelling enough to officially build it into FullStory.
And so it came to pass that Engagement Heatmaps are a new visual mode for Click Maps. They help answer the core question everyone we interviewed was trying to answer: What are my users interacting with on this page?
But Our Work Isn’t Finished ...
Remember the beginning of this post? There were two problems: meeting our customer’s needs for some spiciness in their visual overlays being one. Engagement Heatmaps directly answer that problem—it took us awhile, but we’re finally giving a clear answer to the question, “Does FullStory have heatmaps?” Yes.
The other problem remains: engagement. While we think our new visual mode for aggregate clicks on a page will serve up insights faster and help drive feature adoption, there’s still work to be done helping the feature get heavier use.
For now we’re confident that we’ve breathed new life into a feature we dearly love.
We’re also now confident we know how to get the right insights out of our users when we decide again to iterate on a FullStory product feature.
Turn Up the Heat—Try out Engagement Heatmaps
If you want to give Engagement Heatmaps a spin, here's what you do: While watching a session in FullStory, mouse up and click Page Insights. The replay will pause and an overlay will be generated with your Engagement Heatmap for the paused screen.
Want to learn more about Engagement Heatmaps? Blaze a trail over to the Engagement Heatmaps features page—and if you're not a FullStory user, while you're there you can sign up to try it out!
Finally, once you try our heatmaps out, let us know what you think. We're always looking to improve our product further.