One of the most popular feature requests we’ve had at FullStory has been aggregated user interaction information for a particular page. Web analysts want to see their user behavior in aggregate. They want a visualization that summarizes the user experience.
The de facto solution analytics tools offer for visualizing aggregated clicks and user interactions on a website has been “heatmaps” — like the Predator’s infrared Heads-Up Display applied to web pages.
For web applications, heatmaps are graphical representations that indicate the areas of heightened user interest and interaction. There’s no standard to how these heat maps are built, but generally, red, orange, and yellow areas show higher user interaction and interest. Green and blue areas are for moderate interest. Dark or gray areas are ... forever alone.
Areas of user interaction are determined through eye-tracking tools, session replay tools, or perhaps just aggregated clicks on links.
Heatmaps have been the go-to graphic for showing user interactions on a web application — they’re hot and cool!
Except the more we learned about heatmaps, the more we found problems.
What’s a heatmap really, anyway?
Let's start with a bit of history. The term “heat map” or “heatmap” was trademarked by Cormac Kinney in the 90s to refer to a 2D financial market visualization (Wikipedia; also Kinney’s lapsed Trademark). Today, what most of us think of as heatmaps is entirely different.
According to Kenneth Field (a.k.a. Cartonerd) in his post When is a heat map not a heatmap, most of the data visualizations referred to as “heat maps” are not actually heat maps. They are technically isarithmic density maps.
And what is an isarithmic density map? According to the Geographic Information Systems encyclopedia, an isarithmic map is:
A type of thematic map that represents a continuous field using isolines (lines of equal value) or isopleths (regions of similar value). They are sometimes referred to as a heatmap, although a heatmap is only one type of isarithmic map that represents density.
A popular form of isarithmic map is a contour line map. Think about topographic maps — each connected line in a topographic map represents the same elevation:
Your run-of-the-mill topographic map uses bright line data points — elevation at specific, unique geographic locations — and interpolates (i.e. connects the dots) the missing information across the data points.
That brings us to web application isarithmic density maps—a.k.a. the things we all think of as heatmaps. Web heatmaps don’t have isolines. Instead web heatmaps use isopleths. Isopleths are used to represent regions of similar value — the blue, red, green, and yellow color splotches.
The splotches of color on heatmaps represent aggregated user experiences on a web site or application. As visualizations go, heatmaps make for an easy-to-grok, pretty picture.
We look at heatmap analyses and say, “Ah, I see!” We feel like we understand something deep and meaningful—like we're peeking behind the curtain and seeing a remarkable bit of web analytics.
But there’s a catch — or at least four catches — and if you’re hypnotized by the glowing hotness of heatmaps, you might miss these problems completely.
The four problems with web heatmaps.
1. Lies, damned lies, and [X, Y] positions
In a world of responsive design and device heterogeneity, using a visualization designed for a fixed unchanging aspect ratio is a recipe for failure. “100px down and 350px to the right” doesn’t mean the same thing across two different user visits to your site with each at different browser window sizes and/or screen resolutions.
Ignore these differences and you might find yourself at a confusing dead end. Responsive design and varying display PPIs also make converting between aspect ratios only possible with a pile of assumptions.
Heat mapping tools get around this problem by forcibly siloing the datasets they visualize. You only see heatmaps with aggregate data for users with the exact same screen resolution configuration. 1024×768 users go here. 1366×768 go there. 1280×800 users go in the cupboard. These arbitrary silos mean you aren’t actually seeing the meaningful aggregate across your actual non-same-screen-resolution-using users. Kinda defeats the purpose of the “easy” visualization, right?
And this is just on desktop. There are thousands of different mobile browser resolutions in the market. Most heatmap visualization solutions pretend these differences don’t matter.
Yet they matter a lot.
Averaged and aggregated mouse hovers and un-contextualized [X, Y] click positions cause well-intentioned product teams to make actual product changes based on shoddy information, wasting time and money, negatively affecting the business, and harming customer experience.
2. It’s 2017. Dynamic applications are the norm.
Today, a single web page is no longer a static fixed document. Modern webpages and applications are dynamic. Animations, slide out panels, expanding menus and modal dialogs are the norm. As you might guess, this poses a problem for heatmaps.
You end up with aggregated positional spaghetti.
And this same issue affects everything from product inventory tables on an e-commerce site to data-driven menu options in your business enterprise app.
3. Mapping orange splotch “heatmap insights” to meaningful actions.
Even when the stars align and they provide accurate and meaningful aggregations of user interactions, the heatmap visualization itself doesn’t directly tell you what you want to know. Those aggregated splotches obscure the very thing that was interacted with. You have to read the heatmap tea leaves and interpret the splotches to figure out what you think the users were interacting with. What button, link, or control does that blob correspond to?
You might call this effort “splotch to element mapping:”
After seeing a heatmap and completing your back-of-the-envelope “splotch mappings,” what do you do next?
Here, heatmaps quickly turn into "analytics theater." Analytics theater is when an analysis—via a graphical visualization, chart, graph, metric, ect.—dazzles your senses with signals indicating profound insights while in reality being either completely useless or so difficult to parse into action as to become useless. You can read more about "analytics theater" and false signals here.
Heatmaps are notoriously hard to convert into actionable business intelligence ("actionable insights"), and if you can't make clear use of the insights, heatmaps become little more than beautiful distractions—analytics theater.
4. Yikes! I have to configure all that to make heatmaps work?
Most heatmap solutions ask you to configure complicated URL patterns and regular expressions to tell the system how to interpret interactions on a page-by-page level.
If heatmaps are broken, how do you map aggregated user interactions on your site?
Given the major problems heatmap tools face, we can't help but wonder if heatmaps are more analytics theater than actionable insights.
Having thoroughly examined heatmaps as a possible solution to meet our customer’s requests for a visualization of user clicks on a web application or webpage, we decided to go our own way.
We built Page Insights, which includes both Click Maps and Inspect Mode to make it easy to see how your users are interacting with your website, in aggregate, while gracefully dodging the four major problems of heatmaps.
Page Insights are resolution independent.
FullStory understands the structure of your site. FullStory can translate that into aggregations that make sense across screen resolution, device type, and even the most dynamic and varied data driven interfaces by focusing on Elements. Not just position.
Page Insights take into account what’s on (or not on) the page.
If you watch a playback of a dynamic single page application on FullStory, you realize that many things pop in and out of existence.
Page Insights are smart enough to account for these dynamic states and show you what’s relevant to what you are seeing on the page now. Click analytics for a non-existent popup won’t be shown when the popup isn’t there.
Page Insights are actionable.
You can click and see your answers immediately. And if you have questions you want to drill into, you can search. No orange splotch mapping to your web app required.
See here for an example of how you can run analysis using Click Maps.
Page Insights require zero configuration.
FullStory uses machine learning to know what the pages are in your web site or web application. FullStory understands what pages are the same, and what pages are different. So your insights are appropriately relevant and contextualized. No need to do complicated configuration or setup brittle URL route regular expression voodoo. It just works out of the box.
Bonus! Some new tricks.
Page Insights give you aggregations that respect the searches you do in FullStory, too (If you’re not familiar, here’s an explanation of FullStory search). This helps you answer questions like “What do visitors from my reddit ad campaign click on first?” Or you can see how they compare to “visitors that came from Product Hunt.”
Page Insights make your existing FullStory segments even more useful.
See ya later, web app heatmaps!
As for the default data visualization for web app interaction of yesteryear — that is, isarithmic density maps … er … heatmaps — until the day comes when we can reasonably manage the problems they create, we’re leaving heatmaps where they left us: out in the cold.
And if you want to try visualizing aggregate customer experience on your web application, check out FullStory Page Insights, see what you can discover and take action on, and let us know how we can make it even better.