I attended an interesting usability lecture/meetup today, with a talk by Jared Spool. I enjoyed the talk — it definitely got me thinking, but I disagreed with more than a few points.
Overall, I gathered that the takeaway from the discussion was to emphasize that you need to focus on the things that matter, and that a lot of the metrics that we use to measure website success, like bounce rates, conversion rates, and pageviews, don’t actually tell you what you need to know. I agree, but he seems to take this further, and advocate that these metrics aren’t even worth paying attention to. I disagree.
In his talk, he zeroed in on a couple of particularly bad (in his opinion) metrics:
For evidence that bounce rates don’t matter, he presented a graph over the past month of the bounce rate for an article he wrote. Because the bounce rates stayed nearly constant during the month, with a few random jumps here and there, he seemed to think that it means that it’s a useless metric. Of course he didn’t try, at least from his discussion, to affect the bounce rate (I guess perhaps because he thinks it doesn’t matter).
If he were trying to build his audience larger, you would think that he would actually try to reduce it (it was pretty high, like nearly 100% for the article). He could perhaps add a few highlights of similiar articles from that article page, or optimize which articles he shows. As a visitor, I might see some of those and click around on his site more (ie. reducing his bounce rate). Perhaps I even find that I like the new article that I’m reading, and now I’m thinking about visiting his site again since I’ve seen a couple articles that I like.
Yeah, bounce rates do matter, perhaps not in all cases, but especially in the example he gave, I would think it would matter. By focusing in on his bounce rate, perhaps he could gain a larger audience, which I’m sure is a metric that matters.
Conversion Rates Don’t Matter
Another metric he chose to go after, was the conversion rate. The reason for this…because you can have, for example, a two percent conversion rate, regardless of how many people visit your website. If you have more people visiting your site in the first place, then you’re going to make more money, which is the real goal, not the conversion rate. This sounds good, and of course you’re going to want to get more people to your website, but why not make your site more efficient in trying to convert as many as you can once they get there?
I would even take it further, in that it’s probably cheaper to improve your bounce rate, than to attract more people your website.
Does Anything Matter?
Do these metrics matter? Yes, of course they matter.
I could be wrong, but he seemed to be stating that these metrics don’t matter, because they don’t tell you ‘the why’ behind the any changes to these metrics that may happen. I would argue that they matter because they may be alerting you to a potential problem. I don’t think anyone has ever really stated that metrics are supposed to tell you “the why”. They are a staring point. From there, you can begin to invesitgate the reasons behind it.
In what was seemingly a contradiction in his stance against ‘pointless metrics’, he brought up an example where they saved a company 300,000,000 dollars over the course of the year by solving some shopping cart abandonment issues. They looked at what theoretically could be viewed as a pointless metric…pageviews. They examined each stage of the process when adding an item to a shopping cart through to purchasing, and noting where people seemed to drop out of the process based on the pageview metric. Of course the drop in pageviews that happened during one step of the process didn’t tell them why people were dropping off the site at that point, just like conversion rates and bounce rates don’t tell you why changes in those may or may not be happening. But in his example, the pageview metric told them where to look. Most of the dropoff did occurr during one phase of the process, and they were able to fix the issue.
I did gain quite a bit from the talk, perhaps even moreso than if I agreed with it all. Anything that gets you thinking about notions that you already accept is healthy, but I think I’ll need quite a bit more convincing on several of his points.