Design By Experimentation

September 3, 2013 — 3 Comments

Image

If you are remotely interested in design or building products in general, I’d recommend reading Dan McKinley’s slides or listening to his talk.  He discusses Etsy’s experimentation on the infinite scroll and search re-design features and the results are insightful.  “Bite off design changes in small chunks” is probably not an epiphany for many people, but, the slides walk through the pitfalls of blindly using ourselves as representative users or the perils of a massive redesign without checkpoints.

On Infinite Scroll

I wrote about how infinite scroll is not for everything before.  Turns out Etsy discovered exactly this.  Adding infinite scrolling to search results is like telling the user you will never be done with this task.  A task is only appealing if there is the possibility that in reasonable time, the user can make a dent on it.  Infinite scrolling leaves us feeling exhausted.  Another thing that infinite scrolling on search results tell you is that we gave you everything as we couldn’t figure out what’s best for your query.  It really does not build confidence in the technology.

Dan and this article more or less state the reason for infinite scroll not working in this scenario as unknown.  But, the basic human nature of taking on tasks we can complete seems to be the most fundamental explanation to me.

So, why did Pinterest succeed where Etsy didn’t on this feature?  The goals are somewhat different in each case.  Pinterest is primarily interested in engaging the user with interesting content that will keep the user on the site or app.  Or, simply put, it is a leisure activity.  One that can be addictive and turn into something more time consuming – nevertheless, it is still aimed at being a leisure activity (until other goals are apparent, this is true to the observer).  Here, the user is sitting back browsing through no one particular thing, just catching up with one of their many “social” activities.  Etsy surely also wants to serve the user lots of engaging content – but, they want the users to engage in particular ways that result in more sales at the end of the day.  Especially when the user is actively searching, navigating a never ending set of items is exhausting!

I wrote about Pulse’s change to infinite loading in my earlier article – knowing that I’m never done catching up with news is overwhelming (at the risk of considering myself a representative user).  Search results with an intent to purchase are similar – we want to feel like we are buying the best we can find – and we can never feel that we found the best when served an unending series of products (how would I know without looking at everything?)!

I realize that the savvy ones can set filters and sort by various parameters in a nested fashion to find the things they want.  But the average user doesn’t do that. So, serving the best content and personalizing that for the user is much more of a value add than just adding more items.

The Etsy experiment, while showing interesting data, also appears to be not so well grounded in early analysis to me.  Strictly based on the material I saw, it appears that there was not much of an effort to understand the purpose and advantages of the infinite scrolling feature.  If there was, then potentially there wasn’t a good attempt at matching it to their own needs.  Perhaps there was analysis prior to the experimentation – such hypotheses are exactly what A/B testing is for!  But, in this specific case, the misfit is somewhat obvious in my books at least.

So, I’d revise the design by experimentation just a little:

  • Understand the typical use of the feature you are attempting to add (and its general effect on your type of users)
  • Understand your needs (and write them down)
  • If there is a match, design, develop, measure and iterate in incremental chunks towards a bigger vision
Advertisements

3 responses to Design By Experimentation

Trackbacks and Pingbacks:

  1. How Much Do You Push Your Users? – A LinkedIn Case Study « TechBits by Vidya Narayanan - September 6, 2013

    […] is the age of continuous experimentation.  But, I wonder, just how much it is okay to push the users?  I think there is a point in user […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s