Do You Know What Your Users are Doing? A Guide to Understanding Search Behavior

Understanding your users’ search behavior is a critical part of optimizing your site’s search experience.  Your goal is to tailor your search experience to your users, not to train your users to use your search engine.

Google has taught people to expect magic from a simple, typed query, and even though Google has thousands of search engineers and your company doesn’t, your users will nonetheless expect a Google-like experience from your site.

The first step towards addressing this is to understand how your users use your site’s search engine.  And the best way to do that is to watch them.  Grab some customers (or, short of that, co-workers or friends), sit them down in front of your site, and ask them to try to find something that they’re interested in.  Do this with at least 10 people, and I guarantee you’ll be floored by the ways people interact with your site that never occurred to you.  Here are a few things to look for in particular:

Do the words people type correspond to keywords in your search engine?

For instance, often people use adjectives in front of product names, like “black shoes” or “large shirt,” that aren’t indexed by search engines.  Note all the kinds of words that people use, and make sure those exist as keywords in your search engine so that searches that use those words will return results.

How deep do users page through your search results, and how many pages are usually returned?

The willingness of users to page through search results varies from site to site, and can have a big impact on the effectiveness of your search engine.  On some sites, users only glance at the first few search results before deciding that the site doesn’t have what they’re looking for.  In those cases, it’s vital that you work on reducing the number of overall search results and ensure that those results that are returned are highly relevant.  On other sites, users are quite willing to page through many results.  If that’s the case for your users, you can take advantage of this by adding more keywords to your product listings, so that more results get returned.

Do users try multiple strategies to find what they’re looking for?

Sometimes users will leave after a single failed search.  Other times, they have a few different strategies they employ to find what they’re looking for.  For instance, they may first try a very specific search like “Asics Men’s Venture 4 shoe” followed by a brand- or product-only search if they’re not successful, like “Asics” or “Venture,” and finally a very general search like “running shoe.”  If you see a user trying multiple strategies, ask yourself what you can change about your search engine to make every strategy more successful.  Ideally, each strategy should let a user find what they’re looking for.

As you watch users use your site, also pay attention to whether they’re new or returning customers.  Returning customers have a better understanding of your site and may rely on things they’ve learned in the past to inform their search strategies.  Ask them if this is the case, and what they’ve learned that has changed their behavior.  New users, on the other hand, will have no preconceptions, and you can learn a lot about ways to improve your search experience by watching their first interactions with your site.

The approach we’ve taken so far has involved qualitative research — sitting down with users and watching them.  It’s also helpful to employ quantitative research by analyzing your log files or using an analytics service to try to answer many of these questions at an aggregate level.

By looking through search logs that have been split up by user, you can gain more insight into what users type into your search engine and how they employ different strategies to search.  With some further information about paths users take through your site, you can answer questions about how deep users are willing to page through results.

One of the most useful pieces of information to collect about aggregate usage is overall counts of what people search for.  You’ll find that your search terms follow a power law distribution, which means that a small number of searches make up a large proportion of the total searches done on your site, and a large variety of searches make up the rest.

This discovery brings both good news and bad news.  The good news is that you can get a big bang for your buck by focusing on the most popular searches.  Since these make up a large proportion of your total searches, by improving the results of these searches you’ll be helping a lot of users.  I recommend looking at the top 100 searches for your site and going through them one by one to see for yourself what the results look like.  If any of the results are not optimal, modify your search engine to achieve better results (you may want to look at our articles on Turning a Document Search Engine into a Product Search Engine to help with this).

The bad news, however, is that the large variety of searches that form the “tail” of your search distribution are hard to improve because, well, they’re by nature a large variety of different terms.  Instead of optimizing them one-by-one, I recommend categorizing them and looking for commonalities among them.  You may find, for instance, that a lot of them include adjectives as we talked about above.  If that’s the case, it’s important that you include adjectives in your search engine.  Or you may find that there are a lot of misspellings, in which case you should add or improve the spelling correction ability of your search engine or use an autocomplete system that handles misspellings.  (The frequency with which users misspell words is why we spend a lot of time making the Constructor.io autocomplete service handle misspellings as smoothly as possible.)

By using both qualitative and quantitative research methods, you can gain insight into how your users interact with your site and search engine.  This will inform how you improve your search engine to ensure that it works for your customers, instead of making them work for it.

Does your site have lightning-fast, typo-tolerant autocomplete that ranks results to make you the most money? If not, check out Constructor.io.