Is Your Search Engine Paying Off? 3 Steps to Monitoring Search Conversion Rates

A key question for any site with a search engine is: Is our search engine leading to more conversions? Whether you define a “conversion” as a sale, a sign-up, or some other action, it’s important to monitor how likely your users are to convert after a search. Further, it’s crucial to understand which search terms have higher conversion rates, and which have lower conversion rates, so you can improve your search results, enhance your metadata, or adjust the products or services you sell.

I’ll talk about some different ways to capture and analyze search data to answer these questions.

First, we’ll define the term “search success” as the likelihood that a given search term will lead to a conversion. That is, if 15% of users on your site end up buying something directly after searching for the word “pickle”, then the search success for that term is 15%.

In order to calculate this statistic for all your search terms, you’ll need to do three things:

1) Keep a count of how many times each search term is used on your site
2) Track search terms through to conversion
3) Keep a count of how many times a conversion happens after a search

Let’s talk about each one of these in turn.

1) Keep a count of how many times each search term is used on your site

The first step is store every search that happens on your site.  This can generate a lot of data, so I’d recommend offloading the job to a secondary data store instead of using your primary transactional database. That way, if a bot or malicious user generates a ton of search traffic, there’s no chance that it will affect your core dataset.

Since we’re interested in aggregate search counts, and not individual searches themselves, you can simply increment a counter every time someone does a search. Redis, a key-value data store, is particularly good for situations like this.

I also recommend storing the data asynchronously so it has no chance of delaying the serving of your search results. Since data integrity isn’t crucial for this application (a few lost searches won’t cause a big problem), a “fire and forget” asynchronous call every time a search takes place is a good path to take.

Finally, it’s useful to segment the data by day or week to make historical trends and comparisons possible.

With all that said, the data structure you’ll be creating will be a simple one that uses the day or week combined with the search term as a primary key, and a simple search_count field as another column. (We’ll add conversion_count in step 3.)

2) Track search terms through to conversion

Tracking search terms through to conversion can be done in a number of ways. The simplest approach is store the user’s most recent search in a session variable, and then read from that variable whenever a conversion event occurs. However, this approach has a few drawbacks: 1) if your customers browse with multiple tabs or windows, the search term stored in the session may not always match up with the one that leads to a conversion, 2) if someone bookmarks a search result page or sends it to a friend as a link, the connection to the conversion might be lost, and 3) if there are other conversion paths that don’t involve search, you may get inaccurate conversion statistics.

One option to avoid the first issue is to use the HTML5 sessionStorage property. This stores data separately for each window and tab, so will not cause search/conversion mismatches.

However, the more robust (but labor intensive) approach is to pass search terms through query strings from the search result page through to the conversion page. This assures that the conversion is an actual result of the particular search term, and also allows for bookmarking of and sending links for search result pages.

For example, if your existing links on search result pages look like this…

<a href=”/item/123″>Dill Pickles</a>

… you’d modify your code to dynamically add the search term to the links, like this:

<a href=”/item/123?searchterm=pickle”>Dill Pickles</a>

(Just remember to URL-escape your search terms to avoid XSS attacks.)

You’ll need to modify every link that leads from a search result page to the final checkout page (that’s where the labor intensiveness comes in), including the sign-up flow if that’s part of the process.

3) Keep a count of how many times a conversion happens after a search

Whichever method of tracking search terms you use, when a conversion event happens, you can then increment a counter in the same manner you used in step 1, with an additional column called (for instance) conversion_count. Then, to calculate search success you just divide the conversion_count column by the search_count column, and you’ll have the likelihood that each search term will lead to a conversion.

Analyzing the data

Congratulations! With this data, you have a goldmine of information. The best way to start using it is to review what terms have the highest search success and which have the lowest. Compare the search results from both sets of terms, try to learn what differentiates them, and modify your search engine, your metadata within the search engine, or even the products you sell to improve the search success of low-performing queries.

You can also review our other articles about product search engines and try out some of the techniques suggested there to improve the search success of queries on your site.

From there, you can experiment with different search interfaces to improve search success (perhaps by a/b testing new features and adding an “ab_cell” column to your search success table to track which variations win).  And you can also track search success over time to understand how and why it’s changing.

Although it will take some effort to set up a search tracking system like this, the benefits are enormous, and will provide multiple paths to increase revenue from your search system.

Does your site have lightning-fast, typo-tolerant autocomplete that ranks results to make you the most money? If not, check out Constructor.io.