Turning a Document Search Engine into a Product Search Engine (Part 2)

In our last article, we talked about the differences between product search engines and document search engines, and suggested four ways to make your document search engine work better as a product search engine.

Here are a few more ways to improve the results from your document search engine.

1) Add product properties like sizes, colors, and styles to your keyword lists

You may have a product named “Men’s Formal Shoe — Executive” with keywords “shoe, men’s, formal”. If you look through your search logs, you’ll likely notice people performing searches like “mens formal shoe size 10” or “black mens shoe”. These searches probably won’t show your men’s shoe because the size and color aren’t in the keywords list.

To solve that problem, be sure to add product properties like size, color, and style to your keyword list. That way, searches for those keywords will return the correct results.

2) Harness the power of bi-grams

Suppose you have two products: “Laundry Enhancer Softener – Fresh Spring Waters” and “40 lb. Water Softener Salt Pellets”. Someone searching for “water softener” will only want the second one of these, but your dutiful search engine will return both results because each product name contains the words “water” and “softener”.

Your search engine likely supports the idea of bi-grams: indexing two consecutive words together. (“Bi-grams” refer to sets of two words that are next to each other. You may also see the term “n-grams” which refers to sets of any number (“n”) words that are next to each other.) This lets consecutive words in the product title rank higher if they match consecutive words in the search term. For instance, in this example, “water softener” would be more likely to match the second product because the words “water” and “softener” appear next to each other and in the same order as the search query.

If your search engine doesn’t seem to be using bi-grams to rank search results, look through the configuration settings to see if this is possible.

3) Review stemming rules

Document search engines are designed to simplify words to their root form (or “stem”) so that, for instance, searches for “shoe” and “shoes” return the same results. This is usually a good thing, and saves you from having to add singular and plural forms of every keyword in your dataset to your search engine.

However, stemming can get you into trouble with certain words. “Fishing,” for example, stems to “fish”, which can lead to erratic search results if your data contains both terms. In general, beware of words that can be used as multiple parts of speech (“fish” is a noun, “fishing” is a verb), lest your “cream-colored whips” start appearing in searches for “whipped cream.”

If you’re having trouble with stemming, you can consider indexing both stemmed words and non-stemmed words, and boosting non-stemmed words in the search results. This would cause “whipped cream” to be indexed with exactly those words, so a search for that term would first return whipped cream products.

4) Add popularity metrics to your search index

Document search engines are fine-tuned to return accurate, relevant results based on the words in a document. They usually aren’t particularly concerned about how popular a document is, however. This often works against you if you’re using them to search product names, because you’ll find they rank obscure, little-purchased items higher than bestsellers.

You can work around this by storing a popularity metric for each product in your search engine, and boosting your search results according to that score. A simple way to do this is to simply count how many times each product is sold, and store that number in your search engine. Then, configure your search engine to return better-selling products higher in the results than worse-selling products. This will put bestselling items at the top of your search results, which will more accurately represent what your customers are looking for.

These four approaches can greatly improve the products that your search engine returns. We recommend you periodically review these suggestions together with those in the first article to continually hone your search results and improve your site’s search experience.

For a super simple, quick way to improve your search engine, check out our autocomplete as a service.