How To Choose eCommerce Search Technology?

eCommerce store without a search is like a physical store without consultants. It’s so much easier to simply say what you are looking for and be directed right to it than to wander around the store for hours. 

We came to the point in the market, where we don’t even question whether or not eCommerce website needs a website search – it’s a must. In fact, site search is considered to be the main navigation option for e-stores. Well-optimized on-site search can bring tremendous benefits, starting from an increase in site usage, finishing with higher sales.

If by any chance you still don’t have a site search option on your store, or you are not sure whether it’s worth to invest into making it better, here are the few benefits that eCommerce search might bring:

• Around 30% of all your site visitors will use search
• 25% of them will click on a search suggestion
• Also, they are 5-6 times more likely to convert
• Based on our clients’ data, visitors, who use search, can generate around 30-60% of ALL revenues

We are sure that by now you understood, that the question is not whether to do it, but rather how to do it. But deciding to change something for the better is just a first step in a journey of eCommerce search development. Maybe it is best to hire some developers and do it yourself? Or maybe choose already developed search engine from SaaS vendor? 

Currently, the industry is stuffed with different options, and site search providers, each claiming to be better than others, making it very hard not to get lost. The truth is, that none of the choices are the same, and each of them has its pros and cons.

We made this overview of all the possible options, to help you find the most suitable one:

1. In-House with Elasticsearch or Solr
2. SaaS Vendor
3. The Rise of the Hybrid: SaaS + Search Solution Developers

1. In-House with Elasticsearch or Solr

The first option you could consider is building eCommerce search yourself. Two of the most popular open-source search engines are Elasticsearch and Solr, which we talk about in this article. Of course, developing a site search internally presents its own unique set of challenges, as well as rewards. Let’s start with overviewing the process itself.

There are typically five development stages involved when trying to reach a very relevant search experience:

1. Setting up Elastic or Solr (~1-4 weeks). These open-source search solutions provide a very rudimentary site search. The process of setting them up can take anywhere from 1 to 4 weeks. In this step, everything relies upon performance requirements, requests, and amount of products, thus the timeframe can extend greatly. 

2. Advanced Data Processing (~2-8 weeks). Technology without well structured and optimized data for search will not help much. The data should be prepared for search: attribute values should be normalized, sentences should be processed, by extracting relevant words based on their part in the sentence, and other custom tasks should be done. It can take from 2 weeks to 2 months, depending on the field experience of engineers.

3. Configuring Basic Features (~4-8 weeks). During this step such basic features, as tokenizers, default stemmers, spellcheck, default facets, synonyms and the rest, should be configured. This stage of development can take around 1 or 2 months.

4. Configuring Advanced Features (~20 weeks).  This includes features like advanced stemmers, custom filters, autocomplete, keyword suggestions based on NLP (natural language processing), contextual dynamic facets along with others. This stage typically takes a minimum of 5 months to complete. 

5. Custom Ranking (~1-4 weeks).  Intelligent site search should find not only relevant products but reflect business needs as well. In this manner, there should be custom ranking algorithms developed, which would make sure that the business maximizes its objectives. It can take anywhere from 1-4 weeks of initial work + 1-2 days per month on continuous improvements and experiments.

6. Custom Search Functionality (min. 3 weeks). Considering that every eCommerce business is different, it has very custom needs for search as well. To develop and prepare features like autocomplete box, search assistant, voice search, personalization and the like, could take from 3 weeks to many months – depending on the individual needs.

7. Post-Live Launch Bug Fixes and Algorithm Improvements (~3-4 months). The final stage of development is never truly finished. It’s an ongoing process, but the first 3-4 months is much more intense and involved. 

Even with a fully staffed, highly-experienced development team on-hand, this process can take anywhere from 12 to 15 months, and that’s not so short, especially when with alternatives you could enjoy growing sales in a much shorter period. What’s more, many new and unforeseen challenges often crop up during the development process. For testing periods and other delays and it’s possible that an in-house search, even when using an open-source solution as its core, can take as much as 2 years to be semi-functional. 

One more thing to consider is that if developers haven’t worked with Elastic or Solr in the past, several additional months worth of training and learning the system adds up to the timeline. Of course, all of this differs from developer to developer, but it can be difficult to find those who have existing experience working with Elastic or Solr. Even if you manage to find someone who knows Elastic/Solr, they may not have had experience with building a search product for eCommerce, thus time to get up-to-speed on the various facets of the task can take several more months.

Advantages of In-House Search

Unlimited Customization
Having a search engine built from the ground up allows you the freedom to add a variety of customization that mesh with your business goals and objectives.

Flexibility in Terms of Making Priority Bug Fixes
Where in-house development truly shines is in terms of its resilience. Having a team on-hand can allow for greater modifications and the assignment of priority bug fixes, as the team is separate and independent from solution providers. 

Results Can Be Leveraged Beyond Search
One more advantage is that the result can be extended to multiple types of usage, such as a recommendation engine.  

Disadvantages of In-House Search

In-House Search Team Recruitment
While finding developers with basic experience is moderately simple, finding developers with extensive in-house site search development experience is much more difficult. Because so few developers have these niche skills, they are in incredibly high demand. Likewise, turnover rates are extreme, creating the need to continually on-board and train new developers, and boosting overall costs.

In-House Search Team Management
Not only do these developers need to be hired and on-boarded, but they also need to be managed as a team. This can add strain to existing managerial employees as they must now oversee additional team duties to keep everyone on track. 

It Takes Years to Fully Develop Great Search
As noted above, a great site search requires a great deal of oversight, planning, monitoring, and upkeep. Even with the smoothest of development processes, it can still take years to see the project through to completion. 

In-House site search Requires a Significant Amount of Manual Infrastructure Setup and Maintenance
Beyond the development and managerial talent needed to see the project to fruition, in-house site search also needs reliable, robust and scalable server infrastructure. Once this hardware is set up, it must be maintained and upgraded, as well as continuously monitored so that it functions at optimal capacity. 

“In my experience, launching is not the tricky part of search software. The real work is managing it in production. You have to expect that nodes will occasionally fail, the cluster topology will eventually change, your product will add and update features (which may have unintended consequences for search), traffic will ebb and flow, etc. You may also eventually want to tune relevancy, which is complicated enough that some people make a good living specializing in nothing but relevancy tuning.”

Allison Hammond Zadrozny, Software Engineer at One More Cloud
Source: Quora

The Need to Monitor, Optimize and Improve Search Performance Can Be Tedious and Expensive
In-house site search development is not a “once and done” project. It requires ongoing maintenance, bug fixes, continuous improvements and innovations to stay ahead of the curve and meet customer expectations.

Lack of Relevant, Up-to-Date Technology 
Search is a highly specialized and highly complex set of tasks. Oftentimes, there simply aren’t enough resources to keep investing in something that can take months to launch, let alone optimize efficiently.

Pro tip: If you would like to learn more about building site search yourself, here is a recent video we made about a common misconception that we often hear from CTOs. 

Overall, the high expenses and longer development time can make a truly immersive and engaging site search difficult to realize. On the other hand, a fully-developed in-house site search is highly flexible and there is much less risk in terms of downtime or communication issues.

2. SaaS Vendor

SaaS vendors, too, differ from each other regarding what they have to offer. From simple plug-and-play site search plugins to sophisticated enterprise vendors, every SaaS company has their specific specializations and niches where their product gets the best results. 

As eCommerce site search is the core of their business operations, they’ve already done a great deal of legwork in terms of setting up a robust, reliable site search with the kinds of features that users expect from their search, like natural language processing, autocomplete, spell-check and more. 

Advantages of using a SaaS Vendor

Versatile Solutions that Scale with Your Company
SaaS vendors offer scalable search, so whether you’re a small eCommerce company with a handful of products or a large enterprise with thousands upon thousands of products, the right SaaS vendor can customize your site search engine with a variety of advanced options. 

Predictable Expenses
Because it’s a hosted solution, there’s no need to pay for server hardware, upkeep or maintenance. All of this is factored into a single monthly fee. There’s no need to hire an in-house or outsourced development team, nor pay their salaries or insurance costs.

Better Search in a Shorter Amount of Time
Because SaaS site search vendors specialize in search, they’ve already perfected many of the must-have features that customers demand from their site search experience. Things like natural language processing (NLP), advanced stemming, autocomplete and spellcheck are integrated into the core offering so that businesses can leverage that feature-set to launch a cleaner, more efficient site search in far less time. 

Disadvantages of Using a SaaS Vendor

Lack of Autonomy
Using a SaaS provider means that you’ll be very dependant on the vendor, and SaaS product capabilities. Usually, it is a one-size-fits-all type of solution, with elementary support for maintenance, and narrow possibilities for search improvements in the admin dashboard. As functionalities are very limited, the search potential itself is limited too.

Dashboard is Not a Scalable Solution
Most of the SaaS providers offer a dashboard, there their client can make such simple changes themselves, such as creating redirects, adding synonyms, and modifying rankings. This creates a feeling that you have control of your search and that you are independent of the vendor. However, this is not entirely true. Every time you add a new product or need to change something, you have to set up and add these rules manually product-by-product. If you have thousands of products added every day, that’s much more than a thousand manual actions, which need to be made by hand. 

All in all, with a SaaS vendor, you’re not paying the salary, insurance and other costs of hiring an in-house team, yet you’re enjoying many of the same benefits and features. Also, when compared to other expensive endeavors, there are relatively few risks or disadvantages to working with a SaaS vendor for search.

3. The Rise of the Hybrid: SaaS + Search Solution Developers 

The last alternative is to use the hybrid between the first two options. It is a pretty unique practice in the market, which is offered by SearchNode. The company is changing the definition of what it means to be a SaaS provider, by offering all of the benefits of an eCommerce site search engine, but with the availability and skills of an in-house team. You could call them a hybrid between SaaS and consulting partner.

When using SearchNode eCommerce stores gets excellent site search product within a short period. Most importantly, it is not a one-size-fits-all solution, which means that dedicated search solution developers work on each case individually, not just provide advice, or work only on a global product. After initial implementation, continuous search improvements process starts, so the improvements for search are implemented very quickly. This means that the original search gets even better over time, thus the potential for its quality is unlimited. By doing this, SearchNode can offer the finest search product for its clients, and clients can enjoy much better search experience than they would get with just a usual SaaS search provider. 

Advantages of using a Hybrid Solution

Team of Search Experts
SearchNode works as a consulting partner by fully integrating the service with the eCommerce website and provide an unparalleled customer experience that is seamless, fast and accurate. This one-on-one guidance and investment in success are what makes SearchNode stand out from the crowd of site search SaaS vendors. They can even develop custom features for your site search based on your business needs and goals.

State-of-the-Art Technology
By combining the custom development services, eCommerce search expertise and the intelligent technology into one unique solution, SearchNode is able to improve its own product, algorithms, configuration based on feedback while other teams of search consultants and experts work with you to fix bugs and proactively analyze your site’s search behavior to determine areas of improvement that will bring you the best possible return on investment. 

Scalable Search Improvements
As was mentioned in the previous part, most of the typical SaaS solution offers a dashboard, where the client can easily make some basic changes. SearchNodes’ way is to modify and change core relevancy and data processing algorithms, not individual cases. Or in other words, when there is one rule applied for one product, using intelligent algorithms, it is applied for all of the similar cases, not just that product alone. 

Long-Tail-Query Optimization
Often eCommerce companies do not pay attention to the long-tail of their search queries, although huge revenue potential lies there. Long-tail queries have a lower frequency, but they outnumber the ones in the Head. Despite this, many of them are never even checked because when you try to fix everything by hand (usually in the dashboard), you have enough time to focus only on the most common queries. SearchNode solution makes sure your long tail is optimized as well, for maximum gains. 

Easy and Fast Set-Up
When eCommerce stores start working with SearchNode the only thing they need to do is to provide a data export (feed or API) and discuss their business needs, and ranking rules with the product manager. Afterward, SearchNode’s search engineers adapt search algorithms to meet the client’s business and language specifics, provide custom-ranking algorithms, optimize long-tail queries, and make data-driven improvements. They usually spend 3-6 weeks to build the solution which the client can launch live.

Continuous Search Improvements
When it’s launched live, a team of SearchNode engineers and a client starts a continuous improvements process. Search experts from SearchNode work closely with each client to deliver only the best search experience. This means that the search solution gets even better over time, as a dedicated developers team improves patterns, instead of manual case-by-case fixing.

Disadvantages of using a Hybrid Solution

Dependence on Solution Provider
Similarly, as in the usual SaaS case, you’ll have to count on solution provider a lot. Also, eCommerce companies often feel intimidated, especially when they find out that they won’t get any dashboard kind of solution, but it’s easy to overcome this fear, as when the scalable search solution is offered, there is no point in doing some individual changes by hand. 

You May Need to Relocate Some of Your Employees
Once you start working with a hybrid solution provider, you won’t need to have a team of developers, working full-time for search, as most of the engineering things to ensure best possible search experience will be done by the vendor. If you had a team of engineers/administrators, who worked on hardcoding your search before, you won’t need their services anymore. Thus, you may need to say goodbye to them, or at the best case transfer them to another position.

It’s beneficial to have a product manager who can spend some time per month to collaborate with SearchNode search experts. This way continuous search improvements will bring even more ROI.

It Could Be Not Affordable for Small Online Stores
A hybrid solution is crafted for medium-big eCommerce companies. For smaller ones, with a narrow assortment, and little traffic, a simple solution could be enough. The reason for this is that they don’t need complex, and custom solution, as there is not much space for a search to scale when the product catalog is small. A hybrid solution is relatively costly, so they won’t see enough ROI.

“When we started SearchNode in 2013, our flagship product was developed with a robust API to make it easier for customers to integrate it. However, even with this interface, it was still a challenge for eCommerce companies to customize and configure SearchNode to meet their needs.

That’s when we decided to go beyond a simple API to start creating custom solutions. We have since developed SearchNode into a fully scalable product that is adaptive enough to meet the ever-changing needs of today’s growing eCommerce companies. From our experience developing custom solutions, we also began offering our services as a development partner, helping our clients further expand and build upon their search.

Fast-forward to today and we now have separate teams involved in every aspect of the search experience, from infrastructure and processes to strategy and development. This approach allows our clients to enjoy the best possible ROI and know that they are working with a true market leader. “

Antanas Bakšys – CEO and Co-founder at SearchNode

To wrap up, SearchNode not only delivers outstanding site search capabilities but also work alongside you to help fix bugs, adjust search terms, improve search queries and refine and monitor your site search so that your customers and end-users can enjoy the best possible site search experience. It’s changing the way that today’s business executives look at working with a SaaS to deliver a well-rounded search experience for their users. Most importantly: in a scalable way, not hardcoding with synonyms, redirections, boosts! In the end, SearchNode is like getting a quality of in-house developer team, without having to hire one.

The Final Decision

To conclude, choosing how to develop your eCommerce search, shouldn’t be a fast decision. Before making your final verdict, you first should audit your site search well, to understand its strengths and weaknesses. We developed a framework of the perfect eCommerce search – feel free to use it for analyzing your search. Also, if you want a more extensive examination, we offer a free service of eCommerce search evaluation, which will be conducted by search experts from SearchNode. 

As you can see, there are many different options on the market available, and each comes with its pains and gains. By learning the needs of your search, you will be able to make the best decision possible, and you will start ripening the juicy fruits, that well-optimized eCommerce search can bring. 

Disclaimer: This post was developed based on SearchNodes’ extensive experience working with medium-big eCommerce companies. 

What is your experience in choosing site search technology?
Feel free to share your opinion below in the comments section.