The Evolution from Keyword Search to AI-Powered Search

For many years search was very simple. Search engines would take a query and try to match it to any document that contained those keywords. Keyword search had been around for decades with little innovation.

But then things started to change. Consumer search engines started to develop algorithms based not on keywords but on natural language understanding and AI. This led to the rise of AI-powered search. A user can now ask a question and get a direct answer.

For some reason, none of that innovation made its way to enterprise search platforms. They are still delivering the same old keyword search while the consumer web has continued to evolve. That's why site search and enterprise search are still so bad at most companies.

Keyword-Based Search

The traditional method to power site search is keyword-based search. It works by crawling every page on your website, indexing each word in a document, and running keyword searches against those documents. This is how Google worked 15 years ago, but it's not how it works today.

Often search systems will use features like typo tolerance, synonyms, stemming, and other advanced features to make keyword search better, but it's still simply keyword search. Google switched to a knowledge graph because traditional keyword search just doesn't produce the quality of results that users want.

AI-Powered Search

AI-powered search is a search technology that understands the words of a query in context and produces results relevant to that context. It is built on a knowledge graph and uses natural language processing technologies. The knowledge graph's structure helps the search engine understand the entities, their attributes, and how they are related.

Your customers have been trained by modern search engines to simply ask a question and get a direct answer in return. People type phrases and queries that have meaning, and the words have relationships to one another that convey the user's intent. A search for a "dermatologist who speaks Spanish and takes Cigna" isn't just a set of words - it's a phrase that conveys specific parameters about the visitor's desires. When someone runs that search, they should see dermatologists, not just a list of pages that contain the words "speaks" and "takes."

Key Characteristics of AI-Powered Search

AI-powered search can handle natural language questions. We have seen an explosion of natural language experiences from desktop and mobile search to voice search to chatbots. The ability to interact digitally via natural language is the next big shift in user experience.

AI-powered search leverages multiple algorithms. Different types of data are best searched using different algorithms. Whether you are searching images, help articles, FAQs, or anything else, AI-powered search leverages the best algorithm for that type of data to deliver the best answer.

AI-powered search leverages a knowledge graph in order to deliver a direct answer to your question. A knowledge graph is a semantic data structure, built on real-world entities and relationships. Now when someone asks "What is the phone number for CGH Medical Center?" they get a direct answer instead of a list of documents.

The Problem with Bad Search

Search is on your website, your apps, your help site, your intranet, and nearly every other place people look to engage with your brand. People expect the ability to ask questions in their own words and get a direct answer. That's why the best digital experiences are built on natural language search.

But when users make it to your website or app, often the experience doesn't live up to their expectations.

Let's say your friend recommended a shoe store and you wanted to find out how much a particular pair of shoes costs. You'd probably go to their website and start clicking around. But if it takes more than a few clicks, you're probably going to give up. You don't want to click around on a bunch of pages when you just want to ask a question, the same way you can on Google.

So what do most users do? They give up. They bounce back to exactly where they came from.

And when they bounce back to a search engine, they could find wrong information or, even worse, one of your competitors. Bounce rate is important to consider for internal use cases as well. If a user leaves your platform frustrated without answers, they will remember this negative experience - not to mention, you'll waste time and money answering one-off questions as a result of a high bounce rate.

There is a better way. On Google, if you ask "how much are madison avenue canvas sneakers?" you would get something like this:

An answer to your question. If you bring this experience to your website or app, people can ask questions the same way they're used to doing on Google, Alexa, and Siri.

Note: Enterprise search does not impact 3rd-party search. It only impacts the experience on your own website.

The Negative Impacts of Bad Search

Poor Conversion Rates: People come to you for answers, not more questions. Users are more likely to bounce back to search if they encounter a bad site search experience.

Increased Support Costs: When people can't find the answers they are looking for, they are more likely to email, chat, or call customer support.

Minimal or No Insights: Understanding user intent is key to providing them with the information they seek. If users aren't searching with complex queries, or you aren't able to capture them, you are missing out on valuable insights.

Key Terms and Concepts

Search Engine Results Page (SERP) Components

Search Engine Results Page (SERP) - The pages displayed by search engines in response to a query. These can include links, listings, or other rich content.

Featured Snippet - Search results with rich data displayed. This extra data is often pulled from structured data found in a page's HTML. Common featured snippet types include reviews, recipes, and events.

Local Pack - A list of local listings relevant to your search, typically 3-4 local listings and often including a map of where the results are located.

Knowledge Panel - Rich search results surfaced as information cards when you search for entities (locations, people, products, organizations, etc.). They provide a snapshot of information on a topic based on content around the web that is stored in the search engine's knowledge graph.

Organic Search Results - The web page links that most closely match the user's search query based on factors determined by the search engine. Some organic results appear as links below, but other areas of the SERP like the local pack can be organic as well.

Paid Search Results - Results paid for by the web page owner to show up for specific keywords. These are typically differentiated so consumers can identify which results are organic vs. paid.

Schema.org Markup - A joint effort through collaboration of the major search engines to improve the web by creating a structured data markup schema. Web developers can use this universal vocabulary by embedding it in the HTML of a page to communicate to search engines universally. On-page markup helps search engines understand the information on web pages and provide richer search results.

First-Party Reviews - User reviews collected by a business using their own web properties, often displayed on a business' owned properties such as their website or app.

Third-Party Reviews - User reviews collected on third-party websites such as Google, Facebook, or Yelp.

Site Search Concepts

Bounce Rate - The percentage of site visitors who leave your website after visiting a single page.

Crawling - Using automated software to scan through webpages in order to digest the website's content and make it searchable. Website crawlers follow links on a page to continue to fetch and index content.

Direct Answer - A concise result, such as a knowledge card, phone number, calorie count, or other piece of discrete information that addresses the intent expressed in the user's query. Direct answers are possible when the search technology uses a knowledge graph to map relationships between data, as opposed to indexing keywords.

Entity - A real-world object within your business' knowledge graph, such as a job, an event, a location, a professional, an FAQ, or a product.

Federated Architecture - Also called "Federated Search," a technology framework that allows a search engine to search across multiple data sources and aggregate the results in one unified experience.

Indexing - A technique used to find and organize data by mapping keywords to documents, often pulling from many different databases. (Note: Indexing data is a technique used by both keyword-based and semantic search. This is different from "indexing keywords" as described in keyword-based search above.)

Semantic Search - A search technique in which a search query aims to find keywords and determine the intent and contextual meaning of the query to improve surfaced results.

Document Search - Powered by the Extractive QA algorithm, Document Search searches unstructured documents such as blogs, bios, or help articles and provides direct answers to searchers.

Universal Search - A form of site search that searches across different Vertical Search experiences via a federated architecture to deliver users the best results. It can be used as an entry point into Search on your website.

Vertical Search - A search tool that isolates a user's search to a single entity type, like locations, professionals, or products. The UI of Vertical Search can vary widely and a good Vertical Search should adjust the UI depending on the type of entity being searched.

Machine Learning - The technology by which algorithms learn and improve over time by analyzing data and patterns in that data, without requiring explicit programming or human intervention.

Dynamic Reranking - Uses machine learning to rerank results based on user engagement data. The model reranks results in order of their likelihood to be clicked and becomes smarter over time as it is fed more user interaction data.

Query Rules - Administrators can set custom rules so that when certain search criteria is present, it triggers a particular action in the search results. For example, you can set a rule that if the query contains a certain keyword, a particular entity is boosted in search.

Experience Training - Allows administrators to train the search algorithm by providing feedback on its predictions. With input on a specific search term, you can fix that query instantly and help Search improve the relevancy of results over time.

The Value of AI-Powered Search

When companies adopt AI-powered search technology, they see significant value across every team:

• Marketing teams see a 1.4x increase in onsite conversions with AI-powered search
• Support teams see happier customers with higher CSAT scores and reduced Time to Resolution
• Developers are able to leverage the latest and greatest in AI-powered Natural Language advancements
• Ecommerce teams see a 50% increase in average order value by leveraging machine learning
• Workplace teams drive better productivity as employees find content faster with AI-powered search

More broadly, when you exceed user expectations with a great search experience, it drives success across three areas:

Higher conversion rates - Great answers mean higher engagement. When more people can find what they want on their own, they are more likely to take action.

Lower costs - When users can find answers on their own, you spend less on live chat, support centers, email, and other costly help channels.

Unlock customer intelligence - By seeing how your users search, you gain a new source of customer intelligence that you can use to improve your Knowledge Graph and your other marketing strategies.

Introduction to Yext Search

Yext Search is an AI-powered search platform built to optimize your users' search experience. With the Yext Answers platform, you can:

• Build your own enterprise knowledge graph
• Leverage natural language search algorithms to build AI-powered search experiences
• Push your structured data to 3rd-party search engines
• Build search-optimized web pages built for conversion and scale

Yext Search is different from other search solutions for three reasons:

It understands natural language. Unlike other search solutions that rely on keyword-based search to crawl documents and match text strings, Yext Search uses structured search that understands natural language and delivers clear, concise answers in the form of knowledge cards, maps, and other relevant results. Whether you're searching with a single keyword or a long question, Search understands all kinds of search queries and parses the user's intent to return the right answers.

It's built on the Yext Knowledge Graph. Search is just one of many experiences that your brand's data can power. Your Knowledge Graph stores all of the public facts about your brand and the relationships between them, so that AI-powered services of all kinds can surface answers wherever your customers are searching.

It's built for multiple personas. Anyone can build on top of it - whether you are a line-of-business user or a full-stack developer. We have integration options that come ready out of the box with all the tech and user interface, and we also have other integration options such as a low-level Search API that allow full-stack developers to create other use cases such as a chatbot.

The Multi-Algorithm Approach

In order to provide the best search experience possible, Yext Search incorporates several unique algorithms. The performance of an algorithm will vary depending on the query and the type of results the searcher is looking for, which means that one single algorithm will never be the best all of the time. In order to optimize performance across all queries, Yext Search relies on different algorithms in different circumstances.

After using Natural Language Processing to understand and map the user's input, the Search algorithms can detect the user's location, make spellcheck suggestions, detect filters, output query suggestions, return matched Knowledge Graph entities, return featured snippets and direct answers, and more.

The three algorithms in our Multi-Algorithm story:

• Named Entity Recognition - Best for structured data like locations, events, products, jobs, doctors, or advisors
• Semantic Search - Best for semi-structured data like frequently asked questions
• Document Search - Best for unstructured data such as blogs, news articles, press releases, bios, product manuals, or help articles

Every Yext algorithm gets smarter over time.

Yext Search Use Cases

Yext Search can be applied across five solution categories:

Marketing Search - Deploy AI-powered search on your websites to power site search, or build specialized search experiences like a career site.

Support Search - Add AI-powered search to your external help sites and internal agent portals to resolve customer issues faster.

Developer Search - Add AI-powered search to the products your developers build.

Ecommerce Search - Leverage AI-powered search for product discovery and purchase flows.

Workplace Search - Create a powerful employee intranet search experience.

The same way Google brought AI-powered search to the consumer, Yext is bringing AI-powered search to the enterprise. If you don't want to build everything yourself, you can leverage the App Directory, which is filled with pre-made search solutions and third-party partner apps that can be easily installed or used as templates.