The Search algorithm takes a series of inputs — the user's query, their location, the Search configuration, and the Knowledge Graph — and processes them to return ranked results. It first breaks down the query into tokens, then uses a multi-algorithm approach to determine the best results for each vertical, and finally ranks verticals against each other.
Tokens
Tokenization is breaking down a query into discrete units — words. Tokens are used to determine the candidates for matching to searchable fields. To derive tokens, the algorithm splits individual words based on whitespace, strips out casing and punctuation, and ignores common words (called "stop words") that do not add meaning to the query.
For a query like "What are your products?" the token derived would be products. A candidate for token matches might be an entity type of Product.
How Synonyms Impact Tokens
Synonyms translate tokens to different variations that mean the same thing. For example, although you might have jobs as an entity type, you might want the same results to appear for careers, positions, or vacancies.
How Stop Words Impact Tokens
The Search algorithm has a built-in list of stop words such as of, the, and in. These words can distract from the important tokens of a query — "the best bankers in the tri-state area" becomes "best bankers tri-state area," focusing on the entity, the rating, and the location.
Stop words are treated differently depending on which algorithm is used:
- Keyword Search: Stop words are given a much smaller weight than other words when matching on keyword search fields. Stop words are still used for token matching when no non-stop words are present.
- Inferred Filter: Stop words can still be matched in filters, but the algorithm will not match an inferred filter that only matches stop words. For example, if you have an inferred filter for "Cancer Care" and a stop word for "care," the query "Cancer Care" will pull the filter — but "Urgent Care" will not.
- Semantic Search: Stop words can still be matched on fields with semantic search since this algorithm uses a combination of token matching and semantic similarity scoring.
Set additional stop words with the additionalStopWords property.
How Custom Phrases Impact Tokens
Custom Phrases are multi-word phrases that the algorithm treats as a single unit when matching results. This is useful for brand-specific phrases you don't want partially matching with results.
Custom phrases only affect fields using keyword search, phrase match, or inferred filter. If a field uses semantic search, partial matches may still appear due to semantic similarity.
For example, a taco shop might add "corn tortilla" as a custom phrase to prevent a query for "corn tortilla taco" from returning results like "corn on the cob" or "tortilla soup." However, if the field uses semantic search, a query for "corn tortilla" could still return "flour tortilla" because they are semantically similar.
Search Operators
Search operators allow users to control how tokens are combined in keyword search. Operators are case sensitive and typo-intolerant.
| Operator | Behavior | Example |
|---|---|---|
AND |
Both tokens must appear in a searchable field | red AND dog |
OR |
Either token must appear in a searchable field | blue OR cat |
NOT |
The token must not appear in any searchable field | NOT yellow |
"double quotes" |
The entire phrase must appear in a searchable field | "open now" |
These operators can be combined to apply more complex logic. See the Search Operators reference doc for more.
Multi-Algorithm Approach
Within each vertical, Search takes a multi-algorithm approach based on the type of content being searched. You choose how each type of content is searched by setting searchable fields in the configuration. Learn more in the Searchable and Display Fields unit.
Structured Data: Phrase-Based Searching
For structured data with inferred filtering configured, the algorithm looks for a field value's tokens in the query (e.g., "blue hat" or "doctors who accept blue cross blue shield") and applies those tokens as a strict filter. This works well for structured entities like products, events, and jobs.
The algorithm uses Named Entity Recognition (NER) for location search based on the user's proximity bias and the prominence of the place.
Semi-Structured Data: Semantic Search
Yext Search uses semantic search for FAQs and Help Article names. Instead of relying on keywords, the algorithm embeds the search query and FAQ or Help Article names in vector space and determines the most relevant result. This allows it to identify content that is similar in meaning to the user's question — for example, recognizing that "how is covid transmitted?" is semantically similar to "how does the virus spread?" without requiring synonyms.
Unstructured Data: Keyword Search and Document Search
Keyword search relies on keyword matches between the query and the result. When keyword search is enabled on a field, Search returns results whenever there is a match between a query and field token (unless they are stop words). When ranking, Search considers the relative importance of the token with respect to all of the Knowledge Graph.
Document Search uses the keyword search algorithm to search through long-form documents such as blog posts, help articles, and product manuals. It uses extractive QA to pull out relevant snippets that answer the query.
Entity Ranking Logic
Once the algorithm finds the most relevant results, it ranks entities within the vertical using the following elements (overridden by any custom sorting configured — see the Sorting unit):
- Location radius with location intent: If location intent is present, results are filtered down to entities within an appropriate radius of that location.
- Relevance: Number of matched tokens; entities with the same number of matched tokens but more non-matched tokens rank lower.
- Location distance: Proximity to the user's location or a specified location.
Vertical Ranking Logic
Vertical ranking determines the order in which different categories of content (FAQs, Locations, Help Articles, etc.) appear to best address the user's query. It works in three steps:
Step 1: Gather Clues
For each vertical returned in a search, the algorithm gathers clues to gauge how relevant that vertical is to the user's query. Examples include:
- The name of the vertical
- Any filters applied on the vertical
- Fields from the top result of each vertical
- Any vertical intents provided in the configuration
Step 2: Generate Vertical Scores
The algorithm compares each clue semantically to the user's query — evaluating how "close" they are in meaning, using the same machine learning model as semantic search. Each vertical receives a score between 0 and 1.
Step 3: Rank
Verticals are ranked based on their score. Two additional properties influence final ranking:
- Thresholds: Minimum score requirements a vertical must meet to be returned at all.
- Biases: Adjustments to the final score. A positive bias is added to the vertical score; a negative bias is subtracted.
One caveat: a vertical that produces a displayed featured snippet will always be ranked first, overriding any other vertical ranking logic, since it directly answers the user's query.
You can set intents, thresholds, and biases in the Search configuration for each vertical. Learn more in the Vertical Ranking unit.
Other Components Controlled by the Algorithm
Query Suggestions: Every time a user clicks into a search bar, an empty request is sent and the API returns hardcoded prompts. As the user starts typing, popular queries beginning with the search term are returned.
Spell Check: For a given query, the Search API can return spell check corrections. Clicking on the suggestion re-runs the search with that spelling. Each account has a separate spell-checking dictionary made up of a generic dictionary for the supported language, historical search queries, and your Knowledge Graph.