OpenSearch Optimizations for Production RAG, Part 2: Lexical Retrieval

This is Part 2 of a series on optimizing OpenSearch for production RAG. Part 1 covered semantic retrieval , meaning vector search with Approximate and exact Nearest Neighbor methods. This part covers lexical retrieval, meaning text-based search, and how it complements the semantic path. Introduction Retrieval Augmented Generation (RAG) systems retrieve a set of relevant documents and pass them as context to a Large Language Model (LLM). Part 1 discussed the semantic path, which retrieves documents whose embeddings are near the query embedding in a vector space. This part covers the lexical path, which retrieves documents that share tokens with the query. Lexical retrieval is often described as the older, weaker sibling of semantic retrieval. That framing understates what lexical retrieval contributes. Semantic retrieval is strong at intent and synonymy but weaker at exact identifiers, proper nouns, rare terms, and typo-tolerant partial matches. Lexical retrieval covers exactly those cases, and it does so at low latency with a scoring model that is fully explainable. The two paths are complementary, not substitutes, which is why most production RAG systems run both. The two axes from Part 1 still apply. Recall is whether the relevant documents were retrieved. Latency is how quickly. Every design choice in this article moves along one or both of these axes. This article assumes familiarity with OpenSearch basics such as indices, shards, and mappings; for a refresher, see the References section . Two shapes of RAG corpus Before discussing techniques, it helps to name the two shapes of corpus that lexical retrieval typically serves. The right technique depends on which shape the corpus has. The first shape is a document corpus . Each record is essentially a blob of text: a web page, a knowledge-base article, a support ticket, a research paper. There may be a title and a URL. The rest is a body of prose. Queries against a document corpus ask which documents mention the query terms, and the interesting question is how to score the match well. Field choice is not the hard part, since there are only a few fields and one of them (the body) dominates. The second shape is a structured corpus . Each record has many typed fields: a job-postings corpus with title, company, location, seniority, required skills, and description text; an academic-papers corpus with title, authors, venue, year, keywords, abstract, and body; a customer support corpus with title, product area, resolution status, and body text. The same query can hit different fields for different intents, and metadata drives filtering as much as text matching drives scoring. Field choice, weighting, and filter shape all matter here. Both shapes use BM25 under the hood. Both benefit from the same analyzers. But the query shape differs, the mapping choices differ, and some techniques (like combined fields) apply mostly to one shape and not the other. Each section below calls out which shape it applies to. Term matching for controlled values Not every field should be scored. Some fields hold a small, closed set of controlled values: a country code, a status flag, a document type from a fixed taxonomy, a tenant identifier. For those fields, term matching is the right tool, not scored text matching. The right shape is a keyword field and a term or terms query: "country": { "type": "keyword" }, "status": { "type": "keyword" } GET /records/_search { "query": { "bool": { "filter": [ { "term": { "country": "US" } }, { "terms": { "status": ["published", "reviewed"] } } ] } } } This is exact-match, non-scored, and cache-friendly. There is no analyzer chain to argue with, no tokenization to worry about, no scoring math to interpret. The pitfall is treating an open-ended text field like it were controlled. A field called department that holds free-form user-entered strings like "Software Eng", "software engineering", "eng team", and "engineering group" is not controlled, and term matching on it will miss most of what a reasonable query should retrieve. If the data is not truly controlled at ingest, use text matching, not term matching. Building a parser to force free-form text into enums is a warning sign that the wrong tool has been chosen. The short rule is: term/terms for controlled enums, match/multi_match for free text. If you find yourself writing a parser to bridge the two, the field is not really controlled, and you should treat it as free text. BM25 as the default for free text For everything that is not a controlled enum, BM25 is the default scoring model in OpenSearch and it is a strong baseline. BM25 answers the question “how relevant is this document to this query?” by producing a numeric score per matching document. Higher is better. Documents come back sorted by that score. The score is built from three ideas, each of which captures something intuitive about what makes a document relevant. Term frequency. A document that mentions the query term ten times is more about that term than one that mentions it once. But not linearly: mention it a hundred times and the document is not a hundred times more relevant. BM25 uses a saturating curve, so extra occurrences of the same term give diminishing returns. This is why keyword stuffing does not indefinitely raise BM25 scores. Inverse document frequency. A term that appears in almost every document is a weak relevance signal. The word the occurring in a document says nothing about relevance. A rare term like opentelemetry occurring in a document says a lot. BM25 weights each query term by how rare it is across the whole corpus, so rare terms carry more scoring weight per occurrence than common terms. Length normalization. A long document has more chances to contain any given term by accident than a short one does. Without correction, long documents would always beat short ones on scoring, even when they only mention the query term in passing. BM25 normalizes by document length so a short focused document is not unfairly penalized against a long one. The final score for a document is the sum of these three contributions across all query terms. Two tuning knobs (traditionally called k1 and b) control how aggressively term frequency saturates and how strongly length is normalized. OpenSearch ships with BM25 as the default similarity for text fields with sensible values for both, so no configuration is needed to start. A minimal BM25 query against a single text field looks like this: GET /records/_search { "query": { "match": { "body": "how does bm25 scoring work" } } } Behind the scenes, OpenSearch runs the query text through the same analyzer as the indexed field (the next section explains what an analyzer is), computes BM25 scores for the matching documents, and returns the top results. When a document ranks somewhere surprising, BM25’s score is fully inspectable. Adding "explain": true to the search request returns the per-term score breakdown for each returned document, so you can see which terms contributed and how much. Document corpora: keep it simple For document corpora, resist the temptation to build a multi-field query. Documents in this shape are mostly body text, and complexity in the query rarely earns its cost. A reasonable mapping for a document corpus is one body field with a well-tuned analyzer, plus a lightly boosted title field: PUT /documents { "mappings": { "properties": { "title": { "type": "text", "analyzer": "english" }, "body": { "type": "text", "analyzer": "english" }, "url": { "type": "keyword" } } } } The query is small: GET /documents/_search { "size": 10, "query": { "multi_match": { "query": "gradient checkpointing memory savings", "fields": ["title^2", "body"], "type": "best_fields" } } } A modest boost on title reflects the reality that a term appearing in the title is a stronger relevance signal than the same term appearing once inside a long body. Do not push the boost too high: at 5x or 10x, the title match starts to dominate scoring even when the body match is more substantive, and results get worse. Beyond the title-vs-body split, most document corpora do not have interesting field structure. If a corpus has additional metadata (author, publication date, tags), prefer to use those as filters rather than as scored fields. A scored field must earn its cost: if a match on author is not really a meaningful relevance signal, matching against it just adds noise and slows the query. The shorter the field list, the easier BM25's IDF math is to reason about, the faster the query runs, and the smaller the surface area for scoring bugs. Simple is the right default for documents. Structured corpora: combined fields versus multi-field fanout Structured corpora invite a different mistake. When each record has ten or more text fields, it is natural to write a multi_match across many of them with per-field boosts. For a job-postings corpus: GET /jobs/_search { "query": { "multi_match": { "query": "python backend engineer", "fields": [ "title^3", "company^3", "skills^8", "seniority^6", "team^6", "description^0.5", "location" ], "type": "best_fields" } } } This feels right. It says “match on title, skills, seniority, and so on, with different weights.” In practice, it has two problems. The first is that BM25’s IDF is computed per field. The term python has one document frequency in skills, a different one in title, a different one in description. So the score for the same term differs across fields not just by the boost but by the corpus statistics of each field, and combining them with fixed boost multipliers rarely produces a coherent ranking. The best_fields mode picks the max score across fields, which mutes this problem for the winner but discards signal from the other fields. The second problem is cost. Each field in the fanout adds its own postings-list lookup per query term. Across many fields and many query terms, the query does more work than it needs to. On a large index with high query rates, this shows up as latency. The alternative is to consolidate related fields into one combined field at index time and query that one field: PUT /jobs { "mappings": { "properties": { "title": { "type": "text", "copy_to": "search_text" }, "company": { "type": "text", "copy_to": "search_text" }, "team": { "type": "text", "copy_to": "search_text" }, "seniority": { "type": "text", "copy_to": "search_text" }, "skills": { "type": "text", "copy_to": "search_text" }, "location": { "type": "text", "copy_to": "search_text" }, "search_text": { "type": "text", "analyzer": "english" } } } } copy_to tells OpenSearch, at index time, to concatenate the values of the source fields into the target search_text field. The source fields are still stored and searchable individually if needed; the combined field is an additional inverted index built from the same content. The query then targets one field: GET /jobs/_search { "query": { "match": { "search_text": "python backend engineer" } } } The postings-list work collapses from N lookups per term to one. IDF is computed once, across the whole combined field, so the term statistics are consistent. Scoring becomes easier to reason about because there is one text field doing the work. The cost is that field-specific boosting is no longer possible against the combined field, since the field is a mash of everything. If some sources are genuinely more important than others, you can either keep one high-signal field separate (like skills if skill matches are notably more predictive of relevance) and query it alongside the combined field, or you can weight the sources at concatenation time by repeating their content in the combined field. In practice, a small number of fields outside the combined field is often the cleanest shape: GET /jobs/_search { "query": { "multi_match": { "query": "python backend engineer", "fields": ["skills^8", "search_text"], "type": "best_fields" } } } Two fields instead of seven, and the seven-way fanout is replaced by one composite text field that BM25 can reason about cleanly. The typical outcome, when measured against a representative query set, is that recall stays the same and latency drops substantially. Different production systems have reported multi-x latency reductions on similar consolidations. Combined fields are the single biggest lever for structured lexical retrieval, and the one most search articles skip. How OpenSearch turns text into tokens: the analyzer Almost every design decision in the rest of this article is about a component called the analyzer . This section explains what an analyzer is, because the sections that follow (partial matching, filters, stop-words, multilingual handling, symmetry) all depend on understanding it. When you index a document with a text field, OpenSearch does not store the raw string as the searchable unit. It runs the text through an analyzer, which produces a stream of tokens , and those tokens are what get written to the inverted index. Search works by matching query-side tokens against the tokens that are already in the index. So the analyzer decides what “the same” means for search purposes. The OpenSearch analyzer pipeline in action. Image generated using Gemini AI. An analyzer is a pipeline with three stages: Character filters transform the raw text before tokenization. Common examples are stripping HTML, replacing patterns, and normalizing punctuation. Most analyzers do not use character filters, so this stage is often empty. A tokenizer splits the transformed text into tokens. The standard tokenizer splits on unicode word boundaries. The whitespace tokenizer only splits on whitespace. Language-specific tokenizers (icu_tokenizer, nori, kuromoji) split according to the rules of the language. Token filters transform the token stream after tokenization. This is where most of the interesting work happens: lowercasing, removing stopwords, folding diacritics, applying stemming, generating ngrams, expanding synonyms. Multiple token filters chain together in order. A worked example. Consider the input "Running the Kubernetes cluster". After the standard tokenizer, the tokens are ["Running", "the", "Kubernetes", "cluster"]. After a lowercase token filter, the tokens are ["running", "the", "kubernetes", "cluster"]. After a stop filter (with English stopwords), the tokens are ["running", "kubernetes", "cluster"]. After a snowball (English stemming) filter, the tokens are ["run", "kubernet", "cluster"]. That final list is what OpenSearch actually writes to the inverted index for this document. A future query for "kubernetes" will pass through the same analyzer, producing the token kubernet, which matches the indexed token. A future query for "the running" will produce ["run"] (the stopword is dropped, the verb is stemmed), which matches. Analyzers are configured in the index settings and then referenced from the mapping: PUT /records { "settings": { "analysis": { "analyzer": { "my_analyzer": { "type": "custom", "tokenizer": "standard", "filter": ["lowercase", "stop", "snowball"] } } } }, "mappings": { "properties": { "body": { "type": "text", "analyzer": "my_analyzer" } } } } By default, the analyzer configured on a field runs at both index time (when documents are written) and query time (when the query text is processed). Both sides go through the same pipeline, so the tokens line up and matching works. The exception, covered later under analyzer symmetry, is when index-time and query-time need to differ. With this in mind, most of the rest of the article is easier to state plainly. Stop-word removal is a token filter. Edge-ngram generation is a token filter. Synonym expansion is a token filter. Diacritic folding is a token filter. Multilingual support is a choice of tokenizer plus language-specific token filters. Everything is a chain of stages in the analyzer pipeline. Understanding the pipeline is understanding lexical search. Partial matching: fuzzy versus edge-ngram Users make typos. Users type prefixes. Users hit enter mid-word. The two common ways to handle partial matching are runtime fuzzy matching and index-time edge-ngram tokenization. They have different cost profiles and cover different typo patterns. Fuzzy matching computes an edit-distance expansion at query time. For each query term, OpenSearch generates a set of variants within a small Levenshtein distance (typically one or two edits) and searches for matches on any of them. It is turned on per query: GET /records/_search { "query": { "match": { "body": { "query": "kuberntes", "fuzziness": "AUTO" } } } } Fuzzy expansion catches typos anywhere in a word, including transpositions (kuberntes matches kubernetes) and single-character substitutions. The index does not need any special preparation. But every fuzzy query pays a per-term expansion cost at query time. On a busy cluster with long queries, this cost adds up. Edge-ngram tokenization takes a very different approach. Instead of expanding the query at query time, it expands each document's tokens at index time, so that partial matching becomes plain exact-token matching at query time. Edge-ngram is a token filter (the previous section covered what those are) that emits all prefix substrings of each token it receives, up to a configured length. Configured on a field: PUT /records { "settings": { "analysis": { "filter": { "edge_ngram_2_20": { "type": "edge_ngram", "min_gram": 2, "max_gram": 20 } }, "analyzer": { "edge_ngram_analyzer": { "type": "custom", "tokenizer": "standard", "filter": ["lowercase", "asciifolding", "edge_ngram_2_20"] } } } }, "mappings": { "properties": { "body": { "type": "text", "analyzer": "edge_ngram_analyzer" } } } } To see what this does, walk through what happens at ingest for a document whose body is "kubernetes cluster": The standard tokenizer produces ["kubernetes", "cluster"]. lowercase produces ["kubernetes", "cluster"] (already lowercase in this case). asciifolding produces ["kubernetes", "cluster"] (no diacritics). edge_ngram_2_20 emits every prefix of each token from 2 to 20 characters long, producing ["ku", "kub", "kube", "kuber", "kubern", "kuberne", "kubernet", "kubernete", "kubernetes", "cl", "clu", "clus", "clust", "cluste", "cluster"]. All 15 tokens are then written to the inverted index for this document. The document now appears in the postings list for ku, for kub, for kube, and so on. This is where edge-ngram's cost lives: the inverted index carries many more entries than a plain tokenizer would produce , and every document goes through this expansion at ingest. At query time, the payoff shows up. When a user types kube, the query is processed by the query analyzer (which, as we'll see in the analyzer-symmetry section, should not also do edge-ngram) and produces the single token kube. OpenSearch does one lookup for kube in the inverted index and finds every document whose analyzer previously emitted kube as one of its prefix tokens. The lookup itself is a single index probe. No runtime expansion, no per-term computation, no scan over variants. The prefix work was already done at ingest. This is the key mental shift. Partial matching with edge-ngram is not a query-time algorithm change; it is an index-time storage change. Every document was pre-expanded to hold all its prefixes, and the query is a plain term match against those prefixes. Fuzzy versus edge-ngram: cost trade-offs. The contrast with fuzzy is now clean: Fuzzy leaves the index unchanged and expands the query at query time. Every query pays the expansion cost. Edge-ngram leaves the query unchanged and expands the documents at index time. Every ingested document pays the expansion cost, but every query is cheap. The costs live on opposite sides of the pipeline. Edge-ngram’s tradeoffs follow directly: the inverted index is larger (each word contributes many prefix tokens instead of one), ingest CPU is higher (the analyzer does more work per document), and the technique only catches prefix matches (kube matches kubernetes, but kubrentes does not, because no document was indexed with kubrentes as one of its prefixes). The right choice depends on the typo pattern in the query stream. For type-ahead or autocomplete flows, users are entering prefixes, and edge-ngram is the right tool: the query is cheap and the pattern is exactly what edge-ngram covers. For arbitrary typo tolerance on established typing, fuzzy at query time is more general, at higher query cost. The pattern that most mature production systems settle on is edge-ngram for the fields where partial matching matters, without runtime fuzzy expansion. The reasoning is that prefix matches cover the common case (users typing, or truncated queries), the query cost is low, and the index growth is bounded because ngram range is capped (typically 2 to 20 characters). Runtime fuzzy is kept in reserve for specific use cases where edge-ngram is not enough. A gotcha worth naming here: if a field is indexed with edge-ngram, the same analyzer runs on the query by default. That means the query text also gets edge-ngrammed, so kubernetes becomes eight query tokens (ku, kub, ..., kubernetes) instead of one. This inflates the postings intersection work at query time and can hurt scoring. The fix is to set a different search_analyzer, which is covered in the analyzer symmetry section below. Filters, boosts, and scoring functions Three tools shape which documents come back and how they rank. They look similar in query shape but have different jobs, and mixing them up is a common source of muddy results. Hard filters require a match. A document without the required attribute is not returned. They are unscored and cache-friendly. Use them for attributes that are non-negotiable for correctness: an access control list, a tenant identifier, a status flag, a recency window, a “must be published” requirement. GET /jobs/_search { "query": { "bool": { "must": [ { "match": { "search_text": "python backend engineer" } } ], "filter": [ { "term": { "tenant_id": "acme" } }, { "term": { "status": "open" } }, { "range": { "posted_at": { "gte": "now-30d" } } } ] } } } Soft boosts raise the score for documents with a desired attribute but do not require it. Use them for signals that improve results when present but should not exclude documents when absent: “remote”, “senior level”, “recently posted”: GET /jobs/_search { "query": { "bool": { "must": [ { "match": { "search_text": "python backend engineer" } } ], "should": [ { "term": { "tags": "remote" } }, { "term": { "tags": "senior" } } ], "minimum_should_match": 0 } } } minimum_should_match: 0 is the key: it says "match none of these if you must, but prefer documents that match at least one." Without that, the boost effectively becomes a filter. Scoring functions fold a continuous value directly into the score. Recency is a good example: a document is not "recent yes or no," its relevance degrades smoothly with age. function_score with a decay function expresses this cleanly: GET /jobs/_search { "query": { "function_score": { "query": { "match": { "search_text": "python backend engineer" } }, "functions": [ { "gauss": { "posted_at": { "origin": "now", "scale": "14d", "decay": 0.5 } } } ], "score_mode": "multiply", "boost_mode": "multiply" } } } The right question when adding any signal is: is this a requirement, a preference, or a continuous factor? Requirement is a filter. Preference is a soft boost. Continuous factor is a scoring function. Reaching for the wrong one is a common cause of results that “look reasonable but rank strangely.” A signal that should degrade smoothly (like recency, or a numeric proximity to a target value) degrades badly if expressed as a hard threshold in a filter. A requirement that is expressed as a soft boost lets non-matching documents slip through. Where logic lives: application code versus the analyzer chain Every non-trivial lexical pipeline ends up doing preprocessing on the query text before BM25 sees it. The interesting question is not what preprocessing, but where it runs. There are two homes: application code, and the analyzer chain. Application-code preprocessing is what most teams start with. It is where prototyping happens: pull the query text out, strip some stop words, lowercase, remove some context-specific noise, hand the cleaned string to OpenSearch. This works. It is fast to iterate on. It runs at query time on every request. And, over time, most of the steps that started in application code turn out to belong somewhere else. The alternative is to move the preprocessing into the analyzer chain, which runs at both index time and query time inside OpenSearch. The classic example is stop word removal. In application code, it looks like this: STOP_WORDS = {"a", "an", "the", "of", "for", "and", "or", "what", "how", "does", "when", "where", "near", "me", "is", "are"} def strip_stopwords(query): tokens = query.lower().split() return " ".join(t for t in tokens if t not in STOP_WORDS) The equivalent in the analyzer chain is a filter chained into a custom analyzer: PUT /records { "settings": { "analysis": { "filter": { "english_stops": { "type": "stop", "stopwords": "_english_" } }, "analyzer": { "clean_analyzer": { "type": "custom", "tokenizer": "standard", "filter": ["lowercase", "asciifolding", "english_stops"] } } } }, "mappings": { "properties": { "body": { "type": "text", "analyzer": "clean_analyzer" } } } } Three reasons the analyzer chain is the better home for anything context-free: Multilingual coverage is a config change, not code. OpenSearch’s stop token filter ships language stopword lists for many languages, addressable as _english_, _spanish_, _french_, _german_, and so on. Adding a locale is one line, not a code change. Applied uniformly. The analyzer runs on every document at index time and every query at query time. There is no code path that can accidentally skip it. In application code, someone has to remember to call the filter, and eventually someone somewhere forgets. Free at query time. Index-time work is paid once per document during ingest. Application-code query-time filtering is paid on every request. On a busy cluster at high queries per second, that difference is meaningful. The generalization: any preprocessing that is context-free belongs in the analyzer chain . Stopword removal, lowercasing, diacritic folding, tokenization, ngramming, synonym expansion. All of these are decisions that depend only on the token stream, not on anything specific to the caller or the query slot. What stays in application code : anything context-dependent. If preprocessing needs to know something the analyzer cannot see, it stays in the application. Two examples: A query arrives with a slot like "filter_by_company": "acme", and the application knows to remove the word "acme" from the query text because the slot already restricts results to that company. The analyzer cannot know this, because "acme" is a meaningful search term in a different query where no company filter was provided. This is per-query context. A multi-tenant system knows the tenant name should be stripped from the query text for one tenant but not another, based on tenant configuration. The analyzer cannot reach into tenant config. This is per-tenant context. Most mature lexical pipelines end up with a small amount of context-dependent application-code preprocessing, a rich analyzer chain doing everything else, and a very simple query builder that just wires the two together. The maturity curve is that application code shrinks over time and the analyzer chain grows. Analyzer symmetry and the search_analyzer override OpenSearch runs the same analyzer at index time and at query time by default. This is usually right: the tokens on both sides come out the same, and BM25 matches them directly. Search should match indexing. The exception is edge-ngram. When a field is indexed with edge-ngram, the documents should be edge-ngrammed (so a query for kube can find kubernetes). But the query should not be re-edge-ngrammed, because that turns one query term into many, inflates the postings intersection, and can hurt scoring on short queries. The fix is search_analyzer, which lets index-time and query-time analyzers differ: "body": { "type": "text", "analyzer": "edge_ngram_analyzer", "search_analyzer": "standard_analyzer" } Here edge_ngram_analyzer runs at index time and produces all prefix tokens. standard_analyzer (or whatever your non-ngram analyzer is) runs at query time and produces one token per word. A query for kubernetes becomes one token, which hits the indexed kubernetes token. A query for kube becomes one token, which hits the indexed kube token. Both work, but the query does one postings lookup per word instead of several. The rule to internalize is: index-time and query-time analyzers should be the same unless the field's indexing does something that the query should not repeat . Edge-ngram is the common case. Synonym expansion is another: some teams synonym-expand only at index time (so queries stay short), some only at query time (so the index does not bloat), and getting this wrong in either direction is a real footgun. When search_analyzer is missing on a field where it should exist, queries often work but do more work than they need to. The symptom is high latency with no obvious cause. Multilingual handling Everything above assumes English. Real corpora are usually multilingual. Four topics matter here, each briefly. Tokenization. The standard tokenizer splits on unicode word boundaries and works reasonably for European languages. It handles CJK languages badly. For a corpus that includes Chinese, Japanese, or Korean text, use language-specific tokenizers: icu_tokenizer for general unicode handling, nori for Korean, kuromoji for Japanese, and analysis plugins for Chinese. Getting this wrong produces token streams that BM25 cannot score meaningfully. Stemming. english uses the snowball stemmer to conflate running, runs, ran into run. Snowball has variants for many European languages: spanish, french, german, italian, portuguese, and others. Light stemmers exist for cases where snowball is too aggressive. For languages without well-established stemmers, no stemming is better than bad stemming; over-stemming muddies scoring. Diacritic folding. icu_folding and asciifolding normalize accented characters so that café and cafe produce the same token. This is almost always desirable for user-facing search, since users type without diacritics as often as they type with them. Fold at both index time and query time so tokens align. One index per locale, or one index with a locale field. For a small number of locales with distinct analysis pipelines (English, Japanese, and Arabic), separate indices per locale is often the cleanest shape: each index has its right tokenizer, stemmer, and analyzer chain, and queries route by locale. For many locales, or when locales share analysis (English and Spanish can use similar pipelines with different stopword lists), one index with a locale field and a per-locale analyzer selection is usually simpler to operate. The tradeoff is analysis fidelity against operational overhead: more indices give better analysis, one index gives simpler operations. Where BM25 falls down BM25 is a strong baseline, but it is exact-token scoring with corpus statistics. It does not know synonyms. It does not know that k8s means kubernetes. It does not know that a query for ML and a document about machine learning describe the same concept. Three failure patterns show up in every production lexical pipeline: Abbreviations and acronyms. A query for ML will match documents that literally contain the string ml, and miss documents that describe the same concept as machine learning. Both are correct answers; BM25 sees them as unrelated. Concept versus surface form. A query for distributed tracing retrieves documents that literally contain those words. Documents about opentelemetry, spans, or observability may be just as relevant but score lower or not at all, because the query terms are not in the text. Broad-term versus specific-term queries. A query for database should probably match documents about postgres, mysql, dynamodb, and general database material. BM25 only matches documents whose text contains the token database. It does not know that a document about postgres tuning is a database document. Other techniques exist to bridge these gaps (synonym token filters at index time, upstream query rewriting via rule engines or LLMs), but each comes with real costs: dictionaries to maintain, latency added to the query path, and coverage that is only as good as the rules you wrote. At best, these are stop-gaps until actual semantic retrieval is implemented. The right way to complement lexical retrieval is semantic retrieval. Retrieve semantically alongside lexically , which is the natural bridge to Part 1. The semantic path does not care that distributed tracing and opentelemetry are different tokens; their embeddings are close. Running lexical and semantic retrievers in parallel and merging their results (a hybrid retrieval strategy) is the most common production answer to BM25's failure modes. In a mature production RAG pipeline, lexical retrieval does what it does well (fast, explainable, precise on exact terms) and semantic retrieval covers the rest. Trimming the response payload Part 1 covered response payload trimming for the semantic path. The same lever applies to the lexical path, for the same reason. Conclusion The through-line of Part 2 matches Part 1. Push complexity to the boundaries. Keep the hot path simple. The specific translations for lexical retrieval: Field structure lives in the mapping and the analyzer chain. Move context-free preprocessing (stopwords, folding, ngrams, synonyms) out of application code and into analyzers. Consolidate related fields into combined fields at index time when the query would otherwise fan out across many. One well-tuned field beats N boosted fields for both latency and BM25’s scoring coherence. Match the right tool to the right job. Term queries for controlled enums, match queries for free text, hard filters for requirements, soft boosts for preferences, scoring functions for continuous factors. Do not reach for edge-ngram or fuzzy universally. Pick based on the typo pattern in the query stream. Edge-ngram for prefix and type-ahead, fuzzy for mid-word typos. None of this should be adopted on faith. Every corpus is different. The right approach is to establish a baseline, make one change at a time, and measure recall and latency against that baseline on a representative query set. The right configuration is the one the measurements support, not the one that is expected to work. References A Comprehensive Guide to ElasticSearch and OpenSearch Architecture OpenSearch BM25 OpenSearch Fuzzy Match OpenSearch edge-ngram Other Articles Agentic Inference Deployment Operational Readiness for LLM Services Benchmarking AI Agents OpenSearch Optimizations for Production RAG, Part 2: Lexical Retrieval was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.