AI News Archive: July 19, 2026 — Part 3
Sourced from 500+ daily AI sources, scored by relevance.
- IT2030: Expose the C-Suite Tensions That Could Derail Your AI Ambitions
IT2030: Expose the C-Suite Tensions That Could Derail Your AI Ambitions Gartner
- Moonshot's Kimi K3 outperforms Fable 5 in frontend code but lags far behind in complex math
Moonshot's Kimi K3 is the first Chinese model to top the Code Arena: Frontend rankings, beating Claude Fable 5 and GPT-5.6 Sol by a wide margin. But on advanced math, the gap is stark: Kimi K3 scores only about 39 percent on FrontierMath Tier 4, while models from OpenAI and Anthropic hit close to 90. The article Moonshot's Kimi K3 outperforms Fable 5 in frontend code but lags far behind in complex math appeared first on The Decoder .
- Enjoy Lifetime Access to Claude, Gemini, GPT, and More for Only $60
Enjoy Lifetime Access to Claude, Gemini, GPT, and More for Only $60 PCMag
Score: 39🌐 MovesJul 19, 2026https://www.pcmag.com/deals/enjoy-lifetime-access-to-claude-gemini-gpt-and-more-for-only-60 - Tripo3D Highlights AI-Powered 3D Creation Tools
Tripo3D Highlights AI-Powered 3D Creation Tools USA Today
Score: 39🌐 MovesJul 19, 2026https://www.usatoday.com/press-release/story/35790/tripo3d-highlights-ai-powered-3d-creation-tools/ - AI may make you 'boring' at work, Columbia Business School professor says: How it can hurt your career
Using AI at work in the wrong ways could lead to "gradually falling behind and becoming complacent," said Sandra Matz, a professor at Columbia Business School.
Score: 39🌐 MovesJul 19, 2026https://www.cnbc.com/2026/07/19/how-using-ai-at-work-can-hurt-your-career.html - Your AI Strategy May Be Missing Much Bigger Opportunities. Here’s How to Know
AI didn’t just remove tasks. It removed limits.
Score: 38🌐 MovesJul 19, 2026https://www.inc.com/tony-manganiello/your-ai-strategy-may-be-too-small/91374370 - Your AI Agent Passed Every Eval. Finance Still Killed It.
An AI agent passed every metric in the eval harness I published, then the CFO killed it — its successful resolutions cost more than the humans it replaced. The one metric that predicts whether an agent survives production, and how to measure it without a rebuild. The post Your AI Agent Passed Every Eval. Finance Still Killed It. appeared first on Towards Data Science .
Score: 38🌐 MovesJul 19, 2026https://towardsdatascience.com/your-ai-agent-passed-every-eval-finance-still-killed-it/ - AI text detectors struggle when language models mimic an author's style
Epoch AI tested three leading AI text detectors (Pangram, GPTZero, and Originality.ai) using style-imitated texts. Up to 18 percent of AI-generated passages went undetected. For scientific writing, the miss rate climbed as high as 48 percent, the very genre where these detectors likely see the most real-world use. The article AI text detectors struggle when language models mimic an author's style appeared first on The Decoder .
Score: 38🌐 MovesJul 19, 2026https://the-decoder.com/ai-text-detectors-struggle-when-language-models-mimic-an-authors-style/ - Can an Apple lawsuit derail OpenAI’s hardware plans?
On the latest episode of Equity, we debate whether Apple's lawsuit will cast over OpenAi's much-discussed plans to get into hardware and go public.
Score: 38🌐 MovesJul 19, 2026https://techcrunch.com/2026/07/19/can-an-apple-lawsuit-derail-openais-hardware-plans/ - Why AI-Built Apps Pass Every Test and Still Break in Front of Users
Your coding agent builds the whole app and gets the suite passing in an afternoon. But those tests ran against a handful of placeholder rows, or nothing at all, so the bugs that only show up against real data walked straight through. Photo by Sunder Muthukumaran on Unsplash Last month I handed a coding agent a one-line brief and let it run. It wrote the migrations and wired the routes on top of them, and by the time it stopped, pnpm dev came up clean and the test suite was green on the first try, then I clicked around a build that rendered every page perfectly and had nothing on any of them. The dashboard reported zeros across the board, the orders table held no orders, and the "recent activity" feed was a blank rectangle where the demo was supposed to live. What got me is that the schema underneath was genuinely good — real foreign keys, a couple of unique constraints, a CHECK or two holding the shape togetherf. The agent had done the hard structural work and stopped one step short of what makes an app testable at all, which is getting real, connected data into those tables in the first place. For a long time that gap was a corner case, the kind of thing you’d hit once and shrug off. like it has quietly turned into the default. In June 2026 Supabase reported that more than 60% of new databases on its platform are now launched by some kind of AI tool, with agents deploying the majority of them and Claude Code the largest single contributor since the start of the year — these agents stand up the database and apply the migrations, and more and more they write the whole application on top of it. What they are worst at is the unglamorous step underneath all of it, the one where the tables fill with data that behaves the way real data does. “Just tell the agent to seed it” So you ask for it. Insert some sample rows, t he agent obliges, and that’s exactly where the trouble starts. It fills those tables one at a time, writing INSERT statements in whatever order the schema file happens to declare them. If orders sits above users in that file, the orders go in first, and Postgres throws out the very first row because its user_id points at a user that doesn't exist yet. At that point the agent improvises a fix — it might drop the offending constraint, or stuff user_id with a plausible-looking integer, or shuffle the INSERT order by trial and error until the errors stop scrolling up the terminal. Eventually the errors stop and every table has rows in it. None of those rows add up to a graph, though, and a single query is enough to prove it: SELECT count(*) FROM orders o LEFT JOIN users u ON u.id = o.user_id WHERE u.id IS NULL; -- orders pointing at users that were never generated Anything above zero means the agent handed you type-correct noise dressed up as data. Every user_id validates on its own, a real integer sitting in a believable range, while pointing at a user that was never generated. Empty tables make your tests lie There’s a quieter version of this failure, and it’s worse precisely because nothing turns red. Point a test suite at three hand-seeded rows, or at a table with none, and it goes green, though the green certifies only that the data was too thin to contradict the code. Pagination has no second page to break on, because actually there is no second page, and the N+1 query that would crawl in production returns in a millisecond here, walking its whole five rows. To be honest, I’ve shipped a bug exactly like that quite many times. The test suite was green and coverage looked healthy right up until the feature fell over at the first afternoon it met a table with a few hundred thousand rows after weeks of tests quietly agreeing with an empty database. Not all of what slips through this way is a performance problem. Authorization is the sharpest example — a query meant to scope results to the current user only gets tested when a second user’s rows are sitting in the same table, so seeding a single user leaves the endpoint that would hand back someone else’s record by ID looking completely correct, with no other record around for it to leak. The same emptiness hides tenant-isolation bugs and permission checks that never fire, because the data that would trip them was never there. What “fill it correctly” actually takes When the table-by-table INSERT breaks, it isn't breaking because someone was careless. The break is structural, and the structure is where the real work hides. Ordering the inserts is the thing that looks like it should be the first. Rows have to land parents-before-children, each one arriving only after the rows it references already exist, which is almost never the order the schema file happens to list its tables in. When two tables reference each other in a cycle, one side goes in first with its back-reference left blank, and a second pass fills that reference once both rows exist. Constraints are where it gets fiddly. Each one has to hold while the rows are being generated, not patched in afterward once the errors have scrolled past: for example, a UNIQUE(email) column needs a generator that remembers every address it has already produced, so the next one won't collide with an earlier one. Hand that same generator a CHECK (ship_date >= order_date) and the two dates suddenly have to be produced together, by something that understands they're related, because drawn as independent random values they'll violate the rule about half the time. Even with order and constraints handled, the rows still have to make sense together, and that coherence is the thing most tools skip and the thing that gets you caught: user’s signup timestamp ought to fall before their first order, row that reads status = 'shipped' while carrying no shipped_at is nonsense on its face, and a city that disagrees with its postal_code is exactly what a reviewer clocks two seconds into a live demo, long after a column-at-a-time generator has pronounced every field valid on its own. This is where Faker and everything built like it come apart, because each value gets drawn from its own distribution with no glance at the value sitting next to it, so every field can look impeccable while the row as a whole quietly makes no sense. None of this is hard in the sense of being clever. It’s fiddly and repetitive and easy to get subtly wrong, which makes it exactly the kind of work you don’t want a probabilistic agent improvising at 2 in the morning against a schema it only half-remembers. The landscape, once you stop hand-holding the agent Once you decide the data deserves a real step in your pipeline, the options fall into three rough tiers. When you need a column of believable names, or a throwaway script against a couple of tables, the free and DIY route is enough. That means Faker and its ports across languages, plus whatever SQL you hand-write around them; In short, it stays blind by design to the adjacent column and to the table across the foreign key, which leaves every relationship you care about for you to wire up by hand and keep working. Neosync (closed since August 2025) and Seedfast sit one tier up, the schema-aware generators. Both read the live schema and produce data that keeps every foreign key intact, so the insertion order is never something you spell out by hand; where they mostly differ is in how much of your domain you describe up front and how much they read straight from the schema. This is the tier built for exactly the empty-tables problem the agent left behind. For regulated organizations, the ones that have to mask production data and keep an audit trail alongside whatever they generate, there are the enterprise platforms like GenRocket and Delphix . Which one to reach for When an empty database is fine Not every empty table is a problem worth solving. Three commits into a prototype, with the schema changing by the hour, seeding is wasted motion; the agent’s five throwaway rows are plenty to confirm a page renders and a route responds, and you’re better off skipping the ceremony and moving on. Stability is what changes the math. Once the schema settles and someone wants a realistic demo, or the first test starts to lean on more than one row existing, that empty database becomes a real problem — the thing a reviewer or a customer runs into first. Picking them up on purpose, before a test or a demo quietly starts depending on it, is the part worth owning yourself. Why AI-Built Apps Pass Every Test and Still Break in Front of Users was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- AI Made Everyone in Your Company a Storyteller. That’s Exactly Why You Need to Draw This Line
Decide who owns the story before the next launch, not during it.
- Local Rankings AI Launches Done-for-You Local SEO for Small Businesses
Local Rankings AI Launches Done-for-You Local SEO for Small Businesses azcentral.com and The Arizona Republic
- What I Learned Shipping AI Into a Real Financial Workflow
A behind-the-scenes look at taking generative AI from prototype to production in a regulated, high-stakes environment. Continue reading on Towards AI »
- Why Founders Need a New Operating System to Lead Through AI Disruption
Why Founders Need a New Operating System to Lead Through AI Disruption entrepreneur.com
Score: 36🌐 MovesJul 19, 2026https://www.entrepreneur.com/leadership/why-founders-need-a-new-operating-system-to-lead-through-ai/503716 - Caution warranted over tech threat, investors’ AI exuberance
Volatility comes with rallies. As stock markets in China and overseas advance, they are sending mixed signals. Investors’ love-hate relationship with artificial intelligence (AI) is greatly to blame as they rotate between pure AI plays and their underlying hardware and chip producers. Eddie Yue Wai-man, chief executive of the Hong Kong Monetary Authority, has therefore issued a warning about an AI bubble as well as financial systems being hacked or otherwise harmed by the new tech. Stock indexes...
- Flight Centre to embrace AI agents, ecommerce consolidation
Appoints new executives including chief AI officer.
- Pudu Robotics Showcases Full Product Portfolio at WAIC 2026, Winning the "Most Investor-Attractive Enterprise" Award
Pudu Robotics Showcases Full Product Portfolio at WAIC 2026, Winning the "Most Investor-Attractive Enterprise" Award The Straits Times
- ‘The world of work is changing fast': Barely any workers think their job is safe as worries about AI refuse to go away
Although employment remains strong, widespread worker anxiety around AI continues, damaging productivity.
- AI Is Changing How Businesses Are Found Online. That's Why You Need to Rebuild Your Search Strategy.
AI Is Changing How Businesses Are Found Online. That's Why You Need to Rebuild Your Search Strategy. entrepreneur.com
- 15 AI Tools That Are Actually Saving Businesses Time
15 AI Tools That Are Actually Saving Businesses Time entrepreneur.com
Score: 33🌐 MovesJul 19, 2026https://www.entrepreneur.com/building-a-business/15-ai-tools-that-are-actually-saving-businesses-time - GetHookd Expands Meta Ads Library API Scraper For AI-Driven Ad Research
GetHookd Expands Meta Ads Library API Scraper For AI-Driven Ad Research USA Today
- This $40 AI Book Generator Lets Busy Entrepreneurs Publish Without the Learning Curve
This $40 AI Book Generator Lets Busy Entrepreneurs Publish Without the Learning Curve entrepreneur.com
Score: 31🌐 MovesJul 19, 2026https://www.entrepreneur.com/science-technology/this-40-ai-book-generator-lets-busy-entrepreneurs-publish/504999 - Chinese software firm Deepexi opens DeepWorks public beta
DeepWorks platform includes over 2,000 industry skills and supports multi-agent collaboration.
Score: 30🌐 MovesJul 19, 2026https://www.techinasia.com/endowus-ceo-gregory-van-optimistic-profitability - No more beating around the bush — Arc Search puts AI at the center of your browser experience
Homescreen Heroes: Arc Search adds a layer of AI technology on top of a standard web browsing experience
- 10 Open-Source No-Code AI Platforms for Building LLM Apps, RAG Systems, and AI Agents
10 Open-Source No-Code AI Platforms for Building LLM Apps, RAG Systems, and AI Agents MarkTechPost
- Why CHASING Built CanFish: From Underwater Robotics to Fishing Technology
Why CHASING Built CanFish: From Underwater Robotics to Fishing Technology azcentral.com and The Arizona Republic
- What happens when AI starts gambling on soccer?
What happens when AI starts gambling on soccer? Business Insider
Score: 29🌐 MovesJul 19, 2026https://www.businessinsider.com/new-ai-benchmark-betting-on-world-cup-soccer-2026-7 - This $20 ChatGPT E-Degree Helps Entrepreneurs Use AI to Work Smarter
This $20 ChatGPT E-Degree Helps Entrepreneurs Use AI to Work Smarter entrepreneur.com
Score: 29🌐 MovesJul 19, 2026https://www.entrepreneur.com/science-technology/this-20-chatgpt-e-degree-helps-entrepreneurs-use-ai-to/505000 - Gaining ‘attention marketshare’ in the AI era
As consumers opt for AI-powered search and chatbots, the keyword economy is weakening
- Robotic petals couldn't open aboard Vikram-1 due to extreme space conditions: Cosmoserve
Based on detailed studies, necessary architectural changes will be adopted in the next version of the soft robotic petal software, said Cosmoverse, adding that the test flight provided significant data to validate the system, which otherwise wouldn’t have been obtained through ground tests.
- I Replaced My $400/Month Claude API Bill With GLM-5.2 + vLLM Here’s the Exact Playbook
Last month, three separate engineers on my team opened Slack at 6 AM asking the same question: why did our AI spend spike from $290 to… Continue reading on Towards AI »
- Google Just Published the Blueprint. And aiHelpDesk Is Already Shipping it.
Google’s AI in SRE framework independently validates the architecture we’ve been building at aiHelpDesk. Here’s where we align and where we go further. Google’s SRE team published an essay this week on integrating AI into production operations. It is worth reading in full. It is also, for anyone who has been following aiHelpDesk , striking for a different reason: Google independently arrived at the same core architecture we have been building. And published it as the standard the industry should aim for. This post walks through that alignment and then explains where aiHelpDesk goes beyond what Google describes. Google’s Safety Trifecta is aiHelpDesk’s Governance Google names three requirements for safe AI autonomy in production: Transparency: chain-of-thought logging of every agent decision Real-time risk evaluation: assessing blast radius before any production action executes Progressive authorization: requiring human approval at increasing levels of consequence. This is not a framework we need to implement. It is a description of what aiHelpDesk already ships: Google built “Actus”, which is their Mitigation Safety Verification Agent. Actus appears to be the control plane that enforces these three properties. Actus performs pre-flight safety validations, mandatory dry-runs and maintains an emergency circuit breaker (“Red Button”) for pausing all agentic actions. aiHelpDesk’s Governance layer does the same work: the policy engine enforces what can run, the blast radius check is the pre-flight and gate denial is the circuit breaker. All of that is recorded permanently in the audit log alongside the operator identity and stated reason. We also refer to the above as the WHAT vs. the WHY framework. We did not copy Google. They did not copy us. Two teams working on the same hard problem arrived at the same answer. That is the strongest possible signal that the answer is right. Their autonomy levels are useful vocabulary for aiHelpDesk’s positioning Google defines five levels of SRE autonomy (L0–L4): L0 — Manual: human executes everything L1 — Assisted: monitoring and investigation automated, human decides L2 — Partial autonomy: human approval required for actions L3 — High autonomy: autonomous in bounded, certified scenarios L4 — Full autonomy: multi-step resolution without human involvement aiHelpDesk’s default mode is L2. The step-approval gate is the L2 mechanism: the agent diagnoses and proposes, a human reviews the blast radius and approves each consequential step. Nothing destructive executes without that approval. The path to L3 in aiHelpDesk is the stability cert . When a fault class has accumulated sufficient certified runs, e.g. STABLE(7) attr=oom-kill (7/7) with the judge consistency 94%, the cert becomes an evidence that the agent’s reasoning on this specific fault class is reliable enough to reduce the approval burden. L3 is not a mode you flip on. It is a level of trust you earn, per fault class, per model, with data behind it. See here for details. L4 is explicitly not our target. Full autonomy on the mission critical production database and K8s systems without human involvement in the loop is not a goal aiHelpDesk will pursue. The Judgment Layer is expensive, but it exists precisely because there is a class of decisions that requires operational experience the agent cannot synthesize from traces alone. aiHelpDesk knows which recommendation to prohibit and why. Where aiHelpDesk goes further than Google blueprint describes Google’s evaluation framework is thorough: Bronze/Silver/Gold data quality tiers, LLM-as-judge, nightly runs against past incidents, IRM-Analyzer reconstructing human response trajectories. This is serious engineering and their MTTM numbers (44% reduction for supported incidents) reflect it. But Google’s evaluation is entirely backward-looking . Did the agent handle past incidents correctly? The dataset is real incidents. That effectively makes the measurement retrospective. aiHelpDesk’s approach is forward-looking . The fault catalog contains 32 structured fault scenarios [ code link ]. Each with an injection spec, a teardown and an expected attribution class . aiHelpDesk fault injection testing system injects the fault, the agent diagnoses it, the judge evaluates the diagnosis and the result is recorded in a stability cert that carries forward as a per-fault guarantee: fault: k8s-oomkilled playbook: pbs_k8s_pod_crash_triage taxonomy: 1.0 runs: 7 pass: 7/7 (100%) STABLE(7) attr=oom-kill (7/7) judge: 94%±3% This cert is not a quality process report. It is a proof. It says: on this specific fault class, under this playbook version, with this model, the agent reached the correct attribution conclusion seven times in a row with 94% judge consistency. That is a different kind of claim than “we run nightly evaluations.” Google describes three tiers of data quality: Bronze (heuristic), Silver (programmatic) and Gold (human-verified). aiHelpDesk’s vault calibration shows the same distinction. The output explicitly flags when the human feedback fraction is thin and the accuracy figure reflects LLM self-consistency rather than verified verdicts. We call it a Data Quality Banner . The principle is identical: you are entitled to know when the number you are looking at is based on a shallow signal. The dimension Google does not describe at all is attribution consistency . Not just “did the agent pass”, but “did the agent reach the same conclusion every time, and can we name that conclusion?” Two agents can both pass on a K8s Pod crash fault and reach opposite conclusions: one diagnoses oom-kill (exit code 137, memory limit exceeded), the other diagnoses process-error (exit code 1, configuration failure). Same playbook, same pass rate, different reasoning. See here for the full story. aiHelpDesk’s cert captures the attribution: fault: k8s-oomkilled STABLE(7) attr=oom-kill (7/7) fault: k8s-crashloop STABLE(5) attr=process-error (5/5) An agent that passes, but cannot consistently name what it found is not production-certified on that fault class. Attribution consistency is the difference between a system that gets the right answer and one that knows why. The open-source difference Google’s entire stack is internal. Detectr, AI Alert, Incident Hypothesis, Investigation Dashboards, AI Operator, Actus. All of it appears to be internal. It cannot be audited, forked or deployed. The essay is the blueprint. The implementation is not available. aiHelpDesk is open source. The governance layer, the audit trail, the blast radius enforcement, the step-approval gate, the fault catalog, the evaluation pipeline. All of it is in the public repository. The code, the commit history. Fully verifiable. Every claim in the Customer Bill of Rights has a mechanism you can read, test and confirm is real. On a live system. Your system. A company that reads Google’s essay and asks “how do we get this for our database and K8s operations?” has two options: build it internally over two to three years or use aiHelpDesk today. What the blueprint confirms The governance architecture for safe AI in production operations is not experimental. Google runs it at scale, across thousands of incidents and publishes the results. The Safety Trifecta is real. The L2/L3 autonomy boundary is real. The need for human-verified gold data and explainable agent reasoning is real. aiHelpDesk implements this architecture, open source, for production database and K8s operations, with the addition of forward-looking fault certification and attribution consistency that Google’s essay does not describe. The blueprint has been published. The implementation has been running. aiHelpDesk invented and pioneered the concepts of Operational SRE/DBA Flywheel ( blog post , doc ), Triage Consistency Certification , Informed Consent , Second Opinion and Bill of Rights . The vault commands , the Judge , the playbook diff , the full audit trail , the calibration data quality banner , the model-scoped stability certs and vault cert-compare … are not add-ons. They are the product. Trust is the product. The Judgment Layer covers what to do when the AI’s own improvement proposals fail. And why that case is where the most durable operational knowledge gets encoded. Why aiHelpDesk Playbooks are trustworthy? Because we give you, the customer, an ability to vet, verify and improve them through the methodology that we refer to as Operational SRE/DBA Flywheel , see here and here for details. And yes, as a customer, you can not only confirm that the playbooks that you get with aiHelpDesk out of the box work for your environment, but you can easily bring your own. Both: BYO playbooks (by either importing your existing runbooks or cloning and customizing one of our’s or by creating one from scratch) + BYO faults as well. Because nobody knows your specific databases, your environment and your workload with your upstream/downstream apps better than you do. You know how your database fails. Vendors don’t. At aiHelpDesk, we give you an option to create your own faults and add your own playbooks to triage and rectify them (in addition to the system playbooks we ship, of course). And yes, we don’t depend on a particular model or a model provider. aiHelpDesk is model-neutral . From our standpoint, the LLMs are a disposable commodity . Flip from Gemini to Anthropic and aiHelpDesk should continue to give you exactly the same diagnosis and remediation. Anything shorter than that is a P0 bug. Related Reading aiHelpDesk Flywheel official documentation Your SRE On-Call Runbook Is Already Obsolete. Here’s Why That’s Not Your Fault : Introducing aiHelpDesk Operational SRE/DBA Flywheel The Missing Test Suite for AI Database Operations : You’re about to bet your SRE/DBA on-call rotation on an AI agent. Want to know if it’s any good before the 2am page goes off? Runbooks Rot. Playbooks Learn : Operational SRE/DBA Flywheel: Ops Knowledge That Compounds. Automatically. Improving with every incident We Wanted a Dramatic AI Agent Failure. We Got Something Better : When the Flywheel works: The K8s WAL fault that made us rethink what playbooks are for AI Database Troubleshooting: the PostgreSQL Stat That Looks Like Good News (But Ain’t) : What a bgwriter incident taught us about the difference between reading data and understanding it AI troubleshooted DB pileup and reported success. The locks didn’t care : It’s the story that shows that the model wasn’t bad at reasoning. But it reasoned without the right knowledge. Your AI Just Diagnosed the Outage. Should It Fix It Too? How Decision Hub puts a human at every boundary between knowing and doing. And how we tried to override our own governance model and it said no. Twice. As of this writing, aiHelpDesk is available in Beta. If you run PostgreSQL in production and would like to get help in preparing for the avalanche , consider aiHelpDesk. Reach out to us at info@aiHelpDesk.biz and we’ll be happy to show you what it looks like in practice. Google Just Published the Blueprint. And aiHelpDesk Is Already Shipping it. was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- OpenSearch Optimizations for Production RAG, Part 2: Lexical Retrieval
This is Part 2 of a series on optimizing OpenSearch for production RAG. Part 1 covered semantic retrieval , meaning vector search with Approximate and exact Nearest Neighbor methods. This part covers lexical retrieval, meaning text-based search, and how it complements the semantic path. Introduction Retrieval Augmented Generation (RAG) systems retrieve a set of relevant documents and pass them as context to a Large Language Model (LLM). Part 1 discussed the semantic path, which retrieves documents whose embeddings are near the query embedding in a vector space. This part covers the lexical path, which retrieves documents that share tokens with the query. Lexical retrieval is often described as the older, weaker sibling of semantic retrieval. That framing understates what lexical retrieval contributes. Semantic retrieval is strong at intent and synonymy but weaker at exact identifiers, proper nouns, rare terms, and typo-tolerant partial matches. Lexical retrieval covers exactly those cases, and it does so at low latency with a scoring model that is fully explainable. The two paths are complementary, not substitutes, which is why most production RAG systems run both. The two axes from Part 1 still apply. Recall is whether the relevant documents were retrieved. Latency is how quickly. Every design choice in this article moves along one or both of these axes. This article assumes familiarity with OpenSearch basics such as indices, shards, and mappings; for a refresher, see the References section . Two shapes of RAG corpus Before discussing techniques, it helps to name the two shapes of corpus that lexical retrieval typically serves. The right technique depends on which shape the corpus has. The first shape is a document corpus . Each record is essentially a blob of text: a web page, a knowledge-base article, a support ticket, a research paper. There may be a title and a URL. The rest is a body of prose. Queries against a document corpus ask which documents mention the query terms, and the interesting question is how to score the match well. Field choice is not the hard part, since there are only a few fields and one of them (the body) dominates. The second shape is a structured corpus . Each record has many typed fields: a job-postings corpus with title, company, location, seniority, required skills, and description text; an academic-papers corpus with title, authors, venue, year, keywords, abstract, and body; a customer support corpus with title, product area, resolution status, and body text. The same query can hit different fields for different intents, and metadata drives filtering as much as text matching drives scoring. Field choice, weighting, and filter shape all matter here. Both shapes use BM25 under the hood. Both benefit from the same analyzers. But the query shape differs, the mapping choices differ, and some techniques (like combined fields) apply mostly to one shape and not the other. Each section below calls out which shape it applies to. Term matching for controlled values Not every field should be scored. Some fields hold a small, closed set of controlled values: a country code, a status flag, a document type from a fixed taxonomy, a tenant identifier. For those fields, term matching is the right tool, not scored text matching. The right shape is a keyword field and a term or terms query: "country": { "type": "keyword" }, "status": { "type": "keyword" } GET /records/_search { "query": { "bool": { "filter": [ { "term": { "country": "US" } }, { "terms": { "status": ["published", "reviewed"] } } ] } } } This is exact-match, non-scored, and cache-friendly. There is no analyzer chain to argue with, no tokenization to worry about, no scoring math to interpret. The pitfall is treating an open-ended text field like it were controlled. A field called department that holds free-form user-entered strings like "Software Eng", "software engineering", "eng team", and "engineering group" is not controlled, and term matching on it will miss most of what a reasonable query should retrieve. If the data is not truly controlled at ingest, use text matching, not term matching. Building a parser to force free-form text into enums is a warning sign that the wrong tool has been chosen. The short rule is: term/terms for controlled enums, match/multi_match for free text. If you find yourself writing a parser to bridge the two, the field is not really controlled, and you should treat it as free text. BM25 as the default for free text For everything that is not a controlled enum, BM25 is the default scoring model in OpenSearch and it is a strong baseline. BM25 answers the question “how relevant is this document to this query?” by producing a numeric score per matching document. Higher is better. Documents come back sorted by that score. The score is built from three ideas, each of which captures something intuitive about what makes a document relevant. Term frequency. A document that mentions the query term ten times is more about that term than one that mentions it once. But not linearly: mention it a hundred times and the document is not a hundred times more relevant. BM25 uses a saturating curve, so extra occurrences of the same term give diminishing returns. This is why keyword stuffing does not indefinitely raise BM25 scores. Inverse document frequency. A term that appears in almost every document is a weak relevance signal. The word the occurring in a document says nothing about relevance. A rare term like opentelemetry occurring in a document says a lot. BM25 weights each query term by how rare it is across the whole corpus, so rare terms carry more scoring weight per occurrence than common terms. Length normalization. A long document has more chances to contain any given term by accident than a short one does. Without correction, long documents would always beat short ones on scoring, even when they only mention the query term in passing. BM25 normalizes by document length so a short focused document is not unfairly penalized against a long one. The final score for a document is the sum of these three contributions across all query terms. Two tuning knobs (traditionally called k1 and b) control how aggressively term frequency saturates and how strongly length is normalized. OpenSearch ships with BM25 as the default similarity for text fields with sensible values for both, so no configuration is needed to start. A minimal BM25 query against a single text field looks like this: GET /records/_search { "query": { "match": { "body": "how does bm25 scoring work" } } } Behind the scenes, OpenSearch runs the query text through the same analyzer as the indexed field (the next section explains what an analyzer is), computes BM25 scores for the matching documents, and returns the top results. When a document ranks somewhere surprising, BM25’s score is fully inspectable. Adding "explain": true to the search request returns the per-term score breakdown for each returned document, so you can see which terms contributed and how much. Document corpora: keep it simple For document corpora, resist the temptation to build a multi-field query. Documents in this shape are mostly body text, and complexity in the query rarely earns its cost. A reasonable mapping for a document corpus is one body field with a well-tuned analyzer, plus a lightly boosted title field: PUT /documents { "mappings": { "properties": { "title": { "type": "text", "analyzer": "english" }, "body": { "type": "text", "analyzer": "english" }, "url": { "type": "keyword" } } } } The query is small: GET /documents/_search { "size": 10, "query": { "multi_match": { "query": "gradient checkpointing memory savings", "fields": ["title^2", "body"], "type": "best_fields" } } } A modest boost on title reflects the reality that a term appearing in the title is a stronger relevance signal than the same term appearing once inside a long body. Do not push the boost too high: at 5x or 10x, the title match starts to dominate scoring even when the body match is more substantive, and results get worse. Beyond the title-vs-body split, most document corpora do not have interesting field structure. If a corpus has additional metadata (author, publication date, tags), prefer to use those as filters rather than as scored fields. A scored field must earn its cost: if a match on author is not really a meaningful relevance signal, matching against it just adds noise and slows the query. The shorter the field list, the easier BM25's IDF math is to reason about, the faster the query runs, and the smaller the surface area for scoring bugs. Simple is the right default for documents. Structured corpora: combined fields versus multi-field fanout Structured corpora invite a different mistake. When each record has ten or more text fields, it is natural to write a multi_match across many of them with per-field boosts. For a job-postings corpus: GET /jobs/_search { "query": { "multi_match": { "query": "python backend engineer", "fields": [ "title^3", "company^3", "skills^8", "seniority^6", "team^6", "description^0.5", "location" ], "type": "best_fields" } } } This feels right. It says “match on title, skills, seniority, and so on, with different weights.” In practice, it has two problems. The first is that BM25’s IDF is computed per field. The term python has one document frequency in skills, a different one in title, a different one in description. So the score for the same term differs across fields not just by the boost but by the corpus statistics of each field, and combining them with fixed boost multipliers rarely produces a coherent ranking. The best_fields mode picks the max score across fields, which mutes this problem for the winner but discards signal from the other fields. The second problem is cost. Each field in the fanout adds its own postings-list lookup per query term. Across many fields and many query terms, the query does more work than it needs to. On a large index with high query rates, this shows up as latency. The alternative is to consolidate related fields into one combined field at index time and query that one field: PUT /jobs { "mappings": { "properties": { "title": { "type": "text", "copy_to": "search_text" }, "company": { "type": "text", "copy_to": "search_text" }, "team": { "type": "text", "copy_to": "search_text" }, "seniority": { "type": "text", "copy_to": "search_text" }, "skills": { "type": "text", "copy_to": "search_text" }, "location": { "type": "text", "copy_to": "search_text" }, "search_text": { "type": "text", "analyzer": "english" } } } } copy_to tells OpenSearch, at index time, to concatenate the values of the source fields into the target search_text field. The source fields are still stored and searchable individually if needed; the combined field is an additional inverted index built from the same content. The query then targets one field: GET /jobs/_search { "query": { "match": { "search_text": "python backend engineer" } } } The postings-list work collapses from N lookups per term to one. IDF is computed once, across the whole combined field, so the term statistics are consistent. Scoring becomes easier to reason about because there is one text field doing the work. The cost is that field-specific boosting is no longer possible against the combined field, since the field is a mash of everything. If some sources are genuinely more important than others, you can either keep one high-signal field separate (like skills if skill matches are notably more predictive of relevance) and query it alongside the combined field, or you can weight the sources at concatenation time by repeating their content in the combined field. In practice, a small number of fields outside the combined field is often the cleanest shape: GET /jobs/_search { "query": { "multi_match": { "query": "python backend engineer", "fields": ["skills^8", "search_text"], "type": "best_fields" } } } Two fields instead of seven, and the seven-way fanout is replaced by one composite text field that BM25 can reason about cleanly. The typical outcome, when measured against a representative query set, is that recall stays the same and latency drops substantially. Different production systems have reported multi-x latency reductions on similar consolidations. Combined fields are the single biggest lever for structured lexical retrieval, and the one most search articles skip. How OpenSearch turns text into tokens: the analyzer Almost every design decision in the rest of this article is about a component called the analyzer . This section explains what an analyzer is, because the sections that follow (partial matching, filters, stop-words, multilingual handling, symmetry) all depend on understanding it. When you index a document with a text field, OpenSearch does not store the raw string as the searchable unit. It runs the text through an analyzer, which produces a stream of tokens , and those tokens are what get written to the inverted index. Search works by matching query-side tokens against the tokens that are already in the index. So the analyzer decides what “the same” means for search purposes. The OpenSearch analyzer pipeline in action. Image generated using Gemini AI. An analyzer is a pipeline with three stages: Character filters transform the raw text before tokenization. Common examples are stripping HTML, replacing patterns, and normalizing punctuation. Most analyzers do not use character filters, so this stage is often empty. A tokenizer splits the transformed text into tokens. The standard tokenizer splits on unicode word boundaries. The whitespace tokenizer only splits on whitespace. Language-specific tokenizers (icu_tokenizer, nori, kuromoji) split according to the rules of the language. Token filters transform the token stream after tokenization. This is where most of the interesting work happens: lowercasing, removing stopwords, folding diacritics, applying stemming, generating ngrams, expanding synonyms. Multiple token filters chain together in order. A worked example. Consider the input "Running the Kubernetes cluster". After the standard tokenizer, the tokens are ["Running", "the", "Kubernetes", "cluster"]. After a lowercase token filter, the tokens are ["running", "the", "kubernetes", "cluster"]. After a stop filter (with English stopwords), the tokens are ["running", "kubernetes", "cluster"]. After a snowball (English stemming) filter, the tokens are ["run", "kubernet", "cluster"]. That final list is what OpenSearch actually writes to the inverted index for this document. A future query for "kubernetes" will pass through the same analyzer, producing the token kubernet, which matches the indexed token. A future query for "the running" will produce ["run"] (the stopword is dropped, the verb is stemmed), which matches. Analyzers are configured in the index settings and then referenced from the mapping: PUT /records { "settings": { "analysis": { "analyzer": { "my_analyzer": { "type": "custom", "tokenizer": "standard", "filter": ["lowercase", "stop", "snowball"] } } } }, "mappings": { "properties": { "body": { "type": "text", "analyzer": "my_analyzer" } } } } By default, the analyzer configured on a field runs at both index time (when documents are written) and query time (when the query text is processed). Both sides go through the same pipeline, so the tokens line up and matching works. The exception, covered later under analyzer symmetry, is when index-time and query-time need to differ. With this in mind, most of the rest of the article is easier to state plainly. Stop-word removal is a token filter. Edge-ngram generation is a token filter. Synonym expansion is a token filter. Diacritic folding is a token filter. Multilingual support is a choice of tokenizer plus language-specific token filters. Everything is a chain of stages in the analyzer pipeline. Understanding the pipeline is understanding lexical search. Partial matching: fuzzy versus edge-ngram Users make typos. Users type prefixes. Users hit enter mid-word. The two common ways to handle partial matching are runtime fuzzy matching and index-time edge-ngram tokenization. They have different cost profiles and cover different typo patterns. Fuzzy matching computes an edit-distance expansion at query time. For each query term, OpenSearch generates a set of variants within a small Levenshtein distance (typically one or two edits) and searches for matches on any of them. It is turned on per query: GET /records/_search { "query": { "match": { "body": { "query": "kuberntes", "fuzziness": "AUTO" } } } } Fuzzy expansion catches typos anywhere in a word, including transpositions (kuberntes matches kubernetes) and single-character substitutions. The index does not need any special preparation. But every fuzzy query pays a per-term expansion cost at query time. On a busy cluster with long queries, this cost adds up. Edge-ngram tokenization takes a very different approach. Instead of expanding the query at query time, it expands each document's tokens at index time, so that partial matching becomes plain exact-token matching at query time. Edge-ngram is a token filter (the previous section covered what those are) that emits all prefix substrings of each token it receives, up to a configured length. Configured on a field: PUT /records { "settings": { "analysis": { "filter": { "edge_ngram_2_20": { "type": "edge_ngram", "min_gram": 2, "max_gram": 20 } }, "analyzer": { "edge_ngram_analyzer": { "type": "custom", "tokenizer": "standard", "filter": ["lowercase", "asciifolding", "edge_ngram_2_20"] } } } }, "mappings": { "properties": { "body": { "type": "text", "analyzer": "edge_ngram_analyzer" } } } } To see what this does, walk through what happens at ingest for a document whose body is "kubernetes cluster": The standard tokenizer produces ["kubernetes", "cluster"]. lowercase produces ["kubernetes", "cluster"] (already lowercase in this case). asciifolding produces ["kubernetes", "cluster"] (no diacritics). edge_ngram_2_20 emits every prefix of each token from 2 to 20 characters long, producing ["ku", "kub", "kube", "kuber", "kubern", "kuberne", "kubernet", "kubernete", "kubernetes", "cl", "clu", "clus", "clust", "cluste", "cluster"]. All 15 tokens are then written to the inverted index for this document. The document now appears in the postings list for ku, for kub, for kube, and so on. This is where edge-ngram's cost lives: the inverted index carries many more entries than a plain tokenizer would produce , and every document goes through this expansion at ingest. At query time, the payoff shows up. When a user types kube, the query is processed by the query analyzer (which, as we'll see in the analyzer-symmetry section, should not also do edge-ngram) and produces the single token kube. OpenSearch does one lookup for kube in the inverted index and finds every document whose analyzer previously emitted kube as one of its prefix tokens. The lookup itself is a single index probe. No runtime expansion, no per-term computation, no scan over variants. The prefix work was already done at ingest. This is the key mental shift. Partial matching with edge-ngram is not a query-time algorithm change; it is an index-time storage change. Every document was pre-expanded to hold all its prefixes, and the query is a plain term match against those prefixes. Fuzzy versus edge-ngram: cost trade-offs. The contrast with fuzzy is now clean: Fuzzy leaves the index unchanged and expands the query at query time. Every query pays the expansion cost. Edge-ngram leaves the query unchanged and expands the documents at index time. Every ingested document pays the expansion cost, but every query is cheap. The costs live on opposite sides of the pipeline. Edge-ngram’s tradeoffs follow directly: the inverted index is larger (each word contributes many prefix tokens instead of one), ingest CPU is higher (the analyzer does more work per document), and the technique only catches prefix matches (kube matches kubernetes, but kubrentes does not, because no document was indexed with kubrentes as one of its prefixes). The right choice depends on the typo pattern in the query stream. For type-ahead or autocomplete flows, users are entering prefixes, and edge-ngram is the right tool: the query is cheap and the pattern is exactly what edge-ngram covers. For arbitrary typo tolerance on established typing, fuzzy at query time is more general, at higher query cost. The pattern that most mature production systems settle on is edge-ngram for the fields where partial matching matters, without runtime fuzzy expansion. The reasoning is that prefix matches cover the common case (users typing, or truncated queries), the query cost is low, and the index growth is bounded because ngram range is capped (typically 2 to 20 characters). Runtime fuzzy is kept in reserve for specific use cases where edge-ngram is not enough. A gotcha worth naming here: if a field is indexed with edge-ngram, the same analyzer runs on the query by default. That means the query text also gets edge-ngrammed, so kubernetes becomes eight query tokens (ku, kub, ..., kubernetes) instead of one. This inflates the postings intersection work at query time and can hurt scoring. The fix is to set a different search_analyzer, which is covered in the analyzer symmetry section below. Filters, boosts, and scoring functions Three tools shape which documents come back and how they rank. They look similar in query shape but have different jobs, and mixing them up is a common source of muddy results. Hard filters require a match. A document without the required attribute is not returned. They are unscored and cache-friendly. Use them for attributes that are non-negotiable for correctness: an access control list, a tenant identifier, a status flag, a recency window, a “must be published” requirement. GET /jobs/_search { "query": { "bool": { "must": [ { "match": { "search_text": "python backend engineer" } } ], "filter": [ { "term": { "tenant_id": "acme" } }, { "term": { "status": "open" } }, { "range": { "posted_at": { "gte": "now-30d" } } } ] } } } Soft boosts raise the score for documents with a desired attribute but do not require it. Use them for signals that improve results when present but should not exclude documents when absent: “remote”, “senior level”, “recently posted”: GET /jobs/_search { "query": { "bool": { "must": [ { "match": { "search_text": "python backend engineer" } } ], "should": [ { "term": { "tags": "remote" } }, { "term": { "tags": "senior" } } ], "minimum_should_match": 0 } } } minimum_should_match: 0 is the key: it says "match none of these if you must, but prefer documents that match at least one." Without that, the boost effectively becomes a filter. Scoring functions fold a continuous value directly into the score. Recency is a good example: a document is not "recent yes or no," its relevance degrades smoothly with age. function_score with a decay function expresses this cleanly: GET /jobs/_search { "query": { "function_score": { "query": { "match": { "search_text": "python backend engineer" } }, "functions": [ { "gauss": { "posted_at": { "origin": "now", "scale": "14d", "decay": 0.5 } } } ], "score_mode": "multiply", "boost_mode": "multiply" } } } The right question when adding any signal is: is this a requirement, a preference, or a continuous factor? Requirement is a filter. Preference is a soft boost. Continuous factor is a scoring function. Reaching for the wrong one is a common cause of results that “look reasonable but rank strangely.” A signal that should degrade smoothly (like recency, or a numeric proximity to a target value) degrades badly if expressed as a hard threshold in a filter. A requirement that is expressed as a soft boost lets non-matching documents slip through. Where logic lives: application code versus the analyzer chain Every non-trivial lexical pipeline ends up doing preprocessing on the query text before BM25 sees it. The interesting question is not what preprocessing, but where it runs. There are two homes: application code, and the analyzer chain. Application-code preprocessing is what most teams start with. It is where prototyping happens: pull the query text out, strip some stop words, lowercase, remove some context-specific noise, hand the cleaned string to OpenSearch. This works. It is fast to iterate on. It runs at query time on every request. And, over time, most of the steps that started in application code turn out to belong somewhere else. The alternative is to move the preprocessing into the analyzer chain, which runs at both index time and query time inside OpenSearch. The classic example is stop word removal. In application code, it looks like this: STOP_WORDS = {"a", "an", "the", "of", "for", "and", "or", "what", "how", "does", "when", "where", "near", "me", "is", "are"} def strip_stopwords(query): tokens = query.lower().split() return " ".join(t for t in tokens if t not in STOP_WORDS) The equivalent in the analyzer chain is a filter chained into a custom analyzer: PUT /records { "settings": { "analysis": { "filter": { "english_stops": { "type": "stop", "stopwords": "_english_" } }, "analyzer": { "clean_analyzer": { "type": "custom", "tokenizer": "standard", "filter": ["lowercase", "asciifolding", "english_stops"] } } } }, "mappings": { "properties": { "body": { "type": "text", "analyzer": "clean_analyzer" } } } } Three reasons the analyzer chain is the better home for anything context-free: Multilingual coverage is a config change, not code. OpenSearch’s stop token filter ships language stopword lists for many languages, addressable as _english_, _spanish_, _french_, _german_, and so on. Adding a locale is one line, not a code change. Applied uniformly. The analyzer runs on every document at index time and every query at query time. There is no code path that can accidentally skip it. In application code, someone has to remember to call the filter, and eventually someone somewhere forgets. Free at query time. Index-time work is paid once per document during ingest. Application-code query-time filtering is paid on every request. On a busy cluster at high queries per second, that difference is meaningful. The generalization: any preprocessing that is context-free belongs in the analyzer chain . Stopword removal, lowercasing, diacritic folding, tokenization, ngramming, synonym expansion. All of these are decisions that depend only on the token stream, not on anything specific to the caller or the query slot. What stays in application code : anything context-dependent. If preprocessing needs to know something the analyzer cannot see, it stays in the application. Two examples: A query arrives with a slot like "filter_by_company": "acme", and the application knows to remove the word "acme" from the query text because the slot already restricts results to that company. The analyzer cannot know this, because "acme" is a meaningful search term in a different query where no company filter was provided. This is per-query context. A multi-tenant system knows the tenant name should be stripped from the query text for one tenant but not another, based on tenant configuration. The analyzer cannot reach into tenant config. This is per-tenant context. Most mature lexical pipelines end up with a small amount of context-dependent application-code preprocessing, a rich analyzer chain doing everything else, and a very simple query builder that just wires the two together. The maturity curve is that application code shrinks over time and the analyzer chain grows. Analyzer symmetry and the search_analyzer override OpenSearch runs the same analyzer at index time and at query time by default. This is usually right: the tokens on both sides come out the same, and BM25 matches them directly. Search should match indexing. The exception is edge-ngram. When a field is indexed with edge-ngram, the documents should be edge-ngrammed (so a query for kube can find kubernetes). But the query should not be re-edge-ngrammed, because that turns one query term into many, inflates the postings intersection, and can hurt scoring on short queries. The fix is search_analyzer, which lets index-time and query-time analyzers differ: "body": { "type": "text", "analyzer": "edge_ngram_analyzer", "search_analyzer": "standard_analyzer" } Here edge_ngram_analyzer runs at index time and produces all prefix tokens. standard_analyzer (or whatever your non-ngram analyzer is) runs at query time and produces one token per word. A query for kubernetes becomes one token, which hits the indexed kubernetes token. A query for kube becomes one token, which hits the indexed kube token. Both work, but the query does one postings lookup per word instead of several. The rule to internalize is: index-time and query-time analyzers should be the same unless the field's indexing does something that the query should not repeat . Edge-ngram is the common case. Synonym expansion is another: some teams synonym-expand only at index time (so queries stay short), some only at query time (so the index does not bloat), and getting this wrong in either direction is a real footgun. When search_analyzer is missing on a field where it should exist, queries often work but do more work than they need to. The symptom is high latency with no obvious cause. Multilingual handling Everything above assumes English. Real corpora are usually multilingual. Four topics matter here, each briefly. Tokenization. The standard tokenizer splits on unicode word boundaries and works reasonably for European languages. It handles CJK languages badly. For a corpus that includes Chinese, Japanese, or Korean text, use language-specific tokenizers: icu_tokenizer for general unicode handling, nori for Korean, kuromoji for Japanese, and analysis plugins for Chinese. Getting this wrong produces token streams that BM25 cannot score meaningfully. Stemming. english uses the snowball stemmer to conflate running, runs, ran into run. Snowball has variants for many European languages: spanish, french, german, italian, portuguese, and others. Light stemmers exist for cases where snowball is too aggressive. For languages without well-established stemmers, no stemming is better than bad stemming; over-stemming muddies scoring. Diacritic folding. icu_folding and asciifolding normalize accented characters so that café and cafe produce the same token. This is almost always desirable for user-facing search, since users type without diacritics as often as they type with them. Fold at both index time and query time so tokens align. One index per locale, or one index with a locale field. For a small number of locales with distinct analysis pipelines (English, Japanese, and Arabic), separate indices per locale is often the cleanest shape: each index has its right tokenizer, stemmer, and analyzer chain, and queries route by locale. For many locales, or when locales share analysis (English and Spanish can use similar pipelines with different stopword lists), one index with a locale field and a per-locale analyzer selection is usually simpler to operate. The tradeoff is analysis fidelity against operational overhead: more indices give better analysis, one index gives simpler operations. Where BM25 falls down BM25 is a strong baseline, but it is exact-token scoring with corpus statistics. It does not know synonyms. It does not know that k8s means kubernetes. It does not know that a query for ML and a document about machine learning describe the same concept. Three failure patterns show up in every production lexical pipeline: Abbreviations and acronyms. A query for ML will match documents that literally contain the string ml, and miss documents that describe the same concept as machine learning. Both are correct answers; BM25 sees them as unrelated. Concept versus surface form. A query for distributed tracing retrieves documents that literally contain those words. Documents about opentelemetry, spans, or observability may be just as relevant but score lower or not at all, because the query terms are not in the text. Broad-term versus specific-term queries. A query for database should probably match documents about postgres, mysql, dynamodb, and general database material. BM25 only matches documents whose text contains the token database. It does not know that a document about postgres tuning is a database document. Other techniques exist to bridge these gaps (synonym token filters at index time, upstream query rewriting via rule engines or LLMs), but each comes with real costs: dictionaries to maintain, latency added to the query path, and coverage that is only as good as the rules you wrote. At best, these are stop-gaps until actual semantic retrieval is implemented. The right way to complement lexical retrieval is semantic retrieval. Retrieve semantically alongside lexically , which is the natural bridge to Part 1. The semantic path does not care that distributed tracing and opentelemetry are different tokens; their embeddings are close. Running lexical and semantic retrievers in parallel and merging their results (a hybrid retrieval strategy) is the most common production answer to BM25's failure modes. In a mature production RAG pipeline, lexical retrieval does what it does well (fast, explainable, precise on exact terms) and semantic retrieval covers the rest. Trimming the response payload Part 1 covered response payload trimming for the semantic path. The same lever applies to the lexical path, for the same reason. Conclusion The through-line of Part 2 matches Part 1. Push complexity to the boundaries. Keep the hot path simple. The specific translations for lexical retrieval: Field structure lives in the mapping and the analyzer chain. Move context-free preprocessing (stopwords, folding, ngrams, synonyms) out of application code and into analyzers. Consolidate related fields into combined fields at index time when the query would otherwise fan out across many. One well-tuned field beats N boosted fields for both latency and BM25’s scoring coherence. Match the right tool to the right job. Term queries for controlled enums, match queries for free text, hard filters for requirements, soft boosts for preferences, scoring functions for continuous factors. Do not reach for edge-ngram or fuzzy universally. Pick based on the typo pattern in the query stream. Edge-ngram for prefix and type-ahead, fuzzy for mid-word typos. None of this should be adopted on faith. Every corpus is different. The right approach is to establish a baseline, make one change at a time, and measure recall and latency against that baseline on a representative query set. The right configuration is the one the measurements support, not the one that is expected to work. References A Comprehensive Guide to ElasticSearch and OpenSearch Architecture OpenSearch BM25 OpenSearch Fuzzy Match OpenSearch edge-ngram Other Articles Agentic Inference Deployment Operational Readiness for LLM Services Benchmarking AI Agents OpenSearch Optimizations for Production RAG, Part 2: Lexical Retrieval was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Geology PhD founders at AIgneous Million Whys launch daily trivia app for curious adults worldwide
Geology PhD founders at AIgneous Million Whys launch daily trivia app for curious adults worldwide USA Today
- Summer Is Half Over. I Let AI Plan the Rest of It So I Don't Miss Any More
The summer isn't over yet, and AI is surprisingly good at making sure what's left of it doesn't go to waste.
- How does ChatGPT see the 2026 season going for the UCLA Bruins?
How does ChatGPT see the 2026 season going for the UCLA Bruins? UCLA Wire
- How to Use Claude Code with Kimi K3 (and Switch Models by Typing One Word)
A verified, step-by-step guide to running Claude Code against Moonshot AI’s Kimi K3 using Anthropic-compatible environment variables, plus… Continue reading on Towards AI »
- Fine-Tuning Qwen3 with LoRA Using NVIDIA NeMo AutoModel: A Complete Single-GPU Google Colab Workflow Tutorial
Fine-Tuning Qwen3 with LoRA Using NVIDIA NeMo AutoModel: A Complete Single-GPU Google Colab Workflow Tutorial MarkTechPost
- I'm a mom of 2 kids. Sending voice memos to Claude helps me organize my life.
I'm a mom of 2 kids. Sending voice memos to Claude helps me organize my life. Business Insider
Score: 15🌐 MovesJul 19, 2026https://www.businessinsider.com/mom-two-voice-memos-claude-organize-life-2026-7 - I Helped 50,000 People Set Up Claude Code. Here’s What I Didn’t Tell Them
The setup was the easy 20%. Six months in, here’s what actually breaks, memory that rots without a warning, connectors that say… Continue reading on Towards AI »
- #3 Claude Loops: Design the Tool Loop
How to expose the right tools, hide the dangerous ones, and use permissions so Claude Code can act without wandering. Claude Loops: Design the Tool Loop A workshop is not safer because every tool has a warning label. It is safer because the saw, soldering iron, and paint thinner are not all sitting on the same bench for every job. Claude Code has the same problem. The loop becomes powerful at the exact place it becomes capable of doing damage. Tools Turn Reasoning Into Action Why does the tool loop matter more than another prompt trick? Because tools are where Claude stops being only a model and starts becoming an operator. In Part 1, we looked at the basic loop: observe, act, inspect, repeat. In Part 2, we looked at the context loop: what stays inside the working window and what gets cleared, compacted, or forked away. Part 3 is about the action surface. Claude can answer directly, or it can call a tool. The Claude platform docs describe this as part of normal tool use: Claude evaluates the available tools and can decide when a tool is needed. In Claude Code, those tools are not abstract. They can read files, grep a repo, edit code, run shell commands, call MCP servers, launch subagents, and hand results back into the next turn. That is why a tool is not just a feature. It is a door. A tool loop is safe when Claude can reach the evidence it needs and cannot reach the power it does not. The Model Chooses From What You Expose Why does tool overload make Claude worse instead of better? Because every visible tool becomes part of the decision space. If Claude sees one file system, one shell, and one test command, the next move is relatively legible. If Claude sees fifteen MCP servers, browser automation, database tools, deploy commands, issue trackers, spreadsheets, and a pile of project-specific scripts, the loop has to decide not only what to do but which door is the right door. Sometimes that extra surface is useful. A product research loop may need Linear, GitHub, docs, and a browser. A local bug-fix loop probably does not need production Stripe, analytics exports, and a write-capable database connector. The point is not minimalism for aesthetics. The point is reducing wrong possible actions. Your job is not to show Claude every tool you own. Your job is to show Claude the smallest set of tools that can complete this task with evidence. Permission Rules Are the Harness What makes a tool boundary real instead of aspirational? Permissions. Claude Code’s permissions docs are unusually direct about this. Prompt instructions and CLAUDE.md can shape what Claude tries, but they do not change what Claude Code allows. Permission rules, permission modes, and PreToolUse hooks are what grant or revoke access. That distinction is the whole article. If your CLAUDE.md says “never push to main,” Claude may try to comply. If your permission rules deny Bash(git push *), the push is blocked by the harness. If your prompt says “do not read secrets,” Claude may avoid .env. If your permissions deny Read(./.env), Claude Code can enforce that boundary for the built-in read path. Guidance is useful. Enforcement is different. Permission ladder chart: allow, ask, deny Allow the Boring, Ask on Risk, Deny the Irrelevant How should you decide what goes in allow, ask, and deny? Start from the task, not the tool list. For a normal code fix, a good loop should not ask you before every status check. It should be able to run read-only inspection, test commands, and narrow lint commands without turning the session into a permission-dialog simulator. But broad writes, deploys, pushes, destructive migrations, and production access should create a human checkpoint or vanish entirely from the loop. The practical ladder is: allow routine and reversible actions ask before risky actions that might be necessary deny actions that should never be part of this task That is different from saying “trust Claude” or “do not trust Claude.” Trust is not the switch. Scope is the switch. Use Bare Denies to Hide Tools When should a tool disappear from Claude’s context completely? When using it would be wrong for the task even if Claude asked nicely. The permissions docs make an important distinction. A bare deny rule like Bash removes the tool from Claude's context entirely. A scoped rule like Bash(rm *) leaves Bash visible but blocks matching calls when Claude attempts them. That is a design choice. If this loop should never use MCP database tools, deny the MCP tools as a class or hide that server from the session. If this loop can use Bash but must not push code, keep Bash visible and deny the risky command pattern. Think of bare denies as removing a door from the room. Think of scoped denies as putting a lock on one kind of door handle. Both are useful. They solve different problems. Tool Search Is Better Than Tool Piles Why not preload every specialized tool and hope Claude picks the right one? Because the loop pays attention to the tools it can see. Claude Code supports deferred tool loading through tool search. Instead of putting every possible tool definition into context at startup, Claude can discover relevant tools when needed. That is the same pattern we used in Part 2 for context: keep the main window clean, then pull in what the task demands. This is especially important for MCP-heavy setups. A GitHub server, Slack server, database server, Notion server, browser server, and custom analytics server might all be useful somewhere. That does not mean every loop should see every tool description on every turn. Tool discovery is context hygiene for actions. The best tool surface is not the biggest one. It is the one that makes the next correct action obvious. Hooks Catch What Patterns Miss Where do permission patterns stop being enough? At the edge of messy reality. Command patterns are useful, but they are not a full policy engine. Claude Code’s docs warn that constraining things like network access through Bash command patterns can be fragile. A URL can move through variables, redirects, flags, or wrapper commands. A file can be touched indirectly by a script. That is where hooks matter. PreToolUse hooks let you inspect or block tool calls before they execute. They are useful when a simple allow or deny pattern is not expressive enough. Examples: block destructive database phrases in shell commands reject deploy commands outside a release branch require issue IDs before editing a regulated directory log every MCP call touching customer data block writes when tests are currently failing The hook is not there to make Claude smarter. The hook is there to make the loop less negotiable. Terminal GIF showing Claude Code permissions, allow rules, deny rules, and PreToolUse hook The Dangerous Mode Is Not Always the Loud One When do tool loops become risky in a way that feels innocent? But here’s the thing: the dangerous loop is not always the one that says “delete production.” Sometimes it is the loop with too many harmless tools. A read-only command can still fill context with irrelevant output. A browser tool can wander into research instead of fixing the bug. A GitHub tool can pull in ten issue threads when the failing test already named the cause. A write-capable MCP server can look like a convenience until Claude sees it as the fastest path. Risk is not only damage. Risk is drift. The loop starts with a small task, then the tool surface invites a larger one. Claude reads more, opens more, compares more, and returns with a plan that is technically impressive and operationally unhelpful. That is the quiet failure mode of tool overload. The tools were not dangerous one by one. The surface was too broad for the loop. Design Tools Per Task Phase How should the tool surface change as work moves from exploration to execution? By phase. Exploration needs broad read access, search, and maybe web or docs tools. Implementation needs narrow file reads, edits, and tests. Verification needs commands that prove the change worked. Release work may need GitHub, CI, changelog, and deploy tools, but only after the change is known. Do not treat those as one giant session with one giant tool surface. Give each phase the tools it needs: explore with reads and search implement with edits and local tests verify with test, lint, and diff release with explicit asks around push, deploy, and publish This is why permission modes matter. Plan mode is useful when Claude should explore without editing. Manual approvals are useful when you want to inspect first uses. More permissive modes belong in isolated environments where damage is contained. The right mode is not a personality trait. It is a runtime choice. The Tool Loop Contract How should you ask Claude when tool use should be intentional? Write the tool boundary into the request. Weak request: Code Fix the checkout bug. Better request: Code Use read-only inspection first. Do not call payment, production database, or deploy tools. Run the checkout test suite, inspect the smallest failing path, make the minimal local edit, rerun tests, and stop before any push. For a permissions-first session: Code Before editing, list the tools you expect to need. If a tool outside that set becomes necessary, ask before using it and explain why the loop changed. That prompt does not replace permissions. It makes the human intent legible while the harness enforces the real boundary. The third loop is simple: expose the smallest useful tool surface allow routine evidence gathering ask before risky actions deny irrelevant or dangerous doors use hooks when patterns are too weak change the tool surface when the task phase changes Part 4 will move from tools to verification: how to make Claude prove the loop worked before you trust the answer. Until then… — Sage 🍓 PS: Pick one Claude Code project and remove one tool or MCP server from the default session. If Claude only needs it once a week, it probably should not sit in every loop. Reference Notes Configure permissions Claude Code settings Automate actions with hooks Tool use with Claude Explore the context window Related Reading Claude Loops 1: Stop Using Claude Like Chat Claude Loops 2: Context Is the Hidden Loop Claude Code MCP Servers: How to Connect, Configure, and Use Them #3 Claude Loops: Design the Tool Loop was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
Score: 10🌐 MovesJul 19, 2026https://pub.towardsai.net/3-claude-loops-design-the-tool-loop-9a1cfe599c3e?source=rss----98111c9905da---4 - Google Maps Scraper by Outscraper Helps Businesses Scale Lead Generation
Google Maps Scraper by Outscraper Helps Businesses Scale Lead Generation azcentral.com and The Arizona Republic
- Your next car’s software update could become its biggest security risk
Experts say over-the-air vehicle updates are transforming the auto industry while creating new cybersecurity and national security risks that governments can no longer ignore.
Score: 08🌐 MovesJul 19, 2026https://www.digitaltrends.com/cars/your-next-cars-software-update-could-become-its-biggest-security-risk/ - Last week's roundupInside: Last week's AI, caught up in 5
A recap of the week's AI news and developments.
- [Yoo Choon-sik] Vacancies at heart of Korea's artificial intelligence policy
Despite taking office without a proper preparation period, the Lee Jae Myung administration has achieved remarkable results in several areas that none of its predecessors managed to accomplish within such a short period. At the center of those achievements has been a narrative surrounding artificial intelligence. One example is the surge in exports by South Korean manufacturers of semiconductors and other hardware related to the AI sector as global competition to secure an advantage in AI innova
- SK chief warns AI memory shortage could turn geopolitical
SK Group Chairman Chey Tae-won said the global shortage of AI memory chips is severe enough that foreign governments have started intervening on behalf of their own industries, and warned that Seoul is likely to face similar pressure. He separately argued that SK hynix needs to accelerate capacity expansion at home and abroad, describing today's memory prices as "abnormal" and warning that sustained high prices would invite new competitors and geopolitical retaliation. "Building where we can, as
- Chinese AI chipmaker: Open collaboration is key to the future of AI
China's homegrown AI innovations, such as optoelectronic computing, will help build an open global AI ecosystem across both domestic and international dimensions, said Zou Tianqi, CEO of MakeSens, in an interview with CGTN reporter He Jingyi at WAIC 2026.
- Xi Jinping's key remarks on seizing opportunities brought by AI
Xi Jinping's key remarks on seizing opportunities brought by AI
- Xi Jinping's key remarks on China's role in global AI development
Xi Jinping's key remarks on China's role in global AI development
- S. Korea’s top Go player Shin Jin-seo becomes first pro to beat KataGo
South Korea’s top-ranked Go player Shin Jin-seo pulled off a victory over artificial intelligence program KataGo on Sunday, becoming the first professional player to do so. The 26-year-old clinched a 4.5-point victory over KataGo after a grueling five-hour match. It was the second game of the three-match series that kicked off on Friday in Seoul. Although Shin began the match with a two-stone head start, the victory made him the first professional player to defeat KataGo in an official match. U