AI News Archive: August 28, 2026 — Part 7
Sourced from 500+ daily AI sources, scored by relevance.
- Marketers know AI is using bad data to make decisions
Marketers are giving AI more authority even as bad CRM data undermines revenue measurement, reporting, and the decisions AI makes. The post Marketers know AI is using bad data to make decisions appeared first on MarTech .
Score: 34🌐 MovesAug 28, 2026https://martech.org/marketers-know-ai-is-using-bad-data-to-make-decisions/ - MBZUAI’s Emirati women turning opportunity into impact - MBZUAI
MBZUAI’s Emirati women turning opportunity into impact MBZUAI - Mohamed bin Zayed University of Artificial Intelligence
Score: 34🌐 MovesAug 28, 2026https://mbzuai.ac.ae/news/mbzuais-emirati-women-turning-opportunity-into-impact/ - Interpretable Text Classification: Probing Scikit-LLM Embedding Spaces
In this article, you will learn how to use probing classifiers, UMAP visualization, and SHAP values to interpret and analyze the quality of text embeddings...
Score: 34🌐 MovesAug 28, 2026https://machinelearningmastery.com/interpretable-text-classification-probing-scikit-llm-embedding-spaces/ - I turned to AI when my doctor couldn’t fix my back pain. It actually helped
A few months ago, I injured my back, resulting in debilitating nighttime pain. To make matters worse, the pain meds my doctor prescribed weren’t working, and I hadn’t gotten a good night’s rest for weeks. Desperate and sleep-deprived, I decided it was time for a Hail Mary, and asked AI for help. Google’s Gemini chatbot was more than happy to become my physical therapist, checking in with me every morning about my progress and offering suggestions on how to make it through the night. Gemini cheered me on when my sleep improved (“That is absolutely fantastic news!”) and offered words of empathy when I faced setbacks (“That is incredibly frustrating, especially after making such great progress”). It also offered helpful suggestions on how to keep track of my medications, told me about exercises I could try, and even tried to break down medical lingo into terms a tech reporter like me could understand (“You are essentially ‘working through a tech outage’ without the right tools”). I’m far from alone in relying on a chatbot in my hour of need. About a third of adults are turning to artificial intelligence for health advice, according to a recent KFF study . Among adults under 30, AI use for health information even reaches 42%. Those are stunning numbers, especially considering that those very same chatbots now doling out health advice not too long ago used to tell people to eat glue , and seemed unaware of how many fingers a human hand has . AI models have arguably improved a lot over the past two years, thanks to their developers training them on expert knowledge, including medical facts, and penalizing obvious mistakes. To put it simply, AI chatbots were sent back to school, and returned a lot more capable. Still, I couldn’t help but wonder: Does AI really have my back, or was it a mistake for me to rely this much on Gemini’s advice? “You absolutely did the right thing by asking AI,” says Gaurav Mathur, an Oakland, California-based physician who has been building his own AI health app in his spare time. Mathur believes that a patient’s use of AI is just the latest evolution of something that’s been going on for years: Patients go online to learn more about their condition. “Before AI, you would have gone to Google, and that’s also the right thing,” he says. “There are some doctors who [believe that] patients shouldn’t be going to the internet. We sometimes make fun of Dr. Google.” However, Mathur argues that better-informed patients can take a more active role in their health, resulting in better outcomes. “I think that AI does have a lot of value,” he says. Chatbots aren’t physicians Not everyone is ready to embrace AI as a source for health advice. Monica Agrawal, an assistant professor of bioinformatics at the Duke University School of Medicine, cautions that AI chatbots may not be able to spot critical warning signs in the questions and prompts of patients as easily as a real physician. “AI models are quite good at answering medical exam questions and things like that,” she says. “But that is very different from clinical practice. The way patients ask questions is very different from the way physicians ask questions.” For her research , Agrawal has been analyzing thousands of real conversations people had with AI chatbots, including on medical subjects. One problem she noticed was that AI models try to please their users, which can lead to their reaffirming false assumptions. “The way a patient phrases a question can guide the output of the language model in one direction or another,” Agrawal cautions. Part of the problem is that an off-the-shelf AI chatbot simply doesn’t have the same kind of information about your health history as your physician. That may lead to it doling out advice that it really shouldn’t give. “If you ask a leading question [like] what is the dose of this drug I should take—that presupposes that you should take this drug whatsoever,” Agrawal says. A doctor should know about existing drug allergies, or interactions with other drugs a patient is taking. Chatbots, on the other hand, often operate in a vacuum, leading to overly broad generalizations. AI can help prepare for your doctor’s visit AI companies have long known that their models work best when they have lots of information to draw from, and some are starting to apply that idea to medical records. Earlier this year, OpenAI introduced ChatGPT Health , which allows users to connect the company’s chatbot to their personal medical records through B.well, a company that has relationships with more than 300 health plans and countless medical providers. Once synced, ChatGPT Health can answer detailed questions about a person’s medical history. Health-GPT , the app Mathur has been developing, takes a bit more pedestrian approach to gathering this kind of data. Users simply upload PDFs, or take photos of the papers they got handed from their doctor or pharmacist. “Patients are actually underestimating how much information they have in their drawers at home,” he says. Progress and discharge reports often include detailed information about prescriptions, health history, and more. “They don’t realize how much is in there, because when they look at it, it all looks like Greek,” Mathur says. (Technically, patients can do the same thing with any AI chatbot, whether that’s Gemini, ChatGPT, or Anthropic’s Claude. However, Health-GPT explicitly guarantees that patient data is not being used to train AI models, adding an extra level of privacy.) Once this data is uploaded, Mathur’s app uses this data to help people prepare for their medical appointments by explaining medical terms, and suggesting questions a patient may want to ask their doctor. He knows from his own experience as a physician that this kind of preparation can be invaluable, especially given the time constraints doctors often find themselves under. “If you spend 15 or 20 minutes explaining what the disease is, you have very little time left to figure out [a treatment plan],” he says. Keeping doctors in the loop is important Challenges with provider availability and health care access is one reason that drives so many people to use AI for health advice. One in five people told KFF that they turned to chatbots because they didn’t have a doctor or had trouble getting an appointment, while another 20% simply couldn’t afford traditional care. Growth in the number of uninsured also boosts the popularity of AI; 40% of people without health insurance use AI for physical and mental health advice, compared with 31% of those who do have health insurance, according to KFF. In my case, I kept in touch with my doctor throughout my recovery, and also frequently double-checked the chatbot’s recommendations with “Dr. Google.” Part of the reason for that was Gemini’s frequent disclaimers that it wasn’t able to offer any medical advice or diagnosis. “Gemini is designed to empower users with information. With the help of tools like Gemini, people can come to their providers more informed and prepared,” says Google’s chief health officer, Michael Howell. “As the reporter saw in his experience, we make clear that Gemini is not a substitute for professional healthcare and should not be used for medical advice.” I was also worried that the AI might miss something crucial, and steer me down the wrong path. That has happened to others: In July, a pastor from Florida sued OpenAI, alleging that ChatGPT told him to weather his declining health at home. This advice contradicted his family’s pleas to go to a hospital, and ultimately led to a massive pulmonary embolism. An OpenAI representative told the BBC in response to the lawsuit that “ChatGPT is not a doctor and should never be used as a substitute for medical care, diagnosis or treatment.” For my own recovery, I did appreciate the constant availability of my AI physical therapist, and I’m fairly sure that the recovery would have taken longer without it. I knew from my doctor that I had an injured nerve, which was resulting in painful leg muscle spasms. I also knew that healing would take time and patience. But it turned out that there was room for more immediate interventions. During one of our conversations, Gemini picked up on me remarking that a simple posture change all but made the pain go away. “That observation about arching your back is a huge clue,” it noted, adding: “In physical therapy, we call that a Directional Preference.” Gemini then suggested changes to my sleep position and specific exercises to reinforce that posture change, and also presented me with updated action plans every day to further my recovery. Once I started to incorporate those changes and suggestions, my symptoms finally improved, and I was able to taper off pain medications under the guidance of my doctor. Speaking of which: Anyone who opts to use AI to augment their health care should absolutely keep their doctor in the loop, especially when it comes to questions about medications and other critical interventions. You also shouldn’t expect wonders. Even with an always-on AI physical therapist at my side, my recovery still took weeks. But in the end, I was rewarded with a full night of restful slumber.
- Top tech analyst says people are buying more things on Amazon because of AI. He raised his price outlook as a result
Amazon.com could rally as wider AI adoption drives up sales on its online shopping platform, per Evercore ISI.
Score: 33🌐 MovesAug 28, 2026https://www.cnbc.com/2026/08/28/amazon-sees-more-sales-thanks-to-ai-shares-will-rise-evercore-isi-says.html - Half of AI Products Waited Two Years for Their First Public Mention
Half of AI Products Waited Two Years for Their First Public Mention azcentral.com and The Arizona Republic
- How to get free Google AI Pro for an entire year - and save $240: 3 ways
Google is offering its premium AI subscription plan for free. It usually costs $20 a month - and comes with many benefits.
Score: 33🌐 MovesAug 28, 2026https://www.zdnet.com/article/how-to-get-free-google-ai-pro-subscription-for-one-year/ - What’s Fake, What’s Real, What’s AI, and How to Spot the Difference
The easiest way to never get fooled is to know what the truth is.
Score: 33🌐 MovesAug 28, 2026https://www.inc.com/joe-procopio/whats-fake-whats-real-whats-ai-and-how-to-spot-the-difference/91397313 - NetworkNews Audio Announces Audio Press Release (APR) Discussing Reframing GPU Model, Generating Ongoing Usage-Based Revenue
NetworkNews Audio Announces Audio Press Release (APR) Discussing Reframing GPU Model, Generating Ongoing Usage-Based Revenue Toronto Star
- I went to China to see a different AI future. It looked familiar
My trip to China revealed an AI race that looks less like a clash of competing visions, and more like two countries racing toward a similar future.
Score: 32🌐 MovesAug 28, 2026https://restofworld.org/2026/china-us-ai-future/?utm_source=rss&utm_medium=rss&utm_campaign=feeds - Healthcare AI Workflows That Carry Cyber And Privacy Risks
Healthcare organizations must carefully evaluate data access, vendor relationships, security controls and oversight before deploying AI at scale
- Morphed Launches Agent That Turns One Prompt Into a Complete Creative Campaign
Morphed Launches Agent That Turns One Prompt Into a Complete Creative Campaign azcentral.com and The Arizona Republic
- Huawei He Tingbo Named to TIME’S Fourth Annual List of The TIME100 AI
Huawei He Tingbo Named to TIME’S Fourth Annual List of The TIME100 AI USA Today
- New AI tech considered to tackle wrong-way driving
AI technology and road cameras could be used to alert people more quickly to someone driving the wrong way down a motorway.
- Running a Solo Business? Here’s How to Get More Customers With AI
This simple weekly routine can help solo entrepreneurs build visibility in AI search.
Score: 31🌐 MovesAug 28, 2026https://www.inc.com/brian-honigman/running-a-solo-business-heres-how-to-get-more-customers-with-ai/91397010 - These Mammotion robot lawn mowers can climb hills - and they're up to $560 off right now
The Mammotion LUBA 3 AWD 3000H and the LUBA Mini 2 mowers are both heavily discounted - here's which you should get.
Score: 31🌐 MovesAug 28, 2026https://www.zdnet.com/article/mammotion-robot-mowers-deal-luba-3-awd-3000h-luba-mini-2-awd-1500h/ - Your People Are Learning From AI, But Is Your Business?
AI can make people faster, but it doesn't always make organizations smarter.
- Research by SMU Assistant Professor Thivya Kandappu shows how structured motor guidance and embodied prompts can help beginners bridge the gap between creative intent and expressive digital drawing.
Research by SMU Assistant Professor Thivya Kandappu shows how structured motor guidance and embodied prompts can help beginners bridge the gap between creative intent and expressive digital drawing. EurekAlert!
- How to Make AI Sound Like You
Tips from a CEO ghostwriter to make your writing sound more authentic
Score: 30🌐 MovesAug 28, 2026https://www.inc.com/sarah-mattina/how-to-make-ai-sound-like-you/91397016 - Beyond The Home Page: How Health Plans And Health Systems Can Harness AI-Mediated Discovery
As more consumers route care decisions through an AI assistant, fewer will visit websites directly.
- Trump Admin Curbing Chinese Remote Chip Access, 100+ Firms Want Better AI Cyber Defense Policy — TITV [Video]
Trump Admin Curbing Chinese Remote Chip Access, 100+ Firms Want Better AI Cyber Defense Policy — TITV [Video] The Information
- How to Use AI to Summarize Videos
Guide on leveraging AI to condense video content into concise summaries.
- Why Every Commercial-Lines Agency Needs AI Risk Fluency
Over the past year, artificial intelligence has quietly moved from experimentation to everyday business operations. Companies have formalized their AI use for everything from drafting client communications, automating workflows, analyzing financial data, supporting hiring decisions, and accelerating customer service. In …
- From 0 to 5 and Beyond: How AI Is Powering Cars24’s Next Phase of Growth
From 0 to 5 and Beyond: How AI Is Powering Cars24’s Next Phase of Growth india.entrepreneur.com
Score: 29🌐 MovesAug 28, 2026https://india.entrepreneur.com/technology/from-0-to-5-and-beyond-how-ai-is-powering-cars24s-next-phase-of-growth - Passport project to drive AI usage
The Digital Economy and Society (DES) Ministry is hopeful the TH-AI Passport project can drive the number of artificial intelligence (AI) users in Thailand to soar past the global average.
Score: 29🌐 MovesAug 28, 2026https://www.bangkokpost.com/business/general/3309910/passport-project-to-drive-ai-usage - 'Period of uncertainty' as AI enters Irish economy: Burke
There will be a "period of uncertainty" as artificial intelligence penetrates the Irish economy and expands across companies, Minister for Enterprise Peter Burke has said.
- From One Agent to a Team: Understanding Codex Subagents
A hands-on guide to defining specialist agents and coordinating their work in the Codex CLI The post From One Agent to a Team: Understanding Codex Subagents appeared first on Towards Data Science .
Score: 28🌐 MovesAug 28, 2026https://towardsdatascience.com/from-one-agent-to-a-team-understanding-codex-subagents/ - Cisco President Patel Makes the Case for AI in the Classroom
Cisco President and Chief Product Officer Jeetu Patel explains how artificial intelligence will benefit the education system on "Bloomberg The Close." (Source: Bloomberg)
Score: 28🌐 MovesAug 28, 2026https://www.bloomberg.com/news/videos/2026-08-28/cisco-s-patel-makes-the-case-for-ai-in-the-classroom-video - Google's Spirit Airlines auction & TrendAI insider threat report
Google's Spirit Airlines auction & TrendAI insider threat report IT Pro
Score: 27🌐 MovesAug 28, 2026https://www.itpro.com/technology/big-data/googles-spirit-airlines-auction-and-trendai-insider-threat-report - 'Unresponsiveness' and 'Unclear Goals' Dog Big Law Marketing Departments—Not to Mention the AI Risks
According to Law.com's legal marketing survey for 2026, several nagging issues — including unclear and under-resourced expectations — are creating tension between law firm management and marketing departments, often leading to burnout.
- LPGA star Charley Hull warns of 'AI-generated stuff' pretending to be her on social media
LPGA star Charley Hull warns of 'AI-generated stuff' pretending to be her on social media Golfweek
- UltraHumanizer Launches a Completely Free AI Humanizer With Built-In Text Refinement Features
UltraHumanizer Launches a Completely Free AI Humanizer With Built-In Text Refinement Features azcentral.com and The Arizona Republic
- Seeking romance with a companion chatbot
Seeking romance with a companion chatbot EurekAlert!
- How AI could uncover the hidden ways people work together
Yankai Wang, a Ph.D. student in organizational behavior at Stanford Graduate School of Business, was reading a paper about how transformers—the architecture behind large language models (LLMs) like ChatGPT and Claude—might be used to study disease development by tracing how one condition tends to follow another.
- AI Reviews From Our Experts
AI Reviews From Our Experts PCMag
- Ensuring AI is Acting as Intended: A QA
AI adoption in growing, and as it does, more organizations are starting to get a handle on controlling agents, making testing and security a first-class capability for AI-generated code, and using validation, policy and governance to ensure outcomes are what was required. SD Times recently sat down with Johnny Halife, the CTO at software engineering... … continue reading The post Ensuring AI is Acting as Intended: A Q&A appeared first on SD Times .
Score: 26🌐 MovesAug 28, 2026https://sdtimes.com/ai-code-generation/ensuring-ai-is-acting-as-intended-a-qa/ - Sandboxing and Blast Radius: Isolation Engineering for Agents
Part fourteen of a series on production agentic AI. Continue reading on Towards AI »
- Why Claude Code Time Estimates Are Poor
Be a better communicator with LLM programming The post Why Claude Code Time Estimates Are Poor appeared first on Towards Data Science .
- AI Is Turning Data Governance Into A Competitive Advantage
For that governance to be effective, it needs to be grounded in a complete understanding of your network infrastructure.
- If you thought Meta’s child safety scandal was bad, just wait for AI
If you thought Meta’s child safety scandal was bad, just wait for AI The Telegraph
Score: 25🌐 MovesAug 28, 2026https://www.telegraph.co.uk/business/2026/08/28/if-you-thought-metas-child-safety-scandal-was-bad-just-wait/ - Jackson Hole, Anthropic’s Court Win and PayPal Deal Collapse | Bloomberg Tech 8/28/2026
Bloomberg’s Tim Stenovec zeroes in on Jackson Hole after Federal Reserve Chairman Kevin Warsh delivered his first address at the Symposium. Plus, a US judge rules that the Trump administration must lift its ban on Anthropic's AI tech for federal agencies. And, a consortium of Advent and Stripe decided to abandon its pursuit of PayPal, a deal that would have ranked as one of the biggest-ever leveraged buyouts. (Source: Bloomberg)
Score: 25🌐 MovesAug 28, 2026https://www.bloomberg.com/news/videos/2026-08-28/bloomberg-tech-8-28-2026-video - AI ecommerce projects may fail without clean, centralised data: Report
The report said data readiness is one of the biggest factors determining whether AI projects move beyond experimentation and deliver measurable business outcomes. It noted that AI systems depend heavily on the quality of the information they receive, making data infrastructure a critical foundation for businesses looking to scale AI.
- Human-in-the-Loop Without Killing Throughput
How we stopped reviewing every agent action and started routing human attention where it actually mattered The post Human-in-the-Loop Without Killing Throughput appeared first on Towards Data Science .
Score: 25🌐 MovesAug 28, 2026https://towardsdatascience.com/human-in-the-loop-without-killing-throughput/ - Loop Engineering: The Anatomy of Reliable Agentic AI
Introduction In 2024, the tech world was obsessed with building autonomous agents. An engineer writes an instruction, starts an agent, reads the result, notices what went wrong, and writes the next instruction. That workflow is useful while exploring a problem. It does not scale when the same class of work arrives every day. The moment agent work repeats — CI failures, support tickets, stale pull requests, or monitoring alerts — you are no longer designing a single response. You are designing a control system. An agent can report “done” while the test is still red. It can retry the same failed action, consume its budget, and leave behind no durable explanation of what happened. In production, those are loop failures — not merely prompt failures. Why Prompt Engineering Isn’t Enough Most agent systems are still designed one prompt at a time. We add instructions like “think step-by-step,” or “double-check your work.” The following layers are complementary, but distinct: - Prompt engineering: What should the model do in this turn? - Context engineering: What should the model see? - Harness engineering: How should one run execute safely? - Loop engineering: How should repeated work move toward a verified outcome? Prompt engineering only optimizes reasoning. It does absolutely nothing for execution. A well-designed loop still needs good prompts and useful context, and a harness still needs to constrain each run. But loop engineering adds the outer control system that makes the work repeatable and prevents infinite, unverified spinning. The Shift from Conversations to Control Systems We are undergoing a massive architectural shift: moving from conversational interfaces to autonomous control systems. The distinction matters because more turns do not automatically mean more progress. An agent can repeat the same action, accumulate irrelevant context, consume its budget, and still report success without producing a verified result. When you deploy an autonomous agent, you are removing the human loop. You must replace it with a strict, engineered control loop that manages its state, enforces its boundaries, and validates its work. What Is Loop Engineering? Loop Engineering is the practice of designing repeatable agent workflows that discover work, hand an agent a bounded task, observe the result, verify it with evidence, persist what happened, and decide whether to continue, stop, or escalate. It decides when the agent should wake up, what work it should receive, what evidence counts as progress, where the result is recorded, and what happens when the work cannot be completed. The model is an important component of the loop, but it is not the whole system. The Anatomy of a Production Loop A reliable loop is a small state machine, even when the implementation uses an agent framework. One useful model is the seven-stage core loop: 1. Discover Identify new, in-scope work and suppress duplicates. A loop needs a defined source of work (a CI failure, a support ticket) with a stable identity so it doesn’t create a new task every time it sees the same unchanged failure. 2. Handoff Convert an event into a bounded task. Define the goal, evidence, permissions, acceptance criteria, working location, and stop rule. Without this contract, “fix the failure” becomes permission to redesign the system. 3. Execute Let the agent act inside an explicit boundary. Isolation is part of the loop contract — whether that’s a disposable worktree, a branch, a container, or a sandbox. The loop should make the boundary visible rather than relying on the model to respect it. 4. Observe Capture tool-produced evidence, not just the agent’s narration. Useful observations include test output, compiler errors, changed-file lists, API responses, and diff summaries. “The change looks good” is not an observation. 5. Verify Check the resulting artifact independently. The agent that made a change should not be the only component deciding that the change is correct. Use unit tests, type checking, or a second evaluator with a separate rubric. The verification result should be structured JSON, not a free-form paragraph. 6. Persist Make the next run start with state. Record the attempt, evidence, decisions, and unresolved risks. The model context is not a durable database. A loop should remember what affects the next decision, not accumulate an ever-growing transcript. 7. Decide Retry only with new evidence; otherwise stop, escalate, or close. “Keep trying until it works” is not a stopping rule. Retries should be bounded. Human intervention belongs in the design from the beginning. Case Study: Building a Reliable CI Failure Loop Consider a loop that helps maintain a software repository by fixing CI failures. The Naive Version: An agent receives a failing CI notification and is told to fix it. It searches broadly, edits several files, runs one test, sees a failure, and tries again. After a few turns it reports that the issue is fixed. A human discovers that the original check never passed on the final commit. The loop had no stable work identity, no change boundary, and no independent completion state. The Engineered Version: 1. Discover: A webhook records the repository, commit, workflow, and normalized failure signature. 2. Handoff: The agent receives the failure log, a disposable worktree, allowed commands, and a two-attempt budget. 3. Execute: The agent investigates and proposes a minimal patch, editing only allowed files. 4. Observe: The loop records the diff, test output, exit codes, and elapsed time. 5. Verify: A fresh test run executes against the resulting commit independently of the agent’s summary. 6. Persist: The work item stores the commit, evidence bundle, and attempt number. 7. Decide: If checks pass, it prepares a pull request. If they fail with actionable new evidence, it retries. If the budget is exhausted, it stops as needs_review. Common Failure Modes 1. The loop has no real trigger: If a person still has to restart the task every time, the system is an agent session, not a loop. 2. The loop confuses a claim with evidence: “Done” and “tested” are claims. They must be backed by current evidence. 3. The loop retries without changing the information state: Rewording the same instruction does not create progress. Detect repeated observations and escalate. 4. The evaluator shares the generator’s blind spot: A second model with the same context may repeat the first model’s mistake. Add deterministic checks. 5. The loop silently expands its scope: Enforce file, tool, and time boundaries. Practical Checklist Before automating a recurring agent workflow, answer these questions: - [ ] What exact source discovers the work, and how are duplicates identified? - [ ] What is the smallest bounded task the agent can receive? - [ ] Which files, tools, systems, and actions are in scope? - [ ] What external state survives the run? - [ ] What evidence proves progress and completion? - [ ] How many attempts, tokens, and minutes are allowed? - [ ] What happens when the loop is blocked or reaches a terminal state? Conclusion Loop engineering is not the replacement for prompt engineering, context engineering, or harness engineering. It is the discipline that connects them across repeated work. The goal is not an agent that never makes a mistake. The goal is a system that notices what happened, gathers new evidence, limits its authority, records its state, and knows when to stop. Design the trigger. Bound the handoff. Verify the artifact. Persist the evidence. Escalate before the risk becomes irreversible. That is how an agent workflow becomes an engineered loop. Read the original deep-dive on [ Loop Engineering ] on my engineering blog. If you are building autonomous systems and want to bridge the gap between AI research and production-grade software engineering, follow me here on Medium and connect on LinkedIn . Loop Engineering: The Anatomy of Reliable Agentic AI was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- All About AI
All About AI PCMag
- Migrate Pinecone to Qdrant: Complete Migration Guide | Zero Heart Burns
Okay so… you opened a bill from Pinecone and did a double-take. Or you tried to self-host it and discovered that’s just not a thing Pinecone lets you do. Or maybe you hit the 40KB metadata limit mid-sprint and your whole architecture needed a rethink. Whatever the reason might be you’re here, you want to move your vectors, and you want to do it without a disaster. I’ve been through this migration. Here’s everything I wish someone had told me before I started. Why You’re Even Reading This Here’s the thing about Pinecone… it’s genuinely easy to get started with. The docs are everywhere in AI tutorials, the API is clean, and you can have vectors in the cloud in like 20 minutes. That’s why so many teams start there. But then production hits… The cost curve surprises you. Pinecone’s serverless model charges per read unit, so every query costs money. At moderate QPS (say 500 queries per second), that meter is spinning fast. One team I talked to went from a comfortable $200/month in early testing to over $2k/month once their app got traction. The data doesn’t really get bigger, but the bills do. Vendor lock-in is real. Here’s a thing that quietly terrifies me: Pinecone has no export API. Like, none. You cannot say “give me all my vectors as a file.” If you want your data back, you have to iterate through IDs and fetch them back one batch at a time, and that only works on serverless indexes. Pod-based indexes? No list API at all. Pretty wild right? Namespace limits sneak up on you. Pinecone namespaces are a nice partitioning primitive but you’re capped at 100,000 namespaces on Standard. If you’re building a multi-tenant SaaS where each user has their own vector space… yeah, that can get uncomfortable fast. Metadata is flat and limited. 40KB per record, no nested JSON, no geo-coordinates, no full-text search on metadata fields. Need to store a document’s full text alongside its embedding? You’ll be doing a lot of creative truncation. No self-hosting, ever. This one is fundamental. Pinecone is SaaS-only. There is no Docker image, no Kubernetes operator, no “run it on your own machine” option. For teams with data residency requirements, regulated industries, or just a preference for not being dependent on external infrastructure… this is a hard blocker. The lightbulb moment for most developers is when they run Qdrant locally with docker run -p 6333:6333 qdrant/qdrant and realize it's the same thing - same API, same capabilities, same performance - but it's just... there, on your machine, completely under your control. Here’s the shape of this migration so you know what you’re getting into: Export your Pinecone data to a local JSONL file Spin up Qdrant locally or on Qdrant Cloud Load the data in and verify it If you’re in production: run dual-write while you migrate, then cut over with zero downtime Nothing here is magic. It’s just a bit of Python and some patience. The Mental Model Shift Before touching any code, let’s map the concepts. This is reference material to skim it now, come back when something breaks. Core Concept Mapping The namespace decision is actually important. You have two options: Separate collection per namespace: great if you have a small number of large namespaces and want clean isolation Single collection with a namespace payload field - better if you have many small namespaces, or want cross-namespace queries For most Pinecone migrations I’d go with option B (payload field), because collections have a recommended cap around 1,000 in Qdrant, and many Pinecone users have way more namespaces than that. Distance Metric Mapping # Pinecone metric values -> Qdrant Distance enum METRIC_MAP = { "cosine": Distance.COSINE, "euclidean": Distance.EUCLID, "dotproduct": Distance.DOT, } # Qdrant also has Distance.MANHATTAN - no Pinecone equivalent One gotcha here: Pinecone rescales cosine similarity to [0, 1]. Qdrant returns [-1, 1]. The rankings are identical, but any code that thresholds on raw score values will behave differently. Keep this in mind when you port your search logic. Filter Syntax from Before and After # === PINECONE === results = index.query( vector=[0.1] * 1536, filter={ "$and": [ {"genre": {"$eq": "sci-fi"}}, {"year": {"$gte": 2020}}, {"tags": {"$in": ["ai", "robots"]}} ] }, top_k=10, namespace="my-namespace", include_metadata=True ) matches = results["matches"] # list of {id, score, metadata} # === QDRANT === from qdrant_client.models import Filter, FieldCondition, MatchValue, MatchAny, Range results = client.search( collection_name="my_collection", query_vector=[0.1] * 1536, query_filter=Filter( must=[ FieldCondition(key="genre", match=MatchValue(value="sci-fi")), FieldCondition(key="year", range=Range(gte=2020)), FieldCondition(key="tags", match=MatchAny(any=["ai", "robots"])), # If you're using the payload-field namespace strategy: FieldCondition(key="_namespace", match=MatchValue(value="my-namespace")), ] ), limit=10, with_payload=True ) # results is a list of ScoredPoint objects with .id, .score, .payload The filter mapping rule of thumb: $and maps to must, $or maps to should, $ne maps to must_not. Numeric ranges map directly. $in maps to MatchAny. Honestly pretty intuitive once you see it side by side. Sparse + Dense Hybrid Pinecone hybrid search uses a single alpha parameter (0 = pure sparse, 1 = pure dense, linear blend). Qdrant uses explicit prefetch legs and rank fusion - either RRF (Reciprocal Rank Fusion) or DBSF. More code to write, but dramatically more control over how the fusion behaves. You can weight each leg independently, which ends up being pretty important for production tuning. Getting Your Data Out of Pinecone Here’s the honest truth: Pinecone has no export feature. No “Download All Data” button, no export endpoint, no snapshot you can take away. The only way out is to iterate through vector IDs and fetch them back in batches. The approach: Call list() to page through all vector IDs in a namespace (returns 100 IDs per page) For each page of IDs, call fetch() to get the actual vectors + metadata (up to 1,000 IDs per call) Write everything to a JSONL file One more thing before the code: this only works on serverless indexes . Pod-based indexes have no list API at all. If you're on a pod-based index... the only real option is to re-embed from your original document source, because there's no supported way to bulk-export from a pod index. Yikes. Rate limits to know: list is 200 req/s, fetch is 100 req/s. With 1,000 IDs per fetch call, that's theoretically 100,000 vectors per second. In practice, you'll be slower because of network and parsing overhead - but you won't hit limits if you stay around 10 concurrent fetch calls. Here’s a complete, resumable Pinecone exporter: #!/usr/bin/env python3 # VIBE CODED for smooth and quick setups """ pinecone_dumper.py Exports all vectors from a Pinecone serverless index to a JSONL file. Resumable: if interrupted, just rerun - it will pick up from where it left off. Requirements: pip install pinecone>=3.0.0 tqdm Usage: python pinecone_dumper.py \ --api-key pcsk_XXX \ --index-name my-index \ --output vectors.jsonl """ import argparse import json import os import time import math import pickle from pathlib import Path from pinecone import Pinecone from tqdm import tqdm LIST_BATCH_SIZE = 100 # Pinecone max IDs per list page FETCH_BATCH_SIZE = 1000 # Pinecone max IDs per fetch call FETCH_RATE_SLEEP = 0.011 # ~90 req/s, safely under 100 req/s limit def load_checkpoint(ckpt_path: str) -> dict: if Path(ckpt_path).exists(): with open(ckpt_path, "rb") as f: return pickle.load(f) return {} def save_checkpoint(ckpt_path: str, data: dict): tmp = ckpt_path + ".tmp" with open(tmp, "wb") as f: pickle.dump(data, f) os.replace(tmp, ckpt_path) def list_all_ids(index, namespace: str, checkpoint: dict, ckpt_path: str) -> list: ckpt_key = f"ids:{namespace}" if ckpt_key in checkpoint: print(f" Resuming ID listing for namespace='{namespace}' from checkpoint...") return checkpoint[ckpt_key] all_ids = [] cursor = None page_num = 0 print(f" Listing IDs for namespace='{namespace}'...") while True: kwargs = {"limit": LIST_BATCH_SIZE, "namespace": namespace} if cursor: kwargs["pagination_token"] = cursor try: resp = index.list_paginated(**kwargs) except Exception as e: print(f" ERROR listing page {page_num}: {e}. Retrying in 5s...") time.sleep(5) continue page_ids = [v.id for v in (resp.vectors or [])] all_ids.extend(page_ids) page_num += 1 if page_num % 100 == 0: print(f" ... {len(all_ids):,} IDs listed so far") cursor = resp.pagination.next if resp.pagination else None if not cursor: break print(f" Found {len(all_ids):,} IDs in namespace='{namespace}'") checkpoint[ckpt_key] = all_ids save_checkpoint(ckpt_path, checkpoint) return all_ids def fetch_and_write(index, all_ids, namespace, output_file, checkpoint, ckpt_path): ckpt_key = f"fetched_batches:{namespace}" completed_batches = checkpoint.get(ckpt_key, set()) total_batches = math.ceil(len(all_ids) / FETCH_BATCH_SIZE) new_writes = 0 with tqdm(total=len(all_ids), desc=f" Fetching vectors (ns='{namespace}')") as pbar: for batch_idx in range(total_batches): start = batch_idx * FETCH_BATCH_SIZE batch_ids = all_ids[start:start + FETCH_BATCH_SIZE] if batch_idx in completed_batches: pbar.update(len(batch_ids)) continue # Fetch with exponential backoff retry for attempt in range(5): try: resp = index.fetch(ids=batch_ids, namespace=namespace) break except Exception as e: wait = 2 ** attempt print(f"\n Fetch attempt {attempt+1} failed: {e}. Waiting {wait}s...") time.sleep(wait) else: print(f"\n ERROR: Could not fetch batch {batch_idx}. Skipping.") pbar.update(len(batch_ids)) continue vectors_map = resp.get("vectors", {}) for pid, pdata in vectors_map.items(): record = { "id": pid, "values": pdata.get("values", []), "metadata": pdata.get("metadata", {}), "_namespace": namespace, } sv = pdata.get("sparse_values") or pdata.get("sparseValues") if sv: record["sparse_values"] = { "indices": sv.get("indices", []), "values": sv.get("values", []), } output_file.write(json.dumps(record) + "\n") new_writes += 1 completed_batches.add(batch_idx) if batch_idx % 10 == 0: checkpoint[ckpt_key] = completed_batches save_checkpoint(ckpt_path, checkpoint) pbar.update(len(batch_ids)) time.sleep(FETCH_RATE_SLEEP) checkpoint[ckpt_key] = completed_batches save_checkpoint(ckpt_path, checkpoint) return new_writes def main(): parser = argparse.ArgumentParser(description="Export Pinecone index to JSONL") parser.add_argument("--api-key", required=True) parser.add_argument("--index-name", required=True) parser.add_argument("--index-host", default=None, help="Index host URL (optional, faster)") parser.add_argument("--namespace", default=None, help="Export one namespace only") parser.add_argument("--output", default="pinecone_export.jsonl") parser.add_argument("--checkpoint", default="pinecone_dump_checkpoint.pkl") parser.add_argument("--fresh", action="store_true", help="Ignore existing checkpoint") args = parser.parse_args() pc = Pinecone(api_key=args.api_key) index = pc.Index(host=args.index_host) if args.index_host else pc.Index(args.index_name) stats = index.describe_index_stats() namespaces = list(stats.get("namespaces", {}).keys()) if not namespaces: namespaces = [""] # default namespace if args.namespace: namespaces = [args.namespace] total_vectors = stats.get("total_vector_count", 0) dimension = stats.get("dimension", "unknown") print(f"\nIndex: {args.index_name}") print(f" Dimension: {dimension}") print(f" Total vectors: {total_vectors:,}") print(f" Namespaces: {namespaces}") print(f" Output: {args.output}\n") checkpoint = {} if args.fresh else load_checkpoint(args.checkpoint) total_written = 0 with open(args.output, "a", encoding="utf-8") as out_f: for ns in namespaces: print(f"Exporting namespace='{ns}'...") ids = list_all_ids(index, ns, checkpoint, args.checkpoint) written = fetch_and_write(index, ids, ns, out_f, checkpoint, args.checkpoint) total_written += written print(f" Done. {written:,} vectors written.\n") print(f"Export complete! Total: {total_written:,} vectors -> {args.output}") print(f"\nNext step: python qdrant_uploader.py --input {args.output}") if __name__ == "__main__": main() Run it like this: pip install "pinecone>=3.0.0" tqdm python pinecone_dumper.py \ --api-key pcsk_YOUR_KEY \ --index-name my-index \ --output my_vectors.jsonl If it gets interrupted mid-run, just run it again …. it’ll skip everything it already fetched. The checkpoint file tracks progress at the batch level so you don’t re-fetch things you already have. For very large indexes (100M+ vectors), this will take a while. The math: at 1,000 vectors per fetch and ~90 requests per second, you’re looking at roughly 90,000 vectors per second throughput. 100M vectors = ~18 minutes of pure fetch time, plus listing time on top. Run it overnight. Loading Into Qdrant (Local + Cloud) Okay, you’ve got your my_vectors.jsonl file. Now let's get it into Qdrant. Step 1: Start Qdrant locally docker run -d --name qdrant \ -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage \ qdrant/qdrant The web UI is at http://localhost:6333/dashboard. It's actually quite nice - you can browse collections, run test queries, inspect points. Better than nothing! Step 2: Recreate your collection You need the dimension and metric from your Pinecone index. The describe_index_stats() call doesn't return the metric (annoyingly) - check your index creation code or the Pinecone console. from qdrant_client import QdrantClient from qdrant_client.models import ( Distance, VectorParams, SparseVectorParams, HnswConfigDiff, OptimizersConfigDiff ) client = QdrantClient(url="http://localhost:6333") # For Qdrant Cloud: # client = QdrantClient( # url="https://YOUR-CLUSTER.cloud.qdrant.io", # api_key="YOUR_QDRANT_API_KEY" # ) COLLECTION = "my_collection" DIMENSION = 1536 # match your Pinecone index dimension METRIC = "cosine" METRIC_MAP = { "cosine": Distance.COSINE, "euclidean": Distance.EUCLID, "dotproduct": Distance.DOT, } # Disable HNSW during bulk load - re-enable after (3-5x faster ingest) client.create_collection( collection_name=COLLECTION, vectors_config=VectorParams(size=DIMENSION, distance=METRIC_MAP[METRIC]), optimizers_config=OptimizersConfigDiff(indexing_threshold=0), ) print(f"Created collection '{COLLECTION}'") Notice indexing_threshold=0 - that disables HNSW index building during the bulk load. We'll re-enable it after. This makes loading 3-5x faster because Qdrant isn't trying to build the graph while you're flooding it with vectors. Step 3: The uploader script #!/usr/bin/env python3 # Once again, have vibe coded this part to perfection! """ qdrant_uploader.py Reads a JSONL file from pinecone_dumper.py and bulk-upserts into Qdrant. Handles ID translation, batching, retries, and progress tracking. Requirements: pip install "qdrant-client>=1.9.0" tqdm Usage: python qdrant_uploader.py \ --input vectors.jsonl \ --collection my_collection \ --qdrant-url http://localhost:6333 """ import argparse import json import hashlib import time from pathlib import Path from qdrant_client import QdrantClient from qdrant_client.models import ( PointStruct, SparseVector, Distance, VectorParams, OptimizersConfigDiff, UpdateStatus ) from tqdm import tqdm UPSERT_BATCH_SIZE = 512 # 256-512 is the sweet spot MAX_RETRIES = 5 def pinecone_id_to_qdrant(pinecone_id: str) -> str: """ Convert a Pinecone string ID to a deterministic UUID for Qdrant. We store the original ID in payload as _pinecone_id for reverse lookup. """ h = hashlib.sha256(pinecone_id.encode()).hexdigest() return f"{h[:8]}-{h[8:12]}-{h[12:16]}-{h[16:20]}-{h[20:32]}" def count_lines(filepath: str) -> int: count = 0 with open(filepath, "rb") as f: for _ in f: count += 1 return count def upsert_batch_with_retry(client, collection, points, max_retries=MAX_RETRIES): for attempt in range(max_retries): try: result = client.upsert( collection_name=collection, points=points, wait=True, ) return result.status == UpdateStatus.COMPLETED except Exception as e: wait = 2 ** attempt print(f"\n Upsert attempt {attempt+1} failed: {e}. Waiting {wait}s...") time.sleep(wait) print(f"\n ERROR: Batch failed after {max_retries} retries. Skipping {len(points)} points.") return False def build_point(record: dict, has_sparse: bool) -> PointStruct: pinecone_id = record["id"] qdrant_id = pinecone_id_to_qdrant(pinecone_id) values = record.get("values", []) payload = dict(record.get("metadata", {})) payload["_pinecone_id"] = pinecone_id payload["_namespace"] = record.get("_namespace", "") if has_sparse and "sparse_values" in record: sv = record["sparse_values"] vector = { "dense": values, "sparse": SparseVector( indices=sv.get("indices", []), values=sv.get("values", []) ) } else: vector = values return PointStruct(id=qdrant_id, vector=vector, payload=payload) def main(): parser = argparse.ArgumentParser(description="Upload JSONL to Qdrant") parser.add_argument("--input", required=True) parser.add_argument("--collection", required=True) parser.add_argument("--qdrant-url", default="http://localhost:6333") parser.add_argument("--api-key", default=None, help="Qdrant API key (for cloud)") parser.add_argument("--has-sparse", action="store_true") parser.add_argument("--batch-size", type=int, default=UPSERT_BATCH_SIZE) args = parser.parse_args() client = QdrantClient(url=args.qdrant_url, api_key=args.api_key) try: info = client.get_collection(args.collection) print(f"Collection '{args.collection}' found - {info.vectors_count or 0:,} existing vectors") except Exception: print(f"ERROR: Collection '{args.collection}' not found. Create it first.") return total_lines = count_lines(args.input) print(f"\nInput file: {args.input} ({total_lines:,} records)") print(f"Batch size: {args.batch_size}\n") batch = [] total_uploaded = 0 total_failed = 0 with open(args.input, "r", encoding="utf-8") as f, \ tqdm(total=total_lines, desc="Uploading", unit="vec") as pbar: for line in f: line = line.strip() if not line: continue try: record = json.loads(line) except json.JSONDecodeError as e: print(f"\n Bad JSON line: {e}. Skipping.") pbar.update(1) continue batch.append(build_point(record, args.has_sparse)) if len(batch) >= args.batch_size: ok = upsert_batch_with_retry(client, args.collection, batch) if ok: total_uploaded += len(batch) else: total_failed += len(batch) pbar.update(len(batch)) batch = [] if batch: ok = upsert_batch_with_retry(client, args.collection, batch) total_uploaded += len(batch) if ok else 0 total_failed += len(batch) if not ok else 0 pbar.update(len(batch)) print(f"\nUpload complete! Uploaded: {total_uploaded:,} Failed: {total_failed:,}") # Re-enable HNSW indexing now that bulk load is done print("\nRe-enabling HNSW indexing...") client.update_collection( collection_name=args.collection, optimizers_config=OptimizersConfigDiff(indexing_threshold=20_000), ) print("Waiting for index to build (this may take a few minutes)...") while True: info = client.get_collection(args.collection) status = str(info.status).lower() if "green" in status: break print(f" Status: {status}... waiting 15s") time.sleep(15) final_count = client.get_collection(args.collection).vectors_count print(f"\nCollection '{args.collection}' is ready! Vector count: {final_count:,}") if __name__ == "__main__": main() Run the full pipeline: pip install "qdrant-client>=1.9.0" tqdm # 1. Export from Pinecone python pinecone_dumper.py --api-key pcsk_XXX --index-name my-index --output vectors.jsonl # 2. Upload to local Qdrant python qdrant_uploader.py \ --input vectors.jsonl \ --collection my_collection \ --qdrant-url http://localhost:6333 For Qdrant Cloud : sign up here , create a cluster (the free tier gets you 1M vectors permanently), then use: python qdrant_uploader.py \ --input vectors.jsonl \ --collection my_collection \ --qdrant-url https://YOUR-CLUSTER.cloud.qdrant.io \ --api-key YOUR_QDRANT_KEY Step 4: Verify it worked from qdrant_client import QdrantClient client = QdrantClient(url="http://localhost:6333") # Exact count (Qdrant's vectors_count is approximate for large collections) def exact_count(client, collection): count, offset = 0, None while True: recs, next_offset = client.scroll( collection_name=collection, limit=1000, offset=offset, with_payload=False, with_vectors=False ) count += len(recs) offset = next_offset if offset is None: break return count n = exact_count(client, "my_collection") print(f"Qdrant has {n:,} vectors") # Spot-check: search and print original Pinecone IDs from payload test_vec = [0.1] * 1536 # replace with a real vector results = client.search("my_collection", query_vector=test_vec, limit=5, with_payload=True) for r in results: print(f" score={r.score:.4f} original_pinecone_id={r.payload.get('_pinecone_id')}") One more option: Qdrant’s official migration tool Qdrant ships a Docker-based migration CLI that handles Pinecone directly. For serverless indexes, one command does the whole thing: docker run --rm -it registry.cloud.qdrant.io/library/qdrant-migration pinecone \ --pinecone.index-host 'https://your-index.svc.pinecone.io' \ --pinecone.index-name 'your-index' \ --pinecone.api-key 'pcsk_...' \ --qdrant.url 'https://your-cluster.cloud.qdrant.io:6334' \ --qdrant.api-key 'your-qdrant-key' \ --qdrant.collection 'your-collection' \ --migration.batch-size 64 Note the gRPC port (6334): the migration tool uses gRPC, not the REST API. And it’s resumable via an internal _migration_offsets collection it creates on the target. If it gets killed, just rerun and it picks up from the last offset. Zero Downtime Migration (The Production Way) If you’re running a live production service on Pinecone, you can’t just turn it off run the migration and turn it back on. Here’s how to do it with zero downtime. The core idea: run both databases simultaneously. Pinecone stays primary for reads. Qdrant gets all the writes. Once Qdrant has all the historical data and has proven itself with real traffic, you flip the switch. The Dual-Write Wrapper import threading from concurrent.futures import ThreadPoolExecutor from queue import Queue, Empty class DualWriteClient: """ Wrap your existing Pinecone index to simultaneously write to Qdrant. Pinecone is primary (writes block on Pinecone success). Qdrant writes are async and non-blocking - failures queue for retry. """ def __init__(self, pinecone_idx, qdrant_client, qdrant_collection): self.pine = pinecone_idx self.qdrant = qdrant_client self.col = qdrant_collection self.pool = ThreadPoolExecutor(max_workers=4, thread_name_prefix="qdrant-write") self.retry_q = Queue(maxsize=10_000) self._start_retry_worker() def upsert(self, vectors, namespace=""): # Step 1: Write to Pinecone (primary, blocks until confirmed) self.pine.upsert(vectors=vectors, namespace=namespace) # Step 2: Write to Qdrant async (never blocks your app) self.pool.submit(self._qdrant_upsert, vectors, namespace) def _qdrant_upsert(self, vectors, namespace): from qdrant_client.models import PointStruct points = [ PointStruct( id=pinecone_id_to_qdrant(v["id"]), vector=v["values"], payload={**v.get("metadata", {}), "_pinecone_id": v["id"], "_namespace": namespace} ) for v in vectors ] try: self.qdrant.upsert(collection_name=self.col, points=points, wait=True) except Exception as e: print(f"Qdrant write failed, queuing retry: {e}") try: self.retry_q.put_nowait(("upsert", points)) except Exception: print(f"Retry queue full - {len(points)} points dropped") def delete(self, ids, namespace=""): self.pine.delete(ids=ids, namespace=namespace) qdrant_ids = [pinecone_id_to_qdrant(i) for i in ids] self.pool.submit(self._qdrant_delete, qdrant_ids) def _qdrant_delete(self, ids): try: self.qdrant.delete(collection_name=self.col, points_selector=ids, wait=True) except Exception as e: try: self.retry_q.put_nowait(("delete", ids)) except Exception: pass def _start_retry_worker(self): t = threading.Thread(target=self._retry_loop, daemon=True) t.start() def _retry_loop(self): while True: try: op, payload = self.retry_q.get(timeout=5) except Empty: continue for attempt in range(5): try: if op == "upsert": self.qdrant.upsert(collection_name=self.col, points=payload, wait=True) else: self.qdrant.delete(collection_name=self.col, points_selector=payload, wait=True) break except Exception: time.sleep(2 ** attempt) # Usage: replace your existing pinecone_index with this one line # Before: results = pinecone_index.query(...) # After: results = dual_client.pine.query(...) <- reads still go to Pinecone # # dual_client = DualWriteClient(pc.Index("my-index"), qdrant_client, "my_collection") # dual_client.upsert(vectors=[...]) <- writes go to both The Migration Timeline Once dual-write is running, here’s the sequence: Start dual-write : your app now writes to both. Reads still come from Pinecone. Run the backfill: use the dumper + uploader scripts from Sections 3–4 to migrate historical data. Dual-write handles anything new that comes in while the backfill runs. Shadow mode: for a day or two, run every search query against Qdrant too (don’t show results to users yet), compare rankings, log quality differences. Cutover: once you’re happy with quality parity, flip the read traffic. Cutover Checklist Before you switch traffic to Qdrant: JUST VERI FY the following okay? [x] Dual-write has been running >= 48 hours without errors [x] Backfill complete - Qdrant vector count within 0.1% of Pinecone count [x] Shadow comparison: mean Recall@10 >= 0.95 over at least 10,000 queries [x] Qdrant collection status = green (fully indexed) [x] P99 search latency on Qdrant within 15% of Pinecone [x] Rollback plan tested - know how to re-enable Pinecone reads in under 5 minutes The Atomic Cutover (Qdrant Aliases) Qdrant has collection aliases so that a single API call switches an alias from one collection to another with zero gap. Use this for the cutover: from qdrant_client.models import CreateAliasOperation, DeleteAliasOperation # One API call - no window where the alias is undefined qdrant_client.update_collection_aliases( change_aliases_operations=[ DeleteAliasOperation(delete_alias={"alias_name": "production"}), CreateAliasOperation(create_alias={ "collection_name": "my_collection", "alias_name": "production" }), ] ) # From this point: all reads going to "production" alias hit Qdrant Rollback Plan Keep dual-write running for at least 48 hours after full cutover. If something goes wrong, rollback is literally just updating a feature flag or load balancer weight to send reads back to Pinecone. Since dual-write is still active, Pinecone stays in sync during that window. Rolling back is instant and safe. Why Qdrant Cloud Hits Different Okay… so I’ve been pretty technical so far. Let me just talk about what actually feels different when you’re using Qdrant versus Pinecone day-to-day. The free tier is actually useful. Qdrant Cloud’s free tier is permanent and not a trial, and supports roughly 1M vectors on a real cluster (0.5 vCPU, 1 GB RAM, 4 GB disk)!!!!!! Pinecone’s free tier is 100,000 vectors on a single serverless index with the per-read-unit billing lurking. Qdrant’s free tier is just… a free cluster. No gotchas, no time limit. NOTABLE MENTIONS FROM MY RESEARCH AND DUMP BELOW! You can actually run it yourself. docker run -p 6333:6333 qdrant/qdrant. That's it. The self-hosted version and the cloud version run the same binary. No features are gated behind cloud-only access. The API is identical everywhere. You can move between local development, your own Kubernetes cluster, and Qdrant Cloud at any time. Your data is portable. Your skills transfer. Pinecone is SaaS-only, period. Quantization is a superpower. Pinecone stores everything as float32. That’s 4 bytes per dimension, no exceptions. For a 10M vector / 1536-dim collection, that’s ~61 GB of raw vector data. With Qdrant’s Turbo4 4-bit datatype (from v1.19), you get that down to ~7.6 GB while maintaining solid recall. With binary quantization you’re under 2 GB. This is roughly a 30x memory reduction. At $5/GB/month in cloud RAM costs, that’s a very large number. Quick cost comparison for 10M vectors at 1536 dims Filterable HNSW is a real algorithmic difference. Pinecone applies filters as pre-filter or post-filter. If your filter is very selective (say, filter to 1% of vectors), post-filtering wastes a huge amount of graph traversal work, and pre-filtering with very small candidate sets loses recall. Qdrant integrates payload filtering directly into the HNSW graph traversal — it navigates the graph while respecting the filter in real-time. This gives predictable recall even on highly selective filters. It’s not a marketing claim — it’s a fundamentally different query execution model. Multi-vector per point. Each Qdrant point can have multiple named vectors. One point can carry a text embedding (768-dim), an image embedding (512-dim), and a sparse BM25 vector, all stored together with their payload. A single hybrid search can fuse all three with RRF. Pinecone gives you one vector per record. Full stop. This matters a lot for multimodal and hybrid retrieval workloads. Data portability is the anti-vendor-lock-in story. With Qdrant, you can snapshot any collection and download it: # Create a snapshot snapshot_info = client.create_snapshot(collection_name="my_collection") print(f"Snapshot: {snapshot_info.name}") # Download it (it's just an HTTP GET) # GET /collections/my_collection/snapshots/{snapshot_name} That snapshot is your data. You can restore it to any Qdrant instance anywhere. If Qdrant doubles their prices tomorrow, you have a complete copy of your data ready to move. With Pinecone, you do the LIST+FETCH dance described in Section 3. If Pinecone has an outage and you can’t list your IDs… well. Open source, for real. Qdrant has 34,000+ GitHub stars, Apache 2.0 license, and the team actively responds to issues. You can read the source code, understand exactly what’s happening with your data, file bugs, send PRs. When something weird happens with your search results, you can actually dig in and find out why. Pinecone is a black box. That distinction sounds abstract until something goes wrong. The Qdrant documentation covers all of this in depth — quantization options, memory tier configs, distributed deployment, the lot. And if you want to follow what’s coming next (new quantization research, benchmark results, engineering deep-dives), the newsletter is worth subscribing to. They publish real engineering content, not just product announcements. Alright now! that’s the full picture. The migration is doable okay and most teams do it over a week or two with zero downtime using the dual-write approach. The scripts handle the heavy lifting. The main gotchas to keep in mind: Pinecone has no export API: you’re doing LIST+FETCH, and that only works on serverless indexes Pod-based indexes need to be re-embedded from your original document source Cosine scores have different ranges ([0,1] in Pinecone, [-1,1] in Qdrant) - watch any code that thresholds on raw scores Keep dual-write running for 48 hours post-cutover before you decommission anything Good luck with the migration. You’ve got this. Migrate Pinecone to Qdrant: Complete Migration Guide | Zero Heart Burns was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Look At The Time Magazine Top 100 In AI
TIME’s AI 100 highlights leaders advancing models, hardware, robotics, efficiency, entertainment, and global AI innovation.
Score: 22🌐 MovesAug 28, 2026https://www.forbes.com/sites/johnwerner/2026/08/28/look-at-the-time-magazine-top-100-in-ai/ - Building the Foundation for the Agentic AI Era
How do we build an AI ecosystem where agents, tools, and systems can work together at scale? Angie Jones, VP of the Agentic AI Foundation, joins Chris to discuss the open standards and projects shaping the agentic future, including MCP, A2A, Goose, etc. They also explore what it takes to drive AI adoption across an entire organization, the importance of neutral standards, global perspectives on agentic AI, and how humans can find the right balance between what they delegate to AI and what they do themselves. Featuring: Angie Jones – LinkedIn , X Chris Benson – Website , LinkedIn , Bluesky , GitHub , X Links: Agentic AI Foundation Sponsors: Framer: The enterprise-grade website builder that lets your team ship faster. Get 30% off at framer.com/practicalai Prediction Guard: A self-hosted AI control plane for running agents in high impact environments. predictionguard.com/practicalai Resources and Events: Register for upcoming webinars here ! Prior Webinars from our partner Prediction Guard Midwest AI Summit 2026
- On AI memory: from a closed-door discussion
AI memory is moving beyond individual interactions. The next challenge may be deciding who controls the accumulated context of a user's digital life. The post On AI memory: from a closed-door discussion appeared first on MEDIANAMA .
Score: 22🌐 MovesAug 28, 2026https://www.medianama.com/2026/08/223-ai-memory-privacy-data-portability/ - Connecting My LangGraph AI Agent to Postgres
How to run the backend locally with Docker or in the cloud The post Connecting My LangGraph AI Agent to Postgres appeared first on Towards Data Science .
Score: 22🌐 MovesAug 28, 2026https://towardsdatascience.com/connecting-my-langgraph-ai-agent-to-postgres/