AI News Archive: August 28, 2026 — Part 7
Sourced from 500+ daily AI sources, scored by relevance.
- Interpretable Text Classification: Probing Scikit-LLM Embedding Spaces
In this article, you will learn how to use probing classifiers, UMAP visualization, and SHAP values to interpret and analyze the quality of text embeddings...
Score: 34🌐 MovesAug 28, 2026https://machinelearningmastery.com/interpretable-text-classification-probing-scikit-llm-embedding-spaces/ - What’s Fake, What’s Real, What’s AI, and How to Spot the Difference
The easiest way to never get fooled is to know what the truth is.
Score: 33🌐 MovesAug 28, 2026https://www.inc.com/joe-procopio/whats-fake-whats-real-whats-ai-and-how-to-spot-the-difference/91397313 - How to get free Google AI Pro for an entire year - and save $240: 3 ways
Google is offering its premium AI subscription plan for free. It usually costs $20 a month - and comes with many benefits.
Score: 33🌐 MovesAug 28, 2026https://www.zdnet.com/article/how-to-get-free-google-ai-pro-subscription-for-one-year/ - Top tech analyst says people are buying more things on Amazon because of AI. He raised his price outlook as a result
Amazon.com could rally as wider AI adoption drives up sales on its online shopping platform, per Evercore ISI.
Score: 33🌐 MovesAug 28, 2026https://www.cnbc.com/2026/08/28/amazon-sees-more-sales-thanks-to-ai-shares-will-rise-evercore-isi-says.html - Half of AI Products Waited Two Years for Their First Public Mention
Half of AI Products Waited Two Years for Their First Public Mention azcentral.com and The Arizona Republic
- I went to China to see a different AI future. It looked familiar
My trip to China revealed an AI race that looks less like a clash of competing visions, and more like two countries racing toward a similar future.
Score: 32🌐 MovesAug 28, 2026https://restofworld.org/2026/china-us-ai-future/?utm_source=rss&utm_medium=rss&utm_campaign=feeds - Healthcare AI Workflows That Carry Cyber And Privacy Risks
Healthcare organizations must carefully evaluate data access, vendor relationships, security controls and oversight before deploying AI at scale
- NetworkNews Audio Announces Audio Press Release (APR) Discussing Reframing GPU Model, Generating Ongoing Usage-Based Revenue
NetworkNews Audio Announces Audio Press Release (APR) Discussing Reframing GPU Model, Generating Ongoing Usage-Based Revenue Toronto Star
- Running a Solo Business? Here’s How to Get More Customers With AI
This simple weekly routine can help solo entrepreneurs build visibility in AI search.
Score: 31🌐 MovesAug 28, 2026https://www.inc.com/brian-honigman/running-a-solo-business-heres-how-to-get-more-customers-with-ai/91397010 - Research by SMU Assistant Professor Thivya Kandappu shows how structured motor guidance and embodied prompts can help beginners bridge the gap between creative intent and expressive digital drawing.
Research by SMU Assistant Professor Thivya Kandappu shows how structured motor guidance and embodied prompts can help beginners bridge the gap between creative intent and expressive digital drawing. EurekAlert!
- Huawei He Tingbo Named to TIME’S Fourth Annual List of The TIME100 AI
Huawei He Tingbo Named to TIME’S Fourth Annual List of The TIME100 AI USA Today
- New AI tech considered to tackle wrong-way driving
AI technology and road cameras could be used to alert people more quickly to someone driving the wrong way down a motorway.
- These Mammotion robot lawn mowers can climb hills - and they're up to $560 off right now
The Mammotion LUBA 3 AWD 3000H and the LUBA Mini 2 mowers are both heavily discounted - here's which you should get.
Score: 31🌐 MovesAug 28, 2026https://www.zdnet.com/article/mammotion-robot-mowers-deal-luba-3-awd-3000h-luba-mini-2-awd-1500h/ - Your People Are Learning From AI, But Is Your Business?
AI can make people faster, but it doesn't always make organizations smarter.
- Morphed Launches Agent That Turns One Prompt Into a Complete Creative Campaign
Morphed Launches Agent That Turns One Prompt Into a Complete Creative Campaign azcentral.com and The Arizona Republic
- How to Make AI Sound Like You
Tips from a CEO ghostwriter to make your writing sound more authentic
Score: 30🌐 MovesAug 28, 2026https://www.inc.com/sarah-mattina/how-to-make-ai-sound-like-you/91397016 - Trump Admin Curbing Chinese Remote Chip Access, 100+ Firms Want Better AI Cyber Defense Policy — TITV [Video]
Trump Admin Curbing Chinese Remote Chip Access, 100+ Firms Want Better AI Cyber Defense Policy — TITV [Video] The Information
- Beyond The Home Page: How Health Plans And Health Systems Can Harness AI-Mediated Discovery
As more consumers route care decisions through an AI assistant, fewer will visit websites directly.
- How to Use AI to Summarize Videos
Guide on leveraging AI to condense video content into concise summaries.
- From 0 to 5 and Beyond: How AI Is Powering Cars24’s Next Phase of Growth
From 0 to 5 and Beyond: How AI Is Powering Cars24’s Next Phase of Growth india.entrepreneur.com
Score: 29🌐 MovesAug 28, 2026https://india.entrepreneur.com/technology/from-0-to-5-and-beyond-how-ai-is-powering-cars24s-next-phase-of-growth - Passport project to drive AI usage
The Digital Economy and Society (DES) Ministry is hopeful the TH-AI Passport project can drive the number of artificial intelligence (AI) users in Thailand to soar past the global average.
Score: 29🌐 MovesAug 28, 2026https://www.bangkokpost.com/business/general/3309910/passport-project-to-drive-ai-usage - Why Every Commercial-Lines Agency Needs AI Risk Fluency
Over the past year, artificial intelligence has quietly moved from experimentation to everyday business operations. Companies have formalized their AI use for everything from drafting client communications, automating workflows, analyzing financial data, supporting hiring decisions, and accelerating customer service. In …
- 'Period of uncertainty' as AI enters Irish economy: Burke
There will be a "period of uncertainty" as artificial intelligence penetrates the Irish economy and expands across companies, Minister for Enterprise Peter Burke has said.
- Cisco President Patel Makes the Case for AI in the Classroom
Cisco President and Chief Product Officer Jeetu Patel explains how artificial intelligence will benefit the education system on "Bloomberg The Close." (Source: Bloomberg)
Score: 28🌐 MovesAug 28, 2026https://www.bloomberg.com/news/videos/2026-08-28/cisco-s-patel-makes-the-case-for-ai-in-the-classroom-video - From One Agent to a Team: Understanding Codex Subagents
A hands-on guide to defining specialist agents and coordinating their work in the Codex CLI The post From One Agent to a Team: Understanding Codex Subagents appeared first on Towards Data Science .
Score: 28🌐 MovesAug 28, 2026https://towardsdatascience.com/from-one-agent-to-a-team-understanding-codex-subagents/ - LPGA star Charley Hull warns of 'AI-generated stuff' pretending to be her on social media
LPGA star Charley Hull warns of 'AI-generated stuff' pretending to be her on social media Golfweek
- 'Unresponsiveness' and 'Unclear Goals' Dog Big Law Marketing Departments—Not to Mention the AI Risks
According to Law.com's legal marketing survey for 2026, several nagging issues — including unclear and under-resourced expectations — are creating tension between law firm management and marketing departments, often leading to burnout.
- Google's Spirit Airlines auction & TrendAI insider threat report
Google's Spirit Airlines auction & TrendAI insider threat report IT Pro
Score: 27🌐 MovesAug 28, 2026https://www.itpro.com/technology/big-data/googles-spirit-airlines-auction-and-trendai-insider-threat-report - UltraHumanizer Launches a Completely Free AI Humanizer With Built-In Text Refinement Features
UltraHumanizer Launches a Completely Free AI Humanizer With Built-In Text Refinement Features azcentral.com and The Arizona Republic
- Seeking romance with a companion chatbot
Seeking romance with a companion chatbot EurekAlert!
- How AI could uncover the hidden ways people work together
Yankai Wang, a Ph.D. student in organizational behavior at Stanford Graduate School of Business, was reading a paper about how transformers—the architecture behind large language models (LLMs) like ChatGPT and Claude—might be used to study disease development by tracing how one condition tends to follow another.
- Ensuring AI is Acting as Intended: A QA
AI adoption in growing, and as it does, more organizations are starting to get a handle on controlling agents, making testing and security a first-class capability for AI-generated code, and using validation, policy and governance to ensure outcomes are what was required. SD Times recently sat down with Johnny Halife, the CTO at software engineering... … continue reading The post Ensuring AI is Acting as Intended: A Q&A appeared first on SD Times .
Score: 26🌐 MovesAug 28, 2026https://sdtimes.com/ai-code-generation/ensuring-ai-is-acting-as-intended-a-qa/ - Why Claude Code Time Estimates Are Poor
Be a better communicator with LLM programming The post Why Claude Code Time Estimates Are Poor appeared first on Towards Data Science .
- Sandboxing and Blast Radius: Isolation Engineering for Agents
Part fourteen of a series on production agentic AI. Continue reading on Towards AI »
- Jackson Hole, Anthropic’s Court Win and PayPal Deal Collapse | Bloomberg Tech 8/28/2026
Bloomberg’s Tim Stenovec zeroes in on Jackson Hole after Federal Reserve Chairman Kevin Warsh delivered his first address at the Symposium. Plus, a US judge rules that the Trump administration must lift its ban on Anthropic's AI tech for federal agencies. And, a consortium of Advent and Stripe decided to abandon its pursuit of PayPal, a deal that would have ranked as one of the biggest-ever leveraged buyouts. (Source: Bloomberg)
Score: 25🌐 MovesAug 28, 2026https://www.bloomberg.com/news/videos/2026-08-28/bloomberg-tech-8-28-2026-video - AI Is Turning Data Governance Into A Competitive Advantage
For that governance to be effective, it needs to be grounded in a complete understanding of your network infrastructure.
- If you thought Meta’s child safety scandal was bad, just wait for AI
If you thought Meta’s child safety scandal was bad, just wait for AI The Telegraph
Score: 25🌐 MovesAug 28, 2026https://www.telegraph.co.uk/business/2026/08/28/if-you-thought-metas-child-safety-scandal-was-bad-just-wait/ - AI ecommerce projects may fail without clean, centralised data: Report
The report said data readiness is one of the biggest factors determining whether AI projects move beyond experimentation and deliver measurable business outcomes. It noted that AI systems depend heavily on the quality of the information they receive, making data infrastructure a critical foundation for businesses looking to scale AI.
- Human-in-the-Loop Without Killing Throughput
How we stopped reviewing every agent action and started routing human attention where it actually mattered The post Human-in-the-Loop Without Killing Throughput appeared first on Towards Data Science .
Score: 25🌐 MovesAug 28, 2026https://towardsdatascience.com/human-in-the-loop-without-killing-throughput/ - Loop Engineering: The Anatomy of Reliable Agentic AI
Introduction In 2024, the tech world was obsessed with building autonomous agents. An engineer writes an instruction, starts an agent, reads the result, notices what went wrong, and writes the next instruction. That workflow is useful while exploring a problem. It does not scale when the same class of work arrives every day. The moment agent work repeats — CI failures, support tickets, stale pull requests, or monitoring alerts — you are no longer designing a single response. You are designing a control system. An agent can report “done” while the test is still red. It can retry the same failed action, consume its budget, and leave behind no durable explanation of what happened. In production, those are loop failures — not merely prompt failures. Why Prompt Engineering Isn’t Enough Most agent systems are still designed one prompt at a time. We add instructions like “think step-by-step,” or “double-check your work.” The following layers are complementary, but distinct: - Prompt engineering: What should the model do in this turn? - Context engineering: What should the model see? - Harness engineering: How should one run execute safely? - Loop engineering: How should repeated work move toward a verified outcome? Prompt engineering only optimizes reasoning. It does absolutely nothing for execution. A well-designed loop still needs good prompts and useful context, and a harness still needs to constrain each run. But loop engineering adds the outer control system that makes the work repeatable and prevents infinite, unverified spinning. The Shift from Conversations to Control Systems We are undergoing a massive architectural shift: moving from conversational interfaces to autonomous control systems. The distinction matters because more turns do not automatically mean more progress. An agent can repeat the same action, accumulate irrelevant context, consume its budget, and still report success without producing a verified result. When you deploy an autonomous agent, you are removing the human loop. You must replace it with a strict, engineered control loop that manages its state, enforces its boundaries, and validates its work. What Is Loop Engineering? Loop Engineering is the practice of designing repeatable agent workflows that discover work, hand an agent a bounded task, observe the result, verify it with evidence, persist what happened, and decide whether to continue, stop, or escalate. It decides when the agent should wake up, what work it should receive, what evidence counts as progress, where the result is recorded, and what happens when the work cannot be completed. The model is an important component of the loop, but it is not the whole system. The Anatomy of a Production Loop A reliable loop is a small state machine, even when the implementation uses an agent framework. One useful model is the seven-stage core loop: 1. Discover Identify new, in-scope work and suppress duplicates. A loop needs a defined source of work (a CI failure, a support ticket) with a stable identity so it doesn’t create a new task every time it sees the same unchanged failure. 2. Handoff Convert an event into a bounded task. Define the goal, evidence, permissions, acceptance criteria, working location, and stop rule. Without this contract, “fix the failure” becomes permission to redesign the system. 3. Execute Let the agent act inside an explicit boundary. Isolation is part of the loop contract — whether that’s a disposable worktree, a branch, a container, or a sandbox. The loop should make the boundary visible rather than relying on the model to respect it. 4. Observe Capture tool-produced evidence, not just the agent’s narration. Useful observations include test output, compiler errors, changed-file lists, API responses, and diff summaries. “The change looks good” is not an observation. 5. Verify Check the resulting artifact independently. The agent that made a change should not be the only component deciding that the change is correct. Use unit tests, type checking, or a second evaluator with a separate rubric. The verification result should be structured JSON, not a free-form paragraph. 6. Persist Make the next run start with state. Record the attempt, evidence, decisions, and unresolved risks. The model context is not a durable database. A loop should remember what affects the next decision, not accumulate an ever-growing transcript. 7. Decide Retry only with new evidence; otherwise stop, escalate, or close. “Keep trying until it works” is not a stopping rule. Retries should be bounded. Human intervention belongs in the design from the beginning. Case Study: Building a Reliable CI Failure Loop Consider a loop that helps maintain a software repository by fixing CI failures. The Naive Version: An agent receives a failing CI notification and is told to fix it. It searches broadly, edits several files, runs one test, sees a failure, and tries again. After a few turns it reports that the issue is fixed. A human discovers that the original check never passed on the final commit. The loop had no stable work identity, no change boundary, and no independent completion state. The Engineered Version: 1. Discover: A webhook records the repository, commit, workflow, and normalized failure signature. 2. Handoff: The agent receives the failure log, a disposable worktree, allowed commands, and a two-attempt budget. 3. Execute: The agent investigates and proposes a minimal patch, editing only allowed files. 4. Observe: The loop records the diff, test output, exit codes, and elapsed time. 5. Verify: A fresh test run executes against the resulting commit independently of the agent’s summary. 6. Persist: The work item stores the commit, evidence bundle, and attempt number. 7. Decide: If checks pass, it prepares a pull request. If they fail with actionable new evidence, it retries. If the budget is exhausted, it stops as needs_review. Common Failure Modes 1. The loop has no real trigger: If a person still has to restart the task every time, the system is an agent session, not a loop. 2. The loop confuses a claim with evidence: “Done” and “tested” are claims. They must be backed by current evidence. 3. The loop retries without changing the information state: Rewording the same instruction does not create progress. Detect repeated observations and escalate. 4. The evaluator shares the generator’s blind spot: A second model with the same context may repeat the first model’s mistake. Add deterministic checks. 5. The loop silently expands its scope: Enforce file, tool, and time boundaries. Practical Checklist Before automating a recurring agent workflow, answer these questions: - [ ] What exact source discovers the work, and how are duplicates identified? - [ ] What is the smallest bounded task the agent can receive? - [ ] Which files, tools, systems, and actions are in scope? - [ ] What external state survives the run? - [ ] What evidence proves progress and completion? - [ ] How many attempts, tokens, and minutes are allowed? - [ ] What happens when the loop is blocked or reaches a terminal state? Conclusion Loop engineering is not the replacement for prompt engineering, context engineering, or harness engineering. It is the discipline that connects them across repeated work. The goal is not an agent that never makes a mistake. The goal is a system that notices what happened, gathers new evidence, limits its authority, records its state, and knows when to stop. Design the trigger. Bound the handoff. Verify the artifact. Persist the evidence. Escalate before the risk becomes irreversible. That is how an agent workflow becomes an engineered loop. Read the original deep-dive on [ Loop Engineering ] on my engineering blog. If you are building autonomous systems and want to bridge the gap between AI research and production-grade software engineering, follow me here on Medium and connect on LinkedIn . Loop Engineering: The Anatomy of Reliable Agentic AI was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Migrate Pinecone to Qdrant: Complete Migration Guide | Zero Heart Burns
Okay so… you opened a bill from Pinecone and did a double-take. Or you tried to self-host it and discovered that’s just not a thing Pinecone lets you do. Or maybe you hit the 40KB metadata limit mid-sprint and your whole architecture needed a rethink. Whatever the reason might be you’re here, you want to move your vectors, and you want to do it without a disaster. I’ve been through this migration. Here’s everything I wish someone had told me before I started. Why You’re Even Reading This Here’s the thing about Pinecone… it’s genuinely easy to get started with. The docs are everywhere in AI tutorials, the API is clean, and you can have vectors in the cloud in like 20 minutes. That’s why so many teams start there. But then production hits… The cost curve surprises you. Pinecone’s serverless model charges per read unit, so every query costs money. At moderate QPS (say 500 queries per second), that meter is spinning fast. One team I talked to went from a comfortable $200/month in early testing to over $2k/month once their app got traction. The data doesn’t really get bigger, but the bills do. Vendor lock-in is real. Here’s a thing that quietly terrifies me: Pinecone has no export API. Like, none. You cannot say “give me all my vectors as a file.” If you want your data back, you have to iterate through IDs and fetch them back one batch at a time, and that only works on serverless indexes. Pod-based indexes? No list API at all. Pretty wild right? Namespace limits sneak up on you. Pinecone namespaces are a nice partitioning primitive but you’re capped at 100,000 namespaces on Standard. If you’re building a multi-tenant SaaS where each user has their own vector space… yeah, that can get uncomfortable fast. Metadata is flat and limited. 40KB per record, no nested JSON, no geo-coordinates, no full-text search on metadata fields. Need to store a document’s full text alongside its embedding? You’ll be doing a lot of creative truncation. No self-hosting, ever. This one is fundamental. Pinecone is SaaS-only. There is no Docker image, no Kubernetes operator, no “run it on your own machine” option. For teams with data residency requirements, regulated industries, or just a preference for not being dependent on external infrastructure… this is a hard blocker. The lightbulb moment for most developers is when they run Qdrant locally with docker run -p 6333:6333 qdrant/qdrant and realize it's the same thing - same API, same capabilities, same performance - but it's just... there, on your machine, completely under your control. Here’s the shape of this migration so you know what you’re getting into: Export your Pinecone data to a local JSONL file Spin up Qdrant locally or on Qdrant Cloud Load the data in and verify it If you’re in production: run dual-write while you migrate, then cut over with zero downtime Nothing here is magic. It’s just a bit of Python and some patience. The Mental Model Shift Before touching any code, let’s map the concepts. This is reference material to skim it now, come back when something breaks. Core Concept Mapping The namespace decision is actually important. You have two options: Separate collection per namespace: great if you have a small number of large namespaces and want clean isolation Single collection with a namespace payload field - better if you have many small namespaces, or want cross-namespace queries For most Pinecone migrations I’d go with option B (payload field), because collections have a recommended cap around 1,000 in Qdrant, and many Pinecone users have way more namespaces than that. Distance Metric Mapping # Pinecone metric values -> Qdrant Distance enum METRIC_MAP = { "cosine": Distance.COSINE, "euclidean": Distance.EUCLID, "dotproduct": Distance.DOT, } # Qdrant also has Distance.MANHATTAN - no Pinecone equivalent One gotcha here: Pinecone rescales cosine similarity to [0, 1]. Qdrant returns [-1, 1]. The rankings are identical, but any code that thresholds on raw score values will behave differently. Keep this in mind when you port your search logic. Filter Syntax from Before and After # === PINECONE === results = index.query( vector=[0.1] * 1536, filter={ "$and": [ {"genre": {"$eq": "sci-fi"}}, {"year": {"$gte": 2020}}, {"tags": {"$in": ["ai", "robots"]}} ] }, top_k=10, namespace="my-namespace", include_metadata=True ) matches = results["matches"] # list of {id, score, metadata} # === QDRANT === from qdrant_client.models import Filter, FieldCondition, MatchValue, MatchAny, Range results = client.search( collection_name="my_collection", query_vector=[0.1] * 1536, query_filter=Filter( must=[ FieldCondition(key="genre", match=MatchValue(value="sci-fi")), FieldCondition(key="year", range=Range(gte=2020)), FieldCondition(key="tags", match=MatchAny(any=["ai", "robots"])), # If you're using the payload-field namespace strategy: FieldCondition(key="_namespace", match=MatchValue(value="my-namespace")), ] ), limit=10, with_payload=True ) # results is a list of ScoredPoint objects with .id, .score, .payload The filter mapping rule of thumb: $and maps to must, $or maps to should, $ne maps to must_not. Numeric ranges map directly. $in maps to MatchAny. Honestly pretty intuitive once you see it side by side. Sparse + Dense Hybrid Pinecone hybrid search uses a single alpha parameter (0 = pure sparse, 1 = pure dense, linear blend). Qdrant uses explicit prefetch legs and rank fusion - either RRF (Reciprocal Rank Fusion) or DBSF. More code to write, but dramatically more control over how the fusion behaves. You can weight each leg independently, which ends up being pretty important for production tuning. Getting Your Data Out of Pinecone Here’s the honest truth: Pinecone has no export feature. No “Download All Data” button, no export endpoint, no snapshot you can take away. The only way out is to iterate through vector IDs and fetch them back in batches. The approach: Call list() to page through all vector IDs in a namespace (returns 100 IDs per page) For each page of IDs, call fetch() to get the actual vectors + metadata (up to 1,000 IDs per call) Write everything to a JSONL file One more thing before the code: this only works on serverless indexes . Pod-based indexes have no list API at all. If you're on a pod-based index... the only real option is to re-embed from your original document source, because there's no supported way to bulk-export from a pod index. Yikes. Rate limits to know: list is 200 req/s, fetch is 100 req/s. With 1,000 IDs per fetch call, that's theoretically 100,000 vectors per second. In practice, you'll be slower because of network and parsing overhead - but you won't hit limits if you stay around 10 concurrent fetch calls. Here’s a complete, resumable Pinecone exporter: #!/usr/bin/env python3 # VIBE CODED for smooth and quick setups """ pinecone_dumper.py Exports all vectors from a Pinecone serverless index to a JSONL file. Resumable: if interrupted, just rerun - it will pick up from where it left off. Requirements: pip install pinecone>=3.0.0 tqdm Usage: python pinecone_dumper.py \ --api-key pcsk_XXX \ --index-name my-index \ --output vectors.jsonl """ import argparse import json import os import time import math import pickle from pathlib import Path from pinecone import Pinecone from tqdm import tqdm LIST_BATCH_SIZE = 100 # Pinecone max IDs per list page FETCH_BATCH_SIZE = 1000 # Pinecone max IDs per fetch call FETCH_RATE_SLEEP = 0.011 # ~90 req/s, safely under 100 req/s limit def load_checkpoint(ckpt_path: str) -> dict: if Path(ckpt_path).exists(): with open(ckpt_path, "rb") as f: return pickle.load(f) return {} def save_checkpoint(ckpt_path: str, data: dict): tmp = ckpt_path + ".tmp" with open(tmp, "wb") as f: pickle.dump(data, f) os.replace(tmp, ckpt_path) def list_all_ids(index, namespace: str, checkpoint: dict, ckpt_path: str) -> list: ckpt_key = f"ids:{namespace}" if ckpt_key in checkpoint: print(f" Resuming ID listing for namespace='{namespace}' from checkpoint...") return checkpoint[ckpt_key] all_ids = [] cursor = None page_num = 0 print(f" Listing IDs for namespace='{namespace}'...") while True: kwargs = {"limit": LIST_BATCH_SIZE, "namespace": namespace} if cursor: kwargs["pagination_token"] = cursor try: resp = index.list_paginated(**kwargs) except Exception as e: print(f" ERROR listing page {page_num}: {e}. Retrying in 5s...") time.sleep(5) continue page_ids = [v.id for v in (resp.vectors or [])] all_ids.extend(page_ids) page_num += 1 if page_num % 100 == 0: print(f" ... {len(all_ids):,} IDs listed so far") cursor = resp.pagination.next if resp.pagination else None if not cursor: break print(f" Found {len(all_ids):,} IDs in namespace='{namespace}'") checkpoint[ckpt_key] = all_ids save_checkpoint(ckpt_path, checkpoint) return all_ids def fetch_and_write(index, all_ids, namespace, output_file, checkpoint, ckpt_path): ckpt_key = f"fetched_batches:{namespace}" completed_batches = checkpoint.get(ckpt_key, set()) total_batches = math.ceil(len(all_ids) / FETCH_BATCH_SIZE) new_writes = 0 with tqdm(total=len(all_ids), desc=f" Fetching vectors (ns='{namespace}')") as pbar: for batch_idx in range(total_batches): start = batch_idx * FETCH_BATCH_SIZE batch_ids = all_ids[start:start + FETCH_BATCH_SIZE] if batch_idx in completed_batches: pbar.update(len(batch_ids)) continue # Fetch with exponential backoff retry for attempt in range(5): try: resp = index.fetch(ids=batch_ids, namespace=namespace) break except Exception as e: wait = 2 ** attempt print(f"\n Fetch attempt {attempt+1} failed: {e}. Waiting {wait}s...") time.sleep(wait) else: print(f"\n ERROR: Could not fetch batch {batch_idx}. Skipping.") pbar.update(len(batch_ids)) continue vectors_map = resp.get("vectors", {}) for pid, pdata in vectors_map.items(): record = { "id": pid, "values": pdata.get("values", []), "metadata": pdata.get("metadata", {}), "_namespace": namespace, } sv = pdata.get("sparse_values") or pdata.get("sparseValues") if sv: record["sparse_values"] = { "indices": sv.get("indices", []), "values": sv.get("values", []), } output_file.write(json.dumps(record) + "\n") new_writes += 1 completed_batches.add(batch_idx) if batch_idx % 10 == 0: checkpoint[ckpt_key] = completed_batches save_checkpoint(ckpt_path, checkpoint) pbar.update(len(batch_ids)) time.sleep(FETCH_RATE_SLEEP) checkpoint[ckpt_key] = completed_batches save_checkpoint(ckpt_path, checkpoint) return new_writes def main(): parser = argparse.ArgumentParser(description="Export Pinecone index to JSONL") parser.add_argument("--api-key", required=True) parser.add_argument("--index-name", required=True) parser.add_argument("--index-host", default=None, help="Index host URL (optional, faster)") parser.add_argument("--namespace", default=None, help="Export one namespace only") parser.add_argument("--output", default="pinecone_export.jsonl") parser.add_argument("--checkpoint", default="pinecone_dump_checkpoint.pkl") parser.add_argument("--fresh", action="store_true", help="Ignore existing checkpoint") args = parser.parse_args() pc = Pinecone(api_key=args.api_key) index = pc.Index(host=args.index_host) if args.index_host else pc.Index(args.index_name) stats = index.describe_index_stats() namespaces = list(stats.get("namespaces", {}).keys()) if not namespaces: namespaces = [""] # default namespace if args.namespace: namespaces = [args.namespace] total_vectors = stats.get("total_vector_count", 0) dimension = stats.get("dimension", "unknown") print(f"\nIndex: {args.index_name}") print(f" Dimension: {dimension}") print(f" Total vectors: {total_vectors:,}") print(f" Namespaces: {namespaces}") print(f" Output: {args.output}\n") checkpoint = {} if args.fresh else load_checkpoint(args.checkpoint) total_written = 0 with open(args.output, "a", encoding="utf-8") as out_f: for ns in namespaces: print(f"Exporting namespace='{ns}'...") ids = list_all_ids(index, ns, checkpoint, args.checkpoint) written = fetch_and_write(index, ids, ns, out_f, checkpoint, args.checkpoint) total_written += written print(f" Done. {written:,} vectors written.\n") print(f"Export complete! Total: {total_written:,} vectors -> {args.output}") print(f"\nNext step: python qdrant_uploader.py --input {args.output}") if __name__ == "__main__": main() Run it like this: pip install "pinecone>=3.0.0" tqdm python pinecone_dumper.py \ --api-key pcsk_YOUR_KEY \ --index-name my-index \ --output my_vectors.jsonl If it gets interrupted mid-run, just run it again …. it’ll skip everything it already fetched. The checkpoint file tracks progress at the batch level so you don’t re-fetch things you already have. For very large indexes (100M+ vectors), this will take a while. The math: at 1,000 vectors per fetch and ~90 requests per second, you’re looking at roughly 90,000 vectors per second throughput. 100M vectors = ~18 minutes of pure fetch time, plus listing time on top. Run it overnight. Loading Into Qdrant (Local + Cloud) Okay, you’ve got your my_vectors.jsonl file. Now let's get it into Qdrant. Step 1: Start Qdrant locally docker run -d --name qdrant \ -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage \ qdrant/qdrant The web UI is at http://localhost:6333/dashboard. It's actually quite nice - you can browse collections, run test queries, inspect points. Better than nothing! Step 2: Recreate your collection You need the dimension and metric from your Pinecone index. The describe_index_stats() call doesn't return the metric (annoyingly) - check your index creation code or the Pinecone console. from qdrant_client import QdrantClient from qdrant_client.models import ( Distance, VectorParams, SparseVectorParams, HnswConfigDiff, OptimizersConfigDiff ) client = QdrantClient(url="http://localhost:6333") # For Qdrant Cloud: # client = QdrantClient( # url="https://YOUR-CLUSTER.cloud.qdrant.io", # api_key="YOUR_QDRANT_API_KEY" # ) COLLECTION = "my_collection" DIMENSION = 1536 # match your Pinecone index dimension METRIC = "cosine" METRIC_MAP = { "cosine": Distance.COSINE, "euclidean": Distance.EUCLID, "dotproduct": Distance.DOT, } # Disable HNSW during bulk load - re-enable after (3-5x faster ingest) client.create_collection( collection_name=COLLECTION, vectors_config=VectorParams(size=DIMENSION, distance=METRIC_MAP[METRIC]), optimizers_config=OptimizersConfigDiff(indexing_threshold=0), ) print(f"Created collection '{COLLECTION}'") Notice indexing_threshold=0 - that disables HNSW index building during the bulk load. We'll re-enable it after. This makes loading 3-5x faster because Qdrant isn't trying to build the graph while you're flooding it with vectors. Step 3: The uploader script #!/usr/bin/env python3 # Once again, have vibe coded this part to perfection! """ qdrant_uploader.py Reads a JSONL file from pinecone_dumper.py and bulk-upserts into Qdrant. Handles ID translation, batching, retries, and progress tracking. Requirements: pip install "qdrant-client>=1.9.0" tqdm Usage: python qdrant_uploader.py \ --input vectors.jsonl \ --collection my_collection \ --qdrant-url http://localhost:6333 """ import argparse import json import hashlib import time from pathlib import Path from qdrant_client import QdrantClient from qdrant_client.models import ( PointStruct, SparseVector, Distance, VectorParams, OptimizersConfigDiff, UpdateStatus ) from tqdm import tqdm UPSERT_BATCH_SIZE = 512 # 256-512 is the sweet spot MAX_RETRIES = 5 def pinecone_id_to_qdrant(pinecone_id: str) -> str: """ Convert a Pinecone string ID to a deterministic UUID for Qdrant. We store the original ID in payload as _pinecone_id for reverse lookup. """ h = hashlib.sha256(pinecone_id.encode()).hexdigest() return f"{h[:8]}-{h[8:12]}-{h[12:16]}-{h[16:20]}-{h[20:32]}" def count_lines(filepath: str) -> int: count = 0 with open(filepath, "rb") as f: for _ in f: count += 1 return count def upsert_batch_with_retry(client, collection, points, max_retries=MAX_RETRIES): for attempt in range(max_retries): try: result = client.upsert( collection_name=collection, points=points, wait=True, ) return result.status == UpdateStatus.COMPLETED except Exception as e: wait = 2 ** attempt print(f"\n Upsert attempt {attempt+1} failed: {e}. Waiting {wait}s...") time.sleep(wait) print(f"\n ERROR: Batch failed after {max_retries} retries. Skipping {len(points)} points.") return False def build_point(record: dict, has_sparse: bool) -> PointStruct: pinecone_id = record["id"] qdrant_id = pinecone_id_to_qdrant(pinecone_id) values = record.get("values", []) payload = dict(record.get("metadata", {})) payload["_pinecone_id"] = pinecone_id payload["_namespace"] = record.get("_namespace", "") if has_sparse and "sparse_values" in record: sv = record["sparse_values"] vector = { "dense": values, "sparse": SparseVector( indices=sv.get("indices", []), values=sv.get("values", []) ) } else: vector = values return PointStruct(id=qdrant_id, vector=vector, payload=payload) def main(): parser = argparse.ArgumentParser(description="Upload JSONL to Qdrant") parser.add_argument("--input", required=True) parser.add_argument("--collection", required=True) parser.add_argument("--qdrant-url", default="http://localhost:6333") parser.add_argument("--api-key", default=None, help="Qdrant API key (for cloud)") parser.add_argument("--has-sparse", action="store_true") parser.add_argument("--batch-size", type=int, default=UPSERT_BATCH_SIZE) args = parser.parse_args() client = QdrantClient(url=args.qdrant_url, api_key=args.api_key) try: info = client.get_collection(args.collection) print(f"Collection '{args.collection}' found - {info.vectors_count or 0:,} existing vectors") except Exception: print(f"ERROR: Collection '{args.collection}' not found. Create it first.") return total_lines = count_lines(args.input) print(f"\nInput file: {args.input} ({total_lines:,} records)") print(f"Batch size: {args.batch_size}\n") batch = [] total_uploaded = 0 total_failed = 0 with open(args.input, "r", encoding="utf-8") as f, \ tqdm(total=total_lines, desc="Uploading", unit="vec") as pbar: for line in f: line = line.strip() if not line: continue try: record = json.loads(line) except json.JSONDecodeError as e: print(f"\n Bad JSON line: {e}. Skipping.") pbar.update(1) continue batch.append(build_point(record, args.has_sparse)) if len(batch) >= args.batch_size: ok = upsert_batch_with_retry(client, args.collection, batch) if ok: total_uploaded += len(batch) else: total_failed += len(batch) pbar.update(len(batch)) batch = [] if batch: ok = upsert_batch_with_retry(client, args.collection, batch) total_uploaded += len(batch) if ok else 0 total_failed += len(batch) if not ok else 0 pbar.update(len(batch)) print(f"\nUpload complete! Uploaded: {total_uploaded:,} Failed: {total_failed:,}") # Re-enable HNSW indexing now that bulk load is done print("\nRe-enabling HNSW indexing...") client.update_collection( collection_name=args.collection, optimizers_config=OptimizersConfigDiff(indexing_threshold=20_000), ) print("Waiting for index to build (this may take a few minutes)...") while True: info = client.get_collection(args.collection) status = str(info.status).lower() if "green" in status: break print(f" Status: {status}... waiting 15s") time.sleep(15) final_count = client.get_collection(args.collection).vectors_count print(f"\nCollection '{args.collection}' is ready! Vector count: {final_count:,}") if __name__ == "__main__": main() Run the full pipeline: pip install "qdrant-client>=1.9.0" tqdm # 1. Export from Pinecone python pinecone_dumper.py --api-key pcsk_XXX --index-name my-index --output vectors.jsonl # 2. Upload to local Qdrant python qdrant_uploader.py \ --input vectors.jsonl \ --collection my_collection \ --qdrant-url http://localhost:6333 For Qdrant Cloud : sign up here , create a cluster (the free tier gets you 1M vectors permanently), then use: python qdrant_uploader.py \ --input vectors.jsonl \ --collection my_collection \ --qdrant-url https://YOUR-CLUSTER.cloud.qdrant.io \ --api-key YOUR_QDRANT_KEY Step 4: Verify it worked from qdrant_client import QdrantClient client = QdrantClient(url="http://localhost:6333") # Exact count (Qdrant's vectors_count is approximate for large collections) def exact_count(client, collection): count, offset = 0, None while True: recs, next_offset = client.scroll( collection_name=collection, limit=1000, offset=offset, with_payload=False, with_vectors=False ) count += len(recs) offset = next_offset if offset is None: break return count n = exact_count(client, "my_collection") print(f"Qdrant has {n:,} vectors") # Spot-check: search and print original Pinecone IDs from payload test_vec = [0.1] * 1536 # replace with a real vector results = client.search("my_collection", query_vector=test_vec, limit=5, with_payload=True) for r in results: print(f" score={r.score:.4f} original_pinecone_id={r.payload.get('_pinecone_id')}") One more option: Qdrant’s official migration tool Qdrant ships a Docker-based migration CLI that handles Pinecone directly. For serverless indexes, one command does the whole thing: docker run --rm -it registry.cloud.qdrant.io/library/qdrant-migration pinecone \ --pinecone.index-host 'https://your-index.svc.pinecone.io' \ --pinecone.index-name 'your-index' \ --pinecone.api-key 'pcsk_...' \ --qdrant.url 'https://your-cluster.cloud.qdrant.io:6334' \ --qdrant.api-key 'your-qdrant-key' \ --qdrant.collection 'your-collection' \ --migration.batch-size 64 Note the gRPC port (6334): the migration tool uses gRPC, not the REST API. And it’s resumable via an internal _migration_offsets collection it creates on the target. If it gets killed, just rerun and it picks up from the last offset. Zero Downtime Migration (The Production Way) If you’re running a live production service on Pinecone, you can’t just turn it off run the migration and turn it back on. Here’s how to do it with zero downtime. The core idea: run both databases simultaneously. Pinecone stays primary for reads. Qdrant gets all the writes. Once Qdrant has all the historical data and has proven itself with real traffic, you flip the switch. The Dual-Write Wrapper import threading from concurrent.futures import ThreadPoolExecutor from queue import Queue, Empty class DualWriteClient: """ Wrap your existing Pinecone index to simultaneously write to Qdrant. Pinecone is primary (writes block on Pinecone success). Qdrant writes are async and non-blocking - failures queue for retry. """ def __init__(self, pinecone_idx, qdrant_client, qdrant_collection): self.pine = pinecone_idx self.qdrant = qdrant_client self.col = qdrant_collection self.pool = ThreadPoolExecutor(max_workers=4, thread_name_prefix="qdrant-write") self.retry_q = Queue(maxsize=10_000) self._start_retry_worker() def upsert(self, vectors, namespace=""): # Step 1: Write to Pinecone (primary, blocks until confirmed) self.pine.upsert(vectors=vectors, namespace=namespace) # Step 2: Write to Qdrant async (never blocks your app) self.pool.submit(self._qdrant_upsert, vectors, namespace) def _qdrant_upsert(self, vectors, namespace): from qdrant_client.models import PointStruct points = [ PointStruct( id=pinecone_id_to_qdrant(v["id"]), vector=v["values"], payload={**v.get("metadata", {}), "_pinecone_id": v["id"], "_namespace": namespace} ) for v in vectors ] try: self.qdrant.upsert(collection_name=self.col, points=points, wait=True) except Exception as e: print(f"Qdrant write failed, queuing retry: {e}") try: self.retry_q.put_nowait(("upsert", points)) except Exception: print(f"Retry queue full - {len(points)} points dropped") def delete(self, ids, namespace=""): self.pine.delete(ids=ids, namespace=namespace) qdrant_ids = [pinecone_id_to_qdrant(i) for i in ids] self.pool.submit(self._qdrant_delete, qdrant_ids) def _qdrant_delete(self, ids): try: self.qdrant.delete(collection_name=self.col, points_selector=ids, wait=True) except Exception as e: try: self.retry_q.put_nowait(("delete", ids)) except Exception: pass def _start_retry_worker(self): t = threading.Thread(target=self._retry_loop, daemon=True) t.start() def _retry_loop(self): while True: try: op, payload = self.retry_q.get(timeout=5) except Empty: continue for attempt in range(5): try: if op == "upsert": self.qdrant.upsert(collection_name=self.col, points=payload, wait=True) else: self.qdrant.delete(collection_name=self.col, points_selector=payload, wait=True) break except Exception: time.sleep(2 ** attempt) # Usage: replace your existing pinecone_index with this one line # Before: results = pinecone_index.query(...) # After: results = dual_client.pine.query(...) <- reads still go to Pinecone # # dual_client = DualWriteClient(pc.Index("my-index"), qdrant_client, "my_collection") # dual_client.upsert(vectors=[...]) <- writes go to both The Migration Timeline Once dual-write is running, here’s the sequence: Start dual-write : your app now writes to both. Reads still come from Pinecone. Run the backfill: use the dumper + uploader scripts from Sections 3–4 to migrate historical data. Dual-write handles anything new that comes in while the backfill runs. Shadow mode: for a day or two, run every search query against Qdrant too (don’t show results to users yet), compare rankings, log quality differences. Cutover: once you’re happy with quality parity, flip the read traffic. Cutover Checklist Before you switch traffic to Qdrant: JUST VERI FY the following okay? [x] Dual-write has been running >= 48 hours without errors [x] Backfill complete - Qdrant vector count within 0.1% of Pinecone count [x] Shadow comparison: mean Recall@10 >= 0.95 over at least 10,000 queries [x] Qdrant collection status = green (fully indexed) [x] P99 search latency on Qdrant within 15% of Pinecone [x] Rollback plan tested - know how to re-enable Pinecone reads in under 5 minutes The Atomic Cutover (Qdrant Aliases) Qdrant has collection aliases so that a single API call switches an alias from one collection to another with zero gap. Use this for the cutover: from qdrant_client.models import CreateAliasOperation, DeleteAliasOperation # One API call - no window where the alias is undefined qdrant_client.update_collection_aliases( change_aliases_operations=[ DeleteAliasOperation(delete_alias={"alias_name": "production"}), CreateAliasOperation(create_alias={ "collection_name": "my_collection", "alias_name": "production" }), ] ) # From this point: all reads going to "production" alias hit Qdrant Rollback Plan Keep dual-write running for at least 48 hours after full cutover. If something goes wrong, rollback is literally just updating a feature flag or load balancer weight to send reads back to Pinecone. Since dual-write is still active, Pinecone stays in sync during that window. Rolling back is instant and safe. Why Qdrant Cloud Hits Different Okay… so I’ve been pretty technical so far. Let me just talk about what actually feels different when you’re using Qdrant versus Pinecone day-to-day. The free tier is actually useful. Qdrant Cloud’s free tier is permanent and not a trial, and supports roughly 1M vectors on a real cluster (0.5 vCPU, 1 GB RAM, 4 GB disk)!!!!!! Pinecone’s free tier is 100,000 vectors on a single serverless index with the per-read-unit billing lurking. Qdrant’s free tier is just… a free cluster. No gotchas, no time limit. NOTABLE MENTIONS FROM MY RESEARCH AND DUMP BELOW! You can actually run it yourself. docker run -p 6333:6333 qdrant/qdrant. That's it. The self-hosted version and the cloud version run the same binary. No features are gated behind cloud-only access. The API is identical everywhere. You can move between local development, your own Kubernetes cluster, and Qdrant Cloud at any time. Your data is portable. Your skills transfer. Pinecone is SaaS-only, period. Quantization is a superpower. Pinecone stores everything as float32. That’s 4 bytes per dimension, no exceptions. For a 10M vector / 1536-dim collection, that’s ~61 GB of raw vector data. With Qdrant’s Turbo4 4-bit datatype (from v1.19), you get that down to ~7.6 GB while maintaining solid recall. With binary quantization you’re under 2 GB. This is roughly a 30x memory reduction. At $5/GB/month in cloud RAM costs, that’s a very large number. Quick cost comparison for 10M vectors at 1536 dims Filterable HNSW is a real algorithmic difference. Pinecone applies filters as pre-filter or post-filter. If your filter is very selective (say, filter to 1% of vectors), post-filtering wastes a huge amount of graph traversal work, and pre-filtering with very small candidate sets loses recall. Qdrant integrates payload filtering directly into the HNSW graph traversal — it navigates the graph while respecting the filter in real-time. This gives predictable recall even on highly selective filters. It’s not a marketing claim — it’s a fundamentally different query execution model. Multi-vector per point. Each Qdrant point can have multiple named vectors. One point can carry a text embedding (768-dim), an image embedding (512-dim), and a sparse BM25 vector, all stored together with their payload. A single hybrid search can fuse all three with RRF. Pinecone gives you one vector per record. Full stop. This matters a lot for multimodal and hybrid retrieval workloads. Data portability is the anti-vendor-lock-in story. With Qdrant, you can snapshot any collection and download it: # Create a snapshot snapshot_info = client.create_snapshot(collection_name="my_collection") print(f"Snapshot: {snapshot_info.name}") # Download it (it's just an HTTP GET) # GET /collections/my_collection/snapshots/{snapshot_name} That snapshot is your data. You can restore it to any Qdrant instance anywhere. If Qdrant doubles their prices tomorrow, you have a complete copy of your data ready to move. With Pinecone, you do the LIST+FETCH dance described in Section 3. If Pinecone has an outage and you can’t list your IDs… well. Open source, for real. Qdrant has 34,000+ GitHub stars, Apache 2.0 license, and the team actively responds to issues. You can read the source code, understand exactly what’s happening with your data, file bugs, send PRs. When something weird happens with your search results, you can actually dig in and find out why. Pinecone is a black box. That distinction sounds abstract until something goes wrong. The Qdrant documentation covers all of this in depth — quantization options, memory tier configs, distributed deployment, the lot. And if you want to follow what’s coming next (new quantization research, benchmark results, engineering deep-dives), the newsletter is worth subscribing to. They publish real engineering content, not just product announcements. Alright now! that’s the full picture. The migration is doable okay and most teams do it over a week or two with zero downtime using the dual-write approach. The scripts handle the heavy lifting. The main gotchas to keep in mind: Pinecone has no export API: you’re doing LIST+FETCH, and that only works on serverless indexes Pod-based indexes need to be re-embedded from your original document source Cosine scores have different ranges ([0,1] in Pinecone, [-1,1] in Qdrant) - watch any code that thresholds on raw scores Keep dual-write running for 48 hours post-cutover before you decommission anything Good luck with the migration. You’ve got this. Migrate Pinecone to Qdrant: Complete Migration Guide | Zero Heart Burns was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Harnessing Fabric Copilot for Automated Data Engineering and DAX Generation
Data engineering has historically been a discipline defined by meticulous, often repetitive, manual labor. For years, we spent countless hours writing boilerplate ETL (Extract, Transform, Load) scripts, digging through convoluted documentation to find the correct PySpark syntax, and manually untangling complex data pipelines. We built our systems line by line, often sacrificing speed for stability. However, the paradigm is shifting rapidly. Microsoft Fabric is no longer just a data platform. By integrating large language models directly into our daily workflows, we are transforming Fabric from a passive data system into an AI-native decision engine. With Copilot embedded across every workload, AI functions built into the Warehouse and intelligent code generation available at our fingertips, we are experiencing a monumental shift in how we build and scale analytical solutions. In this comprehensive guide, we will explore how we can leverage Fabric Copilot to completely automate our data engineering processes. We will dive deep into hands-on techniques for auto-generating complex PySpark ETL code, instantly debugging opaque Spark stack traces, and writing highly optimized DAX measures. By harnessing these embedded Copilot features, we are significantly accelerating development across Data Engineering, Data Science, and Power BI. Enabling Copilot in the Workspace Before we can begin auto-generating our pipelines and queries, we must ensure our environment is properly configured to support generative AI workloads. Fabric Copilot is not simply a lightweight add-on; it requires substantial underlying compute resources to process our prompts, analyze our schema metadata, and return highly accurate code suggestions. To utilize these advanced features, we must deeply understand the capacity model, learning exactly how Compute Units (CUs) work and why billing can easily get out of hand if left unmonitored. When we purchase a Microsoft Fabric capacity (for instance, an F64 SKU), we are provisioning a pool of 64 Compute Units. This pool represents our absolute baseline of computational power. Currently, Microsoft requires our workspaces to be backed by at least an F64 capacity (or equivalent Power BI Premium P1 capacity) to enable the Copilot features natively. Once our F64 capacity is actively running, our tenant administrators must explicitly enable the Copilot integrations within the Fabric Admin Portal. We navigate to the tenant settings, locate the Copilot and Azure OpenAI service section, and toggle the switches to allow data to flow to the AI models. By securing this configuration, we ensure that our data engineering teams have immediate, governed access to generative AI without needing to manage separate API keys or external Azure OpenAI endpoints. Copilot in Notebooks for Code Generation, Refactoring, and Optimization The most immediate and impactful area where we see Copilot shine is within the Fabric Notebook experience. When we attach a notebook to our Lakehouse, we are no longer staring at a blank cell, wondering how to best structure our PySpark transformations. Instead, we can utilize natural language to describe our intent, and Copilot will construct the exact syntax required. Let us examine a highly practical, code-based demonstration. In this scenario, we need to process a massive dataset of web telemetry. We want to read raw data, filter out malicious bot traffic using regular expressions, calculate user session durations using complex windowing functions, and write the optimized data back to our Lakehouse. Instead of writing this from scratch, we can simply invoke the IPython magic command %%copilot directly inside our notebook cell. The Code-Based Demo # Example of using Fabric Copilot IPython Magic Commands inside Notebooks # %chat -Prompt Fabric Copilot to construct a complex Spark transformation pipeline %%copilot Write a PySpark script that reads a Delta table named 'raw_clickstream', filters out requests from IP addresses matching a bot pattern regex, calculates session duration per UserID using windowing functions, and writes the result to 'silver_user_sessions' with V-Order enabled. When we execute this cell, Copilot interprets our English instructions, analyzes the context of our Lakehouse, and generates a fully functional, production-ready PySpark script. We will receive an output block that looks remarkably similar to this: from pyspark.sql.functions import col, unix_timestamp, max, min from pyspark.sql.window import Window # 1. Read the raw Delta table df_raw = spark.read.format("delta").table("raw_clickstream") # 2. Filter out bot traffic using a regular expression pattern bot_pattern = r"^(192\.168\.|10\.|172\.(1[6-9]|2[0-9]|3[0-1])\.)" df_filtered = df_raw.filter(~col("ip_address").rlike(bot_pattern)) # 3. Calculate session duration using windowing functions windowSpec = Window.partitionBy("UserID") df_sessions = df_filtered.withColumn("SessionStart", min("EventTimestamp").over(windowSpec)) \ .withColumn("SessionEnd", max("EventTimestamp").over(windowSpec)) \ .withColumn("SessionDurationSeconds", unix_timestamp("SessionEnd") - unix_timestamp("SessionStart")) # 4. Write the result to the Silver layer with V-Order enabled for extreme read performance df_sessions.write.format("delta") \ .mode("overwrite") \ .option("spark.sql.parquet.vorder.enabled", "true") \ .saveAsTable("silver_user_sessions") We can immediately see the profound value in this generated code. Copilot did not just give us a generic template; it correctly imported the required pyspark.sql.functions, successfully implemented the Window specifications, and, crucially, understood the Fabric-specific configuration for enabling V-Order on our Delta write operation. We simply review the code, make any minor adjustments to our specific column names or bot regex patterns, and execute the pipeline. We have just condensed an hour of coding and documentation hunting into a mere ten seconds. Debugging Complex Spark Stack Traces Beyond generation, Copilot serves as an exceptional debugging assistant. We have all experienced the frustration of executing a PySpark script only to be met with a 200-line Java Virtual Machine (JVM) error stack trace. Finding the actual root cause, perhaps a data type mismatch or an out-of-memory error, can be incredibly tedious. When an error occurs in a Fabric Notebook, we now have a “Fix with Copilot” button integrated directly into the error output. When we click this, Copilot reads the entire stack trace, identifies the specific point of failure in our Python code, and explains the issue to us in plain English. More importantly, it provides a rewritten version of our code designed to bypass the error. We transition from spending hours debugging to resolving complex distributed computing errors in minutes. Harnessing Copilot for Optimized DAX Generation While data engineering in PySpark handles our backend transformations, our analytical models ultimately rely on Data Analysis Expressions (DAX) to deliver business value in Power BI. DAX is notoriously difficult to master, and poorly written queries can have devastating financial impacts on our Fabric capacity. Often, we discover that a single, poorly written DAX measure performing row-by-row calculations is responsible for eating 40% of our daily capacity. When iterating through millions of rows inefficiently, the compute spikes, and other workloads in our workspace begin to suffer from throttling. By isolating these specific, inefficient items, we can assign our engineering teams to refactor the code, effectively saving thousands of dollars a month without upgrading the underlying capacity. However, instead of relying solely on manual refactoring, we can leverage Copilot within the Fabric and Power BI environments to generate optimized, set-based DAX measures from the start. We can open the Copilot chat pane while viewing our semantic model and ask it directly: “Calculate the rolling 12-month average for Total Sales Revenue, ensuring that we ignore any cross-filtering from the Geography table, and format the output as Currency.” Copilot will analyze the relationships, tables, and columns within our active semantic model and return a DAX measure utilizing functions like CALCULATE, DATESINPERIOD, and REMOVEFILTERS. By generating DAX through the AI, we avoid common beginner pitfalls like recursive iterators (the dreaded SUMX loops over massive fact tables) and instead receive code that adheres to best practices. This ensures our models render in sub-seconds and our F64 capacity remains healthy and unburdened. Copilot in Data Factory for Text-to-Pipeline Ingestion Workflows Our automation journey does not stop at coding and modeling; it extends into orchestration. Microsoft Fabric Data Factory allows us to orchestrate complex data movement and transformations, but building large-scale pipelines with dozens of activities can be visually overwhelming. Fabric introduces text-to-pipeline capabilities within the Data Factory experience. When we create a new pipeline, we can open the Copilot pane and describe our required workflow in natural language. For instance, we might prompt: “Create a pipeline that copies data from our on-premises SQL Server Customer table into the Lakehouse bronze layer, then triggers the ‘Cleanse_Customers’ notebook, and finally sends an Office 365 email alert upon success or failure.” Upon submitting this prompt, Copilot will automatically place a Copy Data activity, a Notebook activity, and conditional Email activities onto the canvas, automatically linking their success and failure dependencies. We are no longer dragging and dropping boxes endlessly; we are acting as architectural directors, outlining the flow and letting the AI handle the mechanical assembly. We only need to click into the pre-generated activities to select our specific connection credentials. Evaluating Output Quality with Benchmark vs. Human-Written Code As we deeply integrate these AI features into our daily operations, we must maintain a critical perspective. Copilot is an incredibly powerful assistant, but it is not an infallible autopilot. When we evaluate the output quality against benchmark human-written code, we notice distinct patterns. For standard, highly documented operations, such as reading Delta tables, performing standard aggregations, or writing basic DAX time-intelligence functions, Copilot is virtually indistinguishable from a senior engineer. It flawlessly applies best practices and syntax. However, when we introduce highly proprietary, deeply nested, or domain-specific business logic, we must be vigilant against hallucination. Copilot might occasionally assume a column name exists based on standard naming conventions, or it might suggest a PySpark join strategy that is not optimal for our specific data skew. Therefore, we always enforce a rigorous code review process. We treat Copilot’s output as an advanced first draft. We read the generated code, we test it against a sample dataset, and we validate the execution plans. By combining the unprecedented speed of generative AI with the critical thinking and domain expertise of our data engineering teams, we achieve a perfect synergy of velocity and reliability. Conclusion We stand at the forefront of a new era in data architecture. By fully embracing Microsoft Fabric Copilot , we are systematically eliminating the bottlenecks that have plagued data engineering for decades. We are transitioning from writing repetitive ETL boilerplate to orchestrating highly intelligent, automated workflows. Whether we are commanding Copilot to instantly generate complex windowing functions in PySpark, debugging dense execution traces, building text-to-pipeline orchestrations in Data Factory, or writing highly optimized DAX to protect our capacity, we are fundamentally redefining our productivity. We are no longer simply moving data; we are engineering business value at the speed of thought. Hey, I am Sandip Palit , from Kolkata, India. I love to explore what’s new in the Data Science space and share it with the community. I am a Fabric Super User , and in this Microsoft Fabric Playlist, I will share my learnings on Microsoft Fabric and the tips and tricks of using it effectively.. Thank You for reading this article. Please feel free to share your thoughts in the comments section, and give this article a 🌟. Harnessing Fabric Copilot for Automated Data Engineering and DAX Generation was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Engineering a Multi-Agent AI Platform—Part 4: The Night Dreaming Engine
This is Part 4 of a series on engineering AI systems that learn in production. Continue reading on Towards AI »
- On AI memory: from a closed-door discussion
AI memory is moving beyond individual interactions. The next challenge may be deciding who controls the accumulated context of a user's digital life. The post On AI memory: from a closed-door discussion appeared first on MEDIANAMA .
Score: 22🌐 MovesAug 28, 2026https://www.medianama.com/2026/08/223-ai-memory-privacy-data-portability/ - Building the Foundation for the Agentic AI Era
How do we build an AI ecosystem where agents, tools, and systems can work together at scale? Angie Jones, VP of the Agentic AI Foundation, joins Chris to discuss the open standards and projects shaping the agentic future, including MCP, A2A, Goose, etc. They also explore what it takes to drive AI adoption across an entire organization, the importance of neutral standards, global perspectives on agentic AI, and how humans can find the right balance between what they delegate to AI and what they do themselves. Featuring: Angie Jones – LinkedIn , X Chris Benson – Website , LinkedIn , Bluesky , GitHub , X Links: Agentic AI Foundation Sponsors: Framer: The enterprise-grade website builder that lets your team ship faster. Get 30% off at framer.com/practicalai Prediction Guard: A self-hosted AI control plane for running agents in high impact environments. predictionguard.com/practicalai Resources and Events: Register for upcoming webinars here ! Prior Webinars from our partner Prediction Guard Midwest AI Summit 2026
- Look At The Time Magazine Top 100 In AI
TIME’s AI 100 highlights leaders advancing models, hardware, robotics, efficiency, entertainment, and global AI innovation.
Score: 22🌐 MovesAug 28, 2026https://www.forbes.com/sites/johnwerner/2026/08/28/look-at-the-time-magazine-top-100-in-ai/ - Connecting My LangGraph AI Agent to Postgres
How to run the backend locally with Docker or in the cloud The post Connecting My LangGraph AI Agent to Postgres appeared first on Towards Data Science .
Score: 22🌐 MovesAug 28, 2026https://towardsdatascience.com/connecting-my-langgraph-ai-agent-to-postgres/ - I Combined Dense and Sparse Vectors to Search Medical Research
A practical Qdrant experiment on when semantic search helps, when exact terminology matters, and why combining both is useful. Code: GitHub Repository When I started this project, I thought the question was simple: if someone searches biomedical research with a full sentence, surely semantic search should be enough. Then I looked at the kinds of queries doctors and researchers actually write. A query can contain a plain-language request, a disease name, an exact gene symbol, a mutation such as BRAF V600E, and a date restriction at the same time. A search system has to understand the meaning of the sentence without losing the exact notation hidden inside it. That is the experiment I built. I compared three ways to search biomedical papers: Dense search, which turns a query and a paper into vectors so it can retrieve similar meaning. Sparse BM25 search, which rewards useful word overlap and preserves exact terms. Hybrid search, which runs both searches and combines their ranked results. This is a retrieval benchmark, not a diagnosis tool and not a chatbot. The input is a search query. The output is a ranked list of PubMed papers. I deliberately stopped there so I could measure whether the system found useful evidence before a language model or a generated answer could hide a retrieval mistake. Why PubMed makes this harder than ordinary document search PubMed is the US National Library of Medicine’s public catalogue of biomedical research citations and abstracts. It is where developers building literature tools, researchers, and clinicians often start when they need to locate published evidence. The difficulty is that biomedical language mixes several kinds of information: A disease, such as melanoma, which is a type of skin cancer. A gene, such as BRAF or EGFR. Genes are named pieces of DNA that can affect how a cancer grows and responds to treatment. A mutation, such as BRAF V600E, which is a precise change in a gene. In this example, the protein has a different building block at position 600. A clinical-trial ID, often starting with NCT. It is an identifier for a registered study, not a free-form phrase. If someone searches for “treatment evidence for glioma with BRAF,” a useful system should understand the sentence. If they search for KIT L576P, it should not dilute that precise mutation into a generic search for cancer treatment. That tension is why I did not want to test one retrieval method in isolation. The three retrieval methods in plain English A retriever is simply the part of a search system that accepts a query and returns ranked documents. This project has two different retrievers. Dense search: good at meaning Dense search converts text into a list of numbers called a vector or embedding. The idea is that texts with related meaning should end up close together in this numeric space. I used sentence-transformers/all-MiniLM-L6-v2, a lightweight general-purpose text model, to produce the dense vectors. To compare two dense vectors, the system uses cosine similarity. Despite the name, it is just a way to compare the direction of two vectors. A higher score means the model considers the texts more related in meaning. This helps when wording changes. For example, a query that spells out a mutation in descriptive language can still connect to a paper that uses a shorter medical term. Dense search has a weakness: a short identifier can contribute very little to the vector. KIT L576P is only a few characters, but it carries a lot of meaning for the person searching. Sparse BM25 search: good at exact wording Sparse search keeps track of terms rather than representing the whole sentence as one dense vector. I used BM25, a long-established ranking method used in text search. It gives a document more credit when it contains important query terms, while avoiding over-rewarding common words. BM25 is useful here because it keeps exact notation visible. A paper containing BRAF V600E or NCT01234567 is easier to surface when the query uses the same notation. Its weakness is the opposite of dense search. It does not automatically know that “malignant melanocytic tumor” and “melanoma” refer to the same disease. Hybrid search: let both methods vote The hybrid method runs dense and sparse search separately, then combines their lists with reciprocal rank fusion, or RRF. RRF is a simple rule: a paper gets more credit when it appears near the top of either list, and even more credit when both methods rank it highly. This matters because dense and BM25 scores are different kinds of numbers. It would be misleading to add a cosine score directly to a BM25 score. RRF avoids that problem by combining positions in the ranked lists rather than raw scores. Why I chose Qdrant for the experiment I wanted the experiment to stay focused on retrieval instead of building vectorisation, sparse indexing, fusion, and filtering infrastructure from scratch. That is where Qdrant fits well. Qdrant is a vector database. In this project, each paper is stored once as a Qdrant point with two named representations: a dense vector for semantic search and a sparse BM25 vector for exact-term search. The same Python client can query either representation or ask Qdrant to fuse the two ranked lists with RRF. I also needed filters to run during the search. For example, a developer may want papers about a drug or gene, but only after 2020. Qdrant stores these fields as a payload, which is searchable metadata attached to the paper. That lets a query say “find papers similar to this request, but only where the year is 2020 or later and the gene is EGFR.” EGFR is a gene involved in cell growth and is often used as a treatment-relevant marker in cancer research. In short, Qdrant let me keep one corpus, two retrieval methods, metadata filters, and fusion in one small Python pipeline. For a comparison experiment, that made the moving parts easier to reason about and reproduce. The first implementation choice is where Qdrant runs. I kept that decision behind one function, so the same retrieval code can use local storage for reproduction or a Qdrant server when QDRANT_URL is configured. def get_client() -> QdrantClient: url = os.getenv("QDRANT_URL") if url: return QdrantClient( url=url, api_key=os.getenv("QDRANT_API_KEY"), local_inference_batch_size=16, ) QDRANT_PATH.parent.mkdir(parents=True, exist_ok=True) return QdrantClient( path=str(QDRANT_PATH), local_inference_batch_size=16, ) A reader can therefore run the experiment without provisioning a service. Moving the same collection to a server changes the client configuration, not the indexing or retrieval logic. client.create_collection( collection_name="trec_pm_2018_pool", vectors_config={ "dense": models.VectorParams(size=dense_size, distance=models.Distance.COSINE) }, sparse_vectors_config={ "sparse": models.SparseVectorParams(modifier=models.Modifier.IDF) }, ) The dense vector uses cosine similarity. The sparse vector uses BM25-style term weighting with inverse document frequency, which gives rarer terms more influence than common ones. Figure 1. Corpus preparation, two named vector representations, metadata payloads, and the shared Qdrant Query API The dataset: real relevance labels, controlled query wording For the benchmark, I used the TREC 2018 Precision Medicine scientific-abstract task . TREC is a long-running information-retrieval evaluation programme. In this task, precision oncologists created 50 synthetic patient cases, and physicians trained in medical informatics assessed which research records were relevant to each case. The source files are public: TREC 2018 topics : the 50 source patient cases. TREC 2018 abstract relevance judgments : the labels saying which documents were relevant to which case. PubMed : the public source for the title and abstract records fetched by the project. The downloaded corpus is not committed to Git. The repository downloads the numeric PubMed IDs found in the TREC judgment file through the NCBI API and builds the corpus locally. That makes the data path clear without redistributing a large copy of PubMed. The original judgments contain 14,946 numeric PubMed IDs. When I fetched the current title and abstract records, 12,868 usable records were available. The original TREC task also includes conference abstracts from AACR and ASCO. Those are not PubMed records, so I excluded them rather than mixing data sources. I then created seven deterministic versions of each source query. For example, the same case can be expressed as a natural-language question, an exact medical phrase, a query with a synonym, or a query containing an exact mutation. These are controlled rewrites, not new physician-written cases. They reuse the original case’s relevance labels so I can ask one narrow question: what changes when the wording changes but the information needed stays the same? The final evaluation used 210 held-out queries from 30 source cases. All seven rewrites from one source case stayed in the same split, so a near-duplicate version of a test case could not appear in training or validation. The preparation code mirrors the data flow described above. It downloads the official TREC files, parses the topics and relevance judgments, creates the controlled query forms, and extracts only numeric PubMed IDs for the NCBI fetch. download_trec_pm_sources() topics = parse_topics() source_qrels = parse_source_qrels() queries = make_all_controlled_queries(topics) pubmed_ids = ordered_pubmed_ids(source_qrels) records, _ = _fetch_batches( pubmed_ids, batch_size=400, request_delay=0.4, ) The downloader caches every XML batch before parsing it, so an interrupted run can reuse completed downloads. The generated manifest records the source checksums, corpus count, and code version used for the benchmark. Building the index Each paper becomes one Qdrant point. Its title and abstract are joined into the searchable text. The payload keeps readable fields such as the publication year, diseases, genes, drugs, and trial IDs. I used Qdrant Client’s FastEmbed integration rather than writing embedding code by hand. Passing a models.Document object tells the client which model should encode the text for each named vector. point = models.PointStruct( id=paper_id, vector={ "dense": models.Document(text=search_text, model=DENSE_MODEL), "sparse": models.Document(text=search_text, model=SPARSE_MODEL), }, payload=paper_metadata, ) The benchmark does not send all 12,868 papers in one request. It groups the prepared points into batches of 32 and waits for each upsert to finish. points = [make_point(record) for record in records] for batch in batches(points, size=32): client.upsert( collection_name=collection_name, points=batch, wait=True, ) Batching keeps each embedding and upload step bounded, while the completed-document counter makes a long indexing run easier to monitor. That is the whole indexing idea: the same paper gets two ways to be found, plus metadata for filters. Before fusing the rankings, the benchmark runs dense and sparse retrieval separately. Both branches use the same collection, result limit, payload handling, and optional filter. Only the model and named vector change. if mode == "dense": response = client.query_points( collection_name=collection_name, query=models.Document(text=query_text, model=DENSE_MODEL), using="dense", query_filter=query_filter, limit=limit, with_payload=True, ) elif mode == "sparse": response = client.query_points( collection_name=collection_name, query=models.Document(text=query_text, model=SPARSE_MODEL), using="sparse", query_filter=query_filter, limit=limit, with_payload=True, ) Keeping the surrounding query path identical matters for the benchmark: the comparison changes the retrieval representation rather than changing unrelated application behavior. For a hybrid query, I ask for more candidates from each retriever than I plan to show the user. For example, if the final UI needs five papers, pulling 20 candidates from dense search and 20 from BM25 gives RRF a wider set to combine. A paper ranked sixth in both lists can still become useful after fusion. Asking each retriever for only the final five would throw that paper away too early. response = client.query_points( collection_name=collection_name, prefetch=[ models.Prefetch( query=models.Document(text=query_text, model=DENSE_MODEL), using="dense", limit=20, ), models.Prefetch( query=models.Document(text=query_text, model=SPARSE_MODEL), using="sparse", limit=20, ), ], query=models.FusionQuery(fusion=models.Fusion.RRF), limit=5, with_payload=True, ) What I measured The main metric is Recall@20. It asks: of all the papers judged relevant for a query, how many did the system place in its first 20 results? I used 20 because a retrieval layer often hands a small evidence set to the next stage, such as a reranker or an answer-writing model. If a relevant paper is missing from those 20 results, later steps cannot use it. The implementation calculates Recall@20 per query from two sets: every document judged relevant for that query and the document IDs returned in the first 20 positions. def query_recall(ranked, judgments, depth=20): relevant = { document_id for document_id, score in judgments.items() if score > 0 } retrieved = { document_id for document_id, _ in ranked[:depth] } if not relevant: return 0.0 return len(relevant & retrieved) / len(relevant) Those per-query values are averaged for the reported Recall@20. The separate complete-miss count records the harsher case where the intersection is empty. I also counted complete misses: queries where the top 20 had no judged relevant paper. This is simpler to interpret than a single average. If a system gets a high average but completely fails for many real queries, that matters. The benchmark also records additional ranking metrics and latency measurements in the repository. A warm query latency is the time after the model and index have already been loaded into memory. It is useful for comparing repeated local queries, but it is not a production latency claim. Figure 2. Dense and sparse candidates remain independent until reciprocal rank fusion combines their ranks Results: hybrid helped most when a query mixed meaning and notation Here is the held-out Recall@20 result. Higher is better. The headline is not “hybrid wins everything.” It does not. Dense search was strongest when the query was phrased naturally or used synonyms. BM25 was particularly valuable when exact medical terms carried the intent. Hybrid did best overall and was strongest on exact terminology, mutation-like entities, and mixed queries. The practical result was the miss count. Dense had 13 held-out queries with no relevant paper in the first 20. Sparse had 14. Hybrid had 5. One example makes the trade-off concrete. When the system received a descriptive version of a melanoma question that expanded KIT L576P into words, dense search found a relevant paper at rank one. BM25 did not find relevant evidence in its first 20 because the exact notation had disappeared from the query. The opposite happened with the exact mutation form. BM25 placed a highly relevant KIT (L576P) paper first, while dense search returned no relevant paper in its first 20. Neither method was universally better. They failed in different ways. For “What treatment evidence is available for glioma with BRAF?”, a dense and sparse search each found four relevant papers in the top 20, but not the same four. Hybrid found six. That is the useful case for fusion: it expands the evidence set when the two methods bring back different relevant material. Figure 3. Held-out Recall@20 by controlled query category. The winning modality changes with query form Filtering is a separate, useful capability Search meaning and search constraints are different jobs. If a user wants papers about an EGFR-related treatment after 2020, “after 2020” is not something an embedding model should have to guess from the sentence. It is a hard constraint on a metadata field. The project stores structured fields in Qdrant’s payload and applies them at query time. query_filter = models.Filter( must=[ models.FieldCondition( key="genes", match=models.MatchValue(value="EGFR") ), models.FieldCondition( key="publication_year", range=models.Range(gte=2020) ), ] ) Creating the filter is only half of the operation. The same object is passed into the hybrid search, so Qdrant applies the hard constraints while it retrieves and fuses candidates. points = search( client, "treatment evidence for EGFR resistance", mode="hybrid", limit=20, query_filter=query_filter, collection_name="trec_pm_2018_pool", ) The returned points therefore satisfy both payload conditions before they reach any later reranking or answer-generation stage. must mean both conditions must match. The code checks filters one at a time, together, with an impossible value that should return nothing, and with exact-case matching. The point is not that filters are medically intelligent. The point is that once you have reliable metadata, you can keep hard constraints out of fuzzy semantic matching. What this experiment does not prove This is a fair comparison of three retrieval methods on a reconstructed, judged PubMed pool. It is not a full PubMed index, a clinical recommendation system, or a proof that MiniLM is the best model for oncology literature. There are three important limits: The current PubMed records were fetched after the original 2018 task. Some records may be missing or updated compared with the original collection. The seven query forms are controlled rewrites. The original cases and labels were reviewed by experts, but the rewrites were not independently reviewed medical questions. The dense model is a lightweight baseline. A biomedical model, a reranker, or a larger corpus could change the result. Those limits are why I would use this project as a starting point for a literature-retrieval pipeline, not as an end product for clinical decisions. What I would build next The next useful experiment is not to add an LLM immediately. First, I would keep this benchmark fixed and swap one retrieval component at a time: a biomedical embedding model, a reranker that reorders the retrieved papers, or a larger distractor corpus to test scale. Only after measuring retrieval and reranking separately would I put an answer-generation layer on top. That distinction matters: a fluent answer is not evidence that the search system found the right papers. My main takeaway is simple. In biomedical search, users can ask for meaning, exact notation, and hard filters in the same sentence. Dense search and BM25 each cover a different failure mode. Qdrant made it practical to test both representations against the same corpus and combine them without building the search infrastructure from scratch. References Project source code and reproduction instructions TREC 2018 Precision Medicine Track data TREC 2018 Precision Medicine task description PubMed Qdrant Hybrid and Multi-Stage Queries Qdrant FastEmbed Qdrant Filtering I Combined Dense and Sparse Vectors to Search Medical Research was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Before You Build An AI-Powered Enterprise, Build Its Human Foundation
Organizations don’t organically mature into effective AI adoption. At CX Forum West, Colleen Fazio explained how companies can build a solid human foundation in which they can ground their AI initiatives.
Score: 21🌐 MovesAug 28, 2026https://www.forrester.com/blogs/before-you-build-an-ai-powered-enterprise-build-its-human-foundation/ - I turned ChatGPT into my personal development powerhouse — here’s how
This site is my new favorite prompt bank for ChatGPT and beyond.
Score: 21🌐 MovesAug 28, 2026https://www.androidauthority.com/chatgpt-personal-development-aiarion-ai-tool-roundup-3704315/