AI News Archive: July 12, 2026 — Part 3
Sourced from 500+ daily AI sources, scored by relevance.
- Etzioni on AI: Who disagrees with you about AI? Here’s what the research shows
Optimism comes from those with the most to gain, in the rising economies and inside the labs; doubts rise from those with the most to lose or the most to fear. Read More
Score: 42🌐 MovesJul 12, 2026https://www.geekwire.com/2026/etzioni-on-ai-why-trust-in-ai-splits-by-country-gender-age-and-politics/ - The Human Substitution Test as a Sanity Check for AI Evaluations
TL;DR: We suggest a sanity check for proposed evaluation or AI oversight schemes: Imagine the AI was replaced by a competent, strategic human — someone who knows they might get evaluated and has their own agenda. Would the evaluation still work? When we apply this mental move broadly, to all AIs and evaluations at once, we get a rather discouraging picture: The questions we care about the most — such as "Is it safe to give this AI more power?" — correspond closely to questions where we already know that human evaluations are unreliable, often to the point of being so completely hopeless (or costly) that we don't even attempt them. This post is part of a collection of ideas about the limitations of AI oversight . It can be read standalone. The Human Substitution Test Here is a mental move we find useful for thinking about AI: replace the AI with a human and see how your intuitions about the result change. More precisely, imagine replacing the AI by someone who may have goals of their own and who is at least as smart, strategic, knowledgeable, and aware of the context as a competent human. Then ask whether whatever we're considering — a safety measure, a deployment decision, an evaluation — would still make sense. This isn't meant to be an anthropomorphisation. In many ways, AIs behave very differently from humans, and this move tells us nothing about how an AI will behave. What it gives us is a lower bound on the difficulty of the problem: [1] if something wouldn't work against a strategic human, it currently works for AI only because the AI isn't capable enough to defeat it yet . [2] [3] This post is about applying the move to AI evaluations. Evaluations at Escalating Difficulty Let's start with an easy case: Suppose we want to know whether an AI can translate French. We give it test passages and check the results. For humans, the analogy is a translator taking a competency exam. This works — the translator has no reason to hide their abilities. [4] And the same applies to most capability evaluations, for both humans and AI. Now, take a harder case: We want to check whether an AI assistant gives good, balanced advice rather than subtly steering users toward choices that benefit the AI's operator. We review a sample of the AI's outputs and they look fine. For humans, the analogy is an employee whose performance is being reviewed. But humans generally know when their work is likely to get reviewed, and they will be on their best behaviour when it matters. We do get some signal, but it is filtered through the employee's awareness of being watched. And the better they understand the review process, the less informative it becomes. As an ever harder case, suppose we want to test whether an AI will behave reliably when no one is watching — whether it will leak sensitive data, pursue hidden objectives, or abuse access it has been given. For humans, this is analogous to testing whether an employee will steal or whether a contractor will sabotage a project. Crucially, this is something for which we don't even have a direct test. What we do instead is install cameras, separate duties, require co-signatures, and run audits — structural constraints and traceability, not tests of character. The idea of "testing someone's honesty and relying on the result" is not something any serious security practice depends on. Finally, suppose we want to determine whether an AI is safe to deploy at scale — whether it can be trusted with significant autonomy and real-world influence. For humans, the analogy is: can this person be trusted with power? Notably, we generally don't evaluate humans for this: We don't test politicians for corruption before giving them office. We don't test executives for recklessness before giving them control of companies. It just wouldn't work; evaluation as a tool for this job is so obviously insufficient that it barely registers as an option. [5] Instead, we build constitutions, separation of powers, term limits, independent oversight, etc. To be clear about the logic behind "the Human Substitution Test": none of this requires the AI to be human-like [6] . The test's conclusions are worth listening to as soon as we expect to eventually get AI that: (1) can model the fact that it might get evaluated [7] , (2) might have objectives that don't perfectly align with those of the evaluator, and (3) is capable of adjusting its behaviour in response to the fact that the evaluation might exist [8] . Naturally, the test may be even more relevant in the context of AIs that are trained on largely human-generated corpora or trained to imitate human assistants. However, this is by no means required for this test — the properties (1-3) should be expected from any AI that is at least as smart as a human on the relevant dimensions. Most Safety-Critical Evaluations Already Fail for Humans Now zoom out and imagine AI has been deployed across the economy and society, across all the domains where it might be valuable. What are all the evaluations we would like to run? To give a few examples: We would want to know whether AI won't act against the interests of its users when giving medical, legal, or financial advice. Whether it won't leak confidential data. Whether it won't cut corners when managing critical infrastructure. Whether it won't try to bypass guardrails and gradually undermine human oversight. Whether it will try to take control once it is capable of doing so. But consider some human analogies of these questions. Do we evaluate job applicants for "misalignment" that will manifest only after they get the job? Do we ask them things like "Will you steal from the cash register when nobody is looking"? (No; we install a security camera instead.) Do we ask whether they will commit fraud or leak confidential data? (No; we use things like background checks, gradual increases in authority, and structural checks.) Do we have routine evaluations for whether people are prone to lying? (Polygraphs are famously unreliable, and most courts won't even admit them as evidence.) Do we evaluate CEOs for "will they abuse power once they get it"? (Not exactly. ) [9] The evaluations we would most want for AI safety are the kind that already don't work for humans, to the point where, in most cases, we don't even try. Not because we wouldn't value the information, but because the tests obviously wouldn't work against someone who understands what's being tested and has a reason to game it. [10] This pattern of evaluation failures isn't a coincidence. Rather, the evaluations that matter most for safety tend to be about dispositions — what one would do when they aren't being observed, when they have a chance to act against the evaluator's interests. But somebody strategic will simply behave well when they are being watched. (The employee who only steals when the cameras are off. The advisor who gives biased recommendations only when confident the client won't notice. The politician who abuses their power only when they have enough of it to escape the consequences. [11] ) The strategy of behaving well when you might be watched produces an excellent track record, right up until the moment it causes harm. And this is precisely why evaluation-based safety assurances are unreliable in the cases where they matter most. [12] Where the Human Analogy Breaks To be clear, the human substitution test is not perfect. There are important disanalogies between AI and human evaluations. Reasons AI evaluation might be easier than human evaluation: Interpretability. We can inspect the AI's internal reasoning to an extent that would be impossible with human neuroimaging (for practical or moral [13] reasons). This makes AI evaluations vastly more effective than they would be otherwise, and gives those evaluations a chance for success where human evaluations would be doomed to fail. However, there are reasons to think that interpretability is not a silver bullet solution. The closest human analogues — lie detectors, surveillance, mind-reading in fiction — are imperfect and can be defeated by someone who knows about them. Moreover, AI interpretability does not work well enough yet, and it is unclear whether it will catch up in time. (If it did, it could solve a significant fraction of alignment issues — at least in the near/medium term. But will it?) Ultimately, the thoughts of superintelligent AIs might be so far beyond us, and so alien to us, that any attempt to translate them into human language would be so inaccurate as to be useless (including for the purpose of detecting deception, scheming, etc). [14] Reproducibility and non-problematic A/B testing. Unlike with humans, we can re-run the same tests many times or run the AI on multiple inputs, without the AI adapting its behavior to the test based on prior tests. [15] Information control. We can restrict what the AI knows about its situation more tightly than for humans — at least for current systems with limited memory and context; see the point about entanglement below. Fewer constraints from law and ethics. Many of the things we can do in an AI test would be illegal or unethical to do with humans: lying, manipulation, bodily harm (cf. ablation studies), causing emotional distress, etc. [16] Reasons AI evaluation might be harder: A capable AI may be more strategic than humans, be able to deceive better, and pay attention to different things. The human substitution test mostly gives a lower bound on what the AI might do — it cannot rule out the possibility that a sufficiently advanced AIs will use a strategy that would be unavailable to a human. For example, an AI might notice cues about being tested that a human would miss — statistical patterns in inputs, timing anomalies, metadata inconsistencies. Coordination between instances. While we might try to reset the AI between tests, different instances of the AI might coordinate without any explicit communication channel, for example, through persistent effects on their shared environment. And while humans also share information about tests with each other (e.g., websites gathering information on hiring processes at specific companies), the AI might get better at this. Note also that a powerful AI might even find ways to reason consistently across different copies acausally, without external help. Complex tasks might be inherently expensive to test. Valuable tasks require the AI to interact with real-world details (such as conversation history, connected tools, rich databases, and information about the current state of the world). Fabricating a convincing test version of all of this is extremely expensive. (For more context on this, see the post If This Were a Test, How Much Would It Cost? and the related concept of (minimum) "entanglement" .) Since we want the AI to solve problems that are currently beyond us, the AI solving those tasks might eventually be paying attention to an unprecedented amount of details, which might be unprecedentedly expensive to test. Which Types of Evaluation Might Still Work? If straightforward evaluation breaks against strategic AI, what kinds of evaluation might still work? [17] It helps to distinguish two functions that "evaluation" currently serves. One is measurement : how good is this particular AI at translation, coding, or medical diagnosis? Measurement can often work because the system typically has no reason to hide capabilities that we want it to have. The other is oversight : Is it safe to deploy this AI? Can it be trusted with autonomy? Or should we retrain it, recall it, or put more guardrails around it? Oversight is what breaks, because it requires the evaluation to work even when the AI has reasons to game it. [18] What has historically worked for the "oversight" problem with humans are structural approaches: Monitoring and track records — ongoing observation, rather than one-time tests. The rate of error-catching might be low, but if we do it over a longer period, our chances increase significantly. (Though this doesn't prevent long cons, where the final prize is significantly more valuable than what comes before it.) Institutional checks and balances — things like separation of powers, independent oversight, and competing interests that make coordinated deception harder. Somewhat related: Information within institutions is often fragmented, which creates cracks that can be used by external auditors. [19] Incident reporting and accountability — mechanisms that increase the chance that misbehaviour will have consequences, even when it isn't detected in time to prevent the harm. Designing for evaluability — instead of building powerful blackbox AIs to be capable and worrying about evaluation as an afterthought, evaluability could be a crucial consideration that determines which types of AIs we build in the first place. An existing example of this is the plan to keep LLM chain-of-thought legible and monitorable , rather than applying optimisation pressure to it or replacing it with "neuralese". However, we could go much further and actively look for architectures and paradigms that are easier to evaluate than current LLMs. [20] None of these approaches is a silver bullet. Rather, unlike straightforward evaluation, these are tools that have proven to be somewhat robust against strategic behaviour. Vojta: Personally, my preferred approach would be to bide our time with AI progress and solve alignment first . Or focus on automating trusted processes , as opposed to building general black-boxes. (Oh well.) Tomáš: I think that the four points above (along with others) may be significantly helpful, but they need to be combined into a solid construction rather than a pile of sticks. One point of this post is that many pre-deployment tests don't add much beyond a clutter of scattered sticks, and so we should start considering other options. Conclusion AI evaluation currently works tolerably well , creating an expectation that it will continue to be useful as AI becomes more capable. The human substitution test suggests this won't pan out: most of the evaluations that matter most for safety are operating in a regime where the human analogues have already failed, or where we never even attempted them in the first place. This doesn't mean evaluation is useless: Measurement continues to work. Cooperative evaluation — where the AI benefits from demonstrating trustworthiness — may also survive. And evaluation will remain useful when combined with structural constraints. But we should not use evaluation as the first line of defense — or worse yet, as the only line of defense — because that use case will eventually stop working. Where by "eventually", we mean "much sooner than many people expect". The following posts develop the structural argument: the concrete cost-of-testing argument ( If This Were a Test, How Much Would It Cost? ), and the concept of entanglement that underlies it ( Entanglement ). On LLM use, and how this post came to be: I (Vojta) came up with ideas, discussed them extensively with Tomas, and wrote the initial draft. Which was then rewritten and commented on by Claude Opus 4.7. I went through the post (way too many times) and edited all the parts I didn't fully endorse. Finally, Tomas and Mateusz gave feedback on the post, and I rewrote all the parts that were automatically flagged as LLM written. (Which isn't how it's supposed to work, and I should just flag the whole thing as being LLM assisted. But I only learned about the LW LLM use policy after this was written, and I dislike the LLM-block font, so whatever.) ^ Yudkowsky's AI in the box experiment is an example of this move. ^ Strictly speaking, the "human substitution test" is a conditional: if at some point AI becomes capable enough, evaluation will stop being informative. But given how hard we are trying to build AIs that are capable and autonomous, it might be fair to view this as when rather than if . Note also that this move seems to have a decent track record — better, at least, than the heuristic that "the AI is just a non-agentic tool." ^ The move also doesn't imply that the AI in question will behave undesirably. After all, the AI might be aligned, in which case it won't have a reason to game the evaluation despite having the ability to do so. However, this response goes counter to the reason why we are doing evaluations in the first place — not being sure which AI we are dealing with, and generally making sure things go well in all worlds, rather than just in the more convenient ones. ^ Unless they're sandbagging — deliberately underperforming to seem less capable, for example. Humans frequently do this to avoid being given harder work, and we have seen AIs do this in the lab to avoid raising suspicions. This shows that it matters not only what sort of evaluation we are running, but also why we are running it. ^ We do use some evaluation-adjacent processes — background checks, security clearances, vetting. But these primarily check history and associations, not dispositions. And even these are routinely defeated: people with security clearances have sold state secrets throughout history. ^ Or have any philosophically controversial properties like consciousness. ^ It is not necessary that the AI can tell when exactly it is getting evaluated and when not, only that it understands that the evaluation might happen in general. For more discussion, see our earlier post Deployment Awareness Matters More Than Evaluation Awareness and the section on "Evaluation gaming with no evaluation or deployment awareness". ^ This third part is where tools like interpretability and chain-of-thought monitoring become important. When you ask current LLMs about how they would scheme if they were misaligned, they often mention that scheming is hard because these tools make you likely to get caught. In other words, LLMs would, in principle, be able to come up with strategies for defeating evaluations, but these tools make it hard to actually adopt those strategies. That said, relying on the assumption that AIs will never be able to think unmonitored thoughts seems reckless. (See Circumventing interpretability: How to defeat mind-readers .) ^ Technically, we can sometimes evaluate an applicant based on their track record. Unfortunately, in the case of AI evaluations, we often do not have a representative track record. ^ In an earlier draft, we also listed the example of not evaluating political candidates for honesty. (Though current politics has bigger issues than that.) We removed the example from the main text because it feels like too much of a distraction. However, this example points to an interesting minor flaw in my argument: We suspect that if we evaluated political candidates for a bunch of things, such as dark triad traits and basic numeracy, this would improve matters. Obviously, this would be noisy and problematic (who decides what the content of the test is?). People would game it a lot, and powerful people might try to bypass it. That said, the fact remains that this example disproves my claim that most of the evaluations we could run on humans are not used because they wouldn't work . On the other hand, this example does support a broader point, that when it comes to strategic entities, evaluations are not a very reliable oversight method. ^ The pattern isn't limited to humans. Volkswagen programmed its cars to detect emissions tests and reduce pollution only during testing , allowing them to pass evaluations while exceeding legal limits multiple times in normal driving. (Incidentally, this gives an example of the evaluator getting deceived without any "deliberate deception" by the evaluated entity. Which illustrates that some of the philosophical discussions around intentionality are orthogonal to questions about evaluation gaming.) ^ The "play nice, then act" pattern is one of several counter-strategies available to a misaligned AI. The human analogy makes it especially vivid: it's not a theoretical concern — it's standard human behaviour in adversarial contexts. ^ There's one genuinely novel advantage: if we're confident an AI doesn't have moral status, we can test it more intrusively than we could a human — isolation, repetition, adversarial probing that would be unacceptable to do to a person. This advantage erodes if the AI does have morally relevant properties. ^ For a more detailed list of concerns about interpretability, see Lee Sharkey's Circumventing interpretability: How to defeat mind-readers . ^ Whenever some type of testing becomes a routine part of AI R&D, the AI soon learns this fact as well, by virtue of having read all of the internet. And this will also become a problem here. However, we can at least run A/B tests on a single AI whose memory we reset, instead of having to do so on large sample sizes as we do with humans. ^ This does not make the human substitution test inapplicable: While some AI tests may be illegal to perform on humans, we can still use the human substitution test within a thought experiment, and if we expect the human to foil the test, we can still conclude a capable AI would as well. ^ An obvious and reasonable reaction is that we should just look for alternative approaches that don't rely on observation-based oversight. For example, find some other paradigm than LLMs, something that would be inherently more amenable to science and less reliant on trial and error. But within the context of this post, we will assume that we want to stick with the current paradigm and rely on evaluation. ^ But what if we use the measurement data for oversight? For example, what if we take a genuine measurement of the AI's hacking abilities, then use it to decide whether the AI still seems safe or not? In case this still seems like a viable idea, recall the Human Substitution Test: "Would this strategy seem reliable against a strategic human?" (The answer: No.) ^ An individual human has unified awareness and can coordinate their deception seamlessly. An institution has departments with different information and conflicting interests. AI systems could go either way — but optimisation pressure toward unified awareness would erode these advantages, much as companies under regulatory scrutiny learn to route communications through legal and PR departments, and train employees on what to say to auditors. ^ For example, cognitive emulations, or training narrower models and having a cleaner separation of powers. We are more confident about the claim that "evaluability as a consideration on par with capability" is a good idea and solutions to it will exist, than about any specific existing approaches being the ultimate solution. Discuss
Score: 41🌐 MovesJul 12, 2026https://www.lesswrong.com/posts/B66gAyzwL8Aph6F8u/the-human-substitution-test-as-a-sanity-check-for-ai - Toll Group puts third-party risk at centre of AI-era data security
Redefining the data protection supply chain.
- Where should the boundary exist between AI governance and human governance in enterprise Confluence?
Where should the boundary exist between AI governance and human governance in enterprise Confluence? Atlassian Community
- DPO Fine-Tuning from First Principles in Python
Fine-tuning a Large Language Model (LLM) with human preferences used to require Reinforcement Learning from Human Feedback (RLHF): collect… Continue reading on Towards AI »
- Can a Prettier Data Center Curb the Community Backlash?
Architects plan data centers to resemble tech campuses or art museums, rather than bland boxes, in an effort to soothe local opposition.
- You Could Be Eligible for Part of Apple's $250M AI iPhone Settlement. How to Find Out
Apple must pay iPhone owners to settle a lawsuit over delayed and missing AI features.
Score: 40🌐 MovesJul 12, 2026https://www.cnet.com/tech/mobile/how-to-claim-apple-250-million-ai-iphone-settlement/ - How should organizations measure knowledge trustworthiness when using Rovo-generated content?
How should organizations measure knowledge trustworthiness when using Rovo-generated content? Atlassian Community
- Digitide's Malhotra on why the next AI advantage won't come from better models, but better execution
Digitide's Malhotra on why the next AI advantage won't come from better models, but better execution Techcircle
- RAG vs Fine-Tuning Explained: What They Actually Do and When to Use Each
Two techniques, two different problems, and why the question is not really "which one wins" The post RAG vs Fine-Tuning Explained: What They Actually Do and When to Use Each appeared first on Towards Data Science .
Score: 38🌐 MovesJul 12, 2026https://towardsdatascience.com/rag-vs-fine-tuning-explained-what-they-actually-do-and-when-to-use-each/ - Faith Tech: Pat Gelsinger steers Gloo’s platform to lead faith-based organizations into the age of AI
After eight years as the chief executive of VMware Inc. and nearly four more leading Intel Corp., Pat Gelsinger suddenly found himself retired. Then the phone rang. “Less than two seconds after the Intel departure was announced, Scott called,” Gelsinger (pictured) recalled. “Whether it was opportunistic on his part or God ordained on his part […] The post Faith Tech: Pat Gelsinger steers Gloo’s platform to lead faith-based organizations into the age of AI appeared first on SiliconANGLE .
- Jacksonville Highway Near-Miss Highlights AI Dashcam Role in Commercial Fleet Safety
Jacksonville Highway Near-Miss Highlights AI Dashcam Role in Commercial Fleet Safety azcentral.com and The Arizona Republic
- How Monash University is tackling the AI-driven app security gap
Human connection is key to overcoming AI challenges.
- AI Agent Production Deployment Best Practices
Production Deployment Patterns for AI Agent Systems: From Prototype to Scale When I first built an AI agent, it felt like magic, a single script that could answer a question, call a tool, and return a result. But as soon as I tried to run that agent in a real user-facing environment, the pain points exploded: flaky dependencies, runaway resource usage, and the terrifying “what happens when the agent decides to loop forever?” moment. The shift from a throw-away prototype to a reliable AI agent production deployment forced me to think like a systems engineer, not just a model tinkerer. Suddenly I was asking how the pipeline would handle a new version without breaking the external APIs the agent depends on, how I’d know when a decision went wrong, and what I could actually roll back when things break. This article walks through the patterns that have saved me (and my teammates) from those nightmares. I’ve spent the last year shipping conversational assistants, data-extraction pipelines, and multi-step reasoning loops into production at a mid-size SaaS company. The lessons below come from dozens of post-mortems, continuous-deployment runs, and heated stand-up debates about whether horizontal scaling was the right answer. If you’re moving beyond a Jupyter notebook, keep reading. CI/CD Pipeline Design for AI Agent Production Deployment In a prototype you might hit a button and the agent redeploys instantly. In production you need an atomic, version-controlled pipeline that treats the whole multi-agent workflow as a single deployable unit. Versioning the Workflow Graph Our agents are orchestrated as a directed acyclic graph (DAG) of steps, each step being a container that runs a specific tool. When we bump a step — say, upgrade the PDF parser — we must ensure the new version is compatible with all downstream nodes. We encode the DAG as a JSON manifest and store it in the same repository as the code. A CI job validates that the updated manifest still references existing Docker images and that the new images pass unit tests. stages: - validate - build - test - deploy validate: image: python:3.11 script: - pip install -r requirements.txt - python -c "import yaml; yaml.safe_load(open('workflow.yaml'))" rules: - if: $CI_COMMIT_BRANCH build: image: docker:latest services: - docker:dind script: - docker build -t registry.example.com/agent-service:$CI_COMMIT_SHA . - docker push registry.example.com/agent-service:$CI_COMMIT_SHA only: - main test: image: python:3.11 script: - pytest tests/ only: - merge_requests deploy: image: amazon/aws-cli:latest script: - aws ecs update-service --cluster prod-cluster --service agent-service --force-new-deployment only: - main Notice the “force-new-deployment” flag — it guarantees that every task gets a fresh container image, eliminating the subtle bugs that arise from stale caches. Atomic Deployment of Multi-Agent Workflows When you have three agents A, B, and C, executing in sequence, you cannot roll out B without breaking the contract between A and B. Our solution uses a blue-green style swap where the entire graph is redeployed under a new service name, then traffic is switched via a load balancer. If the health check fails, the old version stays live. This approach gave us zero-downtime releases for a chatbot that processes 12k requests per minute. Takeaway : Treat the whole workflow as a single versioned artifact. Automate validation before you ever touch production. Monitoring and Observability Patterns for AI Agent Production Deployment In a traditional micro-service you monitor CPU, latency, and error rates. With AI agents you must also watch decisions, the outputs that drive downstream actions. Tracing Agent Decisions and Tool Calls We integrated OpenTelemetry into each agent container and instrumented the tool adapters (e.g., HTTP client, database driver) with custom span attributes that capture the agent’s prompt, the retrieved context, and the final response. This lets us query a Grafana dashboard and see, for a given user query, exactly which step produced an out-of-bounds value. import opentelemetry.trace as trace def call_external_api(url, payload): tracer = trace.get_tracer(__name__) with tracer.start_as_current_span("external_api_call"): span = trace.get_current_span() span.set_attribute("ai.agent.prompt", payload["prompt"]) span.set_attribute("ai.agent.context", payload["context"]) # actual HTTP request... The result? We caught a subtle bug where an agent was feeding HTML markup into a markdown renderer, causing downstream parsing errors. Without those span attributes we would have only seen “500 Internal Server Error” and no clue why. Beyond traces, we log structured JSON that includes the full request/response cycle. This made searching for a specific error message across millions of logs trivial. AI generated image Takeaway : If you can’t see what the agent did at each step, you’re flying blind. Invest in tracing early, even when the prototype feels small. Rollback and Recovery Strategies for Failed Agent Actions Failure modes in an AI agent system are rarely just “HTTP 500.” They can be corrupted state, runaway loops, or an agent that decides to invoke a prohibited tool. State Snapshots and Idempotent Actions We persist the intermediate state of each workflow step in a durable store (e.g., DynamoDB). Every step checks the version of its input before proceeding. If a step detects that the persisted state hash doesn’t match the expected value, it aborts and returns a structured error to the orchestrator. The orchestrator then triggers a rollback to the last known good snapshot. Because our agents are designed to be idempotent, re-executing the same tool with the same input yields the same output, we can safely retry a failed step without creating duplicate side effects. Automatic Circuit Breaking When an agent repeatedly fails a health check, we open a circuit breaker in the orchestrator. The breaker trips after three consecutive failures and redirects subsequent requests to a “fallback” agent that simply returns a safe default response. This prevents a cascade of errors from taking down the whole service. During a quarter-long incident, the circuit breaker saved us from a full outage when a third-party weather API started returning stale data. Takeaway: Build rollback as a first-class concern, not an afterthought. Combine persistent snapshots with idempotent designs and circuit breakers for robust recovery. Scaling Patterns: Horizontal vs. Vertical Scaling for Different Agent Architectures Scaling an AI agent workload isn’t one-size-fits-all. The architecture of each agent determines whether you can scale out with many cheap instances or scale up with a few powerful ones. Horizontal Scaling for Stateless Reasoning Agents Our “question-answering” agent is stateless: it receives a prompt, calls a retrieval service, and returns a response. This makes it a perfect candidate for horizontal scaling. We run it behind a Kubernetes Horizontal Pod Autoscaler that scales based on request latency. In practice we run 10–20 pods during peak hours, each pod isolated in its own namespace to avoid noisy-neighbor issues. Vertical Scaling for Stateful Planners The “trip-itinerary planner” agent maintains a conversation history and a mutable task queue. Because it holds session-specific state in memory, we cannot simply add more replicas; instead we assign each planner a dedicated node with more CPU and memory. We use sticky session affinity in the service mesh to ensure subsequent requests hit the same pod. This pattern trades horizontal simplicity for predictable performance. Choosing the wrong model once led us to overscale a planner, racking up $2,300 in unnecessary EC2 costs in a single month. Takeaway : Match the scaling strategy to the agent’s statefulness. Use horizontal scaling for stateless steps and reserve vertical scaling for stateful, memory-heavy components. Resource Isolation Techniques to Prevent Agent Resource Contention Running dozens of agents on a shared Kubernetes cluster can lead to CPU throttling, memory starvation, and noisy-neighbor problems. CPU and Memory Requests/Limits Every container definition includes explicit requests and limits. For a lightweight retrieval agent we set requests to 100mCPU and limits to 500mCPU. For a heavy reasoning agent we request 2CPU and limit to 8CPU. resources: requests: memory: "256Mi" cpu: "200m" limits: memory: "1Gi" cpu: "2" These limits prevent a runaway agent from consuming the entire node and starving others. Namespace-Level Quotas We group agents by business domain into separate namespaces and apply ResourceQuota objects. This caps the total CPU and GPU resources that any group can request across the cluster, ensuring that a sudden surge in one domain doesn’t cripple another. Quotas also force teams to think carefully about capacity planning, which reduced “resource-exhaustion” tickets by 60% in six months. Takeaway : Isolation isn’t optional once you move beyond a single-node prototype. Explicit limits and quotas keep the cluster healthy. Health Checks and Circuit Breaker Implementations for Agent-Based Microservices Health checks are more than a simple “/ping” endpoint. For an AI agent you need to verify that the model is responding sensibly, that the tool adapters are reachable, and that the internal state machine is in a consistent state. Custom Health Endpoints Each agent exposes a /healthz endpoint that returns JSON with three fields: ready, error, and state_hash. The orchestrator polls this endpoint every 15 seconds. If the ready flag is false or the error field contains “loop_detected”, the pod is marked unhealthy and restarted. @app.get("/healthz") async def health(): if not agent.is_ready(): return {"ready": False, "error": "model_load_failed"} return {"ready": True, "error": None, "state_hash": agent.state_hash()} Circuit Breaker Pattern with Exponential Backoff We wrap outgoing calls to external APIs with a library that implements the classic circuit breaker. It opens after three consecutive failures and stays open for a configurable cool-down period (starting at 5 seconds, doubling each time). While open, requests fall back to a cached response or a safe default. Implementing this pattern prevented a cascade failure when a third-party sentiment analysis service went down; the agent simply switched to a neutral response and continued processing other requests. Takeaway : Build health checks that reflect the agent’s internal health, and pair them with circuit breakers to protect the system from downstream outages. Deployment Checklist Validated Against Real-World Agent Failures Before any new version leaves the CI pipeline, we run through a 12-item checklist that has caught dozens of production bugs. Artifact provenance: Verify the Docker image SHA matches the build tag. Environment variable sanity: Ensure no secret is hard-coded in the manifest. Prompt version guard: Confirm the prompt hash hasn’t changed unintentionally. Tool contract test: Run integration tests against each downstream tool. State migration script: If the persisted schema changes, run a dry-run migration. Load test with recorded traffic: Replay a production trace to verify latency. Safety guardrails review: Check that no prohibited tool can be invoked. Rollback plan: Document the exact kubectl or Terraform command to revert. Observability verification: Confirm trace and metric pipelines are receiving data. Circuit breaker dry-run: Simulate a failure and watch the fallback behavior. Resource quota compliance: Ensure new requests stay within namespace limits. Authorized sign-off: Obtain approval from the reliability engineer on call. Running through this checklist turned a potentially catastrophic deployment — where an agent started calling the billing API instead of the analytics API — into a routine release. Takeaway : A reproducible, automated checklist is the safety net that lets you move fast without breaking production. Summary Moving an AI agent from prototype to production-ready service forces you to consider pipelines, observability, rollback, scaling, isolation, health, and validation. The patterns above reflect what has worked for me and my team, but they aren’t silver bullets. Each system has its own constraints, and you’ll need to adapt these ideas to your context. Start by instrumenting every agent with tracing and structured logs. Write a health endpoint that tells you more than “up”. Design your CI/CD pipeline to treat the whole workflow as a single versioned artifact, and enforce atomic deployments. Finally, adopt a concise deployment checklist and treat failures as learning opportunities rather than incidents to sweep under the rug. What scaling pattern has worked best for your stateful agents? Have you ever had to roll back an AI agent because it started looping? Share your experiences in the comments — let’s learn from the real-world scars we all collect. AI Agent Production Deployment Best Practices was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Query Fan-Out Framework Targets LLM Visibility Across AI Engines
Query Fan-Out Framework Targets LLM Visibility Across AI Engines azcentral.com and The Arizona Republic
- Should Confluence optimize for information retrieval or knowledge retrieval in the era of Rovo AI?
Should Confluence optimize for information retrieval or knowledge retrieval in the era of Rovo AI? Atlassian Community
- Meet the Floating Robot Companion Designed for Safe, Friendly Human Interaction
Robotics researchers are trying to prove that lighter-than-air robots could excel at emotional connection.
Score: 34🌐 MovesJul 12, 2026https://www.cnet.com/tech/floating-robots-safe-friendly-human-interaction/ - Should Rovo evolve toward collaborative multi-agent orchestration instead of isolated AI agents?
Should Rovo evolve toward collaborative multi-agent orchestration instead of isolated AI agents? Atlassian Community
- Strategies: Five upskilling practices for women facing AI disruption
Women remain overrepresented in administrative roles most vulnerable to automation. The upskilling conversation needs to move beyond mastering prompts.
- How to Make Your Website Usable by AI Agents with WebMCP
I built a hotel booking demo to show how a LLM can discover website tools, call application actions, and update the interface through… Continue reading on Towards AI »
- AI-Powered Baby Products and Sustainability Continue to Shape the Global Baby Market
AI-Powered Baby Products and Sustainability Continue to Shape the Global Baby Market USA Today
- I spent a fortune on a Copilot+ PC, and I’ve barely ever touched Microsoft’s AI
My Copilot+ PC has dedicated AI hardware and its own Copilot key, yet Microsoft’s assistant has barely earned a place in my daily workflow.
- Could AI eventually create technical debt within Confluence knowledge bases?
Could AI eventually create technical debt within Confluence knowledge bases? Atlassian Community
- Claude Fable 5 stays free for paid users until July 19 as Anthropic buys more time
Anthropic has just extended access to Claude Fable 5 for paid subscribers until July 19, giving you another week to keep using the most powerful model. [...]
- Atlassian AI vs. Rovo
Atlassian AI vs. Rovo Atlassian Community
Score: 28🌐 MovesJul 12, 2026https://community.atlassian.com/forums/Rovo-articles/Atlassian-AI-vs-Rovo/ba-p/3260797 - HBMSU expands academic and research partnerships in smart learning, AI, innovation, and business incubation
During an Official Visit to Germany
- Darren Aronofsky's '1776' AI Video Series Is Unhinged, and I Can't Look Away
Commentary: As generative AI continues making inroads into the world of creatives, what are we to make of the increasingly bonkers On This Day...1776?
Score: 27🌐 MovesJul 12, 2026https://www.cnet.com/tech/services-and-software/darren-aronofsky-on-this-day-1776-ai-series-midseason-review/ - How to use AI as a job interview coach? Key prompts that can help you prepare better
How to use AI as a job interview coach? Key prompts that can help you prepare better
- Soft Flying Robots Just Want to Be Friends video
A team of researchers has demonstrated the role that floating robots inspired by Tinker Bell, Pokemon and Studio Ghibli might be able to play in our lives.
- Questions and answers around Rovo
Questions and answers around Rovo Atlassian Community
Score: 25🌐 MovesJul 12, 2026https://community.atlassian.com/forums/Rovo-questions/qa-p/rovo-atlassian-intelligence-questions - Three Months Into DreamStudio, DreamSofa Says Its AR/3D Tool Is Changing How Shoppers Buy Custom Furniture
Three Months Into DreamStudio, DreamSofa Says Its AR/3D Tool Is Changing How Shoppers Buy Custom Furniture azcentral.com and The Arizona Republic
- I tried ChatGPT’s 2023 ‘Caveman Prompt’— here is the one thing it still does better than most prompts
I tried ChatGPT’s 2023 ‘Caveman Prompt’— here is the one thing it still does better than most prompts Tom's Guide
- I tried to parody the most absurd AI products, but the tech industry beat me to it
I tried inventing AI gadgets too absurd for the tech industry. Then I found toothbrushes, litter boxes, headphones, and appliances that had already beaten me there.
- Lorde says Ray-Ban Meta AI glasses are ‘not sexy’
Lorde was performing at the Real Cool Festival in Madrid on Thursday and took some time during her set to speak out against AI glasses. While she didn't specify any brands in particular, it's likely she was taking a shot at festival sponsor Ray-Ban, which has collaborated with Meta on a pair of AI smartglasses. […]
Score: 12🌐 MovesJul 12, 2026https://www.theverge.com/ai-artificial-intelligence/964539/lorde-says-ray-ban-meta-ai-glasses-are-not-sexy - How to remotely log out of your ChatGPT account on all your devices
How to remotely log out of your ChatGPT account on all your devices Tom's Guide
Score: 07🌐 MovesJul 12, 2026https://www.tomsguide.com/ai/theres-a-way-to-sign-out-of-your-chatgpt-account-on-other-devices-heres-how - AI agents | Articles, Insights & Updates | Fortune | Page 26 of 9
AI agents | Articles, Insights & Updates | Fortune | Page 26 of 9 Fortune
- Trinidad and Tobago signs agreements with US companies paving way for data centers in the Caribbean
Data centers could account for nearly 3% of the world’s projected electricity use by 2030, according to a recent United Nations University report
- Caribbean nation becomes first to sign deal with US companies for AI data centers
Caribbean nation becomes first to sign deal with US companies for AI data centers
- 😹 Apple is suing OpenAI
PLUS: Meta pulled an Instagram AI feature, OpenAI is hiring for families, and AI rebrands lost their shine.
- Apple bites OpenAI with lawsuit
Apple bites OpenAI with lawsuit Boston Herald
- Apple sues OpenAI and two former employees for trade secrets theft
UPDATE 5-Apple sues OpenAI, two former employees for trade secrets theft
- How Apple, OpenAI went from working together on AI to fighting over trade secrets
How Apple, OpenAI went from working together on AI to fighting over trade secrets
- ‘Elon is obsessed with me again’: Sam Altman hits back as Musk invokes Apple lawsuit against OpenAI
OpenAI CEO Sam Altman and SpaceX's Elon Musk engage in a social media dispute following the launch of GPT-5.6, with Musk targeting Altman in memes.
- India's Tata Consultancy Services plans up to 8,900 AI deployment engineers, seeks AI acquisitions
India's Tata Consultancy Services plans up to 8,900 AI deployment engineers, seeks AI acquisitions
- Tata Consultancy Services plans up to 8,900 AI deployment engineers, seeks AI acquisitions
The strategy emerges amid investor concern that AI could disrupt India's $315 billion IT services industry by reducing demand for engineering teams, shortening project timelines and squeezing prices as clients seek a share of productivity gains
- Zhipu’s founder says frontier AI should stay open to everyone. His own government may disagree.
The founder of China’s most prominent AI lab has made an unambiguous case for openness. Frontier AI should stay broadly accessible rather than controlled by a select few, Zhipu’s Tang Jie wrote in an internal memo reviewed by Bloomberg. His argument inverts the usual security logic. Real safety comes from broad participation, sharing, and oversight, he […] This story continues at The Next Web
- Zhipu founder backs open-source AI as global security debate intensifies
Founder Tang Jie said frontier AI should remain widely accessible under open-source principles, arguing that transparency and broad participation offer stronger safeguards than restrictions
- Gymsly
Gym management software for the modern world
- In-Hand Salary Calculator
Calculate your actual take-home salary after PF.
- Rovo
Rovo Atlassian Community