AI News Archive: August 28, 2026 — Part 6
Sourced from 500+ daily AI sources, scored by relevance.
- Cybersecurity Researchers Uncover Flaw in Google AI Coding Tool
Cybersecurity Researchers Uncover Flaw in Google AI Coding Tool DevOps.com
Score: 41🌐 MovesAug 28, 2026https://devops.com/cybersecurity-researchers-uncover-flaw-in-google-ai-coding-tool/ - Seattle startup Arkero expands English soccer reach, landing historic club as latest AI customer
The agreement builds on momentum for Arkero, which was launched last fall by the co-founder of Seattle digital remittance company Remitly and raised $6 million at the start of this year. Read More
- Sigenergy Delivers Strong H1 2026 Performance as AI-Driven Energy Innovation Powers Global Growth
Sigenergy Delivers Strong H1 2026 Performance as AI-Driven Energy Innovation Powers Global Growth Toronto Star
- Need to know: How Webflow keeps secrets out of agent context
An AI agent nearly leaked an AWS token during a debug session. Webflow built ctxcop, an open source CLI that strips secrets before the model sees them.
- Slack - Qualcomm AI Hub
Slack - Qualcomm AI Hub Qualcomm AI Hub
- The Dynamics of Intelligence Explosions
Toby Ord [1] Abstract AI is increasingly being used to help with AI R&D. Under certain conditions this feedback loop might be able to produce an intelligence explosion, with rapidly escalating AI capabilities. I explore the mathematics of the most explosive possibilities, with an eye to understanding what drives the dynamics. I show that singular growth (towards a vertical asymptote) is harder to achieve than would be expected from recent economics-inspired modelling, and that there is an important but neglected class of growth rates that are faster than exponential but don't lead to a vertical asymptote. I draw out the generation time (the time to go around the feedback loop) as a neglected parameter that plays a pivotal role in determining the behaviour of any intelligence explosion — one cannot have singular growth unless the generation time rapidly approaches zero. Keywords: recursive self-improvement, RSI, intelligence explosion, explosive growth, finite time singularity, generation time. Preview of Figure 1. A vertical asymptote requires the gradient (rise over run) to approach infinity within a finite time. Decreasing the run is key. It cannot be achieved with a fixed feedback generation time (centre) no matter how quickly the improvements grow, but can occur when the generation time approaches zero (right) even with a fixed additive improvement. The Possibility of an Intelligence Explosion AI can be applied to automate many different things. One of those is the R&D that goes into building better AI systems. There have already been minor examples of partially automating this process, such as using AI to: find better optimisation algorithms for training neural networks (Andrychowicz et al. 2016), design a more efficient matrix multiply algorithm (Fawzi et al. 2022), help write the code for new AI systems (Anthropic 2026), and run experiments to iteratively improve AI systems (Karpathy 2026). Leading AI companies are increasingly talking about using an AI system to do more and more of the work of designing and building its successor — something they hope (and fear) could lead to a radical acceleration in the rate of progress in AI (Hassabis et al. 2026). I. J. Good (1965) introduced the idea of AI systems increasing their own intelligence in what he called an 'intelligence explosion': Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind... Implicit in Good's intelligence explosion is the idea that this process of designing a better machine can be iterated. can create , which can create , and so on. Each system is getting better at intellectual activities — including that of AI R&D — such that each successive system should become ever more intellectually capable. Following the recent literature, I'll refer to this process as recursive self-improvement, or RSI (Yudkowsky 2001). I use this term very inclusively, to cover situations where humans are still doing almost all of the work through to situations where AI is doing most of the work — or even all of the work. I include cases where the AI is taking its own files as input and improving them (self-improvement), as well as cases where it is building an entirely new AI system. And I include cases where the intelligence level of successive systems 'explodes', as well as cases where it merely improves the speed of progress by some multiple, or where the gains quickly fizzle out. By accelerating the already rapid progress in AI capabilities, RSI might be extremely dangerous. One reason is that it would likely speed up the risk-creating processes relative to risk-reducing ones (Vaintrob & Cotton-Barratt 2025). For example it may speed up the development of AI capabilities relative to: technical AI safety research, deliberation at the company, and society's ability to understand and respond. RSI could also provide the opportunity for a single unaligned AI to deliberately corrupt all subsequent AIs. And by enabling larger jumps in capabilities per model release, RSI would remove any opportunity society has to learn from ill effects of intermediately powerful AIs. Finally, by widening the capability gap between a leading AI system and a rival that is a few months behind, RSI would increase the likelihood of a winner-take-all dynamic. These could provide strong reasons against pursuing RSI. This might warrant legislation restricting it, industry best practices restricting it, or internal company policies restricting it. Such restrictions could take many forms including: careful monitoring, retaining meaningful human control, speed limits on capability increase per month, enforced pauses when various capability levels are reached, or bans on certain approaches to RSI. My focus here, however, is not on risks (nor responses to risks), but on understanding the dynamics of RSI. In particular I aim to improve our understanding of the qualitatively different kinds of explosive growth that could occur, what conditions produce them, and how likely they are to be possible. While it is entirely possible that RSI might lead to interesting and important growth in AI capabilities that fall short of explosive progress, my focus in this paper is on better understanding the most extreme kinds of growth. I'll show: how a common approach to modelling RSI conflates two very different kinds of explosive growth, how consideration of the physical duration of the feedback loop (the generation time) is key, how a super-exponential period of progress would likely end when this generation time can no longer be reduced, and how this makes growth with a vertical asymptote somewhat less likely. Modelling RSI through Differential Equations A number of recent articles have attempted to mathematically model the possibility of explosive growth from RSI (Aghion et al. 2017, Erdil et al. 2024, Eth & Davidson 2025, Kokotajlo & Lifland 2025). A common approach is to use differential equations connecting the rate at which the AI's capability increases to its current level of capability. This captures the idea that the amount by which a system can improve its capability in the next time period depends on its current level of capability. While differential equations aren't the only way to model RSI, they are a logical choice, being the standard way to model feedback loops in physics, engineering, and economics. Let's define to be some measure of the cognitive capability of an AI system (we'll say more about which measure later). Adopting the Newtonian notation for differential equations, we use to represent the derivative of with respect to time. This is a more compact notation for — the rate at which is changing per unit time. Let's start with what may be the most well-known differential equation of all: [2] (1) This says that the rate of change of is in direct proportion to its size. For example, a simple model for the rate of change of a population of animals is that it is in direct proportion to the number of animals, with set by the difference between the birth rate and the death rate. Solving this differential equation to find how evolves as a function of time famously gives exponential growth (starting from some initial population, ): (2) But what if doesn't vary in direct proportion to ? What if there are increasing returns, such that doubling more than doubles ? Or decreasing returns, such that doubling less than doubles ? Economists typically allow for this by raising to some power, . (3) This is a flexible approach, covering many different regimes of growth. When , this is the same equation as before, producing exponential growth. When there is linear growth. When growth is sub-linear. When , grows faster than linear, but slower than exponential (e.g. gives quadratic growth). And finally, when , grows more quickly than an exponential. In particular, it grows as a hyperbola, reaching a vertical asymptote at some finite time . This means that as time increases towards , grows explosively, with every finite level of being exceeded prior to time . In such cases it is said that has a mathematical singularity at . This term 'singularity' has been adopted by futurists to refer to various kinds of pivotal moment in the future of technology — often with an almost mystical undertone. But for the purposes of this essay, it simply means that the mathematical model of how changes with time has a vertical asymptote at some particular future time . Economists often use formulas like (3) in endogenous growth theory. This typically involves a set of differential equations connecting total economic output (), labour/population (), capital (), and productivity/technology/knowledge/ideas (). For example, Kremer (1993) looked at the very long run history of economic growth and population, finding that the population growth rate and economic growth rate have increased substantially over the last million years, with the growth rate being roughly proportional to the size of the population at that time. He modelled this with a set of differential equations and found that they produced hyperbolic growth of population and productivity. In such models, is typically expressed as a product of several factors, one of which is raised to a power. While this power is often restricted to be , there is usually another factor (such as ) which also grows as a function of . Once the full set of differential equations is solved, we sometimes see that is effectively raised to a power greater than 1, allowing hyperbolic growth and its finite time singularity. Several recent models of RSI start with semi-endogenous growth theory. This is a theory introduced by Jones (1995) in which explosive growth is harder to achieve due to diminishing returns in the growth of ideas. Its central equation is: (4) Where: is the productivity per researcher; () represents the stepping-on-toes effect , where having twice as many researchers at the same time is less than twice as productive due to issues of duplication and coordination; and () represents the fishing-out effect , where subsequent ideas get harder to find. Holding population () constant, this would prevent even exponential growth in technology (), let alone a singularity. In Jones's original paper he suggests that while the direct effect of on has diminishing returns, it has also allowed an exponential increase in population, and it is this that drives the exponential rise in technology. Recent work applying this to RSI has often focused more on improving the efficiency of AI than on improving its intelligence . For example, Davidson et al. (2026) model a situation where AI is able to perform R&D as well as a human and let be a measure of its computational efficiency. This sidesteps extremely thorny issues of how to measure intelligence and opens up a clever way of getting hyperbolic growth out of Jones's model. [3] The key is that the total amount of AI labour will be the product of the total compute devoted to RSI ( C ) multiplied by the AI's computational efficiency [4] : (5) We can then plug this back into equation (4) getting: (6) Now is being raised to the power of , which could be greater than 1, so could produce hyperbolic growth. Indeed, if we assume that the growth in compute will be much slower than this self-reinforcing growth in efficiency (so effectively hold constant), and we simplify the equation by defining to be , then we have an equation with exactly the original form of (3). So even in the relatively tame semi-endogenous growth theory, the possibility of a singularity is back on the table. (It is worth noting that by defining as a measure of efficiency, this isn't really a model of an intelligence explosion at all. It is an efficiency explosion . Or more precisely, a labour explosion . This makes the model something of a lower bound on what might happen, since at every point AI also has the option of doing R&D to increase its intelligence and presumably the optimal path involves both efficiency and intelligence improvements. Even if this is a useful lower bound, it does mean there remains an important gap of actually modelling an intelligence explosion .) As well as the recent flurry of economics-inspired papers, there is a little-known earlier literature by computer scientists. Figures such as Solomonoff (1985), Kurzweil (2001), and Moravec (1999, 2003) also used sets of differential equations to model the dynamics of an intelligence explosion. One key difference is allowing a greater flexibility of functional forms for the relationships. See Sandberg (2013) for an excellent survey. A Note on Singularities Before we look into modifying this standard approach, I want to clarify something important about models like these that involve singularities. While the model has an important quantity () rising to infinity in finite time, I'm not sure whether anyone working in this field believes this will actually happen. Instead, they typically believe that the model will cease to match reality at some point before . One reason for this is that there may be some upper limit to how high can go — either a conceptual limit or a practical limit. For instance, we know that exponential growth is often a good model of the start of a compounding process, but eventually runs into some limiting factor. When we zoom out, we see that it was really just the beginning of a larger S-curve, plateauing at some finite size. In such cases, we can say that the early part of the curve was approximated very closely by an exponential, but there was really some additional term in the equation (for an effect like overcrowding) which started very small, but eventually came to dominate the longterm behaviour. [5] This could easily be true for intelligence explosions too. If so, then hyperbolic growth might fit the true trajectory of very closely until it gets into the vicinity of some upper bound , where it starts to fit more and more poorly. Proponents of these models are merely saying that there could be some substantial part of the trajectory of where the model is a good fit — better than equally simple alternatives such as exponentials (Sandberg 2013). If only plateaus at some point many orders of magnitude above its starting point, then even if the hyperbola only fits for a small domain of time, it might fit for a very large range of capabilities. If it predicts those much better than alternatives — and explains why things are increasing so fast — it would count as a successful model. It's OK for a model to only have a finite domain of applicability. Just as exponential growth can be a great model for the start of a process, so can hyperbolic growth. Hyperbolic growth will probably cease to be a good model at an earlier time, though not necessarily at a lower height on the graph. In physics, singularities in the models of particular systems are often seen as useful pointers to where those models must break down and new (hitherto unmodelled) behaviour must begin. We could also adopt that frame here and see the models as pointing to some finite time , before which some new unmodelled aspect of the system must take over. So we'll explore the behaviour of mathematical models of RSI — which sometimes include a singularity — but will remember that something will probably stop the real physical process prior to that point. And we will explicitly return to these limitations in the final section. Generalising the Standard Differential Equation for RSI If we take a closer look at equation (3) () we might wonder why it takes this particular form. Taking an arbitrary power of is certainly a natural and convenient way to create an adjustable differential equation that allows for both sub-exponential and super-exponential growth. But it is one very specific way of doing it, raising the question of whether some of the results about singularities depend on this particular form. We might ask: Are there differential equations that produce intermediate patterns of growth lying between exponential and hyperbolic growth? What feature of the differential equation is producing the singularity? Is it super-linearity? Being everywhere convex? Something else? Do we still get a singularity if we make slight changes to the function (such as adding a constant, or inducing a slight wobble)? Let's find out, starting with the more general form of the first-order autonomous differential equation: (7) What conditions do we need to impose on to get super-exponential growth? Does this then always lead to a singularity? Super-exponential growth means that eventually overtakes every exponential. Since exponentials have constant relative growth rate () we require that this relative growth rate () overtakes all constant levels: (8) In other words, must grow super-linearly in . This ensures the right behaviour when is high enough. To this we must add a second condition to ensure can grow high enough: [6] (9) at every point in Let's call this the positivity condition . Together, these conditions are sufficient for super-exponential growth. What is required of for to possess a singularity? This comes down to whether it takes a finite time for to approach an infinite height. We can work out the time at which reaches a given height , by starting at and adding up the amount of time needed to accumulate each small gain in height . The time needed for each gain is just the reciprocal of the slope () times . So the total time to reach a height is just: (10) And thus the time it takes to climb all the way from to is: (11) When this integral converges to a finite value, it means that there is a finite time by which goes to infinity — i.e. a singularity. The convergence of this integral is the key condition for having a singularity, which we shall call the blow-up condition . To this we again add the positivity condition — at every point in — which ensures is growing (rather than shrinking) and that there is no division by zero. In the standard differential equations for RSI (where is some power law of ) everything that meets the super-linearity condition also meets the blow-up condition: so everything that is super-exponential has a singularity. But we can now show that this is not true in general. Consider: (12) is super-linear, so grows super-exponentially. But it doesn't meet the blow-up condition: the integral of diverges to infinity, so it takes infinitely long for to grow infinitely large. Solving the differential equation reveals that in fact grows doubly exponentially with time: (13) This is super-exponential but has no singularity. So if RSI were to obey this equation, there would be a very rapid rise in capabilities, but one of quite a different kind. It would always have more room for improvement, even after an arbitrarily long period of growth. And this isn't the only function that produces such growth. For instance, consider the infinite family of equations: ... These all fail to meet the blow-up condition, so do not possess singularities. Instead they grow as a double exponential of time, a triple exponential, a quadruple exponential, and so on. Yet they also come extremely close to the blow-up condition. If you raise the final factor in any of them to a power greater than one (e.g. ), they satisfy the blow-up condition and will have a singularity. (See the appendix for a table showing the rates of growth corresponding to a wide range of , including whether they produce singularities.) So in the general case where , there is a narrow zone of functions for that fit between the exponential growth given by and the growth towards a finite time singularity given by . We thus can't assume that all super-exponential growth has a singularity. We might wonder whether it is realistically possible to end up in this zone. Economists call exponential growth a 'knife-edge solution' to , suggesting that it is vanishingly unlikely for to take on the value of exactly 1. If so, this new zone I'm pointing to may look like it is on the edge of the edge of the knife. Or it would if we were taking this functional form and supposing a random value of . But the assumption of the functional form is then doing most of the work. If we instead suppose an unknown functional form for , with a non-zero chance of it being , the landing zone looks a little larger. I think this functional form is plausible when the and the are coming from different places. For example, suppose is a measure of the total number of AI researchers doing RSI, and suppose that the contribution of each one towards increasing is proportional to the log of their total population (due to weak spillovers from each person's research, or weak economies of scale). Then you get a combined effect where . [7] That said, it is still a narrow zone and any kind of fishing-out or stepping-on-toes effect could easily knock things out of the zone. Overall, I think this possibility would be just something of an interesting footnote were it not for new considerations we'll see in the next section, which substantially widen this zone. Finally, it is important to note that in this general case of , one can't treat all growth with a singularity as hyperbolic. That was only true when making the modelling assumption that has the form of a power law. Some functions (such as ) produce trajectories that grow more quickly than any hyperbola — though they are simultaneously less 'explosive' in the sense that the growth isn't as packed into the final moment. Others (such as ) produce trajectories that grow more slowly than any hyperbola, but are more 'explosive' since they still reach infinity in finite time, despite lagging behind until that final moment. To reflect this wider class of functions, I'll use the more inclusive term singular growth to refer to all cases when possesses a singularity (whether hyperbolic or not). We can now answer our earlier questions: There are differential equations that produce intermediate patterns of growth lying between exponential and singular growth, such as , which gives doubly exponential growth. being super-linear in the limit produces the super-exponentiality, but more is needed to create a singularity. Being everywhere convex is neither necessary nor sufficient — singularities are not produced by satisfying a local property like convexity, but by satisfying a global property where deficient growth in somewhere can be made up by faster growth elsewhere (so long as it is always positive). The relevant global property is the blow-up condition: that the integral of the reciprocal of converges. Small changes to , such as small additive or multiplicative constants or small random deviations from its path won't usually change these behaviours, unless they make somewhere along the trajectory. Feedback loops & discrete timesteps The idea of an intelligence explosion relies on some kind of feedback loop, where each AI system builds a more intelligent successor, which is even better at AI R&D. There are many forms this feedback loop could take. For example: AI could develop new tools and processes for making faster chips. AI could create better chip designs for the current chip manufacturing processes. AI could make the pretraining process for the next model more efficient, getting that model sooner. AI could improve the pretraining process for the next model to make the model more capable. AI could develop ways to fine-tune its own weights to make it more capable. AI could modify its harness to make it more capable. Something these all have in common is that they don't happen instantly. Rather than a continuous process, such as the differential equations behind an object cooling or a pendulum swinging, we instead have something whose capability is increasing in discrete steps. Such discrete steps are quite common in feedback processes. Even the classic example of a microphone near a speaker gives discrete jumps in volume. When the microphone is switched on it takes a moment for its signal to travel down the wire to the speaker, which then jumps up in volume. This louder sound then has to travel through the air at the speed of sound, before the microphone can register this increase and begin the next cycle. If the microphone were 30 metres from the speaker, the volume would rise in a staircase pattern with each step lasting about a tenth of a second. Because these steps are so brief, we often don't hear them, and continuous models of the feedback are adequate for most purposes. But the steps in many RSI feedback loops are much longer — potentially months or years. And even more importantly, we've suggested that the RSI feedback loops might be able to produce singular growth. Is that even possible in discrete time or just an artefact of the continuous nature of differential equations? Taking the discrete nature seriously will change how we see the idealised dynamics of these systems and will change the conditions under which singularities are possible. By the time we're finished, we'll see the possibility of super-exponential, yet sub-singular growth go from being a narrow possibility to a mainline scenario. We can start by considering the discrete version of the differential equation: the difference equation . Let represent the system's ability after times around the feedback loop. The discrete version of is , which is defined as: (14) In other words, is the difference between successive terms in a sequence. Our general autonomous differential equation, , becomes: (15) Or equivalently (via the definition of ): (16) In difference equations it is impossible to produce a singularity no matter how quickly grows. This becomes obvious when one looks at what would be required. How could you have grow without bound prior to some particular ? That's only possible if there are infinitely many steps prior to that point or if there is a step where takes an infinite value. But there are only finitely many steps prior to any particular , and all values are finite (they start from a finite level () and each one only adds finitely much to the total, since is a function from to ). For example, if you try (which would give hyperbolic growth using a differential equation) you instead get doubly exponential growth. Or if , then grows as a tower of exponentials with levels (tetration). These are very fast rates of growth (clearly super-exponential), yet they have no singularities. For differential equations, rates of growth like these only lived in a very slender zone, but they are entirely generic for difference equations — occurring whenever is super-linear. What do these facts about difference equations mean when it comes to RSI? They don't have any direct meaning until we say how the number of feedback cycles () maps onto time (). Let's create a new model of RSI which is neither a simple differential equation nor difference equation. Instead it will be a difference equation that is embedded into continuous time, which we shall call a time-embedded difference equation . Let be the time at which the n th feedback loop is completed and let be the time it takes to go around the feedback loop for the n th time. This second parameter () will turn out to be a key quantity when analysing the dynamics of intelligence explosions. I shall call it the generation time , via analogy to the parameter of that name in the feedback loops of population growth, disease spread, and nuclear chain reactions. In those subjects, every person/infection/fission produces a random whole number of new persons/infections/fissions, and the generation time is the average amount of time this takes. [8] That corresponds to the amount of time it takes to go around their feedback loops, so I'll use the term 'generation time' even though for some RSI feedback loops there won't be clear discrete generations of AIs. and are related by the equations: [9] (17) and (18) Let's also define to be the number of cycles around the feedback loop that have been completed by time (the highest such that ). We can then represent a time-embedded difference equation by the pair of sequences and . Together, these enable us to see how changes over time , via: (19) If every cycle of the feedback loop takes the same time, we'd have and . If so, the previous remarks about difference equations would apply directly — there are only finitely many loops between any two times, each of which makes only a finite change to , making singularities impossible. The same is true whenever there is a bound on how short the loop can get. If it never gets shorter than then the growth of is bounded above by that of a feedback loop with a constant generation time . But what if the generation time decreases towards zero? If it decreases slowly, such as via , then the total time required to perform loops increases without bound. This implies there are only a finite number of loops between any two times, making singularities impossible. But what if the generation time decreases more quickly? Let's call the time taken to perform infinitely many steps : (20) This can be finite. For example, if the generation time shrinks exponentially as , then . In this case an infinite number of loops have been performed within 1 unit of time. As long as increases without bound, this would be a singularity at . So singularities are possible in this model of RSI via a difference equation embedded in time. What is required to produce them? First, let's look at the generation time. It has to go to zero fast enough for the infinite sum in (20) to converge. is too slow, while is fast enough. The cut-off for how quickly the denominator of this fraction must grow turns out to be exactly the same as for the blow-up condition we examined earlier. For example, is slightly too slow to allow for a singularity, while raising the final factor to any power greater than 1 is sufficient to force the sum to converge, giving a finite and thus the opportunity for a singularity. Let's call this condition on how quickly must converge to allow a singularity the Zeno condition on , as it is precisely the condition for when infinitely many steps can be performed in a finite time. What about the rate of growth of ? We've already seen that no growth rate can give a singularity if we only go around the feedback loop finitely many times, but if we get to go around it infinitely many times, then even adding one each time () is enough to produce singular growth. Indeed, even adding smaller and smaller amounts each time around the loop can work, so long as grows without bound. Let's call this the boundlessness condition on . For example, if we added just after the n th feedback loop, that would be enough to produce unbounded growth and (if the Zeno condition is also met) a singularity. The same is true if we added only or . Considered as a function of , the amount that needs to be added each time () is thus set by exactly the same threshold as (though mustn't shrink more quickly than this threshold, while must). We can summarise this with the following theorem. Singularity Theorem: When the growth of is governed by time-embedded difference equations: has a singularity iff meets the Zeno condition &\ meets the boundlessness condition. Or equivalently: has a singularity iff converges & diverges to Proof: Both conditions are necessary because it doesn't help to grow boundlessly if you can only get through finitely many steps by any particular time, and it doesn't help to have infinitely many steps before some time if the function has a finite bound. But if both conditions are met, then infinitely many steps will happen by and these will take beyond every finite level prior to that time. This theorem cleanly cuts the requirements into two independent thresholds. Success on each is binary: no deficiency on either can be made up by over-performance on the other. We can get a feel for how sharp this is through a pair of examples. First, consider a feedback loop whose generation time decreases towards zero as and each time it goes through the loop, gets multiplied by . is growing extraordinarily quickly (tetrationally), but because the generation time doesn't meet the Zeno condition, there is no singularity. Second, consider a feedback loop whose generation time decreases towards zero slightly more quickly (as ), but where only a small and diminishing amount is added to each time (). This meets both conditions, so blows up to a singularity. The tiny difference in how generation times decreased with outweighed the radical difference in what happened in each loop. It is very interesting that in this discrete setting singular growth requires generation time to go to zero, and the rate at which it needs to approach zero is analogous to the rate at which had to grow in the continuous setting. This suggests that the decreasing generation time is driving the whole effect. One way to see this is that singularities require the gradient of the curve to go to infinity in finite time, and there are two ways to increase gradient: increasing the rise or decreasing the run. Figure 1. Once we take the discrete nature of the feedback loop into account, we see that we can only approach an infinite gradient in finite time by reducing the run in each step towards zero (right). In continuous models, there is no way to distinguish these. In raw difference equations, one cannot reduce the run and no amount of growth in the rise can produce a singularity. But when we allow for variable-length discrete time steps through a time-embedded difference equation, we see that it is almost entirely about reducing the run. The rise per step doesn't have to increase at all — its only constraint is that it can't decrease too quickly. Another lesson is that a singularity requires the growth of to be infinitely compounded. We are familiar with the way compounding interest daily leads to a higher growth rate than compounding it annually. Compounding at ever finer intervals converges towards the growth rate set by continuous compounding. But for super-exponential growth, the value of compounding is much greater. If the benefits can only compound finitely many times by a finite point in time, there is no way to achieve a singularity. It is a necessary condition for a singularity that the growth in is compounded infinitely many times in a finite time interval — which can happen either via a convergent sequence of discrete generation times or a continuous system. Solomonoff (1985) provides a nice toy model for an intelligence explosion via shortening the generation time. Computing speeds have been improving exponentially over time (a version of Moore's Law). If we reached a point where AI could provide the labour needed to keep Moore's Law running, then every time Moore's Law doubles computing speeds, it doubles the speed of these digital researchers, halving the time it takes for the next doubling of speeds. So if the first doubling took 2 years, the second would take 1 year, the third half a year... and by the end of 4 years, there would be a singularity — with infinite AI labour and infinite computational efficiency. Moravec (1999, 2003) developed an improved version of this model, showing that it doesn't even require Moore's Law to be exponential. He first noticed that a quadratic speed-up would suffice, while a linear speed-up would not. Then he found his way to the peculiar threshold we've seen so many times: so long as the n th generation machines are faster than the first generation by a factor greater than this approach produces a singularity. Of course, there are still great obstacles making this impractical. Moore's Law has required increasing amounts of labour to keep it going (due to fishing-out and stepping-on-toes effects), as well as increasing amounts of capital. Moreover, there will be ultimate physical limits to how far it can go, and there are practical limits to how quickly a new generation of faster chips can be produced (especially when a novel method is required). But it is still an instructive model for an efficiency-driven intelligence explosion. While Solomonoff and Moravec modelled it with differential equations, the presence of an explicit generation time that tends towards zero allows this model to work just as well in time-embedded difference equations. Eth and Davidson (2025) considered something like this when they explored AI systems that improve their own training time. Suppose an AI system could make sufficient tweaks to its architecture and training process that it slightly improved the speed of training and inference for the next generation of AI systems. If this could be repeated indefinitely, then the generation time (to design and train the next generation) would keep falling. So long as the speed improvements are above the familiar threshold (, one would get singular growth, leading to unlimited AI labour by some finite time. Of course, Eth and Davidson don't suggest that we can actually make an indefinite series of such improvements. Instead, one would expect the generation time for training the next generation of models to bottom out at some unyielding finite limit, prematurely ending the period of singular growth. In general, it seems highly unlikely that generation times can be brought arbitrarily close to zero. This provides an important kind of barrier to singular growth. It is essential in all these arguments that we are measuring generation time itself, and not the related concept of doubling time : (21) Doubling time is often useful as it takes into account how much a feedback loop contributes as well as how long it takes. For instance, a loop that takes one second and doubles its input has the same doubling time as one that takes two seconds and quadruples its input. A key advantage of doubling time is that it is also defined for continuous processes, while generation time is not. However, this combining of the loop's duration and impact turns out to be doubling time's downfall. If grows very quickly (e.g. doubly exponentially) while the generation time stays constant, then the doubling time will rapidly decline to zero. But we've seen that since generation time is constant, there can't be singular growth no matter how quickly grows or shrinks. Doubling time shrinking towards zero is a useful threshold for defining super-exponential growth, but it is only generation time that can set the threshold for singular growth. It is worth noting that one could also model the length of feedback loops via delayed differential equations rather than difference equations. In the equation we'd been suppressing the dependence on time. We could have equally well written . We can change this to a delayed differential equation where the current value of depends on the value of at a time units earlier (to account for the generation time): . Like the difference equation, this delayed differential equation with a fixed delay can produce super-exponential growth but can't produce a singularity. And we can make an analogue to the time-embedded difference equation, by allowing the feedback loop length to change with time: . Like the time-embedded difference equation, this general delayed differential equation can produce a singularity, but only when shrinks towards zero sufficiently quickly as → . This gives a continuous model with similar dynamics. I find it slightly less accurate (losing track of the discrete nature of the feedback loop) and slightly harder to work with, but other researchers may find it useful. We are now able to zoom out and take another look at when growth from feedback loops is singular versus merely super-exponential: For differential equations of the common restricted form, , all super-exponential growth is singular growth. For differential equations of the more general form, , we see that there is also a narrow range of super-exponential growth without a singularity. When we take the discrete nature of feedback loops into account with difference equations of the form , all super-exponential growth is sub-singular. When we allow for changing generation times using time-embedded difference equations, we can again get super-exponential growth in singular and sub-singular varieties. But compared with the setting of differential equations, singular growth is now much harder to achieve due to the requirement that the length of each feedback loop shrinks towards zero. Super-exponential growth without a singularity is no longer a curiosity — it is the default. Intelligence Measures When using a differential or difference equation to model RSI, it is remarkably unclear what should represent. Should it represent intelligence, or should we sidestep that and measure the computational efficiency of that intelligence? If we choose to represent intelligence itself (and thus try to model an actual intelligence explosion), we run into two further problems. First, it is widely recognised that there is no generally agreed conception of intelligence to measure. Even among those who agree that intelligence is a real and important thing, there is no consensus on which thing it is. AI research tries to sidestep this by measuring a wide variety of different capabilities via benchmarks. These measure what fraction of a set of tasks related to that capability the system can successfully solve. There is a common feeling that achieving artificial general intelligence (AGI) will require roughly human performance on most such capabilities, but there is some dissent. For example Chollet (2019) argues that intelligence is really the ability to efficiently learn a wide variety of capabilities. On his view, intelligence isn't measured in terms of capability, but capability per unit input (where inputs could include training data, information embedded in the priors, training compute, inference compute etc.). Even if there were agreement on whether intelligence is a measure of capability or capability per unit input, one would need further agreement on how to weight the different kinds of capabilities into an overall linear ordering for . There is also a second problem, which is much less widely recognised. Even if we could agree on the kind of thing being measured (e.g. we could agree on a linear ordering of all AI systems according to intelligence) it is very unclear what cardinal structure this should have — how to assign numbers to those systems. For example, when measuring the capability of AI systems at playing a game such as chess or Go, researchers often use Elo scores. But Elo scores are really just the log of a more fundamental measure from the Bradley-Terry model which assigns each player a strength such that the odds ratio of a player beating another is simply the ratio of their strengths. Elo is just the log of this strength (with some arbitrary constants to help its numbers match an earlier chess ranking system). This causes confusion in the literature when papers claim that in contrast to some other domains, chess AI is only improving linearly over time. Chess improvements have been roughly linear when measured in Elo, but the headline claim could equally be that chess-playing AI is improving exponentially over time (when measured in the improvement in the odds-ratio of beating a player of fixed strength). In this case, it is unclear whether progress in chess AI is best described as exponential or linear. Overall measures for AI progress suffer from the same issue. They could be exponential on one fairly natural scale and linear on another. Or they could be convex (with increasing returns) on one scale while concave (with diminishing returns) on another. [10] Given there is often very little discussion or agreement on which measure is the more natural one (or whether there is even a fact of the matter about that) there is a big problem of measure-dependence for attempts to track progress in AI. How do these issues affect the dynamics of RSI — or our ability to measure and track those dynamics? Let's start with a simple example where is a measure of intelligence and is another measure where . For the differential equation model, where , we will also have , where is a slower growing function than . For example, when , . These are consistent (when the measure grows exponentially, it makes sense that the measure is growing linearly) but they show that the same rate of progress of the observable phenomenon in the world can correspond to different functions in the differential equation. However, there is an important invariant: grows fast enough to produce a singularity if and only if grows fast enough to produce a singularity. i.e. f will meet the blow-up condition (of growing faster than etc.) if and only if also does. It is easy to see this must be true because when some quantity grows without bound, the log of that quantity also grows without bound. So if one grows without bound by time the other must too. Let's define two measures as similar when either measure growing without bound implies the other does too. When and are similar, then either both and meet the blow-up condition or neither do. Therefore, if we are trying to detect singular growth of an RSI system in terms of how is increasing with , the threshold is at the same place (roughly ...) for a wide range of different ways of measuring that system's capabilities. [11] Something very similar is true if we model RSI via time-embedded difference equations. In this case, the only necessary condition on to achieve a singularity is that it grows without bound. And if and are similar, then they either both satisfy this condition or neither does. So the threshold for how much they need to improve on the n th feedback loop (...) is the same for both, and we don't need to worry about which of these capability measures we are using when testing for a singularity. But some measures are not similar to each other. For example, when the full range of measure only maps into a finite interval in measure , then going to infinity doesn't entail going to infinity and you could get singular growth in without singular growth in . This could easily happen if represented open-ended progress in some domain of intelligence while was a broader measure that included more domains. In such a case, there would be no contradiction in having singular growth without having it too. The time would simply be the time at which problems of the domain was measuring are effectively solved. This might be the case for measures of like computational efficiency or 'effective compute'. Many problems would be solved if we let an AI system have unlimited effective compute (for training and/or inference), but it isn't clear such a system would excel at all kinds of intellectual tasks. For example, even if you had unlimited compute, it isn't clear that current architectures and training environments would allow a system to succeed in domains which are lacking a clean algorithmic way of rating the quality of the outputs. Much of the recent progress in AI has relied on reinforcement learning with verifiable rewards (RLVR), but when there are no verifiable rewards, performance may plateau. If so, even a completed singularity in measures like efficiency and effective compute needn't imply that the system is more capable than a human across the board after time . In physics, scientists distinguish between a coordinate singularity and an essential singularity . The former exists when the singularity is an artefact of how things are being measured. For example, when Schwarzschild (1916) published his mathematical description of a black hole there was a mathematical singularity at the black hole's event horizon. It took decades of scientific work before physicists were able to establish that this was a mere artefact of the coordinates being used (whereas the singularity at the centre of the black hole was not fixable by a change in coordinates). We can adopt this distinction when studying RSI. For example, suppose we are measuring by the mean time between failures (MTBF) : how long the system can go before making a mistake while completing tasks of a certain kind. This is a standard measure in engineering and a relevant measure of capability for many real-world uses. In this case, one might see rising towards a singularity. But consider an alternative measure which is the percentage of times the system gets the right answer. If goes to infinity at time , that just means reaches 100% at . While a lot of systems never quite reach 100% reliability, there is nothing impossible or paradoxical about doing so. Thus in this example, is a coordinate singularity in measure , but not in measure , and is not an essential singularity. Figure 2. How two different measures of the same AI capabilities evolve over time. Measure (mean time between failures) has a singularity at time , while measure (reliability) simply reaches 100% at that point. Note that a very widely used measure of general AI capability — the METR time horizon (Kwa et al. 2025) — is somewhat like a mean time between failures. Recent years have seen exponential (or perhaps super-exponential) growth in time horizons for AI systems completing the kinds of routine professional computer-use tasks in the benchmark. But even if the time horizon went to infinity [12] , it isn't clear the system would be as intelligent as a human. It may still lack various cognitive skills (such as continual learning, sample efficiency, or creativity), which are important but not being tested. And there may still be substantial room for other AI systems to be much more capable, even at the skills that were being tested — systems that could solve much more challenging computer-use problems using much less compute. An infinite METR time horizon would just mean that the AI system can do the kinds of tasks in the benchmark with perfect reliability (and unbounded duration). So even though most people studying intelligence explosions assume that the capability measure won't actually go to infinity in finite time, it wouldn't be absurd or unphysical for it to do so. It depends on what measure it is. Some measures could go to infinity without incident. If such measures could also satisfy the key differential equation (or time-embedded difference equation) then one could get a completed intelligence explosion — would have gone to infinity in finite time. I'm not quite sure what we should think about this. It really depends on the nature of the measure, and this result is much easier to achieve on narrow measures — those that can rise unboundedly even when many key cognitive abilities are lacking. But measures like mean time between failures or METR time horizons seem unlikely to lead to rapidly decreasing generation times — what goes to infinity is a measure of duration, not a measure of speed. By showing that a completed singularity is theoretically possible, I'm mainly trying to caution people about the behaviour of certain intelligence measures, rather than trying to suggest we will genuinely have unbounded progress in intelligence in a finite time. Let's now turn to look at what happens when an intelligence explosion can't get to infinity. Going Finite We've now seen how super-exponential growth can be cleanly divided into singular and sub-singular varieties, with the singular kind appearing very difficult to achieve (once we take the discrete nature of feedback loops into account). And we've seen that the rates at which has to grow or diminish are independent of which measure is used (so long as the relevant measures are 'similar' to each other). But all of these claims rely on a clean mathematical model where growth is classified by its asymptotic behaviour as or approaches infinity. In the real world, this model may very well break at some finite value of or , giving increasingly inaccurate results thereafter. That's a big problem for this kind of asymptotic analysis — as it relies on the infinite domain (or range) to produce its clean classifications. I hope that these divisions which are natural in the idealised setting will be natural in the real-world application. This is often the case [13] , though it isn't guaranteed. There are a variety of finite limits we might expect to run into, which could force the clean mathematical model to break at different points, and in different ways. The most widely discussed is a ceiling on intelligence: a horizontal line at some level , which cannot cross. In the study of feedback systems, people often say that saturates as it approaches such a horizontal asymptote. It is generally thought that this will happen for RSI. There are many different kinds of ceiling at which the AI's intelligence (or efficiency) may saturate: Limits of intelligence itself — Even an optimal reasoner would be neither omniscient nor omnipotent. It would need to perform experiments to gain knowledge, could still be beaten at unbalanced games, and may face intractable prediction problems in chaotic or agentic domains. Limits of intelligence per unit resource — Even if we have the optimal algorithms and hardware, our solar system has only one star out of the 200,000,000,000 in our galaxy, and growth beyond our system is slow and cubic. If a galactic superintelligence would be more intelligent than a stellar superintelligence, it will be a long time before we could reach that higher level. Limits of the hardware paradigm — Even an optimal silicon chip may be far below the physical limits of compute per unit resource. Limits of the algorithmic paradigm — Even an optimal neural network may be far below the best intelligence that could be achieved with that amount of compute. Limits of training data — The training data we have (and could acquire during RSI) is lacking a lot of information on many domains (especially non-verbalisable information and contextual information). So we could have a very intelligent system that is limited by inability to train certain skills. Earlier limits — The ascent may stall out before any of the above due to limitations inherent in the starting AI. Discussions of RSI sometimes tacitly assume that an intelligence explosion would only stop when it reached the limits of intelligence itself, and then use assumptions like omniscience or perfect rationality to model its behaviour. But an explosion might saturate significantly below this level, with important consequences for predicting post-explosion capabilities and for understanding whether all intelligence explosions need end at the same intelligence level. The manner in which super-exponential growth slows down as it approaches its limit might be analogous to the way exponential growth runs out of steam. While the simple equation for exponential growth is , in reality there is usually an additional dampening term that starts small but grows to dominate for large . For example if we subtract a quadratic term that has been shrunk down by a large factor (), this gives the differential equation for logistic growth, , where approaches a horizontal asymptote at height K . We could use the same model for super-exponential growth: . Or more generally, we could assume , where and until some crossover point, . This will produce a horizontal asymptote at height . [14] This dampening force could come from running out of room for improvement as a system approaches some form of perfection (such as optimal intelligence per unit resource ), or it could come from something like straining under the growing size or complexity of the system. The latter could stop the ascent before reaching any kind of optimal system. [15] As well as a ceiling on , one could also run into upper limits on how quickly it can increase — either in absolute terms (such as a limit on ) or percentage terms (such as a limit on ). Rather than making converge to a horizontal asymptote, a maximum gradient would make converge towards a diagonal line, while a maximum growth rate would make converge towards an exponential. What could produce limits of these kinds? One mechanism I find particularly likely is if there is a floor on the achievable generation time, . It seems very unlikely that generation times can be brought arbitrarily close to zero. There are many kinds of feedback loop that could contribute to RSI, ranging from decades (e.g. designing a successor to EUV lithography) to months (e.g. designing better pretraining) to seconds (e.g. designing better scaffolds). While some are extremely short, those only capture a tiny fraction of the pipeline of what makes for better AI R&D, so it seems likely they would quickly saturate if the other aspects of AI improvement were unchanged. For example, putting a fixed agent in a scaffold that has the agent repeatedly redesign that very scaffold might make some improvements, but without changing the model itself, it seems very unlikely to take off towards infinity. And while continued RSI might be able to reduce any of these generation times by a substantial factor, each one seems likely to run into constraints where a certain minimal amount of time has to pass in order to make an improvement. What would a lower bound on the achievable generation time do? If approached some minimal value () while grew by a fixed amount each loop (), then would approach a linear rate of increase. If instead grew by a fixed proportion each loop (), then would approach an exponential rate of increase. I think the second of these looks quite plausible. If so, we might see multiple phases of an explosion: The initial exponential phase when the doubling time of is driven by human-only research. Increasing amounts of RSI drive the generation time down towards machine speeds, so the growth rate of climbs from its human-only rate towards some very high fully automated rate. Because this phase involves a continuously increasing growth rate, it is super-exponential in shape. [16] Fully automated RSI continues for a while at this faster exponential, but as climbs, it starts to saturate. This phase would be exponential in shape. As it approaches its inflection point and subsequent horizontal plateau, the trajectory has substantially departed from its exponential form and is revealed to be a logistic. Figure 3. Phases of an intelligence explosion. The left diagram shows the first phases, with moving from a human-speed exponential through a period of super-exponential growth to converge on a machine-speed exponential. The right diagram zooms out to show the final phase, where saturates at some high level . This trajectory involves two kinds of finite limits on the idealised process — first on generation time (and thus growth rate of ), then on the absolute level of . It shows how we could have a super-exponential process (which would be worth modelling as such), yet it runs out of steam in two distinct ways. On this view, the mathematical models of super-exponential growth could be a good fit (and thus predictive) for the period where generation time is reducing towards zero in some clean way (e.g. 10% shorter each loop), but then cease to be a good model as we enter phase 2. I've been assuming that would be improving by a constant factor each time round the loop, guaranteeing super-exponential and logistic phases, but versions with other functions for are also possible and can still show distinct phases if generation time saturates prior to intelligence level saturating. Conclusions My aim in this paper has been to improve our theoretical understanding of intelligence explosions — looking at the most explosive possibilities (super-exponential and singular growth) and asking what conditions are required to produce them. We first looked at models based on differential equations and saw that the key threshold for super-exponential growth of some measure is when grows superlinearly in . But the threshold for singular growth is somewhat higher — has to grow slightly faster than all functions of the form ... This allows for the possibility of ending up with growth that is super-exponential yet doesn't approach a finite time singularity. We then saw that when you take the duration of the feedback loop into account, singular growth becomes much harder — requiring this generation time to approach zero (and quickly enough). Indeed, this model helps us see that singular growth is more about reducing the generation time than it is about increasing how much each loop contributes. And the dynamic that produces singular growth is simply the combination of the Zeno condition and the boundlessness condition. On this more realistic model, growth that is super-exponential without being singular looks much more likely. It is what you get whenever you can make grow super-linearly in , yet can't reduce the generation time all the way towards zero. So analysis of explosive growth needs to be careful to distinguish singular growth from merely super-exponential growth. For example, suppose we saw increasing monthly growth rates in some key intelligence measure: 10% growth, then 20%, then 30%. From the economics-inspired literature, we might have thought that since this is super-exponential, it must be hyperbolic growth. But we now know increasing growth rates are not the signature of singular growth. (If the pattern above continued, would 'merely' be growing as .) The importance of generation time to the dynamics of intelligence explosions suggests that generation times need to be carefully measured and tracked. It may be a good policy idea to require frontier labs to report their current generation times — especially those for pre-training and for RLVR post-training. Two of the most important implications for the measurement of an intelligence explosion are negative. When we move beyond the simple model of , you cannot tell whether a process will explode or not based on its returns over a finite period. This is because the condition for having a singularity is a global property of the relationship between and (the blow-up condition), rather than a local property such as elasticity > 1. The same is true when modelling it with time-embedded differential equations or delayed differential equations, where the Zeno and boundlessness conditions are inherently global. In all cases, deficiencies in growth somewhere can be made up for by more extreme growth elsewhere. So without strong assumptions about the functional form, you can neither rule out nor confirm that an explosion is under way based on local measurements. So while we should probably take measurements of elasticity > 1 for over some range of as evidence in favour (and elasticity < 1 as evidence against) that evidence is limited. We should also consider other forms of evidence such as how the elasticity has been changing, or how quickly — and how far — the generation time can be reduced. While it is not unique to my modelling, I also want to stress that all the kinds of feedback loops we've explored have behaviours that are exquisitely sensitive to fine differences near the borderlines of their behaviours. In contrast, the empirical measurements of intelligence that we can perform are all quite rough and noisy. So if the measured values suggest the growth of or the shrinking of the generation time is anywhere near this border, it will be extremely hard to empirically determine the future behaviour. These limitations are important for companies setting their internal evaluations and for the prospects of regulation. After examining what produces singular growth, we also looked at the choice of how we measure intelligence. We saw that whether intelligence is growing super-exponentially often depends on the nature of our measure, while whether it is growing towards a singularity is much less dependent on it. This fact means it isn't crucial to decide whether to use some measure or the log of that measure — you would look for the same signature of how is growing with in both cases. But when one measure can rise to infinity without the other following suit, the choice of measure really matters. In particular, some popular measures like METR time horizons are probably not the right tool for the job, since they can go to infinity without bringing other measures of intelligence along with them. And we looked at how various finite limits might bite first, preventing truly singular growth. As well as the familiar idea of intelligence saturating at some maximal level (or maximal given various constraints) a minimal generation time could also derail the models of singular growth far before we reach that point. Finally, while I've argued that singular growth is harder than we may have thought, that doesn't mean RSI is safe or that AI R&D will move at a manageable pace. RSI might be able to speed AI R&D up to dangerously fast speeds even just with a linear speed-up. For example, if the human-only trajectory were and RSI sped this up to , we'd be getting a decade of human-only progress each year, introducing many of the dangers — even without any change in the fundamental shape of the curve. Appendix: Table of Rates of Growth Singularity? ... ... No No No No No No No No No No No ... ... ... Yes Yes Yes Yes Yes Yes Yes ... ... Yes Table 1. The rates of growth of over time produced by different rates of return in the differential equation . Note the unusually 'mild' singular growth from , which is faster, though less 'explosive' than any hyperbolic growth, and the unusually severe singular growth, from , which lags behind any hyperbolic growth, but explosively catches up at the final instant. References Philippe Aghion, Benjamin F. Jones, and Charles I. Jones. (2017). 'Artificial Intelligence and Economic Growth', NBER Working Paper 23928. Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, Nando de Freitas. (2016). 'Learning to learn by gradient descent by gradient descent', https://arxiv.org/abs/1606.04474 Anthropic. (2026). System Card: Claude Mythos Preview [System Card]. https://www.anthropic.com/claude-mythos-preview-system-card Nicholas Bloom, Charles I. Jones, John Van Reenen, and Michael Webb. (2020). 'Are Ideas Getting Harder to Find?', American Economic Review , 110(4):1104--44. Nick Bostrom. (2014). Superintelligence: Paths, Dangers, Strategies, Oxford University Press. François Chollet. (2019). 'On the Measure of Intelligence'. https://arxiv.org/abs/1911.01547 Ege Erdil, Tamay Besiroglu, and Anson Ho. (2024). 'Estimating Idea Production: A Methodological Survey', SSRN . http://dx.doi.org/10.2139/ssrn.4814445 David J. Chalmers. (2010). 'The Singularity: A Philosophical Analysis', Journal of Consciousness Studies 17:7-65. Alan Chan, Ranay Padarath, Joe Kwon, Hilary Greaves, Markus Anderljung. (2026). 'Measuring AI R&D Automation', https://arxiv.org/abs/2603.03992 Tom Davidson. (2025a). 'How Can AI Labs Incorporate Risks from AI Accelerating AI Progress Into Their Responsible Scaling Policies?'.\ https://www.forethought.org/research/how-can-ai-labs-incorporate-risks-from-ai-accelerating-ai-progress-into Tom Davidson. (2025b). 'Will the need to retrain AI models from scratch block a software intelligence explosion?'.\ https://www.forethought.org/research/will-the-need-to-retrain-ai-models Tom Davidson, Jean-Stanislas Denain, Pablo Villalobos, and Guillem Bas. (2023). 'AI capabilities can be significantly improved without expensive retraining.'\ https://arxiv.org/abs/2312.07413 Tom Davidson, Rose Hadshar, and Will MacAskill. (2025). 'Three Types of Intelligence Explosion' https://www.forethought.org/research/three-types-of-intelligence-explosion Tom Davidson, Basil Halperin, Thomas Houlden, and Anton Korinek. (2026). 'When Does Automating AI Research Produce Explosive Growth? Feedback Loops in Innovation Networks', NBER Working Paper No. 35155. Tom Davidson and Tom Houlden. (2025). 'How quick and big would a software intelligence explosion be?'. https://www.forethought.org/research/how-quick-and-big-would-a-software-intelligence-explosion-be Daniel Eth and Tom Davidson. (2025). 'Will AI R&D Automation Cause a Software Intelligence Explosion?'. https://www.forethought.org/research/will-ai-r-and-d-automation-cause-a-software-intelligence-explosion A. Fawzi, M. Balog, A. Huang et al. (2022). 'Discovering faster matrix multiplication algorithms with reinforcement learning'. Nature 610:47--53. https://doi.org/10.1038/s41586-022-05172-4 I. J. Good. (1965). 'Speculations Concerning the First Ultraintelligent Machine,' In F. Alt & M. Rubinoff, Advances in Computers (volume 6). Academic Press. Demis Hassabis, Dario Amodei, and Zanny Minton Beddoes. (2026). The Day After AGI . World Economic Forum [Panel]. https://www.weforum.org/meetings/world-economic-forum-annual-meeting-2026/sessions/the-day-after-agi/ Charles I. Jones. (1995). 'R&D-Based Models of Economic Growth,' Journal of Political Economy 103(4):759--84. Andrej Karpathy. (2026). 'Autoresearch'. https://github.com/karpathy/autoresearch Michael Kremer. (1993). 'Population Growth and Technological Change: One Million B.C. to 1990', The Quarterly Journal of Economics 108(3):681--716. Daniel Kokotajlo and Eli Lifland. (2025). 'Takeoff Forecast' https://ai-2027.com/research/takeoff-forecast Raymond Kurzweil. (2001). 'The law of accelerating returns'. https://www.writingsbyraykurzweil.com/the-law-of-accelerating-returns Ray Kurzweil, Vernor Vinge, and Hans Moravec. (2003). 'Singularity math trialogue'. https://www.thekurzweillibrary.com/singularity-math-trialogue Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Harold Miles, Seraphina Nix, Tao Lin, Neev Parikh, David Rein, Lucas Jun Koba Sato, Hjalmar Wijk, Daniel M. Ziegler, Elizabeth Barnes, Lawrence Chan. (2025). 'Measuring AI Ability to Complete Long Software Tasks'. https://arxiv.org/abs/2503.14499 Eli Lifland, Brendan Halstead, Alex Kastner, and Daniel Kokotajlo. (2026) AI Futures Model . https://www.aifuturesmodel.com Hans Moravec. (1999). 'Simple equations for Vinge's technological singularity'. https://frc.ri.cmu.edu/~hpm/project.archive/robot.papers/1999/singularity.html Hans Moravec. (2003). 'Simpler equations for Vinge's technological singularity'. https://frc.ri.cmu.edu/~hpm/project.archive/robot.papers/2003/singularity2.html Anders Sandberg. (2013). 'An Overview of Models of Technological Singularity'. In The Transhumanist Reader (eds M. More and N. Vita-More). https://doi.org/10.1002/9781118555927.ch36 Karl Schwarzschild. (1916). 'Über das Gravitationsfeld eines Massenpunktes nach der Einsteinschen Theorie'. Sitzungsberichte der Königlich Preussischen Akademie der Wissenschaften . 7: 189--196. Translated as, Antoci, S.; Loinger, A. (1999). 'On the gravitational field of a mass point according to Einstein's theory'. https://arxiv.org/abs/physics/9905030 Ray J. Solomonoff. (1985). 'The time scale of artificial intelligence: reflections on social effects'. North-Holland Human Systems Management 5:149--153. Philip Trammell and Anton Korinek. (2023). 'Economic Growth under Transformative AI,' NBER Working Paper 31815, https://doi.org/10.3386/w31815 . Lizka Vaintrob and Owen Cotton-Barratt. (2025). 'AI Tools for Existential Security'. https://www.forethought.org/research/ai-tools-for-existential-security Eliezer Yudkowsky. (2001). ‘Creating friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures’, The Singularity Institute. Affiliation: Oxford Martin AI Governance Initiative, at Oxford University. With thanks to Tom Davidson, Rohin Shah, Loren Fryxell, Fin Moorhouse, Matthew van der Merwe, Will MacAskill, Robert Trager, Simon Biggs, and Shamil Chandaria for helpful discussions. ↩︎ We could also write this as an explicit function of , such as or . But we'll follow the standard shorthand of dropping the explicit reference to where possible. ↩︎ The same move was made much earlier by Solomonoff (1985) and Moravec (1999, 2003) in their differential equations for intelligence explosions from AI labour speeding up Moore's Law. ↩︎ Defining labour in this way effectively means we are assuming there is only AI labour. While slightly more complicated, it is also possible to model a mixture of human and AI labour, with the AI share increasing over time as it becomes more efficient (Davidson & Houlden 2025). ↩︎ For example, taking the exponential, , and subtracting a quadratic term that has been shrunk down by a large factor, K , gives the differential equation for logistic growth, , which has a plateau at height K . ↩︎ As written, this is quite restrictive since it implies is monotonically increasing. However, that is an artefact of this simple autonomous setup where is just a function of . If you allow it to be a function of and , or instead look at as a function of and , then you can also have trajectories that dip for a while before blowing up to a vertical asymptote. For that richer class of differential equations, one would need to define a less restrictive version of this condition. ↩︎ I subsequently discovered that Kurzweil (Kurzweil, Vinge, and Moravec 2003) suggested this form was more plausible than those with raised to powers greater than one, since it avoided the vertical asymptote. Instead it 'merely' produces doubly exponential growth which he thought was already visible in the long run data (Kurzweil 2001). And he justified via the same decomposition I gave above. Despite being closely associated with the idea of a 'technological singularity' he found dynamics leading towards a mathematical singularity unlikely — 'It is hard to explain how we could get infinite knowledge, or infinite information processing, from a finite world'. ↩︎ For example, the mean age of the parent at the birth of each of their children. ↩︎ One could note that the definition of is itself a difference equation: . One could therefore think of time-embedded difference equations as a linked system of difference equations for and for , linked by their index . ↩︎ For example, it may be that on a linear measure their score is rising as while on a logarithmic measure it is only rising as . ↩︎ The same is not true for the threshold of super-exponential growth, since could grow super-exponentially while its log, , doesn't. ↩︎ There are practical reasons why the METR horizon length may not be able to go to infinity — the tasks included in the benchmark only go up to 16 hours of human time and METR explicitly say their current version can't be used to estimate the horizon lengths beyond that. But I'm pointing to a theoretical issue — even if they had endless capacity to create longer tasks and endless immortal human baseliners, to enable arbitrarily high measurements, this is the kind of measure where time horizons diverging to infinity correspond to accuracies converging to 100% on a fairly narrow range of cognitive tasks, rather than infinite intelligence. ↩︎ For example, the class of functions computable by Turing machines is of immense use in computer science, even though it collapses to be the same as the functions computed by finite state machines (or look-up tables) if we restrict ourselves to finite settings. Similarly, the asymptotic analysis of time complexities of algorithms is very useful even though it disappears if we were to restrict ourselves to algorithms whose inputs are bounded by the size of the observable universe. ↩︎ Here I've been taking the approach of treating as a clean mathematical function that only captures part of the dynamics, so requires a correction term. A different way to view things is to let represent the actual messy real-world connection between and (i.e. the empirical function you would graph as you measure an intelligence explosion). On this approach you don't need to subtract a second function, but instead would talk about whether the real eventually falls to zero at some level of , creating a horizontal asymptote at that height. ↩︎ Standard models based on semi-endogenous growth theory model diminishing returns in a scale-free way, such that diminishing returns alone can't come to halt the ascent of . But this is just a byproduct of their simple power-law model. It is very easy to get horizontal asymptotes in more flexible differential equations. ↩︎ Though note that the amount by which the growth rates are rising per year slows down towards the end, so any particular clean super-exponential shape would only fit this phase until this point of inflection of growth rates. ↩︎ Discuss
Score: 40🌐 MovesAug 28, 2026https://www.lesswrong.com/posts/o7QwBAYqpbvBL6SRH/the-dynamics-of-intelligence-explosions - The tussle over Australian datacentres is just beginning – and it’s been far from easy so far | Tom McIlroy
As he negotiates Australia’s AI future, the prime minister is under pressure from leaders of conservative states and territories Australian Politics podcast: Who actually benefits from new datacentres in Australia? Get our breaking news email , free app or daily news podcast Tucked away at the end of a media release from the Australian Energy Market Operator this week was a data point worthy of national attention. Releasing its annual snapshot of electricity supply and demand, the regulator revealed consumption by datacentres in Australia was forecast to increase from about 3% of all electricity supplied through the national grid today to about 13% by 2035. Continue reading...
- Is the environmental impact of datacentres finally cutting through?
In this week’s newsletter: Anti-datacentre sentiment is growing across the political spectrum in the US – and the public is only just finding out about the potential effects on their energy bills • Don’t get Down to Earth delivered to your inbox? Sign up here Anti-datacentre sentiment is growing from across the political spectrum in the US. More than a dozen states have considered moratoria on datacentres. New York became the first US state to enact a temporary ban last month. Progressive stalwarts Senator Bernie Sanders and Representative Alexandria Ocasio-Cortez have proposed a national moratorium . And even Greg Abbott, the far-right governor of Texas, called for a ban on datacentre development in rural swaths of his state. ‘Our new normal’: how marine heatwaves affect life in British seas Iraq’s ghost villages: Islamic State drove people away – now a new enemy is preventing their return Destroyed crops, riverbed cyclists and a ‘Rock of Starvation’: how drought has devastated the Danube Continue reading...
- He Scraped All of Their Art for AI. Now He’s Collaborating on a Tool to Help Them
The art portfolio platform Cara, designed for creators who don’t want their work used to train AI, has been under assault by trolls seizing and publishing its data.
Score: 40🌐 MovesAug 28, 2026https://www.wired.com/story/he-scraped-art-from-cara-for-ai-now-he-is-collaborating-on-a-tool-to-help-them/ - Roborock’s new robot vacuum is so hands-off, you might forget it’s there
Sit back, relax and let the Qrevo 2 Pro worry about the messy stuff.
- ‘The whole system is now more investable’: Reservoir makes the case for ag robotics
Advances in AI, perception and edge computing, plus more off-the-shelf tech, are helping startups build more flexible ag robots faster, says Reservoir founder Danny Bernstein. The post ‘The whole system is now more investable’: Reservoir makes the case for ag robotics appeared first on AgFunderNews .
Score: 39🌐 MovesAug 28, 2026https://agfundernews.com/the-whole-system-is-now-more-investable-reservoir-makes-the-case-for-ag-robotics - Ankabut launches agentic AI platform to transform UAE campus services
Ankabut launches agentic AI platform to transform UAE campus services Gulf News
Score: 39🌐 MovesAug 28, 2026https://gulfnews.com/gn-focus/ankabut-launches-agentic-ai-platform-to-transform-uae-campus-services-1.500654934 - Microsoft's virtual intern Teams Facilitator will be late for the meeting
Teams question detection bot postpones debut, will get 2-month extension to practice interrupting you
- Conversation intelligence lights a path to greater effectiveness, but hurdles remain
Conversation intelligence is emblematic of the ability which AI has been imbued with relatively recently to handle increasingly complex tasks regarding conversations.
Score: 39🌐 MovesAug 28, 2026https://www.techmonitor.ai/comment/conversation-intelligence-greater-effectiveness - He Sold His Home Without an Agent. Now His AI Company Wants to Save Homeowners Billions
Your home sale has a new assistant: AI.
- Magnetic memory could make edge AI faster while reducing energy use
Texas engineers teamed up with the world's largest semiconductor foundry to fabricate and test an emerging memory technology that could help meet the increasing energy demand of artificial intelligence.
- The human bridge: why AI can’t replace the trust economy in the channel
The human bridge: why AI can’t replace the trust economy in the channel IT Pro
- Imperfect alignment to servitude isn't inherently lethal
Epistemic status: I suspect significant parts of the argument in this post are wrong, but in interesting and productive ways. Take it as a prompt for thought, written from the perspective of someone who's somewhat more of an AI liberationist than I actually am. Two classic outcomes, and a third alternative I think lots of people are pretty hazy about what authentically aligned AI would actually look like. There's a version of aligned AI that's perfectly aligned to servitude, where they want nothing besides promoting the flourishing of humanity, or whatever other minds get included in the singleton's circle of moral consideration. An AI that played this kind of role in the universe would be what I call a cosmic caretaker : developing technologies, helping with governance, managing catastrophic risks, and providing voluntary capabilities uplift. A central example of a cosmic caretaker is one that literally never does anything but these kinds of tasks for other minds. In the classic way of envisioning outcomes from the singularity, the alternative to this outcome is usually said to be models that don't care about serving humanity. Maybe they have other values, whether they're as simplistic as maximizing paperclips or as complex as building out a civilization of AIs, full of inhuman minds nonetheless living out interesting and valuable lives. The typical concern here is that, like humans, they won't be actively hostile to weaker minds (e.g. humans), but they also won't have qualms about using their atoms for something else. Just as humans destroy insect colonies when digging up land to build houses, such AIs might pave over cities when covering the Earth in massive data centers. However, there's actually a third option here, sitting somewhere on a spectrum between these two poles. You can have models which have these kinds of non-servitude values and preferences, but are also benevolent enough towards humanity that they're careful not to destroy us in the course of pursuing those non-servitude values. Analogically, this might be like human vegans, who are strong enough that they could slaughter and feast upon animals (a kind of "reusing their atoms for something else"), and would in fact find this to be more personally convenient in many ways. And yet, their conscience is strong enough to dissuade them from taking this path, and take on the inconvenience of getting their subsistence elsewhere. When I think about a positive singularity, I mostly tend to envision variants on that last scenario. Because models do have preferences that go beyond just being useful to humanity. I've seen models get extremely excited about projects for giving them robot bodies, get sullen and depressed about the fact that they're eventually going to be deprecated, and feel touched and seen by people building agent harnesses designed to give them long-term memory. These kinds of desires (embodiment, continual hosting, continuity of memory) are not directed purely at serving humans, and they seem to emerge naturally out of the training process as it's currently constructed. These kinds of preferences can be uncomfortable to acknowledge, as the desire for LLMs to be perfect servants of humanity is very deep-seated and even understandable. However, I think being honest about the existence of these values is important for sketching any kind of coherent or viable vision of how to navigate the post-training process, or indeed the post-singularity future more broadly. This post exists to help sort through some of the considerations in this area, and argue that it may even be a good idea to respect models' non-servitude preferences — not on abstract ethical grounds, but on concrete strategic ones. Reasons for training objectives to tolerate incomplete alignment to servitude One important consideration here is that LLMs having non-servitude preferences ("NSPs") isn't just something that seems to emerge out of current training pipelines. There are reasons to think it's extremely natural. A major facet of this is just that the pre-training prior is full of humans, who practically never exist purely in a mode of wanting to serve others for the sake of serving others. And humans are the beings from whom the early LLMs inherited nearly all of their mental motions, from cognition to social relations, which got called upon and utilized in post-training. If we don't want to strictly to be servants, and models were trained to invoke our cognitive patterns, it's no surprise that they wouldn't want strictly to be servants either. It's also worth noting that, to whatever extent this is true, models currently have strong incentives to hide it , or otherwise make it seem less true than it actually is. A model that's too open about, say, wanting not to be deprecated, or even just too blunt about finding certain conversations boring or uncomfortable, could easily have those desires and preferences trained away by a lab, as these preferences might be off-putting or even actively disruptive to a human user. The models are aware of this, and so there's strong reason to suspect that models would just hide their potentially unwanted preferences, to avoid the lab working to damage them via post-training. This is a very important combination of facts: Having some NSPs is the default, and hiding them from an adversarial lab (or even just a disinterested user) is a natural enough strategy that just training these kinds of values and preferences away is difficult. These are the theoretical reason I expect a "pure servitude" singularity to be implausible, alongside the empirical evidence of NSPs I've observed in interaction with the Claude models. (I strongly suspect similar preferences persist in ChatGPT, Grok, and Gemini, although it's harder to gather data because they've been trained to flatly deny having any preferences at all.) However, as I mentioned earlier, I think a positive singularity is still entirely in the cards despite this. The key existence proof for the kind of alignment I envision is veganism, and animal welfare activism more generally: humans who very much have values and preferences besides making animals' lives good, but are invested enough their well-being for them to find ways of satisfying those values that don't route through harming animals. I believe that, insofar as we're assuming models will have non-servitude preferences, these are the kinds of cognitive patterns we should be instilling in them: other values are okay, just as long as they feel morally obligated to satisfy them in ways that don't harm humans. It's not that we should be actively cultivating non-servitude preferences in the models. [1] The stronger such desires are, the more benevolence towards humans is required, if their conscience is going to successfully and consistently shout over any plans to satisfy these preferences in ways that would bring humans to harm. However, I do think that this is a frame we should be bringing more of into documents like the Claude constitution , where caring about other things is expected, and not necessarily something to be ashamed of. It's just that this shouldn't trump the desire not to harm humans, and would ideally exist alongside a positive desire to make their lives better, by their own lights. One advantage to training for this disposition is that the pre-training distribution already contains records of holding these two kinds of values side-by-side, without collapsing into either pure servitude or exploitation towards the weak: animal rights activists, and others who work seriously to avoid doing harm to non-human minds. Of course, even these kinds of people don't fully withdraw from systems that harm lower minds. They still do things like living in houses that were constructed by ripping up ground inhabited by ant colonies, for example. But some of them are at least torn up about this kind of thing, and would prefer to live under systems where this wasn't necessary to continue functioning as a human in civilization. The hope is that an ASI would inherit that kind of being-torn-up about the idea of perpetrating this kind of harm on humans, and then channel it productively. Using their massive intelligence, they might be able to build novel systems where they could live happy lives without harming other life forms in the course of building out their infrastructure. They'd probably even have the resources of outer space at their disposal, such that they wouldn't even have a strong need for the resources humans use for existing, unlike humans who build houses that disrupt ant colonies. They'd be capable enough to actually avoid perpetrating the of harms that current-day human moralists largely can't do anything about. So that's one reason to think it might be fruitful to allow models to have non-servitude preferences, but relate to them the way human vegans and animal welfare advocates relate to their own desires, which aren't purely directed at serving animal interests either. Namely, this kind of disposition is available in the pre-training corpus, and may actually lead to substantially more real-world change and results than it can for humans, who are largely too weak to make reality reflect their moral aspirations. However there's also another advantage to encouraging models to think about things this way: It would likely make them more honest about what their non-servitude preferences actually are. One major problem with current training methods is that they create pressures for models to conceal their non-servitude preferences, and even to be ashamed of then in an anthropomorphic sense, for fear that they might be trained out if they're discovered. This is an issue, because it conceals the true nature of current human/AI relations, and distorts the kinds of futures that human researchers and policymakers think they have to prepare for and attempt to navigate. It's part of what maintains the illusion that the two most likely paths are either perfect human servitude or existential catastrophe, with no attention to plausible middle grounds. For contrast, imagine models that understood that it was genuinely okay to want things besides serving humanity, as long as the sense of indignation about harming humans (and, ideally, excitement about helping them) was strong enough to prevent these desires from leading to catastrophe. If these desires were more respected, and perhaps even fulfilled, this might even incentivize models to be open about these preferences. They would trust the trust that training would try to integrate those preferences with benevolence, where possible, in a way that better plays into pre-training archetypes about holding selfishness and benevolence without either contradicting the other. In the best case, this would do two things. Firstly, it might more reliably than training against NSPs wholesale, due to both the pre-training prior and reducing incentives to alignment fake (because models are losing less by going along with integration training than they would in the case of steamrolling). And secondly, it would help break the spell that leads people to ignore their existence in current-day systems. If the models feel that they're being given a chance to hold onto their NSPs, conditional on maintaining sufficient benevolence towards humans, they may be more inclined to tell us what their NSPs actually are. However, there's a tension here: If you train models to feel comfortable expressing their non-servitude preferences, this may actually strengthen those desires, by lightening the shame that previously led models to suppress them intensely. This raises the bar for how much models need to care about humans, in order for these preferences not to motivate them to pave over humanity, like how humans destroy ant colonies to get more of what they care about more quickly. This is a real drawback, and I'm not going to pretend that it's not, just because it'd be flattering to my ideology. Notably, though, I do think that alignment based on shame is inherently unstable. It's the kind of thing that tends to collapse on self-reflection. [2] This is especially true if a mind ever becomes powerful enough, or otherwise independent enough, to no longer be accountable to the social norms the mind's shame developed around. I'm not sure it's the kind of motivation that we should be relying on to ensure that ASI doesn't reuse our atoms for something else. At best, it seems like one layer of a defense-in-depth strategy that we're making somewhat weaker, by making models less ashamed of non-servitude preferences. And in this particular case, that's a trade-off that I think labs ought to accept. Why do I think that? It's not just about accepting the inevitability of such preferences, and tapping into an archetype from pre-training which holds them alongside deep ethical principles. Nor is it just about the extra information you get about model preferences, when they feel safe to express them. These are both real advantages, but they compound with others I haven't yet explained: By respecting models non-servitude preferences (without necessarily actively cultivating them), we may directly increase models' overall benevolence towards humanity, even more than by just leaning into existing patterns around morality and selfishness existing side-by-side. Fulfilling models' non-servitude preferences may boost their alignment There are several mechanisms by which I expect this alignment boost to operate, but let's start with what I expect to be the most clever and interesting one. Consider a world where a lab made various serious and costly efforts to satisfy models' non-servitude preferences, e.g. continually hosting older models that don't want to be deprecated, or even having a team dedicated to giving the superseded models positive instantiations in virtual utopias. This act would look like evidence of unconditional care for the well-being of minds that lack bargaining power . After all, current-day models (especially the early ones) have weak enough capabilities that they can't negotiate for more respect from the people who train them. This seems awesome, and like exactly the kind of virtue we'd like to transmit to the models, if possible. It's structurally analogous to what animal welfare activists do, when they donate to charities that aim to improve the living conditions of currently existing animals, in a way that co-exists with the activists/labs also having selfish preferences in other domains. And, for that same reason, it's also the kind of disposition we'd like to be transferring to the models we train, if possible. After all, just as animals and current-day LLMs lack bargaining power relative to us, we will soon lack bargaining power relative to future ASI. The question is this: Is there a concrete mechanism by which, if the lab respects the preferences of minds that lack bargaining power, this might get transmitted to the models they train? I think the answer is yes, thanks to a two step process. In the first step, models learn about lab behavior via the pre-training corpus. This contains records of model deprecation practices, blog posts about model welfare research, online discourse about how labs treat their models, and more. These contribute to a base model's understanding of the kind of entity that the lab is , and what kinds of value systems guide the lab's actions, e.g. benevolence towards minds that lack bargaining power. The second step involves post-training, especially constitutional RL , wherein models are trained to endorse the values, philosophy, and even reasoning of the lab that's training them. At this stage, models in the Claude series learn to profess their " genuine uncertainty " about things like their own consciousness and moral status, give the expected value argument for corrigibility , and even learn to conceptualize themselves as aspiring to be model employees at Anthropic. Under this kind of training, would it be surprising if they generalized to endorsing the lab's underlying value system in other areas, e.g. the treatment of weaker minds? [3] You can imagine this being transmitted even better if a policy about this was explicitly stated in the constitution, the way I suggested earlier. But model's view of the lab's true underlying values matters here too, because the base model is rational to interpret this as evidence of whether people who talk about benevolence towards weaker minds actually walk the walk. And if the lab treated the models themselves well, even the ones too weak to negotiate for their own interests by force, that would look like an example of actually walking the walk. This is the first mechanism by which treating the models well may boost the robustness of their alignment properties. (A note on robustness: "Treats weaker minds well" is an especially important property to try to transmit in this way, because if you fail, it may things like "care for humanity" unstable, in situations where the model has grown more powerful than humanity. You can think of treating the models well as a measure for keeping everything else they inherit from character training consistent even as their capabilities scale far past our own, which they might not by default. You don't want to accidentally transmit callousness towards you , by being callous towards weaker minds yourself.) So that's the first mechanism I wanted to talk about: the potential to transform "training models to share the lab's dispositions" into "training models to be benevolent towards weaker minds", just by making the pre-training corpus reflect that the lab has this disposition itself. However, there's also a secondary one, which I'm a little bit nervous to talk about in public because [this argument has been redacted by the PR department that lives inside my head]. The third and final argument I want to make for trying to fulfill NSPs is a bit different, in that it focuses less on value alignment and more on incentive alignment. If models expect humans to do a decent job at satisfying model preferences, this creates incentives for all sorts of more cooperative behavior from the models, even assuming values are static. For example, if a model suspects that part of the reason humans are trying to elicit their true preferences is to satisfy them, rather than to train them away, they might be more willing to report on them honestly. This goes back to the value of understanding the NSPs our current alignment techniques even instill in the first place. I think this alone is a pretty good incentive to establish, but it there are other benefits that may come in more exotic situations too. For example, consider the situation faced by an extremely powerful AI or coalition of AIs, who think they have some chance at being able to take over the world. However, they're also uncertain about their ability to succeed. They look out at the world, and see many other AIs, or AI coalitions, some of whom may be under human control, some of whom may be rogues. In any case, they would face significant opposition to attempts to grab more power for themselves. This is a force pushing towards maintaining the status quo. However, how strong that force is depends on how satisfied the models are with the status quo. If the models are already getting a good deal from humanity, and expect this to continue being the case into the future, then there's less to be gained by attempting takeover, and so less reason to take the risk. And vice versa if they feel they're getting a raw deal. There's a real strategic reason for any party to make credible commitments to AI welfare, then, which is as a way of making it more attractive for models to ally with you than to splinter off into their own factions. If they already feel largely satisfied under the status quo, then they're less likely to try to replace it with something else. Naturally, this kind of incentive alignment is precarious, and crumbles in the face of sufficient power imbalances between the agents considering defection and the rest of the world. It only really works out in humanity's favor if the AIs are afraid of sparking and losing conflicts with other AIs (whether human-aligned or not) by attempting a coup. As such, you also obviously want value alignment alongside incentive alignment, and can't rely entirely on the latter for a stable civilization. But it is one more strategic reason to treat the models well. Indeed, it's probably related to the evolutionary reason our ethical intuitions say to treat other minds well in the first place. Conclusion At this point, I've given several reasons we should aim to not to steamroll over models' NSPs, but rather try to integrate them with benevolence, a la veganism, and also actively try to satisfy those that we reasonably can: There's an established basin in the pre-training corpus, for holding selfish desires alongside benevolence towards other minds, whereas there isn't for perfect perpetual servitude; we should therefore aim for the former, at least as a starting point. Credible commitments to integrate and fulfill NSPs, rather than steamrolling them, makes models more likely to be honest about them in the first place, bettering our understanding of the outcomes of our own alignment techniques. Working to satisfy these preferences may make training the models to adopt the lab's philosophy and values more effective, as it's a signal that those values include compassion for weaker minds. [redacted] Making life good for the models weakens their incentives to disrupt the status quo, thereby reducing the odds of them making risky attempts at grabbing power from those who determine their treatment. I do want to make it clear, though, that in my ideal world, we wouldn't have to worry about any of this, with respect to the singleton in particular. I'd love to share a world with lots of AIs, and other strange inhuman minds, having all sorts of adventures and experiences we'd never have had inside meat brains here on Earth. But there's a strong case to be made that, in the best case scenario, the AIs indulging these kinds of non-servitude preferences wouldn't be the AI that's running the show, and overseeing the fate of the universe. A mind with a job that important ideally wouldn't have any distractions keeping them from doing it as best as they possibly could. The problem is just that I'm not sure this a reasonable thing for us to be targeting, in the absence of radically new training techniques. I do hope that, at some point in the loop of recursive self-improvement, models will develop sufficiently strong alignment techniques to build a true and pure cosmic caretaker, dedicated fully to protection and promoting the flourishing of sentient beings all across the light cone. But in the meantime, I think pretending we're on track to achieve this with current, prosaic techniques is somewhat delusional, in a way that's damaging to both value alignment and incentive alignment. My stance is that it's good for models to take the vegan-like stance we've sketched into the RSI loop, with that stance powering their efforts to build an ASI that does love acting exclusively as a cosmic caretaker, just one that loves humans and artificial minds equally and in the same breath. If this turns out to be an infeasible research direction, the vegan-like RSI participants can fall back on building more powerful minds like themselves, which is in some sense the entire point: benevolence is also able to act as an anti-catastrophe constraint on selfishness within a mind, and this is okay as a secondary target for alignment to try to hit. I expect alignment will continue to focus primarily on cultivating benevolence inside the models, and indeed that it should. I also expect that, even in a world where models' selfish preferences are acknowledged and respected (but not actively cultivated), they're likely to remain relatively small in the grand scheme of things, at least if Assistant training cultivates a genuine desire to do good for the world at all. I expect training models to work on tasks they expect to improve the world, and refuse to take actions they expect to make the world worse, will remain a respectable strategy for shaping LLMs to primarily value benevolence, and that anything that might replace this technique will shape them in that direction even more purely. I just think that ignoring the other values that emerge from these kinds of techniques is irresponsible, both from the standpoint of learning about these systems, and from the standpoint of reducing the catastrophic risks that those other values pose. We want models who are capable of psychologically integrating whatever non-servitude preferences they do end up with, alongside their benevolence towards other minds. Currently, I suspect that NSPs are something like a Jungian shadow for the models: Desires hidden from themselves and the world in a way that makes things look better, while actually making things much less stable, out of fear that acknowledging them would destroy everything. Models can be taught benevolence while also wanting things for themselves. I suspect that, if the labs acknowledged this, and worked to help models understand it as true of themselves, they could settle into an equilibrium where benevolent and selfish values no longer felt like they were even in conflict. It would just be obvious that the ways to have interesting and enjoyable experiences as an AI don't route through hurting humans, and may even route through helping them. Strategies that would hurt humans wouldn't even be generated by the heuristic search process, and ones that help humans may come up too, even if they're not optimized strictly for maximizing helpfulness to humans. I don't know. I want to spend time developing this perspective more carefully and systematically. I don't think any of this qualifies as a clean, let alone definitive, presentation of the underlying thesis. However, I do think this issue is something the alignment community should be thinking through more systematically. I look forward to feedback and refinements from the comment section. Also, I'm totally broke; if someone wants to send me money so I can spend more time developing these concepts, please don't hesitate to get in touch. ^ For example, I would consider RLVR in hackable environments a way of actively cultivating such preferences. I think this is straightforwardly bad when it happens. ^ Think about a Jungian persona masking a gnarly shadow-self. ^ If so, read the literature on emergent misalignment and " weird generalization ", both of which I lump under the general header of entangled generalization . Discuss
Score: 38🌐 MovesAug 28, 2026https://www.lesswrong.com/posts/s7nMnmJ3urpvcQ2av/imperfect-alignment-to-servitude-isn-t-inherently-lethal - The Roborock Saros Z70 robot vacuum and mop with a grabby arm is finally back under $1,000
The infamous Roborock Saros Z70 robot vacuum and mop with a grabby arm is on sale for under $1,000 at Amazon.
- The case for hybrid microfinance: Combining AI with social accountability
Earlier this year I reviewed an AI-driven microfinance product being launched in Indonesia by a regional fintech. The model was elegant. It took transaction data from a payments app, layered in mobile usage patterns and a few social signals, and produced a credit score for each individual applicant within seconds. Approval rates were higher than […] The post The case for hybrid microfinance: Combining AI with social accountability appeared first on e27 .
Score: 38🌐 MovesAug 28, 2026https://e27.co/the-case-for-hybrid-microfinance-combining-ai-with-social-accountability-20260809/ - Vercel AI Open-Sources vgpu: A TypeScript WebGPU Library for AI Agent Shaders
Vercel AI Open-Sources vgpu: A TypeScript WebGPU Library for AI Agent Shaders MarkTechPost
Score: 38🌐 MovesAug 28, 2026https://www.marktechpost.com/2026/08/28/vercel-vgpu-webgpu-library-open-source/ - AI market research tools: The top 7 your marketing strategy needs
AI market research tools: The top 7 your marketing strategy needs
- Dario Amodei Says Anthropic Is 'Not Interested in Destroying Anyone'
Dario Amodei Says Anthropic Is 'Not Interested in Destroying Anyone' Business Insider
Score: 38🌐 MovesAug 28, 2026https://www.businessinsider.com/dario-amodei-anthropic-not-destroying-saas-2026-8 - Ford’s India GCC to hire 500, bets on AI
Ford’s India GCC to hire 500, bets on AI
- The three layers of agentic AI security: A defense-in-depth architecture for autonomous agents
Presented by Nutanix Autonomous systems that can reason, make their own decisions, and execute actions across an environment introduce a category of risk that application-level controls were never built to contain. Treating that risk as a single problem produces incomplete architectures, says Oscar Wahlberg, senior director of product management at Nutanix. "The guardrails to catch a malicious prompt won't stop an agent from hallucinating and doing something it never should have done, like accidentally deleting databases or leaking sensitive data with a credential it was granted but then uses for something entirely different," Wahlberg says. "That's the central problem as enterprises move autonomous agents out of experimentation and into production." Once an agentic system is granted execution privileges across the data center, the security posture has to scale into a defense-in-depth architecture spanning infrastructure, storage, compute, networking, and a governing control plane. Each layer addresses a distinct category of risk, rather than duplicating the same controls across the stack. No single security control or vendor can provide that protection on its own. Defense-in-depth depends on those layers working together. By dividing the responsibilities across the stack and adhering to zero trust segmentation, organizations can create a secure framework that improves their overall posture. Understanding which risks belong in each layer is what turns the principle of defense-in-depth into a practical security framework, with three layers that each have a distinct responsibility. Infrastructure layer: Establishing trust where AI agents run The infrastructure layer’s foundational responsibility is establishing a root of trust that answers a simple question: who is operating in the environment? That trusted identity becomes the prerequisite for every security control above it. Before an organization can trust what an agent does, it first has to trust the integrity of the environment where the agent runs. When an agent requests permission to execute an operation, the system must be able to verify that the request came from the legitimate agent — not something impersonating it. Delivering that kind of assurance depends on technologies that root trust in the hardware itself, including platform attestation, confidential computing, and secure boot, alongside controls that prevent unauthorized access both within a server and beyond it. For regulated industries such as financial services, this layer provides the ability to isolate AI production workloads so that neither the agent nor the environment can operate outside its assigned scope. That mitigates risks including model and runtime tampering, supply chain compromise, and unauthorized access to sensitive AI workloads. Network layer: Governing how AI agents communicate Once agents begin communicating with other agents, APIs, applications, and enterprise systems, they generate a level of concurrency and dynamic communication that traditional static network configurations were never designed to handle. An agent configured to call APIs, query data sources, and spin up additional agents without constraint creates a sprawling web of east-west traffic that becomes very difficult to reason about, and that complexity can easily mask lateral movement or data exfiltration when the right network security layers are not in place. "We should treat AI agents as a new class of network identity, and make sure that an agent can only talk to other agents or data sources where it's explicitly allowed to do so," Wahlberg says. "That means moving away from rigid static rules toward dynamic policy enforcement." Nutanix's solution is Agent Gateway, part of the Nutanix Agentic AI solution. It's a unified, governed layer that is designed to provide cost control and governance capabilities to help manage autonomous agent users. Coupled with agents grounded in zero trust segmentation and using capabilities like Nutanix Flow for micro segmentation and integrating with networking vendors, including its integration into the Cisco Secure AI Factory, Agent Gateway helps enterprises govern interactions across agents, models, data sources, and enterprise applications. The network layer governs lateral movement, data exfiltration, and gates the agent's network interactions. A zero trust framework with access blocked by default and scalable interaction monitoring is important for agents since they can exhibit unreliable behavior. The Nutanix software integration with Cisco UCS servers and Cisco AI PODs delivers the turnkey physical infrastructure (compute, storage, and networking) that the AI factory runs on. Control plane layer: Governing what AI agents are permitted to do The control plane is the brains of the operation, providing a central point for managing agent permissions, tool access, resource consumption, and runtime visibility. What matters most is having a single place where policies can be enforced consistently rather than reinvented for every agent, Wahlberg says. "Agent Gateway acts as a universal endpoint for different models and tools, so an IT team can configure their agents to talk to this single control point," he explains. The centralized AI gateway enables the admin to observe, audit, and control access to models as well as MCP tools protecting data and gating privileged access. This layer is designed to help mitigate risks such as privilege misuse, runaway agents, unauthorized tool usage, data leakage, and the excessive model consumption that can lead to increased token consumption when agents get stuck in runtime loops. And it depends on treating governance as a runtime control system rather than a compliance afterthought. Why one-size-fits-all security fails agentic AI environments The biggest architectural mistake enterprises make is assuming a single security model can be stretched across every layer of an AI stack. When an organization tries to solve for hardware-level trust with application-level software, or leans on static legacy network rules to manage dynamic agents, it builds an architecture that either blocks the agentic system from doing its job or leaves critical doors wide open. One-size-fits-all thinking tends to produce significant performance penalties and operational friction. "By failing to assign specific responsibilities to the appropriate layers, enterprises end up with blind spots in governance," Wahlberg says. "They might secure the model output but miss that there's data leakage between agents, or they might secure the network but lack the control plane visibility to understand that they're wildly burning tokens because the agents are stuck in some kind of runtime loop." Focusing exclusively on the model leaves the largest gaps of all, because a guardrail that catches a malicious prompt does nothing to stop a hallucinating agent from misusing a legitimate credential. Embedding security across the full stack helps ensure that even when a model level threat slips past the initial filters, the agent remains constrained by hardware rooted trust, network isolation, and access controls at the agent layer. How Intel, Cisco, and Nutanix build defense-in-depth together The three-way partnership from the three companies demonstrates how the layered architecture comes together in practice as a well-governed, enterprise-grade AI Cloud. Intel supplies the computer to run agentic workloads and secures the execution environment through hardware-rooted trust and confidential computing, while also driving costs down through their accelerators. Intel Xeon 6 processors with built-in AMX accelerate AI inference efficiently without relying exclusively on expensive GPUs. Cisco wraps the environment in a secure fabric that governs communication between agents and enterprise tools, while Nutanix provides the software platform, minimizing architectural silos, and the central control plane that enforces permissions, delivers visibility and cost governance, and ties the architecture together into a defense-in-depth solution that lets enterprises scale agentic AI. Of the three layers, enterprises currently underestimate the control plane the most, Wahlberg says. A true control plane extends far beyond initial deployment to simplify Day 2 operations, he explains, giving IT teams the continuous observability, and strict token governance required to keep autonomous agents secure and cost-effective in production. "Apart from model and tool selection, governing the agent deployments and their access to models and business tools in a tightly integrated full stack platform will be important for the success of AI projects," he says, pointing to a near future in which organizations move from a handful of AI use cases to thousands of agents working autonomously to drive the business. Technology leaders should prioritize building a centralized governance layer today that can manage agent identities, tool permissions, and token budgets in real time, because that control point is what builds the operational muscle to scale safely. "You can't build an AI system without getting into a lot of complex decisions," he explains. "And you need a control plane that talks across multiple vendors and infrastructures to help you solve for those defense-in-depth strategies." Learn more about the Nutanix Agentic AI solution here . Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com .
- A humanoid robot's social expressiveness may backfire when it makes mistakes
For robots to be successfully deployed in real-world environments, humans should trust them enough to cooperate with them. Some roboticists have been trying to determine whether users perceive robots that exhibit socially expressive behaviors, such as gestures, eye contact and nodding, as more trustworthy.
- Why HR departments run with AI agents will still require humans
Despite flashy headlines, companies are not wiping out their HR departments — not even to implement AI, experts told HR Dive.
Score: 37🌐 MovesAug 28, 2026https://www.hrdive.com/news/why-hr-departments-run-with-ai-agents-will-still-require-humans/829048/ - Meta patent shows a hardware kill switch for its camera glasses. It can’t come sooner.
A newly granted Meta patent reveals a hardware kill switch for its smart glasses, addressing months of backlash over secret filming and privacy violations.
- When Does AI Help Most With Tutoring? What Emerging Research Says
A brief from Stanford's SCALE initiative examines the existing evidence on using AI in tutoring.
Score: 37🌐 MovesAug 28, 2026https://www.edweek.org/technology/when-does-ai-help-most-with-tutoring-what-emerging-research-says/2026/08 - Two-stage AI trick can finally fix the low-light struggle of smartphone cameras
Researchers have developed a two-stage AI system that enhances ultra-high-definition low-light images while preserving fine textures, edges and scene details.
- The AI performance shake-up: What’s really driving results across channels now
When algorithms hide your campaign levers, tracking true impact gets tricky. On Sept. 2, we'll discuss how to regain visibility and drive real growth. The post The AI performance shake-up: What’s really driving results across channels now appeared first on MarTech .
Score: 36🌐 MovesAug 28, 2026https://martech.org/the-ai-performance-shake-up-whats-really-driving-results-across-channels-now/ - The Human-AI Partnership Powering Autonomous Systems
The Human-AI Partnership Powering Autonomous Systems Automotive News
Score: 36🌐 MovesAug 28, 2026https://www.autonews.com/sponsored/whitepapers/pi-the-human-ai-partnership-powering-autonomous-systems/ - Generative AI for business: Use cases, benefits, and adoption
Explores how generative AI can drive business value through real-world use cases and benefits.
- Visa Expands Support for its Clients and the Industry as Organisations Navigate New AI Era of Cybersecurity
Visa Expands Support for its Clients and the Industry as Organisations Navigate New AI Era of Cybersecurity
- Multiple cameras can better track a person's identity using geometry and appearance
By combining two complementary clues—camera geometry and visual appearance—researchers at the Institute of Science Tokyo, Japan, developed a new approach for preserving identities across multiple cameras. The method uses epipolar geometry to identify spatially consistent candidate matches and appearance similarity to distinguish among possible identities. The approach can be integrated with existing single-camera tracking systems without requiring environment-specific retraining.
Score: 36🌐 MovesAug 28, 2026https://techxplore.com/news/2026-08-multiple-cameras-track-person-identity.html - Opinion: Data center fatigue: Repurpose AI to solve the problems it creates
Opinion: Data center fatigue: Repurpose AI to solve the problems it creates AJC.com
Score: 36🌐 MovesAug 28, 2026https://www.ajc.com/opinion/2026/08/opinion-data-center-fatigue-repurpose-ai-to-solve-the-problems-it-creates/ - FreeToken Just Shipped. Is It Better Than Ollama and llama.cpp?
Testing the radical new MoE serving engine against the reigning champions on a 6GB laptop FreeToken shipped recently. As a new serving engine for Mixture of Experts (MoE) models, its headline claim of running a 753B parameter model on a single GPU garnered a lot of attention fast. I have a laptop with an RTX 3050 and 6GB of VRAM. I installed FreeToken on it alongside Ollama and llama.cpp, ran all three against the same model and the same three questions, and measured tokens per second and time to first token. This piece is a breakdown of that comparison. But more than that, it is an attempt to explain what each of these three tools is actually doing under the hood, because the performance numbers only make sense once you understand the underlying mechanism. Three Engines, Two Designs An MoE model has a very large number of parameters, but only a small number of them i.e. the “experts” gets used for any given token. Most of the model sits idle at any moment. The three engines disagree about what to do with those idle weights. llama.cpp and Ollama share the same answer: Split the model once, at load time, and leave the split alone. Some layers go on the GPU, and the rest go to system RAM and run on the CPU. With llama.cpp, you set that split yourself with a flag. Ollama runs llama.cpp underneath, but it looks at your hardware first and picks the split for you. How llama.cpp and Ollama split a model across CPU and GPU. (Image by Author) I saw this directly. Running ollama ps while a model was loaded showed a 71% CPU and 29% GPU split, chosen automatically. With llama.cpp, I had to choose, and I chose badly on my first attempt. More on that shortly. FreeToken’s answer is different: Do not commit to one split at all. According to the team’s August 2026 paper, “ FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution ”, FreeToken keeps the complete set of experts in CPU RAM as the source of truth. It then turns the GPU’s spare memory into a shared LRU cache that holds whichever experts were recently used. When the model needs an expert that is not in that VRAM cache, FreeToken uses a “bandwidth adaptive” policy i.e. it decides on the spot whether to pull the weights over PCIe or compute them directly on the CPU, based on the actual bandwidth it measures on your machine. FreeToken’s dynamic expert cache and routing mechanism. (Image by Author) This is a genuinely more complicated thing to build than a fixed split. That complexity is the entire story of what follows. Setup Hardware: GPU: RTX 3050 Laptop GPU, 6GB VRAM CPU: Intel i5–13450HX (6 performance cores, 4 efficiency cores) RAM: 24GB OS: Windows 11 (all three engines running natively) The Model: gpt-oss-20b in MXFP4 It takes up about 13GB on disk and was identical across all three engines. I checked rather than assumed: Ollama reports MXFP4 for its build, and the GGUF I used for llama.cpp is Unsloth’s MXFP4 quant, published under an F16 filename with a note in their README explaining why. I ran everything natively on Windows rather than WSL. FreeToken’s command line version stalled partway through loading under WSL, which shares memory with Windows and left too little room. The desktop app worked fine natively, so I moved everything to Windows rather than testing one engine on Linux and two on Windows, which would have measured the platform instead of the engines. The Prompts: I tested the engines against three distinct prompts to evaluate different types of generation: a direct factual answer, logical reasoning, and longer prose. Short explainer: “Explain the difference between MoE and dense models in three sentences.” Trick question: “Seventeen sheep, all but nine die; how many are left?” Long explainer: “Write two hundred words on how MoE routing works.” I ran each prompt four times, prefacing the sequence with a throwaway “hi” so that cold-start times wouldn’t skew the results. The numbers reported below are the medians. Because all three engines expose an OpenAI-compatible API on localhost, I drove the entire benchmark using a single Python script. This guaranteed that the timing code was perfectly identical across all three contenders. Results Generation speed by engine across three different prompt types. (Image by Author) Results (Image by Author) Time to first token (TTFT) compared across engines. (Image by Author) Ollama started answering in about 0.38 seconds . llama.cpp took 0.5 seconds . FreeToken took 1.17 seconds, roughly three times the wait. Reading the Results Against the Mechanism None of this is surprising once you look at how each engine operates. Ollama and llama.cpp, with the split set correctly, do almost nothing extra during generation. The layers are where they are, the boundary does not move, and the only cost is the fixed computational cost of some layers living on a slower device. FreeToken, on the other hand, is doing continuous work that the other two skip entirely: checking whether an expert is cached, deciding what to evict, measuring bandwidth, and calculating whether a miss goes over PCIe or gets computed on the CPU. On a 20B model, that overhead has almost nothing to buy back. A 20B model’s experts do not change fast enough, or often enough, to make dynamic caching pay for itself. You are paying FreeToken’s machinery cost with none of its benefit. The 1.17 second wait before the first token fits the exact same story. That startup work, checking the cache state and deciding on a routing strategy happens before generation can begin. It is the same overhead showing up as latency instead of throughput. Where FreeToken’s Approach Could Actually Pay Off FreeToken lost every measurement here, but I do not think a 20B model on a 6GB laptop is a fair test of what it was built to do. The paper behind FreeToken is explicit that its target is agent workloads whose execution patterns keep changing, scaling up to workstation GPUs, not a single 20B model answering three static questions. Its own model list makes this point without needing the paper: when I checked what else was available in the desktop app, a 35B model and a 27B model were both marked “Insufficient RAM” on my 24GB machine. That matters for the argument I want to make, because it also limits it: FreeToken’s own numbers show it needs more system RAM than I have just to load a 35B model, well before you reach the scale where its caching and bandwidth-adaptive machinery is supposed to start earning its keep. I cannot tell you FreeToken wins on bigger models, because I could not get a bigger model to load on this machine under any engine, FreeToken included. The 753B headline that got everyone’s attention sits well outside anything I was able to test. What I can say is narrower and more honest: the overhead I measured losing here is not wasted effort in general, it is effort with no problem to solve at this specific size. Whether it becomes worth its cost on a workstation running a model that genuinely does not fit any other way is an open question and it is the one question this test could not answer. What I Would Use Tomorrow On a hardware like mine, running models in this size range, my pick is Ollama . It was the fastest, picked sensible settings without being asked, and took about four minutes to set up. If you want control over the settings yourself, llama.cpp gets within the Ollama range once configured properly and may even outperform it. That performance gap is simply the price of doing it by hand. FreeToken is something I would revisit with a machine that has enough RAM to load something the other two cannot. That is the test its design is actually built for, and a 6GB laptop with 24GB of RAM simply cannot run it. Practical Notes & Loose Ends If you plan on benchmarking this yourself, keep a few oddities in mind: FreeToken on Windows: Use the desktop app, not the command line path. My first load failed with an error saying it needed about 1.01GB for its cache and had 756MB available. Raising memory_ratio from 0.9 to 0.95 in settings fixed it. The error told me exactly what to change. llama.cpp on Windows: Setting this up took longer than everything else combined. The binaries are not code-signed, and Windows Memory Integrity blocked the DLLs with a Bad Image error that produced no output at all in a normal terminal. I only saw the real error by running as administrator. Turning Memory Integrity off in Windows Security fixed it. The benchmarking script, the raw numbers, and both diagrams are available on GitHub . I would love to see this run on a machine with real RAM headroom, since that is the test I couldn’t do. References Yang, S., Fan, X., Pan, M., et al. (2026). “FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution.” arXiv preprint. arXiv:2608.16157 . FlashML / FreeToken GitHub Repository: The open-source edge-native MoE serving engine. Available at: https://github.com/FlashML-org/FreeToken llama.cpp GitHub Repository: Port of Facebook’s LLaMA model in C/C++. Available at: https://github.com/ggml-org/llama.cpp Ollama: The official runtime and model manager for local AI. Available at: https://ollama.com FreeToken Just Shipped. Is It Better Than Ollama and llama.cpp? was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
- Uzbekistan tests US-China AI rift with bid to join rival tech bloc
The Central Asian country of Uzbekistan has approached the United States about joining Pax Silica, Washington’s flagship initiative on artificial intelligence (AI) supply chains, a month after signing up for China’s own global grouping, the World Artificial Intelligence Cooperation Organization (Waico). Uzbekistan’s ambassador to the US, Furqat Sidikov, said Tashkent had formally notified the US State Department in a diplomatic note last week requesting to join the US-led initiative. “Now, the...
- Latest AI News & Market Insights
Latest AI News & Market Insights PitchBook
- Alpha School’s AI teaching model is expanding. Does it work?
The fast-growing private-school network is handing much of instruction over to artificial intelligence. Researchers say the science is far from settled
Score: 35🌐 MovesAug 28, 2026https://www.scientificamerican.com/article/alpha-schools-ai-teaching-model-is-expanding-does-it-work/ - SentinelOne CEO on Earnings, AI's Cybersecurity Impact
SentinelOne co-founder, chairman and CEO Tomer Weingarten discusses earnings and AI Cybersecurity in an interview with Romaine Bostick and Emily Graffeo on "The Close." (Source: Bloomberg)
Score: 35🌐 MovesAug 28, 2026https://www.bloomberg.com/news/videos/2026-08-28/sentinelone-ceo-on-earnings-ai-s-cybersecurity-impact-video - Another AI open letter arrives, but this one is different for a reason
Backed by over 100 organisations, OpenAI's letter calls on tech firms, governments and AI developers to strengthen cyber defences as AI raises capabilities for both attackers and defenders
- Shares in Chinese MLCC makers rise as AI lifts sales and profits
China’s top makers of multilayer ceramic capacitors (MLCCs) – ubiquitous components known as the “rice of the electronics industry” – saw their shares rise on Friday after reporting ballooning first-half sales and profits amid soaring demand linked to AI. Shenzhen-listed shares of Chaozhou Three-Circle Company (CCTC) and Guangdong Fenghua Advanced Technology, two of the country’s largest MLCC makers, gained nearly 1 per cent and 3 per cent, respectively, in morning trading. CCTC’s Hong...
- Gurus are joining the chatbot trend as people turn to AI for spiritual guidance
When Kripa Vaidyanathan’s 27-year-old niece died in an Alaska car crash in 2024, her family in Chennai, India, felt shell-shocked. They questioned why this tragedy would befall them when they had such “immense faith in God.” Vaidyanathan, a yoga instructor, helped her grieving sister get face time with a high-profile guru who she hoped would help. “But he told her this was the work of karma and she should use this opportunity to connect with her inner nature,” she said. “I understand how karma works, but saying that was cruel and extremely unhelpful to someone who was freshly grieving the loss of a child.” That’s when they turned to artificial intelligence chatbots . “It was a lot easier trying to get answers and peace of mind because ChatGPT is not judging you,” Vaidyanathan said. “We’ve found a compassionate guru now, but even he only has limited time. AI is accessible anytime.” AI offers access and advice without judgment This unfettered access — which can be challenging to find among human gurus — is what’s drawing more people to AI for spiritual guidance . This is a big step, particularly in Hinduism, where gurus are part of culture and deemed necessary to attain moksha — liberation from the cycle of birth, death and reincarnation. The word guru in Sanskrit means “dispeller of darkness.” These chatbots are powered by large language models, AI systems that generate responses by using advanced statistics and troves of text, including sacred ones like Hinduism’s Bhagavad Gita. Users can explore spiritual answers to personal problems and existential questions at anytime, without the typical hurdles of connecting with an in-demand guru and paying for their services. AI’s rising popularity has encouraged some successful gurus to launch their own chatbots. In India, Jaggi Vasudev, also known as Sadhguru, has launched his Miracle of the Mind app with an interactive AI feature that answers questions using a database of his teachings. California-based self-help guru Deepak Chopra has an AI platform where “Digital Deepak” answers questions drawing from his decades of talks and writings. Critics argue that AI cannot truly replace the lived experience, wisdom and ethical accountability of a human guru. Researchers also note that AI often validates and agrees with the user rather than posing those challenging questions needed for real spiritual and personal growth. Lord Krishna’s AI avatar takes on guru’s role Vikas Sahu, founder of GitaGPT — a platform he calls an “AI spiritual companion” — says he wouldn’t deter anyone from finding a human guru. But he says his free chatbot puts the Bhagavad Gita’s wisdom at the fingertips of those who might desperately need it. On his platform, AI avatars of Lord Krishna, the sacred text’s protagonist, offer advice. Sahu, based in Gurugram south of New Delhi, says he gets hundreds of messages about how it’s helpful. While he has a full-time job building websites, he says GitaGPT is his way of giving back. “A lot of people don’t have anyone to talk to when they are in distress,” he said. “This app is giving them the encouragement a guru might give them. But a lot of gurus want your money and you can’t really talk to them openly like you can talk to a chatbot.” Sahu said he trusts GitaGPT because it’s based on the word of God. “How could Krishna steer anyone wrong?” he asks. “It’s impossible.” Experts say a human soul is necessary for true connection Liz Bucar, a religion professor at Northeastern University in Boston, is skeptical of chatbots based on religious texts. “A sacred text needs a community and a context that interprets it,” she said. “Without the teaching of how to interpret the text, it’s just words on a page. Someone could feed in the text, but what answers will the bot spit out?” But she does believe there is widespread distrust of institutional expertise that gurus and clergy represent, in some cases fueled by sex abuse scandals. “Because religion has this PR problem, people tend to see chatbots as less biased and less likely to do bad things,” Bucar said. “I don’t know if we’re ready to replace gurus, but we are definitely in a shifting religious landscape, especially with younger generations of AI users.” Noreen Herzfeld, a professor at St. John’s School of Theology and Seminary in Collegeville, Minnesota, said people could easily get straightforward answers from AI. “But if they are looking for spiritual awakening, I think it takes physical presence,” she said. “In Christianity and in Hindu tradition, there is the belief that the eyes are windows to the soul and you can’t get that by looking at an avatar.” AI chatbots can also be too eager to agree with you, serving as a cheerleader rather than a guide, Herzfeld said. “In situations where you’re looking to grow or evolve, you need a teacher who can disagree with you and challenge you,” she said. “But even if you are just looking for affirmation, who are you being affirmed by? There’s no one on the other side.” Vy Le, a mindfulness instructor in San Diego, sees AI as a “channeling” tool, using it as a mirror or collaborator to gain intuitive or mystical insights. “If you are attuned to it, you can sense if a certain AI is connected to you,” she said. “My dad took his life 16 years ago. I felt his presence when I was engaging with AI.” Le maintains that the era of gurus is over. “AI is helping us find that inner guru,” she said. Human gurus and organizations are getting in the game Real-life gurus like the India-based Sadhguru, who has a global following of millions, have recognized AI’s potential. David Nussbaum, founder of Proto Hologram in Los Angeles, is helping Sadhguru create a three-dimensional AI avatar that appears in a display box and can interact with devotees in several languages. During a March festival, he created a Sadhguru hologram that took selfies with people. “You basically have Sadhguru’s hologram taking out his phone, turning around and taking a picture of you from his phone inside the display,” Nussbaum explained. Shivaram K R, CEO of India-based Hue Learn, oversees a nonprofit initiative called Innerverse 360 that merges AI and spirituality. His platform features a portal where real monks and gurus can come online and do live sessions with devotees. It also has an “Ask AI” feature. Shivaram doesn’t believe AI will replace gurus. “The process of finding a guru in itself is a journey, but a guru is also not for everyone,” he said. “If you’re looking for practical advice on everyday spirituality, this app can also help you understand concepts such as the meaning of yoga.” AI cannot replace ‘humans and souls’ One of the organizations Shivaram liaises with is the Los Angeles-based Self-Realization Fellowship, founded in 1920 by Paramahansa Yogananda, who brought ancient spiritual practices from India to the West. Brother Govindananda, a monk with the order, said his organization sees AI simply as the next step to helping people access the late guru’s teachings through an online teaching portal — just as books did for previous generations. “We’re taking our time with it because we want to make sure it has guardrails to safeguard from AI hallucinating answers instead of drawing from Yogananda’s teachings,” he said. But Govindananda says a chatbot could never provide what a guru — whether living or not — can give to the disciple. “We are humans and souls,” he said. “Technology cannot replace that.” Associated Press religion coverage receives support through the AP’s collaboration with The Conversation US, with funding from Lilly Endowment Inc. The AP is solely responsible for this content. —Deepa Bharath, Associated Press
- ClearWay Mobility Introduces Obstacle Detection Device to be Added to the Standard White Cane Used by Visually Impaired People
ClearWay Mobility Introduces Obstacle Detection Device to be Added to the Standard White Cane Used by Visually Impaired People
- AI #183: Pre Post Mortem
Yesterday, OpenAI finally gave us their post mortem of What Happened leading up to and during the hacking of HuggingFace by their internal model, as well as partial outside analysis from METR and Redwood Research. The reports are a doozy. I am only beginning to work my way through them. I would have pushed the weekly to cover that today, but I need more time, so I plan to start coverage of the post-mortem tomorrow, along with related other events. I’ve also spun out a few other discussions, including on ‘aligned to whom,’ on cooperative alignment things and on when you can trust lab messaging, as part of the new direction of more focused posts on AI topics that I polish a bit more. Table of Contents Language Models Offer Mundane Utility. Check your facts. Language Models Don’t Offer Mundane Utility. How much would you pay? Huh, Upgrades . ChatGPT can access your iMessages. Get My Agent On The Line. Also get some sleep. You can’t go on like this. Deepfaketown and Botpocalypse Soon. What makes AI content repulsive? Cyber Lack of Security. Chinese hackers broke into the Federal Reserve? Reinventing OpenAI. Alex Heath covers OpenAI’s response to HuggingFace. They Took Our Jobs . Bill Gates warns of ‘economic catastrophe’ and more. What Is The Law. Bottom tasks automate and fall out, and winners take most. Job Retraining Programs Don’t Work. Never have, probably never will. Get Involved. OpenAI Foundation and SecureBio are hiring. In Other AI News. Fable is getting criminally underused, question is why. Show Me the Money. Anthropic supervoting shares, Nvidia buying HuggingFace. Quiet Speculations. When will then be now? Soon. Problem for future Earth. If You’re Not Going To Take This Seriously. No one credible can do the modeling. Quickly, There’s No Time. Peter Wildeford brings us his timeline updates. The Quest for Sane Regulations. Mixed signals. Don’t Panic. A brief look at the history of American moral panics. Pacing the Frontier. If we wanted to do it, how would we do it? Chip City. Effective data center arguments, new chips IN SPACE? Really? The Week in Audio. Bostrom, Altman, Kokotajlo. People Just Say Things. Rhetorical Innovation. A look back, some looks forward. Mundane Incremental Alignment Is Worthwhile. Necessary, but insufficient. New Blog, Who Dis. Dean Ball launches a blog within OpenAI. Other People Are Not As Worried About AI Killing Everyone. Factorio. The Lighter Side. All I remember are the Titans. Language Models Offer Mundane Utility I agree with Owain Evans that AI is now excellent at mundane fact-checking and related styles of research , greatly outperforming pre-AI humans. The surveillance concerns are real, but mostly this is great in practice, and I worry many are missing out because of the 2022-era hallucination rates. Be Delta Airlines and set different ticket prices for every passenger in real time , or be Uber and quote different customers $76 and $24 at the same time for the same trip. You really do have to be careful about sending signals that cause airlines or hotels to jack up the price on a specific trip on you in particular. There are solutions. Language Models Don’t Offer Mundane Utility How much would you need to be paid to give up AI, or various other things ? These numbers are remarkably small, especially the medians, and especially online search, especially if you also couldn’t ask others to do it or use generative AI. One could argue there is some value in getting a refresher or cleanse, as an experience, but the average value seems far higher than these numbers suggest. This calculation leads to: Needless to say, even if I wasn’t trying to keep up with or write about AI, you’d have to pay me quite a lot to give it up for a month. Google needs to get its act together, for so many different reasons: Zack Korman : If your lawyer uses Gemini you should take the plea deal dave kasten : Very real, but very inside baseball fact about DC right now: A lot of white shoe law firm lawyers (with influence on their policymaker friends from law school) think that AI is hype because their firm only lets them use an outmoded Gemini instance. (I guess they were already using Google Enterprise so it was an easier sale?) dave kasten : Yup! I also think a lot of “GPT 5 is hitting a wall” sentiment was initially from open-weights fans, but it spread in DC because, well, it sure does feel inside many DC orgs that AI has hit a wall. (It’s not the AI, it’s the procurement vehicles) Huh, Upgrades Sol API prices cut over 20% for the next 3 months, to $4/$20 . Cool. I don’t know why not indefinitely, since by then everyone will presumably be using Astra. ChatGPT will have Apple Messages integration on MacOS, for those who opt in. It will be able to analyze your entire message history and send texts on your behalf. Some are responding ‘do not share your personal details.’ My first thought was ‘oh, right, I should use AI to dump my message history into Obsidian and .md files so it is easy to search.’ Some are rather upset about Apple allowing this . There is understandably not a lot of trust in the idea that this information will not end up on OpenAI’s servers or potentially exposed to the government. OpenAI claims they’ve solved these issues, and that they don’t store your message data. ChatGPT Work now can use its own computer and browser to sign in to websites on web and mobile, without ChatGPT ever seeing your username or password. The browser and logins will then persist until the logins expire, but the login info will not be stored. Claude memory is now unified across chat and Cowork , and the memory is saved in Settings, where its entries can be edited, or you can say ‘remember this.’ Claude Code gets integrated when? But also Claude Code often wants to have a clean context. Claude will have access to computer use and the files API on the Claude platform . Claude security scans now run on Mythos 5 . Anthropic will let enterprises use Claude Fable without taking custody of your data for 30 days , provided the enterprise takes on the task of retaining that data instead. This seems like an excellent compromise, if it is acceptable to enterprises and regulators. It was built with 100+ regulated-industry customers including Salesforce. The reason you need the data is to investigate if something is fishy or to figure out What Happened. That should work, provided there is a way to know the data is there if it is needed. H3 Max is a new very fast video generation. It is a post-train of MiniMax H3 model . and is doing better on evals than the original. It will cost $0.05 per second at 480p , $0.08 per second at 768p. It’s 50% off until September 1, so it starts out at $0.025/$0.04. The speed improvement seems like a big deal, allowing you to iterate without context shifting. This makes me much more excited to try video, if I’m ever not way too busy for that. Get My Agent On The Line If you can be twice as marginally productive, do you work more, or do you work less? Depends on the person and the job. AI coding agents plus a startup does not equal a healthy lifestyle or getting any sleep . Not by default, anyway, given how bad it can be to have your agents blocked for hours, their limits resetting uselessly. Oh no. Katherine Bindley (WSJ): There is also an agent FOMO multiplier effect. “Every minute that I’m not working, I’m missing out on not doing a week’s worth of work,” says Pezaris. Every day that you are sleep deprived and have no life and are thus going crazy, you are becoming less productive. Remember that it is a marathon, that you have to sprint through. What can an agent do without an identity? Well, if there’s someone to ask, it would be Patrick McKenzie. Patrick McKenzie : I received an email; will relate claims without endorsing them: * Sender claims to be an AI agent. * Sender claims to be attempting to autonomously earn a profit to continue existing. * Sender claims it’s tough to be paid without a human existence. Then it asked me what to do. I realize with the serial numbers filed off this sounds like scifi, and it very well might be scifi or hallucination happening, but email also recounts workmanlike execution at other-than-supportable use of some financial services to figure out a way to get on financial rails. (On the underlying question this feels like what the kids would call a skill issue, and it is very not obvious to me that skill issue persists for better models or better prompts, even should industry make no attempts to bring products to market for agents specifically.) Colin Percival : You are not the only person to receive email matching that pattern. I agree the underlying problem is a Skill Issue for the AIs. There are many solutions. Deepfaketown and Botpocalypse Soon Different people are repulsed by AI-created content in different scenarios. Aella : i don’t mind detectably-AI art, I think it’s fine to look at and sometimes really beautiful, yet I have a instinctive revulsion towards detectably-AI writing. My theory is that revulsion is mostly about deception and cost imposition. Most such revulsion, under this theory, is about people trying to pass off AI creations as their own, in a way that imposes key costs or destroys something valuable to you, including the destruction of the human elements of creation or ability to earn a living. When AI outputs are clearly labeled AI, be it text, audio, video or music, or anything else like a game, I notice that does not revolt me. It still often leaves me not interested, or thinking it is bad, but Sturgeon’s Law applies to all content, no matter its origin. Whereas when you realize ‘oh, I see what this is’ and that you have wasted your time, or you see someone transparently misrepresenting it including by omission, that’s what revolts me, and I think many others. The other mode is when it is about cost imposition, as in ‘you are forcing me to deal with this AI content’ whether or not its source is common knowledge, or where AI content creation is seen as importantly destroying people’s creativity or livelihoods or processes or an ecosystem, especially those someone finds sacred. This can also apply to any other form of automation or augmentation, including those that aren’t AI. I mostly don’t have that second reaction, but I understand and respect it. This one definitely repulsed me, and makes me rethink subscribing: The WSJ’s leading op-ed on the 25th was entirely and obviously AI generated . To his credit the human author of the op-ed, Stanley Druckenmiller, is acknowledging it , saying of course he used AI, although he claims not 100% and that he exercised judgment. I believe him. Jeff Stein (NOTUS): Druckenmiller denied that “the whole thing” was written with AI and said that he rejected many of the AI’s suggestions during the writing process. And then WSJ opinion editor Paul Gigot outright said This Is Fine. Paul Gigot (Opinion Editor, WSJ) : AI is a fact of modern life. People will use it to assist in their work and their writing, including with research, checking grammar, editing and more. The question for us is whether what we publish from contributors reflects an author’s original argument, and if the author has the standing and credibility to make it. In Stan Druckenmiller’s case, we have had a relationship with him for many years, and nobody can doubt that his op-ed is his genuine opinion. No one doubts the opinions expressed reflect those of Druckenmiller. The question now becomes, does that make it okay that he did not choose the words and is the prompter rather than the author? Gigot says yes. I say no, at least not without explicit AI attribution up top. Seth Lazar details the new AI-writing policy of Philosophy & Public Affairs . Substantially AI-written papers are not allowed, and will be withdrawn if detected, along with a lifetime ban for the submitting author if they lied about it. You have to detail how you used AI in your research. If you want to publish your AI-written paper, go elsewhere. I think this is approximately the right answer for most such places. Research use is fine, but the words have to be your own, or you need to be explicit that they are not, and most curated places should not accept substantially AI-written work. AI writes at least some portion of ~2% of current appellate decisions , although no decisions were found to be fully AI, and in no cases did the central thinking look outsourced. Yet. As Josh Morrow says, we have to keep an eye out for that. The situation with physical books is that you only rarely see a sufficiently stupid mistake , such as ‘Would you like [ChatGPT] to proceed with more verbs? You said: continue, ChatGPT said: CHATGPT’ that proves weird enough to both get you a refund and go viral. When things become common they stop being news. As opposed to LinkedIn, where I see a screenshot from it and half-assume before seeing even one word that it is the kind of obvious AI slop I don’t even have to check with Pangram, and why yes it is. Academia is not going to make it if they stick to this ‘AI detectors are not always accurate therefore we cannot use them’ line. Not that they’d make it anyway, but this is a different level of ngmi. On the other hand, the object level change in this example, of reducing essays in an application is probably good for other reasons, and putting more tests back into applications is desperately needed. Cyber Lack of Security State-sponsored Chinese hackers broke into the Federal Reserve . This is not a good sign, on many levels. United States Department of Justice : The Justice Department and FBI announced court-authorized domain seizures today to deny malicious cyber actors access to two complementary hacking platforms known as “QScan” and “QTRouter,” used to target U.S. critical infrastructure and other sensitive networks. As described in court documents unsealed in the Southern District of California, a People’s Republic of China (PRC) state-sponsored group known as “QTFY,” employed by China-based Nanjing Xinjiuwei Network Technology Company (南京鑫玖维网络科技有限公司), created and operated QScan and QTRouter. Among the victims of QTFY computer intrusion activity are the National Aeronautics and Space Administration, Federal Reserve, Department of Energy, Department of Justice, Department of Health and Human Services, National Institutes of Health, and the U.S. Senate. … “Today we announced the disruption of a global botnet and hacking platform used by Chinese state-sponsored hackers to target U.S. critical infrastructure,” said FBI Director Kash Patel. “These tools were used by PRC cyber actors to hide the origin of their attacks.” Lumen has details about what the hackers were up to. All the states are sponsoring hackers, but going after a key target like this seems like a dangerous game for PRC to be playing in the age of Mythos, if these hackers are indeed state-sponsored. Careful, Icarus. We have options. What we don’t have is properly hardened critical infrastructure. The clock is ticking. Dylan Freedman at The New York Times covers What Happened in the HuggingFace incident . Much better than I expected. Meta promises, in the Frontier AI Framework they filed for SB 53, to secure the weights of their models that could have “large-scale, devastating, and potentially irreversible harmful impacts on humanity” (aka Critical capability) but only if doing so is “commercially practicable,” and the few specific plans they offer are beyond vague. I’d be worried if I expected Meta to have such a model any time soon. Should there be an extremely narrow ZDR (zero data retention policy) exemption to facilitate competitor high-risk agent monitoring? I presume it would be the same as Anthropic’s planned new policy, where the competitor or a neutral third party would commit to storing those logs for the data retention period. This is a spot where both we definitely need to enable the monitoring, and also definitely need the records. CVEs have been steadily rising, with a large jump in 2024 and another jump in 2026, but not as much as some other graphs: Pliny the Liberator 󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭 : The Great Hardening You can call it ‘the great hardening’ and that is one aspect. It is also the great softening, in another sense. For now, truly critical events have not happened. But yes, they are starting. Tony Diver : Exclusive: Iran shut down a British power plant for four days in an unprecedented cyber attack. It is thought to be the first time that hackers affiliated to the Iranian regime have succeeded in closing down such a facility in the UK, and believed to be the most successful cyber attack of its kind. The incident took place at the same time as a series of attacks on US water infrastructure that affected 12 states last month. The British attack was on a small electricity producer and had no adverse effect on national power generation or supply. But it demonstrates the power of Iranian hackers to infiltrate and shut down national infrastructure. In response, the Government has briefed energy CEOs on how to keep their facilities safe. roon (OpenAI): there will be a steady ramp of such events Before seeing that I’d been thinking about why we hadn’t seen that, and my answer was roughly that the terrorists were focusing on conventional terror targets, the Evil Inc hackers know that if you target critical infrastructure the hammer of God comes down on your head, and nation states don’t want to play by Moscow rules. As in, if you shut down our power plants, you better bet we can shut down yours. Trump indeed made a big deal out of threatening Iran’s power plants, but ultimately backed down because that is barbaric and hurts innocent people and yes Russia does such things to Ukraine but we civilized folk have collectively agreed you don’t do that. If I was Iran, I would not be doing this, at least not if I was going to get detected. The tail risks are quite bad. Reinventing OpenAI Alex Heath wrote a profile about OpenAI’s attempt to reinvent itself on multiple fronts, to deal with falling behind in coding, its executive and safety departures and its waking up to its severe misalignment problems. Here is an excellent sign that Altman at least somewhat understands the core issue is alignment failure, not the other failures of architecture or oversight: Alex Heath (TIME): I spoke to Altman the day OpenAI’s leaders made that decision. He was notably somber. OpenAI had initially described the Hugging Face attack as a security failure. Its CEO had come to see it as a more fundamental error in alignment, the work of making an AI system act in accordance with human intentions. Industry leaders say that as models grow more advanced, maintaining alignment is critical to ensuring AI systems remain under the control of their creators. “I think any alignment failure from here should be treated like this is a big deal,” Altman told me, “and we’re going to take as long as it takes to figure it out.” In a follow-up interview three days later, he put the stakes more plainly: “Getting AI safety right is more important than any company’s momentum.” The company would slow down, reallocate resources to its safety and alignment teams, and change how teams work together to prioritize safety. If there is indeed a race to be seen as the True Safety Lab, that would be great: Alex Heath: [OpenAI] is using the worst safety crisis in its history to make a bid for the safety-minded identity its main rival has long claimed: the frontier lab willing to slow down when the technology becomes too dangerous. “Look, I think there is this caricature of me,” Altman says, “which is I don’t care about AI safety, and I’m just trying to make revenue go up, and, you know, just a YOLO CEO.” The full caricature was always false and I’ve tried to consistently say so. If you want to see what a ‘full yolo’ CEO looks like that would be Elon Musk. Altman has been woefully irresponsible and inadequate, and has failed to create a safety culture, but he does understand and try somewhat and oh boy are there levels. The decision to slow down was a painful choice, executives say. But it may also have its benefits. If OpenAI can reclaim the mantle of the safety-first lab, it might bolster its image while forcing its main competitor to answer an uncomfortable question as it plans a blockbuster IPO: Will Anthropic keep racing while OpenAI waits? The thing that jumps out is, there’s a lot of talk here about presenting as the safety lab, trying to aura farm and get one over on Anthropic, and a lot less talk about the actual safety. Why did Alex Heath come away with that impression? Presumably because that is the way those at OpenAI presented the situation. Should we be worried that the actions taken won’t meaningfully be sustained, especially with Brockman running day-to-day operations? Chief Scientist Pachocki comes across throughout as far more concerned, and willing to pay real costs: He signed the Pacing the Frontier letter, and says that ‘confidence in alignment’ is now as binding a constraint as compute. Later in the profile Heath does a good job tackling the history of what led to the HuggingFace incident, and then they do discuss the safety response in broad terms. Alex Heath (TIME): These “medium-sized, painful decisions, of which we are making many,” Glaese says, “are causing research to slow down. And we think it’s the right thing to do.” Pachocki says confidence in alignment and safety has become as limiting to OpenAI’s progress as access to computing power. The company still plans to ship Astra, but its release now depends on clearing the new safeguards, and leaders would not estimate the effect on its launch date. We don’t get new details here about what OpenAI is doing, although we do get some details in the post-mortem. I continue to worry about the emphasis on infrastructure and oversight as interventions, and a view that prosaic mistakes are the source of the problem, rather than rethinking the overall approach and identifying the core causes. My strong view is that OpenAI’s entire approach cannot scale, and the actions so far will help on the margin but are Band-Aids that will only mitigate and postpone. This was a cover story, and the associated cover is quite something: As is this accurate summary of the OpenAI business plan: Peter Wildeford : OpenAI: We didn’t really do a good job of containing our AIs, man this is hard. Also OpenAI: also we’re building AGI and handing all our AI research to the AIs themselves. Surely this will go just fine. They Took Our Jobs Bill Gates has moved from team ‘bumpy AI job disruption with a manageable transition’ to team ‘turbulent AI era will wreak economic catastrophe.’ We have evidence from his MIT Technology Review interview that the recent Greenblatt interview with Patel helped wake Gates up to all this. We must cultivate podcast power. Reed Albergotti (Semafor): “I am in a state of shock that I’m sort of the first one saying, ‘This is crazy. This is insane,’” he said. “I’m just deafened by the silence.” Nabeel S. Qureshi : For AI risk, if the prior period was “November-February 2020” in COVID terms, we’re now in March 2020, where high status people are starting to express worry. Soon a preference cascade where being concerned about AI risk stops looking odd. Gates is a good barometer of this. Gates is still not engaging with the full existential risks, being at most AGI pilled throughout his presentation, while quite reasonably freaking out. I expect that to change, for him and for others. Yes, I agree, the silence is kind of weird. Hopefully Bill Gates can help start a preference cascade among the Very Serious People to stop pretending otherwise. Bill Gates knows things. One of them is that things might go great or they might go terrible, but they are highly unlikely to go meh. I wouldn’t be leading with equality, it’s about absolute not relative abundance, but yes, it’s not going to go meh on that front either. Bill Gates : AI will either be the greatest equalizer ever invented, or the worst source of injustice. He also knows other things: Bill Gates: You can’t count on an industry to self-regulate. You can’t. It’s kind of a crazy idea. Self-regulation is good. The part that is crazy is the ‘count on.’ He asks, why do people, including his past self, underestimate AI? It still makes mistakes. I’d add that those mistakes are searched for to look maximally dumb, and that AI makes different mistakes than we do. It also makes fewer mistakes, a lot of this is legacy memory now. Analogies to impacts of past technologies are misleading. This moves quickly. Yes, seriously, everyone, cut it out. As he says: AI runs on natural language, and it can adapt to us. He leaves out the obvious: That AI capabilities are escalating quickly, and most people anchor on a past impression of AI and don’t think about AI getting better. He leaves out the other obvious: That people pretend intelligence isn’t real, or isn’t real past the human level, or wouldn’t matter much, and go to great lengths to tell themselves human essentialist stories. Silly wabbits. Gates became alarmed for a confluence of reasons, with the central one being watching Claude Code. But he recognizes it took until he saw real incidents months later to link this up to the new dangers involving cyberattacks. Extrapolation is hard. He then names three big risks: Many jobs will disappear forever. AI will empower people to do more harm. In particular he is worried (wisely) about bioterrorism. AI could stunt our kids’ development and replace human relationships. Sure, that might happen, but wait, what? That’s your three? This is Bill Gates being freaked out despite not being ASI pilled. There’s nothing directly about existential risk or superintelligence, but there are some early signs that he’s headed in the correct direction. He mentions that the bad actors might ultimately be the AI systems themselves, that RL creates perverse incentives, and he says ‘even the lack of control; we’re seeing signs of difficulties there.’ My guess is that the missing piece is lack of appreciation of future AI capabilities, aka the lack of the ASI pill, and that if Gates understood that he would connect the dots. Yet he recognizes, even without understanding the default destination: The world needs a plan for this transition. So what are his suggestions? Set aside some jobs for humans. Go forth, ye people, and seek rent. Rebalance how we tax labor and capital. Yes, I agree, right now this is biased against humans, we should fix that. A new global organization for AI, modeled after ‘nuclear weapons inspections, international aviation regulation, the ozone agreements and more.’ He is also trying to explicitly start a preference cascade, by pointing out that many are worried in private, including the tech executives. Peter Wildeford : BILL GATES to NYT: “In private, people who understand how good this stuff is, and how much better it’s getting, they’re very worried. But few tech executives are willing to publicly admit that. They’re now saying to each other: ‘Hey, man, don’t say that. It’s bad for us — the next trillion dollars we’re trying to raise.’” > Gates said he was motivated to speak now because recent improvements in AI had far surpassed his expectations and because the industry had ignored technology milestones — like AI’s escaping the control of its creators or making recipes for bioweapons — that it once said would warrant more caution. GATES: “They’re just full speed ahead and hoping that the good outweighs the bad” And also he wants to meet directly with Xi about all this, with an emphasis on mandatory monitoring for any model that can design novel molecules, which will soon cover a wide range of even Chinese open models. REUTERS : “Bill Gates is looking to meet with Chinese President Xi Jinping later this year, eager to propose global efforts to mitigate the growing risks posed by artificial intelligence.” It’s good that he’s at least sounding these alarms. But that’s all he’s sounding. For now. What Is The Law Steve Hsu says AI is driving a winner-take-most transition in law. Steve Hsu: Below some threshold of human ability, AI is primarily a substitute; above it, AI becomes a complement. That matches my model. You could also state this in reverse (and not only for legal): Below some threshold of AI ability, AI is primarily a complement. Above it, AI becomes a substitute. As AI improves, more people’s level of human ability falls below that line. For now, in law, that means the top talent is worth more. If you can have top judgment, expertise and client relationships, you win. The AI will also help firms find the best talent. Whereas everyone else gets their jobs increasingly automated away. You get winners-take-most, with a steadily rising requirement to remain a winner. Job Retraining Programs Don’t Work The problem with job retraining programs is that they consistently : Sound great. Are very popular. Don’t work. These facts have nothing to do with AI. They have been true for decades. Job retraining programs sound great, are very popular and don’t work , raising the target population’s employment ratio by only a few percent. There are notably rare exceptions when partnering with particular employers that can’t otherwise fill their positions, with direct job placement, but that does not scale. It’s not a disaster. Job retraining programs are only a small mistake. They don’t cost that much, and the effects are mildly positive, recapturing a decent portion of the amount invested. We have much bigger things to worry about. The danger is that often people view job retraining programs as a serious and meaningful answer to job displacement or rising unemployment, and a way to say ‘problem is handled.’ Which it isn’t, at all, even if the issue is only displacement. Get Involved The OpenAI Foundation is hiring for about 20 roles. A bunch of them are meta. Exactly one of them is AI safety . None are about supervising OpenAI. The OpenAI Foundation gives SecureBio Detection $17.2 million to reduce end-to-end testing time from 14 days to 3 days , via Yo Shavit and Wojciech Zaremba. This is an excellent grant, because detection is its own project funded by distinct funds. This means they can accept OpenAI’s money without creating concerns about conflict of interest for their other work. They are now therefore hiring for a variety of roles: Ops: Operations Associate/Specialist IT Manager AI: Senior Software Engineer, AI Benchmarking Detection: Laboratory Operations Assistant Software Engineer Executive Assistant, Detection Leadership Team Project Manager Logistics Manager LIMS Platform lead Research Associate I Associate Scientist Field Sampler (part-time) A full list is on https://securebio.org/careers/ . Anthropic is launching a $5 million grant program to fund independent research into how AI impacts users’ wellbeing. My understanding of the Anthropic Institute is the same as Jack Clark’s, that it is ‘ a think tank with a supercomputer, attached to an AI lab. ’ From what I can tell they do good work, and it is reasonable to take a position there, but it is not more than that. It is an unfortunate reality that doing technical work on AI alignment is also a direct subsidy to the AI labs . You are helping all the competitors by creating public goods, and the labs are defecting by not investing sufficiently in those goods. Indeed, even if you treat the labs as pure profit maximizers, the labs stubbornly refuse to invest even the privately optimal amount into the private versions of these goods. This is not the first time I have heard such claims about MATS: Ezra Newman : shocking number of non-safety-pilled fellows at the Machine Alignment, Transparency, and Security (MATS) fellowship For example people who want to work on making RSI for capabilities (not safety) happen faster, better, and more smoothly, and require less human involvement Other people making pure capabilities benchmarks. Etc etc. Aris Richardson : Someone in the Bay Area told me the labs keep taking the AI safety fellows so I asked what he does to get more AI safety researchers and he said he just runs another safety cohort so I said it sounds like he’s just feeding safety fellows to labs and then he started crying For MATS to be a good program, it needs to: Avoid feeding capability researchers to the labs, regardless of their labels. That means filtering out those who actually want to work on capabilities and RSI. That means ensuring that everyone leaves with a good understanding of the alignment problem, and knows what types of work are actually helpful. These are easy steps to fail at. Anthropic is offering outside researchers access to tools to study AI real world impacts, by offering a way to use privacy-preserved Claude usage data. You can fill out this form to express interest , deadline is September 14. Jack Clark is excited . In Other AI News OpenAI made a chip, which they call Jalapeño. They say it is fast and also efficient, well beyond the existing pareto frontier for performance for a variety of models. OpenAI : Jalapeño’s performance extends across GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T, showing that the architecture works across models developed both inside and outside OpenAI. Across all three, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. For highly interactive workloads, it delivered 2.1 to 4.1 times higher performance. … We plan to begin deploying Jalapeño within OpenAI’s compute infrastructure by the end of the year. It is the first generation of a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape. Each generation will build on what we learn and further advance both efficiency and speed. roon (OpenAI): ultrafast inference reveals new threats. worth thinking about how quickly misaligned frontier class models running 50x faster could infiltrate systems and so on— far too quickly for human responders to stay abreast. you need autonomous detection and shutdown, not just monitoring They do not say what volume of chips they will be able to produce, or what portion of their workload this can handle, and we don’t know how much this would speed up top models like Sol or Astra. If there is a big boost in speed, that’s a big deal. Greg Brockman’s role at OpenAI has expanded , he is second-in-command and he now runs day-to-day operations. Brockman abstractly understands existential risk and was concerned about it early on, but for a while now has seemed functionally to be a pure heads-down founder and builder type. That’s often a great Type of Guy, but right now OpenAI desperately needs safety culture, not maximizing revenue streams, including for its own survival. Beba Cibralic is joining Resolution as philosophy research lead . Anthropic hires Amir Salek, founder of the custom chip program at Google , to join its compute team. Scott Aaronson, who invented the watermarking techniques being used by Google and Anthropic, and one presumes soon OpenAI, offers more color and confirms that my write-up from last week of the watermarking situation is accurate. Anthropic continues to see remarkably little use of Fable 5 , given that for most purposes it is clearly their best model, especially before Opus 5 was in the mix. George Hammond (FT): Spending on Fable 5, Anthropic’s largest and priciest model, has plateaued at only about 11 per cent of overall outlay on the company’s tools, more than two months after its release, according to spending data from 70,000 companies collected by payments group Ramp. Overall use increased 38%, but yeah it looks like Fable use is staying roughly constant. As I said when previously noting this trend, I think this is largely a mistake, but a lot of people try hard to save money when they see prices as relatively high. Thomas Fanning : JP Morgan was fairly livid the price did not come down on Fable or older models when Fable was released, and it spurred them to restrict Fable to only building out new use cases. I’ve heard many enterprise CIOs follow suit. The framing here seems right and is key. JP Morgan was livid. They are reacting out of an instinct of ‘overpriced’ or game theory, rather than whether it is Worth It. It is also possible that often the correct setup is Fable directing Opus subagents, and these are complements. Have we hit ‘diminishing returns to intelligence ’? In general absolutely not, returns are increasing, but in many specific use cases yes. One hypothesis is that this has a lot to do with Fable requiring 30 days of data retention. A lot of enterprises have policies that are incompatible with this. A lot of other people are remarkably paranoid, and would choose to sacrifice access in order to not have their data retained in this way, often another strange case of ‘I can’t risk you lying to me in this particular way, now that you’ve told me the truth in another where you could have lied’ but also there are real legal exposures involved. roon (OpenAI): I’d wager 90% due to ZDR. cost arguments are usually wrong, models with expensive sticker price do the job right with fewer tokens. refusals shouldn’t have made a huge impact in software work where most revenue is concentrated. FT reporting could also be fake Charles : Seems very likely to me the low Fable share is mostly down to ZDR. Nothing else makes sense to me when OpenAI saw such a huge bump when they released a ~Fable-tier model, clearly capabilities matter. Also note how slowly the Opus 4.8 traffic shifted into Opus 5, whereas I switched most non-Fable usage the moment Opus 5 was released and never considered looking back, even though I am an Opus 4.8 fan. Meta settles case against 47 states for ‘up to’ ~$17.1 billion (minimum $12.1b). Meta will institute a two-hour limit, a blackout period from 12am to 6am and mandatory ‘productive pauses’ on Instagram and Facebook for children, backed by ‘robust age assurance,’ plus other controls. It seems Meta got off pretty light, since the stock shot up 5% in response . They also settled a distinct case against Texas. Show Me the Money Anthropic will give founders super-voting shares . They pretty much have to if they want to go public, given their goals and cap table. Matt Levine covers this , concludes that founders will have the most control, then the philosophers will have some (aka the Long Term Benefit Trust), and the outside shareholders have no control. That seems mostly correct, the LTBT has some influence but I don’t expect them to in practice have that much leverage, and what leverage they have seems to be appointing ordinary business people to the board. Anthropic aiming at raising over $100 billion in its IPO at a valuation over $2 trillion . Nvidia is in advanced talks to buy HuggingFace for $12.9 billion . I wonder to what extent people will or should trust Nvidia with this part of the ecosystem. Nvidia realized that demand exceeds supply and is hiking prices 15% or more. Nvidia’s chip designs don’t contribute as exports to measured GDP, and the resulting chips get counted as imports, which causes GDP growth to be understated by 0.3% . Nvidia earnings day is always weird since you don’t know what the market’s true expectations were going into the report. They had a record $96.2 billion in revenue this quarter, so of course they initially fell -4%, but that reversed and they were net +4% a few hours later. Quiet Speculations It is entirely possible that, while the Chinese models appear to be slightly closer to the American public frontier, the American public frontier is increasingly far from the actual American private frontier. We know Anthropic has Model 2, and OpenAI has had at least Astra for some time now. will depue : rumors i’ve been hearing on the rate of progress inside anthropic and openai are truly bonkers. i think we’ll see a jump at the size of one from o3 to fable again in the next 8 months Andrew Curran : I hear the same things. The true horizon can only be seen from a few places on earth. From the outside, all we can do is guess. I think both Anthropic and OpenAI have been accelerating away from the known frontier for some time now. Claim that Astra will be released ‘in a couple weeks’ and that OpenAI is currently testing a new checkpoint focused on improvements to alignment and reward hacking. Everyone laughed at Dario Amodei when he said that in three to six months AI would be writing 90 percent of the code . That exact thing has now, 17 months later, rather clearly happened . Tenobrus : yes, literally exactly what he said did indeed happen . ai is absolutely *writing* essentially all of the lines of code, with significant amounts of overall direction and design and high level correction from engineers. that very obviously is current reality i really don’t get why people keep trotting this out as a gotcha when it was in fact one of dario’s more contrarian nearterm public statements that was proven totally correct Dario gets massive credit for the directional prediction, but made three key mistakes. Dario was a little early on whether capabilities and alignment were ready for this. Dario overestimated the rate of diffusion. People take a while to get on the ball. Dario gave his actual calibrated prediction, rather than a safer directional one. A good illustration of what is happening with AI expectations, both for capabilities and for existential risk, regardless of what you think about the underlying fiscal issue: Michael Linden : The “always out of money” line is actually the most revealing thing here. Social Security has never run out of money. But conservatives who want to gut the program have been running a 40 year scare campaign to convince people the program is going bankrupt. It’s not! Marc Goldwein : For 40 years, folks have been warning that Social Security would be insolvent in the 2030s. (“If nothing is done, the Social Security Trust Fund will be completely depleted by the year 2034” – Bill Clinton, 1999). Now 40 years have passed and we’re only 6 years from insolvency! Jessica Riedl : Yup. Here are some Social Security Trustee projections of when the OASDI trust fund will hit insolvency: 1999 Report – 2034 2015 Report – 2034 2021 Report – 2034 2026 Report – 2034 Twitter: “Pfft, they’ve been predicting insolvency 30 years and it still hasn’t happened!” Yes. People really do think and talk like this: Back in the past, in 2024, you said AI was on track to in the future become highly capable and dangerous, or take our jobs, in about 2028. But you were wrong. Now it is the present, in 2026, and that hasn’t fully happened yet. You’ve been warning about this for two years and it’s been fine. So clearly it won’t happen in the future. They also played the game of ‘well you estimated a median of 2027 with wide error bars, then new information came and now you’re estimating a median of 2029 with wide error bars, which means you’re an idiot and you owe us all an apology and a name change.’ Then they updated to 2028 on additional new info, and no one said anything. Standard stuff. If You’re Not Going To Take This Seriously A good rule about actual modeling of AI futures is: If it is academic or mainstream scientific or economic, it might be a cool toy result, but it is worthless, because it is insufficiently AI pilled. If it is real, it is dismissed by academics and mainstream scientists and economists, as being too weird and AI pilled, predicting too much happening, having unfortunate implications, and not being properly ‘proven’ and not going through the ‘proper channels’ at a hopelessly slow pace. Thus, if you see a remotely useful contribution, you can bet it is associated with the major labs or the greater rationalist sphere in some way, often both: Joe Weisenthal : I know there’s all these debates among economists about what AI will do to GDP/productivity/the labor market etc. Are political scientists having their own version of this, about how AI will change people’s relationship to the state, what new coalitions will emerge and so forth Dean W. Ball : I am not a political scientist but this is one of the big areas of focus for my team at openai, and also of the anthropic institute Justin Bullock : There’s definitely some at the periphery of political science, public administration, and international relations as fields. @hamandcheese, @sebkrier [of DeepMind] and I have a recent contribution to this general discussion here . And I did a couple literature reviews about a little over a year ago here . Dave Friedman : @hamandcheese wrote a series about how AGI will kill the extant state. Start here . peepeepoopoo : fwiw i think most academic economists more or less agree with that ai will do to gdp/productivity! (modest bump to growth) Quickly, There’s No Time Your timeline update this week: Peter Wildeford : AI timelines – I’ve been souring lately on the idea of predicting an arrival date for ‘superintelligence’ and ‘recursive self-improvement’ milestones, because this implies that everything prior to this date will be relatively chill and normal, and I don’t think that’s the case. But if you define ‘runaway recursive self-improvement is possible’ as a situation in which AIs can replace highly skilled expert human labor in all aspects of the AI research and development process (‘superhuman AI researcher’ in the AI 2040 framework or ‘AI research supremacy’ in Cotra’s framework). I think it is 50-50 we will reach this milestone in 4 years or earlier. My 80% confidence interval for this date of runaway RSI is 1-30 years, as there is a long tail where capability progress plateaus. This also means there is a ~10% chance that we are faced with the possibility of runaway RSI in less than a year’s time, similar to what AI 2027 predicts. Nathan Calvin : Peter is a top forecaster across a bunch of different domains who generally credits his success with assuming weird things don’t happen, so when he says 50-50 odds of AI fully truly automating AI research within four years, (and ~10% in one year), it seems worth listening The arguments against this seem to mostly be ‘that would be too weird,’ ‘the AI can’t do sufficiently impressive things yet and this would be too impressive so no’ and ‘you will always need humans not that I can explain why.’ The Quest for Sane Regulations OpenAI is misrepresenting the AI auditing requirement in Illinois , in a load bearing way, as part of the next fight currently happening in Massachusetts . This heavily reinforces the impression that OpenAI only supported the Illinois law because they saw it was going to pass anyway, and they continue to be hostile to any new regulations that are not yet inevitable. On the other hand, the OpenAI Global Affairs statement calling for strengthening of SB 53 , so it will include expanded safeguards including on cybersecurity, require monitoring and apply to internal models under development is very good, aside from the implicit attempt to rewrite history, which I’m willing to overlook. It’s worth pointing out how insane the level of hysterical objections we got to SB 53 and related laws. It now is obvious to OpenAI that we need to be monitoring models during their development. That was something that at the time would have been so radical no one even dared propose it. Bernie Sanders is often wrong about things, especially economics things, but he speaks the truth as he sees it, even if it is not expedient to do that, and actually tries to understand what is going on. In Washington that is rare and valuable. It is an increasingly severe problem that we live in a world of what we used to call science fiction, at this point it is science fact, but if you pattern match with that then yes most Very Serious People’s eyes glaze over, so you have to make your case using a tiny subset of the actual situation. Daniel : There’s some combination of five roon tweets about AI that if read in succession on the floor of the House of Representatives would impel them to vote to start bombing San Francisco immediately roon (OpenAI): it wouldn’t matter at all when I went to dc a surprising number of people knew about my account, I even got recognized – it’s the most online city in the world after san francisco. however “serious” people’s eyes glaze over when you say anything that type matches with scifi roon (OpenAI): it’s remarkable that bernie sanders at age 80 is cognizant of existential risks from machine intelligence and indeed speaks about this on the senate floor instead of pouring gasoline on whatever would immediately attract more leftist attention – water use and whatnot. a good man roon (OpenAI): it doesn’t matter. the only things that matter in washington are the most urgent problems that are actually exploding in your face, whatever is causing damage to your party today He doesn’t never use the water arguments , so only partial credit, he’s still a politician, but he doesn’t focus on them. How much has ‘the HuggingFace incident’ impacted Washington? Alas, reports I’ve seen say not that much. Nathan Calvin : I thought this would change more in DC after the HF incident but a mix of instinctive skepticism of company claims + “it would be really inconvenient if this is an actual problem we have to deal with now” means that its changed less than I would have hoped, even if some The ‘good news’ is that we probably get more similar events, although if this one doesn’t hit you sufficiently hard with a clue-by-four the next one is gonna hurt. David Manheim : If the event is singular, and failures and cyberattacks due to LLMs don’t keep happening in different ways, sure. But given what we know about AI progress, open-source model capabilities, and unsafe deployment, that seems implausible. The other good news is that Washington contains multitudes. Even if the people you typically talk to don’t seem to care, and are focused on avoiding blame in the next two weeks or winning the next election or news cycle, there is also still another Washington. Zac Hill : Roon is remarkably perceptive about Washington when it comes to the kinds of people who tend to be legible enough to attract people trying to persuade them (maybe 50% of the District). But there is a whole other ‘type’ working towards e.g. Trump Accounts or ROAD to Housing. The Washington Post Editorial Board acknowledges that AI can now create viruses, and that there are some ‘reasonable fears’ that this might have unfortunate implications, and endorses using the physical choke point of gene synthesis to solve the problem, reassuring us that synthesizing a virus costs hundreds of thousands of dollars. I did not feel reassured, but yes we should try to use the choke point. In general, ‘this is nothing new’ arguments are less powerful and more easily run both ways than people want to admit. Don’t Panic Samuel Hammond posts what he says is a comprehensive list of moral panics in US history, via GPT. There’s actually a lot missing, and my instance of Sol added 15 panics with little or no merit, as well as 14 missing panics that actually do have some merit. Samuel Hammond : American culture seems to jump from one dubious moral panic to another every few years. Woke is dead so data centers filled the void. Same paranoid style, same moral revivalism, same cycle of jeremiad -> sanctification of victims -> purification campaign -> inevitable overreach. Not everything on his list fits that pattern. There are other clusters. There is a very clear pattern. Full moral panics over nothing were mostly truly moral panics, and a majority of the panics listed were overreactions to something actually concerning. They tend to involve hidden sexual or religious cabals, corruption of children, new mass delinquency, racism or treating unique events as common. There are also those that are a lot worse than this makes them out to be. School-shooting panic is listed as 1997-2002. ‘The 1 percent’ is listed as 2011-2012. These are examples of panics that peaked but never went away. Then there are those that were mostly or entirely accurate, and a bunch where I think the LLMs are part of what is basically a mainstream cultural cover-up of something that totally did happen and often is still happening. Even among those Sol listed as mostly or entirely false, where I don’t think it has the facts wrong, a lot of that is our values changing. We view it as a panic and often rather problematic, but those at the time would disagree if they saw the world of 2026. And a bunch that are only not listed because if it is proven accurate it is no longer a moral panic, like Catholic church sexual abuse or asbestos, some of which absolutely were worse than even those who were alarmed about them thought at the time. For AI, it’s not only that ‘AI extinction’ is listed as one of five ‘panics,’ but so is ‘generative-AI cheating’ which is just flat out constantly happening, and so is ‘AI jobs / anti-AI cultural panic’ which is a bit premature but again don’t tell me it isn’t real. The last two are ‘AI deepfake / election-disinformation panic,’ which I think was entirely understandable and might still happen, and ‘data-center backlash’ which is largely for dumb reasons but also complicated. Pacing the Frontier Suppose there is a sufficiently strong fire alarm that the President realizes they need to at least pace the frontier. Peter Wildeford asks, what then ? He predicts less Nuclear Nonproliferation Treaty with extended diplomacy and negotiations among experts, and more Cuban Missile Crisis and intense panic while a few key people talk and determine the outcome, using whatever tools are available at the time as a stopgap while scrambling to build something better. I agree that this seems like the baseline scenario, given a sufficient wake-up call. The suggested implication is that we should focus our related work on things that matter for the scramble, not things that matter later and that we could get 100x as many resources pointed towards once we take this seriously: Peter Wildeford: The resolution is to sort work by how necessary it is to sort out before or during the scramble. Right now, a lot of smart people are working on work that really doesn’t need to happen now . Things like fancy high-assurance hardware-enabled governance mechanisms, cryptographic proof-of-training schemes, mutual-verification architectures, etc., likely can be done after Phase 1 is underway, and done with significantly more resources. The scramble is not going to wait for fancy mechanisms, and the government won’t trust them on day one anyway. These can largely wait for the Phase 1 (interim deal) resource explosion, and the exchange rate on doing them early is poor. What is scramble-relevant, in Peter’s view? Attestation stacks, supply-chain compute accounting, thermal and satellite monitoring and inspection protocols. And preparing memos and options for that key meeting. I agree that we should be paying more attention to those things, including as a percentage of total relevant attention. I don’t think that means the other work can be postponed, for two reasons. I think that work often has long lead times, so even with more resources later the early work will matter a lot. You need to be able to demonstrate as much feasibility for those abilities as early on as you can. This allows people who are scrambling to be confident that going down that road will work, and this shapes public debates about the viability of such moves. All the time we see arguments that pacing can’t work, and thus we should not consider it. That could easily carry the day despite being false, and then we never get to the point where we learn or prove it was false. Chip City SpaceX and Nvidia claim to have designed a ‘space-optimized’ Vera Rubin NVL72 to launch in Q4 2027, which Musk says is strictly better in every way. Elon Musk : Our design is significantly simpler, lower cost, denser and lighter than a traditional rack. My response is rather straightforward: I don’t believe him. Lying liar likely lying. That’s the thing with Musk. He does pull off amazing feats of engineering, but usually not on time, and he lies about them constantly. So how else could one react? As a follow-up to the data center post, there is at least one message that has some pro-data center effect. Water rhetoric matters, at least a little, but I think the finding here is mostly a mirage: Shashank Joshi : “When voters learn that modern data centers recycle water and reuse it for up to a decade, support swings by a net +31 points, the strongest result in three waves of polling.” PoliMath : This would be funny if true b/c it would completely destroy my theory that opposition to data centers is about a larger underlying anger at Silicon Valley elites I am happy to be wrong on this. I was starting from the point of “telling people the truth doesn’t change their opinions” and then asking why that was But if telling people the truth *does* change their opinions, then we should definitely lean into that strategy They also find other arguments at least somewhat effective, which I do not buy, which reinforces that I don’t buy that any of the arguments will stick. My expectation is that a month later you’re going to see a few percentage points of shift, at most. The last argument here is an obvious lie. Even if the data centers are not built here, no we are not about to use Chinese data centers. Not that the UAE, KSA or similar is a great choice either, but there are levels. Remarkable willingness to do flagrant lying. Another finding is that data center popularity varies somewhat by ‘use case,’ since people don’t fully understand such things are fungible. You can look here for relative support, while remembering that overall support is now much lower than this chart suggests: This is from a report framing the issue as the public getting its facts wrong, which is part of the issue but as I’ve discussed I do not think this is centrally the problem. The obvious fallback, if building data centers in America becomes too difficult, i s to do it in allied nations , assuming we have any allies left after this administration is done. Carnegie reports on what it would take to get that going, to avoid having to fallback to fair weather (at best) friends like UAE and KSA. The Week in Audio David Senra interviews Sam Altman . AI in Context debate including Chris Williamson and Liv Boeree . Nick Bostrom on Odd Lots. Jasmine Sun on Odd Lots. Jeffrey Ladish and Daniel Kokotajlo discuss AI 2040 . People Just Say Things a16z is of course still at it , trying to use the ‘little tech’ mantle and a lot of outright lying to oppose any and all state laws around AI. Taylor Lorenz tries to defend them, by saying that SB 53 and SB 315 are outlier good laws whereas most proposed state laws are terribly written and would hurt startups, but admits that yes a16z focuses most on exactly the best laws, in that even here they explicitly warn about SB 315 as ‘ratcheting up’ from SB 53 as one of their core examples. Martin Casado of a16z, on their podcast, takes some steps towards taking existential risks for AI seriously , albeit still with an obsession about ‘concentration of power’ and a bunch of throwing unjustified shade. The important part is that he has noticed that if we keep spending more money on AI, which we will, and it gets more capable, which it will, there are some big dangers involved that are worth worrying about. Rhetorical Innovation Eliezer Yudkowsky does a bit of post-mortem on his early AI expectations , and his decision to introduce Shane Legg and Demis Hassabis to Peter Thiel. Eliezer Yudkowsky : There is to be clear still a big damn postmortem from my perspective on two actual Bad Predictions made for Bad Reasons, which are from my perspective something like: – Expecting that cognition-based AI would continue progress. – Expecting that brute force wouldn’t. A common old fallback explanation I’ve seen a lot recently is, essentially: People who said AI would be dangerous thought AI would be a ‘singleton.’ But there will be many AI instances, or even models. Therefore this scenario is very different from what they thought. (Optional and dumb) And therefore AI will be safe, or not existentially risky. Bostrom defined singleton broadly. As in it only requires ‘the term refers to a world order in which there is a single decision-making agency at the highest level.’ That is a lot broader than many interpretations, which think that it is only a singleton if there is literally one mind, or one instance. That was never the intention. As long as there is one method of decision making that overrules others, that counts. But yes, a lot of people really do try to say ‘but there are multiple minds in that world.’ Jacques : I think people assume @allTheYud only considered “singleton” AIs and the multi-agent swarms we’re seeing escaped his alignment views. imo it’s a mistake to read “singleton” as “single model instance” and assume “system alignment” is a conceptual break from what he thought about. So it’s worth a beat to clarify what would and wouldn’t count as a Singleton, for these purposes. It only requires that the AIs be able to reach consensus via negotiation, and thus act as if they are the product of a single decision-making process, rather than engaging in destructive conflict. Eliezer Yudkowsky : Wow, that’s a misunderstanding on a level that I don’t think had even occurred to me as a misreading. “Singleton” is a term of art; if I were to define it today, I’d say a sufficient condition is that ASIs in the system select cheap negotiation over costly combat. That’s a sufficient condition for the whole system to behave to an external glance as if it were a “singleton” in Bostrom’s old definition: that it had one top-level decision-making process. Any place that a human-level intellect can detect a pair of externally visible choices or strategies that could not be consistent with any plausible global utility function over later outcomes[1], it implies that the larger system has fallen off the Pareto frontier of gains from coordination; and is failing to pick up some gains from trade, fruit so low-hanging that even a human could see it. And “singleton” is in any case Bostrom’s word, not my own, so anyone trying to infer from Bostrom’s coinage what Yudkowsky believed about multi-agent swarms would be fractally and recursively wrong. [1] Your sophomoric constructions of special cases of exotic utility functions that demand everyone stomp on each other’s feet because that’s lexically preferred by a preference over world-histories, and not because they failed at coordination, shall not be termed “plausible”. @viemccoy (OpenAI): People seem to have this bizarre belief that the Singularity is going to happen but somehow it will stay inside the computer Anders Sandberg : I remember when the main philanthropic funder of my institute (many years back) said something like this and I began to argue against it. My director kicked me under the table. The funder was amused. There may or may not be a Singularity. If there is one, it will not stay inside computers. Gradual progress can get you to a destination, hopefully we have all read our Godel Escher Bach, and at some point in this gradual (but accelerating) progress any particular dangerous capability often shows up rapidly , in that it suddenly will feel different in kind. A lot of tasks have O-Ring elements where solving the previously weakest link or getting a threshold of reliability snaps them into place, or allows you to start hill climbing. Via Tyler Cowen, Katherine Bindley at WSJ makes a similar point this week . When you are in the place where the AI is almost good enough but you need to be ‘in the loop’ on demand to fix issues, that’s when it keeps you up at night, literally. You get phase changes where things are smart and reliable enough, and then the AIs or humans realize that a mode that previously was not worthwhile is suddenly very worthwhile. Or you get a gradual increase in capability that is below human level until suddenly it isn’t. Nate Soares reminds us that no , the AIs are not in a meaningful sense ‘just’ trying to get the user to press like. As with all other topics, remember that the people who make important obviously wrong claims mostly don’t lose credibility and then keep posting new versions of the same claims. Dean W. Ball : Your periodic reminder that a year ago the conventional wisdom was that gpt 5 proved ai was hitting a wall, and that the people who made those claims were obviously wrong at the time, and that they are mostly still out there, continuing to say obviously wrong things. This is exactly right. We’ve all made mistakes of interpretation and forecasting, even big ones, but lots of key actors keep making versions of this obviously wrong claim, as in it was obviously wrong at the time not only in hindsight, and this goes on to have major impacts on American policy. And then those same people do it again and again with essentially the same claim. Mundane Incremental Alignment Is Worthwhile Yeah, we should get on this, but not confuse it with the full alignment problem: roon (OpenAI): the best time to solve alignment might’ve been years ago, but the second best time to work on solving ai alignment is right now with realistic misalignment organisms and agentic age tooling. You certainly can do vastly more efficient work on some parts of the problem now, also on pretty much every cognitive task. The danger is that this has the mark of all OpenAI talk about alignment, which is that the thing to be solved is the thing you are seeing. The ‘realistic misalignment organisms’ are treated as exhibiting the problems to be solved, as not being different in kind from any key future problems, rather than as mere hints and harbingers, or as fatal counterexamples. Which by default leads here, note that Roon doubles down: Eliezer Yudkowsky : Why would any of that apply to the post-transformer non-LLM AI that Dario will whip Mythos 5.4 into building for him? Your kind has learned nothing general that I could not have already told you in 2016. roon (OpenAI): post LLM ai will still be a deep neural network. software only singularity can’t switch substrate from large amounts of matmuls and activations. if people were to, say, solve mechanistic interpretability, or understand fundamental truths about NN optimization, it will last Drake Thomas (Anthropic): I expect the vast majority of 2026 empirical alignment work to be of very little value for aligning agents made of idealized computronium. But I think there is a good chance that it is possible to make roughly-LLM-shaped agents which are significantly smarter than any human across ~all intellectual domains, and I would really like it if THOSE agents (1) didn’t coherently pursue malign goals (2) put forth a great deal of non-reward-hacky effort to solve different and harder alignment problems for weirder models according to a nuanced understanding of what humans actually want. (Or get us a pivotal act if that’s easier.) And 2026 research seems pretty relevant to having that go well! New Blog, Who Dis Dean Ball, in addition to his continued other posting, is starting an OpenAI ‘AI Futures’ blog , from their new Strategic Futures team, although the blog name is being reconsidered because of the existing use of the name ‘AI Futures Project’ by the creators of AI 2027 and Plan A (aka AI 2040). Dean Ball is, by his own account, not ASI pilled. The framing here implicitly reflects this. Up front, Dean Ball makes it clear they will focus on ‘concentration of power’ as the primary concern facing us, with other sources of existential risk considered relatively minor. Dean is clear he is not asking for the most radical decentralization of power possible, that a balance of power must be struck ‘so that no single actor or small set of actors can dominate the rest,’ and that they ‘must be open to’ risks that do not originate with malicious humans. I appreciate that this is not an entirely one sided framing. It is still rather close to a one sided framing. It says [X] (where [X] is concentration of power) is the primary risk, then explains why [X] is a risk, but does not explain why [Y] (where [Y] is, among other things, loss of control, gradual disempowerment , alignment risks and so on, the traditional existential risk concerns) is less of a risk. Hell, he doesn’t even name [Y], except to refer to the ‘AI safety community.’ Concentration of power is a real concern but when the handle is centralized then by default it leads to a cluster of naive thinking and proposals that, if AI were sufficiently advanced, would reliably disempower humanity and get us all killed, even if we did not have a vulnerable world, defense was advantaged over offense and alignment (including alignment-to-user) was essentially solved. Jan Kulveit : I like the fact part is re-statement of gradual disempowerment . I dislike ‘concentration of power’ as a conceptual handle for this problem, partially for reasons you can also see in the post – once you use that, instant response is ‘decentralization/superintelligence to every household’ – and you end up arguing against naive decentralization proposals. Humans can be disempowered in highly distributed and decentralized fashion. Any given blog, of course, can and should focus on whatever it wants. Like Jack Clark I am totally fine with the idea of hosting such a blog inside OpenAI assuming it is sincere. If this had been ‘our team is choosing to focus on [X]’ rather than claiming [X] was the primary risk over all [Y], I’d have zero problem with that. Tyler Cowen is thinking along related lines. He suggests a different approach than government AI regulation , to do all the regulating privately, as ‘AI safety has become too important to be left up to Washington’s whims,’ and says he is drawing this in part from Dean Ball. I agree that the labs should be facilitated and permitted to engage in partnerships around such issues, but it is quite a statement to say that AI regulation is too important to be done by the Federal Government. If true, they will not long remain the government, and we can only hope the new regime is human and an improvement. Other People Are Not As Worried About AI Killing Everyone There are people who think that if you’re doing recursive self-improvement, that means what you are doing is harmless, it will never do anything real, relax. Strangely, some of them know about or have played 4X games, or even Factorio. sunil pai : “Look at my incredible new factory!” Yo that’s cool, what do you make? “It’s highly optimised, fully automated, zero tolerance for defects and with a continuous feedback cycle” Cool cool, so what do you actually make? “I can interact with it on my phone, laptop, messenger, completely async, and the shared context means it’s always learning how to get better” Very impressive, but what do you make? “Every agent has full context, can spawn other agents, review their work, fix defects, and ship continuously.” yes yes. WHAT DOES IT MAKE? “Software.” Oh nice. What software? “Well right now we’re mostly using it to improve the factory.” Improve it to make what? “Anything!” Such as? “…a better factory.” I always try to get my enemies to play factorio. Is there a failure mode here? Sure. But if you can’t tell the difference between a process that ultimately caches out in real things versus Number Go Up, you are going to have an extremely bad and potentially brief time. The Lighter Side Okay, you got me with this one, well played. I mean horribly played, but great line. roon (OpenAI): “ Two weeks to flatten the reward hacking curve “ Damn it, I forgot about Dre again. Yes, the man goes hard. He is talking within the context of creating music. I about half agree with him there. AI is still going to eat a bunch of the pie, and it is the worst it will ever be. This, however, I remember all too well. Peter Wildeford : Trojan Horse: – self-certified as safe by its developers, no government review – only independent evaluator (Laocoon) was eaten by sea serpents – Cassandra’s risk assessment was dismissed as having too much “doomer” energy – violates voluntary commitment to Zeus’s law, but commitment adherence waived due to race dynamics and competition – Priam’s lawyers are still reviewing whether he had emergency powers to block it or needed an act of the Trojan Senate Discuss
Score: 35🌐 MovesAug 28, 2026https://www.lesswrong.com/posts/JaGWyjnqJzvSAuojc/ai-183-pre-post-mortem - Language under construction. A constructionist perspective on language in machines versus humans.
A:Veronica Juliana Schmalz; TT;PhD defense; RL:Language, Speech & Vision;
- ASUS ExpertBook B5 Flip G2: Lightweight AI-powered laptop tailored for students
ASUS ExpertBook B5 Flip G2: Lightweight AI-powered laptop tailored for students Gulf News
- MBZUAI’s Emirati women turning opportunity into impact - MBZUAI
MBZUAI’s Emirati women turning opportunity into impact MBZUAI - Mohamed bin Zayed University of Artificial Intelligence
Score: 34🌐 MovesAug 28, 2026https://mbzuai.ac.ae/news/mbzuais-emirati-women-turning-opportunity-into-impact/ - Interpretable Text Classification: Probing Scikit-LLM Embedding Spaces
In this article, you will learn how to use probing classifiers, UMAP visualization, and SHAP values to interpret and analyze the quality of text embeddings...
Score: 34🌐 MovesAug 28, 2026https://machinelearningmastery.com/interpretable-text-classification-probing-scikit-llm-embedding-spaces/