Quiet Day, Loud Ideas: OpenAI Caps Astra, AI Shrinks to 66 MiB, and Speech Goes Fully Local

OpenAI Puts the Brakes on Astra

OpenAI says its upcoming Astra model shows such significant advances in agentic coding and cybersecurity that it cannot rule out a "critical" capability level under its own Preparedness Framework. Rather than shipping, the lab is pausing internal work that doesn't meet strengthened controls, tightening network and tool access, and hardening the security of the model's weights, while still aiming to get it "into the hands of defenders." It's one of the clearest public cases of a frontier lab voluntarily constraining a model over cyber-risk.

Claude Code Learns to Talk to Itself

Anthropic's Claude Code now lets one session message another, even across machines, summarising what matters instead of shipping full files and history. Anthropic is also making classifier-mediated "auto mode" the default for paying users; in internal testing the classifier caught around 89% of dangerous shell commands versus just 14% for manual approval alone.

The Harness Matters More Than the Model

New agent-coding analysis found that swapping the harness around a model changed results more than most model upgrades, with performance on one benchmark swinging from roughly 23% to 52% on the same model. The lesson: a smaller model in a great scaffold can rival a giant model in a poor one, so the winning strategy is often the best routing and setup rather than the flashiest checkpoint.

Databricks Cuts AI Spend by up to 90%

Databricks detailed how it slashed internal AI coding costs by up to 90% even as usage grew, via cheaper default models, smart routing of hard tasks to premium models only when needed, spend visibility and adaptive budgets, and trimming context bloat.

vLLM, Now 66 MiB

An engineer ported vLLM's serving stack to C++20, producing a roughly 66 MiB binary with no Python at inference time, versus a ~9.1 GiB install. Output was verified token-for-token against the original while matching throughput, keeping features like continuous batching and paged KV cache.

Blackwell Pushes Qwen Serving to 25K tokens/s

A vLLM and NVIDIA deep dive optimised Qwen serving to around 25,000 tokens per second per GPU on Blackwell-class GB200 hardware, using optimised kernels, hybrid cache and state transfer, and race-free async scheduling. Faster serving means a lower cost per answer for everyone downstream.

Qdrant Turbo4 Shrinks Vector Storage 9x

Qdrant 1.19 introduced Turbo4, storing each vector value in just 4 bits for roughly 9x less storage than a float32 plus quantised copy, trading away some rescoring precision for space and throughput. This is the plumbing behind AI memory and semantic search.

NVIDIA's Speech Stack Goes Fully Local

NVIDIA's automatic speech recognition, text-to-speech, and audio codec models can now run on-device in a compact quantised format, with the community already running them on hardware as small as a Raspberry Pi and building phone voice keyboards. The privacy win: your voice never leaves your device. The remaining gap is an efficient always-on wake-word layer.