TLDR Data 2026-09-14

What if Postgres compute was stateless? Imagine a branch that is a metadata operation that runs in seconds, not hours. What if you could also run transactions and analytics on the same copy of data in real time without delays or CDC? One architecture, two problems solved.

Unlock unlimited storage, elastic compute, durable writes and instant branching

Run analytics against the freshest transactional data

Lyft rebuilt its outdated neighborhood travel-time dataset with more accurate ETAs, better coverage, drivable-location filtering, and self-service fields, improving how pricing and driver heatmaps understand nearby supply. It will now refresh every six months, with time-aware ETAs planned to reflect changing traffic throughout the week.

Diskless Kafka keeps the Kafka API while moving durable payload storage from broker-owned logs to object storage and strongly consistent metadata. The tradeoff is not just cheaper retention: batching, offset assignment, recovery, compaction, latency, and feature support all change, making diskless topics a per-workload storage choice rather than a universal Kafka replacement.

Pinterest's Manas retrieval platform serves billions of embeddings across Home Feed, Search, Related Pins, Ads, and Notifications. Quantization is cutting serving cost by 20–30%, SSD-backed ANN experiments reduce memory and CPU needs, and multi-embedding retrieval moves beyond single-vector two-tower matching toward richer candidate scoring.

Coding agents now produce competent PostgreSQL schemas, but they tend to stack indexes onto hot operational tables. In a measured support-ticket workload, extra indexes nearly doubled update time and WAL while increasing VACUUM work. Index review needs write-path evidence, HOT-update awareness, and usage counters, not just fast read plans.

AI agents can return successful, well-grounded responses that are still wrong because they retrieved invalid context or followed a faulty execution path. Debugging requires end-to-end traces of model calls, tool calls, retrieval inputs, and outputs, plus separate tests for deterministic system behavior and answer quality.

Most AI agents fail confidently, not loudly. They act on stale data and produce wrong refunds, outdated quotes, and contradictory decisions. A context engine computes real-time values at the moment of each decision, so your agents act on current, accurate context. See exactly how it works.

PlanetScale set out to run 1 million queries per second against Neki, its new sharded Postgres, hit that on 5 shards almost immediately, then kept going until 512 shards were serving 118 million queries per second across 1.22 PiB of data. The workload is deliberately narrow: a single-shard point select fetching one row by primary key with no writes, joins, or cross-shard queries, which is what makes the scaling curve clean.

ChatGPT Work's Data agent connects to approved company data and business definitions, letting employees investigate questions and build shareable dashboards using plain language. It respects existing permissions, works with major data and BI platforms, and can recommend or carry out approved follow-up actions.

DuckDB 2.0 speeds up S3 queries by 2-3x through asynchronous I/O, dramatically improves deep recursive queries, and makes semi-structured VARIANT data smaller and faster than JSON. It also adds practical SQL features including triggers, nested schemas, and data-modifying CTEs.

Comparing AI adopters with non-adopters usually exaggerates impact because more engaged customers are also more likely to adopt. Instead, estimate causality using variation that customers did not choose, such as an eligibility threshold, while clearly limiting conclusions to the population and decision it measures.

Large PostgreSQL CDC backfills fail less from historical reads than from WAL retention, delayed slot acknowledgments, long-running transactions, and stale snapshot rows racing live changes. Safer designs keep consuming WAL during chunked reads, checkpoint progress, reconcile overlaps with watermarks and fences, and size chunking around real write rates.

Four production Spark cases cut runtime, compute, S3 operations, and costs by tracing hidden waste beyond top-level CPU and memory metrics.

Tools like dbt turn fragile, undocumented analysis into reproducible systems, while AI often misses the constraints of real workplaces.

Jetpack adds a portable 1-RTT fast-commit path to existing consensus systems, cutting latency for write-heavy workloads by up to 60% while adding redundant work, delaying execution, and requiring careful recovery to remain safe during leader changes.