TLDR Data 2026-09-28

Get advice from 15+ leaders on how to build and monitor reliable intelligent agents for better business outcomes in this book from Amazon Web Services (AWS).

See their unique perspectives on topics such as agentic AI governance, risk management, evaluation, and deployment. Get your digital copy

Robotics Harness Optimization used coding agents to evolve a Graph-as-Policy robot controller, boosting simulated throughput 5.27x and success from 67% to 96%. The strongest version only rewired existing skills, without editing them or using human demonstrations.

Pinterest added a finalization layer to its Kafka, Flink, Spark, and Iceberg CDC pipeline so downstream jobs can tell when an hourly partition is safe to read. Flink checkpoints capture event-time statistics, which become Iceberg snapshot metadata and a non-regressing watermark. The reusable signal makes freshness-versus-completeness explicit without introducing another coordination service.

WHOOP cut a 15.8-million-task internal ML simulation from more than two months to six days by removing per-task HTTP calls, loading the model in workers, and tuning pod CPU. It also documents why Python fork deadlocked native libraries, how spawn fixed it, and how SQS chain state plus atomic S3 writes made retries idempotent.

Airbnb extended Chronon to update guest-journey features from batch refreshes to event-driven processing, cutting feature staleness from roughly two days to under a minute. Push Mode and near-real-time model transforms feed embeddings into online ranking while preserving offline training consistency. The architecture illustrates how feature platforms can bridge streaming updates, experimentation, and low-latency serving.

Postgres logical replication has steadily replaced custom data-movement plumbing with core SQL features: row filters, column lists, partitioned targets, streaming apply, parallel apply, and safer loop prevention. This retrospective connects those releases to practical migration and hub-and-worker scaling patterns. It is a useful checklist for teams revisiting replication slots, failover, write routing, and schema evolution.

PostgreSQL 19 adds built-in REPACK (CONCURRENTLY) for online table rewrites, avoiding extensions while generally running faster and generating less WAL than pg_repack or pg_squeeze. The tradeoffs are significant: it can delay VACUUM across the cluster, briefly block writes during the final swap, and currently has a ceiling of approximately 105 million updates/deletes per run.

Your AI agent's accuracy comes down to the data pipeline behind it. Data Pipelines and Lineage is a guide to engineering that pipeline: source-to-response lineage, data quality checks, and pipeline observability. Read the guide in AWS Marketplace.

Quail is an open-source AI-SQL engine that jointly optimizes query planning and LLM inference, running 1.84x faster than tuned vLLM on average and up to 14x faster on some workloads. It reduces wasted KV-cache work and GPU scheduling overhead, making large-scale LLM filtering and joins much cheaper.

Safe Not Safe is a browser-based PostgreSQL migration checker that flags risky DDL patterns like blocking locks, rewrites, and unsafe constraint rollouts. It runs entirely locally in the browser, with no API, uploads, logs, or account required.

Iceberg's View Spec makes metadata portable by versioning schemas, dependencies, properties, and dialect-tagged SQL text, but it does not standardize execution or authorization. Production portability still depends on REST-catalog support and compatible SQL dialects. Teams should test dependency integrity, concurrent replacement, and cross-engine result consistency before treating views as shared control-plane assets.

Apache Parquet added ALP, a new lossless encoding for floating-point data that delivers compression similar to ZSTD while decoding around 10x faster. It also enables much faster random access and parallel decoding, making it especially useful for prices, coordinates, and scientific measurements.

Netflix describes how Spark jobs on managed compute exchange a cloud execution role for an internally trusted workload identity. A signed metadata payload and AWS identity proof are verified before short-lived certificates are issued for mutual TLS. The pattern separates cloud-provider authorization from application trust, giving the platform stronger workload identity and more auditable access boundaries.

OpenAI's AI data stack centralizes rich metadata to close the context gap, while a Rust rewrite reportedly cut Airflow scheduling latency 35x at 70,000+ concurrent tasks.

Opus 5.5 leads Hex's DataBench, scoring 70.5% at Max reasoning and 67.0% at XHigh, but at relatively high cost and latency.