TLDR Data 2026-09-07

Semantic layers define business metrics, context layers provide supporting knowledge, and ontologies describe entities and their relationships. Together, they help AI agents interpret data, but people still need to keep that knowledge accurate and current.

Most of what a coding agent does is I/O rather than reasoning, such as reading five files to answer a question about one method. Spotify routed that work to two declarative Portal modes, bulk-reader and code-writer, both running Gemini 2.5 Flash on an ephemeral runtime, and reserved the frontier model for problems that need it. A quarter of engineering leaders already spend $200 to $500 per developer per month on tokens, and some are past $2,000.

This post explains why Meta put a managed proxy tier in front of ZippyDB, its massive key-value store. The real lesson is that proxies can turn client-fleet sprawl into a controllable infrastructure boundary: connection fan-in drops roughly 97–98%, batching and coalescing happen across clients, excess traffic from noisy tenants can be selectively rejected, and failover and load-balancing policy can change centrally without waiting for a million clients to update.

Vortex is an open-source columnar format designed to stream training data from S3 to GPUs with minimal copying. Cascading lightweight encodings, layout-aware pruning, and zero-copy memory paths target CPU and NVMe bottlenecks, giving ML platforms a way to keep expensive accelerators fed without converting everything upfront.

AI analytics can fail when agents generate free-form SQL against inconsistent warehouse metadata. Governed systems should route repeated questions through verified semantic-layer tools, parameterized calls, and explicit refusal paths, reserving open-ended SQL for cases humans can inspect. Maintaining definitions is the hard part regardless of model upgrades.

A 100,000-row Postgres load took 41 minutes because each row became its own statement and transaction; COPY cut the same work to 11 seconds. Real batching removes network round trips, parse overhead, and WAL flush barriers, while driver flags and transaction chunking prevent bulk APIs from silently behaving row by row.

Nao's founders argue that AI is shifting data teams toward maintaining business context and enabling self-service analytics. They recommend starting with one focused use case, measuring results, and improving data models before investing in a full semantic layer.

Sail 0.7 pushes Spark-style intermediate state toward object storage with blocking shuffle and checkpoint support. It persists shuffle outputs as compressed Arrow streams so downstream task regions retry without cascading upstream failures, while checkpointing keeps iterative jobs from rebuilding long query plans. The design targets stateless workers, smaller clusters, and better resilience under preemption.

Polars 2.0 makes the streaming engine the default for lazy queries, aiming for much lower memory use and faster execution while accepting that row order is not guaranteed unless requested. It also tightens type coercion and concatenation behavior, and removes deprecated APIs so data mismatches fail earlier instead of hiding in production pipelines.

Apache Iggy is a Rust streaming engine with Kafka-like logs and consumer groups, optimized around thread-per-core execution and modern Linux I/O. It looks promising for low-latency single-node or simpler deployments, but Kafka still has the deeper production story for replication, exactly-once semantics, compaction, and multi-region operations.

Warner Music Group turned a hand-edited anti-counterfeit tracking spreadsheet into governed Databricks tables without taking the spreadsheet away from operators. The pipeline handles shifting tabs, renamed headers, late edits, row fingerprints, and full-overwrite loads so executive reporting can trust a messy business interface that people actually use.

Text-to-SQL assumes users ask questions that a single query can answer, when in practice they bring problems that need a sequence of queries with reasoning over the intermediate results. The paper proposes data investigations as the replacement paradigm and presents D2, a prototype that autonomously searches, reasons over, and collects data to reach evidence-backed conclusions, benchmarked on a Murder Mystery dataset.

DuckFlight lets PostgreSQL tools and Arrow Flight SQL clients read and write the same live DuckDB database without a separate service or copying data.

PostgreSQL 19 Beta 3 adds temporal SQL, idempotent upsert-and-fetch, IGNORE NULLS window behavior, concurrent repacking, DDL extraction helpers, SQL/PGQ graph querying, and stricter NULL handling.