Icebug-format, a new open-source graph standard combining Apache Arrow and Compressed Sparse Rows (CSR) into an immutable, interoperable format, has been released under the MIT license. The format, wh…
Apache Iceberg voted to deprecate equality deletes in V4, forbidding new writes while keeping read support for backward compatibility. The community also debated whether V4 manifests should be Parquet…
PyTorch developers can eliminate GPU idle time caused by CPU bottlenecks by using multi-process data loading with `num_workers` set to the number of CPU cores, `pin_memory=True`, and binary formats li…
Spotify introduced Random Access Parquet (RAP), a storage architecture that adds an external indexing layer over Apache Parquet files to enable low-latency point queries directly against its data lake…
Apache Parquet's Dremel encoding imposes a roughly 3× overhead for fixed-length lists like vector embeddings, but the upcoming Hardwood release introduces a fast path that detects data pages with unif…
OpenObserve, an open-source observability platform, crossed 20,000 GitHub stars on July 15 and offers native LLM monitoring at $3 per day, compared to Datadog's $174 per day for the same workload. The…
A developer at Dremio details the renaissance in columnar file formats driven by AI workloads and hardware evolution. The analysis covers how Parquet's assumptions are breaking under modern demands, a…
The Apache data lakehouse community focused on governance this week, with Parquet voting on versioned releases and Polaris aligning with the incubating Ossie semantics project. Iceberg Rust released R…
AWS launched a new log analytics engine for Amazon OpenSearch Service that reduces storage costs by 70% and improves price-performance, addressing rising telemetry costs from AI and agentic applicatio…
A developer building a virtual running app, In the Long Run, used AI to enrich maps with points of interest from the GeoNames dataset, but found AI's role was limited to supporting data processing rat…
The Apache Iceberg community debated a v2 loadTable endpoint for the REST catalog protocol this week, with Ryan Blue proposing the change to support optional locations and snapshots while moving crede…
Acceldata claimed its Autonomous Data & AI Platform signals "the end of the data lakehouse era," arguing that lakehouse architecture designed for human access fails in the agentic era. AWS launched Am…
Gunnar Morling, a technologist at Confluent and Java Champion, discussed building high-performance Java applications in the data space, including his work on Hardwood, a zero-dependency Java parser fo…