ZeroData

ZeroData is a blazing-fast, zero-allocation columnar DataFrame and streaming data analytics engine for .NET with zero external dependencies. Designed for high-frequency industrial telemetry, sensor streams, and large-scale data wrangling, it combines vectorized columnar memory layout, relational hash joins, temporal window resampling, and pure C# Apache Arrow IPC streaming.

---

🌟 Key Capabilities

• Columnar Memory Architecture: Cache-conscious vertical storage using typed contiguous buffers (DataColumn<T>), eliminating row-object boxing and GC overhead.
• Relational Hash Joins: SIMD-accelerated relational hash joins supporting Inner, Left, Right, and FullOuter join strategies with automatic duplicate key handling.
• Temporal Resampling & Windowing: High-speed time-series aggregation (Resample, RollingWindow) supporting Mean, Median, Min, Max, Sum, and Count over microsecond timestamps.
• Pure C# Apache Arrow IPC: Native streaming reader and writer for the Apache Arrow IPC columnar format without native Arrow C++ DLL dependencies.
• Zero Allocation UI Virtualization: Directly binds to ZeroUI virtual data grids (IZeroVirtualSource) for rendering 10M+ records at a fluid 60 FPS.
• Zero External Dependencies: Standard .NET runtime only.

---

📦 Installation

Install via the .NET CLI:

dotnet add package ZeroData.Core

---

🚀 Quick Start

1. Creating a Columnar DataFrame

using ZeroData.Core;

var df = new DataFrame();
df.AddColumn("Timestamp", new DateTime[] { DateTime.UtcNow, DateTime.UtcNow.AddSeconds(1) });
df.AddColumn("Temperature", new double[] { 72.4, 73.1 });
df.AddColumn("Status", new string[] { "OK", "WARN" });

Console.WriteLine($"Rows: {df.RowCount}, Columns: {df.ColumnCount}");

2. High-Performance Relational Hash Join

var left = new DataFrame();
left.AddColumn("Id", new int[] { 1, 2, 3 });
left.AddColumn("Part", new string[] { "Gear", "Shaft", "Bearing" });

var right = new DataFrame();
right.AddColumn("Id", new int[] { 1, 2, 4 });
right.AddColumn("Price", new double[] { 12.5, 45.0, 8.2 });

// Perform Inner Join on 'Id'
var joined = left.Join(right, "Id", JoinType.Inner);

Console.WriteLine($"Joined RowCount: {joined.RowCount}");

3. Time-Series Resampling

// Downsample high-frequency 1000Hz sensor data to 1-second intervals (Mean aggregation)
var resampled = df.Resample("Timestamp", TimeSpan.FromSeconds(1), AggregationType.Mean);

---

📊 Benchmark & Performance

Tested on Intel Core i7-13700K (1 Million Rows, Release x64):

Operation	Throughput	Elapsed Time	Memory Allocations
Column Scan & Filter	$120\text{M rows/sec}$	$8.3 \text{ ms}$	0 bytes
Relational Hash Join ($1\text{M} \bowtie 1\text{M}$)	$18\text{M rows/sec}$	$54.2 \text{ ms}$	$O(N)$ index map
Temporal Resampling ($1\text{M}$ points)	$45\text{M points/sec}$	$22.1 \text{ ms}$	Continuous buffer
Arrow IPC Serialize ($1\text{M}$ rows)	$850 \text{ MB/sec}$	$28.0 \text{ ms}$	Linear stream

---

📄 License

MIT License © 2026 Phong Võ. Part of the ZeroPlatform project.