{"slug": "edge-ai-with-net-part-4-time-series-that-outlives-the-hardware", "title": "Edge AI with .NET, Part 4: Time Series That Outlives the Hardware", "summary": "A developer outlines a .NET edge-to-cloud time-series architecture for water and power meter fleets, using TimescaleDB hypertables, Hypercore columnstore compression, and continuous aggregates to keep readings tied to logical meter identities rather than device serials. The approach cuts cold-storage costs by 90 to 95 percent and avoids history loss when hardware is swapped or upgraded. The writeup covers the PostgreSQL schema, EF Core mapping, and an anomaly worker, and notes TimescaleDB 2.29.0's incremental and concurrent continuous-aggregate refresh.", "body_md": "You have a fleet of water and power meters, each running a small .NET edge agent that sends readings every minute. The hardware will fail, be replaced, or be upgraded. The readings must not. If your schema ties a reading to a device serial number rather than a logical meter identity, you lose history the moment you swap hardware. If you store readings in a plain PostgreSQL table, you pay ten times the storage cost and wait ten times as long for rolling-window queries.\n\nTimescaleDB (now shipped by TigerData, renamed from Timescale Inc. on June 17, 2025) solves both the storage and the query problem. Version 2.29.0, released July 28, 2026, adds incremental and concurrent continuous-aggregate refresh, vectorised `time_bucket()` from 2.26 onward, and and the Hypercore columnstore engine introduced in 2.18. All of them matter for the patterns below.\n\nThis article covers the schema, the EF Core mapping, and the anomaly worker. Parts 1 to 3 covered the edge agent; here we focus on what happens after the reading lands in the cloud database.\n\nThe first rule: readings reference a *logical meter*, not a device serial. Keep a `meters` lookup table with a surrogate UUID, the physical serial, and install/remove timestamps. When hardware is swapped, insert a new `meters` row, and the old readings keep their original `meter_id`.\n\n```\n-- Locations and meters\nCREATE TABLE locations (\n    location_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),\n    address     TEXT NOT NULL\n);\n\nCREATE TABLE meters (\n    meter_id     UUID PRIMARY KEY DEFAULT gen_random_uuid(),\n    location_id  UUID NOT NULL REFERENCES locations(location_id),\n    device_serial TEXT NOT NULL,\n    meter_type   TEXT NOT NULL CHECK (meter_type IN ('water','power')),\n    installed_at TIMESTAMPTZ NOT NULL,\n    removed_at   TIMESTAMPTZ\n);\n\n-- Raw readings hypertable\nCREATE TABLE readings (\n    time        TIMESTAMPTZ     NOT NULL,\n    meter_id    UUID            NOT NULL,\n    value       DOUBLE PRECISION NOT NULL  -- litres or watt-hours\n);\n\nSELECT create_hypertable('readings', 'time',\n    chunk_time_interval => INTERVAL '7 days');\n\n-- Hypercore columnstore compression after 7 days\nSELECT add_columnstore_policy('readings',\n    after => INTERVAL '7 days',\n    segmentby => ARRAY['meter_id']);\n\n-- Tariff periods for cost calculation\nCREATE TABLE tariff_periods (\n    tariff_id   UUID PRIMARY KEY DEFAULT gen_random_uuid(),\n    valid_from  TIMESTAMPTZ NOT NULL,\n    valid_until TIMESTAMPTZ,          -- NULL means current tariff\n    eur_per_kwh NUMERIC(10,6) NOT NULL\n);\n```\n\nA 7-day chunk interval suits medium-frequency meter data (one reading per minute is 1,440 rows per day per meter). Hypercore's columnstore compresses cold chunks by 90 to 95 percent, so a year of data from a hundred meters costs roughly what a week would in plain PostgreSQL.\n\nStack the aggregates. An hourly rollup feeds a daily rollup, which feeds a monthly rollup. Each is its own hypertable, refreshed incrementally. The `WITH (timescaledb.continuous)` flag and `real_time_aggregate` option mean your anomaly queries always see the latest raw data without a forced refresh.\n\n```\n-- Hourly rollup\nCREATE MATERIALIZED VIEW readings_hourly\nWITH (timescaledb.continuous,\n      timescaledb.materialized_only = false) AS\nSELECT\n    time_bucket('1 hour', time)  AS bucket,\n    meter_id,\n    SUM(value)                   AS total,\n    MIN(value)                   AS min_val,\n    MAX(value)                   AS max_val,\n    COUNT(*)                     AS sample_count\nFROM readings\nGROUP BY bucket, meter_id\nWITH NO DATA;\n\nSELECT add_continuous_aggregate_policy('readings_hourly',\n    start_offset  => INTERVAL '2 hours',\n    end_offset    => INTERVAL '1 hour',\n    schedule_interval => INTERVAL '1 hour');\n\n-- Daily rollup sourced from hourly\nCREATE MATERIALIZED VIEW readings_daily\nWITH (timescaledb.continuous,\n      timescaledb.materialized_only = false) AS\nSELECT\n    time_bucket('1 day', bucket)  AS bucket,\n    meter_id,\n    SUM(total)                    AS total,\n    MIN(min_val)                  AS min_val,\n    MAX(max_val)                  AS max_val,\n    SUM(sample_count)             AS sample_count\nFROM readings_hourly\nGROUP BY time_bucket('1 day', bucket), meter_id\nWITH NO DATA;\n\nSELECT add_continuous_aggregate_policy('readings_daily',\n    start_offset  => INTERVAL '2 days',\n    end_offset    => INTERVAL '1 day',\n    schedule_interval => INTERVAL '1 day');\n```\n\nMonthly rollups follow the same pattern using `'1 month'` and sourcing `readings_daily`.\n\nEF Core has no native understanding of hypertable DDL. The `YC.EntityFrameworkCore.TigerData.TimescaleDB` package (targeting EF Core 10 / Npgsql 10, requires TimescaleDB 2.23+) exposes a Fluent API that injects the correct `migrationBuilder.Sql(...)` calls into generated migrations. For bare projects, execute raw SQL directly in `Up()`.\n\nThe `Reading` entity is straightforward; the important thing is that `MeterId` is the foreign key, not a serial string:\n\n```\n// Reading.cs\npublic class Reading\n{\n    public DateTimeOffset Time     { get; set; }\n    public Guid           MeterId  { get; set; }\n    public double         Value    { get; set; }\n\n    public Meter Meter { get; set; } = null!;\n}\n\n// Meter.cs\npublic class Meter\n{\n    public Guid            MeterId      { get; set; }\n    public Guid            LocationId   { get; set; }\n    public string          DeviceSerial { get; set; } = string.Empty;\n    public string          MeterType    { get; set; } = string.Empty;\n    public DateTimeOffset  InstalledAt  { get; set; }\n    public DateTimeOffset? RemovedAt    { get; set; }\n\n    public Location          Location { get; set; } = null!;\n    public ICollection<Reading> Readings { get; set; } = [];\n}\n\n// MeterDbContext.cs (relevant excerpt)\nprotected override void OnModelCreating(ModelBuilder modelBuilder)\n{\n    modelBuilder.Entity<Reading>(e =>\n    {\n        e.HasKey(r => new { r.Time, r.MeterId });\n        e.Property(r => r.Time).HasColumnType(\"timestamptz\");\n        e.HasOne(r => r.Meter)\n         .WithMany(m => m.Readings)\n         .HasForeignKey(r => r.MeterId);\n    });\n\n    modelBuilder.Entity<Meter>(e =>\n    {\n        e.HasKey(m => m.MeterId);\n        e.Property(m => m.RemovedAt).HasColumnType(\"timestamptz\");\n    });\n\n    // If not using the community package, call the hypertable\n    // DDL from the migration's Up() method via migrationBuilder.Sql()\n}\n```\n\nImportant gotcha: TimescaleDB cannot apply certain schema changes in place on an existing hypertable, it has to recreate the table. Design your schema before production data volumes grow large; adding a column after millions of rows have been ingested is a heavy operation.\n\nAlso: the 2.27.x bloom-filter bug caused compressed `int2`/` SMALLINT` columns to silently miss matching rows in `SELECT` queries. The workaround is to drop the affected sparse indexes manually before upgrading. On 2.28+ this is resolved, but avoid `SMALLINT` on heavily compressed columns until you have confirmed your version.\n\nA single `BackgroundService` polls every minute and runs three independent checks:\n\n**Leak detection.** A water meter with non zero flow for six consecutive hours. Use `readings_hourly` with `real_time_aggregate` so no explicit refresh is needed:\n\n```\npublic class AnomalyWorker(MeterDbContext db,\n                           ILogger<AnomalyWorker> logger,\n                           IAlertService alerts)\n    : BackgroundService\n{\n    protected override async Task ExecuteAsync(CancellationToken ct)\n    {\n        using var timer = new PeriodicTimer(TimeSpan.FromMinutes(1));\n        while (await timer.WaitForNextTickAsync(ct))\n        {\n            await CheckLeaksAsync(ct);\n            await CheckPowerSpikesAsync(ct);\n            await CheckSilentMetersAsync(ct);\n        }\n    }\n\n    private async Task CheckLeaksAsync(CancellationToken ct)\n    {\n        var cutoff = DateTimeOffset.UtcNow.AddHours(-6);\n\n        // readings_hourly is a cagg, queried like a normal table\n        var leaking = await db.Database\n            .SqlQuery<Guid>($\"\"\"\n                SELECT h.meter_id AS \"Value\"\n                FROM   readings_hourly h\n                JOIN   meters m ON m.meter_id = h.meter_id\n                WHERE  m.meter_type = 'water'\n                  AND  m.removed_at IS NULL\n                  AND  h.bucket >= {cutoff}\n                  AND  h.min_val > 0\n                GROUP  BY h.meter_id\n                HAVING COUNT(*) >= 6\n                \"\"\")\n            .ToListAsync(ct);\n\n        foreach (var meterId in leaking)\n            await alerts.RaiseAsync(meterId, \"LeakDetected\", ct);\n    }\n\n    private async Task CheckPowerSpikesAsync(CancellationToken ct)\n    {\n        // 7-day baseline: flag today if today's total > avg + 2*stddev\n        var spikes = await db.Database\n            .SqlQuery<Guid>($\"\"\"\n                WITH baseline AS (\n                    SELECT meter_id,\n                           AVG(total)    AS avg_total,\n                           STDDEV(total) AS std_total\n                    FROM   readings_daily\n                    WHERE  bucket >= NOW() - INTERVAL '8 days'\n                      AND  bucket <  NOW() - INTERVAL '1 day'\n                    GROUP  BY meter_id\n                ),\n                today AS (\n                    SELECT meter_id, SUM(total) AS day_total\n                    FROM   readings_hourly\n                    WHERE  bucket >= date_trunc('day', NOW())\n                    GROUP  BY meter_id\n                )\n                SELECT t.meter_id AS \"Value\"\n                FROM   today t\n                JOIN   baseline b ON b.meter_id = t.meter_id\n                WHERE  t.day_total > b.avg_total + 2 * COALESCE(b.std_total, 0)\n                \"\"\")\n            .ToListAsync(ct);\n\n        foreach (var meterId in spikes)\n            await alerts.RaiseAsync(meterId, \"PowerSpike\", ct);\n    }\n\n    private async Task CheckSilentMetersAsync(CancellationToken ct)\n    {\n        var threshold = DateTimeOffset.UtcNow.AddMinutes(-15);\n\n        var silent = await db.Database\n            .SqlQuery<Guid>($\"\"\"\n                SELECT m.meter_id AS \"Value\"\n                FROM   meters m\n                WHERE  m.removed_at IS NULL\n                  AND  NOT EXISTS (\n                    SELECT 1 FROM readings r\n                    WHERE  r.meter_id = m.meter_id\n                      AND  r.time    >= {threshold}\n                  )\n                \"\"\")\n            .ToListAsync(ct);\n\n        foreach (var meterId in silent)\n            await alerts.RaiseAsync(meterId, \"SilentMeter\", ct);\n    }\n}\n```\n\nThe silent-meter check queries the raw hypertable rather than an aggregate, because chunk pruning on `r.time >= threshold` limits the scan to one or two recent chunks regardless of how many years of history exist.\n\nTariffs change. A reading from 2023 must be costed at the 2023 rate, not today's. Join through `tariff_periods` using a lateral range condition:\n\n```\nSELECT\n    d.meter_id,\n    d.bucket,\n    d.total / 1000.0                   AS kwh,\n    tp.eur_per_kwh,\n    (d.total / 1000.0) * tp.eur_per_kwh AS eur_cost\nFROM readings_daily d\nJOIN tariff_periods tp\n  ON  d.bucket >= tp.valid_from\n  AND (tp.valid_until IS NULL OR d.bucket < tp.valid_until)\nWHERE d.meter_id = @meterId\nORDER BY d.bucket;\n```\n\nStore `valid_until = NULL` for the active tariff. When a new tariff starts, set `valid_until` on the previous row to the change timestamp; insert the new row. That gives you an append only audit trail, so you never lose what a customer paid in an earlier period.\n\nTimescaleDB 2.29 removed PostgreSQL 15 support; if your managed database is on PG15, you are on the 2.28.x train until you upgrade the engine. The Hypercore columnstore is now the recommended default for new installs, so enable it from day one. Retrofitting compression onto a large existing hypertable is possible but slow.\n\nThe community EF Core packages (both `YC.EntityFrameworkCore.TigerData.TimescaleDB` and `CmdScale.EntityFrameworkCore.TimescaleDB`) are thin wrappers that emit raw SQL migrations. They save repetitive boilerplate, but you should read the generated migration SQL before applying it, especially the `compress_segmentby` and `chunk_time_interval` settings, which are hard to change once data is flowing.", "url": "https://wpnews.pro/news/edge-ai-with-net-part-4-time-series-that-outlives-the-hardware", "canonical_source": "https://dev.to/mehdimohseni82/edge-ai-with-net-part-4-time-series-that-outlives-the-hardware-4p14", "published_at": "2026-10-07 14:10:42+00:00", "updated_at": "2026-10-07 14:18:04.403863+00:00", "lang": "en", "topics": ["mlops", "ai-infrastructure"], "entities": ["TimescaleDB", "TigerData", "PostgreSQL", ".NET", "EF Core"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/edge-ai-with-net-part-4-time-series-that-outlives-the-hardware", "markdown": "https://wpnews.pro/news/edge-ai-with-net-part-4-time-series-that-outlives-the-hardware.md", "text": "https://wpnews.pro/news/edge-ai-with-net-part-4-time-series-that-outlives-the-hardware.txt", "jsonld": "https://wpnews.pro/news/edge-ai-with-net-part-4-time-series-that-outlives-the-hardware.jsonld"}}