TL;DR:

  • DuckDB runs as a library with no server process, making it practical to deploy on Raspberry Pi, Jetson, and industrial gateways for local SQL analytics.
  • Its columnar storage and vectorised execution give it a significant performance advantage over SQLite for aggregations, window functions, and time-series queries.
  • DuckDB is not designed for concurrent writes, so pair it with a lightweight write buffer or use it purely for read/query workloads on already-captured sensor data.

Most IoT deployments end up with the same architecture: sensors generate data, a gateway collects it, and everything gets shipped to the cloud for analysis. That works fine until you need a real-time anomaly alert, until your connectivity is intermittent, or until your cloud bill starts looking alarming at scale. The answer is usually some form of edge analytics: do more of the processing locally, send only what matters upstream. The question is which database you reach for when you need SQL on a constrained device.

SQLite is usually the first answer, and fair enough, it’s genuinely excellent for transactional workloads. But if you’re trying to run aggregations over time-series sensor data, GROUP BY window queries, rolling averages over the last hour of temperature readings, it starts to show its limits. SQLite is row-oriented, which is the wrong shape for analytical queries across wide time ranges.

DuckDB is a different beast entirely.

What makes DuckDB interesting for edge

DuckDB is an in-process analytical database. It runs as a library, embedded directly in your Python, C++, Go, or Rust application, with no server daemon, no network socket, no separate process to manage. That’s important for edge deployments where you want minimal overhead and simple deployment: copy the library, write your code, run it. There’s no service to configure, no port to open, no init system integration to worry about.

Under the hood, DuckDB uses columnar storage and vectorised execution. When you’re querying sensor data, you’re typically asking questions like “what was the average temperature per sensor over the last 24 hours, grouped by hour?” That query touches one column repeatedly across thousands of rows. Columnar storage puts that data adjacent in memory, which is exactly what you want for CPU cache efficiency. The vectorised execution model processes batches of values at once using SIMD instructions where available, which is why DuckDB’s GROUP BY performance on time-series data can be an order of magnitude faster than SQLite on the same hardware doing the same query.

Window functions are another area where DuckDB shines. Running totals, sliding averages, rank within a group: all of this is native SQL in DuckDB and performs well even on modest hardware. SQLite’s window function support improved in version 3.25, but the execution model still isn’t optimised for the kind of analytical patterns you encounter in IoT telemetry.

Deploying on constrained hardware

DuckDB runs comfortably on a Raspberry Pi 4 or 5 with 4GB of RAM. For an NVIDIA Jetson Orin NX or Xavier, you’ve got more headroom and can query larger datasets in memory. Industrial gateways running on x86 or ARM with 2-8GB RAM are a reasonable target as well.

Memory configuration is where you need to pay attention. By default, DuckDB will try to use a fair amount of available memory for its buffer pool. On a device where you’re also running your data collection agent, an MQTT broker, and possibly some ML inference, you want to constrain it explicitly:

import duckdb

con = duckdb.connect("sensors.db")
con.execute("SET memory_limit='512MB'")
con.execute("SET threads=2")

The memory_limit setting caps the buffer pool, and reducing threads from the default (which matches CPU count) prevents DuckDB from saturating your cores when other processes need them. On a Raspberry Pi with 4 cores, setting threads to 2 gives your ingestion pipeline breathing room.

DuckDB can also query Parquet and CSV files directly without importing them into a database file. If your sensor logging pipeline writes hourly Parquet files, you can query across them with standard SQL:

SELECT 
    date_trunc('hour', timestamp) AS hour,
    sensor_id,
    avg(temperature) AS avg_temp,
    max(temperature) AS peak_temp
FROM read_parquet('/data/sensors/2026-08-*.parquet')
WHERE sensor_id IN ('zone_a', 'zone_b')
GROUP BY 1, 2
ORDER BY 1, 2;

That runs directly against the files on disk. No import step, no intermediate database. For edge scenarios where data is already being written to Parquet by a separate process, this is genuinely elegant.

A practical Python example

Here’s a pattern for querying rolling temperature sensor data stored in DuckDB:

import duckdb
from datetime import datetime, timedelta

con = duckdb.connect("sensors.db")
con.execute("SET memory_limit='256MB'")

# Assumes a table: sensor_readings(sensor_id TEXT, timestamp TIMESTAMPTZ, temperature FLOAT)
result = con.execute("""
    SELECT
        sensor_id,
        date_trunc('minute', timestamp) AS minute,
        avg(temperature) AS avg_temp,
        avg(temperature) OVER (
            PARTITION BY sensor_id
            ORDER BY date_trunc('minute', timestamp)
            ROWS BETWEEN 9 PRECEDING AND CURRENT ROW
        ) AS rolling_10min_avg
    FROM sensor_readings
    WHERE timestamp >= now() - INTERVAL '1 hour'
    ORDER BY sensor_id, minute
""").fetchdf()

anomalies = result[result["avg_temp"] > result["rolling_10min_avg"] + 5]
print(f"Found {len(anomalies)} anomalous readings in the last hour")

The result comes back as a Pandas DataFrame, which slots naturally into any downstream processing or alerting pipeline.

DuckDB vs InfluxDB: where each one wins

InfluxDB is purpose-built for time-series data ingestion. Its line protocol handles high-frequency writes efficiently, its data model is optimised for tag-based filtering on sensor metadata, and its retention policy system handles data lifecycle automatically. If you’re ingesting thousands of readings per second from hundreds of sensors, InfluxDB’s write path is genuinely better suited to the task.

DuckDB wins on the analytics side. Ad-hoc SQL queries, joins across multiple data sources, complex aggregations, reading from Parquet archives: all of this is more natural and often faster in DuckDB. For edge scenarios where write frequency is moderate and you want flexible query capability, DuckDB is the better choice. You’re not limited to InfluxDB’s query language or its specific data model.

A practical combination, one that several industrial IoT teams are using in 2026, is to write with InfluxDB and export hourly summaries to Parquet, then use DuckDB for analytical queries against those archives. You get the best of both ingestion models.

The limitations you should know about

DuckDB is not designed for concurrent writes. Multiple processes writing to the same database file simultaneously will cause errors. If your architecture has several sensors writing in parallel, you need a write buffer in front of DuckDB, an MQTT subscriber that batches and writes sequentially, or a separate transactional store that you periodically bulk-load from. This isn’t a dealbreaker, but it does mean DuckDB fits best as a query engine sitting downstream of your ingestion layer, not as the ingestion layer itself.

It also doesn’t have a network server mode out of the box. That’s by design, it’s an embedded database. If you need remote query access, you’re either querying through your application or using a separate tool. For most edge scenarios this is fine, but if you have a data engineer who wants to connect a SQL client directly to the device, you’ll need to build that bridge yourself.

None of that changes the core case: for running analytical SQL on sensor data at the edge, DuckDB offers capability that simply wasn’t accessible on constrained hardware a few years ago. The performance is real, the deployment is simple, and the SQL dialect is standard enough that queries translate directly from your cloud analytics environment.