Why General-Purpose Databases Fail for IoT Telemetry
Relational databases like PostgreSQL and MySQL were designed for transactional workloads: reads and writes of individual rows, complex joins across normalised tables, ACID guarantees for business-critical mutations. IoT telemetry is the opposite pattern in almost every dimension.
An IoT deployment generating 10,000 sensor readings per second produces 864 million rows per day. At that rate, a standard PostgreSQL table without partitioning will see write performance degrade within weeks as B-tree indexes grow beyond memory. B-tree indexes — optimised for point lookups and range scans on arbitrary columns — are wasteful when the access pattern is almost always “give me all values for sensor X between time T1 and T2.”
Time-series databases solve this by treating time as a first-class dimension: data is physically ordered by timestamp, partitioned into time chunks, and compressed using delta-of-delta encoding. A well-tuned TSDB achieves 10–50× compression ratios and sustains write rates that would cripple a general-purpose database.
The Three Contenders
InfluxDB 3.0
InfluxDB, developed by InfluxData, is purpose-built for time-series. Version 3.0 is a full rewrite using Apache Arrow and Apache Parquet under the hood, replacing the bespoke TSM storage engine of earlier versions. The new storage layer dramatically improves analytical query performance and long-term compression.
Query language: InfluxDB 3.0 uses SQL as its primary interface (via Apache Arrow Flight SQL), ending the division between Flux and InfluxQL that plagued earlier versions. If your team already knows SQL, this is a significant usability improvement.
Write throughput: InfluxDB 3.0 Cloud Dedicated benchmarks show sustained ingestion above 1 million data points per second per node in favourable conditions. The open-source edge version handles hundreds of thousands of writes per second on modest hardware.
Retention policies: Configurable per-database retention with automatic data expiration. No limits on the number of distinct series in InfluxDB 3.0 — cardinality limits were a major pain point in v1.x, and removing them substantially changes the data modelling options.
Clustering: InfluxDB 3.0 Cloud Dedicated is natively clustered. The open-source edition is single-node; horizontal scaling requires InfluxDB Cloud.
Cost: Open-source is free. InfluxDB Cloud Dedicated starts at roughly $1,500/month for production tiers. Self-hosted on cloud VMs is cost-effective at scale.
TimescaleDB
TimescaleDB is a PostgreSQL extension that adds time-series capabilities — automatic partitioning (hypertables), columnar compression, continuous aggregates, and data tiering — to a fully compatible PostgreSQL database. This is its defining advantage: you get TSDB performance while retaining every PostgreSQL feature, including full SQL, JOINs, foreign keys, PostGIS for geospatial queries, and the entire pg ecosystem.
For UK engineering teams already running PostgreSQL infrastructure, TimescaleDB is often the natural choice — you’re not adding an unfamiliar database to your operational stack.
Query language: Standard SQL with TimescaleDB-specific functions (time_bucket(), first(), last(), locf()). Any PostgreSQL client library works without modification.
Write throughput: With hypertables and native compression enabled, TimescaleDB handles 500K–1M rows per second on a single high-memory node.
Retention policies: Tiered storage with automatic data tiering to S3-compatible object storage. Retention jobs via add_retention_policy().
Cost: Apache 2.0 open-source. Timescale Cloud starts around $50/month for small deployments. Very competitive for teams already running PostgreSQL.
Prometheus
Prometheus is a pull-based monitoring and alerting toolkit with a built-in time-series database. It’s the default observability stack for Kubernetes and microservices environments — if you’re running K3s on edge nodes, you’re probably already familiar with it.
Prometheus scrapes metrics from instrumented applications via HTTP endpoints at configurable intervals. It’s designed for monitoring workloads (hundreds of thousands of active series, scrape intervals of 15–60 seconds) rather than high-frequency IoT sensor ingestion. That distinction matters.
Query language: PromQL — powerful and expressive for operational metrics, but not SQL-compatible. There’s a learning curve.
Retention policies: Default local storage retention is 15 days. Long-term retention requires remote write to a backend (Thanos, Mimir, Grafana Cloud, VictoriaMetrics).
Cost: Fully open-source. Grafana Cloud (which includes a managed Prometheus-compatible backend) has a generous free tier.
Comparison Table
| Dimension | InfluxDB 3.0 | TimescaleDB | Prometheus |
|---|---|---|---|
| Primary use case | IoT telemetry, metrics | IoT telemetry + relational | Infrastructure monitoring |
| Query language | SQL (Arrow Flight SQL) | PostgreSQL SQL | PromQL |
| Max write throughput | >1M points/sec (cloud) | 500K–1M rows/sec | ~500K samples/sec (single node) |
| Cardinality limits | None (v3.0) | None | High cardinality degrades perf |
| Retention policies | Per-DB, automatic | Policy-based + S3 tiering | 15d local; remote write for more |
| Clustering | Cloud-native (managed) | Multi-node (complex) | Via Thanos/Mimir/Cortex |
| PostgreSQL compat | No | Yes (full) | No |
| Managed cloud | InfluxDB Cloud | Timescale Cloud | Grafana Cloud, Mimir |
Use Case Fit
Choose InfluxDB 3.0 when you want a purpose-built TSDB with SQL and no PostgreSQL operational overhead, you have very high write rates (above 500K points/sec), or you’re migrating from InfluxDB 1.x or 2.x.
Choose TimescaleDB when you need to JOIN sensor data with relational tables (devices, customers, locations), your team already operates PostgreSQL infrastructure, you require geospatial queries (PostGIS) alongside time-series data, or you want a single database for both telemetry and business data.
Choose Prometheus when your primary use case is monitoring edge infrastructure and applications rather than device telemetry, you’re running Kubernetes and want native CNCF observability integration, or you need tight alerting integration (Alertmanager).
The Bottom Line
There’s no universal winner here. TimescaleDB is the strongest choice for teams that value SQL familiarity, relational joins, and PostgreSQL ecosystem compatibility — particularly common in UK enterprise environments. InfluxDB 3.0 is the best pure-play TSDB for high-throughput IoT ingestion with minimal operational complexity. Prometheus is the right answer when the primary goal is infrastructure observability, not sensor data analytics. Many mature IoT platforms run all three — Prometheus for platform health, InfluxDB or TimescaleDB for device telemetry — and expose a unified query layer via Grafana.