Processing one row at a time means a branch and a function call per value. Vectorized execution processes thousands at once, and the CPU notices.

A naive query engine evaluates each operator row by row: check a filter, branch, maybe emit. That pattern defeats CPU pipelining and wastes cache bandwidth on function-call overhead. HelixQL's execution engine instead operates on batches of a few thousand values at a time, column by column.

What a batch looks like

Decoded chunk data is materialized into fixed-size batches — 2,048 values per batch, chosen to fit comfortably in L2 cache alongside the operator's working state. A `where value > 500` filter, instead of branching per row, produces a selection bitmap over the whole batch in one tight loop with no branch misprediction on the comparison itself.

from metrics
| where name == 'http.server.duration'
| where value > 500ms and status_code >= 500
| summarize count() by bin(timestamp, 1m)
# both where clauses evaluate as SIMD-friendly bitmap operations over 2048-value batches

Measured CPU impact

On a benchmark scanning 100 million points with two numeric filters and a group-by aggregation, vectorized batch execution completed in 1.4s using a single core. The same query plan executed row-at-a-time (a mode we kept for correctness testing and debugging) took 6.1s — a 4.4x difference attributable almost entirely to eliminated branch mispredictions and better instruction cache locality.

Where vectorization breaks down

Not every operator vectorizes cleanly. Regex-based log filtering, for instance, still evaluates largely per-value since general regex engines don't have a natural batched form — we get some benefit from batching the memory access pattern, but the regex evaluation itself remains the bottleneck, typically 8-10x slower per byte than a numeric comparison at the same batch size.

Why 2,048 and not larger

We tested batch sizes from 256 to 16,384. Beyond 2,048, batches started spilling out of L2 cache on our reference hardware (256KB L2 per core), and the marginal throughput gain from fewer batch-boundary overheads was outweighed by cache misses — we measured a 12% throughput regression at 8,192 versus 2,048 for typical filter-heavy queries.

  • Query execution operates on 2,048-value column batches, not individual rows.
  • Vectorized filter/aggregation is roughly 4.4x faster than row-at-a-time execution on numeric workloads.
  • Regex-based log filters don't vectorize cleanly and remain the slower operator class.