Real-Time Feature Engine
Incremental feature computation for streaming data
Nightly batches leave fraud and recommendation models using hours-old data. A fraudster may transact dozens of times before the pattern reaches your model.
09:41:02
09:41:05
09:41:09
09:41:11
spend_1h: $129.00
txn_count: 1
spend_1h: $18.50
txn_count: 1
spend_1h: $360.22
txn_count: 2
The Problem
Overnight batches leave features stale
Create an incremental pipeline in under 10 minutes. No JVM, custom operators, or stream-processing PhD needed.
Batch limitations
At midnight, your pipeline scans the prior 24 hours, then writes aggregate features by 2am. Depending on arrival time, predictions use features 1 to 25 hours old.
That works for low-velocity domains. In fraud, recommendations, and dynamic pricing, it drives degradation that retraining cannot explain.
# Runs every 60 minutes, scans full table
SELECT card_id,
SUM(amount) AS spend_1h,
COUNT(*) AS txn_count
FROM transactions
WHERE event_ts > NOW() - INTERVAL 1 HOUR
GROUP BY card_id
-- Features stale up to 60 min after each run
-- Full table scan on every triggerHow It Works
Four steps: stream to feature
No logos or lock-in. Feldera reads via standard protocols and writes via standard interfaces.
Link sources
Point Feldera at a Kafka topic, Postgres replication slot, MySQL binlog, or S3 bucket. One config block, no custom connectors.
Write SQL
Keep using SELECT, GROUP BY, JOIN, and window functions. Feldera parses them into an incremental operator plan.
Deploy with deltas
Deploy with one CLI command. Feldera initializes state from existing data, then automatically enters incremental delta mode.
Serve models
Call the feature API from model serving. Features stay current within pipeline latency, not a batch window.
Core Features
An engine for incremental computation
GROUP BY, JOIN, and window functions update incrementally. Joins retain bidirectional delta state; aggregates scale with O(change size), not O(table size).
- Compute only the delta per event
- No historical record reprocessing
- Incremental joins, aggregates, and windows
Read: Apache Kafka, PostgreSQL logical replication (CDC), MySQL binlog, Amazon S3 (Parquet, JSON, CSV). Write: Kafka output topics, REST feature API, Postgres writes, Parquet to S3.
- Kafka, Postgres CDC, MySQL binlog, S3
- Inject ad-hoc events via HTTP webhooks
- YAML config, no Java or Scala needed
Look up by feature key with sub-10ms median response at steady state. Export batches for model training, with typed schemas defined when the pipeline is registered.
- GET /features/{pipeline}/{key} for key lookup
- Export batches for offline training
- Typed schemas, no schema-on-read parsing
Early Access
Practitioner perspectives
We used to recompute our entire transaction velocity table every 15 minutes. Feldera's incremental engine now refreshes those features with each transaction event. Our fraud model responds to what just happened, not 15 minutes earlier. Moving to SQL was straightforward because every query stayed unchanged.
Internal benchmark: sub-100ms feature update latency across five pipeline configurations comparing full-recompute baseline against Feldera incremental engine. Your results will vary by data shape and hardware.
Integrations
Use the stack you have
Open source self-hosted or fully managed cloud. No seat minimums.
Pricing
Start free, scale as needed
Open source self-hosted or fully managed cloud. No seat minimums to get started.
Full SQL incremental engine, all connectors, unlimited local pipelines. Self-hosted, Apache 2.0.
Read the docsManaged hosting, web UI editor, usage dashboard, email support. Up to 15 active pipelines.
Start free trialGet started
Fresh features begin here
Build your first incremental pipeline in under 10 minutes. The open-source tier needs no credit card.