I
I
infirmitive2022-04-20 09:29:05
PostgreSQL
infirmitive, 2022-04-20 09:29:05

How to aggregate stock candles using timescaledb?

I am graduating with a diploma in market data and the task was to display stock candles. There is an installed postgresql 12, rolled up timescaledb. I create tables:

Actually, in data_raw, I filled the table with test data for the last couple of years for several tickers and launched a test client that simulates a live exchange and fills it with random values ​​that are close to real behavior.

So, the data that I uploaded is not aggregated for some reason, but the data that I write to test clients (about 3600 records per minute in the so-called "peak activity") are aggregated into candles as it should. The last 5 minutes in real-time aggregates, and the previous 10 in continuous.

The problem is that the select of candles then takes a long time to complete. Based on explain, it goes through a bunch of chunks. Even after the last candle, it goes into chunks for about 6-7 seconds. limit 100 somewhere around 7-8 seconds. But these are 5-minute candles, the same weekly candles are very long.

There are more such views, I just didn’t add everything. I would like to understand the correct algorithm for using such aggregations, and I will apply them to the rest.

CREATE TABLE IF NOT EXISTS data_raw
(
    ticktime timestamp with time zone NOT NULL,
    ticker text COLLATE pg_catalog."default" NOT NULL,
    board text COLLATE pg_catalog."default",
    exchange text COLLATE pg_catalog."default",
    price double precision,
    amount bigint
);
SELECT create_hypertable('data_raw','ticktime', chunk_time_interval => INTERVAL '1 day', migrate_data := true, if_not_exists => TRUE);

-- Create views

CREATE MATERIALIZED VIEW IF NOT EXISTS candle_5m
            WITH (timescaledb.continuous, timescaledb.create_group_indexes = true) AS
SELECT ticker, board, exchange,
       time_bucket('5 minutes', ticktime) AS bucket,
       FIRST(price, ticktime) as open,
       LAST(price, ticktime) as close,
       MIN(price) as low,
       MAX(price) as high,
       sum(amount*price) as volume
FROM data_raw
GROUP BY ticker, board, exchange, bucket;

SELECT add_continuous_aggregate_policy(
               'candle_5m',
               start_offset => INTERVAL '15 minutes',
               end_offset => INTERVAL '5 minutes',
               schedule_interval => INTERVAL '5 minutes',
               if_not_exists => true
);

Tell me, please, what's wrong with the aggregations nachudil. I'm using Timescale for the first time and haven't read the whole doc yet, but protection is coming soon and there's no time to read it all over again.

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question