S
S
sibvic2019-01-09 07:19:03
Database design
sibvic, 2019-01-09 07:19:03

What is the best database structure for getting new records from the database?

The question arose of how best to organize the structure of the database / table for the fastest receipt of new records. There is a table in which new records are constantly added. In addition to the main data, the table has 2 fields:
Processed and RecordType (nullable). A non-unique index has been created for these fields.
There are several performers who are raking out new entries from the table.

SELECT ...
FROM [Table] AS [a]
WHERE ([a].[Processed] = 0) AND ([a].[RecordType] IS NULL OR ([a].[RecordType] = @__Type_0))

SELECT ...
FROM [Table] AS [a]
WHERE (([a].[Processed] = 0) AND [a].[RecordType] IS NOT NULL) AND ([a].[RecordType] = @__Type_0)

Usually raw records in the table - units. Everything else is history, and is requested extremely rarely.
And the catch is that the first request is executed much faster than the second (tens to hundreds of times). And the more history in the table, the slower it is executed. Fixing to WHERE (([a].[Processed] = 0) AND [a].[RecordType] = @__Type_0) doesn't fix the situation.
And in this regard, I have two questions:
1) Why is the second request so slow? It looks like the query is not using an index, but it's not clear why.
2) Wouldn't it be better to make a separate table for raw records, and then transfer them to the history table? Or is there some other trick?
Tried on Azure DB and on local MS SQL Local DB if that matters.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
sibvic, 2019-01-09
@sibvic

It looks like the Rubber Duck Programming methodology has worked again.
In my case, the index filter helped

CREATE NONCLUSTERED INDEX [IX_...]
    ON [dbo].[Table]([Processed] ASC, [RecordType] ASC);

changed to
CREATE NONCLUSTERED INDEX [IX_...]
    ON [dbo].[Table]([Processed] ASC, [RecordType] ASC) WHERE ([Processed]=(0));

Well, either in EF:
protected override void OnModelCreating(ModelBuilder builder)
        {
            base.OnModelCreating(builder);
            builder.Entity<Table>()
                .HasIndex(x => new { x.Processed, x.RecordType})
                .HasFilter("Processed= 0")//NEW!!!
                .IsUnique(false);
        }

Now both requests are executed equally fast

K
Konstantin Tsvetkov, 2019-01-09
@tsklab

Because the optimizer includes a query by index. Look at the execution plan.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question