Answer the question
In order to leave comments, you need to log in
How to work with a large database using C# + MsSQLL?
Good day!
There is a database in which it is necessary to enter a large amount of data.
Approximately 5 million records per day. Before adding a database to the database, we first check that they are not there, otherwise we edit the existing record.
At first I used a visual generator from the studio, left the plates, found code examples on the Internet.
It was like this:
usersTableAdapter usersTA = new usersTableAdapter();
var usersTable = new users.usersDataTable();
usersTA.Fill(usersTable);
IF (NOT EXISTS(SELECT * FROM [dbo].[users] WHERE [id] = @id))
BEGIN
INSERT INTO [dbo].[users]
(...)
VALUES
(...);
SET @result = 1;
END
ELSE
BEGIN
UPDATE [dbo].[users]
SET ...
WHERE [id] = @id
SET @result = 0;
END
Answer the question
In order to leave comments, you need to log in
How do you transfer records, which are 5000, to storage? 1 piece? Then the time will really increase.
Option 2:
1. Convert all data (all 5000 records) to XML - pass XML to storage, which will parse XML and add/edit data.
2. Load all records (INSERT) into a temporary table in the database. Then you run the storage, which will do the logic of adding / updating data to the main table, comparing two tables - the main and the temporary.
Why pull all the data into memory?
First get a list of unique record keys (or a set of fields by which records are unique) and then pull the record one by one and process it.
Is this the data synchronization you are trying to perform?
That is, you just need to transfer the data by ID code?
If so, then you just need to remember on which ID the last download ended and continue from there.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question