K
K
KrivdaKravdaBum2018-02-22 09:36:39
C++ / C#
KrivdaKravdaBum, 2018-02-22 09:36:39

Local client for parsing XML and updating SQLite tables (C#). Is optimization possible?

A client for updating a table in a SQLite database has been written. Downloads XML from the server, parses and updates the data in the table. The XML contains > 40,000 lines (and the number of lines increases every month), like this:

<card card="05871" article="0016" count="1" timestamp="1519134555"/>

The question arose about optimizing this action, because it takes at least 2 minutes to update the table. Originally written in C#. I want to rewrite everything in C++. Is it worth doing this, will there be any acceleration?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
G
GavriKos, 2018-02-22
@GavriKos

First you need to profile what exactly takes a lot of time.
If it is for "updating the table" (tobish for executing SQL queries) - then rewriting will not give anything - you need to do something with the database or with queries.

S
Stanislav Makarov, 2018-02-22
@Nipheris

The question arose about optimizing this action, because it takes at least 2 minutes to update the table.

Well, it's a well-known joke that by default SQlite makes a transaction for each insert. Run a transaction and commit it once, and all rows will be inserted in a second at most.
You started inventing some mega ways to speed up without exploring the root of the problem. And the root is that SQLite creates a rollback file per transaction, and when there are 40,000 transactions, it will be the same number of file re-creation operations. It's just a fucking heavy operation for the file system, and neither the processor nor multithreading has anything to do with it. You could guess for yourself, because. 40,000 small records is not a volume at all.
The solution is googled in 20 seconds and is written in the SQLite FAQ: www.sqlite.org/faq.html#q19

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question