Answer the question
In order to leave comments, you need to log in
Why does the Python MySQL Connector DB driver work an order of magnitude faster than in other languages?
Tested various technologies to automate data insertion into the database, under the following conditions:
- MyISAM storage system
- MySQL 8/MariaDB 10.4 DBMS
- Insertion of 33,000 rows in 5 fields
- C#/PHP/Node/Python code
- both prepared and normal direct insertion of static values
- type query
INSERT INTO tablename (fieldA, fieldB, fieldC, fieldD, fieldE) VALUES (a, b, c, d, e)
CREATE TABLE `zip_code` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`zip` CHAR(5) NOT NULL,
`city` VARCHAR(50) NOT NULL,
`stid` CHAR(2) NOT NULL,
`state` VARCHAR(50) NOT NULL,
PRIMARY KEY (`id`)
)
COLLATE='utf8_general_ci'
ENGINE=MyISAM
;
Answer the question
In order to leave comments, you need to log in
Offhand: cursor.executemany() connects queries into one big INSERT. Other drivers themselves do not seem to be able to do this.
I think if you manually assemble a multi-row INSERT, then something like this will come out:
INSERT INTO tablename (fieldA, fieldB, fieldC, fieldD, fieldE)
VALUES (a, b, c, d, e),
(a1, b1, c1, d1, e1),
(a2, b2, c2, d2, e2),
...
1k lines are usually inserted by separate inserts when performance is of importance somewhere in the distant and bright future, or from lack of experience.
If the issue of performance is still important - either we perform a multiple insert in one request, or we wrap it in a transaction.
Author, run your own test, but with a transaction around your thousand inserts, the results will be different.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question