How to quickly insert over 100k records in Woocommerce?

Question

WordPress

Hello. I do import of goods from different sources. Now the goods are about 130k. Import on excellent hardware using woocommerce api takes more than 3 days. This is very long, as the data is updated every 6 hours.
Is there a library or ready-made examples for creating goods through mysql, bypassing the api?!
https://pastebin.com/CLg4i1bh sample code that inserts records now.

Answer 1

Ready libraries most likely are not present. Because everyone uses the REST interface or official WooCommerce calls.
As an option, add products via wp_set_object_terms update_post_meta. But this will also involve part of the wordpress API.
https://lukasznowici.info/insert-new-woocommerce-...
If you write queries directly in mysql, there may be problems with correct data insertion. There are a lot of nuances to consider for such a low-level work.
Perhaps it is worth executing all queries in parallel. For example, there are 100 categories, we take the first 10 categories and run 10 import instances. when they import, take another 10.
To make everything much faster, it would be desirable to run scripts through the terminal, without using a web server as an intermediary.

Answer 2

It is initially a wrong idea to insert (or update as the task says) more than 100k products every 6 hours, and it is not reasonable to use WC for such a number of products on any hardware.
Inserting goods directly through mysql is not so easy, there are many relationships between tables. As far as I understand, you are pulling goods from somewhere outside, then there is no way for these goods to save certain parameters and you will need to make constant selections in addition to inserting. It is possible to implement all this and it will work faster, but it is not clear how to maintain and where to look for errors if something goes wrong. Especially on 100k records. By the way, in my experience, image processing gives the most load, if they are added correctly through the wp core.
If you insert all these products once and then update only certain parameters, such as the price, then the task is already more realistic. You need to make a selection of products to get their id and then update the necessary data.