S
S
SmartReactorApp2019-10-19 14:41:22
MySQL
SmartReactorApp, 2019-10-19 14:41:22

How to dump a large database in Mysql on Windows?

There is a large database (in a 7GB archive) at the address of G:\Base\mybase.zip
the latest version of Mysql at G:\MySqlWork\mysql.exe
7zip location C:\Program Files (x86)\7-Zip\7zFM.exe
Name of the Mysql database newBase
Login root
Password 4830
There mybase.zipare about 7 more archives in the archive itself in the format .7z, and in each archive there is a 7z .sqlfile.
How can I upload these dumps to the database using the command line "on the fly" newBase
Along with the database comes a small file readme.txtthat says the following:

You can upload SQL dumps to MySQL with on-the-fly unpacking from 7-Zip - I advise you to do just that, because it is faster and requires less free disk space.
To load data into MySQL at once from all downloaded archives in the current archive folder, use the command:
7z x -so "*.7z" | mysql -uLogin -pPasswordBaseName
Under Windows, all commands must contain full paths to the files 7z.exe and mysql.exe, enclosing them in "double quotes" if the paths contain spaces.
Tables are loaded into MySQL in UTF-8 encoding.

Tried many different commands like
"C:\Program Files (x86)\7-Zip\7zFM.exe" x -so G:\FIAS\fias_07.10.19.zip | G:\MySqlWork\mysql.exe -uroot -p 4830

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
SmartReactorApp, 2019-10-19
@SmartReactorApp

If you have a similar case and a really large database, it is better to change the database storage directory in the my.ini file, since the "command line" will not write errors and you, like me, can wait a day waiting for the result (the space on the system disk is running out and mysql starts to fall).
Everything turned out to be simple) I specified the command a little incorrectly, it was necessary to unpack the main archive so that the internal 7 archives were in the G:\Base\.
In my case, the correct command would be :
"C:\Program Files (x86)\7-Zip\7z.exe" x -so "*.7z" | G:\MySqlWork\mysql.exe -uroot -p newBase
This command must be run from an open folder with archives on the command line, or "*.7z"G:\Base-other\*.7z
The first (main archive) was .zipand inside there were 7 archives .7z- that's why I had to initially unzip the main archive, and then run the command.
If there is an error during the dump ERROR 2006 (HY000) — MySQL server has gone away, open the my.ini file, I had it at the address C:\ProgramData\MySQL\MySQL Server 8.0\my.ini(by default, the folder ProgramDatais hidden by the system) look for the line max_allowed_packet=and set the value above ↑ (I set 400000M). Then restart mysql - I just rebooted the PC, because I did not understand how to restart the mysql server.
Also, after entering the command on the command line, nothing happens, but looking at the "task manager" there is a load on the CPU, RAM and Disk. I connected to the database using "HeidiSQL" and made sure that the new tables were loaded successfully.
Command line state after typing a command
Task Manager after entering the
PS command If you have a similar case and a really large database, it is better to change the database storage directory in the my.ini file, since the "command line" will not write errors and you, like me, can wait a day waiting for the result (place on the system drive ends and mysql starts crashing/shutdown).
Briefly how to do it:

S
Sergey c0re, 2019-10-19
@erge

Not 7zFM.exe but 7z.exe
The first is a window file manager, the second works from the command line,
see options and startup keys:
7-Zip 19.00 (x86) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21
Usage: 7z [...] [...] [@listfile]
a : Add files to archive
b : Benchmark
d : Delete files from archive
e : Extract files from archive (without using directory names)
h : Calculate hash values ​​for files
i : Show information about supported formats
l : List contents of archive
rn : Rename files in archive
t : Test integrity of archive
u : Update files to archive
x : eXtract files with full paths
-- : Stop switches and @listfile parsing
-ai[r[-|0]]{@listfile|!wildcard} : Include archives
-ax[r[-|0] ]{@listfile|!wildcard} : eXclude archives
-ao{a|s|t|u} : set Overwrite mode
-an : disable archive_name field
-bb[0-3] : set output log level
-bd : disable progress indicator
-bs{o|e|p}{0 |1|2} : set output stream for output/error/progress line
-bt : show execution time statistics
-i[r[-|0]]{@listfile|!wildcard} : Include filenames
-m{Parameters} : set compression Method
-mmt[N] : set number of CPU threads
-mx[N] : set compression level: -mx1 (fastest) ... -mx9 (ultra)
-o{Directory} : set Output directory
-p{Password} : set Password
-r[-|0] : Recurse subdirectories
-sa{a|e|s} : set Archive name mode
-scc{UTF-8|WIN|DOS} : set charset for for console input/output
-scs{UTF-8|UTF-16LE|UTF-16BE|WIN|DOS|{id}} : set charset for list files
-scrc[CRC32|CRC64|SHA1|SHA256|*] : set hash function for x, e , h commands
-sdel : delete files after compression
-seml[.] : send archive by email
-sfx[{name}] : Create SFX archive
-si[{name}] : read data from stdin
-slp : set Large Pages mode
-slt : show technical information for l (List) command
-snh : store hard links as links
-snl : store symbolic links as links
-sni : store NT security information
-sns[-] : store NTFS alternate streams
-so : write data to stdout
-spd : disable wildcard matching for file names
-spe : eliminate duplication of root folder for extract command
-spf : use fully qualified file paths
-ssc[-] : set sensitive case mode
-sse : stop archive creating, if it can't open some input file
-ssw : compress shared files
-stl : set archive timestamp from the most recently modified file
-stm{HexMask} : set CPU thread affinity mask (hexadecimal number)
-stx{Type} : exclude archive type
-t{Type} : Set type of archive
-u[ -][p#][q#][r#][x#][y#][z#][!newArchiveName] : Update options
-v{Size}[b|k|m|g] : Create volumes
- w[{path}] : assign Work directory. Empty path means a temporary directory
-x[r[-|0]]{@listfile|!wildcard} : eXclude filenames
-y : assume Yes on all queries
also see - mysql, Command line utility
but I suspect that it still won't work, you need to specify which particular file to unpack and throw into stdout, because they may unpack in the wrong order.
something like this probably:

"C:\Program Files (x86)\7-Zip\7z.exe" x -so G:\FIAS\fias_07.10.19.zip sql_file1.sql | G:\MySqlWork\mysql.exe -uroot -p 4830 newBase
"C:\Program Files (x86)\7-Zip\7z.exe" x -so G:\FIAS\fias_07.10.19.zip sql_file2.sql | G:\MySqlWork\mysql.exe -uroot -p 4830 newBase

and so iterate over everything in order.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question