A
A
Anton2019-02-12 15:16:03
PostgreSQL
Anton, 2019-02-12 15:16:03

Why does wal-g backup-push hang on Finished writing part 3?

Centos 7
Postgresql 9.6
WAL-G v0.2.4 and v0.1.15

cat /usr/local/bin/backup_wal.sh 
#!/bin/bash

. /etc/wal-g.d/server-s3.conf

wal-g backup-push /var/lib/pgsql/9.6/
wal-g backup-list

cat  /etc/wal-g.d/server-s3.conf
#!/bin/bash

export WALE_S3_PREFIX="s3://pg-backups"
export AWS_ACCESS_KEY_ID="xxx"
export AWS_ENDPOINT="http://http://10.233.60.57:9000"
export AWS_S3_FORCE_PATH_STYLE="true"
export AWS_SECRET_ACCESS_KEY="yyy"

export PGDATA=/var/lib/pgsql/9.6/data/
export PGHOST=/var/run/postgresql/.s.PGSQL.5432
export WALG_UPLOAD_CONCURRENCY=2
export WALG_DOWNLOAD_CONCURRENCY=2
export WALG_UPLOAD_DISK_CONCURRENCY=2
export WALG_DELTA_MAX_STEPS=7

cat /var/lib/pgsql/9.6/data/postgresql.conf | grep wal_level
wal_level = replica         # minimal, replica, or logical

Turned off the archive block
cat /var/lib/pgsql/9.6/data/postgresql.conf | grep archive
archive_mode = on
archive_command = '/usr/local/bin/backup_wal.sh'
archive_timeout = 60

/usr/local/bin/backup_wal.sh 
Path:  
INFO: 2019/02/12 15:12:56.554875 Couldn't find previous backup. Doing full backup.
INFO: 2019/02/12 15:12:56.874112 Walking ...
INFO: 2019/02/12 15:12:56.874507 Starting part 1 ...
INFO: 2019/02/12 15:12:56.876594 Starting part 2 ...
INFO: 2019/02/12 15:12:56.996311 Finished writing part 1.
INFO: 2019/02/12 15:13:03.726401 Finished writing part 2.
INFO: 2019/02/12 15:13:04.581864 Starting part 3 ...
INFO: 2019/02/12 15:13:04.582826 /data/global/pg_control
INFO: 2019/02/12 15:13:04.583282 Finished writing part 3.

minio:
5c62b8b72ffeb495341335.png

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
Melkij, 2019-02-12
Patsev @chemtech

archive_command = '/usr/local/bin/backup_wal.sh'
cat /usr/local/bin/backup_wal.sh
#!/bin/bash
. /etc/wal-gd/server-s3.conf
wal-g backup-push /var/lib/pgsql/9.6/
wal-g backup-list

Ahhh?... Are you serious now?
It won't work. archive_command is called per WAL segment - 16mb changes typically. That is, easily dozens of times per second on loaded bases. Trying to backup the cluster from here is a terrible idea.
wal-push in archive_command should be. backup-push separately, usually from krone.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question