A
A
Anton2019-02-13 11:21:01
PostgreSQL
Anton, 2019-02-13 11:21:01

Why does Postgresql write Archive '0000000100000000000000CC' does not exist after restoring WAL-G?

Version wal-g
wal-g -version
v0.1.15 I initialize the
database

/usr/pgsql-9.6/bin/postgresql96-setup initdb
Initializing database ... OK

start database
systemctl start postgresql-9.6
postgres=# create database test1;
CREATE DATABASE

postgres=# \c test1;
You are now connected to database "test1" as user "postgres".
test1=# CREATE TABLE large_test (num1 bigint, num2 double precision, num3 double precision);
CREATE TABLE

test1=# INSERT INTO large_test (num1, num2, num3)
  SELECT round(random()*10), random(), random()*142
  FROM generate_series(1, 20000000) s(i);
INSERT 0 20000000

I run wal-g backup-push
#!/bin/bash
. /etc/wal-g.d/server-s3.conf
wal-g backup-push /var/lib/pgsql/9.6/data/

postgres=# create database test2;
CREATE DATABASE
postgres=# \c test2;
You are now connected to database "test2" as user "postgres".
test2=# CREATE TABLE large_test (num1 bigint, num2 double precision, num3 double precision);
CREATE TABLE
test2=# INSERT INTO large_test (num1, num2, num3)
  SELECT round(random()*10), random(), random()*142
  FROM generate_series(1, 20000000) s(i);
INSERT 0 20000000

list of files in pg_xlog folder after command "create database test2;"
https://hastebin.com/ifonukifus.css
list of files in minio in wal_005 folder in pg-backups bucket
https://hastebin.com/ifaqapacoj.css
Stop Postgresql
Delete everything from /var/lib/pgsql/9.6/data folder
then I run backup_fetch_latest.sh
The contents of backup_fetch_latest.sh
#!/bin/bash
. /etc/wal-g.d/server-s3.conf
wal-g backup-fetch /var/lib/pgsql/9.6/data LATEST

Backup extraction complete.
I add recovery.conf to the /var/lib/pgsql/9.6/data folder
standby_mode = 'on'
restore_command = '/usr/local/bin/wal-fetch.sh "%f" "%p"'

Contents of wal-fetch.sh
cat /usr/local/bin/wal-fetch.sh
#!/bin/bash
. /etc/wal-g.d/server-s3.conf
wal-g wal-fetch $1 $2

systemctl start postgresql-9.6
tail -fn100 /var/lib/pgsql/9.6/data/pg_log/postgresql-Wed.log

Postgresql-Wed.log here
https://hastebin.com/okosecoxel.makefile
locate 0000000100000000000000CC
/var/lib/pgsql/9.6/data/pg_xlog/0000000100000000000000CC

ls -lh /var/lib/pgsql/9.6/data/pg_xlog/0000000100000000000000CC
-rw------- 1 postgres postgres 16M Feb 13 11:05 /var/lib/pgsql/9.6/data/pg_xlog/0000000100000000000000CC

cat  /etc/wal-g.d/server-s3.conf
#!/bin/bash
export WALE_S3_PREFIX="s3://pg-backups"
export AWS_ACCESS_KEY_ID="xxxx"
export AWS_ENDPOINT="http://10.233.60.57:9000"
export AWS_S3_FORCE_PATH_STYLE="true"
export AWS_SECRET_ACCESS_KEY="yyyy"

export PGDATA=/var/lib/pgsql/9.6/data/
export PGHOST=/var/run/postgresql/.s.PGSQL.5432
export WALG_UPLOAD_CONCURRENCY=2
export WALG_DOWNLOAD_CONCURRENCY=2
export WALG_UPLOAD_DISK_CONCURRENCY=2
export WALG_DELTA_MAX_STEPS=7

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question