Regular postgres backup to email and telegram channel

Posted by : on

Category : Infrastructure

Why you may want to read this article

This article is a guide on how to send backup (essentially you can send anything) to telegram and email with the minimum effort and external tools installed (cURL).

All applications that store data in some way.

This implies that we don’t want to lose our data for sure, especially in production environments. To prevent data loss there are a lot of techniques, including: replication, backup, snapshots, etc.

In this case, we will consider a cheap ubuntu server (in my case I’m paying 5$), where we will set up our regular backup mechanism. But you also can do it on your own machine.

My specific use case: I had an ubuntu server running PostgreSQL. I needed regular backups to be sent to my email and to the telegram channel from the ubuntu server.

If you would like to see how to set up your Postgres database with command lines - check this guide.


Steps


The guide itself is pretty simple we will use the curl tool in ubuntu, because all basic servers have it and you do not need to install anything additional


Create Gmail account

Just follow the Gmail instructions, we will name email created {created-email}.

Then we need to turn on App Passwords to use them through curl.

Go here

1.Go to Google Account -> Security -> Signing in to Google -> 2-Step Verification, enable it.

alt_text

2.Go to Google Account -> Security -> Signing in to Google -> App Passwords

alt_text

Copy generated password (we will name it {gmail-password}):

“aaaaaaaaaaaa”

alt_text


Create telegram bot

Follow this instruction

You should get the bot token in format (numbers + numbers with letters), we will name it {bot-token} in the article:

1111111111:AAAAAAAA1aaaAAAAaAaaAaAAa1AAAAAAaA1

After this create new group and get the id of this group.

While creating the group add your bot by the name you gave him while creating.

Now paste https://api.telegram.org/bot{bot-token}/getUpdates in your browser. If it returns an empty response, try to remove and add your bot to the group again, as explained here

Then extract the chat id from JSON, I’ll name this {chat-id} in the article.


Write the script with file name test.sh


#!/bin/sh

TELEGRAM_URL="https://api.telegram.org/bot<bot-token>/sendDocument"

CHAT_ID="<chat-id>"

docker exec<postgres-container> pg_dump <database-name> > dump.sql

echo "created dev backup file in container"

curl -v -F "chat_id=${CHAT_ID}" -F document=@./dump.sql $TELEGRAM_URL

echo "sent dev backup to telegram chat"

curl --url 'smtps://smtp.gmail.com:465' --ssl-reqd --mail-from '<created-email>' --mail-rcpt '<email-where-to-send>’' -F text='Backup' -F attachment=@dump.sql --user '<created-email>:<gmail-password>'

echo "sent dev backup to email"


Test Script

To test your script just we will give the file permission to execute and then execute it.


chmod u+x test.sh

./test.sh

We will see the logs:

alt_text

Telegram result:

alt_text

Gmail result:

alt_text


Automate backup process

For automation we can use well known crontab for ubuntu.


crontab -e

You can find plenty of documentation for this, my file contains the following content:


0 1 \* \* \* /services/backup/prod-db-backup.sh

It means run this script once a day.

/services/backup/prod-db-backup.sh” is a path to my script that contains same content as our test.sh.


Restore from backup (Optional)

First copy dump.sql to the postgres docker container


docker cp dump.sql <container-name>:/dump.sql

Then run this command agains your database:


docker exec -it <container-name> bash

psql -h 127.0.0.1 -U root -d <database-name> < dump.sql

Follow up

Please subscribe to my social media to not miss updates.: Instagram, Telegram

I’m talking about life as a Software Engineer at Microsoft.


Besides that, my projects:

Symptoms Diary: https://symptom-diary.com

Pet4Pet: https://pet-4-pet.com


About Andrii Bui

Hi, my name is Andrii. I'm Software Engineer at Microsoft with 5 years of experience.