Home:ALL Converter>How to create a logical replication subscription with SSL certificates in Google Cloud SQL (PostgreSQL)?

How to create a logical replication subscription with SSL certificates in Google Cloud SQL (PostgreSQL)?

Ask Time:2022-07-14T18:19:07         Author:FOR_SCIENCE

Json Formatter

I would like to use CDC with PostgreSQL's logical replication feature to gather data from different databases in the same instance for a materialized view. However, I cannot get this to work with SSL.

To test the logical replication, I wrote a zsh script that takes care of the publication and subscription.

HOST=?
PORT=?
DB_NAME=?

SSL_CERT=?/client-cert.pem
SSL_KEY=?/client-key.pem
SSL_ROOT_CERT=?/root.crt

SU_PASSWORD=
SU_USER="postgres"

R_USER=?
R_PASSWORD=?

PUBLICATION_NAME="stock_publication"
SUBSCRIPTION_NAME="stock_subscription"
# CONNECTION_INFO="host=$HOST port=$PORT dbname=$DB_NAME \
#     user=$R_USER password=$R_PASSWORD"
CONNECTION_INFO="host=$HOST port=$PORT dbname=$DB_NAME \
    sslmode=allow sslcert=$SSL_CERT sslkey=$SSL_KEY \
    user=$R_USER password=$R_PASSWORD"

exec_query() {
    # Execute a query with SSL connection

    PGPASSWORD=$R_PASSWORD \
    psql "sslmode=allow \
    sslcert=$SSL_CERT \
    sslkey=$SSL_KEY \
    hostaddr=$HOST \
    port=5432 user=$R_USER dbname=$DB_NAME" \
    --command=$1
}

# exec_query
exec_query "DROP PUBLICATION IF EXISTS $PUBLICATION_NAME;"
exec_query "CREATE PUBLICATION $PUBLICATION_NAME FOR TABLE t1, t2, ...;"
exec_query "DROP SUBSCRIPTION IF EXISTS $SUBSCRIPTION_NAME;"
exec_query "CREATE SUBSCRIPTION $SUBSCRIPTION_NAME CONNECTION '$CONNECTION_INFO' PUBLICATION $PUBLICATION_NAME;"

However, creating the subscription fails due to SSL restrictions. So I consequently added the required SSL mode and certificates, but that raised another error.

DROP PUBLICATION
CREATE PUBLICATION
NOTICE:  subscription "stock_subscription" does not exist, skipping
DROP SUBSCRIPTION
ERROR:  certificate is not allowed
DETAIL:  Non-superusers cannot use certificate in the connection setting.

The error surprises me, since Google Cloud SQL does not offer support for the super user role. Instead, the 'cloudsqlsuperuser' role exists. This role is not the same as a traditional PostgreSQL super user, so how am I supposed to mitigate this issue?

For as far as I know, there is not an alternative way to pass the SSL certificates to the 'CREATE SUBSCRIPTION' statement.

I guess that I can try Apache Debezium for CDC instead, but using Postgres' built-in CDC support seems much simpler to me. This is, for streaming data from one Postgres instance to another.

Author:FOR_SCIENCE,eproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/72978929/how-to-create-a-logical-replication-subscription-with-ssl-certificates-in-google
Vaidehi Jamankar :

As you know,Cloud SQL is a managed service,Please read the documentation for more information on the replication and configuration settings.\nPlease check and make sure that you have completed all the prerequisite steps as mentioned on the public docs [3] before running the command.\nKindly try the following suggestions and let me know the result.\nBy following the guide [1] but without a SSL flag and unchecking the ‘Allow only SSL connections’ on the instance, were you able to successfully run the command.And at the same time, on the command, you can set the flag sslmode=require in the create subscription call so the subscriber connection to the primary is always SSL-encrypted.\nYou may also check the documentation for SuperUser [2].\n[1] https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication\n[2] https://cloud.google.com/sql/docs/postgres/users#superuser_restrictions\n[3]https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication#setting-up-logical-replication-with-external-replica",
2022-07-19T06:04:32
yy