Home:ALL Converter>Connecting to Google Cloud SQL (PostgreSQL) locally and through App Engine Flex

Connecting to Google Cloud SQL (PostgreSQL) locally and through App Engine Flex

Ask Time:2018-11-30T03:41:22         Author:Adz

Json Formatter

Locally: My local environment is Mac OS. I'm attempting to connect to my Cloud SQL (PostgreSQL) db using Java 8. I've whitelisted my IP and I am able to connect to the DB using pgadmin. My service account is also project owner for the project that the database is hosted in and the app engine environment

This is the tutorial I was using to connect locally:

https://cloud.google.com/sql/docs/postgres/connect-external-app#java

and my actual code:

        private static void jdbcUrl = String.format(
            "jdbc:postgresql://google/%s?socketFactory=com.google.cloud.sql.postgres.SocketFactory"
                    + "&cloudSqlInstance=%s",
            Configuration.getDatabaseName(),
            Configuration.getInstanceConnectionName());

    public static String getDbConnection() throws Exception {
        LOGGER.info(jdbcUrl);

        Connection connection = DriverManager.getConnection(jdbcUrl, Configuration.getUsername(), Configuration.getPassword());

... }

This is the url call that gets generated in my code (I've altered the original for obvious purposes):

jdbc:postgresql://google/users?socketFactory=com.google.cloud.sql.postgres.SocketFactory&cloudSqlInstance=my-project:europe-west1:my-db

I am unable to connect with this call because of this error:

Nov 29, 2018 7:13:11 PM com.google.cloud.sql.postgres.SocketFactory createSocket
INFO: Connecting to Cloud SQL instance [my-project:europe-west1:my-db] via ssl socket.
Nov 29, 2018 7:13:11 PM com.google.cloud.sql.core.SslSocketFactory getInstance
INFO: First Cloud SQL connection, generating RSA key pair.
Nov 29, 2018 7:13:12 PM com.google.cloud.sql.core.SslSocketFactory fetchInstanceSslInfo
INFO: Obtaining ephemeral certificate for Cloud SQL instance [my-project:europe-west1:my-db].
Nov 29, 2018 7:13:13 PM com.google.cloud.sql.core.SslSocketFactory createAndConfigureSocket
INFO: Connecting to Cloud SQL instance [my-project:europe-west1:my-db] on IP [30.100.40.150].
org.postgresql.util.PSQLException: The connection attempt failed.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:292)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
    at org.postgresql.Driver.makeConnection(Driver.java:454)
    at org.postgresql.Driver.connect(Driver.java:256)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:247)
    at com.ri.ccm.utils.DbConnection.getDbConnection(DbConnection.java:23)
    at com.ri.ccm.routes.DummyRoute.serviceCategoryDetail(DummyRoute.java:22)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:833)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
    at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:206)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1621)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:541)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1593)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:118)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.Server.handle(Server.java:564)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
    at sun.security.ssl.InputRecord.read(InputRecord.java:503)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
    at com.google.cloud.sql.core.SslSocketFactory.createAndConfigureSocket(SslSocketFactory.java:210)
    at com.google.cloud.sql.core.SslSocketFactory.create(SslSocketFactory.java:160)
    at com.google.cloud.sql.postgres.SocketFactory.createSocket(SocketFactory.java:96)
    at org.postgresql.core.PGStream.<init>(PGStream.java:62)
    at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
    ... 64 more

However, I am able to connect and get data back with this call:

jdbc:postgresql://30.100.40.150/users

The only issue is that, I have to whitelist every ip (0.0.0.0/24) for this url to work in App Engine Flex (including mine)! Now I obviously don't want to do that for security reasons.

Through App Engine Flex:

jdbc:postgresql://30.100.40.150/users

I am able to connect with this call^ in App Engine Flex, but again I have to whitelist every IP for this to be able to work. The documentation states that I don't have to whitelist the IP in App Engine Flex for this to work (it should just use the service account). I'm guessing the original call, in the documentation does that. However that does not work. So I'm left at a standstill.

Please, for the love of god can someone help me. It shouldn't be this hard to connect to a database!

All I want to be able to do is to connect to my remote Cloud SQL PostgreSQL database through my local environment and through my App Engine Flex environment without having to whitelist every IP address (and doing it the 'proper' way). I'm using maven profiles to differentiate between my environments.

I've also tried setting up Cloud SQL Proxy for local use, it doesn't work and I get this error:

couldn't connect to "my-project:europe-west1:my-db": read tcp 10.75.120.100:49663->30.100.40.150:3307: read: connection reset by peer

I'm seriously running out of ideas, could this be a firewall port issue? Please help :(

Author:Adz,eproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/53546402/connecting-to-google-cloud-sql-postgresql-locally-and-through-app-engine-flex
David :

Outbound tcp port 3307 access is needed when using Cloud SQL Proxy or the socket factory, but not when using the native PostgreSQL JDBC driver. Make sure you don't have a firewall on your Mac or local network blocking outbound port 3307.",
2018-11-29T20:52:38
yy