Securing Load balanced Authenticated Cloud Run Services using IAP

When designing your cloud run services, you should always consider following points

  1. How to run same service in multiple region for multi-regional support

Both of these features are well documented on Google cloud documentation and can be just implemented by following Google cloud documentation.

But we should remember following points

  1. When using Authenticated Cloud run either as (PubSub push url where we configure audience, although this is optional) , JWT audience for Cloud Run (and GCF) bound ID tokens is region specific. That means if your cloud run (say us-west and us-central) is load balanced, the authentication may not work as expected because the JWT token is region bound and this token is generated by PubSub and will eventually fail if the request lands on the other region cloud run service because the token is generated by PubSub while which cloud run to use is being decided by the Load Balancer.
  • If the used behind the load balancer and audience is matching the cloud run service url , then authentication is performed normally other 403 error is raised

To solve this problem, I propose below architecture

Cloud Run Secured by Cloud IAP

Below is the thought process on this architecture

  1. Cloud run is deployed in 2 different region as Authenticated and Ingress with Internal and Load Balancer only

This is an unsolved and undocumented problem on Google Cloud Platform.

When you configure a IAP on top of Load Balancer/App Engine/Managed Instance Group(MIG) on GCP, GCP created oAuth credentials behind the scene for you. These credentials are with name IAP-{LOAD_BALANCER_NAME} or IAP-{APP_ENGINE_SERVICE_NAME}

The above line is not documented anywhere on GCP or at-least I could not find it on GCP.

Now to give Cloud IAP permission to invoke cloud run service, perform below steps

  1. Go to credentials page

3. Now give Cloud Run invoker permission to this Client Id.

Now to allow users to invoke or authenticate though the Cloud IAP , you must give them IAP WebApp User permission. This way you can expose your cloud run service without giving end users Cloud Run invoker role. You just add them to Cloud IAP.

Solving the same for PubSub Push delivery for multi regional service.

Workflow is as below.

  1. Create a push subscription with Push url pointing to LB domain

Hope you will like this unsolved and undocumented problem for Cloud Run and Load Balancer.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Deepak Verma

Data Analytics | Kubernetes | Cloud Architect | Data Architect | Python