Firewall your GCP Cloud Run Service using Terraform

7 min readDec 10, 2022

Overview

Have you ever been in a situation where you wanted to use a serverless technology like Cloud Run or App Engine, but also wanted to restrict its access to certain IPs ? Well you’re in luck, since we’ll be solving this very problem here. Also to take it a step further, we’ll do this entirely in Terraform so it’s easier to maintain and update in future

As a side note, this article is part of the Akatsuki Games Advent Calendar 2022 series. In case you’re interested, the previous article in this series was on ‘Making a Game using HAXE and Releasing it on Steam’ by Pedro, and the next one is ‘Setting Multiple PropertyAttributes at the same time in Unity’ by Yuyu. The articles are mostly in Japanese but you can always use auto-translate ;)

As always I won’t bore you with any more filler, so let’s jump right in!

What we’ll do

We’ll deploy a basic unsecured Cloud Run service
We’ll then secure the service behind a firewall so that it can only be accessed by a list of authorized IPs
We’ll create the entire infrastructure in Terraform

Prerequisites

Before we start here are some prerequisites for those following along

Knowledge

Basic knowledge of Terraform
Basic knowledge of GCP (or any another cloud provider)
And of course a willingness to learn :)

Tools

Terraform (set up and working). I’d recommend using tfenv to manage your terraform versions
A GCP account and gcloud CLI (set up and working)
Your favourite Text Editor or IDE

Note you’ll be charged for any resources you create! Some of the resources have a free-tier option but others do not so please keep that in mind!

If you already know how this works and just want to see the code, the complete project can be found on GitHub

To keep things simple we’ll be using a local state in this article but the GitHub project contains a backend.tf file that can be uncommented and modified to save your state to Cloud Storage

Also since the main focus is to show you how to firewall your Cloud Run service, this article will assume you already have your Cloud Run container image pushed somewhere and readily accessible. To keep things in scope I’ll be using an example image in this article

Ok let’s start! :)

Terraform

Setup

Create a directory to store all your terraform files (will be referred to as the project directory from now on) and create a providers.tf file with the terraform and provider version constraints

terraform {
  required_version = "~> 1.0"

  required_providers {
    google     = "~> 4.0"
  }
}

provider "google" {
  project = var.project
  region  = var.region
}

Lets also add a variables.tf file to define the GCP Project ID and Region

variable "project" {
  description = "GCP project ID"
}

variable "region" {
  description = "The main region where the resources are created"
}

and also a terraform.tfvars.json file to specify the values of the variables we just defined

{
  "project": "<REPLACE WITH YOUR GCP PROJECT ID>",
  "region": "<REPLACE WITH YOUR GCP PROJECT REGION>,
}

Optional: I’m using tfenv so I’m also using a .terraform-version file specifying the exact version to use (or automatically download if necessary)

Now let’s initialise Terraform and download the necessary plugins.

$ cd ~/examples/firewalled-cloud-run/terraform
$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/google versions matching "~> 4.0"...
- Installing hashicorp/google v4.45.0...
- Installed hashicorp/google v4.45.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

Cloud Run Service

Next lets create the Cloud Run service and test that everything works before we go about restricting access to it.

Create a services.tf file in the project directory with the following code

I’m adding a couple of noauth resources here since this is a public service and I’m not using Cloud IAM. If you’re using it however you can go ahead and delete those blocks and add the corresponding resources instead

Also note the annotation which specifies which traffic to allow (currently all) since we’ll be updating it later.

“run.googleapis.com/ingress” = “all”

Now go ahead and run terraform plan.

$ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # google_cloud_run_service.service will be created
  + resource "google_cloud_run_service" "service" {
      + autogenerate_revision_name = true
      + id                         = (known after apply)
      + location                   = "us-west1"

...
...
...

# google_cloud_run_service_iam_policy.noauth will be created
  + resource "google_cloud_run_service_iam_policy" "noauth" {
      + etag        = (known after apply)
      + id          = (known after apply)
      + location    = "us-west1"

...
...
...
Plan: 2 to add, 0 to change, 0 to destroy

Terraform should report 2 new resources to add. If this looks ok and you’re fine creating the resources then go ahead and apply and enter yes when prompted

$ terraform apply

An execution plan has been generated and is shown below
...
...

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

Enter a value:

If the apply went well you should see the new service in your GCP Cloud Run console. Verify the service URL to see if everything works before proceeding. It’s better to iron out any issues here before attempting to add the extra complication of restricting the service

Restricting Service Access

Next lets create the necessary resources to restrict access to our service

Before doing this however, let’s change the google_cloud_run_service resource to only allow internal and load balancer traffic

If you recall the annotation we made note of earlier we just need a minor change

# Change this
"run.googleapis.com/ingress" = "all"
↓
↓
# To this
"run.googleapis.com/ingress" = "internal-and-cloud-load-balancing"

We’ll be making use of a Serverless NEG (Network Endpoint Group) which will allow us to use a Cloud Run service as the backend of a GCP Load Balancer. Then all we have to do is restrict access to the Load Balancer and we’ll be done!

Create a networking.tf file to add the resources we just mentioned and to connect the Cloud Run service as a backend to the Load Balancer

There’s a lot going on here so let’s break it down.

We first create the IP to attach to the Load Balancer
Next we create a Serverless NEG and attach the Cloud Run Service we created earlier as the backend
We then create a security policy to restrict access to only the list of authorized IPs that are stored in a local variable
Next we use this serverless_negs module to connect all the resources together. If you’re using SSL then go ahead and set the ssl variable to true and update your domain there as well.
And finally we ouput the Load Balancer IP. We’ll need to use this IP to access the Cloud Run service going forward. If you configured SSL in the previous step you can just use your new domain to access it instead

Almost there! Now all we need to do is initialise the new module

$ terraform init
Initializing modules...
Downloading registry.terraform.io/GoogleCloudPlatform/lb-http/google 6.3.0 for service-loadbalancer...
- service-loadbalancer in .terraform/modules/service-loadbalancer/modules/serverless_negs

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/google from the dependency lock file
- Finding hashicorp/google-beta versions matching ">= 3.53.0, < 5.0.0"...
- Finding latest version of hashicorp/random...
- Installing hashicorp/random v3.4.3...
- Installed hashicorp/random v3.4.3 (signed by HashiCorp)
- Using previously-installed hashicorp/google v4.45.0
- Installing hashicorp/google-beta v4.45.0...
- Installed hashicorp/google-beta v4.45.0 (signed by HashiCorp)

...
...

Then let’s run plan

$ terraform plan

...
...

And if everything looks ok let’s go ahead and apply

$ terraform apply

...
...
...

Outputs:

cloud-run-load-balancer-ip = "<The LB IP will appear here>"

Final Test

Next let’s make sure the service firewall is in effect

First try accessing the Cloud Run service from it’s original URL which should be displayed in the Cloud Run GCP console. It should now return a Access Forbidden error since we’re only allowing internal and Load Balancer traffic

Error: Forbidden
Access is forbidden.

Next try accessing it via the Load Balancer IP (or your custom domain if you’re using SSL) and from one of the authorized IPs. You should see be able to access your Service in this scenario
Finally try accessing it again via the Load Balancer IP but this time from one an unauthorized IP. You should get a 403 Forbidden error this time

Ok!

Summary

In this article we utilized a Serverless NEG and a Load Balancer to restrict access to a Cloud Run service to a list of authorized IPs via Terraform

The source for this project can be found on GitHub. We used a local state but the source contains a commented out backend.tf file that you can modify and use to save your state to GCS.

Thanks and see you again on the next adventure :)

PS: If you’re interested in Akatsuki Games you can check us out here!