GCP Deployment

This page explains how to use the Google Cloud Deployment Manager (DM) to set up Forseti for your Google Cloud Platform (GCP) resources. The DM templates help you manage configuration for Forseti Inventory and Scanner. It doesn’t currently schedule or execute Enforcer. You’ll use DM to do the following:

  • Create a Cloud SQL instance and database to store Inventory data.
  • Create a Cloud Storage bucket for Scanner output.
  • Create a Google Compute Engine instance to deploy and run Forseti Security.
  • Manage Forseti Security configuration and automatically run Forseti Inventory and Scanner.

This is the manual version of the Quickstart.

Before you begin

To complete this guide, you will need:

  • A GCP organization.
  • A GCP project for Forseti Security with billing enabled.
  • The ability to assign roles on your organization’s Cloud IAM policy.

Setting up Forseti Security

Setting up gcloud

First, install and configure the gcloud command-line tool so you can run the setup commands:

  1. Download and install the gcloud command-line tool.
  2. Make sure gcloud is configured by running gcloud info and check that the Account and Project displayed match your Forseti Security project. If it doesn’t match, run the following commands to configure gcloud for your Forseti Security project:
    1. Run gcloud auth login and use your Google credentials to authenticate.
    2. Run gcloud init and select your Forseti Security project and Google account.

Creating service accounts

If you are setting up a developer environment, you can just use the Google credentials from when you ran gcloud auth login and go to the next section.

If you are running Forseti on GCP, you’ll need create service accounts with Cloud Identity and Access Management (Cloud IAM) roles to allow Forseti to read GCP data and to manage Forseti modules.

For a detailed explanation of how Forseti Security uses service accounts, refer to “Forseti Service Accounts”.

To create and grant roles to a service account for Forseti Inventory, Scanner, and Enforcer, follow the steps below.

  1. Go to your Google Cloud Platform console and create a new service account.
  2. Create and download a json key for the service account.
  3. Run the following command to assume the service account credentials:
  gcloud auth activate-service-account --key-file=PATH/TO/KEYFILE.json

To create a separate service account for enabling G Suite data collection, follow the steps in “Enabling GSuite Google Groups Collection”.

Assigning roles

In order for Forseti to have access to read data from your GCP environment, you will need to assign roles to a particular member: either the Inventory/Scanner/Enforcer service account or your Google user. You can refer to the official documentation about members for more information.

Also, you can grant the roles on the organization, folder, or project IAM policies.

  • Organization IAM: the member has access to the everything under the organization. Your authed account must have the Organization Admin role to assign the role to another member.

    The command to add IAM policy bindings to the organization IAM is:

    $ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
       --member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
    
  • Folder IAM: the member has access to everything under a particular folder. Your authed account must have the Folder IAM Admin role to assign the role to another member.

    The command to add IAM policy bindings to the folder IAM is:

    $ gcloud alpha resource-manager folders add-iam-policy-binding FOLDER_ID \
       --member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
    
  • Project IAM: the member has access only to a particular project. Your authed account must either have the Owner role on the project or Folder IAM Admin.

    The command to add IAM policy bindings to the folder IAM is:

    $ gcloud projects add-iam-policy-binding PROJECT_ID \
       --member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
    

The MEMBER_TYPE value is either user, group, serviceAccount, or domain.

The MEMBER_NAME is either a domain (e.g. example.com) or an email address (user@example.com).

Examples of MEMBER_TYPE:MEMBER_NAME:

  • user:me@gmail.com
  • serviceAccount:forseti-gcp-reader@your-project-id.iam.gserviceaccount.com
  • group:my-forseti-group@example.com
  • domain:example.com

Use these commands to grant the Forseti roles to your organization IAM policy. If you need to assign the roles on the folder or project level, use the commands from above, with the roles below.

$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/browser
$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/compute.networkViewer
$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/iam.securityReviewer
$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/appengine.appViewer
$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/servicemanagement.quotaViewer
$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/cloudsql.viewer
$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/compute.securityAdmin
$ gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/bigquery.dataViewer

Project Cloud IAM roles

These are necessary for reading/writing Forseti data in Google Cloud Storage and Cloud SQL. Do not assign these outside of the project IAM.

$ gcloud projects add-iam-policy-binding FORSETI_PROJECT_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/storage.objectViewer
$ gcloud rpojects add-iam-policy-binding FORSETI_PROJECT_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/storage.objectCreator
$ gcloud projects add-iam-policy-binding FORSETI_PROJECT_ID \
--member=MEMBER_TYPE:MEMBER_NAME \
--role=roles/cloudsql.client

Enabling APIs

Enable each of the required APIs by running the following command:

  $ gcloud beta service-management enable <API URI>
Name API URI
Admin SDK API admin.googleapis.com
AppEngine Admin API appengine.googleapis.com
Cloud Resource Manager API cloudresourcemanager.googleapis.com
Cloud SQL Admin API sqladmin.googleapis.com
Cloud SQL API sql-component.googleapis.com
Compute Engine API compute.googleapis.com
Deployment Manager API deploymentmanager.googleapis.com

Customizing deployment templates

The Forseti deployment template files can be found in the deployment-templates directory. Make a copy of deploy-forseti.yaml.sample as deploy-forseti.yaml and update at least the following variables:

  • CLOUDSQL_INSTANCE_NAME
    • Instance names must start with a letter and consist of lowercase letters, numbers, and hyphens, such as “valid-instancename-1”.
    • Instance names must be unique and can’t be reused for up to 7 days after deletion.
    • Read more about naming guidelines.
  • SCANNER_BUCKET
    • Add only the bucket name. Don’t include gs://.
    • Make sure the name conforms to bucket naming guidelines.
    • Use the same name for both the Cloud Storage and Compute Engine sections in the template.

By default, the deployment template retrieves the latest stable branch to set release version and src-path. To get a different release archive, change the following values:

  • src-path: the path to the release, such as https://github.com/GoogleCloudPlatform/forseti-security/archive/TAG_NAME.tar.gz.
    • Tag names start with a “v” and can be found on the forseti-security page.
    • To use the master branch, use master as the TAG_NAME.
  • release-version : the tag name, without the “v” prefix.
    • To use the master branch (latest release), set this value to “master”.
  • branch-name: the git branch to deploy.

Following are examples of different values for the src-path and release-version:

  # master branch:
  branch-name: "master"
  # release-version: "1.0"
  src-path: https://github.com/GoogleCloudPlatform/forseti-security

  # v1.0 release:
  # branch-name: "master"
  release-version: "1.0"
  src-path: https://github.com/GoogleCloudPlatform/forseti-security

You can also modify the following templates:

  • cloudsql/cloudsql-instance.py: the template for the Google Cloud SQL instance.
  • cloudsql/cloudsql-database.py: the template for the Google Cloud SQL database.
  • iam/service_acct.py: the template for service accounts.
  • storage/bucket.py: the template for the Google Cloud Storage buckets.
  • compute-engine/forseti-instance.py: the template for the Compute Engine instance where Forseti Security will run.
    • You can customize the startup script.
    • By default, the startup script sets up the environment to install Forseti Security and run the tools hourly.
    • Learn more about Using Startup Scripts.

Deploying Forseti Security

After you configure your deployment template variables, use the following command to create a new deployment:

  gcloud deployment-manager deployments create forseti-security \
    --config path/to/deploy-forseti.yaml

To view your deployment details, access the Cloud Console Deployment Manager dashboard.

Configuring Forseti Settings

You MUST provide a Forseti configuration file before Forseti will run properly. Refer to “Configuring Forseti” to prepare a forseti_conf.yaml.

At the very minimum, you should edit the following values (the following values are specific to GCP deployments and do not necessarily reflect the values you will use for other deployments):

  • db_host: Set this to “localhost” or “127.0.0.1”.
  • db_user: Set this to “root”.
  • db_name: Set this to “forseti_security”.
  • output_path: Set this to gs://YOUR_SCANNER_BUCKET/scanner_violations (where YOUR_SCANNER_BUCKET is the SCANNER_BUCKET value you used in deploy-forseti.yaml).
  • rules_path: Set this to “/home/ubuntu/forseti-security/rules”

Customize Forseti Rules

Customize your Forseti rules by following this guide.

Move Configuration to GCS

After editing your forseti_conf.yaml, copy it to your GCS SCANNER_BUCKET:

gsutil cp configs/forseti_conf.yaml gs://YOUR_SCANNER_BUCKET/configs/forseti_conf.yaml

Next, edit your rules and copy the rules directory to the GCS SCANNER_BUCKET:

gsutil cp -r rules gs://YOUR_SCANNER_BUCKET/

Checking the Forseti setup

At this point, Forseti should be installing on the GCE instance that was created from the Deployment Manager script. You can ssh into the GCE instance and watch the /tmp/deployment.log to watch the progress.

To ssh to your GCE instance, you can either use gcloud compute ssh (official docs) or go to the GCE instance details page from Google Cloud Console > Compute Engine, then click the “SSH” button for your instance. If a window doesn’t pop up right away (or gets blocked), try clicking the SSH button again.

Once you’re logged into the GCE instance, you can “tail” the deployment log (use CTRL-C to exit):

tail -f /tmp/deployment.log

Viewing Forseti Logs

You can also check out the Stackdriver Log Viewer to see the logs generated by Forseti when it runs. Go to Google Cloud Console > Logging > Logs, then set the following dropdowns:

  • First dropdown: GCE VM Instance > All instance_id
  • Second dropdown: “All logs” or “syslog”

If you run into any issues, especially if deployment.log shows errors, please contact the Forseti team at discuss@forsetisecurity.org.