Deploy Forseti Security on Kubernetes

This guide explains how to setup Forseti on Kubernetes.

This is a proof of concept (POC) in the dev branch.

If you decide to use this, consider implementing the k8s hardening recommendations, as well as CIS Benchmarks for GCP, section 7, Kubernetes.

0. Install Pre-Requistes

Create (or use) a project with Forseti PaaS dependencies deployed: GCS buckets, Cloud SQL DB and Service Account(s)

Install Forseti server and client using

• the standard installer
• or the terraform installer.

We are using dev branch for the POC.

The VM’s may be deleted or shutdown after the install as they will be replaced by the k8s solution.

Remaining steps can be run from a Google Cloud Shell VM in Forseti GCP project.

1. Obtain Service Account Keys

Download the Forseti server (and optionally client) service account keys and store them locally on the Cloud Shell VM temporarily for k8s secret creation. It is your responsibility to keep the key files secure.

2. Clone the Forseti Security Git Repo

git clone https://github.com/forseti-security/forseti-security.git
cd forseti-security
git checkout dev

3. Build Docker Image

gcloud builds submit --config install/docker/cloudbuild.yaml .

Dont forget the dot at the end.

Ref. https://cloud.google.com/cloud-build/docs/quickstart-docker#build_using_a_build_config_file

4. Edit the Example Kubernetes Deployment Script

cd install/scripts
vi k8s_setup_forseti.sh

Specify the variables

• New cluster spec or name of existing cluster
• Path to service account keys
• Forseti GCS buckets
• MySQL connection string
• Type of k8s deployment: CronJob or long running server (+ optional client)
• Cron schedule

5. Update Client Config File with Forseti Service IP

If using the client, modify the client config file in the GCS bucket to point to the Forseti Server Cluster IP.

forseti-security/install/scripts/k8s_setup_forseti.sh

export FORSETI_SERVER_IP=10.43.240.3 # k8s Cluster IP for Forseti Server. Don't forget to manually add this to the Client config file in GCS bucket if using Client. 

forseti-security/configs/client/forseti_conf_client.yaml.sample

server_ip: <server cluster ip>

We will move away from this hard coded approach, however this is the current POC implementation.

6. Update the rules_path in server config

If Scanner rules_path defaulted to /home/ubuntu/forseti-security/rules, change it to use the GCS bucket directly:

forseti-security/configs/server/forseti_conf_server.yaml.sample

rules_path: gs://<server bucket>/rules

7. Run the Kubernetes Deployment Script (from its directory)

./k8s_setup_forseti.sh

8. Verify

Monitor the deployment in the GKE web console. Allow approximately 2 minutes for the cluster to spin up and another 30 seconds for the pods to become active.

Long Running Server Example

Run kubectl get pods to get pod ids Connect: kubectl exec -it <CLIENT_POD_ID> -- /bin/bash

Run Forseti commands to verify that Forseti server is working as expected.

k8s Cron Job Example

k8s CronJob example:

Drill into Forseti Workload to see Job History

File Notes