This guide explains how to setup Forseti on Kubernetes.
This is a proof of concept (POC) in the dev
branch.
If you decide to use this, consider implementing the k8s hardening recommendations, as well as CIS Benchmarks for GCP, section 7, Kubernetes.
Create (or use) a project with Forseti PaaS dependencies deployed: GCS buckets, Cloud SQL DB and Service Account(s)
Install Forseti server and client using
We are using dev branch for the POC.
The VM’s may be deleted or shutdown after the install as they will be replaced by the k8s solution.
Remaining steps can be run from a Google Cloud Shell VM in Forseti GCP project.
Download the Forseti server (and optionally client) service account keys and store them locally on the Cloud Shell VM temporarily for k8s secret creation. It is your responsibility to keep the key files secure.
git clone https://github.com/forseti-security/forseti-security.git
cd forseti-security
git checkout dev
gcloud builds submit --config install/docker/cloudbuild.yaml .
Dont forget the dot at the end.
Ref. https://cloud.google.com/cloud-build/docs/quickstart-docker#build_using_a_build_config_file
cd install/scripts
vi k8s_setup_forseti.sh
Specify the variables
If using the client, modify the client config file in the GCS bucket to point to the Forseti Server Cluster IP.
forseti-security/install/scripts/k8s_setup_forseti.sh
export FORSETI_SERVER_IP=10.43.240.3 # k8s Cluster IP for Forseti Server. Don't forget to manually add this to the Client config file in GCS bucket if using Client.
forseti-security/configs/client/forseti_conf_client.yaml.sample
server_ip: <server cluster ip>
We will move away from this hard coded approach, however this is the current POC implementation.
If Scanner rules_path defaulted to /home/ubuntu/forseti-security/rules
,
change it to use the GCS bucket directly:
forseti-security/configs/server/forseti_conf_server.yaml.sample
rules_path: gs://<server bucket>/rules
./k8s_setup_forseti.sh
Monitor the deployment in the GKE web console. Allow approximately 2 minutes for the cluster to spin up and another 30 seconds for the pods to become active.
Run kubectl get pods
to get pod ids
Connect: kubectl exec -it <CLIENT_POD_ID> -- /bin/bash
Run Forseti commands to verify that Forseti server is working as expected.
k8s CronJob example:
Drill into Forseti Workload to see Job History
File | Changes to Support GKE |
---|---|
https://github.com/forseti-security/forseti-security/blob/dev/.dockerignore | Added .dockerignore to reduce Docker image size. |
https://github.com/forseti-security/forseti-security/blob/dev/install/dependencies/apt_packages.txt | Add cron (to install on base image) |
https://github.com/forseti-security/forseti-security/blob/dev/install/docker/base | Install Google Cloud SDK on base image |
https://github.com/forseti-security/forseti-security/blob/dev/install/docker/cloudbuild.yaml | Added optional cache base image build step to reduce build time. Added optional unit tests build step |
https://github.com/forseti-security/forseti-security/blob/dev/install/docker/forseti | chmod +x docker_entrypoint.sh |
https://github.com/forseti-security/forseti-security/blob/dev/install/scripts/cloudsqlproxy.service.template.yaml | Cloud SQL Proxy Cluster IP Service Deployment template |
https://github.com/forseti-security/forseti-security/blob/dev/install/scripts/cloudsqlproxy.template.yaml | Cloud SQL Proxy Deployment template |
https://github.com/forseti-security/forseti-security/blob/dev/install/scripts/docker_entrypoint.sh | docker_entrpoint.sh initialises the container, starts services, runs scan as needed. |
https://github.com/forseti-security/forseti-security/blob/dev/install/scripts/forseti.client.template.yaml | Forseti Client Deployment template |
https://github.com/forseti-security/forseti-security/blob/dev/install/scripts/forseti.cronjob.template.yaml | Forseti CronJob Template |
https://github.com/forseti-security/forseti-security/blob/dev/install/scripts/forseti.server.service.template.yaml | Forseti Server Cluster IP Service Deployment template |
https://github.com/forseti-security/forseti-security/blob/dev/install/scripts/forseti.server.template.yaml | Forseti Server Deployment template |
https://github.com/forseti-security/forseti-security/blob/release-2.16.0/install/scripts/k8s_setup_forseti.sh | Example script to spin up cluster and deploy Forseti to GKE |