This page explains how to set up Forseti for local development.
To complete this guide, you will need:
First, install and configure the gcloud command-line tool to run the setup commands:
gcloud info
and check that the
Account and Project displayed match your Forseti Security project. If it
doesn’t match, run the following commands to configure gcloud for your
Forseti Security project:
gcloud auth login
and use your Google credentials to authenticate.gcloud init
and select your Forseti Security project and Google
account.If you are setting up a developer environment, it’s best to use the credential
from the Forseti service accounts. You can also use your own Google credentials
from when you ran gcloud auth login
, but your personal credentials might drift
and differ from the Forseti service account.
If you are running Forseti on GCP, you’ll need to create service accounts with Cloud Identity and Access Management (Cloud IAM) roles to allow Forseti to read GCP data and to manage Forseti modules.
For a detailed explanation of how Forseti Security uses service accounts, see Forseti Service Accounts.
To create and grant roles to a service account for Forseti Inventory, Scanner, and Enforcer, follow the steps below:
gcloud auth activate-service-account --key-file=PATH/TO/KEYFILE.json
To enable your service account to collect G Suite data, follow the steps in Enabling G Suite Access.
For Forseti to have access to read data from your GCP environment, you will need to assign the roles below to the Forseti service account or to your Google user. It is recommended that you assign roles to the service account, especially if you run Forseti in multiple environments.
Granted at the organization level
roles/appengine.appViewer
(server)roles/bigquery.metadataViewer
(server)roles/browser
(server)roles/cloudasset.viewer
(server)roles/cloudsql.viewer
(server)roles/compute.networkViewer
(server)roles/iam.securityReviewer
(server)roles/orgpolicy.policyViewer
(server)roles/servicemanagement.quotaViewer
(server)roles/serviceusage.serviceUsageConsumer
(server)Granted at the project level
roles/cloudsql.client
(server)roles/logging.logWriter
(server, client)roles/monitoring.metricWriter
(server)roles/storage.objectViewer
(server, client)roles/storage.objectCreator
(server)Granted at the service account level
roles/iam.serviceAccountTokenCreator
(server)To grant the roles on the Cloud IAM policies, use the following commands:
Organization: the member has access to everything under the organization. Your authorized account must have the Organization Admin role to assign the role to another member.
To retrieve your organization id, follow these steps.
To add Cloud IAM policy bindings to the Organization, run the following command:
gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
For exmple:
gcloud organizations add-iam-policy-binding 000000000001 \
--member=serviceAccount:service-account-01@my-foo-project.iam.gserviceaccount.com \
--role=roles/serviceusage.serviceUsageConsumer
Folder: the member has access to everything under a particular folder. Your authorized account must have the Folder Admin role to assign the role to another member.
To add Cloud IAM policy bindings to the Folder, run the following command:
gcloud alpha resource-manager folders add-iam-policy-binding FOLDER_ID \
--member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
Project: the member has access only to a particular project. Your authorized account must have the Owner role on the project or Folder Admin.
To add Cloud IAM policy bindings to the Project, run the following command:
gcloud projects add-iam-policy-binding PROJECT_ID \
--member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
For example:
gcloud projects add-iam-policy-binding my-project-name \
--member=serviceAccount:service-account-01@my-project-name.iam.gserviceaccount.com \
--role=roles/storage.objectCreator
Service Account: grant additional roles to the service account. Your authorized account must have the Owner role on the project that is the source of the service account.
gcloud iam service-accounts add-iam-policy-binding YOUR_SERVICE_ACCOUNT \
--member=serviceACcount:YOUR_SERVICE_ACCOUNT --role=ROLE_NAME
MEMBER_TYPE
user
, group
, serviceAccount
, or domain
MEMBER_NAME
Examples of MEMBER_TYPE:MEMBER_NAME
:
Enable each of the required APIs by running the following command:
gcloud services enable <API URI>
Name | API URI |
---|---|
Admin SDK | admin.googleapis.com |
App Engine Admin | appengine.googleapis.com |
BigQuery | bigquery-json.googleapis.com |
Cloud Asset | cloudasset.googleapis.com |
Cloud Billing | cloudbilling.googleapis.com |
Cloud IAM | iam.googleapis.com |
Cloud SQL Administration | sqladmin.googleapis.com |
Cloud SQL | sql-component.googleapis.com |
Cloud Storage | storage-api.googleapis.com |
Compute Engine | compute.googleapis.com |
Deployment Manager | deploymentmanager.googleapis.com |
Groups Settings | groupssettings.googleapis.com |
Kubernetes Engine | container.googleapis.com |
Resource Manager | cloudresourcemanager.googleapis.com |
Stackdriver Logging | logging.googleapis.com |
Service Management | servicemanagement.googleapis.com |
Service Usage | serviceusage.googleapis.com |
Forseti stores data in Cloud SQL. You can connect to the Cloud SQL instance by using the Cloud SQL proxy to authenticate to GCP with your Google credentials, instead of opening up network access to your Cloud SQL instance. To set up your Cloud SQL instance for Forseti, follow the steps below:
forseti_user
,
with read/write privileges
for Forseti to access the database. Don’t set a password for the new user.
This will allow Cloud SQL Proxy to handle authentication to your instance.forseti_security
.Use the SQL Proxy to proxy your connection to your Cloud SQL instance. Your INSTANCE_CONNECTION_NAME is the Instance Connection Name under Properties on the Cloud SQL dashboard instance details, in the format “PROJECTID:REGION:INSTANCEID”.
If you are using Cloud SDK authentication:
<path/to/cloud_sql_proxy> -instances=INSTANCE_CONNECTION_NAME=tcp:3306
If you are using a service account to authenticate (recommended for production environments):
<path/to/cloud_sql_proxy> \
-instances=<INSTANCE_CONNECTION_NAME>=tcp:3306 \
-credential_file=<PATH_TO_KEY_FILE>
For example:
./cloud_sql_proxy \
-instances=foo-project-name:us-central1:mysql-instance=tcp:3306 \
-credential_file=/usr/local/google/home/foo/foo-project-cert-0d152c0e1bc8.json
If you are setting up a development environment, install MySQL Workbench. This is a GUI tool that will make it much easier to view and query the Forseti tables and data.
Install the necessary python dev tools and packages from apt_packages.txt.
This guide is written for use with Homebrew.
Use the following commands to install the necessary dependencies:
Install python-dev:
The system python that comes with OS X does not allow certain packages (such as six) to be modified. Install brew’s version of python so that separate python packages can be installed and managed.
brew install python@3
Install openssl:
brew install openssl
Install mysql_config:
brew install mysql
Ensure virtualenv is installed in your system. Virtualenv allows you to create multiple environments to contain different modules and dependencies in different projects:
sudo pip3 install virtualenv
Use the following command to create a virtualenv:
# create a virtualenv
mkvirtualenv forseti-security
workon forseti-security
Follow our contributing guidelines to create a fork of the Forseti code, and learn how to submit a pull request (PR).
Use the following command to navigate to your cloned repository and install required build dependencies:
cd forseti-security
pip3 install -q --upgrade -r requirements.txt
Use the following commands to run the Python setup:
python3 setup.py install
Before you run Forseti, you need to edit the forseti_conf_server.yaml
file.
For more information, see Configuring Forseti.
After you complete the above steps, you should be able to run the Forseti server and the command-line interface (CLI) client:
forseti_server \
--endpoint "localhost:50051" \
--forseti_db "mysql+pymysql://root@127.0.0.1:3306/forseti_security" \
--services scanner model inventory explain notifier \
--config_file_path "PATH_TO_YOUR_CONFIG.yaml" \
--log_level=info \
--enable_console_log
To test the server is healthy and running, run the following command in another terminal window , and see if the server responds:
forseti inventory list
If you are installing on Mac OS X with Homebrew and get
a fatal error related to 'openssl/opensslv.h' file not found
, you might need to
export CPPFLAGS
and LDFLAGS
for the openssl package. For more information,
see issue 3489.
To find the CPPFLAGS
and LDFLAGS
information and export them, run the
following command:
brew info openssl
... lots of information ...
There aren't usually any consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:
LDFLAGS: -L/SOME/PATH/TO/openssl/lib
CPPFLAGS: -I/SOME/PATH/TO/openssl/include
Next, copy the LDFLAGS
and CPPFLAGS
values and export them, similar to the
following:
export CPPFLAGS=-I/SOME/PATH/TO/openssl/include
export LDFLAGS=-L/SOME/PATH/TO/openssl/lib
In the above example, /SOME/PATH/TO
represents the path specific to your
system. Make sure to use the values from your terminal.
If on Linux executing mkvirtualenv forseti-security
results in
bash: mkvirtualenv: command not found
then try the following:
sudo pip3 install virtualenv virtualenvwrapper
Now attempt to make a virtual environment again.
If on Linux executing workon forseti-security
results in bash: workon: command not found
then
ensure workon
is in the source path. Try fixing
source path issue by executing the following:
source /usr/local/bin/virtualenvwrapper.sh
and then trying workon forseti-security
again.
You can also put these into your bash profile:
export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh
If on Linux executing python3 setup.py install
results in EnvironmentError: mysql_config not found
then
try the following:
sudo apt install default-libmysqlclient-dev
If on OS X executing python3 setup.py install
results in my_config.h
not found, then try the following:
brew install mysql
brew unlink mysql
brew install mysql-connector-c
sed -i -e 's/libs="$libs -l "/libs="$libs -lmysqlclient -lssl -lcrypto"/g' /usr/local/bin/mysql_config
pip3 install MySQL-python
brew unlink mysql-connector-c
brew link --overwrite mysql
If running the server results in this error:
ImportError: No module named forseti.common.util
Try to add a symlink from the source code to the virtual environment:
cd <virtual environment>/lib/python3.6/site-packages/google/cloud
ln -s <path to your git source code>/google/cloud/forseti forseti