This page explains how to set up Forseti for local development.
To complete this guide, you will need:
First, install and configure the gcloud command-line tool to run the setup commands:
gcloud infoand check that the Account and Project displayed match your Forseti Security project. If it doesn’t match, run the following commands to configure gcloud for your Forseti Security project:
gcloud auth loginand use your Google credentials to authenticate.
gcloud initand select your Forseti Security project and Google account.
If you are setting up a developer environment, it’s best to use the credential
from the Forseti service accounts. You can also use your own Google credentials
from when you ran
gcloud auth login, but your personal credentials might drift
and differ from the Forseti service account.
If you are running Forseti on GCP, you’ll need to create service accounts with Cloud Identity and Access Management (Cloud IAM) roles to allow Forseti to read GCP data and to manage Forseti modules.
For a detailed explanation of how Forseti Security uses service accounts, see Forseti Service Accounts.
To create and grant roles to a service account for Forseti Inventory, Scanner, and Enforcer, follow the steps below:
gcloud auth activate-service-account --key-file=PATH/TO/KEYFILE.json
To enable your service account to collect G Suite data, follow the steps in Enabling G Suite Access.
For Forseti to have access to read data from your GCP environment, you will need to assign the roles below to the Forseti service account or to your Google user. It is recommended that you assign roles to the service account, especially if you run Forseti in multiple environments.
Granted at the organization level
Granted at the project level
Granted at the service account level
To grant the roles on the Cloud IAM policies, use the following commands:
Organization: the member has access to everything under the organization. Your authorized account must have the Organization Admin role to assign the role to another member.
To retrieve your organization id, follow these steps.
To add Cloud IAM policy bindings to the Organization, run the following command:
gcloud organizations add-iam-policy-binding ORGANIZATION_ID \ --member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
gcloud organizations add-iam-policy-binding 000000000001 \ --member=serviceAccount:email@example.com \ --role=roles/serviceusage.serviceUsageConsumer
Folder: the member has access to everything under a particular folder. Your authorized account must have the Folder Admin role to assign the role to another member.
To add Cloud IAM policy bindings to the Folder, run the following command:
gcloud alpha resource-manager folders add-iam-policy-binding FOLDER_ID \ --member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
Project: the member has access only to a particular project. Your authorized account must have the Owner role on the project or Folder Admin.
To add Cloud IAM policy bindings to the Project, run the following command:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member=MEMBER_TYPE:MEMBER_NAME --role=ROLE_NAME
gcloud projects add-iam-policy-binding my-project-name \ --member=serviceAccount:firstname.lastname@example.org \ --role=roles/storage.objectCreator
Service Account: grant additional roles to the service account. Your authorized account must have the Owner role on the project that is the source of the service account.
gcloud iam service-accounts add-iam-policy-binding YOUR_SERVICE_ACCOUNT \ --member=serviceACcount:YOUR_SERVICE_ACCOUNT --role=ROLE_NAME
Enable each of the required APIs by running the following command:
gcloud services enable <API URI>
|App Engine Admin||
|Cloud SQL Administration||
Forseti stores data in Cloud SQL. You can connect to the Cloud SQL instance by using the Cloud SQL proxy to authenticate to GCP with your Google credentials, instead of opening up network access to your Cloud SQL instance. To set up your Cloud SQL instance for Forseti, follow the steps below:
forseti_user, with read/write privileges for Forseti to access the database. Don’t set a password for the new user. This will allow Cloud SQL Proxy to handle authentication to your instance.
Use the SQL Proxy to proxy your connection to your Cloud SQL instance. Your INSTANCE_CONNECTION_NAME is the Instance Connection Name under Properties on the Cloud SQL dashboard instance details, in the format “PROJECTID:REGION:INSTANCEID”.
If you are using Cloud SDK authentication:
If you are using a service account to authenticate (recommended for production environments):
<path/to/cloud_sql_proxy> \ -instances=<INSTANCE_CONNECTION_NAME>=tcp:3306 \ -credential_file=<PATH_TO_KEY_FILE>
./cloud_sql_proxy \ -instances=foo-project-name:us-central1:mysql-instance=tcp:3306 \ -credential_file=/usr/local/google/home/foo/foo-project-cert-0d152c0e1bc8.json
If you are setting up a development environment, install MySQL Workbench. This is a GUI tool that will make it much easier to view and query the Forseti tables and data.
Install the necessary python dev tools and packages from apt_packages.txt.
This guide is written for use with Homebrew.
Use the following commands to install the necessary dependencies:
The system python that comes with OS X does not allow certain packages (such as six) to be modified. Install brew’s version of python so that separate python packages can be installed and managed.
brew install python@3
brew install openssl
brew install mysql
Ensure virtualenv is installed in your system. Virtualenv allows you to create multiple environments to contain different modules and dependencies in different projects:
sudo pip3 install virtualenv
Use the following command to create a virtualenv:
# create a virtualenv mkvirtualenv forseti-security workon forseti-security
Follow our contributing guidelines to create a fork of the Forseti code, and learn how to submit a pull request (PR).
Use the following command to install required build dependencies:
pip3 install -q --upgrade -r forseti-security/requirements.txt
Use the following commands to navigate to your cloned repository and run the Python setup:
cd forseti-security python3 setup.py install
Before you run Forseti, you need to edit the
For more information, see Configuring Forseti.
After you complete the above steps, you should be able to run the Forseti server and the command-line interface (CLI) client:
forseti_server \ --endpoint "localhost:50051" \ --forseti_db "mysql://email@example.com:3306/forseti_security" \ --services scanner model inventory explain notifier \ --config_file_path "PATH_TO_YOUR_CONFIG.yaml" \ --log_level=info \ --enable_console_log
To test the server is healthy and running, run the following command in another terminal window , and see if the server responds:
forseti inventory list
If you are installing on Mac OS X with Homebrew and get
a fatal error related to
'openssl/opensslv.h' file not found, you might need to
LDFLAGS for the openssl package. For more information,
see issue 3489.
To find the
LDFLAGS information and export them, run the
brew info openssl ... lots of information ... There aren't usually any consequences of this for you. If you build your own software and it requires this formula, you'll need to add to your build variables: LDFLAGS: -L/SOME/PATH/TO/openssl/lib CPPFLAGS: -I/SOME/PATH/TO/openssl/include
Next, copy the
CPPFLAGS values and export them, similar to the
export CPPFLAGS=-I/SOME/PATH/TO/openssl/include export LDFLAGS=-L/SOME/PATH/TO/openssl/lib
In the above example,
/SOME/PATH/TO represents the path specific to your
system. Make sure to use the values from your terminal.
If on Linux executing
mkvirtualenv forseti-security results in
bash: mkvirtualenv: command not found then try the following:
sudo pip3 install virtualenv virtualenvwrapper
Now attempt to make a virtual environment again.
If on Linux executing
bash: workon: command not found then
workon is in the source path. Try fixing
source path issue by executing the following:
and then trying
workon forseti-security again.
You can also put these into your bash profile:
export WORKON_HOME=$HOME/.virtualenvs source /usr/local/bin/virtualenvwrapper.sh
If on Linux executing
python3 setup.py install
EnvironmentError: mysql_config not found then
try the following:
sudo apt install default-libmysqlclient-dev
If on OS X executing
python3 setup.py install
my_config.h not found, then try the following:
brew install mysql brew unlink mysql brew install mysql-connector-c sed -i -e 's/libs="$libs -l "/libs="$libs -lmysqlclient -lssl -lcrypto"/g' /usr/local/bin/mysql_config pip3 install MySQL-python brew unlink mysql-connector-c brew link --overwrite mysql
If running the server results in this error:
ImportError: No module named forseti.common.util
Try to add a symlink from the source code to the virtual environment:
cd <virtual environment>/lib/python3.6/site-packages/google/cloud ln -s <path to your git source code>/google/cloud/forseti forseti