google.cloud.forseti.services.model.importer.importer module

Importer implementations.

class EmptyImporter(session, readonly_session, model, dao, _, *args, **kwargs)[source]

Bases: object

Imports an empty model.

run()[source]

Runs the import.

class InventoryImporter(session, readonly_session, model, dao, service_config, inventory_index_id, *args, **kwargs)[source]

Bases: object

Imports data from Inventory.

_add_to_cache(resource, resource_id)[source]

Add a resource to the cache for parent lookup.

Parameters:
  • resource (object) – Resource to put in the cache.
  • resource_id (int) – The database key for the resource.
_convert_billing_account(billing_account)[source]

Convert a billing account to a database object.

Parameters:billing_account (object) – billing account to store.
_convert_bucket(bucket)[source]

Convert a bucket to a database object.

Parameters:bucket (object) – Bucket to store.
_convert_clouddns_resource(resource)[source]

Convert a CloudDNS resource to a database object.

Parameters:resource (dict) – A resource to store.
_convert_cloudsqlinstance(cloudsqlinstance)[source]

Convert a cloudsqlinstance to a database object.

Parameters:cloudsqlinstance (object) – Cloudsql to store.
_convert_computeengine_resource(resource)[source]

Convert an AppEngine resource to a database object.

Parameters:resource (dict) – An appengine resource to store.
_convert_crm_org_policy(org_policy)[source]

Convert an org policy to a database object.

Parameters:org_policy (object) – org policy to store.
_convert_dataset(dataset)[source]

Convert a dataset to a database object.

Parameters:dataset (object) – Dataset to store.
_convert_dataset_policy(dataset_policy)[source]

Convert a dataset policy to a database object.

Parameters:dataset_policy (object) – Dataset policy to store.
_convert_enabled_apis(enabled_apis)[source]

Convert a description of enabled APIs to a database object.

Parameters:enabled_apis (object) – Enabled APIs description to store.
_convert_folder(folder)[source]

Convert a folder to a database object.

Parameters:folder (object) – Folder to store.
_convert_gae_instance_resource(resource)[source]

Convert an AppEngine Instance resource to a database object.

Parameters:resource (dict) – A resource to store.
_convert_gae_resource(resource)[source]

Convert an AppEngine resource to a database object.

Parameters:resource (dict) – A resource to store.
_convert_iam_policy(iam_policy)[source]

Convert an IAM policy to a database object.

Parameters:iam_policy (object) – IAM policy to store.
_convert_kubernetes_cluster(cluster)[source]

Convert an AppEngine resource to a database object.

Parameters:cluster (dict) – A Kubernetes cluster resource to store.
_convert_lien(lien)[source]

Convert a lien to a database object.

Parameters:lien (object) – Lien to store.
_convert_organization(organization)[source]

Convert an organization a database object.

Parameters:organization (object) – Organization to store.
_convert_project(project)[source]

Convert a project to a database object.

Parameters:project (object) – Project to store.
_convert_pubsub_topic(topic)[source]

Convert a PubSub Topic to a database object.

Parameters:topic (object) – Pubsub Topic to store.
_convert_resource(resource, cached=False, display_key='name', email_key='email')[source]

Convert resource to a database object.

Parameters:
  • resource (dict) – A resource to store.
  • cached (bool) – Set to true for resources that have child resources or policies associated with them.
  • display_key (str) – The key in the resource dictionary to lookup to get the display name for the resource.
  • email_key (str) – The key in the resource dictionary to lookup to get the email associated with the resource.
_convert_role(role)[source]

Convert a role to a database object.

Parameters:role (object) – Role to store.
_convert_role_post()[source]

Executed after all roles were handled. Performs bulk insert.

_convert_service_config(service_config)[source]

Convert Kubernetes Service Config to a database object.

Parameters:service_config (dict) – A Service Config resource to store.
_convert_serviceaccount(service_account)[source]

Convert a service account to a database object.

Parameters:service_account (object) – Service account to store.
_convert_serviceaccount_key(service_account_key)[source]

Convert a service account key to a database object.

Parameters:service_account_key (object) – Service account key to store.
_convert_sink(sink)[source]

Convert a log sink to a database object.

Parameters:sink (object) – Sink to store.
_convert_spanner_db_resource(resource)[source]

Convert a Spanner Database resource to a database object.

Parameters:resource (dict) – A resource to store.
_convert_spanner_resource(resource)[source]

Convert a Spanner Instance resource to a database object.

Parameters:resource (dict) – A resource to store.
_full_resource_name(resource)[source]

Returns the parent object, full resource name and type name.

Parameters:
  • resource (object) – Resource whose full resource name and parent
  • be returned. (should) –
Returns:

full resource name for the provided resource.

Return type:

str

_get_parent(resource)[source]

Return the parent object for a resource from cache.

Parameters:resource (object) – Resource whose parent to look for.
Returns:cached object and full resource name
Return type:tuple
_is_role_unique(role_name)[source]

Check to see if the session contains Role with primary key = role_name.

Parameters:role_name (str) – The role name (Primary key of the role table).
Returns:
Whether or not session contains Role with
primary key = role_name.
Return type:bool
_is_root(resource)[source]

Checks if the resource is an inventory root. Result is cached.

Parameters:resource (object) – Resource to check.
Returns:Whether the resource is root or not
Return type:bool
_store_gsuite_membership(child, parent)[source]

Store a gsuite principal such as a group, user or member.

Parameters:
  • child (object) – member item.
  • parent (object) – parent part of membership.
_store_gsuite_membership_post()[source]

Flush storing gsuite memberships.

_store_gsuite_principal(principal)[source]

Store a gsuite principal such as a group, user or member.

Parameters:principal (object) – object to store.
Raises:Exception – if the principal type is unknown.
_store_iam_policy(policy)[source]

Store the iam policy of the resource.

Parameters:policy (object) – IAM policy to store.
Raises:KeyError – if member could not be found in any cache.
_store_resource(resource)[source]

Store an inventory resource in the database.

Parameters:resource (object) – Resource object to convert from.
static _type_name(resource)[source]

Return the type/name for that resource.

This is a simple wrapper for the to_type_name function.

Parameters:resource (object) – Resource to retrieve type/name for.
Returns:type/name representation of the resource.
Return type:str
static model_action_wrapper(session, inventory_iterable, action, post_action=None, flush_count=1000)[source]

Model action wrapper. This is used to reduce code duplication.

Parameters:
  • session (Session) – Database session.
  • inventory_iterable (Iterable) – Inventory iterable.
  • action (func) – Action taken during the iteration of the inventory list.
  • post_action (func) – Action taken after iterating the inventory list.
  • flush_count (int) – Flush every flush_count times.
Returns:

Number of item iterated.

Return type:

int

run()[source]

Runs the import.

Raises:NotImplementedError – If the importer encounters an unknown inventory type.
class ResourceCache[source]

Bases: dict

Resource cache.

__setitem__(key, value)[source]

Overriding to assert the keys does not exist previously.

Parameters:
  • key (object) – Key into the dict.
  • value (object) – Value to set.
Raises:

Exception – If the key already exists in the dict.

by_source(source)[source]

Helper to resolve client provided import sources.

Parameters:source (str) – Source to import from.
Returns:Chosen by source.
Return type:Importer