google.cloud.forseti.services.model.importer.importer module¶

Importer implementations.

class EmptyImporter(session, readonly_session, model, dao, _, *args, **kwargs)[source]¶

Bases: object

Imports an empty model.

run()[source]¶

Runs the import.

class InventoryImporter(session, readonly_session, model, dao, service_config, inventory_index_id, *args, **kwargs)[source]¶

Bases: object

Imports data from Inventory.

_add_to_cache(resource, resource_id)[source]¶

Add a resource to the cache for parent lookup.

Parameters:	resource (object) – Resource to put in the cache. resource_id (int) – The database key for the resource.

_convert_bigquery_table(table)[source]¶

Convert a table to a database object.

Parameters:	table (object) – table to store.

_convert_billing_account(billing_account)[source]¶

Convert a billing account to a database object.

Parameters:	billing_account (object) – billing account to store.

_convert_bucket(bucket)[source]¶

Convert a bucket to a database object.

Parameters:	bucket (object) – Bucket to store.

_convert_clouddns_resource(resource)[source]¶

Convert a CloudDNS resource to a database object.

Parameters:	resource (dict) – A resource to store.

_convert_cloudsqlinstance(cloudsqlinstance)[source]¶

Convert a cloudsqlinstance to a database object.

Parameters:	cloudsqlinstance (object) – Cloudsql to store.

_convert_composite_root(resource)[source]¶

Convert a composite root resource to a database object.

Parameters:	resource (dict) – A resource to store.

_convert_computeengine_resource(resource)[source]¶

Convert an AppEngine resource to a database object.

Parameters:	resource (dict) – An appengine resource to store.

_convert_crm_org_policy(org_policy)[source]¶

Convert an org policy to a database object.

Parameters:	org_policy (object) – org policy to store.

_convert_dataproc_cluster(cluster)[source]¶

Convert a dataproc cluster to a database object.

Parameters:	cluster (object) – Dataproc Cluster to store.

_convert_dataset(dataset)[source]¶

Convert a dataset to a database object.

Parameters:	dataset (object) – Dataset to store.

_convert_dataset_policy(dataset_policy)[source]¶

Convert a dataset policy to a database object.

Parameters:	dataset_policy (object) – Dataset policy to store.

_convert_enabled_apis(enabled_apis)[source]¶

Convert a description of enabled APIs to a database object.

Parameters:	enabled_apis (object) – Enabled APIs description to store.

_convert_folder(folder)[source]¶

Convert a folder to a database object.

Parameters:	folder (object) – Folder to store.

_convert_gae_instance_resource(resource)[source]¶

Convert an AppEngine Instance resource to a database object.

Parameters:	resource (dict) – A resource to store.

_convert_gae_resource(resource)[source]¶

Convert an AppEngine resource to a database object.

Parameters:	resource (dict) – A resource to store.

_convert_gcs_policy(gcs_policy)[source]¶

Convert a gcs policy to a database object.

Parameters:	gcs_policy (object) – Cloud Storage Bucket ACL policy to store.

_convert_iam_policy(iam_policy)[source]¶

Convert an IAM policy to a database object.

Parameters:	iam_policy (object) – IAM policy to store.

_convert_kms_ckv_resource(resource)[source]¶

Convert a KMS CryptoKeyVersion resource to a database object.

Parameters:	resource (dict) – A resource to store.

_convert_kms_resource(resource)[source]¶

Convert a KMS resource to a database object.

Parameters:	resource (dict) – A resource to store.

_convert_kubernetes_binding(kubernetes_clusterrolebinding)[source]¶

Convert a Kubernetes ClusterRoleBinding resource to a database

object.

Parameters:	kubernetes_clusterrolebinding (dict) – A Kubernetes resource to store. (ClusterRoleBinding) –

_convert_kubernetes_cluster(kubernetes_cluster)[source]¶

Convert a Kubernetes Cluster resource to a database object.

Parameters:	kubernetes_cluster (dict) – A Kubernetes cluster resource to store.

_convert_kubernetes_clusterrole(kubernetes_clusterrole)[source]¶

Convert a Kubernetes ClusterRole resource to a database object.

Parameters:	kubernetes_clusterrole (dict) – A Kubernetes ClusterRole resource to store. –

_convert_kubernetes_namespace(kubernetes_namespace)[source]¶

Convert a Kubernetes Namespace resource to a database object.

Parameters:	kubernetes_namespace (dict) – A Kubernetes Namespace resource to store. –

_convert_kubernetes_node(kubernetes_node)[source]¶

Convert a Kubernetes Node resource to a database object.

Parameters:	kubernetes_node (dict) – A Kubernetes Node resource to store.

_convert_kubernetes_pod(kubernetes_pod)[source]¶

Convert a Kubernetes Pod resource to a database object.

Parameters:	kubernetes_pod (dict) – A Kubernetes Pod resource to store.

_convert_kubernetes_role(kubernetes_role)[source]¶

Convert a Kubernetes Role resource to a database object.

Parameters:	kubernetes_role (dict) – A Kubernetes Role resource to store.

_convert_kubernetes_rolebinding(kubernetes_rolebinding)[source]¶

Convert a Kubernetes RoleBinding resource to a database object.

Parameters:	kubernetes_rolebinding (dict) – A Kubernetes RoleBinding resource to store. –

_convert_lien(lien)[source]¶

Convert a lien to a database object.

Parameters:	lien (object) – Lien to store.

_convert_organization(organization)[source]¶

Convert an organization a database object.

Parameters:	organization (object) – Organization to store.

_convert_project(project)[source]¶

Convert a project to a database object.

Parameters:	project (object) – Project to store.

_convert_pubsub_resource(resource)[source]¶

Convert a PubSub resource to a database object.

Parameters:	resource (object) – Pubsub resource to store.

_convert_resource(resource, cached=False, display_key='name', email_key='email')[source]¶

Convert resource to a database object.

Parameters:	resource (Resource) – A resource to store. cached (bool) – Set to true for resources that have child resources or policies associated with them. display_key (str) – The key in the resource dictionary to lookup to get the display name for the resource. email_key (str) – The key in the resource dictionary to lookup to get the email associated with the resource.

_convert_role(role)[source]¶

Convert a role to a database object.

Parameters:	role (object) – Role to store.

_convert_role_post()[source]¶

Executed after all roles were handled. Performs bulk insert.

_convert_service_config(service_config)[source]¶

Convert Kubernetes Service Config to a database object.

Parameters:	service_config (dict) – A Service Config resource to store.

_convert_serviceaccount(service_account)[source]¶

Convert a service account to a database object.

Parameters:	service_account (object) – Service account to store.

_convert_serviceaccount_key(service_account_key)[source]¶

Convert a service account key to a database object.

Parameters:	service_account_key (object) – Service account key to store.

_convert_sink(sink)[source]¶

Convert a log sink to a database object.

Parameters:	sink (object) – Sink to store.

_convert_spanner_db_resource(resource)[source]¶

Convert a Spanner Database resource to a database object.

Parameters:	resource (dict) – A resource to store.

_convert_spanner_resource(resource)[source]¶

Convert a Spanner Instance resource to a database object.

Parameters:	resource (dict) – A resource to store.

_flush_session()[source]¶

Flush the session with rollback on errors.

_full_resource_name(resource)[source]¶

Returns the parent object, full resource name and type name.

Parameters:	resource (object) – Resource whose full resource name and parent be returned. (should) –
Returns:	full resource name for the provided resource.
Return type:	str

_get_parent(resource)[source]¶

Return the parent object for a resource from cache.

Parameters:	resource (object) – Resource whose parent to look for.
Returns:	cached object and full resource name
Return type:	tuple

_is_role_unique(role_name)[source]¶

Check to see if the session contains Role with primary key = role_name.

Parameters:	role_name (str) – The role name (Primary key of the role table).
Returns:	Whether or not session contains Role with primary key = role_name.
Return type:	bool

_is_root(resource)[source]¶

Checks if the resource is an inventory root. Result is cached.

Parameters:	resource (object) – Resource to check.
Returns:	Whether the resource is root or not
Return type:	bool

_store_groups_settings(settings)[source]¶

Store gsuite settings.

Parameters:	settings (object) – settings resource object.

_store_gsuite_membership(child, parent)[source]¶

Store a gsuite principal such as a group, user or member.

Parameters:	child (object) – member item. parent (object) – parent part of membership.

_store_gsuite_membership_post()[source]¶

Flush storing gsuite memberships.

_store_gsuite_principal(principal)[source]¶

Store a gsuite principal such as a group, user or member.

Parameters:	principal (object) – object to store.
Raises:	Exception – if the principal type is unknown.

_store_iam_policy(policy)[source]¶

Store the iam policy of the resource.

Parameters:	policy (object) – IAM policy to store.
Raises:	KeyError – if member could not be found in any cache.

_store_resource(resource)[source]¶

Store an inventory resource in the database.

Parameters:	resource (object) – Resource object to convert from.

static _type_name(resource)[source]¶

Return the type/name for that resource.

This is a simple wrapper for the to_type_name function.

Parameters:	resource (object) – Resource to retrieve type/name for.
Returns:	type/name representation of the resource.
Return type:	str

model_action_wrapper(inventory_iterable, action, post_action=None, flush_count=1000)[source]¶

Model action wrapper. This is used to reduce code duplication.

Parameters:	inventory_iterable (Iterable) – Inventory iterable. action (func) – Action taken during the iteration of the inventory list. post_action (func) – Action taken after iterating the inventory list. flush_count (int) – Flush every flush_count times.
Returns:	Number of item iterated.
Return type:	int

run()[source]¶

Runs the import.

Raises:	NotImplementedError – If the importer encounters an unknown inventory type.

class ResourceCache[source]¶

Bases: dict

Resource cache.

__setitem__(key, value)[source]¶

Overriding to assert the keys does not exist previously.

Parameters:	key (object) – Key into the dict. value (object) – Value to set.
Raises:	Exception – If the key already exists in the dict.

by_source(source)[source]¶

Helper to resolve client provided import sources.

Parameters:	source (str) – Source to import from.
Returns:	Chosen by source.
Return type:	Importer

group_name(group)[source]¶

Create the type:name representation for a group.

Parameters:	group (object) – group to create representation from.
Returns:	group:name representation of the group.
Return type:	str