google.cloud.forseti.services.inventory.cai_temporary_storage module

Inventory temporary storage for Cloud Asset data.

class CaiDataAccess[source]

Bases: object

Access to the CAI temporary store table.

static _extract_asset_data(row)[source]

Extracts the data from the database row.

Parameters:row (dict) – Database row from select query.
Returns:
The dict representation of the asset
data and an Asset metadata along with it.
Return type:Tuple[dict, AssetMetadata]
static clear_cai_data(engine)[source]

Deletes all temporary CAI data from the cai temporary table.

Parameters:engine (object) – Database engine.
Returns:The number of rows deleted.
Return type:int
static fetch_cai_asset(content_type, asset_type, name, engine)[source]

Returns a single resource from the cai temporary store.

Retries query on exception up to 5 times.

Parameters:
  • content_type (ContentTypes) – The content type to return.
  • asset_type (str) – The asset type to return.
  • name (str) – The resource to return.
  • engine (object) – Database engine.
Returns:

The content data for the specified resource.

Return type:

dict

static iter_cai_assets(content_type, asset_type, parent_name, engine)[source]

Iterate the objects in the cai temporary table.

Retries query on exception up to 5 times.

Parameters:
  • content_type (ContentTypes) – The content type to return.
  • asset_type (str) – The asset type to return.
  • parent_name (str) – The parent resource to iter children under.
  • engine (object) – Database engine.
Yields:

object – The content_type data for each resource.

static populate_cai_data(data, engine)[source]

Add assets from cai data dump into cai temporary table.

Parameters:
  • data (file) – A file like object, line delimeted text dump of json data representing assets from Cloud Asset Inventory exportAssets API.
  • engine (object) – Database engine.
Returns:

The number of rows inserted

Return type:

int

class CaiTemporaryStore(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

CAI temporary inventory table.

UNPARENTED_ASSETS = frozenset({'cloudbilling.googleapis.com/BillingAccount', 'cloudresourcemanager.googleapis.com/Organization'})
static _get_parent_name(asset)[source]

Determines the parent name from the resource data.

Parameters:asset (dict) – An Asset object.
Returns:The parent name for the resource.
Return type:str
_sa_class_manager = {'asset_data': <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x7f4ae51893b8>, 'asset_type': <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x7f4ae5189360>, 'content_type': <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x7f4ae5189620>, 'name': <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x7f4ae5eb9308>, 'parent_name': <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x7f4ae5eb91a8>}
asset_data
asset_type
content_type
classmethod delete_all(engine)[source]

Deletes all rows from this table.

Parameters:engine (object) – db engine
Returns:The number of rows deleted.
Return type:int
Raises:Exception – Reraises any exception.
classmethod from_json(asset_json)[source]

Creates a database row object from the json data in a dump file.

Parameters:asset_json (str) – The json representation of an Asset.
Returns:database row dictionary or None if there is no data.
Return type:dict
name
parent_name
class ContentTypes[source]

Bases: enum.Enum

Cloud Asset Inventory Content Types.

iam_policy = 2
resource = 1
_initialize(engine)[source]

Create all tables in the database if not existing.

Parameters:engine (object) – Database engine to operate on.
create_sqlite_db(threads=1)[source]

Create and initialize a sqlite db engine for use as the CAI temp store.

Parameters:threads (int) – The number of threads to support. Pool size is set to 5 greater than the number of threads, so that each thread can get its own connection to the temp database, with a few spare.
Returns:
A tuple containing an engine
object initialized to a temporary sqlite db file, and the path to the temporary file.
Return type:Tuple[sqlalchemy.engine.Engine, str]