google.cloud.forseti.scanner.audit.log_sink_rules_engine module

Log Sinks/Exports rules engine.

Builds the RuleBook (LogSinkRuleBook) from the rule definitions (file either stored locally or in GCS) and compares a resource’s log sinks against the RuleBook to determine whether there are violations. Log Sinks rules can be defined on organization, folder, billing_account and project.

class LogSinkRuleBook(global_configs, rule_defs=None, snapshot_timestamp=None)[source]

Bases: google.cloud.forseti.scanner.audit.base_rules_engine.BaseRuleBook

The RuleBook for Log Sink configs.

Rules from the rules definition file are parsed and placed into a map, which associates the applies_to value and GCP resource (project, folder, billing_account or organization) with the rules defined for it.

Resources are evaulated against matching rules defined with applies_to = “self”. Project resources are also evaulated against rules for ancestor resources defined with applies_to = “children”.

__eq__(other)[source]

Equals.

Parameters:other (object) – Object to compare.
Returns:True or False.
Return type:bool
__ne__(other)[source]

Not Equals.

Parameters:other (object) – Object to compare.
Returns:True or False.
Return type:bool
__repr__()[source]

Object representation.

Returns:The object representation.
Return type:str
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 214
_abc_registry = <_weakrefset.WeakSet object>
add_rule(rule_def, rule_index)[source]

Add a rule to the rule book.

The rule supplied to this method is the dictionary parsed from the rules definition file.

For example, this rule…

# rules yaml: rules:

  • name: a rule mode: required resource:

    • type: organization applies_to: children resource_ids:

      • 11223344
    sink:
    • destination: ‘bigquery.googleapis.com/projects/my-proj/’ filter: ‘logName:”logs/cloudaudit.googleapis.com”’ include_children: ‘

… gets parsed into:

{

‘name’: ‘a rule’, ‘mode’: ‘required’, ‘resource’: [{

‘type’: ‘organization’, ‘applies_to’: ‘children’, ‘resource_ids’: [‘11223344’]

}], ‘sink’: {

‘destination’: ‘bigquery.googleapis.com/projects/my-proj/’, ‘filter’: logName:”logs/cloudaudit.googleapis.com”’, ‘include_children’: ‘

}

}

Parameters:
  • rule_def (dict) – Contains rule definition properties.
  • rule_index (int) – The index of the rule from the rule definitions. Assigned automatically when the rule book is built.
add_rules(rule_defs)[source]

Add rules to the rule book.

Parameters:rule_defs (dict) – Rules parsed from the rule definition file.
find_violations(resource, log_sinks)[source]

Find Log Sink violations in the rule book.

Parameters:
  • resource (gcp_type) – The resource that the log sinks belong to.
  • log_sinks (list) – list of LogSinks for resource.
Returns:

A generator of the rule violations.

Return type:

iterable

supported_resource_types = frozenset({'organization', 'project', 'billing_account', 'folder'})
supported_rule_applies_to = frozenset({'self', 'children'})
class LogSinkRulesEngine(rules_file_path, snapshot_timestamp=None)[source]

Bases: google.cloud.forseti.scanner.audit.base_rules_engine.BaseRulesEngine

Rules engine for Log Sinks.

add_rules(rules)[source]

Add rules to the rule book.

Parameters:rules (list) – The list of rules to add to the book.
build_rule_book(global_configs=None)[source]

Build LogSinkRuleBook from the rules definition file.

Parameters:global_configs (dict) – Global configurations.
find_violations(resource, log_sinks, force_rebuild=False)[source]

Determine whether a resources’s log sink config violates rules.

Parameters:
  • resource (gcp_type) – The resource that the log sinks belong to.
  • log_sinks (list) – list of LogSinks for resource.
  • force_rebuild (bool) – If True, rebuilds the rule book. This will reload the rules definition file and add the rules to the book.
Returns:

A generator of rule violations.

Return type:

iterable

class Rule(rule_name, rule_index, rule)[source]

Bases: object

Rule properties from the rule definition file. Also finds violations.

class RuleViolation(resource_type, resource_id, full_name, rule_name, rule_index, violation_type, sink_destination, sink_filter, sink_include_children, resource_data, resource_name)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, resource_type, resource_id, full_name, rule_name, rule_index, violation_type, sink_destination, sink_filter, sink_include_children, resource_data, resource_name)

Create new instance of RuleViolation(resource_type, resource_id, full_name, rule_name, rule_index, violation_type, sink_destination, sink_filter, sink_include_children, resource_data, resource_name)

__repr__()

Return a nicely formatted representation string

_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('resource_type', 'resource_id', 'full_name', 'rule_name', 'rule_index', 'violation_type', 'sink_destination', 'sink_filter', 'sink_include_children', 'resource_data', 'resource_name')
classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new RuleViolation object from a sequence or iterable

_replace(**kwds)

Return a new RuleViolation object replacing specified fields with new values

_source = "from builtins import property as _property, tuple as _tuple\nfrom operator import itemgetter as _itemgetter\nfrom collections import OrderedDict\n\nclass RuleViolation(tuple):\n 'RuleViolation(resource_type, resource_id, full_name, rule_name, rule_index, violation_type, sink_destination, sink_filter, sink_include_children, resource_data, resource_name)'\n\n __slots__ = ()\n\n _fields = ('resource_type', 'resource_id', 'full_name', 'rule_name', 'rule_index', 'violation_type', 'sink_destination', 'sink_filter', 'sink_include_children', 'resource_data', 'resource_name')\n\n def __new__(_cls, resource_type, resource_id, full_name, rule_name, rule_index, violation_type, sink_destination, sink_filter, sink_include_children, resource_data, resource_name):\n 'Create new instance of RuleViolation(resource_type, resource_id, full_name, rule_name, rule_index, violation_type, sink_destination, sink_filter, sink_include_children, resource_data, resource_name)'\n return _tuple.__new__(_cls, (resource_type, resource_id, full_name, rule_name, rule_index, violation_type, sink_destination, sink_filter, sink_include_children, resource_data, resource_name))\n\n @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new RuleViolation object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 11:\n raise TypeError('Expected 11 arguments, got %d' % len(result))\n return result\n\n def _replace(_self, **kwds):\n 'Return a new RuleViolation object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('resource_type', 'resource_id', 'full_name', 'rule_name', 'rule_index', 'violation_type', 'sink_destination', 'sink_filter', 'sink_include_children', 'resource_data', 'resource_name'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % list(kwds))\n return result\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + '(resource_type=%r, resource_id=%r, full_name=%r, rule_name=%r, rule_index=%r, violation_type=%r, sink_destination=%r, sink_filter=%r, sink_include_children=%r, resource_data=%r, resource_name=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values.'\n return OrderedDict(zip(self._fields, self))\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n resource_type = _property(_itemgetter(0), doc='Alias for field number 0')\n\n resource_id = _property(_itemgetter(1), doc='Alias for field number 1')\n\n full_name = _property(_itemgetter(2), doc='Alias for field number 2')\n\n rule_name = _property(_itemgetter(3), doc='Alias for field number 3')\n\n rule_index = _property(_itemgetter(4), doc='Alias for field number 4')\n\n violation_type = _property(_itemgetter(5), doc='Alias for field number 5')\n\n sink_destination = _property(_itemgetter(6), doc='Alias for field number 6')\n\n sink_filter = _property(_itemgetter(7), doc='Alias for field number 7')\n\n sink_include_children = _property(_itemgetter(8), doc='Alias for field number 8')\n\n resource_data = _property(_itemgetter(9), doc='Alias for field number 9')\n\n resource_name = _property(_itemgetter(10), doc='Alias for field number 10')\n\n"
full_name
resource_data
resource_id
resource_name
resource_type
rule_index
rule_name
sink_destination
sink_filter
sink_include_children
violation_type
find_violations(resource, log_sinks)[source]

Find Log Sink violations in the rule book.

Parameters:
  • resource (gcp_type) – The resource that the log sinks belong to.
  • log_sinks (list) – list of log sinks for resource.
Yields:

namedtuple – Returns RuleViolation named tuple.

_find_blacklist_violations(rule_def, sinks)[source]

Returns log sinks that match the rule definition.

Parameters:
  • rule_def (dict) – sink blacklist rule definition.
  • sinks (list) – list of LogSinks to be matched against blacklist.
Returns:

All LogSinks in sinks that violate the blacklist.

Return type:

list

_find_whitelist_violations(rule_def, sinks)[source]

Returns log sinks that DON’T match the rule definition.

Parameters:
  • rule_def (dict) – sink whitelist rule definition.
  • sinks (list) – list of LogSinks to be matched against whitelist.
Returns:

All LogSinks in sinks that violate the whitelist.

Return type:

list

_parse_sink_rule_spec(sink_spec)[source]

Validates and escapes a sink from a rule config.

Parameters:sink_spec (dict) – A sink definition from a LogSink rule definition.
Returns:A sink definition with fields escaped and globified, or None if sink_spec is invalid.
Return type:dict
_required_sink_missing(rule_def, sinks)[source]

Returns True if no sink matches the rule definition.

Parameters:
  • rule_def (dict) – required sink rule definition.
  • sinks (list) – list of LogSinks to be matched against required sink.
Returns:

True if at least one log sink matches the required sink.

Return type:

bool

_sink_matches_rule(rule_def, sink)[source]

Returns true if the log sink matches the rule’s sink definition.

Parameters:
  • rule_def (dict) – sink rule definition.
  • sink (LogSink) – sink being matched to the rule definition.
Returns:

True if sink matches rule definition.

Return type:

bool