google.cloud.forseti.scanner.audit.bigquery_rules_engine module

Rules engine for Big Query data sets.

class BigqueryRuleBook(rule_defs=None)[source]

Bases: google.cloud.forseti.scanner.audit.base_rules_engine.BaseRuleBook

The RuleBook for Big Query dataset resources.

_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 214
_abc_registry = <_weakrefset.WeakSet object>
classmethod _build_rule(rule_def, rule_index)[source]

Build a rule.

Parameters:
  • rule_def (dict) – A dictionary containing rule definition properties.
  • rule_index (int) – The index of the rule from the rule definitions. Assigned automatically when the rule book is built.
Returns:

rule for the given definition.

Return type:

Rule

classmethod _get_binding_from_old_syntax(rule_def)[source]

Get a binding for configs set with the old syntax.

Default fields to glob as default as that is what the fields used to be set.

Parameters:rule_def (dict) – raw rule definition.
Returns:
If an old style config field is set, returns a single binding
with a single member.
Return type:Binding
add_rule(rule_def, rule_index)[source]

Add a rule to the rule book.

Parameters:
  • rule_def (dict) – A dictionary containing rule definition properties.
  • rule_index (int) – The index of the rule from the rule definitions. Assigned automatically when the rule book is built.
add_rules(rule_defs)[source]

Add rules to the rule book.

Parameters:rule_defs (dict) – rule definitions dictionary.
find_violations(resource, bq_acl)[source]

Find acl violations in the rule book.

Parameters:
  • resource (gcp_type) – The GCP resource associated with the acl. This is where we start looking for rule violations and we move up the resource hierarchy (if permitted by the resource’s “inherit_from_parents” property).
  • bq_acl (BigqueryAccessControls) – The acl to compare the rules against.
Returns:

A generator of the rule violations.

Return type:

iterable

class BigqueryRulesEngine(rules_file_path, snapshot_timestamp=None)[source]

Bases: google.cloud.forseti.scanner.audit.base_rules_engine.BaseRulesEngine

Rules engine for Big Query data sets

add_rules(rules)[source]

Add rules to the rule book.

Parameters:rules (dict) – rule definitions dictionary
build_rule_book(global_configs=None)[source]

Build BigqueryRuleBook from the rules definition file.

Parameters:global_configs (dict) – Global configurations.
find_violations(parent_project, bq_acl, force_rebuild=False)[source]

Determine whether Big Query datasets violate rules.

Parameters:
  • parent_project (Project) – parent project the acl belongs to.
  • bq_acl (BigqueryAccessControls) – Object containing ACL data.
  • force_rebuild (bool) – If True, rebuilds the rule book. This will reload the rules definition file and add the rules to the book.
Returns:

A generator of rule violations.

Return type:

generator

class Binding(role, members)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, role, members)

Create new instance of Binding(role, members)

__repr__()

Return a nicely formatted representation string

_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('role', 'members')
classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new Binding object from a sequence or iterable

_replace(**kwds)

Return a new Binding object replacing specified fields with new values

_source = "from builtins import property as _property, tuple as _tuple\nfrom operator import itemgetter as _itemgetter\nfrom collections import OrderedDict\n\nclass Binding(tuple):\n 'Binding(role, members)'\n\n __slots__ = ()\n\n _fields = ('role', 'members')\n\n def __new__(_cls, role, members):\n 'Create new instance of Binding(role, members)'\n return _tuple.__new__(_cls, (role, members))\n\n @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new Binding object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 2:\n raise TypeError('Expected 2 arguments, got %d' % len(result))\n return result\n\n def _replace(_self, **kwds):\n 'Return a new Binding object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('role', 'members'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % list(kwds))\n return result\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + '(role=%r, members=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values.'\n return OrderedDict(zip(self._fields, self))\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n role = _property(_itemgetter(0), doc='Alias for field number 0')\n\n members = _property(_itemgetter(1), doc='Alias for field number 1')\n\n"
members
role
class Member(domain, group_email, user_email, special_group)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, domain, group_email, user_email, special_group)

Create new instance of Member(domain, group_email, user_email, special_group)

__repr__()

Return a nicely formatted representation string

_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('domain', 'group_email', 'user_email', 'special_group')
classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new Member object from a sequence or iterable

_replace(**kwds)

Return a new Member object replacing specified fields with new values

_source = "from builtins import property as _property, tuple as _tuple\nfrom operator import itemgetter as _itemgetter\nfrom collections import OrderedDict\n\nclass Member(tuple):\n 'Member(domain, group_email, user_email, special_group)'\n\n __slots__ = ()\n\n _fields = ('domain', 'group_email', 'user_email', 'special_group')\n\n def __new__(_cls, domain, group_email, user_email, special_group):\n 'Create new instance of Member(domain, group_email, user_email, special_group)'\n return _tuple.__new__(_cls, (domain, group_email, user_email, special_group))\n\n @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new Member object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 4:\n raise TypeError('Expected 4 arguments, got %d' % len(result))\n return result\n\n def _replace(_self, **kwds):\n 'Return a new Member object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('domain', 'group_email', 'user_email', 'special_group'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % list(kwds))\n return result\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + '(domain=%r, group_email=%r, user_email=%r, special_group=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values.'\n return OrderedDict(zip(self._fields, self))\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n domain = _property(_itemgetter(0), doc='Alias for field number 0')\n\n group_email = _property(_itemgetter(1), doc='Alias for field number 1')\n\n user_email = _property(_itemgetter(2), doc='Alias for field number 2')\n\n special_group = _property(_itemgetter(3), doc='Alias for field number 3')\n\n"
domain
group_email
special_group
user_email
class Mode[source]

Bases: enum.Enum

Rule modes.

BLACKLIST = 'blacklist'
WHITELIST = 'whitelist'
class Rule(rule_name, rule_index, rule_reference)[source]

Bases: object

Rule properties from the rule definition file. Also finds violations.

class RuleViolation(group_email, rule_name, resource_name, user_email, view, rule_index, dataset_id, full_name, role, special_group, resource_id, resource_type, violation_type, resource_data, domain)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, group_email, rule_name, resource_name, user_email, view, rule_index, dataset_id, full_name, role, special_group, resource_id, resource_type, violation_type, resource_data, domain)

Create new instance of RuleViolation(group_email, rule_name, resource_name, user_email, view, rule_index, dataset_id, full_name, role, special_group, resource_id, resource_type, violation_type, resource_data, domain)

__repr__()

Return a nicely formatted representation string

_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('group_email', 'rule_name', 'resource_name', 'user_email', 'view', 'rule_index', 'dataset_id', 'full_name', 'role', 'special_group', 'resource_id', 'resource_type', 'violation_type', 'resource_data', 'domain')
classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new RuleViolation object from a sequence or iterable

_replace(**kwds)

Return a new RuleViolation object replacing specified fields with new values

_source = "from builtins import property as _property, tuple as _tuple\nfrom operator import itemgetter as _itemgetter\nfrom collections import OrderedDict\n\nclass RuleViolation(tuple):\n 'RuleViolation(group_email, rule_name, resource_name, user_email, view, rule_index, dataset_id, full_name, role, special_group, resource_id, resource_type, violation_type, resource_data, domain)'\n\n __slots__ = ()\n\n _fields = ('group_email', 'rule_name', 'resource_name', 'user_email', 'view', 'rule_index', 'dataset_id', 'full_name', 'role', 'special_group', 'resource_id', 'resource_type', 'violation_type', 'resource_data', 'domain')\n\n def __new__(_cls, group_email, rule_name, resource_name, user_email, view, rule_index, dataset_id, full_name, role, special_group, resource_id, resource_type, violation_type, resource_data, domain):\n 'Create new instance of RuleViolation(group_email, rule_name, resource_name, user_email, view, rule_index, dataset_id, full_name, role, special_group, resource_id, resource_type, violation_type, resource_data, domain)'\n return _tuple.__new__(_cls, (group_email, rule_name, resource_name, user_email, view, rule_index, dataset_id, full_name, role, special_group, resource_id, resource_type, violation_type, resource_data, domain))\n\n @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new RuleViolation object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 15:\n raise TypeError('Expected 15 arguments, got %d' % len(result))\n return result\n\n def _replace(_self, **kwds):\n 'Return a new RuleViolation object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('group_email', 'rule_name', 'resource_name', 'user_email', 'view', 'rule_index', 'dataset_id', 'full_name', 'role', 'special_group', 'resource_id', 'resource_type', 'violation_type', 'resource_data', 'domain'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % list(kwds))\n return result\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + '(group_email=%r, rule_name=%r, resource_name=%r, user_email=%r, view=%r, rule_index=%r, dataset_id=%r, full_name=%r, role=%r, special_group=%r, resource_id=%r, resource_type=%r, violation_type=%r, resource_data=%r, domain=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values.'\n return OrderedDict(zip(self._fields, self))\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n group_email = _property(_itemgetter(0), doc='Alias for field number 0')\n\n rule_name = _property(_itemgetter(1), doc='Alias for field number 1')\n\n resource_name = _property(_itemgetter(2), doc='Alias for field number 2')\n\n user_email = _property(_itemgetter(3), doc='Alias for field number 3')\n\n view = _property(_itemgetter(4), doc='Alias for field number 4')\n\n rule_index = _property(_itemgetter(5), doc='Alias for field number 5')\n\n dataset_id = _property(_itemgetter(6), doc='Alias for field number 6')\n\n full_name = _property(_itemgetter(7), doc='Alias for field number 7')\n\n role = _property(_itemgetter(8), doc='Alias for field number 8')\n\n special_group = _property(_itemgetter(9), doc='Alias for field number 9')\n\n resource_id = _property(_itemgetter(10), doc='Alias for field number 10')\n\n resource_type = _property(_itemgetter(11), doc='Alias for field number 11')\n\n violation_type = _property(_itemgetter(12), doc='Alias for field number 12')\n\n resource_data = _property(_itemgetter(13), doc='Alias for field number 13')\n\n domain = _property(_itemgetter(14), doc='Alias for field number 14')\n\n"
dataset_id
domain
full_name
group_email
resource_data
resource_id
resource_name
resource_type
role
rule_index
rule_name
special_group
user_email
view
violation_type
_is_binding_applicable(binding, bigquery_acl)[source]

Determine whether the binding is applicable to the acl.

Parameters:
Returns:

True if the rules are applicable to the given acl, False

otherwise.

Return type:

bool

find_violations(bigquery_acl)[source]

Find BigQuery acl violations in the rule book.

Parameters:bigquery_acl (BigqueryAccessControls) – BigQuery ACL resource.
Yields:namedtuple – Returns RuleViolation named tuple.
frozen_rule_attributes = frozenset({'group_email', 'rule_name', 'resource_name', 'user_email', 'view', 'rule_index', 'dataset_id', 'full_name', 'role', 'special_group', 'resource_id', 'resource_type', 'violation_type', 'resource_data', 'domain'})
rule_violation_attributes = ['resource_type', 'resource_id', 'resource_name', 'full_name', 'rule_name', 'rule_index', 'violation_type', 'dataset_id', 'role', 'special_group', 'user_email', 'domain', 'group_email', 'view', 'resource_data']
class RuleReference(mode, dataset_ids, bindings)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, mode, dataset_ids, bindings)

Create new instance of RuleReference(mode, dataset_ids, bindings)

__repr__()

Return a nicely formatted representation string

_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('mode', 'dataset_ids', 'bindings')
classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new RuleReference object from a sequence or iterable

_replace(**kwds)

Return a new RuleReference object replacing specified fields with new values

_source = "from builtins import property as _property, tuple as _tuple\nfrom operator import itemgetter as _itemgetter\nfrom collections import OrderedDict\n\nclass RuleReference(tuple):\n 'RuleReference(mode, dataset_ids, bindings)'\n\n __slots__ = ()\n\n _fields = ('mode', 'dataset_ids', 'bindings')\n\n def __new__(_cls, mode, dataset_ids, bindings):\n 'Create new instance of RuleReference(mode, dataset_ids, bindings)'\n return _tuple.__new__(_cls, (mode, dataset_ids, bindings))\n\n @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new RuleReference object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 3:\n raise TypeError('Expected 3 arguments, got %d' % len(result))\n return result\n\n def _replace(_self, **kwds):\n 'Return a new RuleReference object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('mode', 'dataset_ids', 'bindings'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % list(kwds))\n return result\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + '(mode=%r, dataset_ids=%r, bindings=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values.'\n return OrderedDict(zip(self._fields, self))\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n mode = _property(_itemgetter(0), doc='Alias for field number 0')\n\n dataset_ids = _property(_itemgetter(1), doc='Alias for field number 1')\n\n bindings = _property(_itemgetter(2), doc='Alias for field number 2')\n\n"
bindings
dataset_ids
mode