charmhelpers.core.unitdata

Intro

A simple way to store state in units. This provides a key value storage with support for versioned, transactional operation, and can calculate deltas from previous values to simplify unit logic when processing changes.

Hook Integration

There are several extant frameworks for hook execution, including

• charmhelpers.core.hookenv.Hooks
• charmhelpers.core.services.ServiceManager

The storage classes are framework agnostic, one simple integration is via the HookData contextmanager. It will record the current hook execution environment (including relation data, config data, etc.), setup a transaction and allow easy access to the changes from previously seen values. One consequence of the integration is the reservation of particular keys (‘rels’, ‘unit’, ‘env’, ‘config’, ‘charm_revisions’) for their respective values.

Here’s a fully worked integration example using hookenv.Hooks:

from charmhelper.core import hookenv, unitdata

hook_data = unitdata.HookData()
db = unitdata.kv()
hooks = hookenv.Hooks()

@hooks.hook
def config_changed():
    # Print all changes to configuration from previously seen
    # values.
    for changed, (prev, cur) in hook_data.conf.items():
        print('config changed', changed,
              'previous value', prev,
              'current value',  cur)

    # Get some unit specific bookeeping
    if not db.get('pkg_key'):
        key = urllib.urlopen('https://example.com/pkg_key').read()
        db.set('pkg_key', key)

    # Directly access all charm config as a mapping.
    conf = db.getrange('config', True)

    # Directly access all relation data as a mapping
    rels = db.getrange('rels', True)

if __name__ == '__main__':
    with hook_data():
        hook.execute()

A more basic integration is via the hook_scope context manager which simply manages transaction scope (and records hook name, and timestamp):

>>> from unitdata import kv
>>> db = kv()
>>> with db.hook_scope('install'):
...    # do work, in transactional scope.
...    db.set('x', 1)
>>> db.get('x')
1

Usage

Values are automatically json de/serialized to preserve basic typing and complex data struct capabilities (dicts, lists, ints, booleans, etc).

Individual values can be manipulated via get/set:

>>> kv.set('y', True)
>>> kv.get('y')
True

# We can set complex values (dicts, lists) as a single key.
>>> kv.set('config', {'a': 1, 'b': True'})

# Also supports returning dictionaries as a record which
# provides attribute access.
>>> config = kv.get('config', record=True)
>>> config.b
True

Groups of keys can be manipulated with update/getrange:

>>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
>>> kv.getrange('gui.', strip=True)
{'z': 1, 'y': 2}

When updating values, its very helpful to understand which values have actually changed and how have they changed. The storage provides a delta method to provide for this:

>>> data = {'debug': True, 'option': 2}
>>> delta = kv.delta(data, 'config.')
>>> delta.debug.previous
None
>>> delta.debug.current
True
>>> delta
{'debug': (None, True), 'option': (None, 2)}

Note the delta method does not persist the actual change, it needs to be explicitly saved via ‘update’ method:

>>> kv.update(data, 'config.')

Values modified in the context of a hook scope retain historical values associated to the hookname.

>>> with db.hook_scope('config-changed'):
...      db.set('x', 42)
>>> db.gethistory('x')
[(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
 (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]

class charmhelpers.core.unitdata.Delta(previous, current)

Bases: tuple

current

Alias for field number 1

previous

Alias for field number 0

class charmhelpers.core.unitdata.DeltaSet

Bases: charmhelpers.core.unitdata.Record

class charmhelpers.core.unitdata.HookData

Bases: object

Simple integration for existing hook exec frameworks.

Records all unit information, and stores deltas for processing by the hook.

Sample:

from charmhelper.core import hookenv, unitdata

changes = unitdata.HookData()
db = unitdata.kv()
hooks = hookenv.Hooks()

@hooks.hook
def config_changed():
    # View all changes to configuration
    for changed, (prev, cur) in changes.conf.items():
        print('config changed', changed,
              'previous value', prev,
              'current value',  cur)

    # Get some unit specific bookeeping
    if not db.get('pkg_key'):
        key = urllib.urlopen('https://example.com/pkg_key').read()
        db.set('pkg_key', key)

if __name__ == '__main__':
    with changes():
        hook.execute()

class charmhelpers.core.unitdata.Record

Bases: dict

class charmhelpers.core.unitdata.Storage(path=None)

Bases: object

Simple key value database for local unit state within charms.

Modifications are not persisted unless flush() is called.

To support dicts, lists, integer, floats, and booleans values are automatically json encoded/decoded.

Note: to facilitate unit testing, ‘:memory:’ can be passed as the path parameter which causes sqlite3 to only build the db in memory. This should only be used for testing purposes.

close()

debug(fh=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>)

delta(mapping, prefix)

return a delta containing values that have changed.

flush(save=True)

get(key, default=None, record=False)

gethistory(key, deserialize=False)

getrange(key_prefix, strip=False)

Get a range of keys starting with a common prefix as a mapping of keys to values.

Parameters:

• key_prefix (str) – Common prefix among all keys
• strip (bool) – Optionally strip the common prefix from the key names in the returned dict

Return dict:

A (possibly empty) dict of key-value mappings

hook_scope(name='')

Scope all future interactions to the current hook execution revision.

set(key, value)

Set a value in the database.

Parameters:

• key (str) – Key to set the value for
• value – Any JSON-serializable value to be set

unset(key)

Remove a key from the database entirely.

unsetrange(keys=None, prefix='')

Remove a range of keys starting with a common prefix, from the database entirely.

Parameters:

• keys (list) – List of keys to remove.
• prefix (str) – Optional prefix to apply to all keys in keys before removing.

update(mapping, prefix='')

Set the values of multiple keys at once.

Parameters:

• mapping (dict) – Mapping of keys to values
• prefix (str) – Optional prefix to apply to all keys in mapping before setting

charmhelpers.core.unitdata.kv()