Lazy Man’s Record Normalizer

normalazy is an experimental, simple Python library which defines a DSL for mapping and normalizing records to a standardized format.

This documentation is quite poor at the moment, but the API is well documented and can be found below.

Installation

From PyPI (The Python Package Index):

pip install normalazy

or from GitHub:

pip install git+https://github.com/vst/normalazy.git

or from GitHub (the development version):

pip install git+https://github.com/vst/normalazy.git@develop

API Documentation

class normalazy.ChoiceKeyField(*args, **kwargs)

Defines a choice mapper for the index of the record provided.

>>> field = ChoiceKeyField(key="a", choices=dict(a=1, b=2))
>>> field.map(None, dict(a="a")).value
1
>>> field = ChoiceKeyField(key="a", choices=dict(a=1, b=2), func=lambda i, r, v: Decimal(str(v)))
>>> field.map(None, dict(a="a")).value
Decimal('1')

class normalazy.Field(name=None, func=None, blank=True, null=True)

Provides a concrete mapper field.

>>> field = Field()
>>> field.map(None, dict()).value
>>> field.map(None, dict()).status == Value.Status.Success
True
>>> field = Field(null=False)
>>> field.map(None, dict()).value
>>> field.map(None, dict()).status == Value.Status.Error
True
>>> field = Field(func=lambda i, r: r.get("a", None))
>>> field.map(None, dict(a="")).value
''
>>> field.map(None, dict(a="")).status == Value.Status.Success
True
>>> field = Field(func=lambda i, r: r.get("a", None), blank=False)
>>> field.map(None, dict(a="")).value
''
>>> field.map(None, dict(a="")).status == Value.Status.Error
True
>>> field = Field(func=lambda i, r: r.get("a", None))
>>> field.map(None, dict()).value
>>> field.map(None, dict(a=1)).value
1
>>> field.map(None, dict(a=1)).status == Value.Status.Success
True

blank

Indicates if the value is allowed to be blank.

Returns:Boolean indicating if the value is allowed to be blank.

func

Returns the mapping function of the field.

Returns:The mapping function of the field.

map(instance, record)

Returns the value of for field as a Value instance.

Parameters:

• instance – The instance for which the value will be retrieved.
• record – The raw record.

Returns:

A Value instance.

name

Returns the name of the field.

Returns:The name of the field.

null

Indicates if the value is allowed to be null.

Returns:Boolean indicating if the value is allowed to be null.

rename(name)

Renames the field.

Parameters:name – The new name of the field.

treat_value(value)

Treats the value and return.

Parameters:value – The value to be treated. Returns:A Value instance.

class normalazy.KeyField(key=None, cast=None, **kwargs)

Provides a mapper field for a given key which belongs to the record. The record can be an object which has __getitem__ method or a simple object just with attribute access.

The method starts reading the source value using the key provided checking __getitem__ method (for iterables such as dict or list), then checks the attribute for simple object attribute access.

>>> field = KeyField(key="a")
>>> field.map(None, dict(a="")).value
''
>>> field.map(None, dict(a="")).status == Value.Status.Success
True
>>> field = KeyField(key="a", blank=False)
>>> field.map(None, dict(a="")).value
''
>>> field.map(None, dict(a="")).status == Value.Status.Error
True
>>> field = KeyField(key="a", func=lambda i, r, v: as_number(v))
>>> field.map(None, dict(a="12")).value
Decimal('12')
>>> field.map(None, dict(a="12")).status == Value.Status.Success
True
>>> field = KeyField(key="a", cast=as_number)
>>> field.map(None, dict(a="12")).value
Decimal('12')
>>> field.map(None, dict(a="12")).status == Value.Status.Success
True
>>> class Student:
...     def __init__(self, name):
...         self.name = name
>>> field = KeyField(key="name")
>>> field.map(None, Student("Sinan")).value
'Sinan'

key

Returns the key of for the field mapping.

map(instance, record)

Returns the value of for field as a Value instance.

Parameters:

• instance – The instance for which the value will be retrieved.
• record – The raw record.

Returns:

A Value instance.

rename(name)

Renames the field.

Parameters:name – The new name of the field.

class normalazy.Record(record)

Provides a record normalizer base class.

>>> class Test1Record(Record):
...     a = KeyField()
>>> record1 = Test1Record(dict(a=1))
>>> record1.a
1
>>> class Test2Record(Record):
...     a = KeyField()
...     b = ChoiceKeyField(choices={1: "Bir", 2: "Iki"})
>>> record2 = Test2Record(dict(a=1, b=2))
>>> record2.a
1
>>> record2.b
'Iki'

We can get the dictionary representation of records:

>>> record1.as_dict()
OrderedDict([('a', 1)])

>>> record2.as_dict()
OrderedDict([('a', 1), ('b', 'Iki')])

Or detailed:

>>> record1.as_dict(detailed=True)
OrderedDict([('a', OrderedDict([('value', '1'), ('status', 1), ('message', None)]))])

>>> record2.as_dict(detailed=True)
OrderedDict([('a', OrderedDict([('value', '1'), ('status', 1), ('message', None)])), ('b', OrderedDict([('value', 'Iki'), ('status', 1), ('message', None)]))])

We can also create a new record from an existing record or dictionary:

>>> class Test3Record(Record):
...     a = KeyField()
...     b = KeyField()
>>> record3 = Test3Record.new(record2)
>>> record3.a
1
>>> record3.b
'Iki'
>>> record3.a == record2.a
True
>>> record3.b == record2.b
True

With dictionary:

>>> record4 = Test3Record.new({"a": 1, "b": "Iki"})
>>> record4.a
1
>>> record4.b
'Iki'
>>> record4.a == record2.a
True
>>> record4.b == record2.b
True

Or even override some fields:

>>> record5 = Test3Record.new(record3, b="Bir")
>>> record5.a
1
>>> record5.b
'Bir'

allvals()

Returns all the value slots.

Returns:A dictionary of all computed value slots.

as_dict(detailed=False)

Provides a JSON representation of the record instance.

Parameters:detailed – Indicates if we need detailed result, ie. with status and message for each field. Returns:A JSON representation of the record instance.

delval(name)

Deletes a stored value.

Parameters:name – The name of the value.

getval(name)

Returns the value slot identified by the name.

Parameters:name – The name of the value slot. Returns:The value slot, ie. the boxed value instance of class Value.

hasval(name)

Indicates if we have a value slot called name.

Parameters:name – The name of the value slot. Returns:True if we have a value slot called name, False otherwise.

classmethod new(record, **kwargs)

Creates a new record from the provided record or dictionary and overriding values from the provided additional named arguments.

Parameters:

• record – The record or dictionary to be copied from.
• kwargs – Named arguments to override.

Returns:

New record.

setval(name, value, status=None, message=None, **kwargs)

Sets a value to the value slot.

Parameters:

• name – The name of the value slot.
• value – The value to be set (Either a Python value or a Value instance.)
• status – The status of the value slot if any.
• message – The message of the value slot if any.
• kwargs – Additional named values as payload to value.

Returns:

The Value instance set.

val_blank(name)

Indicates if the value is blank.

Parameters:name – The name of the value slot. Returns:Boolean indicating if the value is blank.

val_error(name)

Indicates if the value is error.

Parameters:name – The name of the value slot. Returns:Boolean indicating if the value is error.

val_none(name)

Indicates if the value is None.

Parameters:name – The name of the value slot. Returns:Boolean indicating if the value is None.

val_some(name)

Indicates if the value is something other than None or blank.

Parameters:name – The name of the value slot. Returns:Boolean indicating if the value is something other than None or blank.

val_success(name)

Indicates if the value is success.

Parameters:name – The name of the value slot. Returns:Boolean indicating if the value is success.

val_warning(name)

Indicates if the value is warning.

Parameters:name – The name of the value slot. Returns:Boolean indicating if the value is warning.

class normalazy.RecordMetaclass

Provides a record metaclass.

class normalazy.Value(value=None, message=None, status=None, **kwargs)

Defines an immutable [sic.] boxed value with message, status and extra data as payload if required.

>>> value = Value(value=42, message=None, status=Value.Status.Success, extras="41 + 1")
>>> value.value
42
>>> value.message
>>> value.status == Value.Status.Success
True
>>> value.extras
'41 + 1'
>>> value = Value.success(42, date="2015-01-01")
>>> value.value
42
>>> value.status == Value.Status.Success
True
>>> value.date
'2015-01-01'
>>> value = Value.warning(value="fortytwo", message="Failed to convert to integer.", date="2015-01-01")
>>> value.value
'fortytwo'
>>> value.status == Value.Status.Warning
True
>>> value.date
'2015-01-01'
>>> value.message
'Failed to convert to integer.'
>>> value = Value.error(message="Failed to compute the value.", date="2015-01-01")
>>> value.value
>>> value.status == Value.Status.Error
True
>>> value.date
'2015-01-01'
>>> value.message
'Failed to compute the value.'

class Status

Defines an enumeration for value status.

Error = 3

Indicates that value could not be mapped successfully.

Success = 1

Indicates that value is mapped successfully.

Warning = 2

Indicates that value is mapped successfully with warnings.

classmethod Value.error(value=None, message=None, **kwargs)

Provides a convenience constructor for Values instances with errors.

Parameters:

• value – The value of the Value instance to be constructed.
• message – The message, if any.
• kwargs – Extra payload for the value.

Returns:

A Value instance with errors.

classmethod Value.success(value=None, message=None, **kwargs)

Provides a convenience constructor for successful Value instances.

Parameters:

• value – The value of the Value instance to be constructed.
• message – The message, if any.
• kwargs – Extra payload for the value.

Returns:

A successful Value instance.

classmethod Value.warning(value=None, message=None, **kwargs)

Provides a convenience constructor for Values instances with warnings.

Parameters:

• value – The value of the Value instance to be constructed.
• message – The message, if any.
• kwargs – Extra payload for the value.

Returns:

A Value instance with warnings.

normalazy.as_boolean(x, predicate=None)

Converts the value to a boolean value.

Parameters:

• x – The value to be converted to a boolean value.
• predicate – The predicate function if required.

Returns:

Boolean

>>> as_boolean(None)
False
>>> as_boolean("")
False
>>> as_boolean(" ")
True
>>> as_boolean(1)
True
>>> as_boolean(0)
False
>>> as_boolean("1")
True
>>> as_boolean("0")
True
>>> as_boolean("1", predicate=lambda x: int(x) != 0)
True
>>> as_boolean("0", predicate=lambda x: int(x) != 0)
False
>>> as_boolean("1", predicate=int)
True
>>> as_boolean("0", predicate=int)
False
>>> as_boolean("1", int)
True
>>> as_boolean("0", int)
False

normalazy.as_date(x, fmt=None)

Converts the value to a date value.

Parameters:

• x – The value to be converted to a date value.
• fmt – The format of the date string.

Returns:

A datetime.date instance.

>>> as_date(None)
>>> as_date('')
''
>>> as_date("2015-01-01")
datetime.date(2015, 1, 1)
>>> as_date("Date: 2015-01-01", "Date: %Y-%m-%d")
datetime.date(2015, 1, 1)
>>> as_date("Date: 2015-01-01", fmt="Date: %Y-%m-%d")
datetime.date(2015, 1, 1)

normalazy.as_datetime(x, fmt=None)

Converts the value to a datetime value.

Parameters:

• x – The value to be converted to a datetime value.
• fmt – The format of the date/time string.

Returns:

A datetime.date instance.

>>> as_datetime(None)
>>> as_datetime("")
''
>>> as_datetime("2015-01-01 00:00:00")
datetime.datetime(2015, 1, 1, 0, 0)
>>> as_datetime("2015-01-01T00:00:00", "%Y-%m-%dT%H:%M:%S")
datetime.datetime(2015, 1, 1, 0, 0)
>>> as_datetime("2015-01-01T00:00:00", fmt="%Y-%m-%dT%H:%M:%S")
datetime.datetime(2015, 1, 1, 0, 0)

normalazy.as_factor(x)

Converts the value to a factor string.

Parameters:x – Value. Returns:Trimmed, up-cased string value.

>>> as_factor(None)
>>> as_factor("")
''
>>> as_factor("a")
'A'
>>> as_factor(" a ")
'A'

normalazy.as_number(x)

Converts the value to a decimal value.

Parameters:x – The value to be converted to a decimal value. Returns:A Decimal instance.

>>> as_number(None)
>>> as_number(1)
Decimal('1')
>>> as_number("1")
Decimal('1')
>>> as_number(" 1 ")
Decimal('1')

normalazy.as_string(x)

Converts the value to a trimmed string.

Parameters:x – Value. Returns:Trimmed string value.

>>> as_string(None)
>>> as_string("")
''
>>> as_string("a")
'a'
>>> as_string(" a ")
'a'

normalazy.identity(x)

Defines an identity function.

Parameters:x – value Returns:value

>>> identity(None)
>>> identity(1)
1

normalazy.iffnotblank(func)

Wraps a function, returns None if the first argument is empty, invokes the method otherwise.

Parameters:func – The function to be wrapped. Returns:Empty string or the result of the function.

>>> test1 = iffnotblank(lambda x: x)
>>> test1("")
''
>>> test1(1)
1

normalazy.iffnotnull(func)

Wraps a function, returns None if the first argument is None, invokes the method otherwise.

Parameters:func – The function to be wrapped. Returns:None or the result of the function.

>>> test1 = iffnotnull(lambda x: x)
>>> test1(None)
>>> test1(1)
1

Indices and tables

• Index
• Module Index
• Search Page