mt_metadata.base.metadata

Created on Wed Dec 23 20:41:16 2020

copyright:

Jared Peacock (jpeacock@usgs.gov)

license:

MIT

Classes

Base

DotNotationBaseModel

Base model that supports dot notation for setting nested attributes.

MetadataBase

Base class for all metadata objects with Pydantic validation.

Module Contents

class mt_metadata.base.metadata.Base
class mt_metadata.base.metadata.DotNotationBaseModel(**data)

Bases: pydantic.BaseModel

Base model that supports dot notation for setting nested attributes.

This model extends Pydantic’s BaseModel to allow setting nested attributes using dot notation (e.g., ‘location.latitude’ or ‘time_period.start’). It automatically handles both flat and nested dictionary structures.

Parameters:

**data (Any) – Keyword arguments representing field values. Supports both flat keys and dot-notation keys for nested attributes.

Examples

>>> model = DotNotationBaseModel(**{"location.latitude": 45.0})
>>> model = DotNotationBaseModel(**{"location": {"latitude": 45.0}})
update_attribute(attr_name, attr_value)

Update a nested attribute using dot notation.

Parameters:

  • attr_name (str) – Name of the attribute to update, supports dot notation for nested attributes (e.g., ‘time_period.start’)

  • attr_value (Any) – New value for the attribute

Raises:

AttributeError – If the attribute path does not exist

Examples

>>> model.update_attribute("time_period.start", "2020-01-01")
>>> model.update_attribute("latitude", 45.0)
class mt_metadata.base.metadata.MetadataBase(**data)

Bases: DotNotationBaseModel

Base class for all metadata objects with Pydantic validation.

MetadataBase extends DotNotationBaseModel (which inherits from Pydantic’s BaseModel) to provide automatic validation according to metadata standards. It adds functionality beyond dictionaries, supporting JSON, XML, pandas Series, and other formats for metadata interchange.

_skip_equals

Private attribute listing fields to skip in equality comparisons

Type:

list[str]

_fields

Private attribute caching field information

Type:

dict[str, Any]

Notes

  • All field assignments are validated automatically via Pydantic

  • None values are converted to appropriate defaults (empty string or 0.0)

  • Supports nested attribute access via dot notation

  • Thread-safe for read operations after initialization

model_config

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod convert_none_to_empty(values)

Convert None values to empty strings or 0.0 for numeric fields, except for fields that explicitly default to None.

classmethod validate_none_on_assignment(value, info)

Convert None values to appropriate defaults when attributes are set.

This validator runs for all fields due to ‘validate_assignment=True’ in model config. It works generically for string and numeric fields without requiring subclass-specific validators.

Parameters:

  • value (Any) – The value being assigned to the field

  • info (Any) – Pydantic validation info containing field name and metadata

Returns:

Converted value (empty string for str, 0.0 for numeric) or original value

Return type:

Any

Notes

  • For complex types, skips conversion and lets Pydantic handle validation

  • Does NOT convert None if the field explicitly has None as its default

  • Conversion rules: str -> ‘’, float/int -> 0.0

load(other)

Load metadata from various formats and populate attributes.

The other object should have the same attributes as the current object. If there are different attributes, validation may not be accurate. Consider making a new model if you need a different object structure.

Parameters:

other (MetadataBase | dict | str | pd.Series | et.Element) – Source object from which to fill attributes. Supported types: - MetadataBase: Another metadata instance - dict: Dictionary with metadata - str: JSON string representation - pd.Series: Pandas Series with metadata - et.Element: XML Element with metadata

Raises:

MTSchemaError – If the input type is not supported

Examples

>>> metadata = MetadataBase()
>>> metadata.load({"latitude": 45.0, "longitude": -120.0})
>>> metadata.load('{"latitude": 45.0}')
update(other, match=[])

Update attribute values from another like element, skipping None

Parameters:

other (MetadataBase) – other Base object from which to update attributes

copy(update=None, deep=True)

Create a copy of the current metadata object.

This is a wrapper around Pydantic’s copy method with special handling for non-copyable objects like HDF5 references. Non-copyable objects are set to None in the copied object.

Parameters:

  • update (Mapping[str, Any] | None, optional) – Values to change/add in the new model. Note: the data is not validated before creating the new model, so ensure it’s trustworthy. Default is None.

  • deep (bool, optional) – If True, create a deep copy of the object. Default is True.

Returns:

A copy of the current object with updates applied

Return type:

MetadataBase

Raises:

TypeError – If the object contains non-copyable objects and fallback fails

Notes

  • HDF5 references cannot be deep copied and will be set to None

  • If deep copy fails, falls back to dictionary-based copying

Examples

>>> original = MetadataBase(latitude=45.0)
>>> copy = original.copy(update={"latitude": 46.0})
get_all_fields()

Get all field attributes in the Metadata class. Will search recursively and return dotted keys. For instance {location.latitude: …}.

Returns:

A flattened dictionary of dotted keys of all attributes within the class.

Return type:

Dict

get_attribute_list()

return a list of the attributes

Returns:

A list of attribute names

Return type:

list[str]

attribute_information(name=None)

Print descriptive information about attributes.

If name is provided, prints information for that specific attribute. Otherwise, prints information for all attributes.

Parameters:

name (str | None, optional) – Attribute name for a specific attribute. If None, prints information for all attributes. Default is None.

Raises:

MTSchemaError – If the specified attribute name is not found

Examples

>>> metadata.attribute_information("latitude")
>>> metadata.attribute_information()  # Print all attributes
get_attr_from_name(name)

Access attribute from the given name, supporting dot notation.

The name can contain nested object references separated by dots, e.g., ‘location.latitude’ or ‘time_period.start’.

Parameters:

name (str) – Name of attribute to get, may include dots for nested attributes

Returns:

The attribute value

Return type:

Any

Raises:

  • KeyError – If the attribute is not found

  • AttributeError – If the attribute path is invalid

Examples

>>> metadata = MetadataBase(**{'location.latitude': 45.0})
>>> metadata.get_attr_from_name('location.latitude')
45.0

Notes

This is a helper function for names with ‘.’ for easier access when reading from dictionaries or other flat structures.

set_attr_from_name(name, value)

Helper function to set attribute from the given name.

The name can contain the name of an object which must be separated by a ‘.’ for e.g. {object_name}.{name} –> location.latitude

Note

this is a helper function for names with ‘.’ in the name for easier getting when reading from dictionary.

Parameters:

  • name (string) – name of attribute to get.

  • value (type is defined by the attribute name) – attribute value

Example:

>>> b = Base(**{'category.test_attr':10})
>>> b.set_attr_from_name('category.test_attr', '10')
>>> print(b.category.test_attr)
'10'
add_base_attribute()
add_new_field(name, new_field_info)

This is going to be much different from older versions of mt_metadata.

This will return a new BaseModel with the added attribute. Going to use pydantid.create_model from the exsiting attribute information and the added attribute.

Add an attribute to _attr_dict so it will be included in the output dictionary

Parameters:

  • name (str) – name of attribute

  • new_field_info (FieldInfo) – value of the new attribute

Returns:

  • BaseModel – A new BaseModel instance with the added attribute.

  • Should include

  • * annotated –> the data type [ str | int | float | bool ]

  • * required –> required in the standards [ True | False ]

  • * units –> units of the attribute, must be a string

  • * alias –> other possible names for the attribute

  • * options –> if only a few options are accepted, separated by | or – comma.b [ option_01 | option_02 | other ]. ‘other’ means other options available but not yet defined.

  • * example –> an example of the attribute

  • Example:

  • .. code-block:: python

  • from pydantic.fields import FieldInfo

  • new_field = FieldInfo( – annotated=str, default=”default_value”, required=False, description=”new field description”, alias=”new_field_alias”, json_schema_extra={“units”:”km”} )

  • existing_basemodel = MetadataBase()

  • new_basemodel = existing_basemodel.add_new_field(“new_attribute”, new_field)

  • new_basemodel_object = new_basemodel()

Return type:

pydantic.BaseModel

to_dict(nested=False, single=False, required=True)

Convert metadata to a dictionary representation.

Parameters:

  • nested (bool, optional) – If True, return a nested dictionary structure. If False, use dot-notation for nested keys. Default is False.

  • single (bool, optional) – If True, return just the metadata dictionary without the class name wrapper (meta_dict[class_name]). Default is False.

  • required (bool, optional) – If True, return only required elements and elements with non-None values. If False, include all fields. Default is True.

Returns:

Dictionary representation of the metadata

Return type:

dict[str, Any]

Notes

  • Comment objects are converted to simple strings for backward compatibility when they only contain a value (no author or custom timestamp)

  • Numpy arrays, Enums, and nested MetadataBase objects are handled specially

  • Required fields are always included even if None

Examples

>>> metadata.to_dict(nested=True, single=True)
>>> metadata.to_dict(required=False)  # Include all fields
from_dict(meta_dict, skip_none=False)

Fill attributes from a dictionary.

The dictionary can be nested or flat with dot-notation keys. If the dictionary has a single key matching the class name, it will be unwrapped automatically.

Parameters:

  • meta_dict (dict) – Dictionary with keys equal to metadata attribute names. Supports both nested dictionaries and flat dictionaries with dot-notation keys.

  • skip_none (bool, optional) – If True, skip attributes with None values. Default is False.

Raises:

MTSchemaError – If the input is not a valid dictionary

Examples

>>> metadata.from_dict({"latitude": 45.0, "longitude": -120.0})
>>> metadata.from_dict({"location": {"latitude": 45.0}})
to_json(nested=False, indent=' ' * 4, required=True)

Write a json string from a given object, taking into account other class objects contained within the given object.

Parameters:

  • indent (str) – indentation for the json string, default is 4 spaces

  • nested (bool) – make the returned json nested

  • required (bool) – return just the required elements and any elements with non-None values

Returns:

json string representation of the object

Return type:

str

from_json(json_str)

read in a json string and update attributes of an object

Parameters:

json_str (str | Path) – json string or file path to json file

from_series(pd_series)

Fill attributes from a Pandas Series.

Parameters:

pd_series (pd.Series) – Series containing metadata information. The series must be single layered with key names separated by dots for nested attributes (e.g., ‘location.latitude’).

Raises:

MTSchemaError – If the input is not a Pandas Series

Examples

>>> series = pd.Series({"latitude": 45.0, "longitude": -120.0})
>>> metadata.from_series(series)

Notes

Types are not currently enforced from the series - validation occurs via Pydantic after assignment.

to_series(required=True)

Convert attribute list to a pandas.Series

Note

this is a flattened version of the metadata

Parameters:

required (bool) – return just the required elements and any elements with non-None values

Returns:

Series containing the metadata information

Return type:

pandas.Series

to_xml(string=False, required=True)

Convert metadata to an XML representation.

Creates an XML element with type and unit information for each attribute.

Parameters:

  • string (bool, optional) – If True, return XML as a string. If False, return an XML Element. Default is False.

  • required (bool, optional) – If True, include only required elements and elements with non-None values. If False, include all elements. Default is True.

Returns:

XML Element object if string=False, otherwise XML string

Return type:

str | et.Element

Examples

>>> xml_elem = metadata.to_xml()
>>> xml_str = metadata.to_xml(string=True)
from_xml(xml_element)

Fill attributes from an XML element.

Parameters:

xml_element (et.Element) – XML element from which to fill attributes. The element structure should match the metadata schema.

Examples

>>> import xml.etree.ElementTree as et
>>> xml_str = '<metadata><latitude>45.0</latitude></metadata>'
>>> elem = et.fromstring(xml_str)
>>> metadata.from_xml(elem)

Notes

The XML element is converted to a dictionary first, then loaded via the from_dict method.