mt_metadata.base

Base classes for holding metadata and schema objects

MetadataBase Overview

MetadataBase is the foundational class for all metadata objects in mt_metadata, providing robust, validated data structures with extensive serialization capabilities.

Built on Pydantic

MetadataBase inherits from Pydantic’s BaseModel, leveraging the powerful benefits of Pydantic for data validation and management:

  • Automatic Type Validation: All attributes are validated against their defined types at assignment time, catching errors early

  • Data Parsing: Automatically converts and coerces input data to the correct types (e.g., strings to floats, lists to arrays)

  • IDE Support: Full autocomplete and type hints for enhanced developer experience

  • Performance: Fast validation using compiled Rust code (via pydantic-core)

  • Serialization: Built-in support for converting to/from dictionaries and JSON

Extended Functionality

MetadataBase extends Pydantic’s BaseModel with specialized features:

  • Dot-Separated Attribute Names: Set nested attributes using dot notation (e.g., survey.id = “MT001” or station.location.latitude = 45.0)

  • Default Values: Accepts default values from schemas and validates them to proper types automatically

  • Flexible I/O Methods: - to_dict() / from_dict() - Dictionary conversion - to_json() / from_json() - JSON string/file handling - to_xml() / from_xml() - XML element handling - to_series() / from_series() - Pandas Series integration

  • Attribute Introspection: - get_attribute_list() - Get all attribute names - attribute_information - Detailed metadata about each field - update_attribute() - Programmatically update attributes with validation - add_new_field() - Dynamically add new fields with validation rules

  • Standards Compliance: Integrates with metadata standards and schemas for consistent, validated magnetotelluric data interchange

This design ensures that metadata objects are always in a valid state, with type safety, comprehensive validation, and flexible data exchange formats.

Submodules

Classes

MetadataBase

Base class for all metadata objects with Pydantic validation.

BaseDict

BaseDict is a convenience class that can help the metadata dictionaries

Package Contents

class mt_metadata.base.MetadataBase(**data)

Bases: DotNotationBaseModel

Base class for all metadata objects with Pydantic validation.

MetadataBase extends DotNotationBaseModel (which inherits from Pydantic’s BaseModel) to provide automatic validation according to metadata standards. It adds functionality beyond dictionaries, supporting JSON, XML, pandas Series, and other formats for metadata interchange.

_skip_equals

Private attribute listing fields to skip in equality comparisons

Type:

list[str]

_fields

Private attribute caching field information

Type:

dict[str, Any]

Notes

  • All field assignments are validated automatically via Pydantic

  • None values are converted to appropriate defaults (empty string or 0.0)

  • Supports nested attribute access via dot notation

  • Thread-safe for read operations after initialization

model_config

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod convert_none_to_empty(values)

Convert None values to empty strings or 0.0 for numeric fields, except for fields that explicitly default to None.

classmethod validate_none_on_assignment(value, info)

Convert None values to appropriate defaults when attributes are set.

This validator runs for all fields due to ‘validate_assignment=True’ in model config. It works generically for string and numeric fields without requiring subclass-specific validators.

Parameters:

  • value (Any) – The value being assigned to the field

  • info (Any) – Pydantic validation info containing field name and metadata

Returns:

Converted value (empty string for str, 0.0 for numeric) or original value

Return type:

Any

Notes

  • For complex types, skips conversion and lets Pydantic handle validation

  • Does NOT convert None if the field explicitly has None as its default

  • Conversion rules: str -> ‘’, float/int -> 0.0

load(other)

Load metadata from various formats and populate attributes.

The other object should have the same attributes as the current object. If there are different attributes, validation may not be accurate. Consider making a new model if you need a different object structure.

Parameters:

other (MetadataBase | dict | str | pd.Series | et.Element) – Source object from which to fill attributes. Supported types: - MetadataBase: Another metadata instance - dict: Dictionary with metadata - str: JSON string representation - pd.Series: Pandas Series with metadata - et.Element: XML Element with metadata

Raises:

MTSchemaError – If the input type is not supported

Examples

>>> metadata = MetadataBase()
>>> metadata.load({"latitude": 45.0, "longitude": -120.0})
>>> metadata.load('{"latitude": 45.0}')
update(other, match=[])

Update attribute values from another like element, skipping None

Parameters:

other (MetadataBase) – other Base object from which to update attributes

copy(update=None, deep=True)

Create a copy of the current metadata object.

This is a wrapper around Pydantic’s copy method with special handling for non-copyable objects like HDF5 references. Non-copyable objects are set to None in the copied object.

Parameters:

  • update (Mapping[str, Any] | None, optional) – Values to change/add in the new model. Note: the data is not validated before creating the new model, so ensure it’s trustworthy. Default is None.

  • deep (bool, optional) – If True, create a deep copy of the object. Default is True.

Returns:

A copy of the current object with updates applied

Return type:

MetadataBase

Raises:

TypeError – If the object contains non-copyable objects and fallback fails

Notes

  • HDF5 references cannot be deep copied and will be set to None

  • If deep copy fails, falls back to dictionary-based copying

Examples

>>> original = MetadataBase(latitude=45.0)
>>> copy = original.copy(update={"latitude": 46.0})
get_all_fields()

Get all field attributes in the Metadata class. Will search recursively and return dotted keys. For instance {location.latitude: …}.

Returns:

A flattened dictionary of dotted keys of all attributes within the class.

Return type:

Dict

get_attribute_list()

return a list of the attributes

Returns:

A list of attribute names

Return type:

list[str]

attribute_information(name=None)

Print descriptive information about attributes.

If name is provided, prints information for that specific attribute. Otherwise, prints information for all attributes.

Parameters:

name (str | None, optional) – Attribute name for a specific attribute. If None, prints information for all attributes. Default is None.

Raises:

MTSchemaError – If the specified attribute name is not found

Examples

>>> metadata.attribute_information("latitude")
>>> metadata.attribute_information()  # Print all attributes
get_attr_from_name(name)

Access attribute from the given name, supporting dot notation.

The name can contain nested object references separated by dots, e.g., ‘location.latitude’ or ‘time_period.start’.

Parameters:

name (str) – Name of attribute to get, may include dots for nested attributes

Returns:

The attribute value

Return type:

Any

Raises:

  • KeyError – If the attribute is not found

  • AttributeError – If the attribute path is invalid

Examples

>>> metadata = MetadataBase(**{'location.latitude': 45.0})
>>> metadata.get_attr_from_name('location.latitude')
45.0

Notes

This is a helper function for names with ‘.’ for easier access when reading from dictionaries or other flat structures.

set_attr_from_name(name, value)

Helper function to set attribute from the given name.

The name can contain the name of an object which must be separated by a ‘.’ for e.g. {object_name}.{name} –> location.latitude

Note

this is a helper function for names with ‘.’ in the name for easier getting when reading from dictionary.

Parameters:

  • name (string) – name of attribute to get.

  • value (type is defined by the attribute name) – attribute value

Example:

>>> b = Base(**{'category.test_attr':10})
>>> b.set_attr_from_name('category.test_attr', '10')
>>> print(b.category.test_attr)
'10'
add_base_attribute()
add_new_field(name, new_field_info)

This is going to be much different from older versions of mt_metadata.

This will return a new BaseModel with the added attribute. Going to use pydantid.create_model from the exsiting attribute information and the added attribute.

Add an attribute to _attr_dict so it will be included in the output dictionary

Parameters:

  • name (str) – name of attribute

  • new_field_info (FieldInfo) – value of the new attribute

Returns:

  • BaseModel – A new BaseModel instance with the added attribute.

  • Should include

  • * annotated –> the data type [ str | int | float | bool ]

  • * required –> required in the standards [ True | False ]

  • * units –> units of the attribute, must be a string

  • * alias –> other possible names for the attribute

  • * options –> if only a few options are accepted, separated by | or – comma.b [ option_01 | option_02 | other ]. ‘other’ means other options available but not yet defined.

  • * example –> an example of the attribute

  • Example:

  • .. code-block:: python

  • from pydantic.fields import FieldInfo

  • new_field = FieldInfo( – annotated=str, default=”default_value”, required=False, description=”new field description”, alias=”new_field_alias”, json_schema_extra={“units”:”km”} )

  • existing_basemodel = MetadataBase()

  • new_basemodel = existing_basemodel.add_new_field(“new_attribute”, new_field)

  • new_basemodel_object = new_basemodel()

Return type:

pydantic.BaseModel

to_dict(nested=False, single=False, required=True)

Convert metadata to a dictionary representation.

Parameters:

  • nested (bool, optional) – If True, return a nested dictionary structure. If False, use dot-notation for nested keys. Default is False.

  • single (bool, optional) – If True, return just the metadata dictionary without the class name wrapper (meta_dict[class_name]). Default is False.

  • required (bool, optional) – If True, return only required elements and elements with non-None values. If False, include all fields. Default is True.

Returns:

Dictionary representation of the metadata

Return type:

dict[str, Any]

Notes

  • Comment objects are converted to simple strings for backward compatibility when they only contain a value (no author or custom timestamp)

  • Numpy arrays, Enums, and nested MetadataBase objects are handled specially

  • Required fields are always included even if None

Examples

>>> metadata.to_dict(nested=True, single=True)
>>> metadata.to_dict(required=False)  # Include all fields
from_dict(meta_dict, skip_none=False)

Fill attributes from a dictionary.

The dictionary can be nested or flat with dot-notation keys. If the dictionary has a single key matching the class name, it will be unwrapped automatically.

Parameters:

  • meta_dict (dict) – Dictionary with keys equal to metadata attribute names. Supports both nested dictionaries and flat dictionaries with dot-notation keys.

  • skip_none (bool, optional) – If True, skip attributes with None values. Default is False.

Raises:

MTSchemaError – If the input is not a valid dictionary

Examples

>>> metadata.from_dict({"latitude": 45.0, "longitude": -120.0})
>>> metadata.from_dict({"location": {"latitude": 45.0}})
to_json(nested=False, indent=' ' * 4, required=True)

Write a json string from a given object, taking into account other class objects contained within the given object.

Parameters:

  • indent (str) – indentation for the json string, default is 4 spaces

  • nested (bool) – make the returned json nested

  • required (bool) – return just the required elements and any elements with non-None values

Returns:

json string representation of the object

Return type:

str

from_json(json_str)

read in a json string and update attributes of an object

Parameters:

json_str (str | Path) – json string or file path to json file

from_series(pd_series)

Fill attributes from a Pandas Series.

Parameters:

pd_series (pd.Series) – Series containing metadata information. The series must be single layered with key names separated by dots for nested attributes (e.g., ‘location.latitude’).

Raises:

MTSchemaError – If the input is not a Pandas Series

Examples

>>> series = pd.Series({"latitude": 45.0, "longitude": -120.0})
>>> metadata.from_series(series)

Notes

Types are not currently enforced from the series - validation occurs via Pydantic after assignment.

to_series(required=True)

Convert attribute list to a pandas.Series

Note

this is a flattened version of the metadata

Parameters:

required (bool) – return just the required elements and any elements with non-None values

Returns:

Series containing the metadata information

Return type:

pandas.Series

to_xml(string=False, required=True)

Convert metadata to an XML representation.

Creates an XML element with type and unit information for each attribute.

Parameters:

  • string (bool, optional) – If True, return XML as a string. If False, return an XML Element. Default is False.

  • required (bool, optional) – If True, include only required elements and elements with non-None values. If False, include all elements. Default is True.

Returns:

XML Element object if string=False, otherwise XML string

Return type:

str | et.Element

Examples

>>> xml_elem = metadata.to_xml()
>>> xml_str = metadata.to_xml(string=True)
from_xml(xml_element)

Fill attributes from an XML element.

Parameters:

xml_element (et.Element) – XML element from which to fill attributes. The element structure should match the metadata schema.

Examples

>>> import xml.etree.ElementTree as et
>>> xml_str = '<metadata><latitude>45.0</latitude></metadata>'
>>> elem = et.fromstring(xml_str)
>>> metadata.from_xml(elem)

Notes

The XML element is converted to a dictionary first, then loaded via the from_dict method.

class mt_metadata.base.BaseDict(*args, **kwargs)

Bases: collections.abc.MutableMapping

BaseDict is a convenience class that can help the metadata dictionaries act like classes so you can access variables by .name or [name]

Note

If the attribute has a . in the name then you will not be able to access that attribute by class.name.name You will get an attribute error. You need to access the attribute like a dictionary class[‘name.name’]

You can add an attribute by:

>>> b = BaseDict()
>>> b.update({name: value_dict})

Or you can add a whole dictionary:

>>> b.add_dict(ATTR_DICT['run'])

All attributes have a descriptive dictionary of the form:

>>> {'type': data type, 'required': [True | False],
>>> ... 'style': 'string style', 'units': attribute units}

  • type –> the data type [ str | int | float | bool ]

  • required –> required in the standards [ True | False ]

  • style –> style of the string

  • units –> units of the attribute, must be a string

property name
add_dict(add_dict, name=None, keys=None)

Add a dictionary to. If name is input it is added to the keys of the input dictionary

Parameters:

  • add_dict (dict or MutableMapping) – dictionary to add

  • name (str, optional) – name to add to keys, by default None

Examples

>>> s_obj = Standards()
>>> run_dict = s_obj.run_dict
>>> run_dict.add_dict(s_obj.declination_dict, 'declination')
copy()
to_latex(max_entries=7, first_table_len=7)

Convert to LaTeX format

Parameters:

  • max_entries (int, optional) – Maximum number of entries, by default 7

  • first_table_len (int, optional) – Length of first table, by default 7

Returns:

DESCRIPTION

Return type:

TYPE

from_csv(csv_fn)

Read in CSV file as a dictionary

Parameters:

csv_fn (pathlib.Path or str) – csv file to read metadata standards from

Returns:

dictionary of the contents of the file

Return type:

dict

Examples

>>> run_dict = BaseDict()
>>> run_dict.from_csv(get_level_fn('run'))
to_csv(csv_fn)

write dictionary to csv file

Parameters:

csv_fn (TYPE) – DESCRIPTION

Returns:

DESCRIPTION

Return type:

TYPE

to_json(json_fn, indent=' ' * 4)

Write schema standards to json

Parameters:

  • json_fn (str or Path) – full path to json file

  • indent (str, optional) – indentation string, by default “ “ * 4

Returns:

full path to json file

Return type:

Path

from_json(json_fn)

Read schema standards from json

Parameters:

json_fn (str or Path) – full path to json file

Returns:

full path to json file

Return type:

Path