mt_metadata.base ================ .. py:module:: mt_metadata.base .. autoapi-nested-parse:: Base classes for holding metadata and schema objects MetadataBase Overview ===================== MetadataBase is the foundational class for all metadata objects in mt_metadata, providing robust, validated data structures with extensive serialization capabilities. Built on Pydantic ----------------- MetadataBase inherits from Pydantic's BaseModel, leveraging the powerful benefits of Pydantic for data validation and management: * **Automatic Type Validation**: All attributes are validated against their defined types at assignment time, catching errors early * **Data Parsing**: Automatically converts and coerces input data to the correct types (e.g., strings to floats, lists to arrays) * **IDE Support**: Full autocomplete and type hints for enhanced developer experience * **Performance**: Fast validation using compiled Rust code (via pydantic-core) * **Serialization**: Built-in support for converting to/from dictionaries and JSON Extended Functionality ---------------------- MetadataBase extends Pydantic's BaseModel with specialized features: * **Dot-Separated Attribute Names**: Set nested attributes using dot notation (e.g., `survey.id = "MT001"` or `station.location.latitude = 45.0`) * **Default Values**: Accepts default values from schemas and validates them to proper types automatically * **Flexible I/O Methods**: - `to_dict()` / `from_dict()` - Dictionary conversion - `to_json()` / `from_json()` - JSON string/file handling - `to_xml()` / `from_xml()` - XML element handling - `to_series()` / `from_series()` - Pandas Series integration * **Attribute Introspection**: - `get_attribute_list()` - Get all attribute names - `attribute_information` - Detailed metadata about each field - `update_attribute()` - Programmatically update attributes with validation - `add_new_field()` - Dynamically add new fields with validation rules * **Standards Compliance**: Integrates with metadata standards and schemas for consistent, validated magnetotelluric data interchange This design ensures that metadata objects are always in a valid state, with type safety, comprehensive validation, and flexible data exchange formats. Submodules ---------- .. toctree:: :maxdepth: 1 /source/api/mt_metadata/base/helpers/index /source/api/mt_metadata/base/metadata/index /source/api/mt_metadata/base/pydantic_helpers/index /source/api/mt_metadata/base/schema/index Classes ------- .. autoapisummary:: mt_metadata.base.MetadataBase mt_metadata.base.BaseDict Package Contents ---------------- .. py:class:: MetadataBase(**data) Bases: :py:obj:`DotNotationBaseModel` Base class for all metadata objects with Pydantic validation. MetadataBase extends DotNotationBaseModel (which inherits from Pydantic's BaseModel) to provide automatic validation according to metadata standards. It adds functionality beyond dictionaries, supporting JSON, XML, pandas Series, and other formats for metadata interchange. .. attribute:: _skip_equals Private attribute listing fields to skip in equality comparisons :type: list[str] .. attribute:: _fields Private attribute caching field information :type: dict[str, Any] .. rubric:: Notes - All field assignments are validated automatically via Pydantic - None values are converted to appropriate defaults (empty string or 0.0) - Supports nested attribute access via dot notation - Thread-safe for read operations after initialization .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:method:: convert_none_to_empty(values) :classmethod: Convert None values to empty strings or 0.0 for numeric fields, except for fields that explicitly default to None. .. py:method:: validate_none_on_assignment(value, info) :classmethod: Convert None values to appropriate defaults when attributes are set. This validator runs for all fields due to 'validate_assignment=True' in model config. It works generically for string and numeric fields without requiring subclass-specific validators. :param value: The value being assigned to the field :type value: Any :param info: Pydantic validation info containing field name and metadata :type info: Any :returns: Converted value (empty string for str, 0.0 for numeric) or original value :rtype: Any .. rubric:: Notes - For complex types, skips conversion and lets Pydantic handle validation - Does NOT convert None if the field explicitly has None as its default - Conversion rules: str -> '', float/int -> 0.0 .. py:method:: load(other) Load metadata from various formats and populate attributes. The other object should have the same attributes as the current object. If there are different attributes, validation may not be accurate. Consider making a new model if you need a different object structure. :param other: Source object from which to fill attributes. Supported types: - MetadataBase: Another metadata instance - dict: Dictionary with metadata - str: JSON string representation - pd.Series: Pandas Series with metadata - et.Element: XML Element with metadata :type other: MetadataBase | dict | str | pd.Series | et.Element :raises MTSchemaError: If the input type is not supported .. rubric:: Examples >>> metadata = MetadataBase() >>> metadata.load({"latitude": 45.0, "longitude": -120.0}) >>> metadata.load('{"latitude": 45.0}') .. py:method:: update(other, match = []) Update attribute values from another like element, skipping None :param other: other Base object from which to update attributes :type other: MetadataBase .. py:method:: copy(update = None, deep = True) Create a copy of the current metadata object. This is a wrapper around Pydantic's copy method with special handling for non-copyable objects like HDF5 references. Non-copyable objects are set to None in the copied object. :param update: Values to change/add in the new model. Note: the data is not validated before creating the new model, so ensure it's trustworthy. Default is None. :type update: Mapping[str, Any] | None, optional :param deep: If True, create a deep copy of the object. Default is True. :type deep: bool, optional :returns: A copy of the current object with updates applied :rtype: MetadataBase :raises TypeError: If the object contains non-copyable objects and fallback fails .. rubric:: Notes - HDF5 references cannot be deep copied and will be set to None - If deep copy fails, falls back to dictionary-based copying .. rubric:: Examples >>> original = MetadataBase(latitude=45.0) >>> copy = original.copy(update={"latitude": 46.0}) .. py:method:: get_all_fields() Get all field attributes in the Metadata class. Will search recursively and return dotted keys. For instance `{location.latitude: ...}`. :returns: A flattened dictionary of dotted keys of all attributes within the class. :rtype: Dict .. py:method:: get_attribute_list() return a list of the attributes :returns: A list of attribute names :rtype: list[str] .. py:method:: attribute_information(name = None) Print descriptive information about attributes. If name is provided, prints information for that specific attribute. Otherwise, prints information for all attributes. :param name: Attribute name for a specific attribute. If None, prints information for all attributes. Default is None. :type name: str | None, optional :raises MTSchemaError: If the specified attribute name is not found .. rubric:: Examples >>> metadata.attribute_information("latitude") >>> metadata.attribute_information() # Print all attributes .. py:method:: get_attr_from_name(name) Access attribute from the given name, supporting dot notation. The name can contain nested object references separated by dots, e.g., 'location.latitude' or 'time_period.start'. :param name: Name of attribute to get, may include dots for nested attributes :type name: str :returns: The attribute value :rtype: Any :raises KeyError: If the attribute is not found :raises AttributeError: If the attribute path is invalid .. rubric:: Examples >>> metadata = MetadataBase(**{'location.latitude': 45.0}) >>> metadata.get_attr_from_name('location.latitude') 45.0 .. rubric:: Notes This is a helper function for names with '.' for easier access when reading from dictionaries or other flat structures. .. py:method:: set_attr_from_name(name, value) Helper function to set attribute from the given name. The name can contain the name of an object which must be separated by a '.' for e.g. {object_name}.{name} --> location.latitude .. note:: this is a helper function for names with '.' in the name for easier getting when reading from dictionary. :param name: name of attribute to get. :type name: string :param value: attribute value :type value: type is defined by the attribute name :Example: >>> b = Base(**{'category.test_attr':10}) >>> b.set_attr_from_name('category.test_attr', '10') >>> print(b.category.test_attr) '10' .. py:method:: add_base_attribute() .. py:method:: add_new_field(name, new_field_info) This is going to be much different from older versions of mt_metadata. This will return a new BaseModel with the added attribute. Going to use `pydantid.create_model` from the exsiting attribute information and the added attribute. Add an attribute to _attr_dict so it will be included in the output dictionary :param name: name of attribute :type name: str :param new_field_info: value of the new attribute :type new_field_info: FieldInfo :returns: * *BaseModel* -- A new BaseModel instance with the added attribute. * *Should include* * *\* annotated --> the data type [ str | int | float | bool ]* * *\* required --> required in the standards [ True | False ]* * *\* units --> units of the attribute, must be a string* * *\* alias --> other possible names for the attribute* * *\* options --> if only a few options are accepted, separated by | or* -- comma.b [ option_01 | option_02 | other ]. 'other' means other options available but not yet defined. * *\* example --> an example of the attribute* * *Example:* * *.. code-block:: python* * *from pydantic.fields import FieldInfo* * *new_field = FieldInfo(* -- annotated=str, default="default_value", required=False, description="new field description", alias="new_field_alias", json_schema_extra={"units":"km"} ) * *existing_basemodel = MetadataBase()* * *new_basemodel = existing_basemodel.add_new_field("new_attribute", new_field)* * *new_basemodel_object = new_basemodel()* .. py:method:: to_dict(nested = False, single = False, required = True) Convert metadata to a dictionary representation. :param nested: If True, return a nested dictionary structure. If False, use dot-notation for nested keys. Default is False. :type nested: bool, optional :param single: If True, return just the metadata dictionary without the class name wrapper (meta_dict[class_name]). Default is False. :type single: bool, optional :param required: If True, return only required elements and elements with non-None values. If False, include all fields. Default is True. :type required: bool, optional :returns: Dictionary representation of the metadata :rtype: dict[str, Any] .. rubric:: Notes - Comment objects are converted to simple strings for backward compatibility when they only contain a value (no author or custom timestamp) - Numpy arrays, Enums, and nested MetadataBase objects are handled specially - Required fields are always included even if None .. rubric:: Examples >>> metadata.to_dict(nested=True, single=True) >>> metadata.to_dict(required=False) # Include all fields .. py:method:: from_dict(meta_dict, skip_none = False) Fill attributes from a dictionary. The dictionary can be nested or flat with dot-notation keys. If the dictionary has a single key matching the class name, it will be unwrapped automatically. :param meta_dict: Dictionary with keys equal to metadata attribute names. Supports both nested dictionaries and flat dictionaries with dot-notation keys. :type meta_dict: dict :param skip_none: If True, skip attributes with None values. Default is False. :type skip_none: bool, optional :raises MTSchemaError: If the input is not a valid dictionary .. rubric:: Examples >>> metadata.from_dict({"latitude": 45.0, "longitude": -120.0}) >>> metadata.from_dict({"location": {"latitude": 45.0}}) .. py:method:: to_json(nested = False, indent = ' ' * 4, required = True) Write a json string from a given object, taking into account other class objects contained within the given object. :param indent: indentation for the json string, default is 4 spaces :type indent: str :param nested: make the returned json nested :type nested: bool :param required: return just the required elements and any elements with non-None values :type required: bool :returns: json string representation of the object :rtype: str .. py:method:: from_json(json_str) read in a json string and update attributes of an object :param json_str: json string or file path to json file :type json_str: str | Path .. py:method:: from_series(pd_series) Fill attributes from a Pandas Series. :param pd_series: Series containing metadata information. The series must be single layered with key names separated by dots for nested attributes (e.g., 'location.latitude'). :type pd_series: pd.Series :raises MTSchemaError: If the input is not a Pandas Series .. rubric:: Examples >>> series = pd.Series({"latitude": 45.0, "longitude": -120.0}) >>> metadata.from_series(series) .. rubric:: Notes Types are not currently enforced from the series - validation occurs via Pydantic after assignment. .. py:method:: to_series(required = True) Convert attribute list to a pandas.Series .. note:: this is a flattened version of the metadata :param required: return just the required elements and any elements with non-None values :type required: bool :returns: Series containing the metadata information :rtype: pandas.Series .. py:method:: to_xml(string = False, required = True) Convert metadata to an XML representation. Creates an XML element with type and unit information for each attribute. :param string: If True, return XML as a string. If False, return an XML Element. Default is False. :type string: bool, optional :param required: If True, include only required elements and elements with non-None values. If False, include all elements. Default is True. :type required: bool, optional :returns: XML Element object if string=False, otherwise XML string :rtype: str | et.Element .. rubric:: Examples >>> xml_elem = metadata.to_xml() >>> xml_str = metadata.to_xml(string=True) .. py:method:: from_xml(xml_element) Fill attributes from an XML element. :param xml_element: XML element from which to fill attributes. The element structure should match the metadata schema. :type xml_element: et.Element .. rubric:: Examples >>> import xml.etree.ElementTree as et >>> xml_str = '45.0' >>> elem = et.fromstring(xml_str) >>> metadata.from_xml(elem) .. rubric:: Notes The XML element is converted to a dictionary first, then loaded via the from_dict method. .. py:class:: BaseDict(*args, **kwargs) Bases: :py:obj:`collections.abc.MutableMapping` BaseDict is a convenience class that can help the metadata dictionaries act like classes so you can access variables by .name or [name] .. note:: If the attribute has a . in the name then you will not be able to access that attribute by class.name.name You will get an attribute error. You need to access the attribute like a dictionary class['name.name'] You can add an attribute by: >>> b = BaseDict() >>> b.update({name: value_dict}) Or you can add a whole dictionary: >>> b.add_dict(ATTR_DICT['run']) All attributes have a descriptive dictionary of the form: >>> {'type': data type, 'required': [True | False], >>> ... 'style': 'string style', 'units': attribute units} * **type** --> the data type [ str | int | float | bool ] * **required** --> required in the standards [ True | False ] * **style** --> style of the string * **units** --> units of the attribute, must be a string .. py:property:: name .. py:method:: add_dict(add_dict, name=None, keys=None) Add a dictionary to. If name is input it is added to the keys of the input dictionary :param add_dict: dictionary to add :type add_dict: dict or MutableMapping :param name: name to add to keys, by default None :type name: str, optional .. rubric:: Examples >>> s_obj = Standards() >>> run_dict = s_obj.run_dict >>> run_dict.add_dict(s_obj.declination_dict, 'declination') .. py:method:: copy() .. py:method:: to_latex(max_entries=7, first_table_len=7) Convert to LaTeX format :param max_entries: Maximum number of entries, by default 7 :type max_entries: int, optional :param first_table_len: Length of first table, by default 7 :type first_table_len: int, optional :returns: DESCRIPTION :rtype: TYPE .. py:method:: from_csv(csv_fn) Read in CSV file as a dictionary :param csv_fn: csv file to read metadata standards from :type csv_fn: pathlib.Path or str :returns: dictionary of the contents of the file :rtype: dict .. rubric:: Examples >>> run_dict = BaseDict() >>> run_dict.from_csv(get_level_fn('run')) .. py:method:: to_csv(csv_fn) write dictionary to csv file :param csv_fn: DESCRIPTION :type csv_fn: TYPE :returns: DESCRIPTION :rtype: TYPE .. py:method:: to_json(json_fn, indent=' ' * 4) Write schema standards to json :param json_fn: full path to json file :type json_fn: str or Path :param indent: indentation string, by default " " * 4 :type indent: str, optional :returns: full path to json file :rtype: Path .. py:method:: from_json(json_fn) Read schema standards from json :param json_fn: full path to json file :type json_fn: str or Path :returns: full path to json file :rtype: Path