Usage Examples
Here basic usage of the mt_metadata module are demonstrated.
Base Class
mt_metadata.base.Base is the base for which all metadata objects are built upon. Base provides convenience filters to input and output metadata in different formats XML, JSON, Python dictionary, Pandas Series. It also provides functions to help the user understand what’s inside.
[1]:
from mt_metadata.base import Base
[2]:
b = Base()
Methods of Base
Methods of Base include to/from_[json, dict, series, xml] which allows the user to input and output data in various standard formats including
JSON
XML
standard Python dictonary
Pandas.Series
Also included are methods to get information about the standards and attributes included in the metadata object, and the ability to add new attributes. Finally, methods to get and set an attribute by a compound name like location.declination.value.
[3]:
print("\n\t".join(["Methods:"] + [func for func in dir(b) if callable(getattr(b, func)) and not func.startswith("_")]))
Methods:
add_base_attribute
attribute_information
copy
from_dict
from_json
from_series
from_xml
get_attr_from_name
get_attribute_list
set_attr_from_name
to_dict
to_json
to_series
to_xml
update
Add attributes
You can add attibutes to an existing metadata object. All you need is to add a standards dictionary that describes the new attribute.
Here we will add an extra attribute for temperature. We will allow it to only have two options ‘ambient’ or ‘air’. It will be a string but is not required.
[4]:
extra = {
'type': str,
'style': 'controlled vocabulary',
'required': False,
'units': 'celsius',
'description': 'local temperature',
'alias': ['temp'],
'options': [ 'ambient', 'air'],
'example': 'ambient',
'default': None
}
[5]:
b.add_base_attribute("temperature", "ambient", extra)
The __repr__
The base class __repr__ is represented by the JSON representation of the object.
[6]:
b
[6]:
{
"base": {
"temperature": "ambient"
}
}
The __str__
The __str__ of the class is a printed list
[7]:
print(b)
base:
temperature = ambient
Attribute Information and List
There is also a convenience method to get attribute information.
[8]:
b.get_attribute_list()
[8]:
['temperature']
[9]:
b.attribute_information()
temperature:
alias: ['temp']
default: None
description: local temperature
example: ambient
options: ['ambient', 'air']
required: False
style: controlled vocabulary
type: <class 'str'>
units: celsius
==================================================
[10]:
b.attribute_information("temperature")
temperature:
alias: ['temp']
default: None
description: local temperature
example: ambient
options: ['ambient', 'air']
required: False
style: controlled vocabulary
type: <class 'str'>
units: celsius
Validation
Validation of the attribute is the most important part of having a separate module for the metadata. The validation processes
First assures the
typeis the correct type prescribed by the metadata. For example in the above example the prescribed data type fortemperatureis astring. Therefore when the value is set, the validators make sure the value is a string. If it is not it is converted to a string if possible. If not aValueErroris thrown.If the
styleiscontrolled vocabularythen the value is checked againstoptions. Ifotheris in options that allows other options to be input that are not in the list, kind of a accept anything key.If a value of None is given the proper None type is set. If the
styleis a date then the None value for is set to 1980-01-01T00:00:00, or iflistinstylethe value is set to [].
When the standards are first read in if required is True the value is set to the given default value. If required is False the value is set to the appropriate None value.
[11]:
extra = {
'type': float,
'style': 'number',
'required': True,
'units': None,
'description': 'height',
'alias': [],
'options': [],
'example': 10.0,
'default': 0.0
}
b.add_base_attribute("height", 0, extra)
[12]:
b.height = "11.7"
print(b)
base:
height = 11.7
temperature = ambient
[13]:
b.temperature = "fail"
2023-09-27T15:51:24.997266-0700 | ERROR | mt_metadata.base.metadata | __setattr__ | fail not found in options list ['ambient', 'air']
---------------------------------------------------------------------------
MTSchemaError Traceback (most recent call last)
~\AppData\Local\Temp\1\ipykernel_10084\1125618368.py in <cell line: 1>()
----> 1 b.temperature = "fail"
~\OneDrive - DOI\Documents\GitHub\mt_metadata\mt_metadata\base\metadata.py in __setattr__(self, name, value)
388 if not accept:
389 self.logger.error(msg.format(value, options))
--> 390 raise MTSchemaError(msg.format(value, options))
391 if other and not accept:
392 self.logger.warning(msg.format(value, options, name))
MTSchemaError: fail not found in options list ['ambient', 'air']
A more complicated example
We will look at a more complicated metadata object mt_metadata.timeseries.Location
[14]:
from mt_metadata.timeseries import Location
[15]:
here = Location()
here.get_attribute_list()
[15]:
['datum',
'declination.comments',
'declination.epoch',
'declination.model',
'declination.value',
'elevation',
'elevation_uncertainty',
'latitude',
'latitude_uncertainty',
'longitude',
'longitude_uncertainty',
'x',
'x2',
'x_uncertainty',
'y',
'y2',
'y_uncertainty',
'z',
'z2',
'z_uncertainty']
[16]:
here.attribute_information()
latitude:
alias: ['lat']
default: 0.0
description: latitude of location in datum specified at survey level
example: 23.134
options: []
required: True
style: number
type: float
units: degrees
==================================================
longitude:
alias: ['lon', 'long']
default: 0.0
description: longitude of location in datum specified at survey level
example: 14.23
options: []
required: True
style: number
type: float
units: degrees
==================================================
elevation:
alias: ['elev']
default: 0.0
description: elevation of location in datum specified at survey level
example: 123.4
options: []
required: True
style: number
type: float
units: meters
==================================================
latitude_uncertainty:
alias: []
default: None
description: uncertainty in latitude estimation in degrees
example: 0.01
options: []
required: False
style: number
type: float
units: degrees
==================================================
longitude_uncertainty:
alias: []
default: None
description: uncertainty in longitude estimation in degrees
example: 0.01
options: []
required: False
style: number
type: float
units: degrees
==================================================
elevation_uncertainty:
alias: []
default: None
description: uncertainty in elevation estimation
example: 0.01
options: []
required: False
style: number
type: float
units: meters
==================================================
datum:
alias: []
default: None
description: Datum of the location values. Usually a well known datum like WGS84.
example: WGS84
options: ['WGS84', 'NAD83', 'other']
required: False
style: controlled vocabulary
type: string
units: None
==================================================
x:
alias: ['east', 'easting']
default: None
description: relative distance to the center of the station
example: 10.0
options: []
required: False
style: number
type: float
units: meters
==================================================
x2:
alias: ['east', 'easting']
default: None
description: relative distance to the center of the station
example: 10.0
options: []
required: False
style: number
type: float
units: meters
==================================================
y:
alias: ['north', 'northing']
default: None
description: relative distance to the center of the station
example: 10.0
options: []
required: False
style: number
type: float
units: meters
==================================================
y2:
alias: ['north', 'northing']
default: None
description: relative distance to the center of the station
example: 10.0
options: []
required: False
style: number
type: float
units: meters
==================================================
z:
alias: []
default: None
description: relative elevation to the center of the station
example: 10.0
options: []
required: False
style: number
type: float
units: meters
==================================================
z2:
alias: []
default: None
description: relative elevation to the center of the station
example: 10.0
options: []
required: False
style: number
type: float
units: meters
==================================================
x_uncertainty:
alias: []
default: None
description: uncertainty in longitude estimation in x-direction
example: 0.01
options: []
required: False
style: number
type: float
units: meters
==================================================
y_uncertainty:
alias: []
default: None
description: uncertainty in longitude estimation in y-direction
example: 0.01
options: []
required: False
style: number
type: float
units: meters
==================================================
z_uncertainty:
alias: []
default: None
description: uncertainty in longitude estimation in z-direction
example: 0.01
options: []
required: False
style: number
type: float
units: meters
==================================================
declination.comments:
alias: []
default: None
description: any comments on declination
example: estimated from WMM 2016
options: []
required: False
style: free form
type: string
units: None
==================================================
declination.model:
alias: []
default: WMM
description: geomagnetic reference model used to calculate declination
example: WMM
options: ['EMAG2', 'EMM', 'HDGM', 'IGRF', 'WMM', 'unknown', 'other']
required: True
style: controlled vocabulary
type: string
units: None
==================================================
declination.epoch:
alias: []
default: None
description: Epoch for which declination was approximated in.
example: 2020
options: []
required: False
style: free form
type: string
units: None
==================================================
declination.value:
alias: []
default: 0.0
description: declination angle relative to geographic north positive clockwise
example: 12.3
options: []
required: True
style: number
type: float
units: degrees
==================================================
These methods are convenience methods for getting/setting complicated attributes. For instance getting/setting the declination value from a single call. This is helpful when filling metadata from a file.
[17]:
here.set_attr_from_name("declination.value", 10)
print(here)
location:
declination.model = WMM
declination.value = 10.0
elevation = 0.0
latitude = 0.0
longitude = 0.0
[18]:
here.get_attr_from_name("declination.value")
[18]:
10.0
[19]:
# This is the same as
here.declination.value
[19]:
10.0
Dictionary
The basic element that the metadata can be in is a Python dictionary with key, value pairs.
[20]:
here.to_dict()
[20]:
{'location': OrderedDict([('declination.model', 'WMM'),
('declination.value', 10.0),
('elevation', 0.0),
('latitude', 0.0),
('longitude', 0.0)])}
[21]:
here.from_dict(
{
"location": {
"declination.value": -11.0,
"elevation": 759.0,
"latitude": -34.0,
"longitude": -104.0
}
}
)
print(here)
location:
declination.model = WMM
declination.value = -11.0
elevation = 759.0
latitude = -34.0
longitude = -104.0
JSON
JSON is a standard format human/machine readable and well supported in Python. There are methods to to read/write JSON files.
[22]:
# Compact form
print(here.to_json())
{
"location": {
"declination.model": "WMM",
"declination.value": -11.0,
"elevation": 759.0,
"latitude": -34.0,
"longitude": -104.0
}
}
[23]:
here.from_json('{"location": {"declination.model": "WMM", "declination.value": 10.0, "elevation": 99.0, "latitude": 40.0, "longitude": -120.0}}')
print(here)
location:
declination.model = WMM
declination.value = 10.0
elevation = 99.0
latitude = 40.0
longitude = -120.0
[24]:
# Nested form
print(here.to_json(nested=True))
{
"location": {
"declination": {
"model": "WMM",
"value": 10.0
},
"elevation": 99.0,
"latitude": 40.0,
"longitude": -120.0
}
}
[25]:
here.from_json('{"location": {"declination": {"model": "WMM", "value": -12.0}, "elevation": 199.0, "latitude": 20.0, "longitude": -110.0}}')
print(here)
location:
declination.model = WMM
declination.value = -12.0
elevation = 199.0
latitude = 20.0
longitude = -110.0
XML
XML is also a common format for metadata, though not as human readable.
[26]:
print(here.to_xml(string=True))
<?xml version="1.0" encoding="UTF-8"?>
<location>
<declination>
<model>WMM</model>
<value units="degrees">-12.0</value>
</declination>
<elevation units="meters">199.0</elevation>
<latitude units="degrees">20.0</latitude>
<longitude units="degrees">-110.0</longitude>
</location>
[27]:
from xml.etree import cElementTree as et
location = et.Element('location')
lat = et.SubElement(location, 'latitude')
lat.text = "-10"
here.from_xml(location)
print(here)
location:
declination.model = WMM
declination.value = -12.0
elevation = 199.0
latitude = -10.0
longitude = -110.0
Pandas Series
Pandas is a common data base object that is commonly used for columnar data. A series is basically like a single row in a data base.
[28]:
pd_series = here.to_series()
print(pd_series)
declination.model WMM
declination.value -12.0
elevation 199.0
latitude -10.0
longitude -110.0
dtype: object
[29]:
from pandas import Series
location_series = Series(
{
'declination.model': 'WMM',
'declination.value': -14.0,
'elevation': 399.0,
'latitude': -14.0,
'longitude': -112.0
}
)
here.from_series(location_series)
print(here)
location:
declination.model = WMM
declination.value = -14.0
elevation = 399.0
latitude = -14.0
longitude = -112.0