{ "cells": [ { "cell_type": "markdown", "id": "a7212ef8-04d2-41ab-8b7e-b477891b4cdb", "metadata": {}, "source": [ "# Time Series Metadata Examples\n", "\n", "Metadata can be confusing and a daunting task. Hopefully `mt_metadata` makes it easier. Here we will demonstrate some use cases." ] }, { "cell_type": "code", "execution_count": 1, "id": "f5304d5c-e26a-49f5-bcc2-ea796f521270", "metadata": {}, "outputs": [], "source": [ "from mt_metadata.timeseries import (\n", " Auxiliary, Electric, Experiment, Magnetic, Run, Station, Survey\n", ")" ] }, { "cell_type": "markdown", "id": "918474b6-063a-4f9a-b9e9-8d0403b1abd6", "metadata": {}, "source": [ "## Time Series Metadata as a Single Object\n", "\n", "`mt_metadata` was written so that all metadata can reside under a single object. For example, if you have a single station with multiple runs that have multiple channels, you don't want to carry around a single file for each. Each level of metadata, namely `Experiment`, `Survey`, `Station`, `Run` have convenience objects to contain a collection of the next level down. For example `Experiment.surveys`, `Survey.stations` or `Station.runs` or `Run.channels`. This object is a combination of a `List` and `Dictionary`, which allows the user to access items in the collection by index or key word. Each level of metadata also has a `add_`, `get_`, and `remove_` method to add/get/remove an item from the collection. And example below. \n", "\n", "
\n", " Tip:\n", " The benefit of having the experiment in a single object is later you can populate an MTH5 with out data, just the metadata.\n", "
" ] }, { "cell_type": "code", "execution_count": 2, "id": "d19264e5-3267-49ce-a80a-1f889693bcf2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Contents:\n", "------------\n", "\t01 = survey:\n", "\tcitation_dataset.doi = None\n", "\tcitation_journal.doi = None\n", "\tdatum = WGS84\n", "\tgeographic_name = None\n", "\tid = 01\n", "\tname = None\n", "\tnorthwest_corner.latitude = 0.0\n", "\tnorthwest_corner.longitude = 0.0\n", "\tproject = None\n", "\tproject_lead.email = None\n", "\tproject_lead.organization = None\n", "\trelease_license = CC0-1.0\n", "\tsoutheast_corner.latitude = 0.0\n", "\tsoutheast_corner.longitude = 0.0\n", "\tsummary = None\n", "\ttime_period.end_date = 1980-01-01\n", "\ttime_period.start_date = 1980-01-01\n" ] } ], "source": [ "experiment = Experiment()\n", "survey_01 = Survey(id=\"01\")\n", "experiment.add_survey(survey_01)\n", "print(experiment.surveys)" ] }, { "cell_type": "code", "execution_count": 3, "id": "63d8940f-10e8-46de-8417-39eeb93aa649", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{\n", " \"survey\": {\n", " \"citation_dataset.doi\": null,\n", " \"citation_journal.doi\": null,\n", " \"datum\": \"WGS84\",\n", " \"geographic_name\": null,\n", " \"id\": \"01\",\n", " \"name\": null,\n", " \"northwest_corner.latitude\": 0.0,\n", " \"northwest_corner.longitude\": 0.0,\n", " \"project\": null,\n", " \"project_lead.email\": null,\n", " \"project_lead.organization\": null,\n", " \"release_license\": \"CC0-1.0\",\n", " \"southeast_corner.latitude\": 0.0,\n", " \"southeast_corner.longitude\": 0.0,\n", " \"summary\": null,\n", " \"time_period.end_date\": \"1980-01-01\",\n", " \"time_period.start_date\": \"1980-01-01\"\n", " }\n", "}" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "experiment.get_survey(\"01\")" ] }, { "cell_type": "code", "execution_count": 4, "id": "398f6513-c24e-435d-b808-1c02cdee6850", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{\n", " \"survey\": {\n", " \"citation_dataset.doi\": null,\n", " \"citation_journal.doi\": null,\n", " \"datum\": \"WGS84\",\n", " \"geographic_name\": null,\n", " \"id\": \"01\",\n", " \"name\": null,\n", " \"northwest_corner.latitude\": 0.0,\n", " \"northwest_corner.longitude\": 0.0,\n", " \"project\": null,\n", " \"project_lead.email\": null,\n", " \"project_lead.organization\": null,\n", " \"release_license\": \"CC0-1.0\",\n", " \"southeast_corner.latitude\": 0.0,\n", " \"southeast_corner.longitude\": 0.0,\n", " \"summary\": null,\n", " \"time_period.end_date\": \"1980-01-01\",\n", " \"time_period.start_date\": \"1980-01-01\"\n", " }\n", "}" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "experiment.surveys[0]" ] }, { "cell_type": "code", "execution_count": 5, "id": "7ee8da74-aa08-4f7b-b7c9-2282d7a94100", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{\n", " \"survey\": {\n", " \"citation_dataset.doi\": null,\n", " \"citation_journal.doi\": null,\n", " \"datum\": \"WGS84\",\n", " \"geographic_name\": null,\n", " \"id\": \"01\",\n", " \"name\": null,\n", " \"northwest_corner.latitude\": 0.0,\n", " \"northwest_corner.longitude\": 0.0,\n", " \"project\": null,\n", " \"project_lead.email\": null,\n", " \"project_lead.organization\": null,\n", " \"release_license\": \"CC0-1.0\",\n", " \"southeast_corner.latitude\": 0.0,\n", " \"southeast_corner.longitude\": 0.0,\n", " \"summary\": null,\n", " \"time_period.end_date\": \"1980-01-01\",\n", " \"time_period.start_date\": \"1980-01-01\"\n", " }\n", "}" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "experiment.surveys[\"01\"]" ] }, { "cell_type": "markdown", "id": "25a770b7-a1e2-42aa-9a42-b6a0a499f61b", "metadata": {}, "source": [ "### Full Experiment\n", "\n", "This example demonstrates how to build an experiment from scratch and how to interogate it. \n" ] }, { "cell_type": "code", "execution_count": 6, "id": "b94361ad-14c4-4c8d-8783-5d9f000f51f5", "metadata": {}, "outputs": [], "source": [ "experiment = Experiment()\n", "kwargs = {\n", " \"time_period.start\": \"2020-01-01T00:00:00+00:00\",\n", " \"time_period.end\": \"2021-01-01T12:00:00+00:00\"\n", "}\n", "\n", "for survey in [\"One\", \"Two\"]:\n", " survey_obj = Survey(id=survey)\n", " survey_obj.filters = {}\n", " for station in [\"mt01\", \"mt02\"]:\n", " station_obj = Station(id=station, **kwargs)\n", " for run in [\"mt01a\", \"mt01b\"]:\n", " run_obj = Run(id=run, **kwargs)\n", " for ch in [\"ex\", \"ey\"]:\n", " ch_obj = Electric(component=ch, **kwargs)\n", " run_obj.add_channel(ch_obj)\n", " for ch in [\"hx\", \"hy\", \"hz\"]:\n", " ch_obj = Magnetic(component=ch, **kwargs)\n", " run_obj.add_channel(ch_obj)\n", " for ch in [\"temperature\", \"voltage\"]:\n", " ch_obj = Auxiliary(component=ch, **kwargs)\n", " run_obj.add_channel(ch_obj)\n", " run_obj.update_time_period()\n", " station_obj.runs.append(run_obj)\n", " station_obj.update_time_period()\n", " survey_obj.stations.append(station_obj)\n", " survey_obj.update_time_period()\n", " experiment.surveys.append(survey_obj) " ] }, { "cell_type": "markdown", "id": "a91f5194-ac04-4197-80fe-2bc4b4f09ce3", "metadata": {}, "source": [ "#### Pick out a channel\n", "\n", "To pick out a channel you need to know the survey, station, and run. If you don't you can figure out what runs, stations, and surveys are available." ] }, { "cell_type": "code", "execution_count": 7, "id": "3785b127-ed39-4f16-b69f-ee5aaca7533f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Experiment Contents\n", "--------------------\n", "Number of Surveys: 2\n", " Survey ID: One\n", " Number of Stations: 2\n", " Number of Filters: 0\n", " --------------------\n", " Station ID: mt01\n", " Number of Runs: 2\n", " --------------------\n", " Run ID: mt01a\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------\n", " Run ID: mt01b\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------\n", " Station ID: mt02\n", " Number of Runs: 2\n", " --------------------\n", " Run ID: mt01a\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------\n", " Run ID: mt01b\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------\n", " Survey ID: Two\n", " Number of Stations: 2\n", " Number of Filters: 0\n", " --------------------\n", " Station ID: mt01\n", " Number of Runs: 2\n", " --------------------\n", " Run ID: mt01a\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------\n", " Run ID: mt01b\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------\n", " Station ID: mt02\n", " Number of Runs: 2\n", " --------------------\n", " Run ID: mt01a\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------\n", " Run ID: mt01b\n", " Number of Channels: 7\n", " Recorded Channels: ex, ey, hx, hy, hz, temperature, voltage\n", " Start: 2020-01-01T00:00:00+00:00\n", " End: 2021-01-01T12:00:00+00:00\n", " --------------------" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "experiment" ] }, { "cell_type": "markdown", "id": "f56a660b-beee-40c8-9250-ccd62a33d0b8", "metadata": {}, "source": [ "From an experiment you can access a channel knowing either the index or key word of the survey, station, and run. This is a mix of using index values and keys. " ] }, { "cell_type": "code", "execution_count": 8, "id": "c03654af-7b91-4e85-8e62-93e253739da0", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{\n", " \"electric\": {\n", " \"channel_number\": 0,\n", " \"component\": \"ex\",\n", " \"data_quality.rating.value\": 0,\n", " \"dipole_length\": 100.0,\n", " \"filter.applied\": [\n", " false\n", " ],\n", " \"filter.name\": [],\n", " \"measurement_azimuth\": 10.0,\n", " \"measurement_tilt\": 0.0,\n", " \"negative.elevation\": 0.0,\n", " \"negative.id\": null,\n", " \"negative.latitude\": 0.0,\n", " \"negative.longitude\": 0.0,\n", " \"negative.manufacturer\": null,\n", " \"negative.type\": null,\n", " \"negative.x\": -50.0,\n", " \"positive.elevation\": 0.0,\n", " \"positive.id\": null,\n", " \"positive.latitude\": 0.0,\n", " \"positive.longitude\": 0.0,\n", " \"positive.manufacturer\": null,\n", " \"positive.type\": null,\n", " \"positive.x\": 50.0,\n", " \"sample_rate\": 256.0,\n", " \"time_period.end\": \"2021-01-01T12:00:00+00:00\",\n", " \"time_period.start\": \"2020-01-01T00:00:00+00:00\",\n", " \"type\": \"electric\",\n", " \"units\": null\n", " }\n", "}" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ch_ex = experiment.surveys[0].stations[\"mt01\"].runs[0].channels[\"ex\"]\n", "# set some values\n", "ch_ex.dipole_length = 100\n", "ch_ex.measurement_azimuth = 10\n", "ch_ex.negative.x = -50\n", "ch_ex.positive.x = 50\n", "ch_ex.sample_rate = 256\n", "ch_ex" ] }, { "cell_type": "markdown", "id": "6883bf57-cdac-467a-9ec3-512a705a2a78", "metadata": {}, "source": [ "The values are updated in the experiment object. " ] }, { "cell_type": "code", "execution_count": 9, "id": "1d347e4f-cd16-4fa4-b846-5b35bcaa4639", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{\n", " \"electric\": {\n", " \"channel_number\": 0,\n", " \"component\": \"ex\",\n", " \"data_quality.rating.value\": 0,\n", " \"dipole_length\": 100.0,\n", " \"filter.applied\": [\n", " false\n", " ],\n", " \"filter.name\": [],\n", " \"measurement_azimuth\": 10.0,\n", " \"measurement_tilt\": 0.0,\n", " \"negative.elevation\": 0.0,\n", " \"negative.id\": null,\n", " \"negative.latitude\": 0.0,\n", " \"negative.longitude\": 0.0,\n", " \"negative.manufacturer\": null,\n", " \"negative.type\": null,\n", " \"negative.x\": -50.0,\n", " \"positive.elevation\": 0.0,\n", " \"positive.id\": null,\n", " \"positive.latitude\": 0.0,\n", " \"positive.longitude\": 0.0,\n", " \"positive.manufacturer\": null,\n", " \"positive.type\": null,\n", " \"positive.x\": 50.0,\n", " \"sample_rate\": 256.0,\n", " \"time_period.end\": \"2021-01-01T12:00:00+00:00\",\n", " \"time_period.start\": \"2020-01-01T00:00:00+00:00\",\n", " \"type\": \"electric\",\n", " \"units\": null\n", " }\n", "}" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "experiment.surveys[0].stations[\"mt01\"].runs[0].channels[\"ex\"]" ] }, { "cell_type": "markdown", "id": "c54e3a7a-7e88-4007-b974-59859cd21bfd", "metadata": {}, "source": [ "## From Field Notes\n", "\n", "Field notes are important but can be a pain to keep track of, whether they are written in notebooks, kept on tablets, or in various formats. Say you have a spreadsheet of field notes and now you'd like to make some metadata objects." ] }, { "cell_type": "code", "execution_count": 10, "id": "a28e037c-61dc-4fcd-a144-ee7e8197f549", "metadata": {}, "outputs": [], "source": [ "#TODO" ] }, { "cell_type": "code", "execution_count": null, "id": "71f201de-2b61-4b62-afec-da14e09ec257", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13" } }, "nbformat": 4, "nbformat_minor": 5 }