AVG files

MTEdit processing code developed by Zonge International outputs .avg files which have minimal metadata and contain statistical estimates of the impedance, tipper and data errors.

There are a few different flavors of .avg files, earlier versions have minimal metadata, newere ones have more extensive metadata.

IMPORTANT: There is a switch or flag within Zonge’s MTEdit processing code that let’s you set z positive up or down. This is currently not recorded in the .avg metadata and therefore will need to input by the user. The default is z_positive = 'down' cause that is the “right” way.

from mt_metadata import TF_AVG, TF_AVG_TIPPER
from mt_metadata.transfer_functions import TF
from mt_metadata.transfer_functions.io.zonge import ZongeMTAvg

Older AVG file structure

AVG files are not common, here is the basic structure. Some metadata about how the station was processed, orientation of setup, and location, and then the statistical estimates with some parameters on coherency and Fourier Coefficients used.

with open(TF_AVG, "r") as fid:
    lines = fid.readlines()

print("".join(lines[0:18]))

$Survey.Type=NSAMT
$Survey.Array=Tensor
$Tx.Type=Natural
$MTEdit:Version=3.10m applied 2021/01/27
$MTEdit:Auto.PhaseFlip=Yes
$MTEdit:PhaseSlope.Smooth=Robust
$MTEdit:PhaseSlope.toZmag=No
$MTEdit:DPlus.Use=No
$Rx.GdpStn= 24
$Rx.Length=100 M
$Rx.HPR=0,0,180
$GPS.Lat=32.83331167
$GPS.Lon=-107.08305667
$Unit.Length=m
Skp,Freq,      E.mag,      B.mag,      Z.mag,      Z.phz,   ARes.mag,   ARes.%err,Z.perr,  Coher,   FC.NUse,FC.NTry
$Rx.Cmp = Zxx
2,  2.3438E-2, 7.8081E-01, 2.9611E-01, 1.3930E+00, -2228.9, 1.6558E+01, 98.4,     514.3,   0.971,   8,      16

tf_object = TF(fn=TF_AVG)
tf_object.read(z_positive="down")
avg_object = ZongeMTAvg(fn=TF_AVG)
avg_object.read()

tf_object.station_metadata

{
    "station": {
        "channels_recorded": [
            "ex",
            "ey",
            "hx",
            "hy"
        ],
        "data_type": "nsamt",
        "geographic_name": null,
        "id": "24",
        "location.datum": "WGS84",
        "location.declination.model": "WMM",
        "location.declination.value": 0.0,
        "location.elevation": 1414.37487793,
        "location.latitude": 32.83331167,
        "location.longitude": -107.08305667,
        "orientation.method": null,
        "orientation.reference_frame": "geographic",
        "provenance.archive.name": null,
        "provenance.creation_time": "1980-01-01T00:00:00+00:00",
        "provenance.creator.name": null,
        "provenance.software.author": null,
        "provenance.software.name": null,
        "provenance.software.version": null,
        "provenance.submitter.email": null,
        "provenance.submitter.name": null,
        "provenance.submitter.organization": null,
        "release_license": "CC0-1.0",
        "run_list": [
            "001"
        ],
        "time_period.end": "1980-01-01T00:00:00+00:00",
        "time_period.start": "1980-01-01T00:00:00+00:00",
        "transfer_function.coordinate_system": "geopgraphic",
        "transfer_function.data_quality.rating.value": 0,
        "transfer_function.id": "24",
        "transfer_function.processed_by.name": null,
        "transfer_function.processed_date": "1980-01-01",
        "transfer_function.processing_parameters": [
            "mtedit.auto.phase_flip=yes",
            "mtedit.d_plus.use=no",
            "mtedit.phase_slope.smooth=robust",
            "mtedit.phase_slope.to_z_mag=no"
        ],
        "transfer_function.processing_type": null,
        "transfer_function.remote_references": [],
        "transfer_function.runs_processed": [
            "001"
        ],
        "transfer_function.sign_convention": null,
        "transfer_function.software.author": "Zonge International",
        "transfer_function.software.last_updated": "2021/01/27",
        "transfer_function.software.name": "MTEdit",
        "transfer_function.software.version": "3.10m",
        "transfer_function.units": null
    }
}

Statistical Estimates

The data are stored as magnitude and phase of the impedance which are converted to real and imaginary values.

print(avg_object.z[0])
tf_object.impedance[0]

[[-0.85198341-1.1020768j  -1.76236149-1.89288924j]
 [-0.17290554-0.45221147j -0.16184071-0.1545914j ]]

<xarray.DataArray 'impedance' (output: 2, input: 2)>
array([[-0.85198341-1.1020768j , -1.76236149-1.89288924j],
       [-0.17290554-0.45221147j, -0.16184071-0.1545914j ]])
Coordinates:
    period   float64 42.67
  * output   (output) <U2 'ex' 'ey'
  * input    (input) <U2 'hx' 'hy'
Attributes:
    survey:             0
    project:            None
    id:                 24
    name:               None
    latitude:           32.83331167
    longitude:          -107.08305667
    elevation:          1414.37487793
    declination:        0.0
    datum:              WGS84
    acquired_by:        None
    start:              1980-01-01T00:00:00+00:00
    end:                1980-01-01T00:00:00+00:00
    runs_processed:     ['001']
    coordinate_system:  geographic

xarray.DataArray

'impedance'

• output: 2
• input: 2

• (-0.851983410178674-1.1020767980410064j) ... (-0.16184070793551983-...

  array([[-0.85198341-1.1020768j , -1.76236149-1.89288924j],
         [-0.17290554-0.45221147j, -0.16184071-0.1545914j ]])

• Coordinates: (3)
  • period
    ()
    float64
    42.67

    array(42.66575646)

  • output
    (output)
    <U2
    'ex' 'ey'

    array(['ex', 'ey'], dtype='<U2')

  • input
    (input)
    <U2
    'hx' 'hy'

    array(['hx', 'hy'], dtype='<U2')

• Attributes: (14)

  survey :

  0

  project :

  None

  id :

  24

  name :

  None

  latitude :

  32.83331167

  longitude :

  -107.08305667

  elevation :

  1414.37487793

  declination :

  0.0

  datum :

  WGS84

  acquired_by :

  None

  start :

  1980-01-01T00:00:00+00:00

  end :

  1980-01-01T00:00:00+00:00

  runs_processed :

  ['001']

  coordinate_system :

  geographic

print(avg_object.z_err[0])
tf_object.impedance_error[0]

[[1.38180502 2.05773344]
 [0.21920379 0.16689004]]

<xarray.DataArray 'impedance_error' (output: 2, input: 2)>
array([[1.38180502, 2.05773344],
       [0.21920379, 0.16689004]])
Coordinates:
    period   float64 42.67
  * output   (output) <U2 'ex' 'ey'
  * input    (input) <U2 'hx' 'hy'
Attributes:
    survey:             0
    project:            None
    id:                 24
    name:               None
    latitude:           32.83331167
    longitude:          -107.08305667
    elevation:          1414.37487793
    declination:        0.0
    datum:              WGS84
    acquired_by:        None
    start:              1980-01-01T00:00:00+00:00
    end:                1980-01-01T00:00:00+00:00
    runs_processed:     ['001']
    coordinate_system:  geographic

xarray.DataArray

'impedance_error'

• output: 2
• input: 2

• 1.382 2.058 0.2192 0.1669

  array([[1.38180502, 2.05773344],
         [0.21920379, 0.16689004]])

• Coordinates: (3)
  • period
    ()
    float64
    42.67

    array(42.66575646)

  • output
    (output)
    <U2
    'ex' 'ey'

    array(['ex', 'ey'], dtype='<U2')

  • input
    (input)
    <U2
    'hx' 'hy'

    array(['hx', 'hy'], dtype='<U2')

• Attributes: (14)

  survey :

  0

  project :

  None

  id :

  24

  name :

  None

  latitude :

  32.83331167

  longitude :

  -107.08305667

  elevation :

  1414.37487793

  declination :

  0.0

  datum :

  WGS84

  acquired_by :

  None

  start :

  1980-01-01T00:00:00+00:00

  end :

  1980-01-01T00:00:00+00:00

  runs_processed :

  ['001']

  coordinate_system :

  geographic

Newer AVG file structure

The newer .avg files have a bit more metadata. Here’s a look at an example, note that there is now a block of metadata before each component that has information about the channels used to compute that component of the impedance tensor or induction vector. Most of this is captured in the reader. The parts that are not yet are the locations of the specific channels, namely Rx.XYZ and Rx.UTM.

with open(TF_AVG_TIPPER, "r") as fid:
    lines = fid.readlines()

print("".join(lines[0:50]))

$Survey.Type=MT
$Tx.Type=Natural
$MTEdit:Version=3.11n applied 2022/08/02
$MTEdit:Auto.PhaseFlip=No
$MTEdit:PhaseSlope.Smooth=Minimal
$MTEdit:PhaseSlope.toZmag=No
$MTEdit:DPlus.Use=No
$MTFT24:Version=1.30k applied 2022/08/02
$Survey.Datum=WGS84
$Survey.Proj=UTM
$Survey.UTMZone=12
$Line.Name="0"
$Line.Number=0
$GDP.Date=05/16/2022
$GDP.Time=23:46:18.000
$GDP.Type=Zen
$GDP.ProgVer=4457:ZenACQv5.22
$Unit.Length=m
$GPS.Lat=38.6653467
$GPS.Lon=-113.1690717
$GPS.Datum=WGS84
$GPS.UTMZone=12
$Survey.UTMZone=12
$Rx.ASpace=100 m
$Rx.SSpace=100
$Unit.E=uV/m
$Unit.B=nT
$Rx.GdpStn= 22
$Stn.Name= 22
$Rx.HPR=11,0,180

$Rx.Cmp = Zxx
$Rx.Length=100 m
$Rx.Center=311288:4281879:1548.1 m
$Rx.XYZ1=311278.21:4281829.97:0, 311288:4281879:0
$Rx.XYZ2=311297.79:4281928.03:0, 311288:4281879:0
$Rx.UTM1=311278.21:4281829.97:1548.1, 311288:4281879:1548.1
$Rx.UTM2=311297.79:4281928.03:1548.1, 311288:4281879:0
$Ch.ADCardSN=7A07B42B,5EE2EE6B
$Ch.GdpBox=17,17
$Ch.Stn=22,22
$Ch.Number=22,2374
$Ch.Cmp=Ex,Hx
$Ch.CRes=0,0
$Ch.Azimuth=11.3,11.3
$Ch.Incl=0,0
Skp,Freq,      E.mag,      B.mag,      Z.mag,      Z.phz,   ARes.mag,   ARes.%err,Z.perr,  Coher,   FC.NUse,FC.NTry
1,  7.3242E-4, 7.8837E+01, 6.1492E+01, 1.2821E+00, -1908.8, 4.4884E+02, 168,      997,     0.672,   62,     217
1,  9.7656E-4, 1.6841E+01, 3.4432E+01, 4.8911E-01, 929.9,   4.8995E+01, 108.5,    573.6,   0.482,   89,     217

Opening Files with z-positive up

If you accidentally processed the data assuming z is positive up, that’s ok it happens. You can set the parameter z_positive to up. This will read in the data as if z is positive down. If that is not the intention leave the flag as down and just know that your data is in a sinister coordinate system. Also, note these data are not great so don’t mind the numbers, they are not representative of common values.

tf_object = TF(fn=TF_AVG_TIPPER)
tf_object.read(z_positive="up")

tf_object.tipper[0]

<xarray.DataArray 'tipper' (output: 1, input: 2)>
array([[923.7823987 -524.50678723j, -22.74427485 -18.94823397j]])
Coordinates:
    period   float64 1.365e+03
  * output   (output) <U2 'hz'
  * input    (input) <U2 'hx' 'hy'
Attributes:
    survey:             0
    project:            None
    id:                 22
    name:               None
    latitude:           38.6653467
    longitude:          -113.1690717
    elevation:          1548.1
    declination:        0.0
    datum:              WGS84
    acquired_by:        None
    start:              2022-05-16T23:46:18+00:00
    end:                1980-01-01T00:00:00+00:00
    runs_processed:     ['001']
    coordinate_system:  geographic

xarray.DataArray

'tipper'

• output: 1
• input: 2

• (923.7823987037089-524.5067872251244j) (-22.744274847079495-18.9482...

  array([[923.7823987 -524.50678723j, -22.74427485 -18.94823397j]])

• Coordinates: (3)
  • period
    ()
    float64
    1.365e+03

    array(1365.3368286)

  • output
    (output)
    <U2
    'hz'

    array(['hz'], dtype='<U2')

  • input
    (input)
    <U2
    'hx' 'hy'

    array(['hx', 'hy'], dtype='<U2')

• Attributes: (14)

  survey :

  0

  project :

  None

  id :

  22

  name :

  None

  latitude :

  38.6653467

  longitude :

  -113.1690717

  elevation :

  1548.1

  declination :

  0.0

  datum :

  WGS84

  acquired_by :

  None

  start :

  2022-05-16T23:46:18+00:00

  end :

  1980-01-01T00:00:00+00:00

  runs_processed :

  ['001']

  coordinate_system :

  geographic

The newer formats contain channel information, and those are absorbed by the reader.

for ch in tf_object.station_metadata.runs[0].channels:
    print(ch)

electric:
        channel_id = 22
        channel_number = 0
        component = ex
        data_quality.rating.value = 0
        dipole_length = 100.0
        filter.applied = [False]
        filter.name = []
        measurement_azimuth = 11.3
        measurement_tilt = 0.0
        negative.elevation = 0.0
        negative.id = None
        negative.latitude = 0.0
        negative.longitude = 0.0
        negative.manufacturer = None
        negative.type = None
        positive.elevation = 0.0
        positive.id = None
        positive.latitude = 0.0
        positive.longitude = 0.0
        positive.manufacturer = None
        positive.type = None
        sample_rate = 0.0
        time_period.end = 1980-01-01T00:00:00+00:00
        time_period.start = 2022-05-16T23:46:18+00:00
        translated_azimuth = 11.3
        translated_tilt = 0.0
        type = electric
        units = None
electric:
        channel_id = 22
        channel_number = 0
        component = ey
        data_quality.rating.value = 0
        dipole_length = 100.0
        filter.applied = [False]
        filter.name = []
        measurement_azimuth = 101.3
        measurement_tilt = 0.0
        negative.elevation = 0.0
        negative.id = None
        negative.latitude = 0.0
        negative.longitude = 0.0
        negative.manufacturer = None
        negative.type = None
        positive.elevation = 0.0
        positive.id = None
        positive.latitude = 0.0
        positive.longitude = 0.0
        positive.manufacturer = None
        positive.type = None
        sample_rate = 0.0
        time_period.end = 1980-01-01T00:00:00+00:00
        time_period.start = 2022-05-16T23:46:18+00:00
        translated_azimuth = 101.3
        translated_tilt = 0.0
        type = electric
        units = None
magnetic:
        channel_id = 1
        channel_number = 0
        component = hx
        data_quality.rating.value = 0
        filter.applied = [False]
        filter.name = []
        location.elevation = 0.0
        location.latitude = 0.0
        location.longitude = 0.0
        measurement_azimuth = 11.3
        measurement_tilt = 0.0
        sample_rate = 0.0
        sensor.id = 2374
        sensor.manufacturer = None
        sensor.type = None
        time_period.end = 1980-01-01T00:00:00+00:00
        time_period.start = 2022-05-16T23:46:18+00:00
        translated_azimuth = 11.3
        translated_tilt = 0.0
        type = magnetic
        units = None
magnetic:
        channel_id = 2
        channel_number = 0
        component = hy
        data_quality.rating.value = 0
        filter.applied = [False]
        filter.name = []
        location.elevation = 0.0
        location.latitude = 0.0
        location.longitude = 0.0
        measurement_azimuth = 101.3
        measurement_tilt = 0.0
        sample_rate = 0.0
        sensor.id = 287
        sensor.manufacturer = None
        sensor.type = None
        time_period.end = 1980-01-01T00:00:00+00:00
        time_period.start = 2022-05-16T23:46:18+00:00
        translated_azimuth = 101.3
        translated_tilt = 0.0
        type = magnetic
        units = None
magnetic:
        channel_id = 3
        channel_number = 0
        component = hz
        data_quality.rating.value = 0
        filter.applied = [False]
        filter.name = []
        location.elevation = 0.0
        location.latitude = 0.0
        location.longitude = 0.0
        measurement_azimuth = 11.3
        measurement_tilt = 0.0
        sample_rate = 0.0
        sensor.id = 2374
        sensor.manufacturer = None
        sensor.type = None
        time_period.end = 1980-01-01T00:00:00+00:00
        time_period.start = 2022-05-16T23:46:18+00:00
        translated_azimuth = 11.3
        translated_tilt = 0.0
        type = magnetic
        units = None