Skip to content

Merge development into master to issue new release #174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
May 11, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 17 additions & 27 deletions Examples/TimeSeries_RetrieveVisualize.ipynb

Large diffs are not rendered by default.

54 changes: 25 additions & 29 deletions Examples/WaterQualityMeasurements_RetrieveVisualize.ipynb

Large diffs are not rendered by default.

7 changes: 2 additions & 5 deletions Examples/clientenvironment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,9 @@ dependencies:
- yodatools
- requests
- scipy
- xarray
- folium
- geopandas
- gdal=2.2
- owslib
- ulmo
- gdal
- folium
# Jupyter notebook
- ipykernel
- jupyter
Expand Down
30 changes: 15 additions & 15 deletions docs/source/getstarted.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,16 @@ To create a new environment "myenv" with the ``odm2api`` package:

conda create -n myenv -c conda-forge python=2.7 odm2api

Sample Jupyter notebooks
------------------------

These two notebooks are complete, extended examples that illustrate reading from ODM2 databases and using the resulting data and metadata. They use SQLite ODM2 file databases that can be `downloaded here <https://github.com/ODM2/ODM2PythonAPI/tree/master/Examples/data>`_.
A conda environment to run these notebooks can be created with the conda environment file
`clientenvironment.yml <https://github.com/ODM2/ODM2PythonAPI/blob/master/Examples/clientenvironment.yml>`_.

1. `WaterQualityMeasurements_RetrieveVisualize.ipynb <https://nbviewer.jupyter.org/github/ODM2/ODM2PythonAPI/blob/master/Examples/WaterQualityMeasurements_RetrieveVisualize.ipynb>`_

2. `TimeSeries_RetrieveVisualize.ipynb <https://nbviewer.jupyter.org/github/ODM2/ODM2PythonAPI/blob/master/Examples/TimeSeries_RetrieveVisualize.ipynb>`_

Code examples
-------------
Expand All @@ -63,12 +73,14 @@ Connect to an ODM2 database and open the connection for reading.
from odm2api.ODMconnection import dbconnection
import odm2api.services.readService as odm2rs

# A SQLite file-based connection
# -----------------------------------------------------
# 1. A SQLite file-based connection
session_factory = dbconnection.createConnection('sqlite',
'/myfilepath/odm2db.sqlite')
read = odm2rs.ReadODM2(session_factory)

# A connection to a server-based database system
# -----------------------------------------------------
# 2. A server-based database system connection
db_credentials = {
'address': 'ip-or-domainname',
'db': 'dbname',
Expand All @@ -93,21 +105,9 @@ then constructing and issuing a direct ``SQL UPDATE`` statement, like this:
from odm2api.ODMconnection import dbconnection

session_factory = dbconnection.createConnection('postgresql',
**db_cred)
**db_credentials)
DBSession = session_factory.getSession()

sq_str = " UPDATE mytable SET variablecode = 'xyz' WHERE variablecode = 'abc' "
DBSession.execute(sql_str)
DBSession.commit()


Sample Jupyter notebooks
------------------------

These two notebooks are complete, extended examples that illustrate reading from ODM2 databases and using the resulting data and metadata. They use SQLite ODM2 file databases that can be `downloaded here <https://github.com/ODM2/ODM2PythonAPI/tree/master/Examples/data>`_.
A conda environment to run these notebooks can be created with the conda environment file
`clientenvironment.yml <https://github.com/ODM2/ODM2PythonAPI/blob/master/Examples/clientenvironment.yml>`_.

1. `WaterQualityMeasurements_RetrieveVisualize.ipynb <https://github.com/ODM2/ODM2PythonAPI/blob/master/Examples/WaterQualityMeasurements_RetrieveVisualize.ipynb>`_

2. `TimeSeries_RetrieveVisualize.ipynb <https://github.com/ODM2/ODM2PythonAPI/blob/master/Examples/TimeSeries_RetrieveVisualize.ipynb>`_
6 changes: 4 additions & 2 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ ODM2 Python API (odm2api)

A Python-based application programmer's interface for the `Observations Data Model 2 (ODM2) <http://www.odm2.org>`__. Development of ``odm2api`` is done at `<https://github.com/ODM2/ODM2PythonAPI/>`_.

Contents
--------

.. toctree::
:maxdepth: 2
:caption: Contents:

getstarted
odm2models
Expand All @@ -14,7 +16,7 @@ A Python-based application programmer's interface for the `Observations Data Mod
credits

Indices and Search
==================
-----------------

.. toctree::
:maxdepth: 2
Expand Down
23 changes: 12 additions & 11 deletions odm2api/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
BigIntegerType = BigIntegerType.with_variant(mysql.BIGINT(), 'mysql')

DateTimeType = DateTime()
DateTimeType = DateTimeType.with_variant(sqlite.INTEGER(), 'sqlite')
DateTimeType = DateTimeType.with_variant(sqlite.DATETIME(), 'sqlite')


def is_hex(s):
Expand Down Expand Up @@ -463,7 +463,7 @@ class DataLoggerFiles(Base):


class EquipmentModels(Base):
ModelID = Column('modelid', Integer, primary_key=True, nullable=False)
ModelID = Column('equipmentmodelid', Integer, primary_key=True, nullable=False)
ModelManufacturerID = Column('modelmanufacturerid', Integer,
ForeignKey(Organizations.OrganizationID), nullable=False)
ModelPartNumber = Column('modelpartnumber', String(50))
Expand Down Expand Up @@ -537,14 +537,15 @@ class Equipment(Base):
EquipmentCode = Column('equipmentcode', String(50), nullable=False)
EquipmentName = Column('equipmentname', String(255), nullable=False)
EquipmentTypeCV = Column('equipmenttypecv', ForeignKey(CVEquipmentType.Name), nullable=False, index=True)
ModelID = Column('modelid', ForeignKey(EquipmentModels.ModelID), nullable=False)
EquipmentSerialNumber = Column('equipmentseriealnumber', String(50), nullable=False)
ModelID = Column('equipmentmodelid', ForeignKey(EquipmentModels.ModelID), nullable=False)
EquipmentSerialNumber = Column('equipmentserialnumber', String(50), nullable=False)
EquipmentInventoryNumber = Column('equipmentinventorynumber', String(50))
EquipmentOwnerID = Column('equipmentownerid', ForeignKey(People.PersonID), nullable=False)
EquipmentVendorID = Column('equipmentvendorid', ForeignKey(Organizations.OrganizationID), nullable=False)
EquipmentPurchaseDate = Column('equipmentpurchasedate', DateTime, nullable=False)
EquipmentPurchaseOrderNumber = Column('equipmentpurchaseordernumber', String(50))
EquipmentDescription = Column('equipmentdescription', String(500))
EquipmentDocumentationLink = Column('equipmentdocumentationlink', String(255))

PersonObj = relationship(People)
OrganizationObj = relationship(Organizations)
Expand Down Expand Up @@ -580,7 +581,7 @@ class EquipmentUsed(Base):

class MaintenanceActions(Base):
ActionID = Column('actionid', Integer, ForeignKey(Actions.ActionID), primary_key=True, nullable=False)
IsFactoryService = Column('isfactoryservce', Boolean, nullable=False)
IsFactoryService = Column('isfactoryservice', Boolean, nullable=False)
MaintenanceCode = Column('maintenancecode', String(50))
MantenanceReason = Column('maintenancereason', String(50))

Expand Down Expand Up @@ -836,7 +837,7 @@ class ActionAnnotations(Base):

class EquipmentAnnotations(Base):
BridgeID = Column('bridgeid', Integer, primary_key=True, nullable=False)
EquipmentID = Column('valueid', BigIntegerType, ForeignKey(Equipment.EquipmentID), nullable=False)
EquipmentID = Column('equipmentid', BigIntegerType, ForeignKey(Equipment.EquipmentID), nullable=False)
AnnotationID = Column('annotationid', ForeignKey(Annotations.AnnotationID), nullable=False)

AnnotationObj = relationship(Annotations)
Expand Down Expand Up @@ -906,7 +907,7 @@ class ReferenceMaterials(Base):
nullable=False,
index=True
)
ReferenceMaterialOrganizationID = Column('referencematerialoranizationid',
ReferenceMaterialOrganizationID = Column('referencematerialorganizationid',
ForeignKey(Organizations.OrganizationID), nullable=False)
ReferenceMaterialCode = Column('referencematerialcode', String(50), nullable=False)
ReferenceMaterialLotCode = Column('referencemateriallotcode', String(255))
Expand Down Expand Up @@ -951,7 +952,7 @@ class ReferenceMaterialValues(Base):

class ResultNormalizationValues(Base):
ResultID = Column(u'resultid', ForeignKey(Results.ResultID), primary_key=True)
ReferenceMaterialValueID = Column(u'referencematerialvalueid',
ReferenceMaterialValueID = Column(u'normalizedbyreferencematerialvalueid',
ForeignKey(ReferenceMaterialValues.ReferenceMaterialValueID),
nullable=False)

Expand Down Expand Up @@ -1068,8 +1069,8 @@ class CitationExternalIdentifiers(Base):
ExternalIdentifierSystemID = Column('externalidentifiersystemid',
ForeignKey(ExternalIdentifierSystems.ExternalIdentifierSystemID),
nullable=False)
CitationExternalIdentifier = Column('citationexternaldentifier', String(255), nullable=False)
CitationExternalIdentifierURI = Column('citationexternaldentifieruri', String(255))
CitationExternalIdentifier = Column('citationexternalidentifier', String(255), nullable=False)
CitationExternalIdentifierURI = Column('citationexternalidentifieruri', String(255))

CitationObj = relationship(Citations)
ExternalIdentifierSystemObj = relationship(ExternalIdentifierSystems)
Expand Down Expand Up @@ -1678,7 +1679,7 @@ class SpectraResultValues(Base):
ValueDateTime = Column('valuedatetime', DateTimeType, nullable=False)
ValueDateTimeUTCOffset = Column('valuedatetimeutcoffset', Integer, nullable=False)
ExcitationWavelength = Column('excitationwavelength', Float(53), nullable=False)
EmissionWavelength = Column('emmistionwavelength', Float(53), nullable=False)
EmissionWavelength = Column('emissionwavelength', Float(53), nullable=False)
WavelengthUnitsID = Column('wavelengthunitsid', ForeignKey(Units.UnitsID), nullable=False)
CensorCodeCV = Column('censorcodecv', ForeignKey(CVCensorCode.Name), nullable=False, index=True)
QualityCodeCV = Column('qualitycodecv', ForeignKey(CVQualityCode.Name), nullable=False, index=True)
Expand Down
32 changes: 16 additions & 16 deletions odm2api/services/readService.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ def getAnnotations(self, annottype=None, codes=None, ids=None, **kwargs):
a = Annotations
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the annottype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the annottype parameter instead.",
DeprecationWarning, stacklevel=2)
annottype = kwargs['type']
if annottype:
Expand Down Expand Up @@ -238,7 +238,7 @@ def getCVs(self, cvtype, **kwargs):
"""
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the cvtype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the cvtype parameter instead.",
DeprecationWarning, stacklevel=2)
cvtype = kwargs['type']

Expand Down Expand Up @@ -430,7 +430,7 @@ def getMethods(self, ids=None, codes=None, methodtype=None, **kwargs):
"""
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the medtype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the medtype parameter instead.",
DeprecationWarning, stacklevel=2)
methodtype = kwargs['type']

Expand Down Expand Up @@ -517,7 +517,7 @@ def getSamplingFeatures(self, ids=None, codes=None, uuids=None,
"""
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the sftype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the sftype parameter instead.",
DeprecationWarning, stacklevel=2)
sftype = kwargs['type']
if results:
Expand Down Expand Up @@ -591,7 +591,7 @@ def getActions(self, ids=None, acttype=None, sfid=None, **kwargs):
"""
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the acttype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the acttype parameter instead.",
DeprecationWarning, stacklevel=2)
acttype = kwargs['type']
a = Actions
Expand Down Expand Up @@ -641,7 +641,7 @@ def getUnits(self, ids=None, name=None, unittype=None, **kwargs):
"""
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the unittype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the unittype parameter instead.",
DeprecationWarning, stacklevel=2)
unittype = kwargs['type']
q = self._session.query(Units)
Expand Down Expand Up @@ -779,7 +779,7 @@ def getResults(self, ids=None, restype=None, uuids=None, actionid=None, simulati
query = self._session.query(Results)
self._check_kwargs(['type', 'sfid'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the restype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the restype parameter instead.",
DeprecationWarning, stacklevel=2)
restype = kwargs['type']
if restype:
Expand All @@ -798,8 +798,8 @@ def getResults(self, ids=None, restype=None, uuids=None, actionid=None, simulati
if actionid:
query = query.join(FeatureActions).filter_by(ActionID=actionid)
if 'sfid' in kwargs:
warnings.warn('The parameter \'sfid\' is deprecated. '
'Please use the sfids parameter instead and send in a list.',
warnings.warn("The parameter 'sfid' is deprecated. " +
"Please use the sfids parameter instead and send in a list.", # noqa
DeprecationWarning, stacklevel=2)
if kwargs['sfid']:
query = query.join(FeatureActions).filter_by(SamplingFeatureID=kwargs['sfid'])
Expand Down Expand Up @@ -1087,7 +1087,7 @@ def getEquipment(self, codes=None, equiptype=None, sfid=None, actionid=None, **k
"""
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the equiptype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the equiptype parameter instead.",
DeprecationWarning, stacklevel=2)
equiptype = kwargs['type']

Expand Down Expand Up @@ -1191,7 +1191,7 @@ def getExtensionProperties(self, exptype=None, **kwargs):
# Todo what values to use for extensionproperties type
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the exptype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the exptype parameter instead.",
DeprecationWarning, stacklevel=2)
exptype = kwargs['type']
e = ExtensionProperties
Expand Down Expand Up @@ -1222,7 +1222,7 @@ def getExternalIdentifiers(self, eitype=None, **kwargs):
"""
self._check_kwargs(['type'], kwargs)
if 'type' in kwargs:
warnings.warn('The parameter \'type\' is deprecated. Please use the eitype parameter instead.',
warnings.warn("The parameter 'type' is deprecated. Please use the eitype parameter instead.",
DeprecationWarning, stacklevel=2)
eitype = kwargs['type']
e = ExternalIdentifierSystems
Expand Down Expand Up @@ -1415,9 +1415,9 @@ def getResultValues(self, resultids, starttime=None, endtime=None, lowercols=Tru
df.columns = [self._get_columns(ResultValues)[c] for c in df.columns]
else:
warnings.warn(
'In a near-future release, '
'the parameter \'lowercols\' default will be changed to False, '
'and later the parameter may be removed.',
"In a near-future release, " + # noqa
"the parameter 'lowercols' default will be changed to False, " +
"and later the parameter may be removed.", # noqa
DeprecationWarning, stacklevel=2)
return df
except Exception as e:
Expand Down Expand Up @@ -1487,7 +1487,7 @@ def getRelatedModels(self, modid=None, code=None, **kwargs):
"""
self._check_kwargs(['id'], kwargs)
if 'id' in kwargs:
warnings.warn('The parameter \'id\' is deprecated. Please use the modid parameter instead.',
warnings.warn("The parameter 'id' is deprecated. Please use the modid parameter instead.",
DeprecationWarning, stacklevel=2)
modid = kwargs['id']
m = self._session.query(Models).select_from(RelatedModels).join(RelatedModels.ModelObj)
Expand Down
6 changes: 3 additions & 3 deletions tests/test_SessionFactory.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import (absolute_import, division, print_function)

from odm2api.ODM2.models import CVElevationDatum, setSchema
from odm2api.ODMconnection import SessionFactory
from odm2api.models import CVElevationDatum, setSchema

import pytest

Expand All @@ -15,10 +15,10 @@
['mysql"root@Localhost/odm2', 'mysql', 'mysql+pymysql://root@localhost/odm2'],
['postgresql_marchantariats_none', 'postgresql',
'postgresql+psycopg2://postgres:None@localhost/marchantariats',
'marchantariats', 'postgres', None],
'marchantariats', 'postgres', None],
['postgresql_marchantariats_empty', 'postgresql',
'postgresql+psycopg2://postgres@localhost/marchantariats',
'marchantariats', 'postgres', None],
'marchantariats', 'postgres', None],
['sqlite_wof', 'sqlite', 'sqlite:///./tests/spatialite/wof2odm/ODM2.sqlite', None, None, None]
]

Expand Down
4 changes: 2 additions & 2 deletions tests/test_connection.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import (absolute_import, division, print_function)

from odm2api.ODM2.models import CVElevationDatum
from odm2api.ODMconnection import dbconnection
from odm2api.models import CVElevationDatum

import pytest

Expand All @@ -13,7 +13,7 @@
dbs_readonly = [
['mysql_odm2_odm', 'mysql', 'localhost', 'odm2', 'ODM', 'odm'],
['mysql_odm2_root', 'mysql', 'localhost', 'odm2', 'root', None],
['postgresql_marchantariats', 'postgresql', 'localhost', 'marchantariats', 'postgres', 'iforget'],
['postgresql_marchantariats', 'postgresql', 'localhost', 'marchantariats', 'postgres', 'iforget'],
['sqlite_wof', 'sqlite', './tests/spatialite/wof2odm/ODM2.sqlite', None, None, None]
]

Expand Down
Loading