Skip to content

Commit f44b4dd

Browse files
pmltester1pmltester1
committed
first commit
1 parent 36204da commit f44b4dd

File tree

63 files changed

+10559
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+10559
-0
lines changed

‎.gitignore

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
env/
12+
build/
13+
develop-eggs/
14+
dist/
15+
downloads/
16+
eggs/
17+
.eggs/
18+
lib/
19+
lib64/
20+
parts/
21+
sdist/
22+
var/
23+
wheels/
24+
*.egg-info/
25+
.installed.cfg
26+
*.egg
27+
28+
# PyInstaller
29+
# Usually these files are written by a python script from a template
30+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
31+
*.manifest
32+
*.spec
33+
34+
# Installer logs
35+
pip-log.txt
36+
pip-delete-this-directory.txt
37+
38+
# Unit test / coverage reports
39+
htmlcov/
40+
.tox/
41+
.coverage
42+
.coverage.*
43+
.cache
44+
nosetests.xml
45+
coverage.xml
46+
*.cover
47+
.hypothesis/
48+
49+
# Translations
50+
*.mo
51+
*.pot
52+
53+
# Django stuff:
54+
*.log
55+
local_settings.py
56+
57+
# Flask stuff:
58+
instance/
59+
.webassets-cache
60+
61+
# Scrapy stuff:
62+
.scrapy
63+
64+
# Sphinx documentation
65+
docs/_build/
66+
67+
# PyBuilder
68+
target/
69+
70+
# Jupyter Notebook
71+
.ipynb_checkpoints
72+
73+
# pyenv
74+
.python-version
75+
76+
# celery beat schedule file
77+
celerybeat-schedule
78+
79+
# SageMath parsed files
80+
*.sage.py
81+
82+
# dotenv
83+
.env
84+
85+
# virtualenv
86+
.venv
87+
venv/
88+
ENV/
89+
90+
# Spyder project settings
91+
.spyderproject
92+
.spyproject
93+
94+
# Rope project settings
95+
.ropeproject
96+
97+
# mkdocs documentation
98+
/site
99+
100+
# mypy
101+
.mypy_cache/
102+
103+
# General data and images
104+
*.nc
105+
106+
*.zip
107+
108+
# OSx junk
109+
*DS_Store*
110+
111+
# Work in progress projects
112+
*_WIP
113+
WIP_*
114+
115+
116+
# data directories
117+
*data*

‎00_index.ipynb

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
" <img src='./img/EU-Copernicus-EUM_3Logos.png' alt='Logo EU Copernicus EUMETSAT' width='100%'></img>\n",
8+
" </br>\n",
9+
"# Ocean Training Notebooks\n",
10+
"\n",
11+
"This is a collection of Python [link] Jupyter [link] noetbooks covering core principles such as; data searching and acess, data manipulation and filtering, and data visulisation."
12+
]
13+
},
14+
{
15+
"cell_type": "markdown",
16+
"metadata": {},
17+
"source": [
18+
"## structure\n",
19+
"\n",
20+
"The jupyter notebooks are orghanised such that teh basics of data retrieval are covered first, then simple data manipulation followed by more complex fully worked data analysis workflows."
21+
]
22+
},
23+
{
24+
"cell_type": "markdown",
25+
"metadata": {},
26+
"source": [
27+
"These notebooks provide information on multiple sensors and possible uses of the data provided by them, below are some quick links to jump to a specific sensor. You can also use the links at the bottom of the page to advance thruogh the examples.\n",
28+
"\n",
29+
" * [OLCI - Ocean and Land Colour Instrument](./OLCI/11_OLCI_overview.ipynb)\n",
30+
" * [SLSTR - Sea and Land Surface Temperature Radiometer](./SLSTR/21_SLSTR_spatial_interrogation.ipynb)\n",
31+
" * [SRAL - Synthetic Aperture Radar Altimeter](./STM/31_SRAL_SLAs.ipynb) "
32+
]
33+
},
34+
{
35+
"cell_type": "markdown",
36+
"metadata": {},
37+
"source": [
38+
"<br> <a href=\"./index_ocean.ipynb\"><< Index</a><span style=\"float:right;\"><a href=\"./OLCI/11_OLCI_overview.ipynb\">11 - Ocean and Land Colour Instrument - Overview and data access >></a> <hr> <p style=\"text-align:left;\">This project is licensed under the <a href=\"/edit/LICENSE\">MIT License</a> <span style=\"float:right;\"><a href=\"https://gitlab.eumetsat.int/eo-lab-usc-open/ocean\">View on GitLab</a> | <a href=\"https://training.eumetsat.int/\">EUMETSAT Training</a> | <a href=mailto:[email protected]>Contact</a></span></p>"
39+
]
40+
},
41+
{
42+
"cell_type": "code",
43+
"execution_count": null,
44+
"metadata": {},
45+
"outputs": [],
46+
"source": []
47+
}
48+
],
49+
"metadata": {
50+
"kernelspec": {
51+
"display_name": "Python 3",
52+
"language": "python",
53+
"name": "python3"
54+
},
55+
"language_info": {
56+
"codemirror_mode": {
57+
"name": "ipython",
58+
"version": 3
59+
},
60+
"file_extension": ".py",
61+
"mimetype": "text/x-python",
62+
"name": "python",
63+
"nbconvert_exporter": "python",
64+
"pygments_lexer": "ipython3",
65+
"version": "3.6.10"
66+
}
67+
},
68+
"nbformat": 4,
69+
"nbformat_minor": 2
70+
}

‎Configuration_testing/Python_configuration_tester.ipynb

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Python configuration tester"
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
" Version: 1.0\n",
15+
" Date: 12/03/2019\n",
16+
" Author: Ben Loveday and Hayley Evers-King (Plymouth Marine Laboratory)\n",
17+
" Credit: This code was developed for EUMETSAT under contracts for the Copernicus \n",
18+
" programme.\n",
19+
" License: This code is offered as open source and free-to-use in the public domain, \n",
20+
" with no warranty."
21+
]
22+
},
23+
{
24+
"cell_type": "markdown",
25+
"metadata": {},
26+
"source": [
27+
"Running this script will test if you have all the necessary modules installed for the EUMETSAT Copernicus Marine training course. If this runs successfully it will print \"All imported ok\" in the panel at the bottom of the screen. If a module is not properly installed, it will print at the bottom of the screen, along with suggested instructions of how to install it."
28+
]
29+
},
30+
{
31+
"cell_type": "code",
32+
"execution_count": 1,
33+
"metadata": {},
34+
"outputs": [
35+
{
36+
"name": "stdout",
37+
"output_type": "stream",
38+
"text": [
39+
"------------------------------------------------------\n",
40+
"Everything required is installed - you are good to go!\n",
41+
"------------------------------------------------------\n"
42+
]
43+
}
44+
],
45+
"source": [
46+
"import warnings\n",
47+
"warnings.filterwarnings('ignore')\n",
48+
"\n",
49+
"\n",
50+
"required_modules = ['os', 'sys', 'shutil', 'warnings', 'fnmatch', 'datetime', 'logging',\\\n",
51+
" 're', 'requests', 'lxml', 'glob', 'tempfile', 'configparser',\\\n",
52+
" 'fileinput', 'subprocess', 'numpy', 'netCDF4', 'xarray', 'matplotlib',\\\n",
53+
" 'cartopy','math','pandas','shapely', 'gdal']\n",
54+
"\n",
55+
"optional_modules = ['plotly','scipy','rasterio','earthpy','sklearn']\n",
56+
"\n",
57+
"missing_module_flag = False\n",
58+
"\n",
59+
"for module in required_modules:\n",
60+
" cmd='import ' + module\n",
61+
" try:\n",
62+
" exec(cmd)\n",
63+
" except:\n",
64+
" missing_module_flag = True\n",
65+
" print('Required module ' + module + ' not installed')\n",
66+
"\n",
67+
"for module in optional_modules:\n",
68+
" cmd='import ' + module\n",
69+
" try:\n",
70+
" exec(cmd)\n",
71+
" except:\n",
72+
" missing_module_flag = True\n",
73+
" print('Optional module ' + module + ' not installed')\n",
74+
"\n",
75+
"if missing_module_flag:\n",
76+
" print('------------------------------------------------------')\n",
77+
" print('To install a package, please run the following in your conda prompt:')\n",
78+
" print(' ') \n",
79+
" print('conda install <replace-with-missing-module-name>')\n",
80+
" print(' ') \n",
81+
" print(' -------- or --------')\n",
82+
" print(' ')\n",
83+
" print('conda install -c conda-forge <replace-with-missing-module-name>')\n",
84+
" print('------------------------------------------------------')\n",
85+
"else:\n",
86+
" print('------------------------------------------------------')\n",
87+
" print('Everything required is installed - you are good to go!') \n",
88+
" print('------------------------------------------------------')"
89+
]
90+
},
91+
{
92+
"cell_type": "code",
93+
"execution_count": null,
94+
"metadata": {},
95+
"outputs": [],
96+
"source": []
97+
}
98+
],
99+
"metadata": {
100+
"kernelspec": {
101+
"display_name": "Python 3",
102+
"language": "python",
103+
"name": "python3"
104+
},
105+
"language_info": {
106+
"codemirror_mode": {
107+
"name": "ipython",
108+
"version": 3
109+
},
110+
"file_extension": ".py",
111+
"mimetype": "text/x-python",
112+
"name": "python",
113+
"nbconvert_exporter": "python",
114+
"pygments_lexer": "ipython3",
115+
"version": "3.6.10"
116+
}
117+
},
118+
"nbformat": 4,
119+
"nbformat_minor": 2
120+
}

‎Configuration_testing/README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
**CONFIGURATION TESTING**
2+
---
3+
|Description | Jupyter notebooks scripts to check training code and data configuration |
4+
| :-------------| :----------------------------------------------------------- |
5+
|Author | Ben Loveday, Hayley Evers-king (Plymouth Marine Laboratory) |
6+
|Date | 02/2019 |
7+
|Version | v2.1 |
8+
|Project use | Copernicus projects for EUMETSAT |
9+
10+
**MODULES:**
11+
---
12+
| Module | Job |
13+
| :----------------------------------- | :---------------------------------------- |
14+
|Python_configuration_tester.ipynb | This script will check if a course participant has all the required python modules installed and will provide advice to remediate issues if they are encountered. |
15+
|Data_path_checker.ipynb | This script helps participants who are new to the command line connect the training code to the training data. |
16+
|Widget_Functions.py | Is a support script for running Jupyter notebook textbox widgets. |

‎Configuration_testing/widget_functions.py

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Version: 1.0
2+
# Date: 12/03/2019
3+
# Author: Ben Loveday and Hayley Evers-King (Plymouth Marine Laboratory)
4+
# Credit: This code was developed for EUMETSAT under contracts for the
5+
# Copernicus programme.
6+
# License: This code is offered as open source and free-to-use in the
7+
# public domain, with no warranty.
8+
9+
from ipywidgets import widgets
10+
from IPython.display import display
11+
import os
12+
import sys
13+
14+
def handle_submit(sender):
15+
16+
init_path = sender.value
17+
found_path = False
18+
if os.path.exists(init_path):
19+
print('Found path: ' + init_path)
20+
found_path = True
21+
else:
22+
print('Not found path: ' + init_path + '. Please check it is correct')
23+
24+
if found_path:
25+
print("---------------------------------------------------------------")
26+
if '/' in init_path:
27+
file_list = init_path.split('/')
28+
path_command = "os.path.join(" \
29+
+ ','.join('"' + item + '"' for item in file_list) \
30+
+ ")"
31+
elif '\\' in init_path:
32+
file_list = init_path.split('\\')
33+
if ':' in file_list[0]:
34+
file_list[0] = file_list[0]+'/'
35+
file_list = filter(None, file_list)
36+
path_command = "os.path.join("+','.join('"' + item \
37+
+ '"' for item in file_list)+")"
38+
print('The safe, cross-platform way to join this path in python is: ')
39+
print("\033[1m" + path_command + "\033[0m")
40+
print("")
41+
if 'gpt' in sender.value:
42+
msg = 'GPTPATH'
43+
else:
44+
msg = 'MYPATH'
45+
print('Please remember the field in bold, we will refer to it as ' \
46+
+ '"' + msg + '" in later scripts.')
47+
print("---------------------------------------------------------------")
48+
49+
def text_box(msg):
50+
text = widgets.Text(\
51+
placeholder='Please copy your ' + msg + ' path to here...')
52+
display(text)
53+
if 'GPT' in msg:
54+
text.on_submit(handle_submit)
55+
else:
56+
text.on_submit(handle_submit)

0 commit comments

Comments
 (0)