You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<h1id="Contributors">Contributors<aclass="anchor-link" href="#Contributors"></a></h1><p>The core contributors of this tutorial are, in alphabetic order:</p>
<h2id="Citing-the-Weather-Panel-Tutorial">Citing the <em>Weather Panel Tutorial</em><aclass="anchor-link" href="#Citing-the-Weather-Panel-Tutorial"></a></h2><h2id="Contributions">Contributions<aclass="anchor-link" href="#Contributions"></a></h2><p>We welcome any contributions to this tutorial.
30
+
If you see a mistake, please don't hesitate to send us a
31
+
pull request on <ahref="https://github.com/atrisovic/weather-panel.github.io">GitHub</a>.</p>
Copy file name to clipboardExpand all lines: _build/geographical-unit-data.html
+4-3Lines changed: 4 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,20 @@
1
1
---
2
2
title: |-
3
3
Generating geographical unit data
4
-
pagenum: 4
5
4
prev_page:
6
5
url: /weighting-schemes.html
6
+
title: |-
7
+
Weighting schemes
7
8
next_page:
8
9
url: /suggestions.html
10
+
title: |-
11
+
Suggestions for work organization
9
12
suffix: .md
10
-
search: data shapefile regions administrative region grid units r matrix unit weather shape vector need example spatial cell common geographic names information files qgis shp cells states across also points shapefiles not python level using polygons used gridded dataset myshapefile pbsmapping w text values generating geographical economic countries those com gis analysis software www library set name same projection import within next generate entries sparse convert column listed events matching korea area defined ways city web where center adm political usually relevant granularity gadm org study features should much system representation geometry create polygon image arcgis latitude longitude working step fiona
11
13
12
14
comment: "***PROGRAMMATICALLY GENERATED, DO NOT EDIT. SEE ORIGINAL FILES IN /content***"
Copy file name to clipboardExpand all lines: _build/getting-started.html
+4-3Lines changed: 4 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,20 @@
1
1
---
2
2
title: |-
3
3
Getting started
4
-
pagenum: 0
5
4
prev_page:
6
5
url:
6
+
title: |-
7
+
7
8
next_page:
8
9
url: /weather-and-climate-data.html
10
+
title: |-
11
+
Using Weather and Climate Data
9
12
suffix: .md
10
-
search: data weather tutorial climate point region econometrics resource examples socioeconomic resolution variables work particular not science gridded being geographic unit grid t panel regression using economic better understand change responses hsiang through relate outcomes high read assume experience language try provide case specific question important useful www org theoretical foundation following describes space location across information always structure notation introduction welcome study social biophysical systems respond opened huge doors allowing us impacts disaster risk management human behavior sustainable development here relationships uncovered recent years carleton images jpg walk steps necessary cover should aware develop specification shapefiles generate predictor knowledge basic
11
13
12
14
comment: "***PROGRAMMATICALLY GENERATED, DO NOT EDIT. SEE ORIGINAL FILES IN /content***"
Copy file name to clipboardExpand all lines: _build/reduced-form-specification.html
+13-14Lines changed: 13 additions & 14 deletions
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,20 @@
1
1
---
2
2
title: |-
3
3
Developing a reduced-form specification
4
-
pagenum: 2
5
4
prev_page:
6
5
url: /weather-and-climate-data.html
6
+
title: |-
7
+
Using Weather and Climate Data
7
8
next_page:
8
9
url: /weighting-schemes.html
10
+
title: |-
11
+
Weighting schemes
9
12
suffix: .md
10
-
search: t theta temperature k transformation level ps g grid sum data c f weather aggregation n example precipitation county polynomial nabla using variable degree not variables response bins humidity before where circ knots radiation aggregate psi into important below average region our linear weighted spline beta variation across cubic model need cross validation agent text form specification used above also local e process however carbon values doing cells y js available methods good forms min days temperatures heat while air highly very since want us shortwave relative sst functional method grids terms betak betakf observations let take defined function curve
11
13
12
14
comment: "***PROGRAMMATICALLY GENERATED, DO NOT EDIT. SEE ORIGINAL FILES IN /content***"
@@ -25,23 +26,21 @@ <h2 id="2.1.-Choosing-weather-variables">2.1. Choosing weather variables<a class
25
26
<ol>
26
27
<li><em>$T_{min}$, $T_{max}$:</em> Many socioeconomic processes are more sensitive to extreme temperatures than to variation in the average. This is also useful when temperature variation is large, leading to significant differences in cold end and hot end response. These are important metric when heterogeneity between each time unit matters, and capture heat waves and cold spells. Also, note that $T_{min}$ reflects nighttime temperatures while $T_{max}$ is reached in the daytime.</li>
27
28
<li><em>$T_{avg}$:</em> A good mean metric for seeing average response over the temperature support, when there is not much variation in temperature across time unit considered in the study. $T_{avg}$ is most appropriate when there is some natural inertia in the response, so that the dependent variable is responding to a kind of average over the last 24 hours. Note that $T_{avg}$ is often just $(T_{min} + T_{max}) / 2$, unless calculated from sub-daily data.</li>
28
-
<li><ahref="https://www.degreedays.net/introduction"><em>HDD/CDD & GDD:</em></a> Degree days (DD) are a measure of ’how much’ and for ’how long’ the outside air temperature was above or below a certain level.</li>
29
+
<li><ahref="https://www.degreedays.net/introduction"><em>HDD/CDD & GDD:</em></a> Degree days (DD) are a measure of ’how much’ and for ’how long’ the outside air temperature was above or below a certain level.</li>
29
30
</ol>
30
31
</li>
31
-
<li><p><strong>Precipitation:</strong> As described above, precipitation is highly local, poorly measured, and poorly predicted.</p>
32
+
<li><p><strong>Precipitation:</strong> It is a highly local (in space <em>and</em> time), poorly measured, and poorly predicted (see <ahref="#1.5-A-Warning-on-Hydrological-Variables-(Precipitation,-Humidity,-etc.">Section 1.5</a>) weather variable. Precipitation is often used as a control since it is correlated with temperature. However, the strength and direction of this correlation varies significantly by region and time of year (see e.g. <ahref="https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2005GL022760">Trenberth et al. 2005</a>). Furthermore, the same care should be taken when inserting precipitation into a model as any other weather or social variable - what is its expected role? In what form should the data be? etc. Precipitation affects society differently at different spatiotemporal scales - annual precipitation may be useful for studying snowpack trends, drinking water supply, or the effect of droughts on agriculture; maximum precipitation rates may be the relevant metric for flood damages or crop failures. Remember that though means and extremes may be correlated, it's still possible to have a record storm in an unnaturally dry year, or an unnaturally wet year without heavy precipitation. As a result, different metrics of precipitation listed below are often used:</p>
32
33
<ol>
33
34
<li><em>Total precipitation (e.g., over a year)</em>: It is often used as a control, but not a very good reflection of the relevance of precipitation ot a socioeconomic process.</li>
34
35
<li><em>Soil water, potential evapotranspiration rate (PET), Palmer drought severity index (PDSI), and water runoff/availability</em>: These are more appropriate for representing water stress.</li>
35
-
<li><p><em>Number of of rainy/dry days, or moments of the precipitation distribution</em>: The distribution of precipitation often matters more than total.</p>
36
-
<p>Precipitation is an important control to include, even if it’s not the main variable of interest, since temperature and precipitation are correlated. However, we should remember that the properties of precipitation and temperature variables are very different in the way they affect humans. For example, binning of annual temperature variable, keeping high temperature bins small-sized, can explain variation in death rates due to heat waves events. However, if we want to see the variation in death rates due to storm events,using binned annual precipitation is likely not going to give us the variation in death rates, rather we would have to separately account for storm events by using an additional control.</p>
37
-
</li>
36
+
<li><em>Number of of rainy/dry days, or moments of the precipitation distribution</em>: The distribution of precipitation often matters more than total. </li>
38
37
</ol>
39
38
</li>
40
39
<li><p><strong>River discharge rate:</strong> River flows are generally measured at the station-level. While runoff is avaialble in gridded products, it is not a good reflection of water availability. Hydrological models (like VIC) can translate precipitation into river discharges across a region.</p>
41
40
</li>
42
41
<li><p><strong>Wind speed:</strong> The process of interest determines how wind speeds should be measured. For example, normal speeds are important for agriculture, squared speeds for distructive force, and cubic speeds for wind turbine power. Also consider gust velocity, which is generally available.</p>
43
42
</li>
44
-
<li><p><strong>Net primary productivity (NPP):</strong> It is the difference of amount of carbon dioxide that vegetation takes in during photosynthesis and the amount of carbon dioxide released during respiration. The data come from MODIS on NASA’s Terra satellite. Values range from near $0 g of carbon/m^2/day$ (tan) to $6.5 g of carbon/m^2/day$ (dark green). A negative value means decomposition or respiration overpowered carbon absorption; more carbon was released to the atmosphere than the plants took in.</p>
43
+
<li><p><strong>Net primary productivity (NPP):</strong> It is the difference of amount of carbon dioxide that vegetation takes in during photosynthesis and the amount of carbon dioxide released during respiration. The data come from MODIS on NASA’s Terra satellite. Values range from near 0 g of carbon/area/day (tan) to 6.5 g of carbon/area/day (dark green). A negative value means decomposition or respiration overpowered carbon absorption; more carbon was released to the atmosphere than the plants took in.</p>
45
44
</li>
46
45
<li><p><strong>Evapotranspiration rate (ET):</strong> It is the sum of evaporation and plant transpiration from the Earth's land and ocean surface to the atmosphere. Changes in ET is estimated using water stress measure in plants, thereby relating to the agricultural productivity measurement.</p>
47
46
</li>
@@ -92,7 +91,7 @@ <h2 id="2.2.-Common-functional-forms-(pros,-cons,-and-methods)">2.2. Common func
92
91
<li>More independence compared to poly in choosing function knots</li>
93
92
<li><p>Highly parametric due to freedom of choice of knots</p>
94
93
<p>For transforming the temperature data into restricted cubic splines, we need to fix the location and the number of knots. The reference above on cubic splines can be helpful in deciding the knot specifications. As before let the grid $\theta$ temperature be $T_{\theta i t}$. Let us do this exercise for $n$ knots, placed at $t_1<t_2<...<t_n$, then for $T_{\theta i t}$, which is a continuous variable, we have a set of $(n-2)$ new variables. We have:<br>
95
-
$$f(T_{i t}^k)= \sum_{\theta \in \Theta} \psi_{\theta}*\{(T_{\theta i t}-t_k)^3_+ - (T_{\theta i t} - t_{n- 1})^3_+*\frac{t_n-t_k}{t_n-t_{n-1}}+(T_{\theta i t} - t_{n})^3_+*\frac{t_{n-1}-t_k}{t_{n}-t_{n-1}}\}$$ $$\forall k \in {1,2,...,n-2\}$$<br>
94
+
$$f(T_{i t}^k)= \sum_{\theta \in \Theta} \psi_{\theta}*\{(T_{\theta i t}-t_k)^3_+ - (T_{\theta i t} - t_{n- 1})^3_+*\frac{t_n-t_k}{t_n-t_{n-1}}+(T_{\theta i t} - t_{n})^3_+*\frac{t_{n-1}-t_k}{t_{n}-t_{n-1}}\}$$ $$\forall k \in \{1,2,...,n-2\}$$
96
95
where, $\psi_{\theta}$ is the weight assigned to the $\theta$ grid.</p>
97
96
<p>And, each spline term in the parentheses $(\nabla)^3_+$ e.g. $(T_{\theta i t} - t_{n-1})^3_+$ is called a truncated polynomial of degree 3, which is defined as follows:<br>
98
97
$\nabla^3_+=\nabla^3_+$ if $\nabla^3_+>0$<br>
@@ -107,11 +106,11 @@ <h2 id="2.2.-Common-functional-forms-(pros,-cons,-and-methods)">2.2. Common func
107
106
<li>Fitting a line between cutoff values e.g. 25C CDD/0C HDD for temp</li>
108
107
<li>Less parametric and very useful for predicting mid-range response</li>
109
108
<li><p>Linear and highly sensitive to choice of cutoff values</p>
110
-
<p>Linear spline is a special kind of spline function, which has two knots, and the segment between these two knots is a linear function. It is also called ‘restricted’ linear spline, since the segments outside the knots are also linear. To implement this, we first decide location of the two knots, say $t_1<t_2$. Then, closely following the cubic spline method, we get:<br>
109
+
<p>Linear spline is a special kind of spline function, which has two knots, and the segment between these two knots is a linear function. It is also called ‘restricted’ linear spline, since the segments outside the knots are also linear. To implement this, we first decide location of the two knots, say $t_1<t_2$. Then, closely following the cubic spline method, we get:<br>
111
110
$$f(T_{it}^1)=\sum_{\theta \in \Theta} \psi_{\theta}*(T_{\theta i t}-t_2)_+$$<br>
112
111
$$f(T_{it}^2)=-\sum_{\theta \in \Theta} \psi_{\theta}*(T_{\theta i t}-t_1)_+$$<br>
113
112
where, $\psi_{\theta}$ is the weight assigned to the $\theta$ grid.</p>
114
-
<p>And, each spline term in the parentheses $(\nabla)_+$ e.g. $(T_{\theta i t} - t_2)_+$ is called a truncated polynomial of degree 1, which is defined as follows:<br>
113
+
<p>And, each spline term in the parentheses $(\nabla)_+$ e.g. $(T_{\theta i t} - t_2)_+$ is called a truncated polynomial of degree 1, which is defined as follows:<br>
115
114
$\nabla_+=\nabla_+$ if $\nabla_+>0$<br>
116
115
$\nabla_+=0$ if $\nabla_+<0$</p>
117
116
<p>The aggregate transformation is as below:<br>
@@ -143,8 +142,8 @@ <h3 id="Transformation-before-aggregation">Transformation-before-aggregation<a c
143
142
g_k(T_{ps})$$</p>
144
143
<p>We can rearrange this to<br>
145
144
$$y_i = \beta_1 (\sum_{\text{sj in it}} g_1(T_{ps})) + \beta_2 (\sum_{\text{sj in it}} g_2(T_{ps})) + \cdots + \beta_k (\sum_{\text{sj in it}} g_k(T_{ps}))$$</p>
146
-
<p>Or, more simply, $$y_i = \beta<em>1 N</em>{it} g<em>1(T</em>{ps}) + \beta<em>2
<p>where $N_{it}$ is the number of agent-timestep observations represented within region $i$ and reporting period $t$.</p>
149
148
<h3id="Aggregation-before-transformation">Aggregation-before-transformation<aclass="anchor-link" href="#Aggregation-before-transformation"></a></h3><p>When an economic process is occurring at the county level, we need to first do the weather variable aggregation at the county level. We do the weather variable transformation after we have aggregated it to the county level using weighted averaging method, and then run our estimation on the county level data. For example, to estimate the effect of storm events on public service employment at the administrative block level, we need to take into account the fact that hiring/firing of public service employees happens at the block level only. Estimating grid-level effects will lead to wrong estimation, as it would result in zero estimate for those (almost all) grid cells which do not have the block office coordinates, and extremely large values for those (very few) cells, which comprise of the block office coordinates. The mathematical formulation for aggregation-before-transformation can be learned through transformation-before-aggregation formulation described above, with a change that the aggregation step precedes the transformation step. Weather data products can have temporal resolution finer than scale of daily observations. Like spatial aggregation, we can do temporal aggregation to month, year, or decade; however, unlike spatial aggregation, the averaging process is standard in all general cases.</p>
Copy file name to clipboardExpand all lines: _build/suggestions.html
+5-4Lines changed: 5 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,20 @@
1
1
---
2
2
title: |-
3
3
Suggestions for work organization
4
-
pagenum: 5
5
4
prev_page:
6
5
url: /geographical-unit-data.html
6
+
title: |-
7
+
Generating geographical unit data
7
8
next_page:
8
-
url:
9
+
url: /contributors.html
10
+
title: |-
11
+
Contributors
9
12
suffix: .md
10
-
search: data analysis file files project version control bash code single should sh want original recommend workflow command example master script python runall echo dataset directory typically not often steps multiple naming only csv readme git automation py finished relative absolute paths panel organization containing makes easier things also case try keep separate information datasets results formatted figures predictors create possible times storage immutable source changes important good folders contents informative e g names sequence cleandata documentation additional using work swcarpentry github io novice here png defines called written path input suggestions producing having subdirectories zip everything send someone else however
11
13
12
14
comment: "***PROGRAMMATICALLY GENERATED, DO NOT EDIT. SEE ORIGINAL FILES IN /content***"
0 commit comments