Update notebooks (INRIA#616)

Author: lesteve

notebooks/cross_validation_grouping.ipynb

@@ -26,8 +26,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We will recreate the same model used in the previous exercise:\n",
-    "a logistic regression classifier with preprocessor to scale the data."
+    "We will recreate the same model used in the previous notebook:\n",
+    "a logistic regression classifier with a preprocessor to scale the data."
    ]
   },
   {
@@ -138,9 +138,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The cross-validation testing error that uses the shuffling has less\n",
-    "variance than the one that does not impose any shuffling. It means that some\n",
-    "specific fold leads to a low score in this case."
+    "The cross-validation testing error that uses the shuffling has less variance\n",
+    "than the one that does not impose any shuffling. It means that some specific\n",
+    "fold leads to a low score in this case."
    ]
   },
   {
@@ -176,11 +176,11 @@
    "source": [
     "If we read carefully, 13 writers wrote the digits of our dataset, accounting\n",
     "for a total amount of 1797 samples. Thus, a writer wrote several times the\n",
-    "same numbers. Let's suppose that the writer samples are grouped.\n",
-    "Subsequently, not shuffling the data will keep all writer samples together\n",
-    "either in the training or the testing sets. Mixing the data will break this\n",
-    "structure, and therefore digits written by the same writer will be available\n",
-    "in both the training and testing sets.\n",
+    "same numbers. Let's suppose that the writer samples are grouped. Subsequently,\n",
+    "not shuffling the data will keep all writer samples together either in the\n",
+    "training or the testing sets. Mixing the data will break this structure, and\n",
+    "therefore digits written by the same writer will be available in both the\n",
+    "training and testing sets.\n",
     "\n",
     "Besides, a writer will usually tend to write digits in the same manner. Thus,\n",
     "our model will learn to identify a writer's pattern for each digit instead of\n",
@@ -209,14 +209,14 @@
     "\n",
     "It might not be obvious at first, but there is a structure in the target:\n",
     "there is a repetitive pattern that always starts by some series of ordered\n",
-    "digits from 0 to 9 followed by random digits at a certain point. If we look\n",
-    "in details, we see that there is 14 such patterns, always with around 130\n",
-    "samples each.\n",
+    "digits from 0 to 9 followed by random digits at a certain point. If we look in\n",
+    "detail, we see that there are 14 such patterns, always with around 130 samples\n",
+    "each.\n",
     "\n",
-    "Even if it is not exactly corresponding to the 13 writers in the\n",
-    "documentation (maybe one writer wrote two series of digits), we can\n",
-    "make the hypothesis that each of these patterns corresponds to a different\n",
-    "writer and thus a different group."
+    "Even if it is not exactly corresponding to the 13 writers in the documentation\n",
+    "(maybe one writer wrote two series of digits), we can make the hypothesis that\n",
+    "each of these patterns corresponds to a different writer and thus a different\n",
+    "group."
    ]
   },
   {
@@ -326,8 +326,8 @@
    "metadata": {},
    "source": [
     "As a conclusion, it is really important to take any sample grouping pattern\n",
-    "into account when evaluating a model. Otherwise, the results obtained will\n",
-    "be over-optimistic in regards with reality."
+    "into account when evaluating a model. Otherwise, the results obtained will be\n",
+    "over-optimistic in regards with reality."
    ]
   }
  ],

notebooks/cross_validation_time.ipynb

@@ -20,8 +20,8 @@
     "depends on past information.\n",
     "\n",
     "We will take an example to highlight such issues with non-i.i.d. data in the\n",
-    "previous cross-validation strategies presented. We are going to load\n",
-    "financial quotations from some energy companies."
+    "previous cross-validation strategies presented. We are going to load financial\n",
+    "quotations from some energy companies."
    ]
   },
   {
@@ -70,14 +70,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We will repeat the experiment asked during the exercise. Instead of using\n",
-    "random data, we will use real quotations this time. While it was obvious that\n",
-    "a predictive model could not work in practice on random data, this is the\n",
-    "same on these real data. So here, we want to predict the quotation of Chevron\n",
-    "using all other energy companies' quotes.\n",
-    "\n",
-    "To make explanatory plots, we will use a single split in addition to the\n",
-    "cross-validation that you used in the introductory exercise."
+    "Here, we want to predict the quotation of Chevron using all other energy\n",
+    "companies' quotes. To make explanatory plots, we first use a train-test split\n",
+    "and then we evaluate other cross-validation methods."
    ]
   },
   {
@@ -158,9 +153,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Surprisingly, we get outstanding generalization performance. We will investigate\n",
-    "and find the reason for such good results with a model that is expected to\n",
-    "fail. We previously mentioned that `ShuffleSplit` is an iterative\n",
+    "Surprisingly, we get outstanding generalization performance. We will\n",
+    "investigate and find the reason for such good results with a model that is\n",
+    "expected to fail. We previously mentioned that `ShuffleSplit` is an iterative\n",
     "cross-validation scheme that shuffles data and split. We will simplify this\n",
     "procedure with a single split and plot the prediction. We can use\n",
     "`train_test_split` for this purpose."
@@ -228,10 +223,10 @@
     "testing. But we can also see that the testing samples are next to some\n",
     "training sample. And with these time-series, we see a relationship between a\n",
     "sample at the time `t` and a sample at `t+1`. In this case, we are violating\n",
-    "the i.i.d. assumption. The insight to get is the following: a model can\n",
-    "output of its training set at the time `t` for a testing sample at the time\n",
-    "`t+1`. This prediction would be close to the true value even if our model\n",
-    "did not learn anything, but just memorized the training dataset.\n",
+    "the i.i.d. assumption. The insight to get is the following: a model can output\n",
+    "of its training set at the time `t` for a testing sample at the time `t+1`.\n",
+    "This prediction would be close to the true value even if our model did not\n",
+    "learn anything, but just memorized the training dataset.\n",
     "\n",
     "An easy way to verify this hypothesis is to not shuffle the data when doing\n",
     "the split. In this case, we will use the first 75% of the data to train and\n",
@@ -292,7 +287,8 @@
    "source": [
     "We see that our model cannot predict anything because it doesn't have samples\n",
     "around the testing sample. Let's check how we could have made a proper\n",
-    "cross-validation scheme to get a reasonable generalization performance estimate.\n",
+    "cross-validation scheme to get a reasonable generalization performance\n",
+    "estimate.\n",
     "\n",
     "One solution would be to group the samples into time blocks, e.g. by quarter,\n",
     "and predict each group's information by using information from the other\n",
@@ -324,8 +320,8 @@
     "\n",
     "Another thing to consider is the actual application of our solution. If our\n",
     "model is aimed at forecasting (i.e., predicting future data from past data),\n",
-    "we should not use training data that are ulterior to the testing data. In\n",
-    "this case, we can use the `TimeSeriesSplit` cross-validation to enforce this\n",
+    "we should not use training data that are ulterior to the testing data. In this\n",
+    "case, we can use the `TimeSeriesSplit` cross-validation to enforce this\n",
     "behaviour."
    ]
   },
@@ -349,9 +345,9 @@
    "metadata": {},
    "source": [
     "In conclusion, it is really important to not use an out of the shelves\n",
-    "cross-validation strategy which do not respect some assumptions such as\n",
-    "having i.i.d data. It might lead to absurd results which could make think\n",
-    "that a predictive model might work."
+    "cross-validation strategy which do not respect some assumptions such as having\n",
+    "i.i.d data. It might lead to absurd results which could make think that a\n",
+    "predictive model might work."
    ]
   }
  ],

notebooks/parameter_tuning_nested.ipynb

@@ -245,7 +245,7 @@
     "hyper-parameters and to train the refitted model.\n",
     "\n",
     "Because of the above, one must keep an external, held-out test set for the\n",
-    "final evaluation the refitted model. We highlight here the process using a\n",
+    "final evaluation of the refitted model. We highlight here the process using a\n",
     "single train-test split."
    ]
   },
@@ -316,7 +316,7 @@
     "refitted tuned model.\n",
     "\n",
     "In practice, we only need to embed the grid-search in the function\n",
-    "`cross-validate` to perform such evaluation."
+    "`cross_validate` to perform such evaluation."
    ]
   },
   {