typos in readmes

Author: vpj

docs/normalization/batch_norm/readme.html

@@ -82,11 +82,11 @@ <h3>Internal Covariate Shift</h3>
 For example, let&rsquo;s say there are two layers $l_1$ and $l_2$.
 During the beginning of the training $l_1$ outputs (inputs to $l_2$)
 could be in distribution $\mathcal{N}(0.5, 1)$.
-Then, after some training steps, it could move to $\mathcal{N}(0.5, 1)$.
+Then, after some training steps, it could move to $\mathcal{N}(0.6, 1.5)$.
 This is <em>internal covariate shift</em>.</p>
 <p>Internal covariate shift will adversely affect training speed because the later layers
-($l_2$ in the above example) has to adapt to this shifted distribution.</p>
-<p>By stabilizing the distribution batch normalization minimizes the internal covariate shift.</p>
+($l_2$ in the above example) have to adapt to this shifted distribution.</p>
+<p>By stabilizing the distribution, batch normalization minimizes the internal covariate shift.</p>
 <h2>Normalization</h2>
 <p>It is known that whitening improves training speed and convergence.
 <em>Whitening</em> is linearly transforming inputs to have zero mean, unit variance,
@@ -95,9 +95,9 @@ <h3>Normalizing outside gradient computation doesn&rsquo;t work</h3>
 <p>Normalizing outside the gradient computation using pre-computed (detached)
 means and variances doesn&rsquo;t work. For instance. (ignoring variance), let
 <script type="math/tex; mode=display">\hat{x} = x - \mathbb{E}[x]</script>
-where $x = u + b$ and $b$ is a trained bias.
-and $\mathbb{E}[x]$ is outside gradient computation (pre-computed constant).</p>
-<p>Note that $\hat{x}$ has no effect of $b$.
+where $x = u + b$ and $b$ is a trained bias
+and $\mathbb{E}[x]$ is an outside gradient computation (pre-computed constant).</p>
+<p>Note that $\hat{x}$ has no effect on $b$.
 Therefore,
 $b$ will increase or decrease based
 $\frac{\partial{\mathcal{L}}}{\partial x}$,
@@ -106,14 +106,14 @@ <h3>Normalizing outside gradient computation doesn&rsquo;t work</h3>
 <h3>Batch Normalization</h3>
 <p>Whitening is computationally expensive because you need to de-correlate and
 the gradients must flow through the full whitening calculation.</p>
-<p>The paper introduces simplified version which they call <em>Batch Normalization</em>.
+<p>The paper introduces a simplified version which they call <em>Batch Normalization</em>.
 First simplification is that it normalizes each feature independently to have
 zero mean and unit variance:
 <script type="math/tex; mode=display">\hat{x}^{(k)} = \frac{x^{(k)} - \mathbb{E}[x^{(k)}]}{\sqrt{Var[x^{(k)}]}}</script>
 where $x = (x^{(1)} &hellip; x^{(d)})$ is the $d$-dimensional input.</p>
 <p>The second simplification is to use estimates of mean $\mathbb{E}[x^{(k)}]$
 and variance $Var[x^{(k)}]$ from the mini-batch
-for normalization; instead of calculating the mean and variance across whole dataset.</p>
+for normalization; instead of calculating the mean and variance across the whole dataset.</p>
 <p>Normalizing each feature to zero mean and unit variance could affect what the layer
 can represent.
 As an example paper illustrates that, if the inputs to a sigmoid are normalized
@@ -126,17 +126,17 @@ <h3>Batch Normalization</h3>
 like $Wu + b$ the bias parameter $b$ gets cancelled due to normalization.
 So you can and should omit bias parameter in linear transforms right before the
 batch normalization.</p>
-<p>Batch normalization also makes the back propagation invariant to the scale of the weights.
-And empirically it improves generalization, so it has regularization effects too.</p>
+<p>Batch normalization also makes the back propagation invariant to the scale of the weights
+and empirically it improves generalization, so it has regularization effects too.</p>
 <h2>Inference</h2>
 <p>We need to know $\mathbb{E}[x^{(k)}]$ and $Var[x^{(k)}]$ in order to
 perform the normalization.
 So during inference, you either need to go through the whole (or part of) dataset
 and find the mean and variance, or you can use an estimate calculated during training.
 The usual practice is to calculate an exponential moving average of
 mean and variance during the training phase and use that for inference.</p>
-<p>Here&rsquo;s <a href="https://nn.labml.ai/normalization/layer_norm/mnist.html">the training code</a> and a notebook for training
-a CNN classifier that use batch normalization for MNIST dataset.</p>
+<p>Here&rsquo;s <a href="mnist.html">the training code</a> and a notebook for training
+a CNN classifier that uses batch normalization for MNIST dataset.</p>
 <p><a href="https://colab.research.google.com/github/lab-ml/nn/blob/master/labml_nn/normalization/batch_norm/mnist.ipynb"><img alt="Open In Colab" src="https://colab.research.google.com/assets/colab-badge.svg" /></a>
 <a href="https://web.lab-ml.com/run?uuid=011254fe647011ebbb8e0242ac1c0002"><img alt="View Run" src="https://img.shields.io/badge/labml-experiment-brightgreen" /></a></p>
             </div>