|
11 | 11 | "cell_type": "markdown", |
12 | 12 | "metadata": { |
13 | 13 | "colab_type": "text", |
14 | | - "id": "S5Uhzt6vVIB2" |
| 14 | + "id": "" |
15 | 15 | }, |
16 | 16 | "source": [ |
17 | 17 | "<table align=\"center\">\n", |
|
33 | 33 | "cell_type": "markdown", |
34 | 34 | "metadata": { |
35 | 35 | "colab_type": "text", |
36 | | - "id": "FbVhjPpzn6BM" |
| 36 | + "id": "" |
37 | 37 | }, |
38 | 38 | "source": [ |
39 | 39 | "# Deep Learning Basics\n", |
|
80 | 80 | "metadata": { |
81 | 81 | "colab": {}, |
82 | 82 | "colab_type": "code", |
83 | | - "id": "dzLKpmZICaWN" |
| 83 | + "id": "" |
84 | 84 | }, |
85 | 85 | "outputs": [ |
86 | 86 | { |
|
149 | 149 | "cell_type": "markdown", |
150 | 150 | "metadata": { |
151 | 151 | "colab_type": "text", |
152 | | - "id": "59veuiEZCaW4" |
| 152 | + "id": "" |
153 | 153 | }, |
154 | 154 | "source": [ |
155 | 155 | "### Build the model\n", |
|
169 | 169 | "metadata": { |
170 | 170 | "colab": {}, |
171 | 171 | "colab_type": "code", |
172 | | - "id": "Lhan11blCaW7" |
| 172 | + "id": "" |
173 | 173 | }, |
174 | 174 | "outputs": [], |
175 | 175 | "source": [ |
|
189 | 189 | "cell_type": "markdown", |
190 | 190 | "metadata": { |
191 | 191 | "colab_type": "text", |
192 | | - "id": "qKF6uW-BCaW-" |
| 192 | + "id": "" |
193 | 193 | }, |
194 | 194 | "source": [ |
195 | 195 | "### Train the model\n", |
|
209 | 209 | "metadata": { |
210 | 210 | "colab": {}, |
211 | 211 | "colab_type": "code", |
212 | | - "id": "xvwvpA64CaW_" |
| 212 | + "id": "" |
213 | 213 | }, |
214 | 214 | "outputs": [ |
215 | 215 | { |
|
295 | 295 | "cell_type": "markdown", |
296 | 296 | "metadata": { |
297 | 297 | "colab_type": "text", |
298 | | - "id": "oEw4bZgGCaXB" |
| 298 | + "id": "" |
299 | 299 | }, |
300 | 300 | "source": [ |
301 | 301 | "Next, compare how the model performs on the test dataset:" |
|
307 | 307 | "metadata": { |
308 | 308 | "colab": {}, |
309 | 309 | "colab_type": "code", |
310 | | - "id": "VflXLEeECaXC" |
| 310 | + "id": "" |
311 | 311 | }, |
312 | 312 | "outputs": [ |
313 | 313 | { |
|
330 | 330 | "cell_type": "markdown", |
331 | 331 | "metadata": { |
332 | 332 | "colab_type": "text", |
333 | | - "id": "yWfgsmVXCaXG" |
| 333 | + "id": "" |
334 | 334 | }, |
335 | 335 | "source": [ |
336 | 336 | "Compare the RMSE measure you get to the [Kaggle leaderboard](https://www.kaggle.com/c/boston-housing/leaderboard). An RMSE of 2.651 puts us in 5th place." |
|
360 | 360 | "cell_type": "markdown", |
361 | 361 | "metadata": { |
362 | 362 | "colab_type": "text", |
363 | | - "id": "DLdCchMdCaWQ" |
| 363 | + "id": "" |
364 | 364 | }, |
365 | 365 | "source": [ |
366 | 366 | "The MNIST dataset containss 70,000 grayscale images of handwritten digits at a resolution of 28 by 28 pixels. The task is to take one of these images as input and predict the most likely digit contained in the image (along with a relative confidence in this prediction):\n", |
|
376 | 376 | "metadata": { |
377 | 377 | "colab": {}, |
378 | 378 | "colab_type": "code", |
379 | | - "id": "7MqDQO0KCaWS" |
| 379 | + "id": "" |
380 | 380 | }, |
381 | 381 | "outputs": [], |
382 | 382 | "source": [ |
|
391 | 391 | "cell_type": "markdown", |
392 | 392 | "metadata": { |
393 | 393 | "colab_type": "text", |
394 | | - "id": "Wz7l27Lz9S1P" |
| 394 | + "id": "" |
395 | 395 | }, |
396 | 396 | "source": [ |
397 | 397 | "We scale these values to a range of 0 to 1 before feeding to the neural network model. For this, we divide the values by 255. It's important that the *training set* and the *testing set* are preprocessed in the same way:" |
|
403 | 403 | "metadata": { |
404 | 404 | "colab": {}, |
405 | 405 | "colab_type": "code", |
406 | | - "id": "bW5WzIPlCaWv" |
| 406 | + "id": "" |
407 | 407 | }, |
408 | 408 | "outputs": [], |
409 | 409 | "source": [ |
|
420 | 420 | "cell_type": "markdown", |
421 | 421 | "metadata": { |
422 | 422 | "colab_type": "text", |
423 | | - "id": "Ee638AlnCaWz" |
| 423 | + "id": "" |
424 | 424 | }, |
425 | 425 | "source": [ |
426 | 426 | "Display the first 5 images from the *training set* and display the class name below each image. Verify that the data is in the correct format and we're ready to build and train the network." |
|
432 | 432 | "metadata": { |
433 | 433 | "colab": {}, |
434 | 434 | "colab_type": "code", |
435 | | - "id": "oZTImqg_CaW1" |
| 435 | + "id": "" |
436 | 436 | }, |
437 | 437 | "outputs": [ |
438 | 438 | { |
|
461 | 461 | "cell_type": "markdown", |
462 | 462 | "metadata": { |
463 | 463 | "colab_type": "text", |
464 | | - "id": "59veuiEZCaW4" |
| 464 | + "id": "" |
465 | 465 | }, |
466 | 466 | "source": [ |
467 | 467 | "### Build the model\n", |
|
475 | 475 | "metadata": { |
476 | 476 | "colab": {}, |
477 | 477 | "colab_type": "code", |
478 | | - "id": "9ODch-OFCaW4" |
| 478 | + "id": "" |
479 | 479 | }, |
480 | 480 | "outputs": [], |
481 | 481 | "source": [ |
|
502 | 502 | "cell_type": "markdown", |
503 | 503 | "metadata": { |
504 | 504 | "colab_type": "text", |
505 | | - "id": "gut8A_7rCaW6" |
| 505 | + "id": "" |
506 | 506 | }, |
507 | 507 | "source": [ |
508 | 508 | "Before the model is ready for training, it needs a few more settings. These are added during the model's *compile* step:\n", |
|
518 | 518 | "metadata": { |
519 | 519 | "colab": {}, |
520 | 520 | "colab_type": "code", |
521 | | - "id": "Lhan11blCaW7" |
| 521 | + "id": "" |
522 | 522 | }, |
523 | 523 | "outputs": [], |
524 | 524 | "source": [ |
|
531 | 531 | "cell_type": "markdown", |
532 | 532 | "metadata": { |
533 | 533 | "colab_type": "text", |
534 | | - "id": "qKF6uW-BCaW-" |
| 534 | + "id": "" |
535 | 535 | }, |
536 | 536 | "source": [ |
537 | 537 | "### Train the model\n", |
|
551 | 551 | "metadata": { |
552 | 552 | "colab": {}, |
553 | 553 | "colab_type": "code", |
554 | | - "id": "xvwvpA64CaW_" |
| 554 | + "id": "" |
555 | 555 | }, |
556 | 556 | "outputs": [ |
557 | 557 | { |
|
579 | 579 | "cell_type": "markdown", |
580 | 580 | "metadata": { |
581 | 581 | "colab_type": "text", |
582 | | - "id": "W3ZVOhugCaXA" |
| 582 | + "id": "" |
583 | 583 | }, |
584 | 584 | "source": [ |
585 | 585 | "As the model trains, the loss and accuracy metrics are displayed. This model reaches an accuracy of about 98.68% on the training data." |
|
589 | 589 | "cell_type": "markdown", |
590 | 590 | "metadata": { |
591 | 591 | "colab_type": "text", |
592 | | - "id": "oEw4bZgGCaXB" |
| 592 | + "id": "" |
593 | 593 | }, |
594 | 594 | "source": [ |
595 | 595 | "### Evaluate accuracy\n", |
|
603 | 603 | "metadata": { |
604 | 604 | "colab": {}, |
605 | 605 | "colab_type": "code", |
606 | | - "id": "VflXLEeECaXC" |
| 606 | + "id": "" |
607 | 607 | }, |
608 | 608 | "outputs": [ |
609 | 609 | { |
|
627 | 627 | "cell_type": "markdown", |
628 | 628 | "metadata": { |
629 | 629 | "colab_type": "text", |
630 | | - "id": "yWfgsmVXCaXG" |
| 630 | + "id": "" |
631 | 631 | }, |
632 | 632 | "source": [ |
633 | 633 | "Often times, the accuracy on the test dataset is a little less than the accuracy on the training dataset. This gap between training accuracy and test accuracy is an example of *overfitting*. In our case, the accuracy is better at 99.19%! This is, in part, due to successful regularization accomplished with the Dropout layers." |
|
637 | 637 | "cell_type": "markdown", |
638 | 638 | "metadata": { |
639 | 639 | "colab_type": "text", |
640 | | - "id": "xsoS7CPDCaXH" |
| 640 | + "id": "" |
641 | 641 | }, |
642 | 642 | "source": [ |
643 | 643 | "### Make predictions\n", |
|
653 | 653 | "metadata": { |
654 | 654 | "colab": {}, |
655 | 655 | "colab_type": "code", |
656 | | - "id": "Gl91RPhdCaXI" |
| 656 | + "id": "" |
657 | 657 | }, |
658 | 658 | "outputs": [ |
659 | 659 | { |
|
771 | 771 | "cell_type": "markdown", |
772 | 772 | "metadata": { |
773 | 773 | "colab_type": "text", |
774 | | - "id": "x9Kk1voUCaXJ" |
| 774 | + "id": "" |
775 | 775 | }, |
776 | 776 | "source": [ |
777 | 777 | "The above shows the prediction of the network by choosing the neuron with the highest output. While the output layer values add 1 to one, these do not reflect well-calibrated measures of \"uncertainty\". Often, the network is overly confident about the top choice that does not reflect a learned measure of probability. If everything ran correctly you should get an animation like this:\n", |
|
793 | 793 | "colab": { |
794 | 794 | "collapsed_sections": [], |
795 | 795 | "name": "tutorial_deep_learning_basics.ipynb", |
796 | | - "private_outputs": true, |
797 | 796 | "provenance": [], |
798 | 797 | "toc_visible": true, |
799 | 798 | "version": "0.3.2" |
|
0 commit comments