Get the full ebook with Bonus Features for a Better Reading Experience on ebookmeta.

com

Ensemble Learning for AI Developers Learn Bagging

Stacking and Boosting Methods with Use Cases Alok
Kumar Mayank Jain

https://ebookmeta.com/product/ensemble-learning-for-ai-
developers-learn-bagging-stacking-and-boosting-methods-with-
use-cases-alok-kumar-mayank-jain/

OR CLICK HERE

DOWLOAD NOW

Download more ebook instantly today at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Ensemble Methods for Machine Learning - MEAP Version 6

Gautam Kunapuli

https://ebookmeta.com/product/ensemble-methods-for-machine-learning-
meap-version-6-gautam-kunapuli/

ebookmeta.com

Deep Learning Approach for Natural Language Processing,

Speech, and Computer Vision: Techniques and Use Cases 1st
Edition L. Ashok Kumar
https://ebookmeta.com/product/deep-learning-approach-for-natural-
language-processing-speech-and-computer-vision-techniques-and-use-
cases-1st-edition-l-ashok-kumar/
ebookmeta.com

Digital Supply Chain and Logistics with IoT Practical

Guide Methods Tools and Use Cases for Industry Andreas
Holtschulte
https://ebookmeta.com/product/digital-supply-chain-and-logistics-with-
iot-practical-guide-methods-tools-and-use-cases-for-industry-andreas-
holtschulte/
ebookmeta.com

Next Generation Electrified Vehicles Optimised for the

Infrastructure 1st Edition Bernhard Brandstätter

https://ebookmeta.com/product/next-generation-electrified-vehicles-
optimised-for-the-infrastructure-1st-edition-bernhard-brandstatter/

ebookmeta.com
The Guest is a Goner Sedona Spirit Cozy Mystery 1 1st
Edition Carly Winter

https://ebookmeta.com/product/the-guest-is-a-goner-sedona-spirit-cozy-
mystery-1-1st-edition-carly-winter/

ebookmeta.com

Handbook of the Anthropocene Humans between Heritage and

Future Nathanaël Wallenhorst

https://ebookmeta.com/product/handbook-of-the-anthropocene-humans-
between-heritage-and-future-nathanael-wallenhorst/

ebookmeta.com

Essential Math for AI (Fifth Early Release) Hala Nelson

https://ebookmeta.com/product/essential-math-for-ai-fifth-early-
release-hala-nelson/

ebookmeta.com

The Vulnerable Observer Anthropology That Breaks Your

Heart 25th anniversary edition Ruth Behar

https://ebookmeta.com/product/the-vulnerable-observer-anthropology-
that-breaks-your-heart-25th-anniversary-edition-ruth-behar/

ebookmeta.com

Machine Learning for Data Science Handbook: Data Mining

and Knowledge Discovery Handbook, 3rd Lior Rokach

https://ebookmeta.com/product/machine-learning-for-data-science-
handbook-data-mining-and-knowledge-discovery-handbook-3rd-lior-rokach/

ebookmeta.com
Scar High Heat BBW Mountain Man Instalove 1st Edition
Kelsie Calloway

https://ebookmeta.com/product/scar-high-heat-bbw-mountain-man-
instalove-1st-edition-kelsie-calloway-2/

ebookmeta.com
Alok Kumar and Mayank Jain

Ensemble Learning for AI Developers

Learn Bagging, Stacking, and Boosting Methods
with Use Cases
Alok Kumar
Gurugram, India

Mayank Jain
Gurugram, India

Any source code or other supplementary material referenced by the

author in this book is available to readers on GitHub via the book’s
product page, located at https://​www.​apress.​com/​us/​book/​978-1-
4842-5939-9. For more detailed information, please visit http://​www.​
apress.​com/​source-code.

ISBN 978-1-4842-5939-9 e-ISBN 978-1-4842-5940-5

https://doi.org/10.1007/978-1-4842-5940-5

© Alok Kumar and Mayank Jain 2020

Standard Apress

The use of general descriptive names, registered names, trademarks,

service marks, etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the
relevant protective laws and regulations and therefore free for general
use.

The publisher, the authors and the editors are safe to assume that the
advice and information in this book are believed to be true and accurate
at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, expressed or implied, with respect to the
material contained herein or for any errors or omissions that may have
been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.
Distributed to the book trade worldwide by Springer Science+Business
Media New York, 233 Spring Street, 6th Floor, New York, NY 10013.
Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-
[email protected], or visit www.springeronline.com. Apress Media,
LLC is a California LLC and the sole member (owner) is Springer
Science + Business Media Finance Inc (SSBM Finance Inc). SSBM
Finance Inc is a Delaware corporation.
This book is dedicated to my family for their unfettered support.
—Alok Kumar
To my family and Anand Sir, who unbounded my mind.
—Mayank Jain
Introduction
Ensemble learning is fast becoming a popular choice for machine
learning models in the data science world. Ensemble methods combine
the output of machine learning models in many interesting ways. Even
after years of working on machine learning projects, we were unaware
of the power of ensemble methods, as this topic is usually neglected or
only given a brief overview in most machine learning courses and
books. Like many others, we came to know about the power of
ensemble methods by checking competitive machine learning
scenarios. Competitive machine learning platforms, like Kaggle, offer an
unbiased review of machine learning techniques. For the past few
years, ensemble learning methods have consistently outperformed
competitive metrics. This itself speaks to the benefit of learning
ensembling techniques. The objective of this book is to help you learn
ensembles and apply them effectively in real-world situations.
This book starts by explaining why ensemble learning techniques
are needed. The objective of Chapter 1 is to build a basic understanding
of various ensembling techniques. Chapters 2, 3, and 4, cover various
ensemble techniques and are grouped by how we mix training data,
models, and combinations. In these chapters, you learn about some of
the most important ensemble learning techniques like random forests,
bagging, bootstrap aggregating, stacking, and cross validation methods.
Chapter 5 covers ensemble learning libraries. The libraries help you
accelerate your experimentation and implementation. Chapter 6 covers
techniques for integrating ensembles into real-world machine learning
workflows.
This book presents a condensed, easy-to-understand way to learn
and apply ensemble learning concepts in real-world use-cases, which
otherwise takes a lot of back and forth learning for a budding data
scientist. The code in this book (Python scripts) can be used as an
extension to your projects or for reference.
Acknowledgments
We would really like to thank the team at Apress for all their help and
support in our journey writing this book. It has been a pleasure to write
this book, and the team at Apress are certainly a big part of that. Many
thanks to Acquisitions Editor, Celestin Suresh John, who was a great
help in narrowing the scope of this project and making it more
accessible. Many thanks to our development coordinators and our
editors, Aditee Mirashi and Laura C. Berendson. Their continuous
follow-up helped us tremendously with keeping us on track and
focused. We are also thankful to Ashutosh Parida for his valuable
technical feedback.
As always, we would like to thank our family for all their help and
support. Without their support and assistance, we couldn’t have
completed this book.
Lastly, we would like to thank the open source community, who has
helped democratize the knowledge. All the libraries used in this book
come from open source projects.
Table of Contents
Chapter 1:​Why Ensemble Techniques Are Needed
Mixing Training Data
Mixing Combinations
Mixing Models
Summary
Chapter 2:​Mixing Training Data
Decision Trees
Sampling a Dataset
Sampling Without Replacement (WOR)
Sampling with Replacement (WR)
Bagging (Bootstrap Aggregating)
k-Fold Cross-Validation
Stratified k-Fold Cross-Validation
Summary
Chapter 3:​Mixing Models
Voting Ensembles
Hard Voting
Averaging/​Soft Voting
Hyperparameter Tuning Ensembles
Horizontal Voting Ensembles
Snapshot Ensembles
Summary
Chapter 4:​Mixing Combinations
Boosting
AdaBoost
 Gradient Boosting
XGBoost
Stacking
Summary
Chapter 5:​Using Ensemble Learning Libraries
ML-Ensemble
Multilayer Ensembles
Ensemble Model Selection
Scale XGBoost via Dask
Dask Arrays and Dataframes
Dask-ML
Scaling XGBoost
Microsoft LightGBM
AdaNet
Summary
Chapter 6:​Tips and Best Practices
Feature Selection Using Random Forests
Feature Transformations with Ensembles of Trees
Building a Preprocessing Pipeline for a Random Forest
Classifier
Isolation Forest for Outlier Detection
Scaling Ensembles with Dask
Preprocessing
Hyperparameter Search
Distributed Ensemble Fitting
Summary
 Thank You
Index
About the Authors
Alok Kumar
is an AI practitioner and Innovation Lead
at Publicis Sapient. He has extensive
experience in leading strategic initiatives
and driving cutting-edge, fast-paced
innovations. His work has won several
awards. Alok is passionate about
democratizing AI knowledge. He
manages multiple nonprofit learning and
creative groups in National capital region
of india NCR. Find him on LinkedIn at
https://in.linkedin.com/in/al
oksaan .

Mayank Jain
is a technology manager and AI/ML
expert at the Publicis Sapient Kepler
Innovation Lab. He has more than 10
years of experience working on cutting-
edge projects that involve making
computers see and think by using
techniques like deep learning, machine
learning, and computer vision. He has
written for several international
publications, he has several patents in
his name, and has been awarded
multiple times for his contributions. Find
him on LinkedIn at
https://in.linkedin.com/in/mayankjain7 .
About the Technical Reviewer
Ashutosh Parida
is the head of analytics at Manipal Global
Education Services in Bangalore. He has
a decade of experience in data science,
including vision, natural-language
understanding, recommendation
engines, and forecasting. He has been
site lead for multiple projects and has
launched products that serve millions of
users. He also has open source
contributions to his credit. Ashutosh has
a bachelor’s degree in computers science
and engineering from IIIT-Hyderabad. He
has been in the IT industry for 15 years,
having worked with Oracle, Samsung,
Akamai, and Qualcomm. Find him on
LinkedIn at
https://www.linkedin.com/in/ashutoshparida/ .
© Alok Kumar and Mayank Jain 2020
A. Kumar, M. Jain, Ensemble Learning for AI Developers
https://doi.org/10.1007/978-1-4842-5940-5_1

1. Why Ensemble Techniques Are

Needed
Alok Kumar1 and Mayank Jain1
(1) Gurugram, India

According to the Cambridge dictionary, an ensemble is defined as a

group of things or people acting or taken together as a whole. The
word ensemble was first used in the context of musicians who
regularly play together. An ensemble of musicians is the sum of
individual compositions by multiple musicians. Similarly, in machine
learning, ensemble learning is a combination of multiple machine
learning techniques performed together.
Ensemble learning is fast becoming a go-to standard for getting an
accuracy boost in machine learning models in data science. Ensemble
methods do this through combining the output of machine learning
models in many interesting ways; just like an ensemble of musicians
combine their individual performances in multiple different ways to
achieve a great composition.
Your role as a data scientist is to become a concertmaster or
architect who can leverage the strength of individual machine learning
(ML) models and combine them in interesting ways to achieve the ML
equivalent of a great composition (i.e., a world-class machine learning
model).
Let’s look at another analogy to build a better understanding of the
benefit of ensemble methods. Let’s suppose that you want to invest in
the stock market. You are interested in a particular stock, but you are
not sure about its future outlook, and so you decide to seek advice. You
reach out to a financial advisor who has a 75% accuracy in making the
right predictions. You decide to check with other financial advisors,
who give you similar advice. In a case where each of the advisors
suggest that you to buy the stock, what is the accuracy rate of this
collective advice?
More often than not, the collective advice of multiple experts beats
the accuracy of any one advisor, especially in varied financial situations.
Similarly, in machine learning, ensemble methods from multiple
machine learning models tend to have a better generalized
performance than any single machine learning model, especially in
varied conditions/cases or in the long run.
In this book, we take you through ways to combine the output of
multiple machine learning models under the umbrella of ensemble
learning.
Ensemble learning techniques can be further divided under a broad
umbrella of three classes: mixing training data, mixing
combinations, and mixing models. We will briefly try to build your
understanding of each class.

Mixing Training Data

Anyone who has read about the evolution of species knows that it is
very important for any species to have sufficient genetic diversity.
Species that have less genetic diversity tend to die out, even if they are
fit for the current environment.
One of the reasons for this phenomenon is that even though a
species (or current training data) has become expert in its
environment, whenever there is an adverse environmental condition,
like a new lethal disease, it is not able to adapt, and the species
becomes prone to extinction.
One of the ways for any species to develop sufficient genetic
diversity is to divide its population and let them evolve inside different
environmental conditions. The reason for success in this approach is
that dividing the population into different groups and exposing them to
different environments leads them to evolve based on the new
environments, which leads to increased genetic diversity. This prevents
the population from becoming homogeneous, and in adversity, it
ensures that at least some of the subgroups of species will survive.
 Taking this insight to machine learning leads us to the first
ensemble learning variant, which is Mixing training data . Instead of
training a single, large classifier on all the training data, we divide the
training data into multiple chunks, and train separate classifiers on
each subset of training data. In the end, the output of all of these
classifiers are combined (see Figure 1-1).
This approach ensures that classifiers capture sufficient diversity as
they are trained (evolve) on a subset of the population. By combining
the output of these diverse learners, we are able to achieve superior
accuracy compared to a case in which we trained a single learner on a
whole population (training data).
This kind of division of training data is called bagging . You learn
more about this technique in Chapter 2.

Figure 1-1 Mixing training data using bagging

Mixing Combinations
To better understand what mixing, or varying, combinations mean in
the context of ensemble learning, let’s look at an analogy of the learning
process of students in a class.
Random documents with unrelated
content Scribd suggests to you:
 AT THE BOSTON BANQUET.

I N his Speech at the Annual Banquet of the Boston Merchants’

Association in December, 1889, Mr. Grady said:

Mr. President: Bidden by your invitation to a discussion of the

race problem—forbidden by occasion to make a political speech—I
appreciate in trying to reconcile orders with propriety the
predicament of the little maid who, bidden to learn to swim, was yet
adjured, “Now, go, my darling, hang your clothes on a hickory limb,
and don’t go near the water.”
The stoutest apostle of the church, they say, is the missionary, and
the missionary, wherever he unfurls his flag, will never find himself in
deeper need of unction and address than I, bidden to-night to plant
the standard of a Southern Democrat in Boston’s banquet hall, and
discuss the problem of the races in the home of Phillips and of
Sumner. But, Mr. President, if a purpose to speak in perfect
frankness and sincerity; if earnest understanding of the vast interests
involved; if a consecrating sense of what disaster may follow further
misunderstanding and estrangement, if these may be counted to
steady undisciplined speech and to strengthen an untried arm—then,
sir, I find the courage to proceed.
Happy am I that this mission has brought my feet at last to press
New England’s historic soil, and my eyes to the knowledge of her
beauty and her thrift. Here, within touch of Plymouth Rock and
Bunker Hill—where Webster thundered and Longfellow sang,
Emerson thought and Channing preached—here in the cradle of
American letters, and almost of American liberty, I hasten to make
the obeisance that every American owes New England when first he
stands uncovered in her mighty presence. Strange apparition! This
stern and unique figure—carved from the ocean and the wilderness
—its majesty kindling and growing amid the storms of winters and of
wars—until at last the gloom was broken, its beauty disclosed in the
sunshine, and the heroic workers rested at its base—while startled
kings and emperors gazed and marveled that from the rude touch of
this handful, cast on a bleak and unknown shore, should have come
the embodied genius of human government, and the perfected
model of human liberty! God bless the memory of those immortal
workers—and prosper the fortunes of their living sons—and
perpetuate the inspiration of their handiwork.
Two years ago, sir, I spoke some words in New York that caught
the attention of the North. As I stand here to reiterate, as I have done
everywhere, every word I then uttered—to declare that the
sentiments I then avowed were universally approved in the South—I
realize that the confidence begotten by that speech is largely
responsible for my presence here to-night. I should dishonor myself
if I betrayed that confidence by uttering one insincere word, or by
withholding one essential element of the truth. Apropos of this last,
let me confess, Mr. President—before the praise of New England
has died on my lips—that I believe the best product of her present
life is the procession of 17,000 Vermont Democrats that for twenty-
two years, undiminished by death, unrecruited by birth or conversion,
have marched over their rugged hills, cast their Democratic ballots,
and gone back home to pray for their unregenerate neighbors, and
awake to read the record of 26,000 Republican majority. May the
God of the helpless and the heroic help them—and may their sturdy
tribe increase!
Far to the south, Mr. President, separated from this section by a
line, once defined in irrepressible difference, once traced in fratricidal
blood, and now, thank God, but a vanishing shadow, lies the fairest
and richest domain of this earth. It is the home of a brave and
hospitable people. There, is centered all that can please or prosper
humankind. A perfect climate, above a fertile soil, yields to the
husbandman every product of the temperate zone. There, by night
the cotton whitens beneath the stars, and by day the wheat locks the
sunshine in its bearded sheaf. In the same field the clover steals the
fragrance of the wind, and the tobacco catches the quick aroma of
the rains. There, are mountains stored with exhaustless treasures;
forests, vast and primeval, and rivers that, tumbling or loitering, run
wanton to the sea. Of the three essential items of all industries—
cotton, iron and wool—that region has easy control. In cotton, a fixed
monopoly—in iron, proven supremacy—in timber, the reserve supply
of the Republic. From this assured and permanent advantage,
against which artificial conditions cannot much longer prevail, has
grown an amazing system of industries. Not maintained by human
contrivance of tariff or capital, afar off from the fullest and cheapest
source of supply, but resting in Divine assurance, within touch of field
and mine and forest—not set amid costly farms from which
competition has driven the farmer in despair, but amid cheap and
sunny lands, rich with agriculture, to which neither season nor soil
has set a limit—this system of industries is mounting to a splendor
that shall dazzle and illumine the world.
That, sir, is the picture and the promise of my home—a land better
and fairer than I have told you, and yet but fit setting, in its material
excellence, for the loyal and gentle quality of its citizenship. Against
that, sir, we have New England, recruiting the Republic from its
sturdy loins, shaking from its overcrowded hives new swarms of
workers and touching this land all over with its energy and its
courage. And yet, while in the Eldorado of which I have told you, but
15 per cent. of lands are cultivated, its mines scarcely touched and
its population so scant that, were it set equidistant, the sound of the
human voice could not be heard from Virginia to Texas—while on the
threshold of nearly every house in New England stands a son,
seeking with troubled eyes some new land in which to carry his
modest patrimony, the strange fact remains that in 1880 the South
had fewer Northern-born citizens than she had in 1870—fewer in ’70
than in ’60. Why is this? Why is it, sir, though the sectional line be
now but a mist that the breath may dispel, fewer men of the North
have crossed it over to the South than when it was crimson with the
best blood of the Republic, or even when the slaveholder stood
guard every inch of its way?
There can be but one answer. It is the very problem we are now to
consider. The key that opens that problem will unlock to the world
the fairest half of this Republic, and free the halted feet of thousands
whose eyes are already kindling with its beauty. Better than this, it
will open the hearts of brothers for thirty years estranged, and clasp
in lasting comradeship a million hands now withheld in doubt.
Nothing, sir, but this problem, and the suspicions it breeds, hinders a
clear understanding and a perfect union. Nothing else stands
between us and such love as bound Georgia and Massachusetts at
Valley Forge and Yorktown, chastened by the sacrifices at Manassas
and Gettysburg, and illumined with the coming of better work and a
nobler destiny than was ever wrought with the sword or sought at the
cannon’s mouth.
If this does not invite your patient hearing to-night—hear one thing
more. My people, your brothers in the South—brothers in blood, in
destiny, in all that is best in our past and future—are so beset with
this problem that their very existence depends upon its right solution.
Nor are they wholly to blame for its presence. The slave-ships of the
Republic sailed from your ports—the slaves worked in our fields. You
will not defend the traffic, nor I the institution. But I do hereby declare
that in its wise and humane administration, in lifting the slave to
heights of which he had not dreamed in his savage home, and giving
him a happiness he has not yet found in freedom—our fathers left
their sons a saving and excellent heritage. In the storm of war this
institution was lost. I thank God as heartily as you do that human
slavery is gone forever from the American soil. But the freedman
remains. With him a problem without precedent or parallel. Note its
appalling conditions. Two utterly dissimilar races on the same soil—
with equal political and civil rights—almost equal in numbers, but
terribly unequal in intelligence and responsibility—each pledged
against fusion—one for a century in servitude to the other, and freed
at last by a desolating war—the experiment sought by neither, but
approached by both with doubt—these are the conditions. Under
these, adverse at every point, we are required to carry these two
races in peace and honor to the end.
Never, sir, has such a task been given to mortal stewardship.
Never before in this Republic has the white race divided on the rights
of an alien race. The red man was cut down as a weed, because he
hindered the way of the American citizen. The yellow man was shut
out of this Republic because he is an alien and inferior. The red man
was owner of the land—the yellow man highly civilized and
assimilable—but they hindered both sections and are gone! But the
black man, affecting but one section, is clothed with every privilege
of government and pinned to the soil, and my people commanded to
make good at any hazard, and at any cost, his full and equal heirship
of American privilege and prosperity. It matters not that every other
race has been routed or excluded, without rhyme or reason. It
matters not that wherever the whites and blacks have touched, in
any era or in any clime, there has been irreconcilable violence. It
matters not that no two races, however similar, have lived anywhere
at any time on the same soil with equal rights in peace! In spite of
these things we are commanded to make good this change of
American policy which has not perhaps changed American prejudice
—to make certain here what has elsewhere been impossible
between whites and blacks—and to reverse, under the very worst
conditions, the universal verdict of racial history. And driven, sir, to
this superhuman task with an impatience that brooks no delay—a
rigor that accepts no excuse—and a suspicion that discourages
frankness and sincerity. We do not shrink from this trial. It is so
interwoven with our industrial fabric that we cannot disentangle it if
we would—so bound up in our honorable obligation to the world, that
we would not if we could. Can we solve it? The God who gave it into
our hands, He alone can know. But this the weakest and wisest of us
do know; we cannot solve it with less than your tolerant and patient
sympathy—with less than the knowledge that the blood that runs in
your veins is our blood—and that when we have done our best,
whether the issue be lost or won, we shall feel your strong arms
about us and hear the beating of your approving hearts.
 The resolute, clear-headed, broad-minded men of the South—the
men whose genius made glorious every page of the first seventy
years of American history—whose courage and fortitude you tested
in five years of the fiercest war—whose energy has made bricks
without straw and spread splendor amid the ashes of their war-
wasted homes—these men wear this problem in their hearts and
their brains, by day and by night. They realize, as you cannot, what
this problem means—what they owe to this kindly and dependent
race—the measure of their debt to the world in whose despite they
defended and maintained slavery. And though their feet are hindered
in its undergrowth, and their march encumbered with its burdens,
they have lost neither the patience from which comes clearness, nor
the faith from which comes courage. Nor, sir, when in passionate
moments is disclosed to them that vague and awful shadow, with its
lurid abysses and its crimson stains, into which I pray God they may
never go, are they struck with more of apprehension than is needed
to complete their consecration!
Such is the temper of my people. But what of the problem itself?
Mr. President, we need not go one step further unless you concede
right here the people I speak for are as honest, as sensible, and as
just as your people, seeking as earnestly as you would in their place,
to rightly solve the problem that touches them at every vital point. If
you insist that they are ruffians, blindly striving with bludgeon and
shotgun to plunder and oppress a race, then I shall sacrifice my self-
respect and tax your patience in vain. But admit that they are men of
common sense and common honesty—wisely modifying an
environment they cannot wholly disregard—guiding and controlling
as best they can the vicious and irresponsible of either race—
compensating error with frankness, and retrieving in patience what
they lose in passion—and conscious all the time that wrong means
ruin,—admit this, and we may reach an understanding to-night.
The President of the United States in his late message to
Congress, discussing the plea that the South should be left to solve
this problem, asks: “Are they at work upon it? What solution do they
offer? When will the black man cast a free ballot? When will he have
the civil rights that are his?” I shall not here protest against the
partisanry that, for the first time in our history in time of peace, has
stamped with the great seal of our government a stigma upon the
people of a great and loyal section, though I gratefully remember that
the great dead soldier who held the helm of state for the eight
stormiest years of reconstruction never found need for such a step;
and though there is no personal sacrifice I would not make to
remove this cruel and unjust imputation on my people from the
archives of my country! But, sir, backed by a record on every page of
which is progress, I venture to make earnest and respectful answer
to the questions that are asked. I bespeak your patience, while with
vigorous plainness of speech, seeking your judgment rather than
your applause, I proceed step by step. We give to the world this year
a crop of 7,500,000 bales of cotton, worth $45,000,000, and its cash
equivalent in grain, grasses and fruit. This enormous crop could not
have come from the hands of sullen and discontented labor. It comes
from peaceful fields, in which laughter and gossip rise above the
hum of industry, and contentment runs with the singing plow.
It is claimed that this ignorant labor is defrauded of its just hire. I
present the tax-books of Georgia, which show that the negro, 25
years ago a slave, has in Georgia alone $10,000,000 of assessed
property, worth twice that much. Does not that record honor him, and
vindicate his neighbors? What people, penniless, illiterate, has done
so well? For every Afro-American agitator, stirring the strife in which
alone he prospers, I can show you a thousand negroes, happy in
their cabin homes, tilling their own land by day, and at night taking
from the lips of their children the helpful message their State sends
them from the schoolhouse door. And the schoolhouse itself bears
testimony. In Georgia we added last year $250,000 to the school
fund, making a total of more than $1,000,000—and this in the face of
prejudice not yet conquered—of the fact that the whites are
assessed for $368,000,000, the blacks for $10,000,000, and yet 49
per cent. of the beneficiaries are black children—and in the doubt of
many wise men if education helps, or can help, our problem.
Charleston, with her taxable values cut half in two since 1860, pays
more in proportion for public schools than Boston. Although it is
easier to give much out of much than little out of little, the South with
one-seventh of the taxable property of the country, with relatively
larger debt, having received only one-twelfth as much public land,
and having back of its tax-books none of the half billion of bonds that
enrich the North—and though it pays annually $26,000,000 to your
section as pensions—yet gives nearly one-sixth of the public-school
fund. The South since 1865 has spent $122,000,000 in education,
and this year is pledged to $37,000,000 for state and city schools,
although the blacks paying one-thirtieth of the taxes get nearly one-
half of the fund.
Go into our fields and see whites and blacks working side by side.
On our buildings in the same squad. In our shops at the same forge.
Often the blacks crowd the whites from work, or lower wages by the
greater need or simpler habits, and yet are permitted because we
want to bar them from no avenue in which their feet are fitted to
tread. They could not there be elected orators of the white
universities, as they have been here, but they do enter there a
hundred useful trades that are closed against them here. We hold it
better and wiser to tend the weeds in the garden than to water the
exotic in the window. In the South, there are negro lawyers,
teachers, editors, dentists, doctors, preachers, multiplying with the
increasing ability of their race to support them. In villages and towns
they have their military companies equipped from the armories of the
State, their churches and societies built and supported largely by
their neighbors. What is the testimony of the courts? In penal
legislation we have steadily reduced felonies to misdemeanors, and
have led the world in mitigating punishment for crime, that we might
save, as far as possible, this dependent race from its own weakness.
In our penitentiary record 60 per cent. of the prosecutors are
negroes, and in every court the negro criminal strikes the colored
juror, that white men may judge his case. In the North, one negro in
every 1865 is in jail—in the South only one in 446. In the North the
percentage of negro prisoners is six times as great as native whites
—in the South, only four times as great. If prejudice wrongs him in
southern courts, the record shows it to be deeper in northern courts.
I assert here, and a bar as intelligent and upright as the bar of
Massachusetts will solemnly indorse my assertion, that in the
southern courts, from highest to lowest, pleading for life, liberty or
property, the negro has distinct advantage because he is a negro,
apt to be over-reached, oppressed—and that this advantage reaches
from the juror in making his verdict to the judge in measuring his
sentence. Now, Mr. President, can it be seriously maintained that we
are terrorizing the people from whose willing hands come every year
$1,000,000,000 of farm crops? Or have robbed a people, who
twenty-five years from unrewarded slavery have amassed in one
State $20,000,000 of property? Or that we intend to oppress the
people we are arming every day? Or deceive them when we are
educating them to the utmost limit of our ability? Or outlaw them
when we work side by side with them? Or re-enslave them under
legal forms when for their benefit we have even imprudently
narrowed the limit of felonies and mitigated the severity of law? My
fellow countryman, as you yourself may sometimes have to appeal
to the bar of human judgment for justice and for right, give to my
people to-night the fair and unanswerable conclusion of these
incontestible facts.
But it is claimed that under this fair seeming there is disorder and
violence. This I admit. And there will be until there is one ideal
community on earth after which we may pattern. But how widely it is
misjudged! It is hard to measure with exactness whatever touches
the negro. His helplessness, his isolation, his century of servitude,
these dispose us to emphasize and magnify his wrongs. This
disposition, inflamed by prejudice and partisanry, has led to injustice
and delusion. Lawless men may ravage a county in Iowa and it is
accepted as an incident—in the South a drunken row is declared to
be the fixed habit of the community. Regulators may whip vagabonds
in Indiana by platoons, and it scarcely arrests attention—a chance
collision in the South among relatively the same classes is gravely
accepted as evidence that one race is destroying the other. We
might as well claim that the Union was ungrateful to the colored
soldiers who followed its flag, because a Grand Army post in
Connecticut closed its doors to a negro veteran, as for you to give
racial significance to every incident in the South, or to accept
exceptional grounds as the rule of our society. I am not one of those
who becloud American honor with the parade of the outrages of
either section, and belie American character by declaring them to be
significant and representative. I prefer to maintain that they are
neither, and stand for nothing but the passion and the sin of our poor
fallen humanity. If society, like a machine, were no stronger than its
weakest part, I should despair of both sections. But, knowing that
society, sentient and responsible in every fibre, can mend and repair
until the whole has the strength of the best, I despair of neither.
These gentlemen who come with me here, knit into Georgia’s busy
life as they are, never saw, I dare assert, an outrage committed on a
negro! And if they did, not one of you would be swifter to prevent or
punish. It is through them, and the men who think with them—
making nine-tenths of every southern community—that these two
races have been carried thus far with less of violence than would
have been possible anywhere else on earth. And in their fairness
and courage and steadfastness—more than in all the laws that can
be passed or all the bayonets that can be mustered—is the hope of
our future.
When will the black cast a free ballot? When ignorance anywhere
is not dominated by the will of the intelligent; when the laborer
anywhere casts a vote unhindered by his boss; when the vote of the
poor anywhere is not influenced by the power of the rich; when the
strong and the steadfast do not everywhere control the suffrage of
the weak and shiftless—then and not till then will the ballot of the
negro be free. The white people of the South are banded, Mr.
President, not in prejudice against the blacks—not in sectional
estrangement, not in the hope of political dominion—but in a deep
and abiding necessity. Here is this vast ignorant and purchasable
vote—clannish, credulous, impulsive and passionate—tempting
every art of the demagogue, but insensible to the appeal of the
statesman. Wrongly started, in that it was led into alienation from its
neighbor and taught to rely on the protection of an outside force, it
cannot be merged and lost in the two great parties through logical
currents, for it lacks political conviction and even that information on
which conviction must be based. It must remain a faction—strong
enough in every community to control on the slightest division of the
whites. Under that division it becomes the prey of the cunning and
unscrupulous of both parties. Its credulity is imposed on, its patience
inflamed, its cupidity tempted, its impulses misdirected—and even its
superstition made to play its part in a campaign in which every
interest of society is jeopardized and every approach to the ballot-
box debauched. It is against such campaigns as this—the folly and
the bitterness and the danger of which every southern community
has drunk deeply—that the white people of the South are banded
together. Just as you in Massachusetts would be banded if 300,000
black men, not one in a hundred able to read his ballot—banded in
race instinct, holding against you the memory of a century of slavery,
taught by your late conquerors to distrust and oppose you, had
already travestied legislation from your statehouse, and in every
species of folly or villainy had wasted your substance and exhausted
your credit.
But admitting the right of the whites to unite against this
tremendous menace, we are challenged with the smallness of our
vote. This has long been flippantly charged to be evidence, and has
now been solemnly and officially declared to be proof of political
turpitude and baseness on our part. Let us see. Virginia—a State
now under fierce assault for this alleged crime—cast in 1888 75 per
cent. of her vote. Massachusetts, the State in which I speak, 60 per
cent. of her vote. Was it suppression in Virginia and natural causes
in Massachusetts? Last month Virginia cast 69 per cent. of her vote,
and Massachusetts, fighting in every district, cast only 49 per cent. of
hers. If Virginia is condemned because 31 per cent. of her vote was
silent, how shall this State escape in which 51 per cent. was dumb?
Let us enlarge this comparison. The sixteen southern States in 1888
cast 67 per cent. of their total vote—the six New England States but
63 per cent. of theirs. By what fair rule shall the stigma be put upon
one section, while the other escapes? A congressional election in
New York last week, with the polling-place in touch of every voter,
brought out only 6000 votes of 28,000—and the lack of opposition is
assigned as the natural cause. In a district in my State, in which an
opposition speech has not been heard in ten years, and the polling-
places are miles apart—under the unfair reasoning of which my
section has been a constant victim—the small vote is charged to be
proof of forcible suppression. In Virginia an average majority of
10,000, under hopeless division of the minority, was raised to
42,000; in Iowa, in the same election, a majority of 32,000 was wiped
out, and an opposition majority of 8000 was established. The change
of 42,000 votes in Iowa is accepted as political revolution—in
Virginia an increase of 30,000 on a safe majority is declared to be
proof of political fraud. I charge these facts and figures home, sir, to
the heart and conscience of the American people, who will not
assuredly see one section condemned for what another section is
excused!
If I can drive them through the prejudice of the partisan, and have
them read and pondered at the fireside of the citizen, I will rest on
the judgment there formed and the verdict there rendered!
It is deplorable, sir, that in both sections a larger percentage of the
vote is not regularly cast, but more inexplicable that this should be so
in New England than in the South. What invites the negro to the
ballot-box? He knows that, of all men, it has promised him most and
yielded him least. His first appeal to suffrage was the promise of
“forty acres and a mule.” His second, the threat that Democratic
success meant his re-inslavement. Both have proved false in his
experience. He looked for a home, and he got the freedman’s bank.
He fought under the promise of the loaf, and in victory was denied
the crumbs. Discouraged and deceived, he has realized at last that
his best friends are his neighbors, with whom his lot is cast, and
whose prosperity is bound up in his—and that he has gained nothing
in politics to compensate the loss of their confidence and sympathy
that is at last his best and his enduring hope. And so, without leaders
or organization—and lacking the resolute heroism of my party friends
in Vermont that makes their hopeless march over the hills a high and
inspiring pilgrimage—he shrewdly measures the occasional agitator,
balances his little account with politics, touches up his mule and jogs
down the furrow, letting the mad world jog as it will!
The negro vote can never control in the South, and it would be
well if partisans in the North would understand this. I have seen the
white people of a State set about by black hosts until their fate
seemed sealed. But, sir, some brave man, banding them together,
would rise, as Elisha rose in beleaguered Samaria, and touching
their eyes with faith, bid them look abroad to see the very air “filled
with the chariots of Israel and the horsemen thereof.” If there is any
human force that cannot be withstood, it is the power of the banded
intelligence and responsibility of a free community. Against it,
numbers and corruption cannot prevail. It cannot be forbidden in the
law or divorced in force. It is the inalienable right of every free
community—and the just and righteous safeguard against an
ignorant or corrupt suffrage. It is on this, sir, that we rely in the South.
Not the cowardly menace of mask or shotgun; but the peaceful
majesty of intelligence and responsibility, massed and unified for the
protection of its homes and the preservation of its liberty. That, sir, is
our reliance and our hope, and against it all the powers of the earth
shall not prevail. It was just as certain that Virginia would come back
to the unchallenged control of her white race—that before the moral
and material power of her people once more unified, opposition
would crumble until its last desperate leader was left alone vainly
striving to rally his disordered hosts—as that night should fade in the
kindling glory of the sun. You may pass force bills, but they will not
avail. You may surrender your own liberties to Federal election law,
you may submit, in fear of a necessity that does not exist, that the
very form of this government may be changed—this old State that
holds in its charter the boast that “it is a free and independent
commonwealth”—it may deliver its election machinery into the hands
of the government it helped to create—but never, sir, will a single
State of this Union, North or South, be delivered again to the control
of an ignorant and inferior race. We wrested our State government
from negro supremacy when the Federal drumbeat rolled closer to
the ballot-box and Federal bayonets hedged it deeper about than will
ever again be permitted in this free government. But, sir, though the
cannon of this Republic thundered in every voting district of the
South, we still should find in the mercy of God the means and the
courage to prevent its re-establishment!
I regret, sir, that my section, hindered with this problem, stands in
seeming estrangement to the North. If, sir, any man will point out to
me a path down which the white people of the South divided may
walk in peace and honor, I will take that path though I took it alone—
for at the end, and nowhere else, I fear, is to be found the full
prosperity of my section and the full restoration of this Union. But, sir,
if the negro had not been enfranchised, the South would have been
divided and the Republic united. His enfranchisement—against
which I enter no protest—holds the South united and compact. What
solution, then, can we offer for the problem? Time alone can disclose
it to us. We simply report progress and ask your patience. If the
problem be solved at all—and I firmly believe it will, though nowhere
else has it been—it will be solved by the people most deeply bound
in interest, most deeply pledged in honor to its solution. I had rather
see my people render back this question lightly solved than to see
them gather all the spoils over which faction has contended since
Catiline conspired and Cæsar fought. Meantime we treat the negro
fairly, measuring to him justice in the fullness the strong should give
to the weak, and leading him in the steadfast ways of citizenship that
he may no longer be the prey of the unscrupulous and the sport of
the thoughtless. We open to him every pursuit in which he can
prosper, and seek to broaden his training and capacity. We seek to
hold his confidence and friendship, and to pin him to the soil with
ownership, that he may catch in the fire of his own hearthstone that
sense of responsibility the shiftless can never know. And we gather
him into that alliance of intelligence and responsibility that, though it
now runs close to racial lines, welcomes the responsible and
intelligent of any race. By this course, confirmed in our judgment and
justified in the progress already made, we hope to progress slowly
but surely to the end.
The love we feel for that race you cannot measure nor
comprehend. As I attest it here, the spirit of my old black mammy
from her home up there looks down to bless, and through the tumult
of this night steals the sweet music of her croonings as thirty years
ago she held me in her black arms and led me smiling into sleep.
This scene vanishes as I speak, and I catch a vision of an old
Southern home, with its lofty pillars, and its white pigeons fluttering
down through the golden air. I see women with strained and anxious
faces, and children alert yet helpless. I see night come down with its
dangers and its apprehensions, and in a big homely room I feel on
my tired head the touch of loving hands—now worn and wrinkled,
but fairer to me yet than the hands of mortal woman, and stronger
yet to lead me than the hands of mortal man—as they lay a mother’s
blessing there while at her knees—the truest altar I yet have found—
I thank God that she is safe in her sanctuary, because her slaves,
sentinel in the silent cabin or guard at her chamber door, puts a
black man’s loyalty between her and danger.
I catch another vision. The crisis of battle—a soldier struck,
staggering, fallen. I see a slave, scuffling through the smoke, winding
his black arms about the fallen form, reckless of the hurtling death—
bending his trusty face to catch the words that tremble on the
stricken lips, so wrestling meantime with agony that he would lay
down his life in his master’s stead. I see him by the weary bedside,
ministering with uncomplaining patience, praying with all his humble
heart that God will lift his master up, until death comes in mercy and
in honor to still the soldier’s agony and seal the soldier’s life. I see
him by the open grave, mute, motionless, uncovered, suffering for
the death of him who in life fought against his freedom. I see him
when the mound is heaped and the great drama of his life is closed,
turn away and with downcast eyes and uncertain step start out into
new and strange fields, faltering, struggling, but moving on, until his
shambling figure is lost in the light of this better and brighter day.
And from the grave comes a voice saying: “Follow him! Put your
arms about him in his need, even as he puts his about me. Be his
friend as he was mine.” And out into this new world—strange to me
as to him, dazzling, bewildering both—I follow! And may God forget
my people—when they forget these!
Whatever the future may hold for them—whether they plod along
in the servitude from which they have never been lifted since the
Cyrenian was laid hold upon by the Roman soldiers and made to
bear the cross of the fainting Christ—whether they find homes again
in Africa, and thus hasten the prophecy of the psalmist who said:
“And suddenly Ethiopia shall hold out her hands unto God”—
whether, forever dislocated and separated, they remain a weak
people beset by stronger, and exist as the Turk, who lives in the
jealousy rather than in the conscience of Europe—or whether in this
miraculous Republic they break through the caste of twenty
centuries and, belying universal history, reach the full stature of
citizenship, and in peace maintain it—we shall give them uttermost
justice and abiding friendship. And whatever we do, into whatever
seeming estrangement we may be driven, nothing shall disturb the
love we bear this Republic, or mitigate our consecration to its
service. I stand here, Mr. President, to profess no new loyalty. When
General Lee, whose heart was the temple of our hopes and whose
arm was clothed with our strength, renewed his allegiance to the
government of Appomattox, he spoke from a heart too great to be
false, and he spoke for every honest man from Maryland to Texas.
From that day to this, Hamilcar has nowhere in the South sworn
young Hannibal to hatred and vengeance—but everywhere to loyalty
and to love. Witness the soldier standing at the base of a
Confederate monument above the graves of his comrades, his
empty sleeve tossing in the April wind, adjuring the young men about
him, to serve as honest and loyal citizens the government against
which their fathers fought. This message, delivered from that sacred
presence, has gone home to the hearts of my fellows! And, sir, I
declare here, if physical courage be always equal to human
aspiration, that they would die, sir, if need be, to restore this Republic
their fathers fought to dissolve!
Such, Mr. President, is this problem as we see it; such is the
temper in which we approach it: such the progress made. What do
we ask of you? First, patience; out of this alone can come perfect
work. Second, confidence; in this alone can you judge fairly. Third,
sympathy; in this you can help us best. Fourth, give us your sons as
hostages. When you plant your capital in millions, send your sons
that they may help know how true are our hearts and may help to
swell the Anglo-Saxon current until it can carry without danger this
black infusion. Fifth, loyalty to the Republic—for there is sectionalism
in loyalty as in estrangement. This hour little needs the loyalty that is
loyal to one section and yet holds the other in enduring suspicion
and estrangement. Give us the broad and perfect loyalty that loves
and trusts Georgia alike with Massachusetts—that knows no south,
no north, no east, no west; but endears with equal and patriotic love
every foot of our soil, every State in our Union.
A mighty duty, sir, and a mighty inspiration impels every one of us
to-night to lose in patriotic consecration whatever estranges,
whatever divides. We, sir, are Americans—and we fight for human
liberty. The uplifting force of the American idea is under every throne
on earth. France, Brazil—these are our victories. To redeem the
earth from kingcraft and oppression—this is our mission. And we
shall not fail. God has sown in our soil the seed of his millennial
harvest, and he will not lay the sickle to the ripening crop until his full
and perfect day has come. Our history, sir, has been a constant and
expanding miracle from Plymouth Rock and Jamestown all the way
—aye, even from the hour when, from the voiceless and trackless
ocean, a new world rose to the sight of the inspired sailor. As we
approach the fourth centennial of that stupendous day—when the
old world will come to marvel and to learn, amid our gathered
treasures—let us resolve to crown the miracles of our past with the
spectacle of a Republic compact, united, indissoluble in the bonds of
love—loving from the lakes to the Gulf—the wounds of war healed in
every heart as on every hill—serene and resplendent at the summit
of human achievement and earthly glory—blazing out the path, and
making clear the way up which all the nations of the earth must
come in God’s appointed time!
 BEFORE THE BAY STATE CLUB.

D URING Mr. Grady’s Visit to Boston, in 1889, he was a Guest of the

Bay State Club, before whom he Delivered the following Speech:

Mr. President and Gentlemen: I am confident you will not

expect a speech from me this afternoon, especially as my voice is in
such a condition that I can hardly talk. I am free to say that it is not a
lack of ability to talk, because I am a talker by inheritance. My father
was an Irishman, my mother was a woman; both talked. I came by it
honestly.
I don’t know how I could take up any discussion here on any topic
apart from the incidents of the past two days. I saw this morning
Plymouth Rock. I was pulled up on top of it and was told to make a
speech.
It reminded me of an old friend of mind, Judge Dooley, of Georgia,
who was a very provoking fellow and was always getting challenged
to duels, and never fighting them. He always got out of it by being
smarter than the other fellow. One day he went out to fight a man
with one leg, and he insisted on bringing along a bee gum and
sticking one leg into it so that he would have no more flesh exposed
than his antagonist. On the occasion I am thinking of, however, he
went out to fight with a man who had St. Vitus’s dance, and the
fellow stood before him holding the pistol cocked and primed, his
hand shaking. The judge went quietly and got a forked stick and
stuck it up in front of him.