Introduction ML

1 - Definition of ML
Interview Transcript: "Defining Machine Learning"

 Michael: Hi Michael.
 Charles: Hey Charles, how's it going?
 Michael: It's going quite well, how's it going with you?
 Charles: Good. Good.
 Michael: Good Good. So, today I thought we would talk a little bit about the philosophy of
Machine Learning.
 Charles: Oh, I hate philosophy.
 Michael: I don't like it much either, although I am a doctor of philosophy.
 Charles: Oh, that's very impressive.
 Michael: Aren't you a doctor of philosophy too?
 Charles: I am, it's kind of impressive.
 Michael: It is kind of impressive. So what we wanted to kind of get across today was a little bit
about why we, the class is structured the way it is. What the different parts are. And maybe go a
little bit of back and forth about what we think you should be getting out of the course. That
seem reasonable?
 Charles: Sure.
 Michael: Okay. Well, so, first off, by the way, before we get started, I wanted to thank you for
coming down to Atlanta, and joining me in these beautiful, studios.
 Charles: Well, it's, it's, it's very nice to be here. Thank you for inviting me.
 Michael: Oh, no, no, thank you for coming, Michael.
 Charles: Thank you for asking me to do the course. This has been a lot of fun.
 Michael: Oh, the whole point was to be able to do the course with you, Michael. We like each
other, and that's one of the things that we want you to get, want to get across in this class,
because we like machine learning. We've a lot of stuff in common, but I'm not sure we
completely agree on the most important parts of machine learning and why we do the things
that we do.
 Charles: Hm, all right.
 Michael: So I think people in the outside world, Michael, would claim that you're more
theoretical than I am.
 Charles: In theory.
 Michael: In theory, and I'm more practical than you are.
 Charles: Practically.
 Michael: At least in practice. And hopefully some of that tension will come out in the class. But I
think in order to see why that tension works that way, you have to understand what machine
learning is. So, Michael.
 Charles: Right.
 Michael: What's machine learning?
  Charles: It's about proving theorems.
 Michael: [LAUGH] No.
 Charles: No?
 Michael: I would not say it's about proving theorems, although proving theorems is often
important in machine learning.
 Charles: I agree with that.
 Michael: Okay. So we're on the same page.
 Charles: We're partially on the same page. What is machine learning?
 Michael: What is machine learning?
 Charles: Give me a definition.
 Michael: So it is computational statistics. How's that for a definition?
 Charles: That is a definition. It is wrong on so many levels. However, a lot of people would agree
with that statement. They would say that machine learning is really just applied statistics.
 Michael: Not applied statistics. Computational statistics.
 Charles: Computationally applied statistics. I don't like that definition. I think that it's a bit too
narrow. I think that machine learning is about this broader notion of building artifacts,
computational artifacts, typically. That learn over time based on experience. And then in
particular, it's not just the act of building these artifacts that matter, it's the math behind it. It's
the science behind it. It's the engineering behind it, and it's the computing behind it. It's
everything that goes into building intelligent artifacts that almost by necessity have to learn over
time. You buy that?
 Michael: Yeah, so you, you have data, and you do analysis of the data and try to glean things
from the data. And you used, various kinds of computational structure to do that, so,
computational statistics.
 Charles: I don't think that's computational statistics.

2 - Supervised Learning
 Charles: This class is divided into three subclasses, three parts. They are supervised learning.
 Michael: Yeah.
 Charles: Unsupervised learning, and reinforcement. So, what do you think supervised learning
is?
 Michael: So, I think of supervised learning as being the problem of taking labelled data sets,
gleaning information from it so that you can label new data sets.
 Charles: That's fair. I call that function approximation. So, here's an example of supervised
learning. I'm going to give you an input and an output. And I'm going to give them to you as
pairs, and I want you to guess what the function is.
 Michael: Sure. Okay?
 Charles: Okay. 1, 1.
 Michael: Uh-huh.
 Charles: 2, 4.
 Michael: Wait, hang on, is 1 the input and 1 the output, and 2 the input, and 4 the output?
 Charles: Correct.
 Michael: Alright. I think I am on to you.
 Charles: 3, 9.
 Michael: Okay.
 Charles: 4, 16.
 Michael: Nice.
 Charles: 5, 25. 6, 36. 7, 49.
 Michael: Nice. This is a very hip data set.
 Charles: It is. What's the function?
 Michael: It's hip to be squared.
 Charles: Exactly. Maybe. So if you believe that's true, then tell me if the input is 10, what's the
output?
 Michael: 100.
 Charles: And that's right, if it turns out, in fact, that the function is x squared. But the truth is, we
have no idea whether this function is x squared. Not really.
 Michael: I have a pretty good idea.
 Charles: You do?
 Michael: Well-
 Charles: Where's that idea come from?
 Michael: It comes from having spoken with you over a long period of time. And plus, you know,
math.
 Charles: And plus math. Well, I'm going to-
 Michael: You can't say I'm wrong.
 Charles: You're wrong.
 Michael: Oh.
 Charles: Yeah, I did.
 Michael: You just said I was wrong.
 Charles: No, you've talked to me for a long time, and plus math. I agree with that.
 Michael: Okay.
 Charles: But I'm going to claim that you're making a leap of faith.
 Michael: Hm.
 Charles: Despite being a scientist, by deciding that the input is 10 and the output is 100.
 Michael: Sure. I would agree with that.
 Charles: What's that leap of faith?
 Michael: Well, I mean, from what you told me, it's still consistent with lots of other mappings
from input to output like 10 gets mapped to 11.
 Charles: Right or everything is x squared except 10.
 Michael: Sure.
 Charles: Or everything is x, x squared up to 10.
 Michael: Right, that would be mean-
 Charles: That would be mean-
  Michael: But it's not logically impossible.
 Charles: What would be the median?
 Michael: A-ha.
 Charles: Thank you very much. I, I was saving that one up.

3 - Induction and Deduction

 Charles: Well, what you're doing in order to make that work and what you end up doing in
supervised learning and functions approximation in general, is you make some fundamental
assumptions about the work, right? You decide that you have a well-behaved function that is
consistent with the data that you're given, and with that, you're able to generalize, and in fact
that is the fundamental problem in machine learning. It is generalization. Now what's behind all
of this is I'm going to claim, Michael, you jump in whenever you disagree is
 Michael: I disagree sorry to soon, go ahead.
 Charles: is bias and in particular
 Michael: bias
 Charles: inductive bias.
 Michael: Inductive bias.
 Charles: Right, so all of machine learning or certainly supervised learning is about induction, as
opposed to deduction.
 Michael: I see. Induction of course being a problem of going from examples to a more general
rule.
 Charles: Right, specifics to generalities. By contrast, deduction is?
 Michael: Be the opposite. It would be going from a general rule to specific instances, basically
like reasoning.
 Charles: Right, and in fact, a lot of AI in the beginning was about deductive reasoning, about
logic programming, those sorts of things, where you have certain rules, and you deduce only
those things that follow immediately from those rules. So for example, you'd have something
like A implies B. That's a rule in the universe. And then I tell you A. So if you A implies B is a rule
in the universe, and I tell you A,
 Michael: That A implies B.
 Charles: And therefore you can infer that?
 Michael: And A. B.
 Charles: B. You have A implies B, you have A, that implies B.
 Michael: Okay.
 Charles: That's what we just said. But what,
 Michael: that's deduction.
 Charles: That's deduction, but what we just did was not deduction. Before then when I asked
you one, one, two, four, three, nine, four sixteen and so forth.
 Michael: Right,
 Charles: we did induction.
 Michael: That was induction.
 Charles: Induction is more about did the sun rise yesterday?
 Michael: yes.
 Charles: Did the sun rise the day before that?
 Michael: yes.
 Charles: Did the sun rise the day before that?
 Michael: I slept in. Did the sun rise the day before that?
 Charles: Yes.
 Michael: Yes. So the sun has risen every day. Is the sun going to rise tomorrow?
 Charles: I sure hope so.
 Michael: We all hope so, and we all act like it does, because if it doesn't, then there are a whole
bunch of other things we ought to be doing besides sitting in this studio and having this
interview.
 Charles: I think we should warn the plants.
 Michael: [LAUGH] I don't think the plants are going to care.
 Charles: They are. They really need sun. I think we all need sun, Mike.
 Michael: Eh.
 Charles: So, the idea there is induction is crucial, and the inductive bias is crucial. We'll talk
about all of this in, in the course.
 Michael: Kay.
 Charles: But that's kind of a fundamental notion behind supervised learning and machine
learning in general.
 Michael: I agree with that.
 Charles: Agreed?
 Michael: Yeah.
 Charles: Alright, so we're on the same page. So that's supervised learning. Supervised learning,
you can talk about it in these high muckity muck ways, but at the end of the day, it's function
approximation. It's figuring out how to take a bunch of training examples and coming up with
some function that generalizes beyond the data that you see.
 Michael: So, why wouldn't you call it function induction, then?
 Charles: because someone said supervised learning first. Well, there is a-
 Michael: No, no, no, no. Sorry. You said supervised learning is function approximation and I want
to say, why don't you say supervised learning is function induction.
 Charles: As opposed to function approximation?
 Michael: Yeah.
 Charles: Okay. It's a
 Michael: Approximate function induction.
 Charles: Or induction of approximate, of.
 Michael: Approximate functions?
 Charles: Approximate functions, something like that, yeah.
 Michael: You don't want to induce an approximate function, you want to induce the actual
function.
 Charles: Yeah, but sometimes you can't,
 Michael: Yeah.
 Charles: Because sometimes you think it's quadratic, but it's not.
  Michael: I have that as a plaque on my wall.
 Charles: You do?
 Michael: No.
 Charles: Yeah I didn't think so. Okay so that's supervised learning

4- "Diving into Reinforcement Learning"

 Charles: All right. So that's supervised learning and unsupervised learning. That's pretty good.
The last one is reinforcement learning.
 Michael: [SOUND].
 Charles: Now reinforcement learning is what we both do, so Michael does a little bit of
reinforcement learning here and there. You've got how many papers published in reinforcement
learning?
 Michael: All of them. [LAUGH] Several. I have several.
 Charles: The man has like a hundred papers of reinforcement learnings and in fact he wrote with
his colleagues the great summary journal article bringing every one up to date on what
reinforcement learning was like back in 1990.
 Michael: Yeah like 112 years ago.
 Charles: 1992.
 Michael: People are saying yeah we should probably somebody should write a new one because
the other ones getting a little long in the dude.
 Charles: But there's been books written on machine learning system.
 Michael: That's right.
 Charles: It's a very popular field. That's why we're both in it. Michael tends to prove a lot of
things,.
 Michael: It is not, that is not why I'm in it.
 Charles: What, I didn't, wait, what?
 Michael: You said it's a very popular field and that's why we're in it.
 Charles: No, no, no, no, no. Did I say that?
 Michael: That's what I heard.
 Charles: I didn't mean to say that.
 Michael: [SOUND] Let's run it back and see.
 Charles: It's a very popular, yeah, let's do that again because I did not mean to say that. It is a
very popular field. Perhaps because you're in it Michael.
 Michael: I don't think that's it. When I was an undergraduate, I thought the thing that I really
want to understand. I liked AI, I liked the whole idea of AI. But what I really want to understand
is how can you learn to be better from experience? And like I, I built a tic-tac-toe playing
program, and like, I want this tic-tac-toe playing program to get really good at tic-tac-toe.
because I was always interested in the most practical society impacting problems.
 Charles: I think that generalized pretty well to world hunger.
 Michael: Eventually. So so that is what got me interested in it, and I was, I didn't even know
what it was called for a long time. So I started doing reinforcement learning, and then
discovered that it was interesting and popular.
 Charles: Right. Well, I certainly wouldn't suggest that we're doing the science that we're doing
because it's popular. We're doing it because we're interested in it.
 Michael: Yes.
 Charles: And I'm interested in reinforcement learning because in some sense, it kind of
encapsulates all the things I happen to care about. I come from a sort of general AI background,
and I care modeling people. I care about building smart agents that have to live in in world that
other smart agents, thousands of them, hundreds of thousand of them, thousands of them.
Some of them might be human and have to feel some way to predict what to do over time. So,
from a sort a technical point of view, if we can think of re, in, supervised learning as function
approximation and unsupervised learning as, you know.
 Michael: Concise-
 Charles: Concise, impact description, what's the difference between something like
reinforcement learning and those two? Supervised learning.
 Michael: So often the way that supervised learning oh, sorry, reinforcement learning is
described is, is learning from delayed reward.
 Charles: Mm-hm. So instead of the feedback that you get in supervised learning which is here's
what you should do. And the feedback that you get in unsupervised learning which is the
feedback in reinforcement learning may come several steps after the decisions that you've
actually made.
 Michael: So a good example of that, or the easy example of that would be, actually your tic-tac-
toe program, right? So, you do something in tic-tac-toe, you put an X in the center and then you
put a, let's say, an O over here.
 Charles: Oh.
 Michael: And then I put an X right here.
 Charles: Nice.
 Michael: And then you ridiculously put an O in the center.
 Charles: Which allows me to put an X over here and I win.
 Michael: All right.
 Charles: Now what's interesting about that is, I didn't tell you what happened until the very end
when I said X wins.
 Michael: Right. And now I know I made a mistake somewhere along the way but I don't know
exactly where. I may have to kind of roll back the game in my mind and eventually figure out
where it is that I went off track, and what it is that I did wrong.
 Charles: And in the full generality of reinforcement learning, you may have never made a
mistake. It may simply be that's the way games go but you would like to know which of the
moves you made mattered. Now, if it were a civilized learning problem, I would have put the X
here, he would have put the O there, and it would have been called that's Good. I would have
put the X here, and when he put the O there, it would have been that's Bad, the O goes here.
 Michael: Mm-hm. Right.
 Charles: Or something like that. It would have told you where he should have put the O. But
here, all he gets is eventually some kind of signal saying, you did something well. You did
something poorly and even then it's only relative to the other signals that you might have
gotten.
  Michael: Right, so then reinforcement learning is in some sense harder because nobody's telling
you what to do. You have to work it out on your own.
 Charles: Yeah it's like playing a game without knowing any of the rules. Or at least knowing how
you win or lose. But being told every once in awhile that you've won or you've lost, okay, now-
 Michael: Sometimes I feel like that.
 Charles: I know man.

5 : "Comparison of Machine Learning Components"

 Charles: Okay, so we've got these three little bits of machine learning here. And there are a lot
of tools and techniques that are inside that.
 Michael: Mm-hm.
 Charles: And I think that's great. And we're going to be trying to teach you a lot of those tools
and techniques and sort of ways to connect them together. So by the way, as Michael was
pointing out, there are kind of ways that these things might help each other, unsupervised
learning might help supervised learning. It's actually much deeper than that. It turns out you,
even though unsupervised learning is clearly not the same as supervised learning at the level
that we described it, in some ways they're exactly the same thing. Supervised learning you have
some bias. Oh, it's a quadratic function, induction make sense. All these kind of assumptions you
make.
 Michael: Mm.
 Charles: And in unsupervised learning, I told you that we don't know whether this clusters is
better than this cluster, dividing by sex is better than dividing by height, or, or hair color or
whatever.
 Michael: Mm.
 Charles: But ultimately you make some decision about how to cluster, and that means implicitly
there's some assume signal. There's some assume set of labels that you think make sense. Oh, I
think things that look alike should somehow be clustered together.
 Michael: Mm.
 Charles: Or things that are near one another should cluster together. So in some ways it's still
kind of like supervised learning. You can certainly turn any supervised learning problem into an
unsupervised learning problem.
 Michael: Mm, mm.
 Charles: Right? So in fact, all of these problems are really the same kind of problem.
 Michael: Yeah, well there's two things that I'd want to add to that. One is that in some sense, in
many cases, you can formulate all these different problems as some form of optimization. In
supervised learning you want something that, that labels data well and so you're, the thing
you're trying to optimize is find me a function that, that does that scores it. In reinforcement
learning we're trying to find a behavior that scores well. And unsupervised learning we usually
have to make up some kind of a criterion, and then we find a way of clustering the data,
organizing the data so that it scores well. So that was the first point I wanted to make.
 Charles: Do you learn about that on the street?
 Michael: I learned it in a Math book.
 Charles: Yes you [NOISE] I'm, I'm going to move on And so here's the thing.
 Michael: Alright.
 Charles: Everything just Michael just said except the last part is true. But there's actually a sort
of deeper thing going on here. To me, if you think about the commonalities of everything we've
just said, it boils down to one thing data, data, data, data, data, data. Data is king in machine
learning. Now Michael would call himself a computer scientist.
 Michael: Oh, yeah.
 Charles: And I would call myself a computationalist.
 Michael: What?
 Charles: What if I'm in a college of computing at a department of computer science? I believe in
computing and computation as being the ultimate thing. So I would call myself a
computationalist, and Michael would probably agree with that just to keep this discussion
moving.
 Michael: Let's say.
 Charles: Right. So we're computationalists. We believe in computing. That's a good thing.
 Michael: Sure.
 Charles: Many of our colleagues who do computations tend to think in terms of algorithms. They
think in terms of what are the series of steps I need to do in order to solve some problem? Or
they.
 Michael: [CROSSTALK].
 Charles: Might think in terms of theorems. If I try to describe this problem in a particular way, is
it solvable quizzically by some algorithm?
 Michael: Yeah.
 Charles: And, truthfully, machine learning is a lot of that. But the difference between the person
who's trying to solve our problem as an AI person or as a computing person and somebody
who's trying to solve our problem as a machine learning person is that the algorithm stops being
central, the data starts being central. And so what I hope you get out of this class, or at least
part of the stuff that you do, is understanding that you have to believe the data, you have to do
something with the data, you have to be consistent with the data. The algorithms that fall out of
all that are algorithms, but they're algorithms that take the data as primary or at least
important.
 Michael: I'm going to go with co-equal.
 Charles: So the algorithms and data are co-equal.
 Michael: Well if you believe in Lisp, they're the same thing.
 Charles: Exactly!
 Michael: Alright. So there you go.
 Charles: They knew back in the 70s.
 Michael: So it turns out we do agree on most things.
 Charles: [NOISE] That was close.
 Michael: Excellent! So, the rest of the semester will go exactly like this.
 Charles: [LAUGH].
 Michael: [LAUGH] Except you won't see us. You'll see our hands though.
 Charles: This side. This side.
 Michael: You'll see our hands, though. Thank you, Michael.
 Charles: It's all right.
 Michael: [LAUGH] What?
 Charles: [LAUGH] What? [LAUGH].
 Michael: That was good, that took me back to when I was four. Okay, so.
 Charles: Senor Wences.
 Michael: Hm?
 Charles: It's called Señor Wences.
 Michael: Yes, I know.
 Charles: Yeah, okay.
 Michael: I'm not that much younger than you are.
 Charles: Little bit.
 Michael: Ten, 12 years only.
 Charles: No come on.
 Michael: You can count gray hairs. Anyway the point is the rest of the semester will go like this.
We will talk about supervised learning and a whole series of algorithms. Step back a little bit and
talk about the theory behind them, and try to connect theory of machine learning with theory of
computing notions, or at least that kind of basic idea. What does it mean to be a hard problem
versus an easier problem? Will move into randomized optimization and unsupervised learning
where we will talk about all the issues that we brought up here and try to connect them back to
some of the things that we did in the section on supervised learning. And then finally, we will
spend our time on reinforcement learning. And a generalization of these traditional
reinforcement learning, which involves multiple agents. So we'll talk about a little bit of.
 Michael: Mm-hm.
 Charles: Game theory, which Michael loves to talk about. I love to talk about. And the
applications of all the stuff that we've been learning to solving problems of how to actually act in
the world? How to build that world out to do something? Or build that agent to play a game or
to teach you how to do whatever you, you need to be taught how to do? But at the end of the
day, we're going to teach you how to think about data, how to think about algorithms, and how
to build artifacts that you know, will learn?
 Michael: Let's do this thing.
 Charles: Excellent. All right. Well thank you Michael.
 Michael: Sure.
 Charles: I will see you next time we're in the same place at the same time.