Skip to content

Exploration vs Exploitation for EI and POI #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 9, 2016
Merged

Exploration vs Exploitation for EI and POI #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8b4e936
Added a Xi parameter to the EI acquisition function, see:
stmax82 Mar 7, 2016
c24cad2
*) added the xi parameter to the poi acq function
stmax82 Mar 7, 2016
ceb3b37
bugfix: selecting the "poi" acquisition function actually selected th…
stmax82 Mar 7, 2016
8b5a8ca
Merge branch 'master' into explore_exploit
stmax82 Mar 7, 2016
885c3d7
notebook update
stmax82 Mar 7, 2016
bfcd0ed
default value of xi set to 0.0 (results in the old behaviour)
stmax82 Mar 7, 2016
ea4118d
moved example notebook
stmax82 Mar 7, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion bayes_opt/bayesian_optimization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,7 @@ def maximize(self,
n_iter=25,
acq='ucb',
kappa=2.576,
xi=0.0,
**gp_params):
"""
Main optimization method.
Expand Down Expand Up @@ -272,7 +273,7 @@ def maximize(self,
self.plog.reset_timer()

# Set acquisition function
self.util = UtilityFunction(kind=acq, kappa=kappa)
self.util = UtilityFunction(kind=acq, kappa=kappa, xi=xi)

# Initialize x, y and find current y_max
if not self.initialized:
Expand Down
18 changes: 10 additions & 8 deletions bayes_opt/helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,13 @@ class UtilityFunction(object):
An object to compute the acquisition functions.
"""

def __init__(self, kind='ucb', kappa=1.96):
def __init__(self, kind, kappa, xi):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to give both kappa and xi default values so when the object is instantiated with a particular acquisition function in mind, the user doesn't have to bother with an unrelated parameter.

I get that this is not an issue in the context of the maximize method, since there these values have defaults. But it just feels a bit weird thinking about creating a UtilityFunciton object to run, say UCB, and having to set xi. Do you agree?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends - let me explain why I removed the default values from the UtilityFunction class:

As far as I can tell instances of the UtilityFunction class are normally only created internally by the BayesianOptimization class - not by the user himself. The user only uses the maximize method (which still has default values).

The relevant line of code in method maximize is:

def maximize(self, init_points=5, n_iter=25, acq='ucb', kappa=2.576, xi=0.0, **gp_params):
    ...
    self.util = UtilityFunction(kind=acq, kappa=kappa, xi=xi)

Here all parameters (acq, kappa, xi,...) are known and all parameters have to be passed on to the UtilityFunction class anyway. Forgetting to specify one of the parameters at that point should actually result in an error.

So I'd remove the default values from the UtilityFunction class, unless there are use cases where the user wants to create UtilityFunction instances himself.

Another (and design technically better) solution would be to split the UtilityFunction class into multiple classes for UCB, EI and POI.. then UCB gets a kappa parameter and EI and POI get a xi parameter.. but I wouldn't do that yet / until more acquisition functions with even more different parameters get added.

"""
If UCB is to be used, a constant kappa is needed.
"""
self.kappa = kappa

self.xi = xi

if kind not in ['ucb', 'ei', 'poi']:
err = "The utility function " \
Expand All @@ -28,33 +30,33 @@ def utility(self, x, gp, y_max):
if self.kind == 'ucb':
return self._ucb(x, gp, self.kappa)
if self.kind == 'ei':
return self._ei(x, gp, y_max)
return self._ei(x, gp, y_max, self.xi)
if self.kind == 'poi':
return self._ucb(x, gp, y_max)
return self._poi(x, gp, y_max, self.xi)

@staticmethod
def _ucb(x, gp, kappa):
mean, var = gp.predict(x, eval_MSE=True)
return mean + kappa * np.sqrt(var)

@staticmethod
def _ei(x, gp, y_max):
def _ei(x, gp, y_max, xi):
mean, var = gp.predict(x, eval_MSE=True)

# Avoid points with zero variance
var = np.maximum(var, 1e-9 + 0 * var)

z = (mean - y_max)/np.sqrt(var)
return (mean - y_max) * norm.cdf(z) + np.sqrt(var) * norm.pdf(z)
z = (mean - y_max - xi)/np.sqrt(var)
return (mean - y_max - xi) * norm.cdf(z) + np.sqrt(var) * norm.pdf(z)

@staticmethod
def _poi(x, gp, y_max):
def _poi(x, gp, y_max, xi):
mean, var = gp.predict(x, eval_MSE=True)

# Avoid points with zero variance
var = np.maximum(var, 1e-9 + 0 * var)

z = (mean - y_max)/np.sqrt(var)
z = (mean - y_max - xi)/np.sqrt(var)
return norm.cdf(z)


Expand Down
285 changes: 285 additions & 0 deletions examples/exploitation vs exploration.ipynb
View file
Open in desktop

Large diffs are not rendered by default.