Exploration vs Exploitation for EI and POI #15
Conversation
"2.3.2 Exploration-exploitation trade-of" http://arxiv.org/pdf/1012.2599v1.pdf
*) added notebook showing the effect of the xi parameter for EI and POI acquisition functions
…e "ucb" acquisition function :/
| @@ -10,11 +10,13 @@ class UtilityFunction(object): | |||
| An object to compute the acquisition functions. | |||
| """ | |||
|
|
|||
| def __init__(self, kind='ucb', kappa=1.96): | |||
| def __init__(self, kind, kappa, xi): | |||
There was a problem hiding this comment.
I think it makes sense to give both kappa and xi default values so when the object is instantiated with a particular acquisition function in mind, the user doesn't have to bother with an unrelated parameter.
I get that this is not an issue in the context of the maximize method, since there these values have defaults. But it just feels a bit weird thinking about creating a UtilityFunciton object to run, say UCB, and having to set xi. Do you agree?
There was a problem hiding this comment.
It depends - let me explain why I removed the default values from the UtilityFunction class:
As far as I can tell instances of the UtilityFunction class are normally only created internally by the BayesianOptimization class - not by the user himself. The user only uses the maximize method (which still has default values).
The relevant line of code in method maximize is:
def maximize(self, init_points=5, n_iter=25, acq='ucb', kappa=2.576, xi=0.0, **gp_params):
...
self.util = UtilityFunction(kind=acq, kappa=kappa, xi=xi)
Here all parameters (acq, kappa, xi,...) are known and all parameters have to be passed on to the UtilityFunction class anyway. Forgetting to specify one of the parameters at that point should actually result in an error.
So I'd remove the default values from the UtilityFunction class, unless there are use cases where the user wants to create UtilityFunction instances himself.
Another (and design technically better) solution would be to split the UtilityFunction class into multiple classes for UCB, EI and POI.. then UCB gets a kappa parameter and EI and POI get a xi parameter.. but I wouldn't do that yet / until more acquisition functions with even more different parameters get added.
|
Thanks for this! I have one small comment, but besides that I think this is ready to merge. |
Exploration vs Exploitation for EI and POI
I added the Xi parameter to the EI and POI acquisition functions to control exploration vs exploitation as explained in http://arxiv.org/pdf/1012.2599v1.pdf
The default is xi=0.0 (old behaviour).
Example notebook:
https://github.com/stmax82/BayesianOptimization/blob/explore_exploit/examples/exploitation%20vs%20exploration.ipynb
oh and I fixed a bug... previously specifying the acq function "poi" returned "ucb"...!