Feature/parameter dependency

[Engineero]

Updated maximization routine to accept constraints using the SLSQP optimizer. Added tests. This resolves issue #94.

It looks like the system will still guess outside of the constraints sometimes, but the updated sklearn_example, at least in my testing, showed that it mostly respects the constraints. The example shown for the maximize method still finds the global max, despite the constraint trying to restrict it to values above the 5000th index.

[codecov-io]

Codecov Report

Merging #95 into master will decrease coverage by 0.04%.
The diff coverage is 87.5%.

@@            Coverage Diff             @@
##           master      #95      +/-   ##
==========================================
- Coverage   82.87%   82.82%   -0.05%     
==========================================
  Files           4        4              
  Lines         292      297       +5     
  Branches       35       38       +3     
==========================================
+ Hits          242      246       +4     
  Misses         45       45              
- Partials        5        6       +1

Impacted Files	Coverage Δ
bayes_opt/bayesian_optimization.py	66.37% <100%> (+0.29%)	⬆️
bayes_opt/helpers.py	89.41% <83.33%> (-0.72%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 317f735...52086fd. Read the comment docs.

[Engineero]

I just realized that pull request #45 addresses this same issue. I believe our approaches are a bit different, so it may be worth looking in to both to see what you like best. Please merge one of these soon!

[fmfn]

Thank you @Engineero , I will take a look at both. I'm a bit behind reviewing issues and PRs but will get around doing that soon.

[fmfn]

My fear with implementing constraints on the acquisition function optimization is that the algorithm might get stuck probing the same edge points. Since the Gaussian Process is not constrained it may yield optimal points outside the allowed bounds, leading the acq_max function to return the same edge point over and over and the GP not being updated (since no new knowledge is gained), in a never ending loop.

In my mind reparametrizing the target function (embedding the constraints in the new parametrization) seems like a more robust solution. However that can be hard or even impossible at times. Alternatively I could see a very clever choice or kernel also being a solution to this problem.

[zhuyanhao]

Hello @fmfn / @Engineero ,

I'm new to this package and I'm using it to solve a constrained problem. I had a feeling that the acquisition function should also been changed as the original ac functions are no longer valid for the constraint problem. Is there any related workings since this post?

Thanks,
Yanhao

[bwheelz36]

Closing this as we have a constrain model implemented now in #344