Optimization could be a bottleneck in VQE

[MatteoBarbieriE4]

In variational algorithms, such as VQE and QAOA, there is the necessity to use optimizers, such as COBYLA.

However in some work we found that the optimization of the parameters is a bit of a bottleneck: running the quantum circuit on GPU is very fast, however the minimization of the cost function takes a lot of time. Profiling the usage of the CPU and the GPU it seems that the optimization takes place on the CPU and this is maybe the reason why we observe vary long times of the whole variational algorithm.

Any thoughts on this?
Are there optimizers that run on GPU?