Hi @leolettuce, thanks for the question.
For the LightningGPU runs, anything under 20 qubits will be faster on the CPU — the GPU execution is optimal for 21-30 range workloads, and especially when calculating multiple expectation values. The plot range from 21 - 27 qubits on the blog post should hold for the given GPUs and comparing against the same number of CPU threads. This is largely due to overheads in setting up of the GPU device, as well as internal to the cuQuantum library. It is a known issue, and we should see improvements with future versions of the NVIDIA cuQuantum library releases.
The data was collected on a DGXA100 box, so you should be able to reproduce it over the range the plot shows. For QAOA, depending on how you have setup the circuit, the depth may also not be sufficient to take advantage of the GPU.
We should see the best performance for circuits with multiple expectation values, beyond 20 qubits, and with a number of layers of depth. The blog-post problem should be listed below the plot, and shows the evaluation of a Jacobian over multiple parameters for a strongly-entangling layered circuit.
Feel free to let me know if there are any follow-up questions.