Qualcs device and qml.grad() timings

I am wanting to run simulations with a higher number of qubits, so I realized that I might need to switch off the default qubit and into some other device. Looking through the device options, I think that qulacs might suit my needs, to I used this device. However, I am quickly finding that qml.grad()-function seems significantly slower using this device. I tried to run an N = 16 qubit example where I simply plot the gradient to see how it looks, and a gradient calculation using backprop which took 6 seconds now seemingly takes 102.68 seconds. Using finite differences instead, the calculation is faster (3.55 sec instead of ~5 seconds using default). I am not familiar with qulacs, so I am wondering why this is? Is it because it is highly optimized for forward calls and not for gradient-calculations like the default qubit is? I also tried this with the lightning qubit, where I find similar trends (qml.grad() takes 84 sec, finite difference takes 2.72 sec). My goal is essentially to run maxcut using QAOA for qubit counts of around 20+ but with some classical post processing using a neural network. Which qubit-device would be reccomended for such needs? Note that I also wish to calculate the gradient, either through backprop or other methods implemented in pennylane or using finite differences.

At some parts, the cost function is given by qml.expcal(H), while in other the output is qml.sample() which is post processed by a neural network. I wish to optimize the parts with qml.exval() using the gradient functinalities in Pennylane while the qml.sample() part is handled through finite differences.

I tried to upload the python file that reproduces the findings that I noted, however cannot seem to attach any files into the topic (the file is of .py type, so I can’t seem why I would get any errors).

Any aid in understanding why the qml.grad() function gives so different timings would be appreciated, from which I can choose the correct device.

Hi @Viro, thanks for the question.

Is it because it is highly optimized for forward calls and not for gradient-calculations like the default qubit is?

I believe you have nailed the issue here. PennyLane’s built-in simulator can leverage backpropagation and hence can be much faster for pipelines that require gradients. Qulacs, from my last recollection, supports parameter-shift rule gradients only.

@Viro if you’d like to learn more about the performance differences between backprop and parameter-shift in the context of variational algorithms, you might find this demo interesting: https://pennylane.ai/qml/demos/tutorial_backprop.html

In the meantime, you could also try out the lightning.qubit device. By default, this device will also use the parameter-shift rule, but if you are using shots=None, it also supports an optimized gradient method called the ‘adjoint’ method.

You can use it like so:

dev = qml.device("lightning.qubit", wires=2)

@qml.qnode(dev, diff_method="adjoint")
def circuit(weights):

Seems interesting. Hoping to give it a spin, I changed the diff-method to adjoint, however I seemingly get this error:

OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

I also tried the fix by setting os.environ[“KMP_DUPLICATE_LIB_OK”]=“TRUE”, however this does not solve the issue and causes a crash. Is this a known issue, and how do I go about solving it?

Note that this issue does not appear when I don’t specify the diff_method.

Hi @Viro thanks for letting us know. Can I ask for some information that may help us identify the cause of this:

  • Can you provide us with a minimum working example of the script that replicates the issue?
  • Are you using a conda env or virtualenv for your Python environment?
  • Are you running on an M1 or Intel Mac?
  • Do you have brew installed, and if so, is clang or libomp installed through brew?

As I mentioned earlier, I cannot seem to attach any files for some weird reason, so I copy pasted the example. It is really short, so I hope it’s fine that I copy pasted it.

import pennylane as qml
from pennylane import numpy as np

dev = qml.device('lightning.qubit',wires = 2,shots = 10000)
devAnalytical = qml.device('lightning.qubit',wires = 2)

def circuit(params):
    qml.Hadamard(wires = 0)
    qml.RX(params,wires =  0)
    return qml.expval(qml.PauliZ(wires = 0))

waht = qml.QNode(circuit,devAnalytical,diff_method='adjoint')
params = np.array([0.4])

This does indeed replicate the issue I mentioned arlier.

I am running a conda environment. My packages are as follows:

# Name                    Version                   Build  Channel
_ipyw_jlab_nb_ext_conf    0.1.0            py39hecd8cb5_0  
absl-py                   1.0.0                    pypi_0    pypi
anaconda-client           1.9.0            py39hecd8cb5_0  
anaconda-navigator        2.1.1                    py39_0  
anyio                     2.2.0            py39hecd8cb5_1  
appdirs                   1.4.4                    pypi_0    pypi
appnope                   0.1.2           py39hecd8cb5_1001  
argon2-cffi               20.1.0           py39h9ed2024_1  
astunparse                1.6.3                    pypi_0    pypi
async_generator           1.10               pyhd3eb1b0_0  
attrs                     21.4.0             pyhd3eb1b0_0  
autograd                  1.3                      pypi_0    pypi
autoray                   0.2.5                    pypi_0    pypi
babel                     2.9.1              pyhd3eb1b0_0  
backcall                  0.2.0              pyhd3eb1b0_0  
backports                 1.1                pyhd3eb1b0_0  
backports.functools_lru_cache 1.6.4              pyhd3eb1b0_0  
backports.tempfile        1.0                pyhd3eb1b0_1  
backports.weakref         1.0.post1                  py_1  
beautifulsoup4            4.10.0             pyh06a4308_0  
blas                      1.0                         mkl  
bleach                    4.1.0              pyhd3eb1b0_0  
bottleneck                1.3.2            py39he3068b8_1  
brotli                    1.0.9                hb1e8313_2  
brotlipy                  0.7.0           py39h9ed2024_1003  
bzip2                     1.0.8                h1de35cc_0  
ca-certificates           2021.10.26           hecd8cb5_2  
cachetools                4.2.4                    pypi_0    pypi
certifi                   2021.10.8        py39hecd8cb5_2  
cffi                      1.15.0           py39hc55c11b_1  
chardet                   4.0.0           py39hecd8cb5_1003  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
click                     8.0.3              pyhd3eb1b0_0  
clyent                    1.2.2            py39hecd8cb5_1  
conda                     4.11.0           py39hecd8cb5_0  
conda-build               3.21.7           py39hecd8cb5_0  
conda-content-trust       0.1.1              pyhd3eb1b0_0  
conda-env                 2.6.0                         1  
conda-package-handling    1.7.3            py39h9ed2024_1  
conda-repo-cli            1.0.4              pyhd3eb1b0_0  
conda-token               0.3.0              pyhd3eb1b0_0  
conda-verify              3.4.2                      py_1  
cryptography              3.4.8            py39h2fd3fbb_0  
cvxgraphalgs              0.1.2                    pypi_0    pypi
cvxpy                     1.1.18                   pypi_0    pypi
cycler                    0.11.0             pyhd3eb1b0_0  
debugpy                   1.5.1            py39he9d5cce_0  
decorator                 4.4.2                    pypi_0    pypi
defusedxml                0.7.1              pyhd3eb1b0_0  
dill                      0.3.4                    pypi_0    pypi
docplex                   2.22.213                 pypi_0    pypi
ecos                      2.0.10                   pypi_0    pypi
entrypoints               0.3              py39hecd8cb5_0  
ffmpeg                    4.2.2                h97e5cf8_0  
filelock                  3.4.2              pyhd3eb1b0_0  
flatbuffers               2.0                      pypi_0    pypi
fonttools                 4.25.0             pyhd3eb1b0_0  
freetype                  2.11.0               hd8bbffd_0  
future                    0.18.2           py39hecd8cb5_1  
gast                      0.4.0                    pypi_0    pypi
gettext                   0.21.0               h7535e17_0  
giflib                    5.2.1                haf1e3a3_0  
glob2                     0.7                pyhd3eb1b0_0  
gmp                       6.2.1                h23ab428_2  
gnutls                    3.6.15               hed9c0bf_0  
google-auth               2.3.3                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
google-pasta              0.2.0                    pypi_0    pypi
grpcio                    1.43.0                   pypi_0    pypi
icu                       58.2                 h0a44026_3  
idna                      3.3                pyhd3eb1b0_0  
importlib-metadata        4.8.2            py39hecd8cb5_0  
importlib_metadata        4.8.2                hd3eb1b0_0  
intel-openmp              2021.4.0          hecd8cb5_3538  
ipykernel                 6.4.1            py39hecd8cb5_1  
ipython                   7.31.1           py39hecd8cb5_0  
ipython_genutils          0.2.0              pyhd3eb1b0_1  
ipywidgets                7.6.5              pyhd3eb1b0_1  
jedi                      0.18.1           py39hecd8cb5_0  
jinja2                    2.11.3             pyhd3eb1b0_0  
joblib                    1.1.0              pyhd3eb1b0_0  
jpeg                      9d                   h9ed2024_0  
json5                     0.9.6              pyhd3eb1b0_0  
jsonschema                3.2.0              pyhd3eb1b0_2  
jupyter_client            7.1.2              pyhd3eb1b0_0  
jupyter_core              4.9.1            py39hecd8cb5_0  
jupyter_server            1.4.1            py39hecd8cb5_0  
jupyterlab                3.2.1              pyhd3eb1b0_1  
jupyterlab_pygments       0.1.2                      py_0  
jupyterlab_server         2.10.2             pyhd3eb1b0_1  
jupyterlab_widgets        1.0.0              pyhd3eb1b0_1  
keras                     2.7.0                    pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
kiwisolver                1.3.1            py39h23ab428_0  
lame                      3.100                h1de35cc_0  
lcms2                     2.12                 hf1fd2bf_0  
libarchive                3.4.2                haa3ed63_0  
libclang                  12.0.0                   pypi_0    pypi
libcxx                    12.0.0               h2f01273_0  
libffi                    3.3                  hb1e8313_2  
libgfortran               3.0.1                h93005f0_2  
libiconv                  1.16                 h1de35cc_0  
libidn2                   2.3.2                h9ed2024_0  
liblief                   0.10.1               h23ab428_1  
libllvm11                 11.1.0               h9b2ccf5_0  
libopus                   1.3.1                h1de35cc_0  
libpng                    1.6.37               ha441bb4_0  
libsodium                 1.0.18               h1de35cc_0  
libtasn1                  4.16.0               h9ed2024_0  
libtiff                   4.2.0                h87d7836_0  
libunistring              0.9.10               h9ed2024_0  
libuv                     1.40.0               haf1e3a3_0  
libvpx                    1.7.0                h378b8a2_0  
libwebp                   1.2.0                hacca55c_0  
libwebp-base              1.2.0                h9ed2024_0  
libxml2                   2.9.12               hcdb78fc_0  
llvm-openmp               12.0.0               h0dcd299_1  
llvmlite                  0.37.0           py39he4411ff_1  
lz4-c                     1.9.3                h23ab428_1  
markdown                  3.3.6                    pypi_0    pypi
markupsafe                2.0.1            py39h9ed2024_0  
matplotlib                3.5.0            py39hecd8cb5_0  
matplotlib-base           3.5.0            py39h4f681db_0  
matplotlib-inline         0.1.2              pyhd3eb1b0_2  
mistune                   0.8.4           py39h9ed2024_1000  
mkl                       2021.4.0           hecd8cb5_637  
mkl-service               2.4.0            py39h9ed2024_0  
mkl_fft                   1.3.1            py39h4ab4a9b_0  
mkl_random                1.2.2            py39hb2f4e1b_0  
mpmath                    1.2.1                    pypi_0    pypi
munkres                   1.1.4                      py_0  
navigator-updater         0.2.1                    py39_1  
nbclassic                 0.2.6              pyhd3eb1b0_0  
nbclient                  0.5.3              pyhd3eb1b0_0  
nbconvert                 6.3.0            py39hecd8cb5_0  
nbformat                  5.1.3              pyhd3eb1b0_0  
ncurses                   6.3                  hca72f7f_2  
nest-asyncio              1.5.1              pyhd3eb1b0_0  
nettle                    3.7.3                h230ac6f_1  
networkx                  2.5.1                    pypi_0    pypi
ninja                            pypi_0    pypi
notebook                  6.4.6            py39hecd8cb5_0  
ntlm-auth                 1.5.0                    pypi_0    pypi
numba                     0.54.1           py39hae1ba45_0  
numexpr                   2.8.1            py39h2e5f0a9_0  
numpy                     1.20.3           py39h4b4dc7a_0  
numpy-base                1.20.3           py39he0bd621_0  
oauthlib                  3.1.1                    pypi_0    pypi
olefile                   0.46               pyhd3eb1b0_0  
openh264                  2.1.1                h8346a28_0  
openssl                   1.1.1m               hca72f7f_0  
opt-einsum                3.3.0                    pypi_0    pypi
osqp                      0.6.2.post5              pypi_0    pypi
packaging                 21.3               pyhd3eb1b0_0  
pandas                    1.3.5            py39h743cdd8_0  
pandocfilters             1.5.0              pyhd3eb1b0_0  
parso                     0.8.3              pyhd3eb1b0_0  
pbr                       5.8.0                    pypi_0    pypi
pennylane                 0.20.0                   pypi_0    pypi
pennylane-lightning       0.20.1                   pypi_0    pypi
pennylane-qiskit          0.20.0                   pypi_0    pypi
pennylane-qulacs          0.16.0                   pypi_0    pypi
pexpect                   4.8.0              pyhd3eb1b0_3  
pickleshare               0.7.5           pyhd3eb1b0_1003  
pillow                    8.4.0            py39h98e4679_0  
pip                       21.2.4           py39hecd8cb5_0  
pkginfo                   1.8.2              pyhd3eb1b0_0  
ply                       3.11                     pypi_0    pypi
prometheus_client         0.13.1             pyhd3eb1b0_0  
prompt-toolkit            3.0.20             pyhd3eb1b0_0  
protobuf                  3.19.3                   pypi_0    pypi
psutil                    5.8.0            py39h9ed2024_1  
ptyprocess                0.7.0              pyhd3eb1b0_2  
py-lief                   0.10.1           py39h23ab428_1  
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pycosat                   0.6.3            py39h9ed2024_0  
pycparser                 2.21               pyhd3eb1b0_0  
pygments                  2.10.0             pyhd3eb1b0_0  
pyjwt                     2.1.0            py39hecd8cb5_0  
pyopenssl                 21.0.0             pyhd3eb1b0_1  
pyparsing                 3.0.4              pyhd3eb1b0_0  
pyqt                      5.9.2            py39h23ab428_6  
pyrsistent                0.18.0           py39hca72f7f_0  
pysocks                   1.7.1            py39hecd8cb5_0  
python                    3.9.7                h88f2d9e_1  
python-constraint         1.4.0                    pypi_0    pypi
python-dateutil           2.8.2              pyhd3eb1b0_0  
python-libarchive-c       2.9                pyhd3eb1b0_1  
python.app                3                py39hca72f7f_0  
pytorch                   1.10.2                  py3.9_0    pytorch
pytz                      2021.3             pyhd3eb1b0_0  
pyyaml                    6.0              py39hca72f7f_1  
pyzmq                     22.3.0           py39he9d5cce_2  
qdldl                     0.1.5.post0              pypi_0    pypi
qiskit                    0.34.0                   pypi_0    pypi
qiskit-aer                0.10.1                   pypi_0    pypi
qiskit-ibmq-provider      0.18.3                   pypi_0    pypi
qiskit-ignis              0.7.0                    pypi_0    pypi
qiskit-optimization       0.3.1                    pypi_0    pypi
qiskit-terra              0.19.1                   pypi_0    pypi
qt                        5.9.7                h468cd18_1  
qtpy                      1.11.2             pyhd3eb1b0_0  
qulacs                    0.3.0                    pypi_0    pypi
readline                  8.1.2                hca72f7f_1  
requests                  2.27.1             pyhd3eb1b0_0  
requests-ntlm             1.1.0                    pypi_0    pypi
requests-oauthlib         1.3.0                    pypi_0    pypi
retworkx                  0.10.2                   pypi_0    pypi
ripgrep                   12.1.1                        0  
rsa                       4.8                      pypi_0    pypi
ruamel_yaml               0.15.100         py39h9ed2024_0  
scipy                     1.7.3            py39h8c7af03_0  
scs                       3.0.1                    pypi_0    pypi
seaborn                   0.11.2             pyhd3eb1b0_0  
semantic-version          2.6.0                    pypi_0    pypi
send2trash                1.8.0              pyhd3eb1b0_1  
setuptools                58.0.4           py39hecd8cb5_0  
sip                       4.19.13          py39h23ab428_0  
six                       1.16.0             pyhd3eb1b0_0  
snakeviz                  2.1.1                    pypi_0    pypi
sniffio                   1.2.0            py39hecd8cb5_1  
soupsieve                 2.3.1              pyhd3eb1b0_0  
sqlite                    3.37.0               h707629a_0  
stevedore                 3.5.0                    pypi_0    pypi
symengine                 0.8.1                    pypi_0    pypi
sympy                     1.9                      pypi_0    pypi
tbb                       2021.5.0             haf03e11_0  
tensorboard               2.7.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tensorflow                2.7.0                    pypi_0    pypi
tensorflow-estimator      2.7.0                    pypi_0    pypi
tensorflow-io-gcs-filesystem 0.23.1                   pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
terminado                 0.9.4            py39hecd8cb5_0  
testpath                  0.5.0              pyhd3eb1b0_0  
tk                        8.6.11               h7bc2e8c_0  
toml                      0.10.2                   pypi_0    pypi
torchaudio                0.10.2                 py39_cpu    pytorch
torchvision               0.11.3                 py39_cpu    pytorch
tornado                   6.1              py39h9ed2024_0  
tqdm                      4.62.3             pyhd3eb1b0_1  
traitlets                 5.1.1              pyhd3eb1b0_0  
tweedledum                1.1.1                    pypi_0    pypi
typing_extensions           pyh06a4308_0  
tzdata                    2021e                hda174b7_0  
ujson                     4.2.0            py39he9d5cce_0  
urllib3                   1.26.8             pyhd3eb1b0_0  
wcwidth                   0.2.5              pyhd3eb1b0_0  
webencodings              0.5.1            py39hecd8cb5_1  
websocket-client          1.2.3                    pypi_0    pypi
wheel                     0.37.1             pyhd3eb1b0_0  
widgetsnbextension        3.5.1            py39hecd8cb5_0  
x264                      1!157.20191217       h1de35cc_0  
xmltodict                 0.12.0             pyhd3eb1b0_0  
xz                        5.2.5                h1de35cc_0  
yaml                      0.2.5                haf1e3a3_0  
zeromq                    4.3.4                h23ab428_0  
zipp                      3.7.0              pyhd3eb1b0_0  
zlib                      1.2.11               h4dc903c_4  
zstd                      1.4.9                h322a384_0  

I have an intel mac.

Apple clang version 13.0.0 (clang-1300.0.27.3)
Target: x86_64-apple-darwin21.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I think this is the stats that you are looking for? Not really all too familiar with all of this, hehe. As a side note, I’d like to mention that I have encountered similar issues with regards to running qiskit, and had to use the os.environ[“KMP_DUPLICATE_LIB_OK”]=“TRUE” to get those simulations to run, so that might be an indicator as to where the issue might be? Hope this is the information that you guys are looking for, and let me know if there is some other information that might be missing

Thanks @Viro
I suspect the issue here is that another package in your environment has already brought in libomp (or another OpenMP variant). Unfortunately, this seems to be a known issue with MacOS, as you mention, so the best option is for us to find a mitigation strategy that works.

I think there are several options we can try that may help solve this:

  • Attempting to use the environment variable trick (but you say this does not help in your case).
  • Avoid using Conda for the python environment, and simply create a bare Python3 env using python3 -m venv pyenv && source ./pyenv/bin/activate along with pip to install all packages. This will ensure the PyPI builds of packages are brought in, and in many cases tend to be better supported than the conda versions. If a conda package is causing the OpenMP library issue, this one will likely solve that.
  • If you wish to stay with conda, maybe try creating a new conda environment and attempt to only install the bare minimum packages; in this case, conda install nomkl before any others may be required to favour use of OpenBLAS over MKL, which should avoid bringing in any additional OpenMP libraries. I suspect the culprit here may be libiomp5 being brought in through numpy, so this may help.

Can you try these options and let us know if it helps?