Proper gradient and differentiation method for default.tensor with tensorflow

PennyLane Help
1

I have a GAN with a quantum generator using pennylane for the generator.
I use tensorflow (tf.GradientTape) to train the model.
I want to use default.tensor so I can make use of mps. The problem is that when running the circuit on default.tensor the function tape.gradient takes incredibly long while using default.qubit does not take as long. I was wondering what the optimal parameters are for the default.tensor device as I don’t think backpropagation works for instance.

I am now using: "default.tensor",, method="mps" and interface='tf', diff_method="parameter-shift".

Also the qml.qnn.KerasLayer does not seem to work with default.tensor and thus I have the QNode wrapped in custom layer i.e. class QuantumLayer(tf.keras.layers.Layer): etc.

Thank you in advance for the help.

Output from qml.about():

Name: PennyLane
Version: 0.39.0
Summary: PennyLane is a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
Home-page: https://github.com/PennyLaneAI/pennylane
Author: 
Author-email: 
License: Apache License 2.0
Location: /Users/lucas/MPS_Quantum_Finance/.venv/lib/python3.10/site-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, packaging, pennylane-lightning, requests, rustworkx, scipy, toml, typing-extensions
Required-by: PennyLane-Cirq, PennyLane_Lightning

Platform info:           macOS-10.16-x86_64-i386-64bit
Python version:          3.10.15
Numpy version:           1.26.4
Scipy version:           1.14.1
Installed devices:
- default.clifford (PennyLane-0.39.0)
- default.gaussian (PennyLane-0.39.0)
- default.mixed (PennyLane-0.39.0)
- default.qubit (PennyLane-0.39.0)
- default.qutrit (PennyLane-0.39.0)
- default.qutrit.mixed (PennyLane-0.39.0)
- default.tensor (PennyLane-0.39.0)
- null.qubit (PennyLane-0.39.0)
- reference.qubit (PennyLane-0.39.0)
- cirq.mixedsimulator (PennyLane-Cirq-0.39.0)
- cirq.pasqal (PennyLane-Cirq-0.39.0)
- cirq.qsim (PennyLane-Cirq-0.39.0)
- cirq.qsimh (PennyLane-Cirq-0.39.0)
- cirq.simulator (PennyLane-Cirq-0.39.0)
- lightning.qubit (PennyLane_Lightning-0.39.0)
None
2

Hi @Lucas ,

I noticed a couple of things that can help:

  • default.tensor does not currently support differentiation.
    If you need differentiation then default.qubit might be best. If you want to find other ways of using default.tensor you can check out our demos that use default.tensor or the documentation for default.tensor.

  • diff_method="parameter-shift" is the slowest method.
    Should you choose to change the device, it’s generally best not to set the diff_method. If you don’t set it then PennyLane will automatically choose the best diff_method for the device you’re using.

  • qnn.KerasLayer will be deprecated soon.
    I would recommend using PyTorch instead if possible. The demo on TorchLayers can be helpful here.

  • If possible, upgrade your version of PennyLane.
    While this is probably not the cause for the issues you’re seeing, it’s always best to use the latest PennyLane version. At the moment the latest stable version is v0.40. You can always check what’s the latest version at the top of the pennylane.ai homepage. You can upgrade the version with pip install pennylane --upgrade.

I hope this helps!

3

Thank you for your reply.
As my research is based on mps is there another way of using mps with differentiation on pennylane?

4

Also on the Pennylane website it says that default.tensor does support differentiation when using the parameter shift rule: https://pennylane.ai/devices/default-tensor

So is it that parameter-shift rule just much slower and that’s why the code the code is slow?

Thank you again for your help

5

Hi @Lucas ,

You’re totally right. default.tensor doesn’t natively support differentiation with backprop and adjoint, which are more performant. But PennyLane does support other methods such as parameter-shift on all devices including default.tensor.

One way you can test the cause for the slowdown is to set diff_method="parameter-shift" when using default.qubit. If you get a big slowdown then this will be an indication that the diff_method might be the main cause.

Please let us know what you find with this test! You can try using tools like timeit to find execution times or a profiler such as snakeviz to get a deeper look into the slower processes. Let me know if you need an example on how to use timeit.

I hope this helps!

6

Hi Catalina,

Thank you for the tip. I checked and:

When using default.qubit the code is fine with parameter-shift (though it is obviously slower then using best). It is just when changing to default.tensor that it slows down massively.
So it would seem the issue is with changing the device on the Qnode. However, the slow down occurs when I am training the generator(or discriminator) in my GAN where I call the Qnode for creating a fake sample for the discriminator.

The tf.GradientTape method has to propagate through the generator, even though the generator’s weights aren’t being updated or any gradients calculated, which only happens later. So what I am thinking is that if the generator’s computational graph is complex or inefficient (maybe because mps involves more steps), this is the bottleneck. Not necessarily computing the gradients but just tracking the operations in the quantum circuit itself.

I hope this makes sense and I was clear enough with my explanation.
Thank you.

7

Thanks for sharing these insights @Lucas ! They’re very useful.

Are you able to share some code that we could use to investigate the issue further?
For example, if you based you code on the PennyLane demo on Quantum GANs, what parts of the code did you change? If you can share an example that we can reproduce then we may be able to identify the cause of the slowdown.