Google Collab Crash for 20 qubit simulation

PennyLane Help
1

Hey! I think I posted this before but I’m having trouble again with this.

I’m simulating a 20 qubit system on Google Collab, using the lightning.gpu backend. On Google-collab, I’m using the A100 runtime as well.

Here is a self-contained version of the code:

# Test circuit:
# Imports relevant packages:
!pip install pennylane custatevec-cu12 pennylane-lightning-gpu
!pip install pyscf

# imports and relevant defines:
import pennylane as qml
from pennylane import qchem
from pennylane import numpy as np
import time

# Simulation starting parameters:
symbols = ["O", "H", "H", "O", "H", "H"]

# geometry array for water-dimer
x = np.array([ 0.        ,  0.        ,  0.        ,  1.563923  ,  0.98312309,
        -0.60365174, -0.78395943,  1.43700227,  1.04747968,  4.26606551,
         0.56799183,  1.68673518,  4.63354675,  1.15239926,  3.50338239,
         6.10594353,  0.1835633 ,  1.19292837], requires_grad=True)

active_electrons = 16
active_orbitals = 10


# define the hamiltonian needed to compute cost-function
def H(x):
    return qml.qchem.molecular_hamiltonian(symbols, x, charge=0, active_electrons = active_electrons, active_orbitals = active_orbitals, method = "pyscf")[0]

# Define Hartree-Fock State
hf = qml.qchem.hf_state(electrons=active_electrons, orbitals=active_orbitals*2)
print(hf)


# other parameters
qubits = active_orbitals*2
singles_select = np.load('singles_select_1610.npy').tolist()
doubles_select = np.load('doubles_select_1610.npy').tolist()
hf_state = hf
num_wires = qubits


# Define Quantum Circuit to take expectation value:
dev = qml.device("lightning.gpu", wires=num_wires, batch_obs=True)
@qml.qnode(dev, interface="autograd", diff_method="adjoint")
def circuit(params, obs, wires):
    # prepares Hartee-fock state:
    qml.BasisState(hf_state, wires=wires)

    # apply all single excitations
    for i, singles in enumerate(singles_select):
        qml.SingleExcitation(params[i], wires=singles)

    # apply all double excitations
    for j, doubles in enumerate(doubles_select):
        qml.DoubleExcitation(params[j + len(singles_select)], wires=doubles)

    # returns expectation value of the ansatz prepared from this quantum circuit:
    return qml.expval(obs)

########### Define the Objective Function: ###########
def cost(params, x):
    hamiltonian = H(x)
    return circuit(params, obs=hamiltonian, wires=range(num_wires))


# Define circuit parameters:
theta = np.array([0.0] * (len(doubles_select) + len(singles_select)), requires_grad=True) 
# Define Optimizer:
opt_theta = qml.GradientDescentOptimizer(stepsize=0.2)

# Perform a single update of theta:
theta.requires_grad = True
x.requires_grad = False

theta, _ = opt_theta.step(cost, theta, x)
print("Done with theta optimization!")

I’ve tried implementing the adjoint diffrentiation method as you can see for the QNode parameter. If anyone as any other methods to reduce memory, such as maybe some form of parallelization or an alternate differentiation, that would be much appreciated! I know it should it be possible to run this because I’ve seen posts of people simulating >25 qubits before!

By the way, I’ve also attached a file of the selected gates I used (‘singles_select_1610.npy’ and ‘doubles_select_1610.npy’) If you’re running this, if you could just simply put the files in the same directory as the code, that’s all that’s needed to run!

Thanks so much!

3

Updated code that’s fully self-contained (has all gates within it)

# Test circuit:
# Imports relevant packages:
!pip install pennylane custatevec-cu12 pennylane-lightning-gpu
!pip install pyscf

# imports and relevant defines:
import pennylane as qml
from pennylane import qchem
from pennylane import numpy as np
import time

# Simulation starting parameters:
symbols = ["O", "H", "H", "O", "H", "H"]

# geometry array for water-dimer
x = np.array([ 0.        ,  0.        ,  0.        ,  1.563923  ,  0.98312309,
        -0.60365174, -0.78395943,  1.43700227,  1.04747968,  4.26606551,
         0.56799183,  1.68673518,  4.63354675,  1.15239926,  3.50338239,
         6.10594353,  0.1835633 ,  1.19292837], requires_grad=True)

active_electrons = 16
active_orbitals = 10


# define the hamiltonian needed to compute cost-function
def H(x):
    return qml.qchem.molecular_hamiltonian(symbols, x, charge=0, active_electrons = active_electrons, active_orbitals = active_orbitals, method = "pyscf")[0]

# Define Hartree-Fock State
hf = qml.qchem.hf_state(electrons=active_electrons, orbitals=active_orbitals*2)
print(hf)


# other parameters
qubits = active_orbitals*2
singles_select = [[0, 16], [0, 18], [1, 17], [1, 19], [2, 16], [2, 18], [3, 17], 
 [3, 19], [4, 16], [4, 18], [5, 17], [5, 19], [6, 16], [6, 18], [7, 17], [7, 19], 
  [8, 16], [8, 18], [9, 17], [9, 19], [10, 16], [10, 18], [11, 17], [11, 19], 
   [12, 16], [12, 18], [13, 17], [13, 19], [14, 16], [14, 18], [15, 17], [15, 19]]
doubles_select = [[0, 1, 16, 17], [0, 1, 16, 19], [0, 1, 17, 18], [0, 1, 18, 19],
 [0, 2, 16, 18], [0, 3, 16, 17], [0, 3, 16, 19], [0, 3, 17, 18], [0, 3, 18, 19],
[0, 4, 16, 18], [0, 5, 16, 17], [0, 5, 16, 19], [0, 5, 17, 18],
[0, 5, 18, 19], [0, 6, 16, 18], [0, 7, 16, 17], [0, 7, 16, 19],
[0, 7, 17, 18], [0, 7, 18, 19], [0, 8, 16, 18], [0, 9, 16, 17],
[0, 9, 16, 19], [0, 9, 17, 18], [0, 9, 18, 19], [0, 10, 16, 18],
[0, 11, 16, 17], [0, 11, 16, 19], [0, 11, 17, 18], [0, 11, 18, 19],
[0, 12, 16, 18], [0, 13, 16, 17], [0, 13, 16, 19], [0, 13, 17, 18],
[0, 13, 18, 19], [0, 14, 16, 18], [0, 15, 16, 17], [0, 15, 16, 19],
[0, 15, 17, 18], [0, 15, 18, 19], [1, 2, 16, 17], [1, 2, 16, 19],
[1, 2, 17, 18], [1, 2, 18, 19], [1, 3, 17, 19], [1, 4, 16, 17],
[1, 4, 16, 19], [1, 4, 17, 18], [1, 4, 18, 19], [1, 5, 17, 19],
[1, 6, 16, 17], [1, 6, 16, 19], [1, 6, 17, 18], [1, 6, 18, 19],
[1, 7, 17, 19], [1, 8, 16, 17], [1, 8, 16, 19], [1, 8, 17, 18],
[1, 8, 18, 19], [1, 9, 17, 19], [1, 10, 16, 17], [1, 10, 16, 19],
[1, 10, 17, 18], [1, 10, 18, 19], [1, 11, 17, 19], [1, 12, 16, 17],
[1, 12, 16, 19], [1, 12, 17, 18], [1, 12, 18, 19], [1, 13, 17, 19],
[1, 14, 16, 17], [1, 14, 16, 19], [1, 14, 17, 18], [1, 14, 18, 19],
[1, 15, 17, 19], [2, 3, 16, 17], [2, 3, 16, 19], [2, 3, 17, 18],
[2, 3, 18, 19], [2, 4, 16, 18], [2, 5, 16, 17], [2, 5, 16, 19],
[2, 5, 17, 18], [2, 5, 18, 19], [2, 6, 16, 18], [2, 7, 16, 17],
[2, 7, 16, 19], [2, 7, 17, 18], [2, 7, 18, 19], [2, 8, 16, 18],
[2, 9, 16, 17], [2, 9, 16, 19], [2, 9, 17, 18], [2, 9, 18, 19],
[2, 10, 16, 18], [2, 11, 16, 17], [2, 11, 16, 19], [2, 11, 17, 18],
[2, 11, 18, 19], [2, 12, 16, 18], [2, 13, 16, 17], [2, 13, 16, 19],
[2, 13, 17, 18], [2, 13, 18, 19], [2, 14, 16, 18], [2, 15, 16, 17],
[2, 15, 16, 19], [2, 15, 17, 18], [2, 15, 18, 19], [3, 4, 16, 17],
[3, 4, 16, 19], [3, 4, 17, 18], [3, 4, 18, 19], [3, 5, 17, 19],
[3, 6, 16, 17], [3, 6, 16, 19], [3, 6, 17, 18], [3, 6, 18, 19],
[3, 7, 17, 19], [3, 8, 16, 17], [3, 8, 16, 19], [3, 8, 17, 18],
[3, 8, 18, 19], [3, 9, 17, 19], [3, 10, 16, 17], [3, 10, 16, 19],
[3, 10, 17, 18], [3, 10, 18, 19], [3, 11, 17, 19], [3, 12, 16, 17],
[3, 12, 16, 19], [3, 12, 17, 18], [3, 12, 18, 19], [3, 13, 17, 19],
[3, 14, 16, 17], [3, 14, 16, 19], [3, 14, 17, 18], [3, 14, 18, 19],
[3, 15, 17, 19], [4, 5, 16, 17], [4, 5, 16, 19], [4, 5, 17, 18],
[4, 5, 18, 19], [4, 6, 16, 18], [4, 7, 16, 17], [4, 7, 16, 19],
[4, 7, 17, 18], [4, 7, 18, 19], [4, 8, 16, 18], [4, 9, 16, 17],
[4, 9, 16, 19], [4, 9, 17, 18], [4, 9, 18, 19], [4, 10, 16, 18],
[4, 11, 16, 17], [4, 11, 16, 19], [4, 11, 17, 18], [4, 11, 18, 19],
[4, 12, 16, 18], [4, 13, 16, 17], [4, 13, 16, 19], [4, 13, 17, 18],
[4, 13, 18, 19], [4, 14, 16, 18], [4, 15, 16, 17], [4, 15, 16, 19],
[4, 15, 17, 18], [4, 15, 18, 19], [5, 6, 16, 17], [5, 6, 16, 19],
[5, 6, 17, 18], [5, 6, 18, 19], [5, 7, 17, 19], [5, 8, 16, 17],
[5, 8, 16, 19], [5, 8, 17, 18], [5, 8, 18, 19], [5, 9, 17, 19],
[5, 10, 16, 17], [5, 10, 16, 19], [5, 10, 17, 18], [5, 10, 18, 19],
[5, 11, 17, 19], [5, 12, 16, 17], [5, 12, 16, 19], [5, 12, 17, 18],
[5, 12, 18, 19], [5, 13, 17, 19], [5, 14, 16, 17], [5, 14, 16, 19],
[5, 14, 17, 18], [5, 14, 18, 19], [5, 15, 17, 19], [6, 7, 16, 17],
[6, 7, 16, 19], [6, 7, 17, 18], [6, 7, 18, 19], [6, 8, 16, 18],
[6, 9, 16, 17], [6, 9, 16, 19], [6, 9, 17, 18], [6, 9, 18, 19],
[6, 10, 16, 18], [6, 11, 16, 17], [6, 11, 16, 19], [6, 11, 17, 18],
[6, 11, 18, 19], [6, 12, 16, 18], [6, 13, 16, 17], [6, 13, 16, 19],
[6, 13, 17, 18], [6, 13, 18, 19], [6, 14, 16, 18], [6, 15, 16, 17],
[6, 15, 16, 19], [6, 15, 17, 18], [6, 15, 18, 19], [7, 8, 16, 17],
[7, 8, 16, 19], [7, 8, 17, 18], [7, 8, 18, 19], [7, 9, 16, 18],
[7, 10, 16, 17], [7, 10, 16, 19], [7, 10, 17, 18], [7, 10, 18, 19],
[7, 11, 16, 18], [7, 12, 16, 17], [7, 12, 16, 19], [7, 12, 17, 18],
[7, 12, 18, 19], [7, 13, 16, 18], [7, 14, 16, 17], [7, 14, 16, 19],
[7, 14, 17, 18], [7, 14, 18, 19], [7, 15, 16, 18], [8, 9, 16, 17],
[8, 9, 16, 19], [8, 9, 17, 18], [8, 9, 18, 19], [8, 10, 17, 19],
[8, 11, 16, 17], [8, 11, 16, 19], [8, 11, 17, 18], [8, 11, 18, 19],
[8, 12, 17, 19], [8, 13, 16, 17], [8, 13, 16, 19], [8, 13, 17, 18],
[8, 13, 18, 19], [8, 14, 17, 19], [8, 15, 16, 17], [8, 15, 16, 19],
[8, 15, 17, 18], [8, 15, 18, 19], [9, 10, 17, 19], [9, 11, 16, 17],
[9, 11, 16, 19], [9, 11, 17, 18], [9, 11, 18, 19], [9, 12, 17, 19],
[9, 13, 16, 17], [9, 13, 16, 19], [9, 13, 17, 18], [9, 13, 18, 19],
[9, 14, 17, 19], [9, 15, 16, 17], [9, 15, 16, 19], [9, 15, 17, 18],
[9, 15, 18, 19], [10, 11, 16, 17], [10, 11, 16, 19], [10, 11, 17, 18],
[10, 11, 18, 19], [10, 12, 16, 18], [10, 13, 16, 17], [10, 13, 16, 19],
[10, 13, 17, 18], [10, 13, 18, 19], [10, 14, 16, 18], [10, 15, 16, 17],
[10, 15, 16, 19], [10, 15, 17, 18], [10, 15, 18, 19], [11, 12, 16, 17],
[11, 12, 16, 19], [11, 12, 17, 18], [11, 12, 18, 19], [11, 13, 16, 18],
[11, 14, 16, 17], [11, 14, 16, 19], [11, 14, 17, 18], [11, 14, 18, 19],
[11, 15, 16, 18], [12, 13, 16, 17], [12, 13, 16, 19], [12, 13, 17, 18],
[12, 13, 18, 19], [12, 14, 16, 18], [12, 15, 16, 17], [12, 15, 16, 19],
[12, 15, 17, 18], [12, 15, 18, 19], [13, 14, 16, 17], [13, 14, 16, 19],
[13, 14, 17, 18], [13, 14, 18, 19], [13, 15, 16, 17], [13, 15, 16, 19],
[13, 15, 17, 18], [13, 15, 18, 19], [14, 15, 16, 17], [14, 15, 16, 19],
[14, 15, 17, 18], [14, 15, 18, 19]]
hf_state = hf
num_wires = qubits


# Define Quantum Circuit to take expectation value:
dev = qml.device("lightning.gpu", wires=num_wires, batch_obs=True)
@qml.qnode(dev, interface="autograd", diff_method="adjoint")
def circuit(params, obs, wires):
    # prepares Hartee-fock state:
    qml.BasisState(hf_state, wires=wires)

    # apply all single excitations
    for i, singles in enumerate(singles_select):
        qml.SingleExcitation(params[i], wires=singles)

    # apply all double excitations
    for j, doubles in enumerate(doubles_select):
        qml.DoubleExcitation(params[j + len(singles_select)], wires=doubles)

    # returns expectation value of the ansatz prepared from this quantum circuit:
    return qml.expval(obs)

########### Define the Objective Function: ###########
def cost(params, x):
    hamiltonian = H(x)
    return circuit(params, obs=hamiltonian, wires=range(num_wires))


# Define circuit parameters:
theta = np.array([0.0] * (len(doubles_select) + len(singles_select)), requires_grad=True) 
# Define Optimizer:
opt_theta = qml.GradientDescentOptimizer(stepsize=0.2)

# Perform a single update of theta:
theta.requires_grad = True
x.requires_grad = False

theta, _ = opt_theta.step(cost, theta, x)
print("Done with theta optimization!")

I’ve also tried circuit cutting using @partial(qml.cut_circuit, auto_cutter=True) and while the program doesn’t crash, the program is taking super long

4

Hi @ImranNasrullah ,

Thanks for sharing your full code.
There may be several things happening here.

On one hand the molecule you’re looking at is big and the number of gates is large too. 20 qubits is on the limit of what you can do on a laptop, but if you’re adding a lot of complexity you may run out of RAM. The fact that circuit cutting is helping prevent your circuit from crashing supports this hypothesis that you’re maxing out your RAM.

Using a GPU can indeed help in this case but since you’re using Colab you only have access to limited resources there. It’s possible that Colab is restricting your access to GPUs if you’re using them too much under their free tier.

Do you have the possibility of further optimizing your circuit before you run it?
You can try using for example qubit tapering for this.

I hope this helps!

5

Hey @CatalinaAlbornoz ,

Thanks for the reply! I can try qubit tapering and let you know how that works.

In addition, would you know if increasing number of steps in the gradient calculation of the gates in Adapt-VQE can help decrease the number of gates useful? So far, the number of steps I have is 20.

Or would there be any other way to decrease number of gates, because to my understanding, represent this entangling is what takes up the memory

Thanks!

6

Hi @ImranNasrullah,

Usually the memory requirements are mainly influenced by the number of qubits and gates but in different factors. Adding an extra qubit will duplicate your memory requirements while adding one extra gate won’t. This means that you’ll need to reduce the number of gates by a lot in order to stop colab from crashing.

The number of steps might help you achieve a higher accuracy. Generally 20 steps is very very small. Most algorithms use over 100 steps. However I wouldn’t increase this unless you reduce the number of qubits or gates.

I hope this helps!