Multiple resets crash SPSAOptimizer() in ver 0.35

Hi, looks like PennyLane ver 0.35 improved memory usage for the feed-forward circuits but it broke optimization for qml.SPSAOptimizer()
Attached demonstrator runs in ver 0.34

core@3fa108d51d4a:~/issues$ pip3 list |grep Pen
PennyLane                 0.34.0
PennyLane-Cirq            0.34.0
PennyLane-Lightning       0.34.0
PennyLane-Lightning-GPU   0.34.0
PennyLane-qiskit          0.34.0
PennyLane-SF              0.29.0

I see a reasonable output - the code is crunching the data

0: ──RX(0.01)──RY(0.01)──┤↗│  │0⟩──X──┤↗│  │0⟩──X─┤  <Z>
                          ╚════════╝   ║        ║       
                                       ╚════════╝        
 run circ with 2 resets ...
input X=[0.011 0.012]  Y=1.00   shots=1000  elaT=0.0 sec  nReset=2
  Optimization loop ...
Step 1: Cost = 1.6848  elaT=0.0 sec
Step 2: Cost = 1.6848  elaT=0.0 sec
Step 3: Cost = 1.6900  elaT=0.0 sec

But when I switch to

core@1f35d1998d35:~/issues$  pip3 list |grep Pen
PennyLane                       0.35.1
PennyLane-Cirq                  0.34.0
PennyLane_Lightning             0.35.1
PennyLane_Lightning_GPU         0.35.1
PennyLane-qiskit                0.35.1
PennyLane-SF                    0.29.0

I only pass testing that circuit can be executed and I get this crash in the params = optimizer.step(..)

Optimization loop ...
Traceback (most recent call last):
  File "/PennyLane/issues/./reset_mutip_SPSAOpt.py", line 56, in <module>
    params = optimizer.step(lambda p: cost(p, target_value), params)
  File "/usr/local/lib/python3.10/dist-packages/pennylane/optimize/spsa.py", line 219, in step
    g = self.compute_grad(objective_fn, args, kwargs)
….
  File "/usr/lib/python3.10/functools.py", line 889, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/usr/local/lib/python3.10/dist-packages/pennylane/devices/qubit/apply_operation.py", line 268, in apply_mid_measure
    sample = np.random.binomial(1, probs[1])
  File "numpy/random/mtrand.pyx", line 3486, in numpy.random.mtrand.RandomState.binomial
  File "_common.pyx", line 430, in numpy.random._common.check_constraint
ValueError: p < 0, p > 1 or p is NaN

This is complete demonstrator
BTW, if only 1 reset is used the optimizer still works in 0.35

import pennylane as qml
from pennylane import numpy as np
num_qubit=1 ; shots=1000
dev = qml.device('default.qubit', wires=num_qubit,shots=shots)
from time import time

nReset=2

@qml.qnode(dev)
def circuit(params):
    qml.RX(params[0], wires=0)
    qml.RY(params[1], wires=0)
    for j in range(nReset):
        m = qml.measure(0,reset=True)
        qml.cond(m, qml.PauliX)(0)
    return qml.expval(qml.PauliZ(0) )

# Initialize parameters
params = np.array([0.011, 0.012], requires_grad=True)

print(qml.draw(circuit, decimals=2)(params), '\n')
T0=time()
print(' run circ with %d resets ...'%nReset)
y = circuit(params)
elaT=time()-T0
print('input X=%s  Y=%.2f   shots=%d  elaT=%.1f sec  nReset=%d'%(params,y,shots,elaT, nReset))

#....  training  code
def cost(params, target_value):
    global function_calls
    return (circuit(params) - target_value) ** 2

num_steps=3
optimizer=qml.SPSAOptimizer(maxiter=num_steps)


# Set the target expectation value
target_value = -0.3
T0=time()

print('\n  Optimization loop ...')
for i in range(num_steps):

    params = optimizer.step(lambda p: cost(p, target_value), params)
    prev_cost = cost(params, target_value)
    elaT=time()-T0    
    print("Step %d: Cost = %.4f  elaT=%.1f sec" % (i+1, float(prev_cost),elaT))

Please advice how to make it work for nReset=30
Thanks, Jan

Hi @Jan_Balewski

I copied and pasted your code and tested it with nReset=5 and 10 and 30, and it’s working. I have :

PennyLane                                    0.35.1
PennyLane_Lightning                          0.35.1
PennyLane-qiskit                             0.34.1

For nReset=30:

0: ──RX(0.01)──RY(0.01)──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩
                          ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║   ║      
                                       ╚════════╝   ║        ║   ║        ║   ║        ║   ║      
                                                    ╚════════╝   ║        ║   ║        ║   ║      
                                                                 ╚════════╝   ║        ║   ║      
                                                                              ╚════════╝   ║      
                                                                                           ╚══════
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  
                                                                                                  

───X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X
   ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
   ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
   ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
   ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
   ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
═══╝   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
       ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
                    ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
                                 ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║
                                              ╚════════╝   ║        ║   ║        ║   ║        ║
                                                           ╚════════╝   ║        ║   ║        ║
                                                                        ╚════════╝   ║        ║
                                                                                     ╚════════╝
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               
                                                                                               

───┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
                 ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
                              ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║
                                           ╚════════╝   ║        ║   ║        ║   ║        ║
                                                        ╚════════╝   ║        ║   ║        ║
                                                                     ╚════════╝   ║        ║
                                                                                  ╚════════╝
                                                                                            
                                                                                            
                                                                                            
                                                                                            
                                                                                            
                                                                                            
                                                                                            
                                                                                            
                                                                                            
                                                                                            

───┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
    ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
                 ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║   ║        ║
                              ╚════════╝   ║        ║   ║        ║   ║        ║   ║        ║
                                           ╚════════╝   ║        ║   ║        ║   ║        ║
                                                        ╚════════╝   ║        ║   ║        ║
                                                                     ╚════════╝   ║        ║
                                                                                  ╚════════╝
                                                                                            
                                                                                            
                                                                                            

───┤↗│  │0⟩──X──┤↗│  │0⟩──X──┤↗│  │0⟩──X─┤  <Z>
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ║        ║   ║        ║   ║        ║       
    ╚════════╝   ║        ║   ║        ║       
                 ╚════════╝   ║        ║       
                              ╚════════╝        

 run circ with 30 resets ...```

Hi Christophe,
thanks for verifying. To be sure, the printing of circuit always worked for me, but the crash happens after run irc with 30 resets ... - I’m assuming you have seen the reports of finishing epoch as well.

How do we proceed? This time I’m 99.9% sure I reported the software stack correctly.
Out of 3 modules you listed 2 are the same:

PennyLane                                    0.35.1
PennyLane_Lightning                          0.35.1

I have more libs and an older PennyLane-qiskit - but this one is not used in this demo, I presume.
This is the Docker file I used to build the image:
https://bitbucket.org/balewski/jannersc/src/master/dockerVaria/ubuntu/Dockerfile.ubu22-qiskit1.0
To be precise I’m using Podman, not Docker.
May I suggest you look into it - is there anything wrong? Could you check if you can run the same test using this image? Unless you hove other suggestion how to converge?
Thanks
Jan

follow up . Below is minimal Docker file. It installs only Python3 & this 2 PennyLane modules

Python 3.10.12
core@b0bc701f69cd:~/issues$ pip3 list |grep Pen
PennyLane           0.35.1
PennyLane_Lightning 0.35.1

I still see the crash for nReset=2

0: ──RX(0.01)──RY(0.01)──┤↗│  │0⟩──X──┤↗│  │0⟩──X─┤  <Z>
                          ╚════════╝   ║        ║       
                                       ╚════════╝        

 run circ with 2 resets ...
..
 File "/usr/local/lib/python3.10/dist-packages/pennylane/devices/qubit/apply_operation.py", line 268, in apply_mid_measure
    sample = np.random.binomial(1, probs[1])
  File "mtrand.pyx", line 3412, in numpy.random.mtrand.RandomState.binomial
  File "_common.pyx", line 395, in numpy.random._common.check_constraint
ValueError: p < 0, p > 1 or p is NaN

Can you confirm/deny the code posted earlier in this thread crashes inside image produced by the Dockerfile below ?

FROM ubuntu:22.04
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=America/Los_Angeles
RUN echo 1a-AAAAAAAAAAAAAAAAAAAAAAAAAAAAA  OS update   && \
    apt-get update && \
    apt-get install -y  build-essential python3-pip   python3-tk python3-scipy python3-dev   && \
    apt-get clean all
RUN echo 3a-AAAAAAAAAAAAAAAAAAAAAAAAAAAAA   pennylane  && \
    pip3 install pennylane

I noticed the crash is in plain numpy , whereas you typically overwrite np with numpy from PennyLane - maybe this shadowing is not working properly in this container?

Hi @Jan_Balewski

I had a quick look at your example, and can reproduce the issue. The problem is due to a floating-point difference that is taking 1.0 to 1.0000000000000002 (namely, the difference between a value and and the next largest value representable by incrementing the machine epsilon value to it), which then triggers the np.random.binomial function to complain about using a value above 1.0.

All in all, this is likely stemming from a call through numpy for the probabilities outputs we use for the implementation. The simplest fix for us is to round values on the order of 10^{-16} by capping it at 15 decimal places, which should fix this, and not affect any other results. It could be possible to play with using older versions of numpy, which may have different floating point outputs, but I suspect the above plan should alleviate the issue entirely.

I can confirm playing with this locally allows me to have 30 resets.
We’ll get this in place for the next release.

1 Like

May I ask when do you expect this fix to be deployed? I’d like to plan my research, thanks, Jan

Hi @Jan_Balewski

You can follow the PR here if you want to work directly with the master branch upon merging of this work. Otherwise, the next release (v0.36) will be May 7th.