Optimizing Neural Network Training by Parallelizing Execution on Strawberry Fields Fock Devices

PennyLane Development
1

Greetings to all members of the forum

I would like to share a fascinating challenge that I am currently facing in my research project. Specifically, I am attempting to simulate a neural network using the device strawberryfields.fock, which comprises of 8 qmodes. During this simulation, I execute 100 epochs, with each epoch utilizing 100 images for training purposes. In order to enhance efficiency, I have lowered the dimensionality of each image to 8.

fock_device

It is important to note that the calculations involved in this process are complex and time-consuming, especially when employing 8 qmodes. As such, I have decided to optimize the process by running the calculation in parallel. This will entail dividing the training set into batches, and assigning each batch to a separate processor for execution. Subsequently, the data obtained from each processor will be consolidated and utilized to continue with the process.
I would like to present a snippet of code that I have been working on, which is integral to my research project. The code in question is as follows:

def loss_every_batches(X_train, y_train, parameters, CPUs):
result = [.]
for i, (X, y) in enumerate(create_minibatches(CPUs, X_train, y_train)):
total = X.shape[0]
y_hat_ = np.array([(circuit(x, parameters)) for x in X])
loss = np.mean((y - y_hat_) ** 2)
result.append((loss, total, i))
return result

for i in range(100):
result = loss_every_batches(X_train, y_train, parameters, CPUs)
# additional code here

The purpose of this code is to calculate the loss for each batch of data during the training of a neural network. The loss_every_batches function uses the create_minibatches function to split the data into smaller batches that can be processed in parallel using the CPUs parameter. The results are then stored in a list for later analysis.

I have encountered a challenge when attempting to use the QNodeCollection function to run a circuit with different batches of data. Despite my attempts, I have been unable to find a solution using this method.

I would greatly appreciate any suggestions or ideas that you may have on how to overcome this challenge. Your expertise and insights are invaluable to me, and I am grateful for your time and attention.

Thank you for your consideration.

2

Hey @andresulead! Sorry for the delay here. Will get back to you as soon as possible!

3

Hey @andresulead! QNodeCollection is going to be deprecated soon (v0.31 — we’re currently at 0.30), so I would avoid using it if possible! The feature that is “replacing” this was introduced in v0.25: parameter broadcasting. That said, the strawberryfields.fock doesn’t support parameter broadcasting, and we are phasing out support for the pennylane-sf plugin as well.

I’m not sure I completely understand how you would like QNodeCollection to behave, but if you want to do something like

@qml.qnode(dev)
def qnode1(x):
    ...

@qml.qnode(dev)
def qnode2(x):
    ...

batch1 = np.array([0.1])
batch2 = np.array([0.4])

qnodes = qml.QNodeCollection([qnode1, qnode2])

print(qnodes(batch1, batch2))

this won’t work because QNodeCollection will try to feed both batch1 and batch2 into qnode1 and qnode2 and both of those QNodes only take one argument.

At the end of the day, since there are deprecated features being used here I really recommend switching to pure Strawberryfields and combining it with Tensorflow to handle batching. We have some tutorials on this: Tutorials — Strawberry Fields

I hope this helps!