Hey @dancbeaulieu!
Glad that you’re set up ok for sampling from the hardware, but I’m not sure what the issue is with integrating the transfer learning code. I think the only solution here is to share the full script that you are running.
Hello,
Here is the code I am using, hope you can help me figure this out. I have the PT weights file from the GitHub link above.
#!/usr/bin/env python
coding: utf-8
Copyright 2019 Xanadu Quantum Technologies Inc.
Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
“”“Running a hybrid image classifier on IBM or Rigetti quantum processors.”""
import matplotlib.pyplot as plt
PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
from torch.optim import lr_scheduler
import torchvision
from torchvision import datasets, models, transforms
Pennylane
import pennylane as qml
from pennylane import numpy as np
Other tools
import time
import os
import copy
import qiskit
from qiskit import QuantumCircuit, transpile, assemble, IBMQ#, Aer
from qiskit.visualization import *
from qiskit import IBMQ
os.environ[“OMP_NUM_THREADS”] = “1”
os.environ[“CUDA_VISIBLE_DEVICES”] = “1”
Setting of the main parameters of the network model and of the training process.
These should match the topology of the saved pre-trained model (quantum_weights_pt).
n_qubits = 4 # number of qubits
step = 0.0004 # learning rate
batch_size = 4 # number of samples for each training step
num_epochs = 1 # number of training epochs
q_depth = 6 # depth of the quantum circuit (number of variational layers)
gamma_lr_scheduler = 0.1 # learning rate reduction applied every 10 epochs.
n_quantum_layers = 15 # Keep 15 even if not all are used.
q_delta = 0.01 # Initial spread of random quantum weights
rng_seed = 0 # seed for random number generator
start_time = time.time() # start of the computation timer
data_dir = “data/hymenoptera_data” # path of dataset
n_wires = 4
provider = IBMQ.load_account()
#Changed my details for hub, group and project because of security concerns
IBMQ.get_provider(hub=‘hub’, group=‘group’, project=‘project’)
dev2 = qml.device(‘qiskit.ibmq’, wires=n_wires, backend=‘ibmq_bogota’, provider=provider, shots=100)
Choose between the two quantum backends: ‘ibm’ or ‘rigetti’.
========= QPU ==========
backend = “ibm”
backend = ‘rigetti’
========================
Set the chosen backend as a PennyLane device.
if backend == “ibm”:
#token = “” # Insert your personal IBM token. Remove the token when sharing your code!
dev = qml.device(‘qiskit.ibmq’, wires=n_wires, backend=‘ibmq_qasm_simulator’)
#if backend == “rigetti”:
dev = qml.device(“forest.qpu”, device=“Aspen-4-4Q-A”, shots=1024)
print("Device capabilities: ", dev.capabilities()[“backend”])
Configure PyTorch to use CUDA, only if available. Otherwise simply use the CPU.
print(“Initializing backend device…”)
device = torch.device(“cuda:0” if torch.cuda.is_available() else “cpu”)
Dataset loading
data_transforms = {
“train”: transforms.Compose(
[
# transforms.RandomResizedCrop(224), # uncomment for data augmentation
# transforms.RandomHorizontalFlip(), # uncomment for data augmentation
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
# Normalize input channels using mean values and standard deviations of ImageNet.
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
]
),
“val”: transforms.Compose(
[
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
]
),
}
get_ipython().system(‘pwd’)
image_datasets = {
x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms) for x in [“train”, “val”]
}
dataset_sizes = {x: len(image_datasets) for x in [“train”, “val”]}
class_names = image_datasets[“train”].classes
Initialize dataloader
torch.manual_seed(rng_seed)
dataloaders = {
x: torch.utils.data.DataLoader(image_datasets, batch_size=batch_size, shuffle=True)
for x in [“train”, “val”]
}
Function to plot images
def imshow(inp, title=None):
“”“Display image from tensor.
Args:
inp (tensor): input image
title (string): title of the image
“””
inp = inp.numpy().transpose((1, 2, 0))
# We apply the inverse of the initial normalization operation.
mean = np.array([0.485, 0.456, 0.406])
std = np.array([0.229, 0.224, 0.225])
inp = std * inp + mean
inp = np.clip(inp, 0, 1)
plt.imshow(inp)
if title is not None:
plt.title(title)
Hybrid transfer learning model (classical-to-quantum).
We first define some quantum layers that will comprise the quantum circuit.
def H_layer(nqubits):
“”“Layer of single-qubit Hadamard gates.
Args:
nqubits (int): number of qubits
“””
for idx in range(nqubits):
qml.Hadamard(wires=idx)
def RY_layer(w):
“”“Layer of parametrized qubit rotations around the y axis.
Args:
w (tensor): list of rotation angles; one for each qubit
“””
for idx, element in enumerate(w):
qml.RY(element, wires=idx)
def entangling_layer(nqubits):
“”“Layer of CNOTs followed by another shifted layer of CNOT.
Args:
nqubits (int): number of qubits
“””
# In other words it should apply something like :
# CNOT CNOT CNOT CNOT… CNOT
# CNOT CNOT CNOT… CNOT
for i in range(0, nqubits - 1, 2): # loop over even indices: i=0,2,…N-2
qml.CNOT(wires=[i, i + 1])
for i in range(1, nqubits - 1, 2): # loop over odd indices: i=1,3,…N-3
qml.CNOT(wires=[i, i + 1])
Let us define the quantum circuit by using the PennyLane qnode decorator .
The structure is that of a typical variational quantum circuit:
1. All qubits are first initialized in a balanced superposition of up and down states,
then they are rotated according to the input parameters (local embedding);
2. Successively a sequence of trainable rotation layers and constant entangling layers is applied.
This block is responsible for the main computation necessary to solve the classification problem.
3. Eventually, for each qubit, the local expectation value of the Z operator is measured.
This produces a classical output vector, suitable for additional post-processing.
@qml.qnode(dev2, interface=“torch”)
def q_net(q_in, q_weights_flat):
“”“Quantum cricuit
Args:
q_in (tensor): input features
q_weights_flat (tensor): variational parameters
Returns:
tuple: expectation values of PauliZ for each qubit
“””
# Reshape weights
q_weights = q_weights_flat.reshape(n_quantum_layers, n_qubits)
# Start from state |+> , unbiased w.r.t. |0> and |1>
H_layer(n_qubits)
# Embed features in the quantum node
RY_layer(q_in)
# Sequence of trainable variational layers
for k in range(q_depth):
entangling_layer(n_qubits)
RY_layer(q_weights[k + 1])
# Expectation values in the Z basis
return [qml.expval(qml.PauliZ(j)) for j in range(n_qubits)]
We can now define a custom torch.nn.Module representing a dressed quantum circuit.
This is is a concatenation of:
1. A classical pre-processing layer (nn.Linear)
2. A classical activation function (torch.tanh)
3. A constant np.pi/2.0 scaling factor.
2. The previously defined quantum circuit (q_net)
2. A classical post-processing layer (nn.Linear)
The input of the module is a batch of vectors with 512 real parameters (features)
and the output is a batch of vectors with two real outputs (associated with the two
classes of images: ants and bees).
class Quantumnet(nn.Module):
def init(self):
super().init()
self.pre_net = nn.Linear(512, n_qubits)
self.q_params = nn.Parameter(q_delta * torch.randn(n_quantum_layers * n_qubits))
self.post_net = nn.Linear(n_qubits, 2)
def forward(self, input_features):
"""Full classical-quantum network.
Args:
self
input_features (tensor): input image
Returns:
tuple: output logits of the hybrid network
"""
pre_out = self.pre_net(input_features)
q_in = torch.tanh(pre_out) * np.pi / 2.0
# Apply the quantum circuit to each element of the batch, and append to q_out
q_out = torch.Tensor(0, n_qubits)
q_out = q_out.to(device)
for elem in q_in:
q_out_elem = q_net(elem, self.q_params).float().unsqueeze(0)
q_out = torch.cat((q_out, q_out_elem))
return self.post_net(q_out)
We are finally ready to build our full hybrid classical-quantum network. We follow the transfer learning approach.
First load the classical pre-trained network ResNet18 from the torchvision.models zoo.
The model is downloaded from Internet and it may take a long time (only the first time).
model_hybrid = torchvision.models.resnet18(pretrained=True)
Freeze all the weights since they should not be trained.
for param in model_hybrid.parameters():
param.requires_grad = False
Replace the last fully connected layer with our trainable dressed quantum circuit (Quantumnet).
model_hybrid.fc = Quantumnet()
Use CUDA or CPU according to the “device” object.
model_hybrid = model_hybrid.to(device)
Load model from file
model_hybrid.fc.load_state_dict(torch.load(“quantum_weights.pt”, map_location=“cpu”))
We apply the model to the test dataset to compute the associated loss and accuracy.
criterion = nn.CrossEntropyLoss()
running_loss = 0.0
running_corrects = 0
n_batches = dataset_sizes[“val”] // batch_size
it = 0
print(
“Results of the model testing on a real quantum processor.”,
file=open(“results_” + backend + “.txt”, “w”),
)
print("QPU backend: " + backend, file=open(“results_” + backend + “.txt”, “a”))
for inputs, labels in dataloaders[“val”]:
model_hybrid.eval()
inputs = inputs.to(device)
labels = labels.to(device)
batch_size_ = len(inputs)
with torch.set_grad_enabled(False):
outputs = model_hybrid(inputs)
, preds = torch.max(outputs, 1)
loss = criterion(outputs, labels)
running_loss += loss.item() * batch_size
batch_corrects = torch.sum(preds == labels.data).item()
running_corrects += batch_corrects
print(“Iter: {}/{}”.format(it + 1, n_batches + 1), end="\r", flush=True)
# log to file
print(
“Iter: {}/{}”.format(it + 1, n_batches + 1),
end="\r",
flush=True,
file=open(“results_” + backend + “.txt”, “a”),
)
it += 1
epoch_loss = running_loss / dataset_sizes[“val”]
epoch_acc = running_corrects / dataset_sizes[“val”]
print("\nTest Loss: {:.4f} Test Acc: {:.4f} ".format(epoch_loss, epoch_acc))
Log to file
print(
"\nTest Loss: {:.4f} Test Acc: {:.4f} ".format(epoch_loss, epoch_acc),
file=open(“results_” + backend + “.txt”, “a”),
)
#!pwd
#torch.save(model_hybrid.state_dict(), ‘modelcnn.pt’)
Compute and visualize the predictions for a batch of test data.
The figure is saved as a .png file in the working directory.
images_so_far = 0
num_images = batch_size
fig = plt.figure(“Predictions”)
model_hybrid.eval()
with torch.no_grad():
for i, (inputs, labels) in enumerate(dataloaders[“val”]):
inputs = inputs.to(device)
labels = labels.to(device)
outputs = model_hybrid(inputs)
, preds = torch.max(outputs, 1)
for j in range(inputs.size()[0]):
images_so_far += 1
ax = plt.subplot(num_images // 2, 2, images_so_far)
ax.axis(“off”)
ax.set_title("[{}]".format(class_names[preds[j]]))
imshow(inputs.cpu().data[j])
if images_so_far == num_images:
fig.savefig("predictions" + backend + “.png”)
break
if images_so_far == num_images:
break
Dear @dancbeaulieu,
Thank you for posting the exact code snippet!
Just to make sure: when creating the provider object that is then passed as provider=provider to qml.device, are IBMQ specific details being used instead of the placeholder strings? So instead of explicitly passing hub='HUB', group='GROUP', project='PROJECT', is 'HUB', 'GROUP', etc. substituted with a valid IBMQ hub, group, etc.?
Curious, because I managed to send a circuit to IBMQ with the snippet above to the Bogota IBMQ machine by having dev2 = qml.device('qiskit.ibmq', wires=n_wires, backend='ibmq_bogota', shots=100). At the time, the job was in the queue for 7 minutes and ran for ~10 seconds. It also appeared well in IBMQ Experience.
Hi @Tom_Bromley
Why do I get ModuleNotFoundError: No module named ‘pennylane’ when running the tutorial on lab.quantum-computing?
Hi @_risto,
That will likely be due to some installation issues. Could you check the output of pip freeze? Does PennyLane appear in the output?
Pennylane and Pennylane-qiskit both appear:
PennyLane==0.16.0
PennyLane-qiskit==0.16.0
Yes it does (among other things), when I wrote it in PowerShell, but I am opening my jupyter notebook ipynb in qiskit quantum lab.
@_risto, you could try executing the following command at the top of your notebook in IBM quantum lab to install PennyLane:
!pip install pennylane
!pip install pennylane_qiskit
Hi @Tom_Bromley
Thank you. It worked. When importing the data (data dir = …/_data/hymenoptera_data), do I need to create a Docker file? It says there is no ```
‘/home/jovyan’ file, however one could create one from Docker? I know it is a different platform, so just as inquiry perhaps, any information would be helpful.
Hi @_risto,
It seems that something is going wrong with the path where the input data file was created for the demo. Is this error raised when running the demonstration in a Jupyter notebook in Qiskit Quantum Lab? It is strange that ‘/home/jovyan’ appears in the error.
Perhaps a solution would be to create the file by first closer inspecting the directory structure that is used with the Jupyter notebook:
import os
os.getcwd()
This will help to see what the current working directory is. Then, the correct directory structure and file could be created using commands from the Jupyter notebook.
Not sure if creating a Docker file and using a Docker container would be necessary.
Hi @antalszava
Thanks for the answer. I get:
‘/home/jovyan’
Yes, the jupyter notebook is running via ibmq lab.quantum. When running just jupyter, I have no problems with importing data.
When I use os.path.abspath I get: FileNotFoundError: [Errno 2] No such file or directory: ‘/home/jovyan/C:\Users\risto\Desktop\quantum_computing\Quantum transfer learning_bees\_data\hymenoptera_data/train’
I was able to solve my problem by changing the backend from IBMQ_BOGOTA to IBM_LAGOS. No idea why that makes any difference. It ran both ways, but when I ran on IBMQ_BOGOTA it ran instantly and didn’t access the IBMQ system or leave and logs, so believe it was executing locally and not leaving a log or any information on the IBMQ backend. When I ran on IBM_LAGOS it worked, put me in the queue for the QPU, and was present in the IBMQ backend logs.
2 Likes
What do you mean by that? To create an account on IBM Nigeria and run it over there?
IBM_LAGOS is the name of a quantum machine, unfortunately I can no longer get my jobs to run on the IBMQ system. Was previously trying to use IBMQ_BOGOTA (a different quantum machine name). Was able to get them running a few times, but no longer.
@_risto the error message seems to be somewhat related to a mix between Linux and Windows paths 
Following up on @dancbeaulieu’s suggestion, changing the backend can be done when creating the device in PennyLane:
dev = qml.device('qiskit.ibmq', wires=2, backend='ibmq_lagos')
This assumes that the 'ibmq_lagos' backend is available.
Hi @antalszava
I have tried that, but still get the error: FileNotFoundError: [Errno 2] No such file or directory: '/home/jovyan/C:…
Hi @_risto,
Just to confirm: do you have the required folder structure in the IBMQ quantum lab system? Tried the demo and after creating the folder structure, the initial FileNotFoundError: [Errno 2] No such file or directory: '../_data/hymenoptera_data/train' error was resolved.
It’s worth noting, that I was only able to create folders & files from within the same folder where the notebook was. Therefore, I changed the path declaration to data_dir = "_data/hymenoptera_data".
Could you maybe execute !ls -la in one of the cells and submit the output?
Hi @antalszava
I get:
total 473
drwxrwxr-x 1 jovyan 1000 0 Aug 2 20:47 .
drwxr-xr-x 1 root root 4096 Jan 27 2021 …
drwxr-xr-x 1 jovyan 1000 0 Aug 2 20:47 .cache
drwxr-xr-x 1 jovyan 1000 0 Aug 2 20:53 .config
drwxr-xr-x 1 jovyan 1000 0 Aug 2 20:53 .ipynb_checkpoints
drwxr-xr-x 1 jovyan 1000 0 Aug 2 20:47 .ipython
lrwxrwxrwx 1 jovyan 1000 13 Aug 11 18:15 .jupyter -> /tmp/.jupyter
drwxr-xr-x 1 jovyan 1000 0 Aug 2 20:47 .local
drwxr-xr-x 1 jovyan 1000 0 Aug 11 18:15 .qiskit
lrwxrwxrwx 1 jovyan 1000 20 Aug 11 18:15 qiskit-textbook -> /tmp/qiskit-textbook
lrwxrwxrwx 1 jovyan 1000 25 Aug 11 18:15 qiskit-tutorials -> /tmp/qiskit-iqx-tutorials
-rw-r–r-- 1 jovyan 1000 186366 Aug 11 18:19 resnet152_bees_qbits4_batch4_IBMQ.ipynb
-rw-r–r-- 1 jovyan 1000 11267 Aug 3 17:29 Untitled1.ipynb
drwxr-xr-x 1 jovyan 1000 0 Aug 4 16:54 ‘Untitled Folder’
-rw-r–r-- 1 jovyan 1000 275498 Aug 4 17:07 ‘Wide ResNet-101-2_4qbits.ipynb’
What do you mean by: required folder structure in the IBMQ quantum lab system? It is in the same folder as other ipynb resnet152 files, thus the same structure in regards to hymenoptera data.
│── tutorials
│ ├── _data
│ ├── ├── hymenoptera_data
│ ├── quantum_transfer_learning
│ ├── ├── tutorial_quantum_transfer_learning.ipynb
│ ├── ├── tutorial_quantum_transfer_learning_IBMQ.ipynb
required folder structure in the IBMQ quantum lab system
It would be the folder structure of the data required by the demo:
│ _data
│ ├── hymenoptera_data
│ ├── ├── train
│ ├── ├── ├── ants
│ ├── ├── ├── bees
│ ├── ├── val
│ ├── ├── ├── ants
│ ├── ├── ├── bees
Based on the path data_dir = "../_data/hymenoptera_data", the following folder structure should indeed work well:
│ _data
│ ├── hymenoptera_data
│ ├── ├── train
│ ├── ├── ├── ants
│ ├── ├── ├── bees
│ ├── ├── val
│ ├── ├── ├── ants
│ ├── ├── ├── bees
│ quantum_transfer_learning
│ ├── tutorial_quantum_transfer_learning.ipynb
However, based on your output, I’m not sure I see tutorial_quantum_transfer_learning.ipynb. Is the tutorial in resnet152_bees_qbits4_batch4_IBMQ.ipynb?
Also, checking the folder one level up, I get the following output:
!ls ..
jovyan
Therefore, I’ve changed the line data_dir = “…/_data/hymenoptera_data” to data_dir = "_data/hymenoptera_data". This should work with folder structure for example as follows:
│ _data
│ ├── hymenoptera_data
│ ├── ├── train
│ ├── ├── ├── ants
│ ├── ├── ├── bees
│ ├── ├── val
│ ├── ├── ├── ants
│ ├── ├── ├── bees
│ tutorial_quantum_transfer_learning.ipynb
Assuming that tutorial_quantum_transfer_learning.ipynb is the notebook with the tutorial that was created on IBMQ quantum lab.