Hi,
I’m trying to build a quantum convolution model to do image classificaiton. My way of doing this is
-
First using
qml.qnn.KerasLayerto buld the trainable quantum filter calledqlayer. -
Then implementing the convolution operation in my custom Keras
Modelreference to here.
My problem is that although the output dimension seems reasonable to me, when calling model.fit(), the gradient of training parameters in qlayer doesn’t exist. Detailed code and error is shown in the following:
First constructing the quantum filter circuit and transforming it to KerasLayer. The comment out line is used for testing.
import pennylane as qml
from pennylane import numpy as np
from tensorflow import keras
n_train = 50
n_test = 30
filter_size = 2
n_qubits = filter_size**2
dev = qml.device("lightning.qubit", wires=n_qubits, shots=None)
def encoding(pixel):
[qml.RY(pixel[i], wires=[i]) for i in range(n_qubits)]
def circuit_1(params):
qml.BasicEntanglerLayers(weights=params, wires=range(n_qubits))
@qml.qnode(dev)
def q_filter(inputs,params):
encoding(inputs)
circuit_1(params)
return qml.expval(qml.PauliZ(0))
# print(q_filter([1.0,4,54,44], [[3.44,0.45,4.44,5.43]]))
n_layers = 1
weight_shapes = {"params": (n_layers, n_qubits)}
qlayer = qml.qnn.KerasLayer(q_filter, weight_shapes,output_dim=1)
# test
# fig, ax = qml.draw_mpl(q_filter)([1.0,4,54,44], [[3.44,0.45,4.44,5.43]])
# fig.show()
And the following is my QCNN model
class QCNN(tf.keras.Model):
def __init__(self, num_filters, filter_size, num_params):
super(QCNN,self).__init__()
self.num_filters = num_filters
self.filter_size = filter_size
# using two filter now
self.quantum_filter_1 = qml.qnn.KerasLayer(q_filter, weight_shapes,output_dim=1, name='quantum_filter')
self.quantum_filter_2 = qml.qnn.KerasLayer(q_filter, weight_shapes,output_dim=1, name='quantum_filter')
self.hidden = Dense(128, activation = 'relu')
self.flatten = Flatten()
self.dense = Dense(10, activation='softmax')
def call(self, inputs):
# The output length of one sample after the convolution operation with stride = 1
output_length = inputs.shape[1] - self.filter_size + 1
num_sample = inputs.shape[0]
output_all = []
quantum_filter_list = [self.quantum_filter_1, self.quantum_filter_2]
for a in range(num_sample):
# output shape for one image
output = np.zeros((output_length, output_length, self.num_filters))
for i in range(output_length):
for j in range(output_length):
for f in range(self.num_filters):
# convolution windows, now only applying to one channel image
sub_input = inputs[a,i:i+self.filter_size, i:i+self.filter_size, :]
# flatten into 1-D
sub_input = tf.reshape(sub_input, [-1])
quantum_filter = quantum_filter_list[f]
output[i][j][f] = quantum_filter(sub_input)
output = tf.convert_to_tensor(output)
# list of tensor
output_all.append(output)
# Stacks a list of rank-R tensors into one rank-(R+1) tensor.
output_all = tf.stack(output_all)
x = self.flatten(output_all)
x = self.dense(x)
return x
Now test for the model output dimension
# test on some data
#(batch_size, image_height, image_width, channel)
input_shape = (4, 10, 10, 1)
i = tf.random.normal(input_shape)
model = QCNN(2,2,4)
print(f"model_output_shape: {model(i).shape}")
model.summary()
model_output_shape: (4, 10)
Model: "qcnn_21"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantum_filter (KerasLayer) multiple 4
quantum_filter (KerasLayer) multiple 4
dense_40 (Dense) multiple 20864
flatten_20 (Flatten) multiple 0
dense_41 (Dense) multiple 1290
=================================================================
Total params: 22,162
Trainable params: 22,162
Non-trainable params: 0
_________________________________________________________________
The output dimension is correct (4,10), and the Param# catch the training parameters in quantum_filter. The output shape say it’s multiple because I use a quantun_filter layer multiple times to convolve the whole image, this will have multiple output according to Guide to the Functional API - Keras 2.0.2 Documentation (faroit.com). I’m not sure whether it will be a problem to have multiple output for each layer, but seems the final output dimension looks great so I keep going to train the model:
model = QCNN(2,2,4)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
r = model.fit(x_train_small, y_train, epochs=10, validation_data=(x_test_small, y_test))
and the output says
Epoch 1/10
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
1/2 [==============>...............] - ETA: 36s - loss: 2.3712 - accuracy: 0.0312WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
2/2 [==============================] - 80s 44s/step - loss: 2.3464 - accuracy: 0.1000 - val_loss: 2.3679 - val_accuracy: 0.1333
Epoch 2/10
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
1/2 [==============>...............] - ETA: 37s - loss: 2.2755 - accuracy: 0.1562WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
2/2 [==============================] - 81s 44s/step - loss: 2.2671 - accuracy: 0.1400 - val_loss: 2.3573 - val_accuracy: 0.1333
Epoch 3/10
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
1/2 [==============>...............] - ETA: 37s - loss: 2.2450 - accuracy: 0.1250WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
2/2 [==============================] - ETA: 0s - loss: 2.2473 - accuracy: 0.1200
where the dataset is the downscale mnist dataset:
mnist_dataset = keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist_dataset.load_data()
# Reduce dataset size
x_train = x_train[:n_train]
y_train = y_train[:n_train]
x_test = x_test[:n_test]
y_test = y_test[:n_test]
# Normalize pixel values within 0 and 1
x_train = x_train / 255
x_test = x_test / 255
# # Add extra dimension for "color" channels
x_train = np.array(x_train[..., tf.newaxis])
x_test = np.array(x_test[..., tf.newaxis])
print(f"train_images_shape: {x_train.shape}")
# use Bilinear Interpolation for downscaling
x_train_small = tf.image.resize(x_train, (10,10)).numpy()
x_test_small = tf.image.resize(x_test, (10,10)).numpy()
print(f"x_train_reshape: {x_train_small.shape}")
How can I solve this warning saying that the gradient of my trianing parameter in my quantum_fliter layer doesn’t exist? And reading the loss, it looks like my model indeed isn’t training. I doubt that maybe this occur because of the multiple output, but couldn’t understand why because when print out the output dimension of the model it indeed give me (batch_size, 10), which match to my y_train dimension. I have read Data re-uploading impelementation in hybrid NN with keras layer - PennyLane Help - Xanadu Discussion Forum, but I think my way using keras with Qnode is very similar to here Turning quantum nodes into Keras Layers — PennyLane documentation, so it should be no problem.
Thanks for you patient reading, the code I povide is very long, hope I explain it clearly 
If there is any problem to reproduce myt result please let me know.
. Usually all this means is that somewhere you’re asking for a TensorFlow operation to be computed with a 
!
