Hi,
I’m trying to build a quantum convolution model to do image classificaiton. My way of doing this is

First using
qml.qnn.KerasLayer
to buld the trainable quantum filter calledqlayer
. 
Then implementing the convolution operation in my custom Keras
Model
reference to here.
My problem is that although the output dimension seems reasonable to me, when calling model.fit()
, the gradient of training parameters in qlayer
doesn’t exist. Detailed code and error is shown in the following:
First constructing the quantum filter circuit and transforming it to KerasLayer
. The comment out line is used for testing.
import pennylane as qml
from pennylane import numpy as np
from tensorflow import keras
n_train = 50
n_test = 30
filter_size = 2
n_qubits = filter_size**2
dev = qml.device("lightning.qubit", wires=n_qubits, shots=None)
def encoding(pixel):
[qml.RY(pixel[i], wires=[i]) for i in range(n_qubits)]
def circuit_1(params):
qml.BasicEntanglerLayers(weights=params, wires=range(n_qubits))
@qml.qnode(dev)
def q_filter(inputs,params):
encoding(inputs)
circuit_1(params)
return qml.expval(qml.PauliZ(0))
# print(q_filter([1.0,4,54,44], [[3.44,0.45,4.44,5.43]]))
n_layers = 1
weight_shapes = {"params": (n_layers, n_qubits)}
qlayer = qml.qnn.KerasLayer(q_filter, weight_shapes,output_dim=1)
# test
# fig, ax = qml.draw_mpl(q_filter)([1.0,4,54,44], [[3.44,0.45,4.44,5.43]])
# fig.show()
And the following is my QCNN
model
class QCNN(tf.keras.Model):
def __init__(self, num_filters, filter_size, num_params):
super(QCNN,self).__init__()
self.num_filters = num_filters
self.filter_size = filter_size
# using two filter now
self.quantum_filter_1 = qml.qnn.KerasLayer(q_filter, weight_shapes,output_dim=1, name='quantum_filter')
self.quantum_filter_2 = qml.qnn.KerasLayer(q_filter, weight_shapes,output_dim=1, name='quantum_filter')
self.hidden = Dense(128, activation = 'relu')
self.flatten = Flatten()
self.dense = Dense(10, activation='softmax')
def call(self, inputs):
# The output length of one sample after the convolution operation with stride = 1
output_length = inputs.shape[1]  self.filter_size + 1
num_sample = inputs.shape[0]
output_all = []
quantum_filter_list = [self.quantum_filter_1, self.quantum_filter_2]
for a in range(num_sample):
# output shape for one image
output = np.zeros((output_length, output_length, self.num_filters))
for i in range(output_length):
for j in range(output_length):
for f in range(self.num_filters):
# convolution windows, now only applying to one channel image
sub_input = inputs[a,i:i+self.filter_size, i:i+self.filter_size, :]
# flatten into 1D
sub_input = tf.reshape(sub_input, [1])
quantum_filter = quantum_filter_list[f]
output[i][j][f] = quantum_filter(sub_input)
output = tf.convert_to_tensor(output)
# list of tensor
output_all.append(output)
# Stacks a list of rankR tensors into one rank(R+1) tensor.
output_all = tf.stack(output_all)
x = self.flatten(output_all)
x = self.dense(x)
return x
Now test for the model output dimension
# test on some data
#(batch_size, image_height, image_width, channel)
input_shape = (4, 10, 10, 1)
i = tf.random.normal(input_shape)
model = QCNN(2,2,4)
print(f"model_output_shape: {model(i).shape}")
model.summary()
model_output_shape: (4, 10)
Model: "qcnn_21"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantum_filter (KerasLayer) multiple 4
quantum_filter (KerasLayer) multiple 4
dense_40 (Dense) multiple 20864
flatten_20 (Flatten) multiple 0
dense_41 (Dense) multiple 1290
=================================================================
Total params: 22,162
Trainable params: 22,162
Nontrainable params: 0
_________________________________________________________________
The output dimension is correct (4,10), and the Param# catch the training parameters in quantum_filter. The output shape say it’s multiple because I use a quantun_filter
layer multiple times to convolve the whole image, this will have multiple output according to Guide to the Functional API  Keras 2.0.2 Documentation (faroit.com). I’m not sure whether it will be a problem to have multiple output for each layer, but seems the final output dimension looks great so I keep going to train the model:
model = QCNN(2,2,4)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
r = model.fit(x_train_small, y_train, epochs=10, validation_data=(x_test_small, y_test))
and the output says
Epoch 1/10
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
1/2 [==============>...............]  ETA: 36s  loss: 2.3712  accuracy: 0.0312WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
2/2 [==============================]  80s 44s/step  loss: 2.3464  accuracy: 0.1000  val_loss: 2.3679  val_accuracy: 0.1333
Epoch 2/10
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
1/2 [==============>...............]  ETA: 37s  loss: 2.2755  accuracy: 0.1562WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
2/2 [==============================]  81s 44s/step  loss: 2.2671  accuracy: 0.1400  val_loss: 2.3573  val_accuracy: 0.1333
Epoch 3/10
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
1/2 [==============>...............]  ETA: 37s  loss: 2.2450  accuracy: 0.1250WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['qcnn_22/quantum_filter/params:0', 'qcnn_22/quantum_filter/params:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
2/2 [==============================]  ETA: 0s  loss: 2.2473  accuracy: 0.1200
where the dataset is the downscale mnist dataset:
mnist_dataset = keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist_dataset.load_data()
# Reduce dataset size
x_train = x_train[:n_train]
y_train = y_train[:n_train]
x_test = x_test[:n_test]
y_test = y_test[:n_test]
# Normalize pixel values within 0 and 1
x_train = x_train / 255
x_test = x_test / 255
# # Add extra dimension for "color" channels
x_train = np.array(x_train[..., tf.newaxis])
x_test = np.array(x_test[..., tf.newaxis])
print(f"train_images_shape: {x_train.shape}")
# use Bilinear Interpolation for downscaling
x_train_small = tf.image.resize(x_train, (10,10)).numpy()
x_test_small = tf.image.resize(x_test, (10,10)).numpy()
print(f"x_train_reshape: {x_train_small.shape}")
How can I solve this warning saying that the gradient of my trianing parameter in my quantum_fliter
layer doesn’t exist? And reading the loss, it looks like my model indeed isn’t training. I doubt that maybe this occur because of the multiple output, but couldn’t understand why because when print out the output dimension of the model it indeed give me (batch_size, 10), which match to my y_train dimension. I have read Data reuploading impelementation in hybrid NN with keras layer  PennyLane Help  Xanadu Discussion Forum, but I think my way using keras with Qnode is very similar to here Turning quantum nodes into Keras Layers — PennyLane documentation, so it should be no problem.
Thanks for you patient reading, the code I povide is very long, hope I explain it clearly If there is any problem to reproduce myt result please let me know.