Making quanvolutional neural net weights trainable

Hi i am trying to make the weights of this network trainable in this tutorial https://pennylane.ai/qml/demos/tutorial_quanvolution.html
using this layer https://pennylane.readthedocs.io/en/stable/code/api/pennylane.qnn.KerasLayer.html?highlight=keraslayer#pennylane.qnn.KerasLayer
However I am unsure as to what arguments i should pass for weight shapes and output dimensions. Can I please get some help ?

Thank you

Hey @vijpandaturtle!

Having a look at the tutorial, is this the part you are trying to convert into a KerasLayer:

dev = qml.device("default.qubit", wires=4)
# Random circuit parameters
rand_params = np.random.uniform(high=2 * np.pi, size=(n_layers, 4))

@qml.qnode(dev)
def circuit(phi=None):
    # Encoding of 4 classical input values
    for j in range(4):
        qml.RY(np.pi * phi[j], wires=j)

    # Random quantum circuit
    RandomLayers(rand_params, wires=list(range(4)))

    # Measurement producing 4 classical output values
    return [qml.expval(qml.PauliZ(j)) for j in range(4)]

(feel free to share any code you have and we can take a closer look)

For the above, you first need to update the signature of circuit() so that the phi argument is changed to inputs, since KerasLayer requires an argument of this name to pass input data to. You should also remove the =None part to make the gradient accessible with respect to the input data. You can then add rand_params as an argument to circuit.

Then, we need to tell KerasLayer the shape of rand_params so they can be initialized within KerasLayer. To do this, you can define:

weight_shapes = {"rand_params": (n_layers, 4)}

It is then simply a case of running

qml.qnn.KerasLayer(circuit, weight_shapes, output_dim=4)

to convert to a Keras-compatible layer.

Hope this helps!
Tom

1 Like

Thank you for the reply @Tom_Bromley
The problem with using a keras layer just for the circuit is that i cannot include the actual convolution function (pasted the code below) in the network

def quanv(image):
    """Convolves the input image with many applications of the same quantum circuit."""
    out = np.zeros((14, 14, 4))

    # Loop over the coordinates of the top-left pixel of 2X2 squares
    for j in range(0, 28, 2):
        for k in range(0, 28, 2):
            # Process a squared 2x2 region of the image with a quantum circuit
            q_results = circuit(
                phi=[image[j, k, 0], image[j, k + 1, 0], image[j + 1, k, 0], image[j + 1, k + 1, 0]]
            )
            # Assign expectation values to different channels of the output pixel (j/2, k/2)
            for c in range(4):
                out[j // 2, k // 2, c] = q_results[c]
    return out

i even tried including the circuit logic inside the convolution function to see if it works

def quanv(inputs, conv_params):
    out = np.zeros((14, 14, 4))

    # Loop over the coordinates of the top-left pixel of 2X2 squares
        for j in range(0, 28, 2):
            for k in range(0, 28, 2):
            # Process a squared 2x2 region of the image with a quantum circuit
                win=[inputs[j, k, 0], inputs[j, k + 1, 0], inputs[j + 1, k, 0], inputs[j + 1, k + 1, 0]]
            
                for j in range(4):
                    qml.RY(np.pi * win[j], wires=j)
                RandomLayers(conv_params, wires=list(range(4)))
                q_results = [qml.expval(qml.PauliZ(j)) for j in range(4)]
            # Assign expectation values to different channels of the output pixel (j/2, k/2)
                for c in range(4):
                    out[j // 2, k // 2, c] = q_results[c]
    return out

there may be some issues with the array shapes but this is a rough sketch of what i’m trying to do. I’m probably going wrong in many place please do let me know :confused:

Hey @vijpandaturtle,

Ah ok I see, good question! Since the current KerasLayer doesn’t support applying this style of convolution, you could create your own Keras layer that does!

This layer could inherit from qml.qnn.KerasLayer and edit the call() method to apply the convolution. I had a quick go at doing this to give you an idea:

import tensorflow as tf

class ConvQLayer(qml.qnn.KerasLayer):
    
    def call(self, inputs):
        
        batches = inputs.shape[0]
        out = tf.Variable(tf.zeros((batches, 14, 14, 4)))
        
        # Loop over the coordinates of the top-left pixel of 2X2 squares
        for j in range(0, 28, 2):
            for k in range(0, 28, 2):
                # Process a squared 2x2 region of the image with a quantum circuit
                qnode_inputs = tf.stack([inputs[:, j, k, 0], inputs[:, j, k + 1, 0], inputs[:, j + 1, k, 0], inputs[:, j + 1, k + 1, 0]], axis=1)
                q_results = super().call(qnode_inputs)

                out[:, j // 2, k // 2].assign(q_results)

        return out

qlayer_conv = ConvQLayer(circuit, weight_shapes, output_dim=4)
batches = 2
inputs = np.random.random((batches, 28, 28, 3))
out = qlayer_conv(inputs)

This probably needs a lot more work to double check that it’s functioning as expected, but this would be the general idea!

Thanks,
Tom

1 Like

Thank you so much for that !! :grinning:

1 Like

@Tom_Bromley I am using the same code snippet and it gives me a warnings “gradients do not exist”. I have set the argument name to ‘inputs’ and removed the ‘None’ assigned to it. Is there anything else that needs to be done to make gradients accessible ?
Thanks

Hi @vijpandaturtle,

Could you help out with posting the code snippet that results in the warnings? From the PennyLane side, the previous suggestions by Tom should suffice for making the gradients be accessible. These warnings seem to be specific to TensorFlow and in certain cases still without specific resolutions. Having said that, perhaps we could uncover something specific to the example. :slightly_smiling_face:

@antalszava the gradient error seems to go away once i updated the tensorflow version. however, i am still getting errors regarding matrix compatibility

InvalidArgumentError: Matrix size-incompatible: In[0]: [4,784], In[1]: [4,10] [Op:MatMul]

How does the keras layer handle batch processing ? I think that might the reason for this issue.

batches = inputs.shape[0]
out = tf.Variable(tf.zeros((batches, 14, 14, 4)))
        

Hi @vijpandaturtle,

That’s great news!

A single KerasLayer object is meant for a single batch. An example of how integrating KerasLayer with a tf.keras.models.Sequential model and specifying the number of batches for the model as a whole can be found in the Additional example section.

For the InvalidArgumentError it indeed seems like there is something going on with the shapes, which could be related to batching (the second component of [4,784] for example seems to be the product of 14, 14, 4).

Unfortunately, it is challenging to see what exactly could be going wrong without the complete code snippet that yields the error. Could you please post it?

Thank you for the reply @antalszava here are the code snippets

class ConvQLayer(qml.qnn.KerasLayer):
    
    def call(self, inputs):
        
        batches = inputs.shape[0]
        out = tf.Variable(tf.zeros((batches, 14, 14, 4)))
        
        # Loop over the coordinates of the top-left pixel of 2X2 squares
        for j in range(0, 28, 2):
            for k in range(0, 28, 2):
                # Process a squared 2x2 region of the image with a quantum circuit
                qnode_inputs = tf.stack([inputs[:, j, k, 0], inputs[:, j, k + 1, 0], inputs[:, j + 1, k, 0], inputs[:, j + 1, k + 1, 0]], axis=1)
                q_results = super().call(qnode_inputs)
                out[:, j // 2, k // 2, :].assign(q_results)
        return out

weight_shapes = {'conv_params': (n_layers,4)}

def MyModel():
    """Initializes and returns a custom Keras model
    which is ready to be trained."""
    model = keras.models.Sequential([
        ConvQLayer(circuit, weight_shapes, output_dim=4),
        keras.layers.Flatten(),
        keras.layers.Dense(10, activation="softmax")
    ])

    model.compile(
        optimizer='adam',
        loss="sparse_categorical_crossentropy",
        metrics=["accuracy"],
    )
    return model

q_model = MyModel()

q_history = q_model.fit(
    train_images,
    train_labels,
    validation_data=(test_images, test_labels),
    batch_size=4,
    epochs=n_epochs,
    verbose=2,
)


Hey @vijpandaturtle,

I had another look just now and also had a problem with accessing the gradient. Unfortunately it wasn’t clear to me what the problem was :thinking: This will likely take some time troubleshooting before we work out the problem. Remember that KerasLayer does not support convolution, so this is definitely a more “research” level question. If you’re interested in looking deeper into this and providing some feedback that would be great, otherwise I’d recommend following the established style in the tutorial.

@Tom_Bromley thank you for the reply. yes i will look into it a little deeper this time. however, do let me know if you find something :slight_smile:

1 Like

Hi @antalszava,

I have faced the same issue. Have you found a fix to this? It will be very helpful. Thanks

Hey @Emmanuel_OM, welcome to the forum!

Could you elaborate on the issue you’re facing? It can also be useful to share your complete code so that we can troubleshoot the issue more directly.

Thanks!

Thank you for the reply @Tom_Bromley. I am using the code snippets that @vijpandaturtle report above. The only difference is the optimizer and loss function (used for a binary classification problem)

    def MyModel():
    """Initializes and returns a custom Keras model
    which is ready to be trained."""
    model = keras.models.Sequential([
        ConvQLayer(circuit, weight_shapes, output_dim=4),
        keras.layers.Flatten(),
        keras.layers.Dense(1, activation="sigmoid")
    ])

model.compile(
        optimizer='adam',
        loss="binary_crossentropy",
        metrics=["binary_accuracy"],
    )
    return model

Thanks @Emmanuel_OM. From my side I didn’t have any breakthrough with getting the gradient from ConvQLayer to be accessible. Not sure if @vijpandaturtle had any luck?

Unfortunately, this approach to a quantum convolutional layer does not work out of the box with KerasLayer and would require some careful work to get the gradients properly accessible. For now, I’d recommend using the core PennyLane approach (i.e., without KerasLayer) that is used in the tutorial.

Thanks!