Keras Layer - Single QBIT

PennyLane Help
1

I am trying to create a keras layer for training a simple NN that would use a single qubit and covert centigrade to fahrenheit.

Code is as below , I am getting error , 'InvalidArgumentError: Input tensor must be at least 2D: [1] [Op:BiasAdd]'.

The disconnect is that I am not able to understand the use of weights_shape. Can you explain use of weights_shape with examples ?

import pennylane as qml
import tensorflow as tf
import numpy as np

n_qubits = 1
dev = qml.device("default.qubit", wires=n_qubits)

@qml.qnode(dev)
def qnode(inputs, weights):
    qml.templates.AngleEmbedding(inputs, wires=range(n_qubits))
    qml.templates.StronglyEntanglingLayers(weights, wires=range(n_qubits))
    return qml.expval(qml.expval(qml.PauliZ(0)))


weight_shapes = {"weights": (1, n_qubits, 3)}

qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=1)
clayer1 = tf.keras.layers.Dense(1)
clayer2 = tf.keras.layers.Dense(1, activation="linear")
model = tf.keras.models.Sequential([clayer1, qlayer, clayer2])

X = np.array([-40, -10,  0,  8, 15, 22,  38],  dtype=float)
Y = np.array([-40,  14, 32, 46, 59, 72, 100],  dtype=float)

X1 = X.reshape((len(X), 1))
Y1 = Y.reshape((len(Y), 1))

opt = tf.keras.optimizers.SGD(learning_rate=0.5)
model.compile(opt, loss='mse',)


model.fit(X1, Y1, epochs=60, batch_size=1)
2

Hey @Hemant_Gahankari,

The following is a tweak of your code that should get it working:

import pennylane as qml
import tensorflow as tf
import numpy as np

n_qubits = 1
layers = 1
data_dimension = 1
dev = qml.device("default.qubit", wires=n_qubits)


@qml.qnode(dev)
def qnode(inputs, weights):
    qml.templates.AngleEmbedding(inputs, wires=range(n_qubits))
    qml.templates.StronglyEntanglingLayers(weights, wires=range(n_qubits))
    return [qml.expval(qml.PauliZ(i)) for i in range(n_qubits)]


weight_shapes = {"weights": (layers, n_qubits, 3)}

qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits)
clayer1 = tf.keras.layers.Dense(n_qubits)
clayer2 = tf.keras.layers.Dense(data_dimension, activation="linear")
model = tf.keras.models.Sequential([clayer1, qlayer, clayer2])

X = np.array([-40, -10,  0,  8, 15, 22,  38],  dtype=np.float32)
Y = np.array([-40,  14, 32, 46, 59, 72, 100],  dtype=np.float32)

X = X.reshape((len(X), data_dimension))
Y = Y.reshape((len(Y), data_dimension))

opt = tf.keras.optimizers.SGD(learning_rate=0.5)
model.compile(opt, loss='mse',)

model.fit(X, Y, epochs=60, batch_size=1)

Note that, although training proceeds without error, you should look at changing the hyperparameters to improve the fit. For example, I recommend lowering the learning rate, increasing the number of qubits and increasing the depth.

It looks like your use of weight_shapes was fine. This is a dictionary that provides the shapes of the internal weights used in the qnode, so that qlayer can randomly initialize them. For example, if we had

@qml.qnode(dev)
def qnode(inputs, w1, w2, w3):
    qml.templates.AngleEmbedding(inputs, wires=range(n_qubits))
    qml.RX(w1, wires=0)
    qml.templates.BasicEntanglerLayers(w2, wires=range(n_qubits))
    qml.Rot(w3, wires=1)
    ...

We would have a corresponding

weight_shapes = {"w1": 1, "w2": (layers, n_qubits), "w3": 3}
3

Hi Tom ,

Thanks a lot the clarifications. It was really helpful.

In order to make it a bit simpler , I made a few changes to the code. Since deg c and Fahrenheit are linearly related like y = mx + c , I though of having two weights through RX and RZ gates.

if I just keep clayer1 , NN converges very quickly. But if I use qlayer or clayer1+qlayer or clayer1 + qlayer + clayer2 , NN does not converge.

Do you see any problem with my approach ?

import pennylane as qml
import tensorflow as tf
import sklearn.datasets
import numpy as np

n_qubits = 1
layers = 1
data_dimension = 1

dev = qml.device("default.qubit", wires=n_qubits)

@qml.qnode(dev)
def qnode(inputs, m, c):
    qml.templates.AngleEmbedding(inputs, wires=range(n_qubits))
    qml.RX(m, wires=0)
    qml.RZ(c, wires=0)
    return [qml.expval(qml.PauliZ(i)) for i in range(n_qubits)]

    

weight_shapes = {"m": 1, "c":1}

qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits)
clayer1 = tf.keras.layers.Dense(n_qubits)
clayer2 = tf.keras.layers.Dense(data_dimension, activation="linear")

model = tf.keras.models.Sequential([qlayer])

X = np.array([-40, -10,  0,  8, 15, 22,  38],  dtype=float)
Y = np.array([-40,  14, 32, 46, 59, 72, 100],  dtype=float)
X1 = X.reshape((len(X), 1))
Y1 = Y.reshape((len(Y), 1))

model.compile(loss='mean_squared_error',

              optimizer=tf.keras.optimizers.Adam(0.01))
4

Dear @Hemant_Gehankari,

Your question ventures away from coding issues and into the wonderful world of QML research :slight_smile:

I can only guess here (without doing the research myself), but one thing I would be aware of is the data input scaling. It’s a great exercise to actually look what the maths of the qlayer is doing: It is a transformation where the inputs enter as sine/cosine functions. Now, these functions are periodic, and you will have to make sure that the data entering the quantum layer lies roughly in an interval of length 2\pi. You can find some more information in the second question of this discussion thread.

5

Hi ,

What I thought is that AngleEmbedding is meant to embed data into quantum states, so that it can be processed further using quantum circuits. Correct me if I am wrong.

Also I am hoping to get this simple sample working so that acts as a basis for further exploration like more qbits - more features etc. It can also act as the simplest example for many new Pennylane users.

Hope we can make it work. As mentioned if we use just one classic layer clayer1 - just one neuron , it trains in seconds and is pretty accurate.

Request you folks to help.

6

Hi @Hemant_Gahankari. If I may jump in as well to comment: you are correct that the AngleEmbedding template is used to embed classical values into quantum states which can then be used, and optimized, in quantum circuits.

Why it’s not converging to a solution when adding a qlayer is, as @Maria_Schuld mentioned, more of a research question and is difficult to answer without diving a bit deeper into the research itself. An idea would be, as also was touched upon above, to scale the data to lie within a 2\pi interval, since the data is embedded in angles, inside this range.

Let us know if you figure out a way to train this circuit!

7

Hello ,

I ran the the additional example code in,

https://pennylane.readthedocs.io/en/stable/code/api/pennylane.qnn.KerasLayer.html

colab notebook and checked the data created by sklearn.datasets.make_moons() , followed by X = tf.constant(data[0])

This data is fed to the model and qlayer. This data is not in the range of 0 to 2 pi. Looks like ,

qml.templates.AngleEmbedding(inputs, wires=range(n_qubits))

converts it to the angles.

Now for my example I used the following code to rescale the data ,

from sklearn.preprocessing import MinMaxScaler

X = np.array([-40, 0, 8, 15, 22,  38],  dtype=np.float32)
Y = np.array([-40, 32, 46, 59, 72, 100],  dtype=np.float32)

X = X.reshape((len(X), 1))
Y = Y.reshape((len(Y), 1))

scale_x = MinMaxScaler()
scale_y = MinMaxScaler()

x = scale_x.fit_transform(X)
y = scale_y.fit_transform(Y)

This solved the problem of network not converging. With scaled data, loss converges, just even with qlayer alone, but converges faster if I have clayer1 + qlayer. But I am still not sure why the data as is was not good for the network to converge.

I If you folks are interested I can share the fully working code.

8

Thanks @Hemant_Gahankari!

It is quite common even in standard ML to preprocess the data, so I’m not surprised we need to do it here too.

For AngleEmbedding, data does not necessarily need to be in the range [0, 2pi). However, the embedding is periodic so that data outside this range has an equivalent point in [0, 2pi). For example, we can encode 3pi but it will be equivalent to pi. Hence, if we do not normalize our data to be within the [0, 2pi) interval then we run the risk of unique data points (e.g., pi and 3pi) being mapped to the same embedded state. Of course, the clayer1 could learn to normalize, but this should give a motivation of why things are tougher with unnormalized data.

Another thing to consider is that the output of the quantum layer is between -1 and 1. Without a subsequent classical layer, we definitely would not be able to hit temperatures outside of this region. However, if we normalize the data then this is now possible. Again, introducing a classical layer after the quantum layer gives us a bit more freedom, but it still might be tougher with unnormalized data.

Hope this helps!
Tom

9

Thank you so much. Appreciate your time and help.

1 Like