Reshape Error in Timeseries when removing LSTM Layer

Hi Pennylane devs/support team,
I’m working in a team on a project to evaluate how efficient VQCs are for predicting future points in time series, but we struggle with the interface between pennylane and TensorFlow because it’s our first time working with it. I saw other questions about batches, but I’m unsure if they are totally related, so I wanted to create my own topic.

The current goal is to implement a tf model that takes a qml circuit to train on a dataset and make predictions on it. We have a hybrid model that works fine with the integrated quantum layers, but when I start on implementing a model that takes a circuit as input, I realize that the input/output shapes are not trivial. The only config that worked was for one batch or when using a LSTM layer beforehand. The optimal goal would be a tf model that looks like this:

    def create_pvc_model(self, circuit, config):
        inputs = Input(shape=(config['time_steps'], 1))
        qlayer = qml.qnn.KerasLayer(, circuit.get_weights(), output_dim=1)(dense1)
        output = Dense(config['future_steps'], activation='linear')(qlayer)

        model = Model(inputs=inputs, outputs=output)
        return model

Because we might use circuits that have a different amount of wires than the Input/Output size, I added layers to adapt to the right size. There might be a more elegant way to archive this, but my first intuition looked like this:

    def create_pvc_model(self, circuit, config):
        inputs = Input(shape=(config['time_steps'], 1))
        dense1 = Dense(circuit.get_wires(), activation='relu')(inputs)
        qlayer = qml.qnn.KerasLayer(, circuit.get_weights(), output_dim=1)(dense1)
        qlayer = tf.reshape(qlayer, (-1, 1))
        output = Dense(config['future_steps'], activation='linear')(qlayer)

        model = Model(inputs=inputs, outputs=output)
        return model

To give a better idea, the very basic example circuit looked like this:

class new_RYXZ_Circuit(BaseCircuit):
    def __init__(self):
        self.weight_shapes = {"weights_0": (1,), "weights_1": (1,), "weights_2": (1,)}
        self.n_wires = 1

    def run(self):
        training_device = qml.device("default.qubit", wires=self.n_wires)

        @qml.qnode(training_device, interface='tf', diff_method='backprop')
        def _circuit(inputs, weights_0, weights_1, weights_2):
            qml.RY(inputs[0], wires=0)
            qml.RX(weights_0, wires=0)
            qml.RY(weights_1, wires=0)
            qml.RZ(weights_2, wires=0)
            return qml.expval(qml.PauliZ(wires=0))

        return _circuit

    def get_weights(self):
        return self.weight_shapes

    def get_wires(self):
        return self.n_wires

We tried a number of different changes but feel overwhelmed because we can’t grape the underlying structure when and when not the shapes changes. The general error always looks like this:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Exception encountered when calling layer 'keras_layer' (type KerasLayer).

{{function_node __wrapped__Reshape_device_/job:localhost/replica:0/task:0/device:CPU:0}} Input to reshape is a tensor with 1 values, but the requested shape has 3200 [Op:Reshape] name: 

Call arguments received by layer 'keras_layer' (type KerasLayer):
  • inputs=tf.Tensor(shape=(64, 50, 1), dtype=float32)

I’m still unsure if the problem is trivial or if there are fundamental limits to what we can do with our general idea. It would help a lot if you could point us in the right direction or suggest an alternative way of implementing the concept that perhaps circumvents the issue. I’m saying thanks in advance, any help might bring us far enough to get the gears running again.

I will link all relevant scripts if you want to take a closer look, some modules might need to be imported but should be trivial: (4.2 KB) (1.3 KB) (5.3 KB) (2.4 KB) (3.3 KB) (5.4 KB) (2.0 KB)

Name: PennyLane
Version: 0.36.0
Summary: PennyLane is a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
Home-page: GitHub - PennyLaneAI/pennylane: PennyLane is a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
License: Apache License 2.0
Location: C:\Users\TomBi\PycharmProjects\ProbDist_QCP2\VQC_project\desktop_venv\Lib\site-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, pennylane-lightning, requests, rustworkx, scipy, semantic-version, toml, typing-extensions
Required-by: PennyLane_Lightning

Platform info: Windows-10-10.0.22621-SP0
Python version: 3.11.8
Numpy version: 1.26.4
Scipy version: 1.13.1
Installed devices:

  • default.clifford (PennyLane-0.36.0)
  • default.gaussian (PennyLane-0.36.0)
  • default.mixed (PennyLane-0.36.0)
  • default.qubit (PennyLane-0.36.0)
  • default.qubit.autograd (PennyLane-0.36.0)
  • default.qubit.jax (PennyLane-0.36.0)
  • default.qubit.legacy (PennyLane-0.36.0)
  • (PennyLane-0.36.0)
  • default.qubit.torch (PennyLane-0.36.0)
  • default.qutrit (PennyLane-0.36.0)
  • default.qutrit.mixed (PennyLane-0.36.0)
  • null.qubit (PennyLane-0.36.0)
  • lightning.qubit (PennyLane_Lightning-0.36.0)

Hi @Cap_Cap ,

Your problem is indeed a bit complex so it’s taking me some time to find a good suggestion for you.
Here are some tips that can help in the meantime:

  1. Using Torch instead of TensorFlow can ensure better compatibility into the future so if possible I’d recommend using that instead.
  2. This demo shows an example of using time series with Torch, Covalent and PennyLane
  3. Making a minimal version of your problem can help you debug things like batching and sizes
  4. Batching/broadcasting doesn’t always work. At the end of the QNode docs you can see a section on Parameter broadcasting which can help you. Also here on our quantum circuits section of the docs.

I hope this can help you keep debugging the issue. If you have any additional insights or a smaller version of the code feel free to post it here. If I find something I’ll also let you know.

I hope this helps!

Thanks for the answer, your links helped me a lot. I somehow overlooked the @qml.batch_input which I think is exactly what I need, but there are some complications on hand. These are a lot more specific, so I hope it makes it easier for you. I also reduced the relevant parts to a single executable file that presents the problem. (6.9 KB)

First, I’m unsure if I set the (argnum) right for batch input. In the current example, the batch size is 64 and the size of an input sample is 32 so if I understand the documentation right:

It should be @qml.batch_input(argnum=(64, 32)) like I wrote in my script

def get_complex_circuit_and_weights():
    training_device = qml.device("default.qubit", wires=5)

    @qml.qnode(training_device, interface='tf')
    @qml.batch_input(argnum=(64, 32)) # In this line
    def example_complex_circuit(inputs, weights_RY, weights_RX, weights_RZ):
        qml.AmplitudeEmbedding(features=inputs, wires=range(5), normalize=True)
        qml.broadcast(qml.RY, wires=range(5), pattern="single", parameters=weights_RY)
        qml.broadcast(qml.RX, wires=range(5), pattern="single", parameters=weights_RX)
        qml.broadcast(qml.RZ, wires=range(5), pattern="single", parameters=weights_RZ)
        qml.broadcast(qml.CNOT, wires=range(5), pattern="all_to_all")
        return [qml.expval(qml.PauliZ(i)) for i in range(5)]

    weight_shapes = {"weights_RY": 5, "weights_RX": 5, "weights_RZ": 5}
    return example_complex_circuit, weight_shapes

But perhaps I just understand it wrong.
For now, changing this number doesn’t have an impact.
This second problem, perhaps triggers the first or vice versa that I’m not sure.

The second problem:

C:\Users\TomBi\OneDrive\.SoSe24\QC_Seminar\Projekt\scrips\ProbDist_QCP3\.lappy_venv\Scripts\python.exe C:\Python_project\ProbDist_QCP_true\VQC_project\Tom\main\ 
2024-06-20 17:03:17.057134: I tensorflow/core/util/] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
WARNING:tensorflow:From C:\Users\TomBi\OneDrive\.SoSe24\QC_Seminar\Projekt\scrips\ProbDist_QCP3\.lappy_venv\Lib\site-packages\keras\src\ The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

WARNING:tensorflow:From C:\Users\TomBi\OneDrive\.SoSe24\QC_Seminar\Projekt\scrips\ProbDist_QCP3\.lappy_venv\Lib\site-packages\keras\src\ The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.

2024-06-20 17:03:22.929681: I tensorflow/core/platform/] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
  File "C:\Python_project\ProbDist_QCP_true\VQC_project\Tom\main\", line 170, in <module>
    fit_and_evaluate_model(dataset, tf_model_simple_circuit)
  File "C:\Python_project\ProbDist_QCP_true\VQC_project\Tom\main\", line 156, in fit_and_evaluate_model
    model = TF_Model(model_with_circuit)
  File "C:\Python_project\ProbDist_QCP_true\VQC_project\Tom\main\", line 71, in __init__
    self.model = model()
  File "C:\Python_project\ProbDist_QCP_true\VQC_project\Tom\main\", line 97, in tf_model_simple_circuit
    quantum_layer = qml.qnn.KerasLayer(circuit, weights, output_dim=1)(transform_layer)
  File "C:\Users\TomBi\OneDrive\.SoSe24\QC_Seminar\Projekt\scrips\ProbDist_QCP3\.lappy_venv\Lib\site-packages\pennylane\qnn\", line 331, in __init__
    self._signature_validation(qnode, weight_shapes)
  File "C:\Users\TomBi\OneDrive\.SoSe24\QC_Seminar\Projekt\scrips\ProbDist_QCP3\.lappy_venv\Lib\site-packages\pennylane\qnn\", line 358, in _signature_validation
    raise TypeError(
TypeError: QNode must include an argument with name inputs for inputting data

The error says that the qnode requires an argument called inputs to function and that it can’t find it. The thing is that the qnode has already an argument that says inputs that only exists for this purpose, so I don’t know what more it requires. It’s clearly related to batch_input, but there is no mentioning of it in the docs, so I don’t really know what options there are.

I included two version of models to give a better idea what the end goal would optimally be able to do, but only getting the complex version to work would already be fine and a big improvement.

We are already deep in the project, so changing libraries would be the last option, but I will change if there is no viable solution to the current situation.
This version of the script should be easily understandable, but I want to answer any questions that would bring me closer to a working solution. So if there are any parts of the code that are unclear, feel free to write it.

Hi @Cap_Cap ,

I’ve noticed two things. On one hand, your decorators are not in the right order. @qml.qnode should be directly above the definition of your function to turn it into a qnode (no other decorators in between).

On the other hand the batch_input transform should be used a bit different. See in the example here in the docs for batch_input we use @partial(qml.batch_input, argnum=1). I’m adding the full example below for reference.

argnum indicates the index of the circuit arguments which are trainable. In this case circuit takes two arguments: inputs and weights. The index for inputs is zero and the index for weights is one. So when we specify argnum=1 it tells the program to find the gradients with respect to weights. Here we don’t worry about the size of weights, it’s just about how many arguments circuit takes.

Note: Remember to import partial with: from functools import partial

Below is the full example code. I hope it helps you!

from functools import partial
dev = qml.device("default.qubit", wires=2, shots=None)

@partial(qml.batch_input, argnum=1)
@qml.qnode(dev, diff_method="parameter-shift", interface="tf")
def circuit(inputs, weights):
    qml.RY(weights[0], wires=0)
    qml.AngleEmbedding(inputs, wires=range(2), rotation="Y")
    qml.RY(weights[1], wires=1)
    return qml.expval(qml.Z(1))

x = tf.random.uniform((10, 2), 0, 1)
w = tf.random.uniform((2,), 0, 1)
circuit(x, w)

Thanks for helping out. I finally got it to work, setting the argnum of @partial(qml.batch_input) to the index of the actual input did the trick. From the documentation I was convinced that it needs the weights and general structure but the @partial(qml.batch_input, argnum=0) and a reshaped_inputs = tf.keras.layers.Reshape((config[‘time_steps’],))(inputs) made it fully functional.

I will post the scripts for completion’s sake so other people perhaps find a useful solution in this.

Final tensorflow script that actually worked as intened (7.1 KB)

The general structure of the tensor flow model and an example circuit that I use in my project:

The Model

from keras.models import Model
from tensorflow.keras.layers import Dense, Input, Flatten
from tensorflow.keras.optimizers import Adam
from main_pipline.input.div.logger import logger
import pennylane as qml
import tensorflow as tf
from tqdm import tqdm

class PACModel:
    def __init__(self, variable_circuit, config):
        self.config = config
        self.circuit = variable_circuit(config)
        self.model = self.create_pac_model(self.circuit, config)

    def train(self, dataset):
        x_train = dataset['input_train']
        y_train = dataset['output_train']

        epochs = self.config['epochs']
        batch_size = self.config['batch_size']
        steps_per_epoch = len(x_train) // batch_size

        history = {'loss': []}

        for epoch in tqdm(range(epochs), desc="Training Progress"):
            epoch_loss = 0
            for step in range(steps_per_epoch):
                batch_start = step * batch_size
                batch_end = (step + 1) * batch_size
                x_batch = x_train[batch_start:batch_end]
                y_batch = y_train[batch_start:batch_end]

                batch_loss = self.model.train_on_batch(x_batch, y_batch)
                epoch_loss += batch_loss
            history['loss'].append(epoch_loss / steps_per_epoch)
  "Epoch {epoch + 1}/{epochs} loss: {epoch_loss / steps_per_epoch}")
            tqdm.write(f"Epoch {epoch + 1}/{epochs}, Loss: {epoch_loss / steps_per_epoch}")
        return history

    def evaluate(self, dataset):
        x_test = dataset['input_test']
        y_test = dataset['output_test']

        pred_y_test_data = self.model.predict(x_test)
        loss = self.model.evaluate(x_test, y_test)
        return pred_y_test_data, loss

    def predict(self, x_test):
        return self.model.predict(x_test)

    def create_pac_model(self, circuit, config):
        inputs = Input(shape=(config['time_steps'], 1))
        reshaped_inputs = tf.keras.layers.Reshape((config['time_steps'],))(inputs)
        quantum_layer = qml.qnn.KerasLayer(, circuit.get_weights(), output_dim=1)(reshaped_inputs)
        model = Model(inputs=inputs, outputs=quantum_layer)
        model.compile(optimizer=Adam(learning_rate=config['learning_rate']), loss=config['loss_function'])
        return model

An example circuit that can be used:

    def run(self):
        training_device = qml.device("default.qubit", wires=self.n_wires)

        @partial(qml.batch_input, argnum=0)
        @qml.qnode(training_device, interface='tf')
        def example_complex_circuit(inputs, weights_RY, weights_RX, weights_RZ):
            qml.AmplitudeEmbedding(features=inputs, wires=range(self.n_wires), normalize=True)
            qml.broadcast(qml.RY, wires=range(self.n_wires), pattern="single", parameters=weights_RY)
            qml.broadcast(qml.RX, wires=range(self.n_wires), pattern="single", parameters=weights_RX)
            qml.broadcast(qml.RZ, wires=range(self.n_wires), pattern="single", parameters=weights_RZ)
            qml.broadcast(qml.CNOT, wires=range(self.n_wires), pattern="all_to_all")
            return [qml.expval(qml.PauliZ(i)) for i in range(self.n_wires)]

        return example_complex_circuit

    def get_weights(self):
        return self.weight_shapes

    def get_wires(self):
        return self.n_wires

Nice @Cap_Cap ! Thanks for posting the full solution :raised_hands: