# Hybrid Quantum-Classical network

**URL:** <https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647>\
**Category:** PennyLane Help\
**Created:** [October 29, 2020, 2:47pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647 "2020-10-29T14:47:35Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![NikSchet](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/nikschet/32/786_2.png) [@NikSchet](https://discuss.pennylane.ai/u/NikSchet)\
**Post date:** [October 29, 2020, 2:47pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/1 "2020-10-29T14:47:35Z")

</div>

Hello all i am using this code with very nice results

```
     nqubits=2
    device = qml.device('default.qubit', wires=nqubits)

    # Define QNode
    @qml.qnode(device)
    def qnode(inputs, weights):
    qml.templates.AngleEmbedding(inputs, wires=range(nqubits))
    qml.templates.StronglyEntanglingLayers(weights, wires=range(nqubits))
        return [qml.expval(qml.PauliZ(i)) for i in range(nqubits)]

    # define weight_shapes
    weight_shapes = {"weights": (n_layers, nqubits, 3)}
    # Define inputs and qnode trainable weights
    qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=nqubits)
    clayer = tf.keras.layers.Dense(output_size)
    model = tf.keras.Sequential([qlayer, clayer])

    opt = tf.keras.optimizers.SGD(learning_rate=0.05)
    model.compile(opt, loss='binary_crossentropy',metrics=['accuracy','mse', 'mae',"binary_accuracy"])

```

I am wondering if i can just alter the Classical layer so i can have a more options. So can i use something like this for clayer? (note dropout)

```
NN = Sequential()
NN.add(Dense(5, input_dim=d, kernel_initializer='uniform', activation='relu'))
NN.add(Dropout(rate=dropout_rate))
NN.add(Dense(2, kernel_initializer='uniform', activation='relu'))
NN.add(Dropout(rate=dropout_rate))
NN.add(Dense(2, activation='sigmoid'))
clayer = NN

model = tf.keras.Sequential([qlayer, clayer])

```

Upon implementation of the code above i get some results but the sketch is having difficulties in converging, maybe it is not running as it was suppose to run?

If it is not possible maybe for backprogatation issues how can i implement such a classical layer?

Thank you very much for the support

---

<div class="post-metadata">

**Author:** ![antalszava](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/antalszava/32/171_2.png) [@antalszava](https://discuss.pennylane.ai/u/antalszava)\
**Post date:** [October 29, 2020, 7:06pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/2 "2020-10-29T19:06:39Z")

</div>

Hi @NikSchet,

Thanks so much for your question! 🙂

That sounds odd indeed and it might need further investigation to uncover what exactly is happening here. The new classical layer would not be expected to cause a difference here when using backpropagation.

One approach could be trying to interpolate between the `clayer` defined using `NN` (including dropout) and the single layer. E.g., tying 2 simple layers and see how it trains, maybe that way it can be uncovered what is causing the difficulty at least (maybe dropout?).

Hope this helps with a bit of direction!

---

<div class="post-metadata">

**Author:** ![NikSchet](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/nikschet/32/786_2.png) [@NikSchet](https://discuss.pennylane.ai/u/NikSchet)\
**Post date:** [October 30, 2020, 12:37pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/3 "2020-10-30T12:37:59Z")

</div>

Thank you very much for your answer. So problem is that classical layer needs too many epochs to converge (1000 epochs,) as a result it makes no sense to try this specific approach because in a quantum processor unit that would require too many processing time. And i guess this is the reason why you have the tranfer learning demo in which you pre-train the classical model and then you apply a quantum node. (any ideas what transformations should i do to that demo to apply my classical neural network?)

So what i am trying to do is similar to the transfer learning method.

1. I pre train only the classical neural network and save it
2. Ι use the hybrid code with the saved neural network

This is like a jump start for the code to converge faster.

---

<div class="post-metadata">

**Author:** ![Tom\_Bromley](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/tom_bromley/32/129_2.png) [@Tom\_Bromley](https://discuss.pennylane.ai/u/Tom_Bromley)\
**Post date:** [October 30, 2020, 9:56pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/4 "2020-10-30T21:56:15Z")

</div>

Hi @NikSchet,

Transfer learning does sound like an exciting approach, and should be promising.

One thing I noticed is that your model is a quantum layer followed by a classical layer. This is slightly different to the transfer learning [demo](https://pennylane.ai/qml/demos/tutorial_quantum_transfer_learning.html) we have where we have a classical layer feeding into a quantum one. I don’t think is should be a problem though, although the link to “transfer learning” as a concept may not be as strong.

In terms of getting it to work, it should be a case of:

- Creating a purely classical model and training it.
- Creating a second hybrid model composed of the quantum layer and then the second half of the previous model.
- Ensuring that the parameters of the classical part of the hybrid are initialized to match the classical trained model (you could just reuse the layers).
- Ensuring that the classical parameters are not trained, which I believe should be a case of setting the `trainable` property for each layer.

---

<div class="post-metadata">

**Author:** ![NikSchet](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/nikschet/32/786_2.png) [@NikSchet](https://discuss.pennylane.ai/u/NikSchet)\
**Post date:** [October 31, 2020, 9:26am UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/5 "2020-10-31T09:26:04Z")

</div>

Τhank you very much. Actually you did a very good observation the order of layers play a significant role in my code.

So if use first the classical pretrain model and then the quantum node you get much much higher accuracy.

---

<div class="post-metadata">

**Author:** ![NikSchet](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/nikschet/32/786_2.png) [@NikSchet](https://discuss.pennylane.ai/u/NikSchet)\
**Post date:** [November 2, 2020, 6:23pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/6 "2020-11-02T18:23:38Z")

</div>

I have one more weird question , so for my hybrid network i use

`modelh = tf.keras.Sequential([saved\_model,qlayer,])

What if i use something repeatedly like this

`modelh = tf.keras.Sequential([qlayer,qlayer,qlayer,qlayer,])

Wouldn’t that be equivalent to the data-reuploading classifier?!?

---

<div class="post-metadata">

**Author:** ![antalszava](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/antalszava/32/171_2.png) [@antalszava](https://discuss.pennylane.ai/u/antalszava)\
**Post date:** [November 3, 2020, 4:01pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/7 "2020-11-03T16:01:49Z")

</div>

Hi @NikSchet,

One thing to keep in mind is that both the input and the output of `qml.qnn.KerasLayer`, is classical information (e.g., for a vector of data to be encoded we obtain the expectation value of an Hermitian operator, measurement outcomes, etc.). Contrary to this we can consider quantum layers as unitary transformations making up a quantum circuit where the input of the layer is a quantum state and the output is a quantum state as well. In PennyLane such layers are defined in the `QNode`.

The data-reuploading classifier could be expressed using a `QNode` which is later used with a single `qml.qnn.KerasLayer`. We have a cost function that uses the output of the `QNode` (the expectation value of an Hermitian operator that corresponds to the fidelity of two states). In particular note, that the layers mentioned for the data-reuploading classifier are unitaries of the same quantum circuit (and `QNode`).

---

<div class="post-metadata">

**Author:** ![kevinkawchak](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/kevinkawchak/32/1801_2.png) [@kevinkawchak](https://discuss.pennylane.ai/u/kevinkawchak)\
**Post date:** [September 11, 2023, 12:23am UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/8 "2023-09-11T00:23:35Z")

</div>

Hello,

1. Is it possible now to implement quantum specific dropout variants in PennyLane on A) layers or B) qubits of parameterized quantum circuits?

2. Will there be a new PQC model to include customizations directly to quantum circuits analogous to advanced deep learning techniques?

References:  
a) [Overfitting in quantum machine learning and entangling dropout | SpringerLink](https://link.springer.com/article/10.1007/s42484-022-00087-9)  
b) [[1804.00633] Circuit-centric quantum classifiers](https://arxiv.org/abs/1804.00633)

---

<div class="post-metadata">

**Author:** ![CatalinaAlbornoz](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/catalinaalbornoz/32/1196_2.png) [@CatalinaAlbornoz](https://discuss.pennylane.ai/u/CatalinaAlbornoz)\
**Post date:** [September 12, 2023, 12:18am UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/9 "2023-09-12T00:18:41Z")

</div>

Hi @kevinkawchak , thank you for your question. Let me check with the team and get back to you.

---

<div class="post-metadata">

**Author:** ![Guillermo\_Alonso](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/guillermo_alonso/32/816_2.png) [@Guillermo\_Alonso](https://discuss.pennylane.ai/u/Guillermo_Alonso)\
**Post date:** [September 12, 2023, 3:23pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/10 "2023-09-12T15:23:50Z")

</div>

Hey @kevinkawchak !  
Regarding to the first question: In PennyLane, the first thing you do is define a device with a number of shots. Even if you could randomly choose whether to set certain CNOTs gates, all shots would be executed with respect to the same configuration. If you really want to sometimes place them and sometimes don’t, you would have to run several circuits of one shot and post process the results. In the short term I would say this is the only way.

Could you give more details about 2?

---

<div class="post-metadata">

**Author:** ![kevinkawchak](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/kevinkawchak/32/1801_2.png) [@kevinkawchak](https://discuss.pennylane.ai/u/kevinkawchak)\
**Post date:** [September 12, 2023, 9:53pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/11 "2023-09-12T21:53:21Z")

</div>

Hello @Guillermo_Alonso,  
Such as freezing earlier layers of quantum circuits as training progresses.

---

<div class="post-metadata">

**Author:** ![Guillermo\_Alonso](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/guillermo_alonso/32/816_2.png) [@Guillermo\_Alonso](https://discuss.pennylane.ai/u/Guillermo_Alonso)\
**Post date:** [September 12, 2023, 10:11pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/12 "2023-09-12T22:11:09Z")

</div>

In that case, I would take a look at this [optimizer](https://docs.pennylane.ai/en/stable/code/api/pennylane.AdaptiveOptimizer.html).  
Instead of freezing the initial layers, you step by step increase the depth of the circuit. I would say the results should be equivalent 😄

---

<div class="post-metadata">

**Author:** ![kevinkawchak](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/kevinkawchak/32/1801_2.png) [@kevinkawchak](https://discuss.pennylane.ai/u/kevinkawchak)\
**Post date:** [September 15, 2023, 5:09am UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/13 "2023-09-15T05:09:21Z")

</div>

Thank you for the info.

---

<div class="post-metadata">

**Author:** ![kevinkawchak](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/kevinkawchak/32/1801_2.png) [@kevinkawchak](https://discuss.pennylane.ai/u/kevinkawchak)\
**Post date:** [September 24, 2023, 6:10pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/14 "2023-09-24T18:10:43Z")

</div>

Hello,

From Huynh, L., et al. 2023 “Quantum-Inspired Machine Learning: a Survey”, the authors state:

“In the case of QVAS models, comparable performance is often only observed when classical models are deliberately scaled back in terms of architecture size, the number of parameters and/or number of input features.”

a) Is there an anticipated break-even point for reaching a number of quantum algorithm parameters to outperform classical deep neural networks?  
b) Are there current studies where quantum variational algorithms at lower numbers of parameters are the exclusive way to assist deep learning networks for speedups or other improvements? Thank you.

Reference:

> **[2308.11269.pdf](https://arxiv.org/pdf/2308.11269.pdf)**
>
> 2038.46 KB

---

<div class="post-metadata">

**Author:** ![CatalinaAlbornoz](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/catalinaalbornoz/32/1196_2.png) [@CatalinaAlbornoz](https://discuss.pennylane.ai/u/CatalinaAlbornoz)\
**Post date:** [September 26, 2023, 7:56pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/15 "2023-09-26T19:56:53Z")

</div>

Hi @kevinkawchak ,

Great questions. At the moment classical machine learning is more powerful than quantum machine learning for almost everything. There are some specific cases where quantum computers are more powerful but this is usually only the case for toy problems at the moment.

[This blog post](https://pennylane.ai/blog/2022/03/why-measuring-performance-is-our-biggest-blind-spot-in-quantum-machine-learning/) by Maria Schuld can give you an excellent perspective on this topic.

At the moment researchers over the world are looking into understanding quantum machine learning better in order to find potential for advantage. As far as I know there’s no anticipated break-even point. The question itself is not the best because in quantum algorithms don’t necessarily perform better with more parameters, and there are so many other aspects that change in a quantum algorithm that it’s not necessarily a good way to compare the two.

That being said, the demo on [quantum advantage in learning from experiments](https://pennylane.ai/qml/demos/tutorial_learning_from_experiments), based on a paper cited at the beginning of the demo, shows one of these specific examples where quantum algorithms can have an advantage over classical ones.

---

<div class="post-metadata">

**Author:** ![kevinkawchak](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/kevinkawchak/32/1801_2.png) [@kevinkawchak](https://discuss.pennylane.ai/u/kevinkawchak)\
**Post date:** [September 27, 2023, 7:34pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/16 "2023-09-27T19:34:37Z")

</div>

Hello,

Do any of the 6 Numpy quantum specific optimizers act in ways to change the circuit over the course of the run?

Reference:  
[https://docs.pennylane.ai/en/stable/introduction/interfaces.html](https://docs.pennylane.ai/en/stable/introduction/interfaces.html)

---

<div class="post-metadata">

**Author:** ![Guillermo\_Alonso](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/guillermo_alonso/32/816_2.png) [@Guillermo\_Alonso](https://discuss.pennylane.ai/u/Guillermo_Alonso)\
**Post date:** [September 29, 2023, 3:53pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/17 "2023-09-29T15:53:15Z")

</div>

Hi @kevinkawchak ! The Adaptive [Optimizer](https://docs.pennylane.ai/en/stable/code/api/pennylane.AdaptiveOptimizer.html) modify the circuit during the training.

In the case of this optimizer, if you do `step_and_cost`, 3 values are returned instead of the usual 2. The first one refers to the circuit in that particular iteration.

I hope that helps!

---

<div class="post-metadata">

**Author:** ![kevinkawchak](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/kevinkawchak/32/1801_2.png) [@kevinkawchak](https://discuss.pennylane.ai/u/kevinkawchak)\
**Post date:** [October 6, 2023, 9:36pm UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/18 "2023-10-06T21:36:06Z")

</div>

Hello @Guillermo_Alonso, is there a ML demo that AdaptiveOptimizer would work good in?

---

<div class="post-metadata">

**Author:** ![Guillermo\_Alonso](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/guillermo_alonso/32/816_2.png) [@Guillermo\_Alonso](https://discuss.pennylane.ai/u/Guillermo_Alonso)\
**Post date:** [October 7, 2023, 1:58am UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/19 "2023-10-07T01:58:50Z")

</div>

At the moment we only show this feature in this [chemistry demo](https://pennylane.ai/qml/demos/tutorial_adaptive_circuits).  
However, you could adjust it to another problem of interest in QML 🙂

---

<div class="post-metadata">

**Author:** ![kevinkawchak](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/kevinkawchak/32/1801_2.png) [@kevinkawchak](https://discuss.pennylane.ai/u/kevinkawchak)\
**Post date:** [October 8, 2023, 5:12am UTC](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647/20 "2023-10-08T05:12:59Z")

</div>

Hello Guillermo,  
I’m looking to use a specific quantum circuit for one mini-batch, a different quantum circuit for the next mini-batch, and other quantum circuits for the remainder of the ml workflow. The demo and documentation appear to be for finding the best algorithm, but I’m not seeing the tools I would need to use several different circuits systematically throughout a run.

[Next page](https://discuss.pennylane.ai/t/hybrid-quantum-classical-network/647.md?page=2)