# TF/Pennylane Regression

**URL:** <https://discuss.pennylane.ai/t/tf-pennylane-regression/3994>\
**Category:** PennyLane Help\
**Created:** [January 30, 2024, 11:41pm UTC](https://discuss.pennylane.ai/t/tf-pennylane-regression/3994 "2024-01-30T23:41:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Corey](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/corey/32/2626_2.png) [@Corey](https://discuss.pennylane.ai/u/Corey)\
**Post date:** [January 30, 2024, 11:41pm UTC](https://discuss.pennylane.ai/t/tf-pennylane-regression/3994/1 "2024-01-30T23:41:48Z")

</div>

Hello! For the life of me I cannot seem to get a quantum network to converge very well on a very simple 1D regression problem. The classical Tensorflow (no quantum) converges with just 1 layer and 3 neurons — so I know it’s incredibly easy to converge. However, the hybrid network is giving me a fit as the results converge _very slowly_ relative to the classical convergence and often just gives a flat line – a completely wrong answer.

I have tried countless permutations of qubits, layers, etc, and nothing helps. I’m definitely missing something, because such a simple problem shouldn’t be this hard. There is a sensitivity to the initial conditions, unsurprisingly, so I tried to constrain the qnode parameters to 0 \< param \< pi, hoping this would help the convergence. I haven’t been able to figure how to set the qnode initial parameters yet though, as a print just show the same old TF default parameters.

Here is my code:

```auto
import pennylane as qml
from pennylane import numpy as np
import autograd as ag
import matplotlib.pyplot as plt
import tensorflow as tf
from scipy.stats import qmc
from silence_tensorflow import silence_tensorflow
silence_tensorflow() # to remove typcast warnings ....
tf.keras.backend.set_floatx("float64")
tf.keras.backend.clear_session()

def flatten(xss):
    return [x for xs in xss for x in xs]

n_qubits = 10
n_layers = 4
neurons_per_classical_layer = 1

dev = qml.device("default.qubit", wires=n_qubits)

@qml.qnode(dev)
def qnode(inputs, weights):
    qml.AngleEmbedding(inputs, wires=range(n_qubits))
    qml.templates.StronglyEntanglingLayers(weights, wires=range(n_qubits))
    return [qml.expval(qml.PauliZ(wires=i)) for i in range(n_qubits)]

weight_shapes = {"weights": (n_layers, n_qubits, 3)}

#################################################################
# Prepare data
engine = qmc.LatinHypercube(d=1)
x = engine.random(n_qubits)
y = np.zeros(x.shape)
y[: n_qubits] = (x[: n_qubits]*(1 - np.e) + np.exp(x[: n_qubits]) - 1)/(1 - np.e)
x,y = map(tf.convert_to_tensor, [x,y])
print("x:", *(f"{x:.3f}" for x in flatten(x)))
print("y:", *(f"{x:.3f}" for x in flatten(y)))

################################################################
# initialize weights randomly from a Gaussian distribution
initializer = tf.keras.initializers.RandomUniform(minval=-0.0, maxval=np.pi, seed=0)
weight_specs = {"weights": {"initializer": "random_uniform"}}. # HOW DO I SET THIS?

qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits, weight_specs = weight_specs)
input_layer = tf.keras.layers.Input(shape=(1,))
hidden0 = tf.keras.layers.Dense(neurons_per_classical_layer, activation="tanh",kernel_initializer=initializer)(input_layer)
hidden1 = qlayer(hidden0)
output_layer = tf.keras.layers.Dense(1, activation=None)(hidden1)
model = tf.keras.Model(input_layer, output_layer)

model.summary()

def custom_mean_squared_error(y_true, y_pred):
    return tf.math.reduce_mean(tf.square(y_true - y_pred), axis=-1)

##################################################################
# Run the QML
#model.compile(optimizer=tf.optimizers.SGD(learning_rate=0.01),loss=custom_mean_squared_error)
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01,amsgrad=False),loss=custom_mean_squared_error)
fitting = model.fit(x, y, epochs=500, verbose=2,batch_size=n_qubits)
##################################################################
# Plot Results
fig = plt.figure(figsize =(6, 6))
plt.plot(fitting.history['loss'])
plt.show()

y_predict = model.predict(x)
fig = plt.figure(figsize =(6, 6))
ax = plt.axes()
xx = np.linspace(0, 1, 100) 
yy = (xx*(1 - np.e) + np.exp(xx) - 1)/(1 - np.e)
ax.plot(xx, yy, color='black',linewidth=2.0, label='Analytic');
plt.scatter(x,y_predict,color = 'red', s = 100, label='Predicted')
plt.xlabel("x", fontsize=18)
plt.ylabel("u(x)", fontsize=18)
plt.legend()
plt.show()

```

I’m fairly certain there is a bug. I _should_ be able to solve this with minimal quantum layers only.

Any help/guidance is appreciated!  
-Corey

---

<div class="post-metadata">

**Author:** ![isaacdevlugt](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/isaacdevlugt/32/1159_2.png) [@isaacdevlugt](https://discuss.pennylane.ai/u/isaacdevlugt)\
**Post date:** [January 31, 2024, 11:10pm UTC](https://discuss.pennylane.ai/t/tf-pennylane-regression/3994/2 "2024-01-31T23:10:46Z")

</div>

Hey @Corey,

This is a little strange indeed. To initialize your parameters differently for any Tensorflow/Keras layer, you can use the `kernel_initializer` and `bias_initializer` keyword arguments like so (purely classical tf):

```auto
import tensorflow as tf

custom_weight_initializer = tf.keras.initializers.RandomNormal(mean=0.0, stddev=0.01)
custom_bias_initializer = tf.keras.initializers.Zeros()

# Create a Dense layer with custom initializers
dense_layer = tf.keras.layers.Dense(
    units=64,
    activation='relu',
    kernel_initializer=custom_weight_initializer,
    bias_initializer=custom_bias_initializer
)

```

Luckily, because `qml.qnn.KerasLayer` inherits from Tensorflow’s `Layer` class, you can do the same with a PennyLane QNode turned KerasLayer 🙂

I ran your code and these were the figures that were generated. Is that the same thing you got?

 ![output1](https://canada1.discourse-cdn.com/flex012/uploads/pennylane/original/2X/4/4334cf061c2806cfdb851e44afb7176a5c21becf.png)  
 ![output2](https://canada1.discourse-cdn.com/flex012/uploads/pennylane/original/2X/9/913e9df9f8efa50256c1ed76432f2309daac7d81.png)

---

<div class="post-metadata">

**Author:** ![Corey](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/corey/32/2626_2.png) [@Corey](https://discuss.pennylane.ai/u/Corey)\
**Post date:** [February 1, 2024, 10:01pm UTC](https://discuss.pennylane.ai/t/tf-pennylane-regression/3994/3 "2024-02-01T22:01:19Z")

</div>

I’ve actually been able to get good results with the code below, though I still am not sure how to constrain the qnode parameters to [0:pi]

```auto
import tensorflow as tf
from tensorflow import keras;    
from tensorflow.keras import layers;
from tensorflow.keras import metrics

n_qubits = 10 
layers = 2
data_dimension = 1 # output
param = {'num_epochs': 1000}

xmin = 0
xmax = 1 #np.pi

num_of_data = n_qubits
#X = np.random.uniform(high=2 * np.pi, size=(num_of_data,1))
#Y = np.sin(X[:,0])
X = np.random.uniform(low=xmin,high=xmax, size=(num_of_data,1))
xx = np.linspace(xmin, xmax, num_of_data)
X[0:num_of_data,0] = xx # Try this later X = np.array(X).reshape(num_of_data, )
Y = (X[:,0]*(1 - np.e) + np.exp(X[:,0]) - 1)/(1 - np.e)
print("x: ",X)
print("y: ",Y)

dev = qml.device("lightning.qubit", wires=n_qubits)
@qml.qnode(dev, diff_method='adjoint')
def qnode(inputs, weights):
    #print("qnode inputs: ",inputs) # why do these change? Is it because I have a layer before? Is it optimizing w.r.t these?
    qml.templates.AngleEmbedding(inputs, wires=range(n_qubits))
    qml.templates.StronglyEntanglingLayers(weights, wires=range(n_qubits))
    return [qml.expval(qml.PauliZ(i)) for i in range(n_qubits)] # why isn't this: return qml.expval(qml.PauliZ(0))
    #return qml.expval(qml.PauliZ(0))

weight_shapes = {"weights": (layers, n_qubits,3)}
qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits)

# This works well! But I'm afraid it is just classical optimizing at the first layer
#clayer1 = tf.keras.layers.Dense(n_qubits, activation='linear')
#clayer2 = tf.keras.layers.Dense(data_dimension, activation="linear")
#model = tf.keras.models.Sequential([clayer1,qlayer,clayer2])

# This works and only uses a quantum layer, but what is the input shape???  
#clayer2 = tf.keras.layers.Dense(data_dimension, activation=None) #"linear")
#model = tf.keras.models.Sequential([qlayer,clayer2])

# shape=(n_qubits,) actually found the answer but was slow as xmas
# input_layer = tf.keras.layers.Input(shape=(n_qubits,))
# [[x1,y1],[x2,y2],...] --->? shape=(2,0)
# What we have is [[x1][x2][x3]...[xm]] -> m-data points -> shape=(1,)
# opposed to [[x1,x2,x3,...,xm],[x1,x2,x3,...,xm],...,[x1,x2,x3,...xm]] -> m-dimensions -> shape=(m,)
input_layer = tf.keras.layers.Input(shape=(1,))
hidden0 = qlayer(input_layer)
output_layer = tf.keras.layers.Dense(1, activation=None)(hidden0)
model = tf.keras.Model(input_layer, output_layer)
model.summary()

opt = tf.keras.optimizers.Adam(learning_rate=0.05)
model.compile(opt, loss='mse')
hist = model.fit(X, Y, epochs=param['num_epochs'], verbose=2, batch_size=10000) #1)#024)
loss = hist.history['loss']

# lossのグラフ
fig = plt.figure(figsize =(4, 4))
plt.plot(range(param['num_epochs']), 10*np.log10(loss), marker='.', label='loss')
plt.legend(loc='best', fontsize=10)
plt.grid()
plt.xlabel('epoch')
plt.ylabel('loss')
plt.show()

```

Here are my results

 ![Screen Shot 2024-02-01 at 4.00.50 PM](https://canada1.discourse-cdn.com/flex012/uploads/pennylane/original/2X/f/f0031c584273bd84c2b147db45dd4f1ff750d7e8.png)

---

<div class="post-metadata">

**Author:** ![isaacdevlugt](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/isaacdevlugt/32/1159_2.png) [@isaacdevlugt](https://discuss.pennylane.ai/u/isaacdevlugt)\
**Post date:** [February 2, 2024, 11:41pm UTC](https://discuss.pennylane.ai/t/tf-pennylane-regression/3994/4 "2024-02-02T23:41:15Z")

</div>

Nice work! To get initial parameters within a certain range, try out `RandomUniform`: [tf.keras.initializers.RandomUniform &nbsp;|&nbsp; TensorFlow v2.15.0.post1](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/RandomUniform)

Just set the `minval` and `maxval` keyword args to be what you want 🙂. Let me know if that works!