Hello! For the life of me I cannot seem to get a quantum network to converge very well on a very simple 1D regression problem. The classical Tensorflow (no quantum) converges with just 1 layer and 3 neurons — so I know it’s incredibly easy to converge. However, the hybrid network is giving me a fit as the results converge very slowly relative to the classical convergence and often just gives a flat line – a completely wrong answer.
I have tried countless permutations of qubits, layers, etc, and nothing helps. I’m definitely missing something, because such a simple problem shouldn’t be this hard. There is a sensitivity to the initial conditions, unsurprisingly, so I tried to constrain the qnode parameters to 0 < param < pi, hoping this would help the convergence. I haven’t been able to figure how to set the qnode initial parameters yet though, as a print just show the same old TF default parameters.
Here is my code:
import pennylane as qml
from pennylane import numpy as np
import autograd as ag
import matplotlib.pyplot as plt
import tensorflow as tf
from scipy.stats import qmc
from silence_tensorflow import silence_tensorflow
silence_tensorflow() # to remove typcast warnings ....
tf.keras.backend.set_floatx("float64")
tf.keras.backend.clear_session()
def flatten(xss):
return [x for xs in xss for x in xs]
n_qubits = 10
n_layers = 4
neurons_per_classical_layer = 1
dev = qml.device("default.qubit", wires=n_qubits)
@qml.qnode(dev)
def qnode(inputs, weights):
qml.AngleEmbedding(inputs, wires=range(n_qubits))
qml.templates.StronglyEntanglingLayers(weights, wires=range(n_qubits))
return [qml.expval(qml.PauliZ(wires=i)) for i in range(n_qubits)]
weight_shapes = {"weights": (n_layers, n_qubits, 3)}
#################################################################
# Prepare data
engine = qmc.LatinHypercube(d=1)
x = engine.random(n_qubits)
y = np.zeros(x.shape)
y[ : n_qubits] = (x[ : n_qubits]*(1 - np.e) + np.exp(x[ : n_qubits]) - 1)/(1 - np.e)
x,y = map(tf.convert_to_tensor, [x,y])
print("x:", *(f"{x:.3f}" for x in flatten(x)))
print("y:", *(f"{x:.3f}" for x in flatten(y)))
################################################################
# initialize weights randomly from a Gaussian distribution
initializer = tf.keras.initializers.RandomUniform(minval=-0.0, maxval=np.pi, seed=0)
weight_specs = {"weights": {"initializer": "random_uniform"}}. # HOW DO I SET THIS?
qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits, weight_specs = weight_specs)
input_layer = tf.keras.layers.Input(shape=(1,))
hidden0 = tf.keras.layers.Dense(neurons_per_classical_layer, activation="tanh",kernel_initializer=initializer)(input_layer)
hidden1 = qlayer(hidden0)
output_layer = tf.keras.layers.Dense(1, activation=None)(hidden1)
model = tf.keras.Model(input_layer, output_layer)
model.summary()
def custom_mean_squared_error(y_true, y_pred):
return tf.math.reduce_mean(tf.square(y_true - y_pred), axis=-1)
##################################################################
# Run the QML
#model.compile(optimizer=tf.optimizers.SGD(learning_rate=0.01),loss=custom_mean_squared_error)
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01,amsgrad=False),loss=custom_mean_squared_error)
fitting = model.fit(x, y, epochs=500, verbose=2,batch_size=n_qubits)
##################################################################
# Plot Results
fig = plt.figure(figsize =(6, 6))
plt.plot(fitting.history['loss'])
plt.show()
y_predict = model.predict(x)
fig = plt.figure(figsize =(6, 6))
ax = plt.axes()
xx = np.linspace(0, 1, 100)
yy = (xx*(1 - np.e) + np.exp(xx) - 1)/(1 - np.e)
ax.plot(xx, yy, color='black',linewidth=2.0, label='Analytic');
plt.scatter(x,y_predict,color = 'red', s = 100, label='Predicted')
plt.xlabel("x", fontsize=18)
plt.ylabel("u(x)", fontsize=18)
plt.legend()
plt.show()
I’m fairly certain there is a bug. I should be able to solve this with minimal quantum layers only.
Any help/guidance is appreciated!
-Corey
