The loss function decreases, but the test accuracy is always 0.5

RX1 · November 23, 2022, 2:25pm

@qml.qnode(dev1)   
def conv_net(features, para):
    """
    qml.MottonenStatePreparation(state_vector=Norm1DArray(features), wires=range(wires1))
    qml.adjoint(convolutional_layer)(para[:63])
    qml.CNOT(wires=[0, 3])
    qml.CNOT(wires=[1, 4])
    qml.CNOT(wires=[2, 5])
    """
    #qml.CRY(para[0], wires=[3, 4])
    #qml.CRY(para[1], wires=[4, 5])
    #qml.CRY(para[2], wires=[5, 3])
    """    
    qml.CRY(para[63], wires=[3, 4])
    qml.CRY(para[64], wires=[4, 5])
    qml.CRY(para[65], wires=[5, 3])
    """
    qml.MottonenStatePreparation(state_vector=Norm1DArray(features), wires=range(wires1))
    for W in para:
        layer(W)

    return qml.expval(qml.PauliZ(5))

def variational_classifier(para, bias, features):
    return conv_net(features, para) + bias

def cost(para, bias, features, labels):
    predictions = [variational_classifier(para, bias, f) for f in features]
    return square_loss(labels, predictions)

digits = datasets.load_digits()
features, labels = digits.data, digits.target

# only use first two classes
features = features[np.where((labels == 0) | (labels == 1))]
labels = labels[np.where((labels == 0) | (labels == 1))]

num_train = int(0.75 * len(labels))
num_test = len(labels) - num_train

# normalize data
features = features / np.linalg.norm(features, axis=1).reshape((-1, 1))
index = np.random.permutation(range(len(labels)))
print(index)

x_train = features[index[:num_train]]
y_train = labels[index[:num_train]]
x_test = features[index[num_train:]]
y_test = labels[index[num_train:]]
print(x_train, x_train.shape)
print(y_train, y_train.shape)
print(x_test, x_test.shape)
print(y_test, y_test.shape)

opt = qml.NesterovMomentumOptimizer(0.05)
batch_size = 5

# train the variational classifier
weights = para_init
bias = bias_init

for it in range(max_iterations):

    # Update the weights by one optimizer step
    batch_index = np.random.randint(0, num_train, (batch_size,))
    x_train_batch = x_train[batch_index]
    y_train_batch = y_train[batch_index]
    weights, bias, _, _ = opt.step(cost, weights, bias, x_train_batch, y_train_batch)
    #print(weights)

    # Compute predictions on train and validation set
    predictions_train = [np.sign(variational_classifier(weights, bias, f)) for f in x_train]
    predictions_val = [np.sign(variational_classifier(weights, bias, f)) for f in x_test]

    # Compute accuracy on train and validation set
    acc_train = accuracy(y_train, predictions_train)
    acc_val = accuracy(y_test, predictions_val)

    print( "Iter: {:5d} | Cost: {:0.7f} | Acc train: {:0.7f} | Acc validation: {:0.7f} " 
          "".format(it + 1, cost(weights, bias, features, labels), acc_train, acc_val))

I use the reference case: Variational classifier — PennyLane documentation for the binary classification of 0 and 1 in MNIST. And tried different Ansatz and optimizers, the result is that the loss function drops, but the test accuracy and training accuracy are always around 0.5. How to solve it?

@Maria_Schuld
@isaacdevlugt
@CatalinaAlbornoz

CatalinaAlbornoz · November 29, 2022, 3:11am

Hi @RX1 It would seem that your model isn’t generalizing well for the training data. I would first try making a smaller example of your problem just to make sure that the issue isn’t in the code. Then you can make a conclusion about whether your circuit is maybe not being able to generalize or if it can be something else.

You can try for instance modifying your ansatz, your cost function or something else.

I hope this helps!

Topic		Replies	Views
Variational Circuits for Solving Differential Equations PennyLane Help	6	621	April 13, 2022
An error occurred replacing NesterovMomentumOptimizer with QNGOptimizer. How do I do this? PennyLane Help	1	433	November 11, 2022
GD w.r.t. qml.sample dependent cost function raises ValueError PennyLane Help	1	856	April 4, 2021
QNGOptimizer with Variational Classifiers PennyLane Help	2	305	October 3, 2022
Gradient Descent of real scalar-output function PennyLane Help	1	557	September 28, 2020

The loss function decreases, but the test accuracy is always 0.5

Related Topics