Can a quantum generator in Hybrid GAN generate RGB images? The generator is quantum, and the discriminator is classical.
The basic example of GAN is DCGAN Tutorial — PyTorch Tutorials 2.0.0+cu117 documentation
and for QGAN is Quantum GANs — PennyLane documentation
I just need a quantum generator for the data used in the DCGAN tutorial. How to achieve it?
Hey @mass_of_15!
You can create hybrid PyTorch-PennyLane models with qml.qnn.TorchLayer. Here is an example where it’s used: Turning quantum nodes into Torch Layers — PennyLane documentation
For your application — a quantum generator and a classical discriminator — the QGAN demo you linked in your post and the demo I linked above should be enough to get started!
Can we also interface with Convolutional layers other than Linear layers, as shown in your demo: Turning quantum nodes into Torch Layers — PennyLane documentation ?
Yep — interfacing with PyTorch isn’t limited to certain layer types, activation functions, etc. Mix and match to your heart’s content!
Hi @mass_of_15 I am trying to do the same thing, I have RGB images and I would like to train with a patch GAN with standard discriminator and quantum generator. May we help each other?
Sure. I am sorry for the delay in replying. What seems to be the problem?
HI @mass_of_15, I modified the demo quantum gan from pennylane in order to adapt the code to my dataset that is composed by 300 RGB images. I resized images to 64x64 and batch size=32, but the training cell is still running for infinite time or I get a memory crash for the total utilization of RAM both on google colab and locally. So, I tried to reduce the dimension of the dataset from 300 items to 30 and the batchsize from 32 to 1 and the training completed, but after 200 epochs the output images are black. Can I ask you if there are errors in the code or in the visualization of images?? Do you have an example of implementation of quantum gan with RGB images?
for k in range (len(test_images)):
fig,axs = plt.subplots(1, 1, sharey=False, tight_layout=True, figsize=(2,2), facecolor=‘white’)
#axs.matshow(np.squeeze(test_images[k].permute(1,2,0)))
axs.matshow(test_images[k].T)
!pip install pennylane custatevec-cu11 pennylane-lightning-gpu
# Library imports
import math
import random
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import pennylane as qml
# Pytorch imports
import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
from torch.utils.data import Dataset, DataLoader
# Set the random seed for reproducibility
seed = 999
torch.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
# Number of workers for dataloader
workers = 2
# Batch size during training
batch_size = 32
# Spatial size of training images. All images will be resized to this
# size using a transformer.
image_size = 64
# Number of channels in the training images. For color images this is 3
nc = 3
# Size of z latent vector (i.e. size of generator input)
nz = 13
# Size of feature maps in generator
ngf = 64
# Size of feature maps in discriminator
ndf = 64
# Number of training epochs
num_epochs = 100
# Learning rate for optimizers
lr = 0.0002
# Beta1 hyperparam for Adam optimizers
beta1 = 0.5
# Number of GPUs available. Use 0 for CPU mode.
ngpu = 1
from google.colab import drive
drive.mount('/content/drive')
device = torch.device("cuda:0" if (torch.cuda.is_available() and ngpu > 0) else "cpu")
import torchvision.datasets as dset
ImgLocation='/content/drive/MyDrive/Colab Notebooks/Subset_Dil_Bos/'
dataset = dset.ImageFolder(root=ImgLocation,
transform=transforms.Compose([
transforms.Resize(image_size),
transforms.CenterCrop(image_size),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
]))
dataloader = torch.utils.data.DataLoader(dataset, batch_size=batch_size,shuffle=True, num_workers=workers)
def weights_init(m):
classname = m.__class__.__name__
if classname.find('Conv') != -1:
nn.init.normal_(m.weight.data, 0.0, 0.02)
elif classname.find('BatchNorm') != -1:
nn.init.normal_(m.weight.data, 1.0, 0.02)
nn.init.constant_(m.bias.data, 0)
class Discriminator(nn.Module):
"""Fully connected classical discriminator"""
def __init__(self,ngpu):
super().__init__()
self.ngpu = ngpu
self.model = nn.Sequential(
# Inputs to first hidden layer (num_input_features -> 64)
nn.Linear(3*image_size * image_size, 64,device=device),
nn.ReLU(),
nn.Linear(64, 64,device=device),
nn.ReLU(),
# First hidden layer (64 -> 16)
nn.Linear(64, 16,device=device),
nn.ReLU(),
# Second hidden layer (16 -> output)
nn.Linear(16, 1,device=device),
nn.Sigmoid(),
)
def forward(self, x):
return self.model(x)
discriminator = Discriminator(ngpu).to(device)
# Handle multi-gpu if desired
if (device.type == 'cuda') and (ngpu > 1):
discriminator = nn.DataParallel(discriminator, list(range(ngpu)))
# Apply the weights_init function to randomly initialize all weights
# to mean=0, stdev=0.2.
discriminator.apply(weights_init)
# Print the model
print(discriminator)
# Quantum variables
n_qubits = 13 # Total number of qubits / N
n_a_qubits = 2 # Number of ancillary qubits / N_A
q_depth = 6 # Depth of the parameterised quantum circuit / D
n_generators = 6 # Number of subgenerators for the patch method / N_G
dev = qml.device("lightning.gpu", wires=n_qubits)
@qml.qnode(dev, interface="torch") #, diff_method="parameter-shift"
def quantum_circuit(noise, weights):
weights = weights.reshape(q_depth, n_qubits)
# Initialise latent vectors
for i in range(n_qubits):
qml.RY(noise[i], wires=i)
# Repeated layer
for i in range(q_depth):
# Parameterised layer
for y in range(n_qubits):
qml.RY(weights[i][y], wires=y)
# Control Z gates
for y in range(n_qubits - 1):
qml.CZ(wires=[y, y + 1])
return qml.probs(wires=list(range(n_qubits)))
def partial_measure(noise, weights):
# Non-linear Transform
probs = quantum_circuit(noise, weights)
probsgiven0 = probs[: (2 ** (n_qubits - n_a_qubits))]
probsgiven0 /= torch.sum(probs)
# Post-Processing
probsgiven = probsgiven0 / torch.max(probsgiven0)
return probsgiven
class PatchQuantumGenerator(nn.Module):
"""Quantum generator class for the patch method"""
def __init__(self, n_generators, q_delta=1):
"""
Args:
n_generators (int): Number of sub-generators to be used in the patch method.
q_delta (float, optional): Spread of the random distribution for parameter initialisation.
"""
super().__init__()
self.q_params = nn.ParameterList(
[
nn.Parameter(q_delta * torch.rand(q_depth * n_qubits), requires_grad=True)
for _ in range(n_generators)
]
)
self.n_generators = n_generators
def forward(self, x):
# Size of each sub-generator output
patch_size = 2 ** (n_qubits - n_a_qubits)
# Create a Tensor to 'catch' a batch of images from the for loop. x.size(0) is the batch size.
images = torch.Tensor(x.size(0), 0).to(device)
# Iterate over all sub-generators
for params in self.q_params:
# Create a Tensor to 'catch' a batch of the patches from a single sub-generator
patches = torch.Tensor(0, patch_size).to(device)
for elem in x:
q_out = partial_measure(elem, params).float().unsqueeze(0)
patches = torch.cat((patches, q_out))
# Each batch of patches is concatenated with each other to create a batch of images
images = torch.cat((images, patches), 1)
return images
lrG = 0.3 # Learning rate for the generator
lrD = 0.01 # Learning rate for the discriminator
num_iter = 100# Number of training iterations
discriminator = Discriminator(ngpu).to(device)
generator = PatchQuantumGenerator(n_generators).to(device)
# Binary cross entropy
criterion = nn.BCELoss()
# Optimisers
optD = optim.SGD(discriminator.parameters(), lr=lrD)
optG = optim.SGD(generator.parameters(), lr=lrG)
real_labels = torch.full((batch_size,), 1.0, dtype=torch.float, device=device)
fake_labels = torch.full((batch_size,), 0.0, dtype=torch.float, device=device)
# Fixed noise allows us to visually track the generated images throughout training
fixed_noise = torch.rand(image_size, n_qubits, device=device) * math.pi / 2
# Iteration counter
counter = 0
# Collect images for plotting later
results = []
while True:
for i, (data, _) in enumerate(dataloader):
# Data for training the discriminator
data = data.reshape( -1,image_size * image_size*3) #data=(32,12288)
real_data = data.to(device) #real_data=(32,12288)
# Training the discriminator
# Noise follwing a uniform distribution in range [0,pi/2)
noise = torch.rand(batch_size, n_qubits, device=device) * math.pi / 2 #noise=(32,13)
fake_data = generator(noise) #fake_data=(32,12288)
discriminator.zero_grad()
outD_real = discriminator(real_data).view(-1) #(outD_real=32)
errD_real = criterion(outD_real, real_labels) #(criterion(32, 32))
outD_fake = discriminator(fake_data.detach()).view(-1) #(outD_fake=32)
errD_fake = criterion(outD_fake, fake_labels) #(criterion(32,32))
# Propagate gradients
errD_real.backward()
errD_fake.backward()
errD = errD_real + errD_fake
optD.step()
# Training the generator
generator.zero_grad()
outD_fake = discriminator(fake_data).view(-1) #outD_fake=32
errG = criterion(outD_fake, real_labels) #criterion(32,32)
errG.backward()
optG.step()
counter += 1
# Show loss values
if counter % 10 == 0:
print(f'Iteration: {counter}, Discriminator Loss: {errD:0.3f}, Generator Loss: {errG:0.3f}')
test_images = generator(fixed_noise).view(image_size,3,image_size,image_size).cpu().detach()
# Save images every 50 iterations
if counter % 50 == 0:
results.append(test_images)
if counter == num_iter:
break
if counter == num_iter:
break
I saw your code. In the below line:
test_images = generator(fixed_noise).view(image_size,3,image_size,image_size).cpu().detach()
I think it should be
test_images = generator(fixed_noise).view(batch_size,3,image_size,image_size).cpu().detach()
Just check and see if there is any improvement.
Also can I ask you what type of images are you generating?
The images of my dataset are comics images from Dilbert, I attached one image as an example.
I tried to change the line code that you have suggested (substituting image_size with batch_size also in fixed_noise), but the result is the same. If I use the entire dataset of 300 images and batch_size=32 the code will block during the training in an infinite loop. If I used 30 images and batch_size=1 I get the same generated images that I attached in the previous email.
Do you have any suggestions?
You can try the code and dataset in github: https://github.com/Elyon7jkd/qGAN.git
I also had the same problem. I tried to increase epoch cycles, like
num_epoch = len(dataloader)*500 # if batch_size > 1
This got my results. I used a batch size of 16 for 1800 images.
Also, can you rerun the code, but this time add the following line into the code during the training session:
G_losses.append(errG.item())
D_losses.append(errD.item())
Try plotting the graph and visualizing both generator and discriminator losses. Send your training loss graph.
You can use this link to understand better: DCGAN Tutorial — PyTorch Tutorials 2.1.0+cu121 documentation.
Also, I think GAN needs more images to train at least 1000.
The problem is that if I use a dataset so large i.e. 300 items, the memory crash both on colab and locally where I have 32 GB RAM). I tried also with a batch size of 16, but I need to use the completed dataset so there is the same problem. I tried now also to increase the epochs as you suggested but there is the problem that the training cell is still running for infinite time without resulting no one epoch. Can you try to run the code that I shared on github and using my dataset and give me a feedback? I have also already studied the link of the dc gan that you have send me and kept some suggestion from this. Can you share your code used for rgb images so I can compare with mine?
thank you
I ran your code. Can I ask you where you are running this code? Is it CPU or GPU? I am using an NVIDIA DGX A-100 GPU to run this code. Also, I have made a few changes.
batch_size = 32
# Quantum variables
n_qubits = 13 # Total number of qubits / N
n_a_qubits = 2 # Number of ancillary qubits / N_A
q_depth = 2 # Depth of the parameterized quantum circuit / D
n_generators = 6 # Number of subgenerators for the patch method / N_G
....
# Binary cross entropy
criterion = nn.BCELoss()
# Optimisers
optD = optim.SGD(discriminator.parameters(), lr=lrD)
optG = optim.SGD(generator.parameters(), lr=lrG)
# Fixed noise allows us to visually track the generated images throughout training
fixed_noise = torch.rand(batch_size, n_qubits, device=device) * math.pi / 2
# Iteration counter
counter = 0
# Collect images for plotting later
results = []
while True:
for i, (data, _) in enumerate(dataloader):
# Data for training the discriminator
data = data.reshape(-1, -1,image_size * image_size*3)
real_data = data.to(device)
b_size = real_data.size(0)
real_labels = torch.full((b_size), 1.0, dtype=torch.float, device=device)
fake_labels = torch.full((b_size), 0.0, dtype=torch.float, device=device)
# Training the discriminator
noise = torch.rand(b_size, n_qubits, device=device) * math.pi / 2
fake_data = generator(noise)
discriminator.zero_grad()
outD_real = discriminator(real_data).view(-1)
errD_real = criterion(outD_real, real_labels)
# Noise following a uniform distribution in range [0,pi/2)
outD_fake = discriminator(fake_data.detach()).view(-1)
errD_fake = criterion(outD_fake, fake_labels)
# Propagate gradients
errD_real.backward()
errD_fake.backward()
errD = errD_real + errD_fake
optD.step()
# Training the generator
generator.zero_grad()
outD_fake = discriminator(fake_data).view(-1)
errG = criterion(outD_fake, real_labels)
errG.backward()
optG.step()
counter += 1
# Show loss values
if counter % 10 == 0:
print(f'Iteration: {counter}, Discriminator Loss: {errD:0.3f}, Generator Loss: {errG:0.3f}')
test_images = generator(fixed_noise).view(batch_size,3,image_size,image_size).cpu().detach()
# Save images every 50 iterations
if counter % 50 == 0:
results.append(test_images)
if counter == num_iter:
break
if counter == num_iter:
break
This code is able to run the complete epoch cycle.
Hi, does anyone know how to code Quantum Batch GAN? Can anyone explain to me how it works and how it is different from Quantum Patch GAN, which is explained in Quantum GANs | PennyLane Demos
Please tell me where the code changes in Quantum Batch GAN compared to Quantum Patch GAN.
I read the paper that mentions Quantum Batch GAN at https://arxiv.org/pdf/2010.06201.pdf, but I am unable to understand it properly. Please help.
HI @mass_of_15, thank you for your help, but I applied your changes to the code and when I run the training cell I get this message:
RuntimeError Traceback (most recent call last)
[<ipython-input-17-fc6910e7b28e>](https://localhost:8080/#) in <cell line: 7>()
10
11 # Data for training the discriminator
---> 12 data = data.reshape(-1, -1,image_size * image_size*3)
13
14 real_data = [data.to](http://data.to)(device)
RuntimeError: only one dimension can be inferred
I don't understand why this error occurred because the shape of data is (32,3,64,64), so it should be correct.
I run the code using GPU, locally I have a personal computer Acer Predator Helios 300 with Nvidia geforce RTX 3080Ti 8GB and 32GB RAM.
Sorry the updated is:
data = data.reshape(-1,image_size * image_size*3)
Have you made any changes to the discriminator or generator?? I have seen that you reduce the number of layers of the generator from 6 to 2. Anyway, the code with your changes get the error of the previous mail. I tried to change the training cell in this way in order to not reshape ‘data’, but I get another error:
#for i, (data, _) in enumerate(dataloader):
for i, data in enumerate(dataloader, 0):
#data = data.reshape(-1, -1,image_size * image_size*3)
real_data = data[0].to(device)
b_size = real_data.size(0)
real_labels = torch.full((b_size,), 1.0, dtype=torch.float, device=device)
fake_labels = torch.full((b_size,), 0.0, dtype=torch.float, device=device)
…
RuntimeError Traceback (most recent call last)
[<ipython-input-25-57b2a0e2398e>](https://localhost:8080/#) in <cell line: 7>()
28
29 discriminator.zero_grad()
---> 30 outD_real = discriminator(real_data).view(-1)
31
32 errD_real = criterion(outD_real, real_labels)
The discriminator, I believe, is still the same.
class Discriminator(nn.Module):
"""Fully connected classical discriminator"""
def __init__(self,ngpu):
super().__init__()
self.ngpu = ngpu
self.model = nn.Sequential(
# Inputs to first hidden layer (num_input_features -> 64)
nn.Linear(3*image_size * image_size, 64),
nn.ReLU(),
nn.Linear(64, 64),
nn.ReLU(),
# First hidden layer (64 -> 16)
nn.Linear(64, 16),
nn.ReLU(),
# Second hidden layer (16 -> output)
nn.Linear(16, 1),
nn.Sigmoid(),
)
def forward(self, x):
return self.model(x)
Ok, now the code is running, but I have the same problem, it does not manage to complete the iteration because the training cell is still running. The iterations do not compare during the training. I am training both on colab and locally using jupyter (anaconda). I don’t understand if the problem is my hardware, but I think it is performant (I have already trained GANs but without a quantum generator). Can you send me your output images after the training? Thank you
It is really slow but after 100 epochs this is the output. I can try to go on training, but maybe there are some problems??
My output is this for 300 epoch cycles.
My generator and discriminator losses are:
Iteration: 10, Discriminator Loss: 5.564, Generator Loss: 0.017
Iteration: 20, Discriminator Loss: 1.745, Generator Loss: 0.241
Iteration: 30, Discriminator Loss: 1.013, Generator Loss: 0.494
Iteration: 40, Discriminator Loss: 0.735, Generator Loss: 0.668
Iteration: 50, Discriminator Loss: 0.625, Generator Loss: 0.778
Iteration: 60, Discriminator Loss: 0.646, Generator Loss: 0.796
Iteration: 70, Discriminator Loss: 0.563, Generator Loss: 0.865
Iteration: 80, Discriminator Loss: 0.550, Generator Loss: 0.897
Iteration: 90, Discriminator Loss: 0.507, Generator Loss: 0.945
Iteration: 100, Discriminator Loss: 0.501, Generator Loss: 0.959
Iteration: 110, Discriminator Loss: 0.476, Generator Loss: 0.996
Iteration: 120, Discriminator Loss: 0.486, Generator Loss: 1.010
Iteration: 130, Discriminator Loss: 0.425, Generator Loss: 1.079
Iteration: 140, Discriminator Loss: 0.488, Generator Loss: 1.041
Iteration: 150, Discriminator Loss: 0.425, Generator Loss: 1.087
Iteration: 160, Discriminator Loss: 0.407, Generator Loss: 1.122
Iteration: 170, Discriminator Loss: 0.381, Generator Loss: 1.163
Iteration: 180, Discriminator Loss: 0.370, Generator Loss: 1.196
Iteration: 190, Discriminator Loss: 0.366, Generator Loss: 1.211
Iteration: 200, Discriminator Loss: 0.362, Generator Loss: 1.222
Iteration: 210, Discriminator Loss: 0.335, Generator Loss: 1.294
Iteration: 220, Discriminator Loss: 0.330, Generator Loss: 1.286
Iteration: 230, Discriminator Loss: 0.336, Generator Loss: 1.291
Iteration: 240, Discriminator Loss: 0.358, Generator Loss: 1.281
Iteration: 250, Discriminator Loss: 0.323, Generator Loss: 1.323
Iteration: 260, Discriminator Loss: 0.298, Generator Loss: 1.394
Iteration: 270, Discriminator Loss: 0.293, Generator Loss: 1.407
Iteration: 280, Discriminator Loss: 0.277, Generator Loss: 1.454
Iteration: 290, Discriminator Loss: 0.325, Generator Loss: 1.382
Iteration: 300, Discriminator Loss: 0.272, Generator Loss: 1.491
I also want to tell you when we should use Quantum Patch GAN. When N < log(M), (base 2), where N is the number of qubits and M is the dimension of the image, QPatch GAN is used. So in our case, we have 64x64x3 images, which give M = 12288. So log(M) = log(12288) = 13. So our number of qubits, N, should be less than 13.