I need to train a deep learning model with numerical data. The size of dataset is so large that it takes too long time to train the model. Is there any method to preprocess the data using quantum methods so that the training process can be sped up. The only requirement is that we can not do anything to the model. For example, we are not allowed to change the classical deep learning model to a quantum version of deep learning model. We can only process the data using some quantum methods.
Hello @gojiita_ku, you can check out this tutorial that mentions Quantum Preprocessing https://pennylane.ai/qml/demos/tutorial_quanvolution.html. I hope this helps.
Hello @brtymn. Thanks for your insightful suggestion! I tried this method before. But this method in general requires an inner loop that iterates over the entire dataset which might take a very long time in the case of large dataset (my dataset consists of around 100,000 samples with 600 features). I guess this inner loop stems from the lack of quantum devices that support vectorization.
Hi @gojiita_ku,
One of the PennyLane developers suggested the following ideas:
If the model is taking too long to train you should aim to reduce the input size of the training data i.e data compression. It could be possible to use quantum embeddings to do this. This might be able to help by reducing the input dimensions of the training data to speed things up. Though there won’t be any quantum advantage since the model is restricted to classical information processing (there are no quantum layers/operations). In this case, classical data compression methods will likely be easier to implement/understand and produce better results.
On the ML side (non-quantum), without changing the structure of the model you could alter some of the hyperparameters for increased training speed e.g
- increase the learning rate
- take a smaller number of epochs
- initialize model weights
- standardization (mean removal and variance scaling) of the input data
Changing these parameters may have adverse effects on the accuracy and loss.
I hope this helps!
Thanks so much for your reply as well as those insightful ideas suggested by the PennyLane developer!
I agree that quantum embeddings, especially amplitude embedding, are powerful techniques to reduce the feature dimension and help speed up the model training. I’ll try this method.
Besides the number of features, data has another dimension: the number of samples. I think it would be also great to reduce the number of samples using quantum embedding method. But I have no idea how to do that. For example, could we feed N data samples into a quantum embedding layer and we finally get logN samples which lead to similar model performance (e.g. classification accuracy)? Do you have any idea with quantum methods in this direction?
Hi @gojiita_ku, I think the idea of reducing the number of samples is interesting. Something you could do is use classical pre-processing to eliminate duplicated data. You could also remove samples that are very similar according to some criteria you define. However, this is likely to affect the quality of your result.
In fact, most classical algorithms improve their performance as you increase the number of samples, so the efforts are more on how to generate more samples, not reduce them. Note that samples are fundamentally different pieces of data that come from a same model or source. In theory there should be no connection between your data points, other than the underlying model you’re trying to uncover or represent with your circuit, so it’s hard to decide how to combine your data. I personally have never seen a quantum application in this sense.
Please let me know if this makes sense for you!
Hi @CatalinaAlbornoz, thank you for your reply. Yes, in most cases, people are making effort on generating more samples to improve model performance. But in my case, the dataset is too large. I have about 100,000 samples with 600 features. It is impractical to train my ML model on this dataset in my local environment. So the requirement of my project is to reduce the number of samples using quantum methods. I notice one interesting example about amplitude encoding in this page, but I don’t know if it could work for my case. I actually post a new question about it in the forum. If you are interested in it, you may take a look.
Thank you @gojiita_ku, I will take a look at your new question.