When removing the kernel in XgboostRegression the result doesnt change, which is strange. In documentation of xgboost, I found no hint, How kernel could be used?
Hi @Kursat_KILIC,
Welcome! 
PennyLane currently doesn’t have a dedicated integration with the XGBoost library, therefore some features might not be compatible.
Having said that, this seems to be something out of scope for PennyLane and rather be related to using XGBoost and in specific the XGBRegressor class: does it actually take a kernel keyword argument? 
For future reference, including code in a text format helps us with addressing issues in a swift manner.
Thanks for your reply. Thought ir was kernel question because when I removed the kernel the result was still same. Therefore, I suppose entegration problem between QML kernel and xgboost.
Additionally, apart from xgboost regression, I have tried SVR and it takes too long time for prediction. In this case, pennylane may have shortcoming for regression. So, I have asked how could I use kernel for xgboost?
If you would like to see all code I would like to share it in forum.
Thank you
Hi @Kursat_KILIC,
Could it be that the result remained the same because passing the kernel kwarg (independent of PennyLane) doesn’t influence the functioning of the XGBRegressor class?
Frankly, I’m too familiar with XGBoost, but a good start would be to see a general regression example that contains a minimal function and then attempt to attempt to swap that function with a PennyLane QNode.
Let us know, if any questions or errors arise when using PennyLane or if perhaps others could benefit from your findings on performing predictions (e.g., even with SVR) with PennyLane.
1 Like
Thanks for your reply and all your consideration. I want to show Quantum Kernel SVR and as you will see in the following code, the SVR has a poor correlation. How could be obtained strong correlation?
Here is the code:
import pandas as pd
import xgboost
import shap
import numpy as np
from sklearn.metrics import mean_squared_error
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
X, y = load_boston(return_X_y=True)
from sklearn.datasets import load_boston
boston_dataset = load_boston()
boston = pd.DataFrame(boston_dataset.data, columns=boston_dataset.feature_names)
boston['MEDV'] = boston_dataset.target
boston.head()
import pennylane as qml
from pennylane.templates import AngleEmbedding, StronglyEntanglingLayers
from pennylane.operation import Tensor
import matplotlib.pyplot as plt
np.random.seed(42)
# To see important feature:
import shap
# train an XGBoost model
X, y = shap.datasets.boston()
model = xgboost.XGBRegressor().fit(X, y)
# explain the model's predictions using SHAP
# (same syntax works for LightGBM, CatBoost, scikitlearn, transformers, Spark, etc.)
explainer = shap.Explainer(model)
shap_values = explainer(X)
# visualize the first prediction's explanation
shap.plots.waterfall(shap_values[0])
from sklearn.preprocessing import StandardScaler
X, y = load_boston(return_X_y=True)
X = pd.DataFrame(np.c_[boston['LSTAT'], boston['PTRATIO'],boston['NOX'],boston['B']], columns = ['LSTAT','PTRATIO','NOX','B'])
y = boston['MEDV']
X = X[:100]
y = y[:100]
scaler = StandardScaler().fit(X)
X_scaled = scaler.transform(X)
X_train, X_test, y_train, y_test = train_test_split(X_scaled,y, test_size=0.20, random_state=42)
# Here is the Kernel:
dev_kernel = qml.device("default.qubit", wires=n_qubits)
projector = np.zeros((2**n_qubits, 2**n_qubits))
projector[0, 0] = 1
@qml.qnode(dev_kernel)
def kernel(x1, x2):
"""The quantum kernel."""
AngleEmbedding(x1, wires=range(n_qubits))
qml.adjoint(AngleEmbedding)(x2, wires=range(n_qubits))
return qml.expval(qml.Hermitian(projector, wires=range(n_qubits)))
kernel(X_train[0,:], X_train[0,:])
# Model SVR:
from sklearn.svm import SVR
svm = SVR(kernel=kernel_matrix).fit(X_train, y_train)
# Prediction:
predictions = svm.predict(X_test)
r2_score(y_test,predictions)
# R^2 Score:
0.5875492460061702
# Without Kernel:
svm1 = SVR().fit(X_train, y_train)
# Prediction:
predictions = svm1.predict(X_test)
r2_score(y_test,predictions)
# R^2 Score:
0.5972305912931388
As you can see with the kernel is slightly lower than without kernel.
Hi @Kursat_KILIC,
Thank you for sharing your approach! Not entirely sure, how stronger correlations could be extracted. Could ask around in our team, but it can be that we might not have a clear direction that we could provide here.
Hi, thanks for your reply. Do I need to create a new topic for that question in the forum?
I can ask as well in Slack.
Hi again, I have shared that discussion in Slack.
Hey @Kursat_KILIC!
I’d guess that your quantum kernel is not outperforming the standard because the quantum circuit is fairly simple, involving two angle embeddings. In fact, you can get quite creative in how the circuit is constructed, as long as it can input the two datapoints and return a value. My recommendation would be to investigate alternative circuits by seeing how things perform using a variety of operations and templates.
For example, what if you inserted a StronglyEntanglingLayers template in between your angle embeddings, using fixed weights. The model may perform better, or it may not - the best practices in this setting are still being developed. You could check out this paper from @Maria_Schuld for intuition on the expressive power of the circuit and how you might improve it.
I also want to point out that we recently added a kernels module to PennyLane that has some useful functionality, e.g., for finding the kernel_matrix. Note that this module is in the development version of PennyLane and you will have to install using these instructions.
Thanks for your reply. I will try to apply your suggestion. By the way, some of community members will be able to provide new point of view and we can discuss again.
Thank you.
1 Like