So, @thegajan, I found the reason for the bad performance. It is actually no bug in the PennyLane-Cirq plugin, but in the Data-reuploading classifier example.
The example should not use the internal state of the simulator to make predictions, because this is not hardware-friendly. In this special case using the internal state causes an error because the Cirq plugin applies gates under the hood to diagonalize the Hermitian observables with eigenvalues in ascending order. This means that the final state of the simulation is different for the Cirq simulator and the default qubit simulator which causes erroneous predictions and therefore poor performance.
We will work on fixing the Data-reuploading classifier as soon as possible!