CleitonForge: neutral Rust benchmarking found Rz sign convention bug (fidelity = 0.0 on QAOA)

Quantum Computing
1

Hi PennyLane community!

I built CleitonForge, a neutral benchmarking framework that routes identical quantum circuits through multiple Rust-native backends simultaneously and compares statevector fidelity. No Python simulation — pure Rust, with a Python wrapper via PyO3.

What I found: While benchmarking QAOA circuits, one backend (quantrs2) returned fidelity = 0.0 silently. No error, no warning. The root cause is a reversed Rz gate sign convention vs IBM/OpenQASM 3:

  • OpenQASM 3 standard: Rz(λ) = diag(e^{-iλ/2}, e^{+iλ/2})
  • quantrs2: uses the opposite sign

Quantum Volume benchmarks are completely blind to this because Haar-random circuits are statistically symmetric — the sign error cancels. Structured circuits like QAOA diverge completely.

Links:

Would love to hear if PennyLane has encountered similar convention mismatches when interfacing with other backends. Happy to answer questions!

2

Hi @cleiton_augusto, welcome to the Forum!

I did hear about this minus sign problem from someone comparing IBM with a different platform (not PennyLane) but I cannot remember which one.

I haven’t heard of anyone having this issue with PennyLane, but of course it depends on what you compare it against. Since different platforms use different conventions it can be tricky to know what to use, and it can be slow to always double check everything with the documentation (and for some frameworks the docs are not very good).

For quantum compilation for example one option is defining exchange formats. This inspired the creation of jeff. Here’s the headline description from the repo:

jeff is a structured, extensible, and simple interchange format for quantum compilers. Designed to foster collaboration and accelerate research by providing a common language for compilers to communicate.

jeff is an open source project, still small and with lots of room to grow, but feel free to experiment with it or ask here if you have questions about it. I can forward your question to the maintainers of jeff.

I hope this helps!

1 Like
3

Hi @CatalinaAlbornoz, thanks for the warm welcome and for the pointer to jeff!

Just to clarify the distinction — CleitonForge and jeff are complementary tools, not overlapping ones:

  • jeff is an interchange format for quantum compilers — it solves the “how do we serialize a circuit to pass between tools” problem
  • CleitonForge is a benchmark/testing layer — it solves the “are two backends computing the same unitary matrix?” problem

The Rz sign bug I found is a good example of why both matter: even if two frameworks share circuits via a perfect interchange format, if one applies diag(e^{-iλ/2}, e^{+iλ/2}) and the other applies diag(e^{+iλ/2}, e^{-iλ/2}), the shared circuit still produces orthogonal states at runtime. The format is fine; the gate semantics diverge.

Since this thread, we shipped QGCS v0.1 (Quantum Gate Convention Standard) — a 12-check phase-sensitive conformance suite that catches exactly this class of bug. The key insight is that Quantum Volume and probability-based tests are algebraically blind to the sign inversion: for circuits built from Rz + Ry + CX, the sign flip produces U_wrong = U*_ref, which gives identical measurement probabilities for all basis states. Only amplitude-ratio checks (inspecting the complex phase directly) can detect it.

Results so far against 4 Rust backends: native/roqoqo/q1tsim pass 12/12; quantrs2 passes 10/12.

Preprint (with the formal proof): Neutral Cross-Framework Benchmarking Reveals Rz Sign Convention Divergence in Quantum Simulation Backends | Zenodo
Source + cforge certify CLI: GitHub - cleitonaugusto/CleitonForge: Neutral cross-backend benchmarking for quantum circuit simulators. Found a silent Rz convention bug that breaks QAOA fidelity to zero. · GitHub

I’d genuinely love to run QGCS against PennyLane’s statevector device — do you know if PennyLane’s qml.device("default.qubit") follows the OpenQASM 3 convention Rz(λ) = diag(e^{-iλ/2}, e^{+iλ/2})? A quick sanity check:

import pennylane as qml
import numpy as np

dev = qml.device("default.qubit", wires=1)

@qml.qnode(dev)
def rz_witness():
    qml.Hadamard(0)
    qml.RZ(np.pi / 2, wires=0)
    return qml.state()

sv = rz_witness()
ratio = sv[1] / sv[0]
print(ratio)  # +i → OpenQASM 3 ✅ | −i → inverted ❌

If it prints +0.000+1.000j, PennyLane is compliant. Would be great to add it to the conformance table.

4

Thanks for the clarification @cleiton_augusto .

I can see in our docs for RZ that we do follow that convention.

When I run your code I get 2.2204460492503136e-16+1j , so it again confirms that we follow the convention.

That being said, this doesn’t necessarily mean that we follow all OpenQASM 3 conventions. Remember to check the docs if things seem off for other gates or other situations.

I hope this helps!