# Parameter-shift rules **URL:** \ **Category:** PennyLane Help\ **Created:** [July 26, 2020, 10:24pm UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496 "2020-07-26T22:24:21Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![amir](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@amir](https://discuss.pennylane.ai/u/amir)\ **Post date:** [July 26, 2020, 10:24pm UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496/1 "2020-07-26T22:24:21Z")
In the documentation for “[Parameter-shift rules](https://pennylane.ai/qml/glossary/parameter_shift.html)” it talks about constructing partial derivatives by evaluating the same circuit with different parameters: ![](https://canada1.discourse-cdn.com/flex012/uploads/pennylane/original/1X/88d59239ba8e7df6f30d21566ae927fc6574d20b.png) How is this any different than numerical differentiation though, mentioned [here](https://pennylane.ai/qml/glossary/automatic_differentiation.html)? I see one mention towards this on the page: “While this construction bears some resemblance to the numerical finite-difference method for computing derivatives, here s is finite rather than infinitesimal.” What’s a concrete example of s (i.e. what’s the significance of calling it out as finite vs. infinitesimal)? Adding a little bit more to this as I read more through the documentation. The page on quantum gradient mentions: “This means that quantum gradients can be computed by quantum computers,” By _computed_ here does it mean that the gradient function itself is being output as the result of the computation or that simply the gradient is being computed on a quantum computer by calling the circuit multiple times with a parameter shift? ---
**Author:** ![josh](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/josh/32/100_2.png) [@josh](https://discuss.pennylane.ai/u/josh)\ **Post date:** [July 27, 2020, 4:11am UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496/2 "2020-07-27T04:11:14Z")
Hi @amir! > How is this any different than numerical differentiation though, mentioned [here](https://pennylane.ai/qml/glossary/automatic_differentiation.html)? The parameter-shift rule provides **exact** , **analytic** gradients, whereas numerical differentiation such as finite-difference provides only an **approximation** to the gradient. As an example, we can compute the parameter-shift gradient rule for a simple function such as f(x) = \sin(x), and compare this to finite-differences 🙂 We know that the gradient is given by: \begin{align} f'(x) = \cos(x) \end{align} By making use of the trig identity \begin{align} \cos(x) = \frac{\sin(x+s) - \sin(x - s)}{2\sin(s)} \end{align} we can now write the gradient as \begin{align} f'(x) = \frac{f(x+s) - f(x - s)}{2\sin(s)} \end{align} That is, the _exact gradient_ of f(x) is given by evaluating the function at the points x+s and x-s, and computing a linear combination. This is exact **for any value of s**. Compare this instead to [finite-differences](https://en.wikipedia.org/wiki/Finite_difference): \begin{align} \cos(x) = \frac{\sin(x+h) - \sin(x - h)}{h} +\mathcal{O}(h^2) \end{align} This is an **approximation** that is only valid for h\ll 1. Furthermore, numerical differentiation is prone to numerical instability, unlike the parameter-shift rule which is exact. So as you can see from the example, the parameter-shift rule takes into account structural information about f(x) to allow the exact gradient of f(x) to be computed by simply taking additional function evaluations. However, this requires f(x) to satisfy particular conditions, so is not universal! > What’s a concrete example of s (i.e. what’s the significance of calling it out as finite vs. infinitesimal)? Within PennyLane, we typically take s=\pi/2. It is advantageous to maximise the shift on near-term noisy quantum devices, as this allows us to compute expectation values further apart in parameter-space and to limit the effects of shot-noise. > simply the gradient is being computed on a quantum computer by calling the circuit multiple times with a parameter shift? Yes this is exactly it! ---
**Author:** ![amir](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@amir](https://discuss.pennylane.ai/u/amir)\ **Post date:** [July 27, 2020, 4:21am UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496/3 "2020-07-27T04:21:59Z")
Thanks for your comprehensive answer. One note: Should \begin{align} f'(x) = \frac{f(x+s) - f(x - s)}{2\sin(s)} \end{align} actually be: \begin{align} f'(x) = \frac{f(x+s) - f(x - s)}{2f(s)} \end{align} ? Follow-up question - is _symbolic_ differentiation the same as what you provide with the exact, analytic gradient in the example above? Also, do you think it might be helpful to others if I updated the documentation with a PR including what you’ve mentioned? ---
**Author:** ![josh](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/josh/32/100_2.png) [@josh](https://discuss.pennylane.ai/u/josh)\ **Post date:** [July 27, 2020, 5:16am UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496/4 "2020-07-27T05:16:01Z")
To answer your first question, no — while both equations are correct, I deliberately left the denominator as \sin(s) as it is independent of x. In general, we want to limit the number of calls to our function f(x), as it might be expensive to compute. In this case, we only need two evaluations of f to compute the gradient; the denominator \sin(s) can be easily and efficiently computed independently 🙂 > Follow-up question - is _symbolic_ differentiation the same as what you provide with the exact, analytic gradient in the example above? No, symbolic differentiation is different yet again! We have three ways to compute derivatives: - **Numerical differentiation:** Numerical approximations to the gradient that can be used with arbitrary functions, for example finite-differences. Can be unstable, and can introduce floating point error. - **Automatic differentiation:** An exact, numerical approach to computing derivatives. Each component of the computation provides a rule for its derivative with respect to the input. At the end of the computation, the chain rule is used to combine all these gradient rules and determine the total gradient. - **Symbolic differentiation:** Manipulates mathematical expressions directly to determine the gradient rule/function. This is similar to how you would compute the gradient manually by hand, or using a computer algebra system such as Mathematica. For more explanation, this Stack Overflow question is quite helpful: [Difference between symbolic differentiation and automatic differentiation? - Stack Overflow](https://stackoverflow.com/questions/43455320/difference-between-symbolic-differentiation-and-automatic-differentiation) > Also, do you think it might be helpful to others if I updated the documentation with a PR including what you’ve mentioned? For sure! PRs adding improvements to our documentation are always encouraged 🙂 ---
**Author:** ![amir](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@amir](https://discuss.pennylane.ai/u/amir)\ **Post date:** [July 29, 2020, 11:30pm UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496/5 "2020-07-29T23:30:46Z")
Okay, so I’m seeking for a little more precision with the definition here. In your original response you said that you’re employing “exact, **analytic** gradients”. With the three ways to compute derivatives the one that is getting used by pennylane is automatic differentiation, which says it is an “exact, **numerical** approach”. So, is it numerical or analytic? When I read numerical I think numerical differentiation. When I read analytic I think symbolic differentiation. Something I’m intuiting though is that automatic differentiation is this sweet spot between both. One more thought on this, too. Would it be possible hypothetically to go double-wide on the circuit and calculate both f(x+s) and f(x - s) and take the expectation value of that difference in a single circuit, which would reduce circuit calls by half? ---
**Author:** ![jmarrazola](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/jmarrazola/32/255_2.png) [@jmarrazola](https://discuss.pennylane.ai/u/jmarrazola)\ **Post date:** [July 30, 2020, 6:13pm UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496/6 "2020-07-30T18:13:07Z")
Hi Amir, Yes, the use of the word “analytic” may be a bit confusing. Let me try to clarify. One method to compute gradients is finite difference. Here we approximate the gradient of a function for example as \frac{d f(x)}{dx}\approx \frac{f(x+\epsilon)-f(x-\epsilon)}{2\epsilon}. I believe this is what Josh meant by numerical differentiation. Note that it gives an approximation of the derivative, whose quality depends on \epsilon. That’s why it can be unstable. Another method is the parameter-shift rule. The idea here is to express the gradient as a linear combination of the function evaluated at two separate points. Consider f(x)=\sin(x). Its derivative, \cos(x), can be written as \cos(x) = \frac{\sin(x+\pi/4)-\sin(x-\pi/4)}{\sqrt{2}}, so in this case we have \frac{d f(x)}{dx}=\frac{f(x+\pi/4)-f(x-\pi/4)}{\sqrt{2}}. Notice that the “shift” from x is not tiny, it’s \pi/4 and the equation is exact, i.e., it is not an approximation. A similar trick can be used for gradients of quantum gates. This is what Josh meant regarding exact, analytic gradients. Finally, symbolic differentiation is just that, it’s literally writing the derivative as a mathematical formula. To answer your final question, typically you can’t do this. If x is a parameter of the circuit, then you need to change the parameter to x+s and x-s separately, and individually calculate expectation values. The only exception I can think of is if you can “absorb” the shift into an observable. So if you can define an observable O\_1 whose expectation value is f(x+s) and another observable O\_2 whose expectation value is f(x-s), where the expectation is taken with respect to _the circuit in the same configuration_, then from linearity you can define the observable O\_3 = O\_1-O\_2 whose expectation value is f(x+s)-f(x-s), which you can calculate or estimate directly. I’m not sure this is even possible. Hope that helps! ---
**Author:** ![nathan](https://yyz2.discourse-cdn.com/flex012/user_avatar/discuss.pennylane.ai/nathan/32/85_2.png) [@nathan](https://discuss.pennylane.ai/u/nathan)\ **Post date:** [July 30, 2020, 6:29pm UTC](https://discuss.pennylane.ai/t/parameter-shift-rules/496/7 "2020-07-30T18:29:23Z")
Hi @amir Just to add on what @jmarrazola mentioned, you _could_ take a circuit on N qubits, and run two copies of it on a “double wide” set of 2N qubits “in parallel” (i.e., never interacting between the first N qubits and the last N qubits). You can effectively think of that as having two independent QPUs. However, it’s a lot cheaper to double the number of runs than to double the number of qubits, so this strategy is not very resource efficient 🙂