Add a tool for measuring expectation values of Pauli strings with readout error mitigation by ddddddanni · Pull Request #7067 · quantumlib/Cirq

ddddddanni · 2025-02-14T00:41:18Z

This PR does the following two things:

Improved Readout Benchmarking: Modified run_shuffled_with_readout_benchmarking to accept a list of qubit tuples (List[Tuple[Qubit]]). This allows for calculating the readout error rate on specific qubit subsets, ensuring alignment with the qubit sets used in non-identity Pauli operators within Pauli strings.
Added a tool to compute expectation values for Pauli operators. This tool calculates both unmitigated and readout-error-mitigated expectation values for each circuit, and integrates the results from the readout benchmarking.

…of qubits as input

… readout error mitigation

…CalibrationResult. Besides, fixed some lints

ddddddanni · 2025-02-14T00:54:32Z

Hey @eliottrosenberg and @NoureldinYosri if you could take a look at this. Thanks!

eliottrosenberg

Very nice work Danni!! This looks really great!

A few initial comments:

Regarding line 55 in shuffle_circuits_with_readout_benchmarking.py, it should be possible to specify 0 for num_random_bitstrings in order to do the measurement without readout mitigation (or have a different way of running without readout mitigation).

I think you went with lists of tuples for circuits_to_pauli because you can't put a cirq.Circuit into a dictionary, but you can put a cirq.FrozenCircuit into a dictionary, so maybe that would be more convenient (not sure). Also, the object that measure_pauli_strings outputs is pretty complicated, with lots of nested tuples, so maybe we want to create some data class for the output?

…_random_bitstrings, and add a test to cover the situation. The design is the run_shuffled_with_readout_benchmarking method will return a empty SingleQubitReadoutCalibrationResult if no calibration is actually done. 2. Allow measure_pauli_strings to take num_random_bitstrings = 0. In this case, no mitigation is actually done, and the mitigated result and unmitigated result are the same. Also add a test to handle this situation. 3. Make the return type of measure_pauli_strings a data class.

ddddddanni · 2025-02-18T07:42:33Z

Very nice work Danni!! This looks really great!

A few initial comments:

Regarding line 55 in shuffle_circuits_with_readout_benchmarking.py, it should be possible to specify 0 for num_random_bitstrings in order to do the measurement without readout mitigation (or have a different way of running without readout mitigation).

I think you went with lists of tuples for circuits_to_pauli because you can't put a cirq.Circuit into a dictionary, but you can put a cirq.FrozenCircuit into a dictionary, so maybe that would be more convenient (not sure). Also, the object that measure_pauli_strings outputs is pretty complicated, with lots of nested tuples, so maybe we want to create some data class for the output?

Done.
In the new commit, I allow shuffle_circuits_with_readout_benchmarking to take 0 for num_random_bitstrings, and add a test to cover the situation. The design is the run_shuffled_with_readout_benchmarking method will return a empty SingleQubitReadoutCalibrationResult if no calibration is actually done.
I also allow measure_pauli_strings to take num_random_bitstrings = 0. In this case, no mitigation is actually done, and the mitigated result and unmitigated result are the same. Also add a test to handle this situation. (not sure if we really want to have this situation supported?)
Besides, I make the return type of measure_pauli_strings a data class CircuitToPauliStringsMeasurementResult. Thanks for the suggestion, the code is more readable with this change.

eliottrosenberg

Thanks, Danni!! I tried testing it in this colab and got the error shown there.

ddddddanni · 2025-02-24T11:01:09Z

Thanks, Danni!! I tried testing it in this colab and got the error shown there.

Can you try again? The problem should be fixed in the latest commit ;)

eliottrosenberg

Thanks, Danni! I tested it again, and there seems to be a bug in the way it is computing the expectation value. I tried it on a simulator without noise in a case where the expectation value should be 1 and got something close to 0. https://colab.sandbox.google.com/drive/1_on4xIHQNMH_2km3RjlfoZxBcYim9A3y

Let me know if you have any trouble identifying the cause of this and I can step in and help.

eliottrosenberg · 2025-02-26T04:18:39Z

@ddddddanni Here is a simpler example that illustrates the bug: https://colab.sandbox.google.com/drive/1DUOEbrJSHIvlIN1CI7T6lug4zGDB5nyG

ddddddanni · 2025-02-26T10:37:55Z

@ddddddanni Here is a simpler example that illustrates the bug: https://colab.sandbox.google.com/drive/1DUOEbrJSHIvlIN1CI7T6lug4zGDB5nyG

Thanks for catching this! This is due to the code doesn't cover the situation that some pauli strings could be pauli I. I will fix it.

NoureldinYosri

@ddddddanni LGTM ... I will take another look after you fix the failing CIs

NoureldinYosri · 2025-02-26T20:06:55Z

+from cirq.study import ResultDict
+
+
+@dataclasses.dataclass


prefer to use attrs over dataclasses

NoureldinYosri · 2025-02-26T20:08:37Z

+        elif pauli_op == ops.Y:
+            # Rotate to Y basis: Rx(pi/2)
+            operations.append(ops.rx(np.pi / 2)(qid_list[qubit_index]))
+        # No operation needed for Pauli Z or I (identity)


there shouldn't be identity in this case

NoureldinYosri · 2025-02-26T20:16:01Z

+    # Extract unique qubit tuples from all circuits
+    unique_qubit_tuples = set()
+    for circuit in circuits_to_pauli.keys():
+        unique_qubit_tuples.add(tuple(sorted(set(circuit.all_qubits()))))


circuit.all_qubits() returns a frozenset, so you can dop the set here and elsewhere

ddddddanni · 2025-03-02T12:51:37Z

@ddddddanni Here is a simpler example that illustrates the bug: https://colab.sandbox.google.com/drive/1DUOEbrJSHIvlIN1CI7T6lug4zGDB5nyG

Done! Can you try test again? The issue was Pauli I was incorrectly treated as Z in expectation calculation, and now the issue is fixed, and I also modified tests to test this.

eliottrosenberg

Thanks, @ddddddanni! This is really great! One more request: can we multiply the result of measuring the PauliString by its coefficient? For example, right now your code does not distinguish between the following PauliStrings:

cirq.PauliString(cirq.Z.on_each(cirq.LineQubit.range(2)))

and

-1 * cirq.PauliString(cirq.Z.on_each(cirq.LineQubit.range(2)))

eliottrosenberg · 2025-03-05T01:15:49Z

(And when you implement my previous suggestion, make sure to also scale the statistical uncertainty by abs(coefficient))

…ng by its coefficient 2. Added a function to validate the input. Also added some tests for the validation function. 3. Fixed lint

ddddddanni · 2025-03-10T04:46:06Z

Thanks, @ddddddanni! This is really great! One more request: can we multiply the result of measuring the PauliString by its coefficient? For example, right now your code does not distinguish between the following PauliStrings:
cirq.PauliString(cirq.Z.on_each(cirq.LineQubit.range(2)))
and
-1 * cirq.PauliString(cirq.Z.on_each(cirq.LineQubit.range(2)))

Sure! I added some codes to multiple the coefficient with the result of measuring the paulistring. Also added some more codes to validate the input and tests.

eliottrosenberg

Thanks, @ddddddanni, this is great!

…o validate the input pauli coefficient must be real.

ddddddanni · 2025-03-12T13:09:26Z

@ddddddanni LGTM ... I will take another look after you fix the failing CIs

Thanks! Failing checks should be fixed now ;)

codecov · 2025-03-12T16:48:45Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.14%. Comparing base (eddb281) to head (1af26c2).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #7067    +/-   ##
========================================
  Coverage   98.14%   98.14%            
========================================
  Files        1098     1100     +2     
  Lines       95816    96186   +370     
========================================
+ Hits        94034    94403   +369     
- Misses       1782     1783     +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

NoureldinYosri

LGTM%style comments

NoureldinYosri · 2025-03-12T16:36:02Z

+from cirq.study import ResultDict
+
+
+@attrs.define


if this is intended to be an immutable class prefere @attrs.frozen else prefer @attrs.mutable

Sure, changed it to @attrs.frozen

NoureldinYosri · 2025-03-12T16:36:36Z

+    mitigated_stddev: float
+    unmitigated_expectation: float
+    unmitigated_stddev: float
+    calibration_result: Optional[SingleQubitReadoutCalibrationResult]


since this is optional, it should have a default value = None

NoureldinYosri · 2025-03-12T16:36:49Z

+    calibration_result: Optional[SingleQubitReadoutCalibrationResult]
+
+
+@attrs.define


NoureldinYosri · 2025-03-12T16:37:35Z

+
+def _validate_input(
+    circuits_to_pauli: Dict[circuits.FrozenCircuit, list[ops.PauliString]],
+    rng_or_seed: Union[np.random.Generator, int],


in most scientific libraries, the rng comes last

Sure! Moved this to the end.

NoureldinYosri · 2025-03-12T17:14:00Z

+        if qubit in pauli_string:
+            pauli_op = pauli_string[qubit]
+            if pauli_op == ops.X:
+                operations.append(ops.ry(-np.pi / 2)(qubit))


can you add a comment that this is hadamard?

Suggested change

operations.append(ops.ry(-np.pi / 2)(qubit))

operations.append(ops.ry(-np.pi / 2)(qubit)) # =cirq.H

NoureldinYosri · 2025-03-12T17:19:16Z

+    """Builds a list of empty confusion matrices"""
+    cms: list[np.ndarray] = []
+    for _ in range(qubits_length):
+        cms.append(_build_one_qubit_confusion_matrix(0, 0))


this is is not an empty matrix ... if the intention is to create a placeholder list of np.array then you can do

return [np.empty(0)]*qubits_length

if you need them to be 2x2 array, then

return [np.empty((2,2))]*qubits_length

Sure! I changed the method to directly return the [_build_one_qubit_confusion_matrix(0, 0) for _ in range(qubits_length)]

NoureldinYosri · 2025-03-12T17:20:18Z

+
+    Returns:
+        A list of PauliStringMeasurementResult objects, where each object contains:
+            - The Pauli string that was measured.


I think this is redundant since this description should be in the docstring of PauliStringMeasurementResult

Sure! Changed.

NoureldinYosri · 2025-03-12T17:20:59Z

+def measure_pauli_strings(
+    circuits_to_pauli: Dict[circuits.FrozenCircuit, list[ops.PauliString]],
+    sampler: work.Sampler,
+    rng_or_seed: Union[np.random.Generator, int],


NoureldinYosri · 2025-03-12T17:24:29Z

+    readout_qubits = [qubits_1, qubits_2]
+
+    # Generate random input circuits and append measurements
+    input_circuit_1 = rqcg.generate_library_of_2q_circuits(


I think we should create a method to create the input_circuits and use it here and elsewhere

input_circuits = _create_test_circuits()

NoureldinYosri · 2025-03-12T17:26:48Z

+    circuit_3 = cirq.FrozenCircuit(_create_ghz(8, qubits_3))
+
+    circuits_to_pauli: Dict[cirq.FrozenCircuit, list[cirq.PauliString]] = {}
+    circuits_to_pauli[circuit_1] = [


I know about the tradeoff between damp and dry tests and to me this feels a bit too damp ... maybe

circuits_to_pauli[circuit_1] = [_generate_random_pauli_string(qubits_1, True) for _ in range(3)]

NoureldinYosri · 2025-03-17T17:53:28Z

great work @ddddddanni ... please fix the coverage CI and then we can merge this PR

ddddddanni · 2025-03-18T23:48:19Z

great work @ddddddanni ... please fix the coverage CI and then we can merge this PR

Thanks!! The coverage check should be fixed now ; )

@NoureldinYosri

…dout error mitigation (quantumlib#7067) * Modify the shuffle_circuit measurement and allow it to handle a list of qubits as input * Add a new tool for measuring expectation values of Pauli strings with readout error mitigation * Allow the measure_pauli_strings to also return the SingleQubitReadoutCalibrationResult. Besides, fixed some lints * 1. Allow shuffle_circuits_with_readout_benchmarking to take 0 for num_random_bitstrings, and add a test to cover the situation. The design is the run_shuffled_with_readout_benchmarking method will return a empty SingleQubitReadoutCalibrationResult if no calibration is actually done. 2. Allow measure_pauli_strings to take num_random_bitstrings = 0. In this case, no mitigation is actually done, and the mitigated result and unmitigated result are the same. Also add a test to handle this situation. 3. Make the return type of measure_pauli_strings a data class. * Fix a issue that caused calibration_results lookup failure * Fix: Pauli I was incorrectly treated as Z in expectation calculation * 1. Added some codes to multiply the result of measuring the PauliString by its coefficient 2. Added a function to validate the input. Also added some tests for the validation function. 3. Fixed lint * Fix a broken test * pauli_string.qubits returns the self.keys which are already unique. Thus remove the set() acts on the pauli_string.qubits * Fix type and coverage check. Besides, adds a input validation check to validate the input pauli coefficient must be real. * Address comments by @NoureldinYosri * Fix the coverage check * Fix lint line too long --------- Co-authored-by: eliottrosenberg <61400172+eliottrosenberg@users.noreply.github.com> Co-authored-by: Noureldin <noureldinyosri@google.com>

ddddddanni added 2 commits February 13, 2025 14:46

Modify the shuffle_circuit measurement and allow it to handle a list …

18f99be

…of qubits as input

Add a new tool for measuring expectation values of Pauli strings with…

d3126c8

… readout error mitigation

ddddddanni requested review from a team and vtomole as code owners February 14, 2025 00:41

ddddddanni requested a review from tanujkhattar February 14, 2025 00:41

CirqBot added the size: L 250< lines changed <1000 label Feb 14, 2025

ddddddanni added 2 commits February 13, 2025 16:41

Merge branch 'main' into pauli-string-readout

b66caa6

Allow the measure_pauli_strings to also return the SingleQubitReadout…

b6e645e

…CalibrationResult. Besides, fixed some lints

NoureldinYosri self-requested a review February 14, 2025 01:46

NoureldinYosri self-assigned this Feb 14, 2025

eliottrosenberg requested changes Feb 15, 2025

View reviewed changes

ddddddanni requested a review from eliottrosenberg February 18, 2025 07:43

Merge branch 'main' into pauli-string-readout

3765833

eliottrosenberg requested changes Feb 19, 2025

View reviewed changes

Fix a issue that caused calibration_results lookup failure

17cf34d

ddddddanni requested a review from eliottrosenberg February 24, 2025 11:01

eliottrosenberg requested changes Feb 26, 2025

View reviewed changes

NoureldinYosri reviewed Feb 26, 2025

View reviewed changes

Fix: Pauli I was incorrectly treated as Z in expectation calculation

d342fef

ddddddanni requested a review from eliottrosenberg March 2, 2025 12:51

eliottrosenberg requested changes Mar 4, 2025

View reviewed changes

1. Added some codes to multiply the result of measuring the PauliStri…

55adfc3

…ng by its coefficient 2. Added a function to validate the input. Also added some tests for the validation function. 3. Fixed lint

ddddddanni requested a review from eliottrosenberg March 10, 2025 04:46

Merge branch 'main' into pauli-string-readout

7a60050

eliottrosenberg approved these changes Mar 12, 2025

View reviewed changes

Fix type and coverage check. Besides, adds a input validation check t…

e8c448c

…o validate the input pauli coefficient must be real.

ddddddanni requested a review from NoureldinYosri March 12, 2025 13:09

NoureldinYosri reviewed Mar 12, 2025

View reviewed changes

Address comments by @NoureldinYosri

fd16e0e

ddddddanni requested a review from NoureldinYosri March 17, 2025 06:15

NoureldinYosri approved these changes Mar 17, 2025

View reviewed changes

Fix the coverage check

a97f613

ddddddanni requested a review from NoureldinYosri March 18, 2025 23:48

ddddddanni and others added 4 commits March 18, 2025 16:51

Fix lint line too long

ff27552

Merge branch 'main' into pauli-string-readout

9b8431e

Merge branch 'main' into pauli-string-readout

2eda743

Merge branch 'main' into pauli-string-readout

1af26c2

NoureldinYosri enabled auto-merge March 25, 2025 00:07

NoureldinYosri added this pull request to the merge queue Mar 25, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 25, 2025

NoureldinYosri added this pull request to the merge queue Mar 25, 2025

Merged via the queue into quantumlib:main with commit b3561b4 Mar 25, 2025
38 checks passed

ddddddanni mentioned this pull request Apr 9, 2025

Add the capability to accept group Paulis that can be measured using same measurement results #7236

Merged

mhucka mentioned this pull request Apr 18, 2025

Improve reporting of errors & efficiency of pr-size-labeler #7288

Merged

ddddddanni mentioned this pull request May 20, 2025

Add parameter sweep support for Pauli string measurements with readout mitigation #7358

Closed

		calibration_result: Optional[SingleQubitReadoutCalibrationResult]


		@attrs.define

	operations.append(ops.ry(-np.pi / 2)(qubit))
	operations.append(ops.ry(-np.pi / 2)(qubit)) # =cirq.H

Conversation

ddddddanni commented Feb 14, 2025

Uh oh!

ddddddanni commented Feb 14, 2025

Uh oh!

eliottrosenberg left a comment

Choose a reason for hiding this comment

Uh oh!

ddddddanni commented Feb 18, 2025

Uh oh!

eliottrosenberg left a comment

Choose a reason for hiding this comment

Uh oh!

ddddddanni commented Feb 24, 2025

Uh oh!

eliottrosenberg left a comment

Choose a reason for hiding this comment

Uh oh!

eliottrosenberg commented Feb 26, 2025

Uh oh!

ddddddanni commented Feb 26, 2025

Uh oh!

NoureldinYosri left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ddddddanni commented Mar 2, 2025

Uh oh!

eliottrosenberg left a comment

Choose a reason for hiding this comment

Uh oh!

eliottrosenberg commented Mar 5, 2025

Uh oh!

ddddddanni commented Mar 10, 2025

Uh oh!

eliottrosenberg left a comment

Choose a reason for hiding this comment

Uh oh!

ddddddanni commented Mar 12, 2025

Uh oh!

codecov bot commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

NoureldinYosri left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

NoureldinYosri left a comment •

edited

Loading

codecov bot commented Mar 12, 2025 •

edited

Loading