Skip to main content

Defining the QuOp: part I

Discussion

Group discussion, 15:35–16:30, Wednesday 4 November 2026

Questions for discussion

These questions frame the afternoon's group discussion. They are grouped by theme, they are open, and we do not expect to settle them all on the day: the aim is to find out where there is already consensus, where there is not, and what part II of the series should take up.

Theme 1

Purpose and what is being measured

  1. Should the QuOp be used to judge whether a device is suited to a particular use case? For example, if an end user wants to run a materials-science application, should a QuOp count give a useful indication of how much quantum compute that application needs?
  2. Should the QuOp be platform-independent? That is, should a definition that works for neutral atoms apply equally to superconducting qubits, trapped ions, photonics or any other modality?
  3. Who is the QuOp for: funders and policymakers tracking national milestones, end users choosing hardware, or developers measuring their own progress? Does the answer change the definition?
  4. What would a machine have to demonstrate to be credibly called MegaQuOp-, GigaQuOp- or TeraQuOp-capable, and should the QuOp measure what a device has actually run or what it is projected to be able to run?

Theme 2

Operation counting

  1. Which operations should count as "quantum operations", and what gate set should be assumed? QUOPS, for example, assumes single-qubit rotations and CNOT gates.
  2. How much weight should different operations carry? QUOPS, for example, counts a single-qubit Pauli rotation as one operation and a CNOT as two.
  3. Is it necessary to abstract away from discrete gate sets such as Clifford+T to keep comparisons fair between devices with no error correction, partial error correction and full fault tolerance? Or do comparisons remain fair if the QuOp counts, say, T gates?
  4. Should mid-circuit measurements, resets and feed-forward operations count, and if so, how?
  5. Should the QuOp be defined by the most demanding operation in the available instruction set, such as distilled T or CCZ gates (or another non-Clifford gate) in a fault-tolerant architecture?
  6. Does a QuOp count refer to the size of a single computation that runs reliably, or can operations accumulate across many shots and repetitions?
  7. Should operations applied in parallel count the same as operations applied in sequence, and should idling qubits, which still accumulate errors, contribute to the count?

Theme 3

Circuits and applications

  1. Is one set of circuits enough, or would it be better to have several (say five) QuOp scores, each probing performance in a different regime? QUOPS, for example, uses dense, highly connected circuits, which may not be representative of many applications. Additional regimes could include:
    1. low connectivity (for example, nearest-neighbour interactions only)
    2. low gate density
    3. Toffoli-heavy circuits
    4. application-specific circuits (for example, quantum simulation)
  2. Should uncertainty about the shape of future application circuits be handled by allowing different shapes under a single score, or by reporting separate scores for different shapes? Does the question go away with application-specific QuOps, and is that a feature or a bug of application-specific QuOps?
  3. How important is it that the QuOp can compare digital and analogue quantum computers? Are a couple of application-specific circuits (for example, Trotterised quantum simulation or QAOA) enough to do this, if they are still scored by counting the operations in the reference digital implementation?
  4. How should the QuOp apply to measurement-based quantum computing (MBQC)?
  5. Should the reference tasks use only classically hard circuits? What about quantum simulation with symmetries, or Clifford circuits with only a few T gates, which may be classically tractable?
  6. Should the reference circuits be fixed and published in advance, or drawn at random at test time so that devices cannot be tuned to them?
  7. How should the reference circuits scale as devices grow, and can the benchmark still be run once they are too large to simulate classically?

Theme 4

Accuracy

  1. Are randomised mirror circuits sufficient to check that a quantum computer has executed a circuit correctly?
  2. Are certification-based verification approaches, such as the verification-inspired benchmarking presented by Elham Kashefi, sufficient for testing accuracy?
  3. Should pass/fail checks use a single success threshold, or should results be reported at several thresholds?
  4. If discrete gate sets such as Clifford+T are used, how should the error from approximate gate synthesis be taken into account? A mirror circuit may pass even with low-precision synthesis, because both halves of the mirror share the same imprecision, and verification-based approaches may have the same blind spot. Should a different synthesis be enforced for each gate?
  5. Should some reference circuits have classically verifiable outputs, instead of relying on verification or mirror circuits?
  6. What other verification methods should be considered for the fault-tolerant regime?
  7. How many repetitions are needed to establish a score with statistical confidence, and should error bars be part of the reported result?
  8. Should accuracy be judged on average over many circuits or in the worst case, and how should rare but catastrophic failures be treated?

Theme 5

Speed

  1. Should speed simply be the total computation (in QuOps) divided by the total wall-clock time?
  2. How can we accurately characterise decoder response time and latency, and the throughput requirements they place on the system?
  3. What should count towards the wall-clock time: compilation, calibration, queueing, classical pre- and post-processing, repeated shots?
  4. Should size and speed be combined into a single figure or reported separately, so that a small, fast machine is not confused with a large, slow one?

Theme 6

Rules and optimisation

  1. What compilation is allowed? Is approximate compilation allowed? What about using additional ancilla qubits?
  2. Should error mitigation, or other classical post-processing that improves the results, be allowed?
  3. Who should define, run and audit the benchmark: hardware developers themselves, a national measurement institute, or another independent third party?
  4. How can the rules stay stable enough to compare results over time, yet evolve as hardware and algorithms improve?

Theme 7

Reporting

  1. High-performance computing distinguishes measured FLOP/s from a theoretical peak FLOP/s that ignores data movement, latency and so on. Should there be a similar distinction for QuOps?
  2. "quops" or "QuOps"? How should the term be written and pronounced?
  3. What must be reported alongside a QuOp score? For example, scores for different applications or reference tasks, and speed.
  4. Should measures of overhead, such as the ratio of physical to logical qubits or heralding and post-selection rates, be reported?
  5. Should results be published in a common, machine-readable format, together with the circuits, raw data and analysis code, so that others can reproduce them?
  6. How should a definition of the QuOp relate to the quantum computing benchmark standards being developed internationally?