Live data from Hacker News

Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

zenodo.org

1–8 of 8 posts

Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

#1
Author here.

Background, for anyone who hasn't followed this fight: IBM's iron–sulfur SQD results (Sci. Adv. 2025) are one of the flagship "quantum computers are useful for chemistry now" claims, and a published critique (arXiv:2501.07231) argues the quantum samples never beat classical selected-CI at matched cost. That argument is still live.

Both sides have been arguing about energies. Neither measured which electronic state these calculations actually converge to.

So I measured it. ⟨S²⟩ comes out between 4.7 and 7.0 depending on the system and the subspace size, where the target these papers name is a singlet at ⟨S²⟩ = 0. Every starting guess I tried lands in the same place, and the error at convergence is about the size of the spin-state ladder itself, which is the physics under dispute.

IBM ships a mitigation for this. Run exactly as shipped, using their driver, their recovery loop and their own spin_square() diagnostic, it moves the ground energy by under a nanohartree while making the subspace 4.00x bigger: 194,481 determinants against 48,600. It makes a singlet representable. It never produces one. Their solver also takes a spin_sq argument that would target the singlet directly, and the default pipeline never sets it.

I filed that narrow part on their tracker yesterday, before posting this: https://github.com/Qiskit/qiskit-addon-sqd/issues/337. A maintainer answered and closed it the same day, and his answer is the useful part. The flag, in his words, augments the sampled subspace by using alpha CI strings as beta strings and vice versa. That is a statement about which determinants span the space, not about what spin the returned state comes out in, which is exactly what the measurement says. He did not contest the numbers. The second question, whether the default path is meant to reach spin_sq at all, is still unanswered.

The obvious objection is that this is all my own reimplementation, so here's the part that isn't. IBM's data-availability archive for the flagship paper contains the raw hardware measurement records: 2,457,600 shots on [2Fe-2S] from December 2023, 3,163,742 outcomes on [4Fe-4S] from April 2024, plus the integrals and the optimized circuit's parameters. Running their shots through their own pipeline, [2Fe-2S] reaches low ⟨S²⟩ but sits 248 mHa off their own reference, and [4Fe-4S] converges to a spin-pure triplet, Var(S²) = 3e-6, a genuine S=1 eigenstate: a clean state, and the wrong one, 1,438 mHa from their reference.

Two things in that archive need no analysis from me at all. Their uniform-random null control matches or beats the hardware samples in every published [2Fe-2S] comparison. And their largest [4Fe-4S] runs, at subspace dimension 10⁸, trail their own classical HCI file by 149 mHa.

On the AI angle, since that's half of why this is on HN: Claude did the initial audit end to end in about 72 hours under my direction, and it's been through many rounds of adversarial review since. The part I'd defend as actually interesting isn't the speed. It caught five defects in its own work through pre-registered validation gates, one of them by cross-checking against IBM's own published energy tables, and it retracted its own strongest pro-quantum finding when the new instrument showed that result was a spin-sector artifact. AUDIT_TRAIL.md has the timeline. REPRO_MAP.md maps every claim to a file and a command that regenerates it.

If you want to kill this, and I mean that, here's how. Exhibit any state in a spin-completed ground manifold of these benchmarks with ⟨S²⟩ Preprint: https://doi.org/10.26434/chemrxiv.15006382/v1

The limitations section is real: comparison dimensions are fixed, the [4Fe-4S] reference is approximate, and the 58.6M "samples" file is deduplicated, so the shot-level distribution of their optimal-circuit numerics isn't publicly auditable. It's short. Worth reading before the hot take.

Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters
zenodo.org

Re: Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

#3
Claude slop detected. Exhibits: 1. "That argument is still live." 2. "It makes a singlet representable. It never produces one." 3. "...and the default pipeline never sets it." 4. "...so here's the part that isn't." 5. "If you want to kill this, and I mean that, here's how." 6. "The limitations section is real:..."

Re: Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

#4
Sorry for asking, but I've seen too much Physics-AI-slop lately. This is close to my research area, and it looks interesting but I have not read it completely.

1) Do you have a degree in physics, computer science or something?

2) Can you answer without using a computer which are the eigenvalues of the 2x2 matrix [[0, 1], [1, 0]]?

Re: Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

#5

Sorry for asking, but I've seen too much Physics-AI-slop lately. This is close to my research area, and it looks interesting but I have not read it completely. 1) Do you have a degree in physics, computer science or something? 2) Can you answer without using a computer which are the eigenvalues of the 2x2 matrix [[0, 1], [1, 0]]?

1) Not physics. I came at this from the computing side and taught myself the chemistry.

2) +1 and -1. Trace 0, determinant -1, so λ² - 1. Eigenvectors (1,1)/√2 and (1,-1)/√2.

Fair worry. Easier to run it than argue about it — two seconds in a browser, installs nothing:

https://colab.research.google.com/github/PureStateLabs/sqd-s...

It runs two archives through IBM's own pipeline. Spin completion quadruples the determinant count. The energy moves less than a nanohartree. ⟨S²⟩ doesn't move at all — sits at 4.83, target is 0.

Re: Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

#6
post #3

Claude slop detected. Exhibits: 1. "That argument is still live." 2. "It makes a singlet representable. It never produces one." 3. "...and the default pipeline never sets it." 4. "...so here's the part that isn't." 5. "If you want to kill this, and I mean that, here's how." 6. "The limitations section is real:..."

It's disclosed in the paper, not hidden. An AI agent ran the computations and tooling under my direction, and the disclosure says explicitly that agent-executed work should not be taken on trust. That's why every quantitative claim is machine-re-derived from the raw archives, 368 of them, and why the agent's own error record ships with the paper, including the finding it retracted.

The core result takes about two seconds to check in a browser, nothing to install:

https://colab.research.google.com/github/PureStateLabs/sqd-s...

IBM's archived data, IBM's pipeline. If a number is wrong I'll publish that.

Re: Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

#7

Sorry for asking, but I've seen too much Physics-AI-slop lately. This is close to my research area, and it looks interesting but I have not read it completely. 1) Do you have a degree in physics, computer science or something? 2) Can you answer without using a computer which are the eigenvalues of the 2x2 matrix [[0, 1], [1, 0]]?

1) Not physics. I came at this from the computing side and taught myself the chemistry. 2) +1 and -1. Trace 0, determinant -1, so λ² - 1. Eigenvectors (1,1)/√2 and (1,-1)/√2. Fair worry. Easier to run it than argue about it — two seconds in a browser, installs nothing: https://colab.research.google.com/github/PureStateLabs/sqd-s... It runs two archives through IBM's own pipeline. Spin completion quadruples the determ…

> 2) +1 and -1.

Good, good. Sorry, I was super busy today. I hope to read it tomorrow.

Re: Spin audit of SQD/QSCI quantum-chemistry benchmarks on iron–sulfur clusters

#8

Earlier quoted context omitted.

1) Not physics. I came at this from the computing side and taught myself the chemistry. 2) +1 and -1. Trace 0, determinant -1, so λ² - 1. Eigenvectors (1,1)/√2 and (1,-1)/√2. Fair worry. Easier to run it than argue about it — two seconds in a browser, installs nothing: https://colab.research.google.com/github/PureStateLabs/sqd-s... It runs two archives through IBM's own pipeline. Spin completion quadruples the determ…

> 2) +1 and -1. Good, good. Sorry, I was super busy today. I hope to read it tomorrow.

Hi! Sorry for the delay.

> 2) +1 and -1.

I like to ask eigenvalues of that matrix. It's relevant to QM and sadly it's a very high bar that is very useful to filter and try to get a good conversation.

My research area is similar, but we use smaller molecules, so I can't understand all the details here.

I took a look an it makes sense. I can't be sure if it's 100% correct, but it's not obviously wrong (and I've seen a lot of obviously wrong post in the /newest page, so I start assuming everything is nonsense until I take a look).

It took a while to understand that [2Fe-2S] is actually Fe2S6C4H12. I guess it's obvious for people working with more biology background. My handwaving is not good enough if the unpaired electron in the Fe couple to S=0 or S=1, but after fooling around in google it looks like S=0 is the correct answer.

Assuming your calculations are correct, they get =4.7 that is really a lot and shows it has contamination from S=2 or higher. I can't imagine how to get S=1 with the spin in both Fe in the same direction, but a higher spin confuses me, it may be related to some additional freedom using the d orbitals that is beyond my handwaving.

They claim:

> We study the ground-state properties of these molecular systems in the _ = 0 and ^2 = 0 subspace

So as you say they are trying to get a S^2=0 state, not only an energy bound. Some contamination is expected, but 4.7 is too much in my opinion.

You say:

> A caveat that our own third finding compels: at residual 3.5 × 10−4 in a manifold this dense, energy stability does not certify the eigenvector, so we did not stop at 1.287: the certification run below closes the point.

How many orbitals are near that energy? Do you have a graphic?

> Table 1: [2Fe-2S] followed sector-root energies and spin content vs subspace size D

It's super interesting. It looks like the better energy is caused by the spin contamination. This is not unusual or unexpected, but that's too much contamination in my opinion.

It's hard to suggest what to do with this. But perhaps a super short version (2 or 3 pages?) that get's directly to the results in table 1 may help.