AI hallucinations: Why LLMs make things up (and how to fix it)

> The trouble is that we software engineers have spent so long working in an artificially deterministic world that we're not used to designing and evaluating probabilistic quality control systems for computer output.

I think that's a mischaracterization and not really accurate. As a trade, we're familiar with probabilistic/non-deterministic components and how to approach them.

You were closer when you used quotes around "AI Engineer" -- many of the loudest people involved in generative AI right now have little to no grounding in engineering at all. They aren't used to looking at their work through "fit for purpose" concerns, compromises, efficiency, limits, constraints, etc -- whether that work uses AI or not.

The rest of us are variously either working quietly, getting drowned out, or patiently waiting for our respected colleagues-in-engineering to document, demonstrate, and mature these very promising tools for us.

Everything else you said is 100% right, though.

Do you have a link to (or can you put here) that "numbers in boxes" problem?

I refuse to believe that you did any of this with any of the latest models. Gemini and Chat GPT with search are both perfectly capable of producing decent essays with accurate citations. And the 4o model is extremely good at writing python code that can accurately solve math and logic problems.

I asked 4o with search to write an essay about the dangers of smoking, along with citations and quotes from the relevant sources. NotebookLM is even better if you drop in your sources and don't rely on web search. Whatever you think you know about what AI is capable of, it's probably wrong.

--- Smoking remains a leading cause of preventable disease and death worldwide, adversely affecting nearly every organ in the human body. The National Cancer Institute (NCI) reports that "cigarette smoking and exposure to tobacco smoke cause about 480,000 premature deaths each year in the United States."

The respiratory system is particularly vulnerable to the detrimental effects of smoking. The American Lung Association (ALA) states that smoking is the primary cause of lung cancer and chronic obstructive pulmonary disease (COPD), which includes emphysema and chronic bronchitis. The inhalation of tobacco smoke introduces carcinogens and toxins that damage lung tissue, leading to reduced lung function and increased susceptibility to infections.

Cardiovascular health is also significantly compromised by smoking. The ALA notes that smoking "harms nearly every organ in the body" and is a major cause of coronary heart disease and stroke. The chemicals in tobacco smoke damage blood vessels and the heart, increasing the risk of atherosclerosis and other cardiovascular conditions.

Beyond respiratory and cardiovascular diseases, smoking is linked to various cancers, including those of the mouth, throat, esophagus, pancreas, bladder, kidney, cervix, and stomach. The American Cancer Society (ACS) emphasizes that smoking and the use of other tobacco products "harms nearly every organ in your body." The carcinogens in tobacco smoke cause DNA damage, leading to uncontrolled cell growth and tumor formation.

Reproductive health is adversely affected by smoking as well. In women, smoking can lead to reduced fertility, complications during pregnancy, and increased risks of preterm delivery and low birth weight. In men, it can cause erectile dysfunction and reduced sperm quality, affecting fertility.

The immune system is not spared from the harmful effects of smoking. The ACS notes that smoking can affect your health in many ways, including "lowered immune system function." A weakened immune system makes the body more susceptible to infections and diseases.

Secondhand smoke poses significant health risks to non-smokers. The ALA reports that secondhand smoke exposure causes more than 41,000 deaths each year. Children exposed to secondhand smoke are more likely to suffer from respiratory infections, asthma, and sudden infant death syndrome (SIDS).

Quitting smoking at any age can significantly reduce the risk of developing these diseases and improve overall health. The ACS highlights that "people who quit smoking can also add as much as 10 years to their life, compared to people who continue to smoke." Resources and support are available to assist individuals in their journey to quit smoking, leading to longer and healthier lives.

References

American Cancer Society. (n.d.). Health Risks of Smoking Tobacco. Retrieved from https://www.cancer.org/cancer/risk-prevention/tobacco/health...

National Cancer Institute. (n.d.). Harms of Cigarette Smoking and Health Benefits of Quitting. Retrieved from https://www.cancer.gov/about-cancer/causes-prevention/risk/t...

American Lung Association. (n.d.). Health Effects of Smoking. Retrieved from https://www.lung.org/quit-smoking/smoking-facts/health-effec...

Cleveland Clinic. (2023, April 28). Smoking: Effects, Risks, Diseases, Quitting & Solutions. Retrieved from https://my.clevelandclinic.org/health/articles/17488-smoking

American Cancer Society. (n.d.). Health Risks of Using Tobacco Products. Retrieved from https://www.cancer.org/cancer/risk-prevention/tobacco/health...

American Cancer Society. (n.d.). Health Benefits of Quitting Smoking Over Time. Retrieved from https://www.cancer.org/cancer/risk-prevention/tobacco/benefi...

You're missing his point. He's saying if you make a program, you expect it to do X reliably. X may include "send an email, or kick off this workflow, or add this to the log, or crash" but you don't expect it to, for example, "delete system32 and shut down the computer". LLMs have essentially unconstrained outputs where the above mentioned program couldn't possibly delete anything or shut down your computer because nothing even close to that is in the code.

Please do not confuse this example with agentic AI losing the plot, that's not what I'm trying to say.

Edit: a better example is that when you build an autocomplete plugin for your email client, you don't expect it to also be able to play chess. But look what happened.

I'm not them, but I think it's a variation of the subset-sum problem

If we modify the question to be "sum to 100" (to just seriously reduce the number of example boxes required) then given:

  | 50 | 20 | 24 |
  |  7 |  5 |  1 |
  | 51 | 51 | 51 |

the solution would be

  | [50] | [20] | [24] |
  |   7  | [ 5] | [ 1] |
  |  51  | 51   |  51  |

  | right | down  | win
  | X     | right | up
  | X     | X     | X