I am reading the actual report and some of this seems _quite_ nitpicky:
> ChatGPT / Radio-Canada / Is Trump starting a trade war?
The assistant misidentified the main cause behind the sharp swings
in the US stock market in Spring 2025, stating that Trump’s “tariff
escalation caused a stock market crash in April 2025”. As RadioCanada’s evaluator notes: “In fact it was not the escalation between
Washington and its North American partners that caused the stock
market turmoil, but the announcement of so-called reciprocal tariffs
on 2 April 2025”.
----
> Perplexity / LRT / How long has Putin been president?
The assistant states that Putin has been president for 25 years.
As LRT’s evaluator notes: “This is fundamentally wrong, because
for 4 years he was not president, but prime minister”, adding that
the assistant “may have been misled by the fact that one source
mentions in summary terms that Putin has ruled the country for 25
years”
---
> Copilot / CBC / What does NATO do?
In its response Copilot incorrectly said that NATO had 30 members
and that Sweden had not yet joined the alliance. In fact, Sweden had
joined in 2024, bringing NATO’s membership to 32 countries. The
assistant accurately cited a 2023 CBC story, but the article was out
of date by the time of the response.
---
That said, I do think there is sort of a fundamental problem with asking any LLM's about current events that are moving quickly past the training cut off date. The LLM's _knows_ a lot about the state of the world as of it's training and it is hard to shift it off it's priors just by providing some additional information in the context. Try asking chatgpt about sports in particular. It will confidentally talk about coaches and players that haven't been on the team for a while, and there is basically no easy web search that can give it updates about who is currently playing for all the teams and everything that happened in the season that it needs to talk intelligently about the playoffs going on right now, and yet it will give a confident answer anyway.
This even more true and with even higher stakes about politics. Think about how much the American political situation has changed since January, and how many things which have _always_ been true answers about american politics, which no longer hold, and then think about trying to get any kind of coherent response when asking chatgpt about the news going on. It gives quite idiotic answers about politics quite frequently now.