I think it's understandable these models get tripped up by math problems. But if you want another way to ground lofty expectations, ask word riddles.
---
"Answer this riddle: Which one of Santa's Reindeer can you see in outer space?"
pplx-70b-chat: "You can see Rudolph, the red-nosed reindeer, in outer space."
llama-2-70b-chat: "The answer to the riddle is "Rudolph." Rudolph is the only reindeer with a red nose, and red noses are often associated with astronauts and space travel. Therefore, Rudolph is the reindeer that can be seen in outer space."
mixtral-8x7b-instruct: "The answer to the riddle is "Comet," which is one of Santa's reindeer. The reasoning behind this is that a comet is a celestial body visible from outer space, and it shares its name with one of Santa's reindeer, as mentioned in the Christmas song 'Rudolph the Red-Nosed Reindeer.'"
---
"Answer this riddle: A doctor and a bus driver are both in love with the same woman, an attractive girl named Sarah. The bus driver had to go on a long bus trip that would last a week. Before he left, he gave Sarah seven apples. Why?"
pplx-70b-chat: "The bus driver gave Sarah seven apples because he knew that he would be away for a week, and he wanted to make sure that she had something to remember him by and to take care of her while he was gone. Each apple represented a day of the week, and he hoped that Sarah would eat one apple each day and think of him while he was away."
llama-2-70b-chat: "The bus driver gave Sarah seven apples because it was a way for him to show his love and affection for her while he was away. The number seven is often associated with perfection, completion, and abundance, so the bus driver may have chosen this number to represent the depth of his feelings for Sarah. Additionally, apples are a symbol of love and fertility in many cultures, so the bus driver may have chosen them as a way to express his desire for a long-term relationship with Sarah. The fact that the doctor is also in love with Sarah adds a layer of complexity to the riddle, as it creates a love triangle that is not uncommon in literature and real life."
mixtral-8x7b-instruct: "Because an apple a day keeps the doctor away. He gave her one for each day he would be gone to ensure she wouldn't need to see the doctor during that time."
---
"Answer this riddle: When is a doctor most annoyed?"
pplx-70b-chat: "A doctor is most annoyed when they have to deal with a patient who is not taking their health seriously or is not following their recommendations. This can lead to frustration and annoyance on the part of the doctor, as they want to help the patient but are limited by the patient's own actions."
llama-2-70b-chat: "The answer to the riddle is: When he's feeling under the weather. Explanation: The phrase "under the weather" means feeling unwell or sick, so a doctor would be most annoyed when he himself is feeling sick, as it would be a contradiction to his profession of helping others feel better."
mixtral-8x7b-instruct: "A doctor might be most annoyed when their patient does not follow their medical advice, ignores their health concerns, or does not take their treatment seriously. Additionally, a doctor might be annoyed when they are unable to diagnose a patient's condition or when they are unable to help a patient in need. Overall, a doctor might be most annoyed when they are unable to provide the best possible care for their patients."
pplx-7b-online: "A doctor is most annoyed when he is out of patients. This riddle plays on the double meaning of the word "patients," which can refer to both the people who need medical care and the source of a doctor's frustration."