Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What a crossword AI reveals about humans' way with words

wired.com

Post reply on HN
Print view View original

11–20 of 50 posts

Re: What a crossword AI reveals about humans' way with words

#11

Post by karamanolev »

bvm wrote: post #9

> The crossword solver is a closed system—it can’t just Google the answers. yet from the wiki: > Probabilities for individual words or phrases in the puzzle are computed using relatively simple statistical techniques based on features such as previous appearances of the clue, number of Google hits for the fill. https://en.wikipedia.org/wiki/Dr.Fill I don't think it's fair to describe it as a closed-system if it's Goo…

Can't the number of Google hits for all words in the dictionary be enumerated and stored before entry into the competition? Wouldn't that keep it a closed system, even if it uses metrics like that?

Re: What a crossword AI reveals about humans' way with words

#12

Post by sweezyjeezy »

bvm wrote: post #7

I would be interested to see how they perform on UK-style cryptic crosswords ( https://en.wikipedia.org/wiki/Cryptic_crossword ) where there's much more play/subversion/meta-approach to the "rules".

I think it's pretty unlikely you'll be able to learn the UK rules to a human level using brute force ML - take a fairly easy clue like ‘Drunken men are more despicable' - you need to split this as (wordplay = drunken (anagram of) "men are") / (meaning = "more despicable") = "MEANER"

This is really hard for supervised learning - the reward is quite sparse (e.g. did you get it right / how many characters did you get right) but the task is reasonably complex (e.g. it would have to learn how to spot/execute arbitrary length anagrams on its own, already something that is nontrivial for ML). Sparse + complex usually means gradient descent will fail or converge to a more trivial minima e.g. only look for synonyms in the clue.

I reckon you would have to codify the different types of cryptic clues manually for this work well.

Re: What a crossword AI reveals about humans' way with words

#13

Post by wongarsu »

bvm wrote: post #9

> The crossword solver is a closed system—it can’t just Google the answers. yet from the wiki: > Probabilities for individual words or phrases in the puzzle are computed using relatively simple statistical techniques based on features such as previous appearances of the clue, number of Google hits for the fill. https://en.wikipedia.org/wiki/Dr.Fill I don't think it's fair to describe it as a closed-system if it's Goo…

Humans are allowed to prepare however they like, so anything from the real world should be fair game as long as you don't add anything during the solving process itself.

Re: What a crossword AI reveals about humans' way with words

#14

Post by ximeng »

sweezyjeezy wrote: post #12
bvm wrote: post #7

I would be interested to see how they perform on UK-style cryptic crosswords ( https://en.wikipedia.org/wiki/Cryptic_crossword ) where there's much more play/subversion/meta-approach to the "rules".

I think it's pretty unlikely you'll be able to learn the UK rules to a human level using brute force ML - take a fairly easy clue like ‘Drunken men are more despicable' - you need to split this as (wordplay = drunken (anagram of) "men are") / (meaning = "more despicable") = "MEANER" This is really hard for supervised learning - the reward is quite sparse (e.g. did you get it right / how many characters did you get ri…

https://www.crosswordgenius.com/ not sure how this is done but it looks pretty good.

https://unlikely.ai/cryptic-crossword-genius-unlikely-ai-art... has a little more detail.

https://crosswordgenius.com/clue/who-may-get-drunk-with-real... example solution with anagram.

Re: What a crossword AI reveals about humans' way with words

#15

Post by sweezyjeezy »

ximeng wrote: post #14
sweezyjeezy wrote: post #12

Earlier quoted context omitted.

I think it's pretty unlikely you'll be able to learn the UK rules to a human level using brute force ML - take a fairly easy clue like ‘Drunken men are more despicable' - you need to split this as (wordplay = drunken (anagram of) "men are") / (meaning = "more despicable") = "MEANER" This is really hard for supervised learning - the reward is quite sparse (e.g. did you get it right / how many characters did you get ri…

https://www.crosswordgenius.com/ not sure how this is done but it looks pretty good. https://unlikely.ai/cryptic-crossword-genius-unlikely-ai-art... has a little more detail. https://crosswordgenius.com/clue/who-may-get-drunk-with-real... example solution with anagram.

Very cool - the fact that it explains the solution makes me almost certain they have codified the rules here.

Re: What a crossword AI reveals about humans' way with words

#16

Post by tasogare »

ptero wrote: post #6
isolli wrote: post #3

"But, unlike more than 200 human solvers, it wasn’t perfect on all of the puzzles: It got waylaid on two of them and finished with errors." I'm curious, what kind of error are we talking about? Words that don't exist, or another solution to a problem that may not have a unique solution?

Non-existent words should be easy to check for. My guess is answers not (quite) matching the clues. Analogies and cultural references are probably hard for AI to get: is an "empire fighting knight" historical figure or Jedi? One level of such referencing is doable, 2-3 likely very hard.

> Non-existent words should be easy to check for.

I'm interested in your magical solution because this is a hard problem.

Re: What a crossword AI reveals about humans' way with words

#17

Post by gorgoiler »

ximeng wrote: post #14
sweezyjeezy wrote: post #12

Earlier quoted context omitted.

I think it's pretty unlikely you'll be able to learn the UK rules to a human level using brute force ML - take a fairly easy clue like ‘Drunken men are more despicable' - you need to split this as (wordplay = drunken (anagram of) "men are") / (meaning = "more despicable") = "MEANER" This is really hard for supervised learning - the reward is quite sparse (e.g. did you get it right / how many characters did you get ri…

https://www.crosswordgenius.com/ not sure how this is done but it looks pretty good. https://unlikely.ai/cryptic-crossword-genius-unlikely-ai-art... has a little more detail. https://crosswordgenius.com/clue/who-may-get-drunk-with-real... example solution with anagram.

Crossword Genius is from one of the Amazon Alexa founders.

Re: What a crossword AI reveals about humans' way with words

#18

Post by tclancy »

djrockstar1 wrote: post #4
isolli wrote: post #3

"But, unlike more than 200 human solvers, it wasn’t perfect on all of the puzzles: It got waylaid on two of them and finished with errors." I'm curious, what kind of error are we talking about? Words that don't exist, or another solution to a problem that may not have a unique solution?

Also, as a novice to crosswords, how do puzzle makers ensure that they have a unique solution?

They don’t necessarily. Some times this is done on purpose at harder difficulties, the clueing is intentionally vague to lead you down blind alleys.

Re: What a crossword AI reveals about humans' way with words

#19

Post by tsimionescu »

tasogare wrote: post #16
ptero wrote: post #6

Earlier quoted context omitted.

Non-existent words should be easy to check for. My guess is answers not (quite) matching the clues. Analogies and cultural references are probably hard for AI to get: is an "empire fighting knight" historical figure or Jedi? One level of such referencing is doable, 2-3 likely very hard.

> Non-existent words should be easy to check for. I'm interested in your magical solution because this is a hard problem.

You should have given more detail on why this problem is deceptively hard. I am guessing that the simple solution of looking up the word in the dictionary seems to work ok (especially in the context of an artificial competition, which doesn't have to accept uncommon spellings, words in other languages etc), but still breaks down hard because of proper names, which are common in cross-words.

Re: What a crossword AI reveals about humans' way with words

#20

Post by SamBam »

sweezyjeezy wrote: post #15
ximeng wrote: post #14

Earlier quoted context omitted.

https://www.crosswordgenius.com/ not sure how this is done but it looks pretty good. https://unlikely.ai/cryptic-crossword-genius-unlikely-ai-art... has a little more detail. https://crosswordgenius.com/clue/who-may-get-drunk-with-real... example solution with anagram.

Very cool - the fact that it explains the solution makes me almost certain they have codified the rules here.

Actually, one of the interesting things about cryptic crosswords is that, in some ways, the rules are far more codified.

The specifics definitely depend on the paper, of course, but basics like the fact that the majority of the clues are actually two halves, clueing the word two different ways, and that anagrams are always explicitly clues using words such as "scrambled," "drunken," "off" or whatever could actually make it easier for the human to guide the ML.

Post reply on HN