Live data from Hacker News

Retiring a Great Interview Problem

thenoisychannel.com

91–100 of 122 posts

Re: Retiring a Great Interview Problem

#91
post #88
post #85

Earlier quoted context omitted.

You could store the dictionary in a trie in O(m) time before you start. But I don't think it's true that it makes it O(n); being able to tell that the prefix you're considering is the prefix of some word doesn't help you, because you still don't know if the suffix of the string after that word is segmentable. (Or even whether the string contains the entire word.)

It does help you. You're following a dynamic programming solution. Suppose that the maximum dictionary word is of length k. Then for each position in the string you have a maximum of k previous positions that you're tracking for, "We could start a next word here." Once you've scanned the string, only then do you discover whether the whole string is segmentable. Put another way, while you're processing you don't immed…

It's true that the trie allows you to prune the search tree, but I don't think that gets you to O(N). The maximum-dictionary-word-length check does get you to O(N), though, or rather O(kN) if you consider the dictionary as part of the input.

Re: Retiring a Great Interview Problem

#92
post #8

Only slightly joking: re.findall("(your|dict|here)", "yourword") I like the idea of constructing a state machine to do all the matching.

There's a library for constructing a deterministic regular expression from a dictionary, by the way, which would right away give you the exponential-time result. If you wrapped it in a * and applied it with a guaranteed-linear-time regular-expression engine like RE2, you'd find out whether the string was segmentable (and as a bonus you wouldn't have to construct the deterministic RE yourself) but I don't know if you'd get the actual segments.

Re: Retiring a Great Interview Problem

#93

Earlier quoted context omitted.

Yes, from a fast reading, your solution appears to be correct, and so does 'memoization', but, still, in spite of common practice in computing, it's not a dynamic programming algorithm because what 'dynamic programming' is goes back to R. Bellman and his student Dreyfus and the book Dreyfus and Law. There may be a problem with what you outline: That substring [i, j) is a word is not so impressive! In addition we need…

I'm confused. Are you saying that if you have a DP table A then to compute A[i+1] you are only allowed to use A[i]? This question is classic DP. A[i] stores the segmentation of the first i letters if it exists (or it can just store a boolean, depending on implementation). Then to compute A[i] you look at the j<i see if A[j] is non-empty and if the [j,i) substring is in the dictionary. If you find a valid j you set A[…

There is what 'dynamic programming' is, and there is the 'word segmentation' problem of this thread.

The problem is interesting, and so are algorithms to solve it.

My view is that algorithms to solve the problem are not really dynamic programming.

If strain, then can regard the code I include below to solve the problem as dynamic programming, but can do such things quite generally.

For what 'dynamic programming' is, see the original books by R. Bellman or:

Stuart E.\ Dreyfus and Averill M.\ Law, {\it The Art and Theory of Dynamic Programming,\/} ISBN 0-12-221860-4, Academic Press, New York, 1977.\ \

Dimitri P.\ Bertsekas, {\it Dynamic Programming: Deterministic and Stochastic Models,\/} ISBN 0-13-221581-0, Prentice-Hall, Englewood Cliffs, NJ, 1987.\ \

George L.\ Nemhauser, {\it Dynamic Programming,\/} ISBN 0-471-63150-7, John Wiley and Sons, New York, 1966.\ \

Dimitri P.\ Bertsekas and Steven E.\ Shreve, {\it Stochastic Optimal Control: The Discrete Time Case,\/} ISBN 0-12-093260-1, Academic Press, New York, 1978.\ \

E.\ B.\ Dynkin and A.\ A.\ Yushkevich, {\it Controlled Markov Processes,\/} ISBN 0-387-90387-9, Springer-Verlag, Berlin, 1979.\ \

Wendell H.\ Fleming and Raymond W.\ Rishel, {\it Deterministic and Stochastic Optimal Control,\/} ISBN 0-387-90155-8, Springer-Verlag, Berlin, 1979.\ \

For the word segmentation problem, for some positive integer n, we are given a string of length n of characters of the alphabet, and we are given a dictionary of words. We are to show that the string can be partitioned ('segmented') into a sequence of words in the dictionary or show that no such partitioning exists.

A partitioning is a 'feasible segmentation'. We are only looking for a feasible segmentation and not all feasible segmentations.

Consider the string

     'catsdoll'
Okay, the 'substring'

     'cat'
is a word. But that word will not be in a feasible segmentation because the string that remains

     = 'sdoll'
has no 'feasible segmentation'.

Generally, even if there is a feasible segmentation, just because we have found a word does not mean that that word will be in a feasible segmentation.

Generally we suspect that somehow the solution will be 'iterative', 'incremental', maybe 'recursive'. So, in software terms, we suspect that we will have a do-loop with

     i = 1, 2, ..., n
or

     i = n, n - 1, ..., 1
For 1 Somehow when we get out of this look we want a feasible segmentation or a claim that there is none.

Let's consider a loop with

     i = 1, 2, ..., n
Okay, let's let b(i) = j > 0, i > j, if s[1, i] has a feasible segmentation and let b(i) = 0 otherwise. When b(i) = j, then s[1, j] has a feasible segmentation and s[j + 1, i] is a word in the dictionary.

In the loop, the pass for i gets the value of b(i) and in this uses the values b(j), 1 If we come out of the loop with b(n) = 0 or n = 0, then we conclude that there is no solution.

Fine.

But it's not really dynamic programming. The obvious candidate for 'stages' would be i = 1, 2, ..., n. But in stage i, we have to consider essentially 'stages' 1 But if define the stages, decisions, state transition functions, and optimal value functions in relatively tricky ways, then we might be able to regard the problem as dynamic programming.

Below is some corresponding code. The code is in Rexx which is an elegant old 'scripting' language developed something over 25 years age by Mike Cowlishaw at IBM. It is fair to regard all the elementary values as just strings. If in addition, the strings are legal base 10 numbers, then Rexx can perform arithmetic with up to 50 significant decimal digits.

One special feature is a.x for any string x regarded as an 'index'. Then a.x will be a string.

The code:

     /* WORD01.RXS --                                      */
     /*                                                    */
     /*   Solution to 'word segmentation' exercise at      */
     /*                                                    */
     /*        http://news.ycombinator.com/item?id=2859182 */
     /*                                                    */
     /* Created at 15:40:00 on Monday, August 8th, 2011.   */

       exec_name = 'word01.rxs'

     /*   Read dictionary into 'stem' variable dict. from  */
     /*   file in_file:                                    */

       in_file = 'dict.dat'

     /*   Set the 'default' value of 'stem' variable       */
     /*   dict.:                                           */

       dict. = 0
       Do Forever
         If Lines(in_file) = 0 Then Leave
         word = Strip( Linein(in_file) )
         dict.word = 1
       End

     /*   So, now, a string word is in the dictionary if   */
     /*   and only if dict.word = 1                        */

     /*   The given string to check for a feasible 'word   */
     /*   segmentation' solution is:                       */

       input_string = ''
       input_string = 'a'
       input_string = 'b'
       input_string = 'up'
       input_string = 'downup'
       input_string = 'down'
       input_string = 'downzzzz'
       input_string = 'aintgottaword'
       input_string = 'gotword'
       input_string = 'catsdoll'
       input_string = 'therecanbeanotherreasontocontactbefore'

     /*   Then we find the length of this given string:    */

       n = Length( input_string )

     /*   Here is the description of the main logic:  For  */
     /*   i = 1, 2, ..., n, we find b.i so that            */
     /*                                                    */
     /*             /  0,  if Substr( input_string, 1, i ) */
     /*            |       has no solution                 */
     /*            |                                       */
     /*     b.i =  0, otherwise.                   */
     /*                                                    */
     /*   When b.i = j > 0, then                           */
     /*                                                    */
     /*        Substr( input_string, 1, j )                */
     /*                                                    */
     /*   has a solution and                               */
     /*                                                    */
     /*        Substr( input_string, j + 1, i - j )        */
     /*                                                    */
     /*   is a word in the dictionary.                     */

     /*   At in b.jm1 below, it is convenient for the      */
     /*   logic to have:                                   */

       b.0 = 1

     /*   Pass through this look determines if             */
     /*                                                    */
     /*        Substr( input_string, 1, i )                */
     /*                                                    */
     /*   has a feasible word segmentation:                */

       Do i = 1 To n

     /*   We preemptively set b.i = 0 and then in the      */
     /*   loop on j change the value if appropriate:       */

         b.i = 0

     /*   This loop violates the spirit of a dynamic       */
     /*   program with stages i:                           */

         Do j = i To 1 By -1
           jm1 = j - 1
           If b.jm1 = 0 Then Iterate
           word = Substr( input_string, j, i - j + 1 )
           If dict.word = 0 Then Iterate
           b.i = j
           Leave
         End

       End

     /*   Since in the loop on i above we had i = 1 To n,  */
     /*   it is essentially inevitable that here we will   */
     /*   have to work from i = n down to 1:               */

       If b.n = 0 | n = 0
         Then
           Do
             Say "There is no solution."
           End
         Else
           Do
             Say 'There is a solution:'
             k = 0
             i = n
             Do Forever
               j = b.i
               If j = 0 Then
                 Do
                   If i = 1 Then Leave
                   i = i - 1
                   Iterate
                 End
               k = k + 1
               segment.k = Substr( input_string, j, i - j + 1 )
               If j = 1 Then Leave
               i = j - 1
             End
             m = k
             Do k = m To 1 By -1
               Say Format( m - k + 1, 5) segment.k
             End
           End

Re: Retiring a Great Interview Problem

#94
post #91
post #88

Earlier quoted context omitted.

It does help you. You're following a dynamic programming solution. Suppose that the maximum dictionary word is of length k. Then for each position in the string you have a maximum of k previous positions that you're tracking for, "We could start a next word here." Once you've scanned the string, only then do you discover whether the whole string is segmentable. Put another way, while you're processing you don't immed…

It's true that the trie allows you to prune the search tree, but I don't think that gets you to O(N). The maximum-dictionary-word-length check does get you to O(N), though, or rather O(kN) if you consider the dictionary as part of the input.

The trie check contains the maximum-dictionary-word-length check as an obvious special case and therefore its worst case is the same order of magnitude efficiency as the other check.

You do have the cost of descending a level in the trie. But that can be made constant (with a jump table) or else the log of the size of the alphabet (which is a constant for all intents and purposes).

Re: Retiring a Great Interview Problem

#95
post #84

Earlier quoted context omitted.

Did you expect people to be able to do this by hand? I mean infront of an IDE I could do this in 5-10 minutes. But on a blackboard or pen and paper there is little chance I would get anywhere close to a correct answer.

I don't understand how that's possible. I mean, I'm going to assume that you're telling the truth, but I don't understand how it's possible for that knowledge to reside so entirely in your IDE rather than in your head. Do you mean you know that you would use an Enumeration, but not the names of the Enumeration and Hashtable methods that you would use? If you're trying to debug a piece of code and it's calling the wro…

I think onemoreact meant that after relying on a good IDE for awhile you can easily forget whether the method is named table.keys() or table.getKeys(), hasElements() or hasMoreElements(), stuff like that. Forgetting such things doesn't impair your ability to understand or debug the code.

Re: Retiring a Great Interview Problem

#96
post #6

Just for fun I decided to rewrite his first version in Haskell. This is probably not idiomatic, though. segment_string :: String -> Set String -> Maybe String segment_string [] _ = Nothing segment_string str dict = if str `member` dict then Just str else let pairs = zip (inits str) (tails str) pairInDict (x, y) = x `member` dict && y `member` dict in do (x, y)

How about this, this will compute more than pairs

let dict = Data.Set.fromList["apple", "pie", "bread", "applepie", "piebread"]

let fn q = concat.Data.List.map (\(x,y) -> if (member x dict) then if y == "" then [[x]] else Data.List.map (x:) (fn y) else [] ) $ tail $ zip (inits q) (tails q)

fn "applepiebread" [["apple","pie","bread"],["apple","piebread"],["applepie","bread"]]

Re: Retiring a Great Interview Problem

#97
post #84

Earlier quoted context omitted.

I don't understand how that's possible. I mean, I'm going to assume that you're telling the truth, but I don't understand how it's possible for that knowledge to reside so entirely in your IDE rather than in your head. Do you mean you know that you would use an Enumeration, but not the names of the Enumeration and Hashtable methods that you would use? If you're trying to debug a piece of code and it's calling the wro…

I think onemoreact meant that after relying on a good IDE for awhile you can easily forget whether the method is named table.keys() or table.getKeys(), hasElements() or hasMoreElements(), stuff like that. Forgetting such things doesn't impair your ability to understand or debug the code.

Java's idiosyncratic interfaces don't always help either: there's Enumeration which has nextElement() and hasMoreElements(), and there's Iterator which has next() and hasNext().

Re: Retiring a Great Interview Problem

#98

Earlier quoted context omitted.

In 1997, before for-each and generics, it would've been very verbose. I learned Java in 2006 and haven't written much in a while, so I maybe slightly off, but I think you'd either get an Iterator for the Hashtable's keys and repeatedly call Iterator.next() and Hashtable.get(), or get the length of Hashtable.keys() and use a traditional C-style for loop. Okay, I couldn't resist looking up the old docs, so this looks a…

FYI - there's a potential defect there - keys are not required to be strings so you should lookup the value before you cast.

I was relying on the assumption from the original question that the keys and values were all Strings: "...a Hashtable containing String keys and values and printing them all out."

Re: Retiring a Great Interview Problem

#99

Earlier quoted context omitted.

The "obvious" selection of stages does work. When you are at position j you check all i<j until you find one where there is a segmentation up to i, and the substring [i,j) is a word. Memoization is just syntactic (semantic?) sugar on top of this and he provides the code that basically implements the above. In programming competition circles it's common to just say "it's dynamic programming" when the stages are semi-o…

Yes, from a fast reading, your solution appears to be correct, and so does 'memoization', but, still, in spite of common practice in computing, it's not a dynamic programming algorithm because what 'dynamic programming' is goes back to R. Bellman and his student Dreyfus and the book Dreyfus and Law. There may be a problem with what you outline: That substring [i, j) is a word is not so impressive! In addition we need…

I just put a bottom-up reformulation of the dynamic-programming solution at http://canonical.org/~kragen/sw/inexorable-misc/wordseg.c, in the function "segment". The stages are the prefixes of s: the substrings s[0:0], s[0:1], s[0:2],... s[0:n], where n is the length of s. The state of some stage s[0:i] is a finite map from all of its segmentable prefixes s[0:j] {j∈[0,i)} to segmentations of those prefixes. (Only one segmentation per prefix.) The transition function may leave the state unchanged, or it may add a pair to the map, mapping s[0:i] to some segmentation, which it can do if ∃j:∃w∈dict: (s[0:j] || word) = s[0:i], in which case the segmentation is (state[s[0:j]], word).

(In my program, the state[] table is represented simply as a vector of integers: seglen[i] is simply the length of the last word of state[s[0:i]], or 0 if s[0:i] is not present in the finite map. This is sufficient to efficiently reconstruct a segmentation.)

This is completely different from your formulation where you're thinking about i+1, i+2, etc. It's not surprising that you think that your formulation isn't dynamic programming!

Now, this is a solution by forward induction or "bottom-up dynamic programming", and I wrote it that way because it's easier to see the mapping to dynamic programming. But if you solve the problem by backward induction or "top-down dynamic programming" instead, you may be able to solve the problem a lot more efficiently, because you can avoid computing most of the table entries. And that's what happens if you just write the recurrence out directly as a recursive function and then memoize it.

Re: Retiring a Great Interview Problem

#100
post #94
post #91

Earlier quoted context omitted.

It's true that the trie allows you to prune the search tree, but I don't think that gets you to O(N). The maximum-dictionary-word-length check does get you to O(N), though, or rather O(kN) if you consider the dictionary as part of the input.

The trie check contains the maximum-dictionary-word-length check as an obvious special case and therefore its worst case is the same order of magnitude efficiency as the other check. You do have the cost of descending a level in the trie. But that can be made constant (with a jump table) or else the log of the size of the alphabet (which is a constant for all intents and purposes).

I suppose that depends on what you store in your trie. If every node contains a field for the length of the longest path below it, then yes. But that's not a normal thing to store in a trie, and it wasn't at all obvious to me that that was what you meant.
Post reply on HN