Live data from Hacker News

LINQ Ruined My Favorite Interview Question

scottchamberlin.tumblr.com

181–190 of 218 posts

Re: LINQ Ruined My Favorite Interview Question

#181
post #176
post #154

Earlier quoted context omitted.

Ooh, this is fun! In Ruby, without imports/requires: def toptenwords(str) words = str.split words.sort_by{|word| words.count(word)}.uniq.reverse.take(10) end or as a one-liner, without any variable declarations in the function scope: def toptenwords(str) str.split.sort_by{|word| str.split.count(word)}.uniq.reverse.take(10) end

I think this version and spenuke's version are impractically slow, but yours is actually O(N²). If you give it an 824 000 word input, it will do 0.6 quadrillion word comparisons, which you will probably not be willing to wait for. A more practical solution: counts = Hash.new { 0 } IO.read('bible-pg10.txt').split.each { |w| counts[w] += 1; } counts.keys.sort_by { |w| -counts[w] }.take 10 This is still O(N lg N) instea…

I actually was working on a hash-based solution first, using group_by, but I switched to using just array after seeing the Python version. I didn't really think about the time complexity of the count operation inside the sort_by operation, thanks for pointing that out.

You definitely can do a hash-based solution in a single expression in Ruby. Here's a very ugly and kludgy example that you could probably improve on if you wanted to. I don't think it's n^2 because the group_by just counts the occurrences of each word and returns a hash where the count is the key:

str.split.group_by{|w| str.split.count(w)}.sort_by{|k,v| k}.reverse.flatten.uniq.keep_if{|w| w.is_a?(String)}.take(10)

I'm also trying to work out a better way to do this using "chunk" because although hashes are fast to access, they are not fundamentally sortable, and sort_by returns a 2d array just like chunk does anyway.

Re: LINQ Ruined My Favorite Interview Question

#182

What is it with people's obsession over lines of code? As someone who doesn't do C# or LINQ, that second solution seems to me like someone really wanted to have as few lines of code as possible. I don't claim to have an impressive programming pedigree, but while I take simplicity and performance into account, I never take "conciseness" into account. Conciseness usually means "this is opaque as shit but at least it's…

Less code is less places to insert bugs and less to read. The majority of time spent "writing" software is actually spent reading the existing code, so "less to read" is really important. "Concise" does not mean "short"; it means "short and clear". Obviously if your short code is opaque or bug-prone then you're defeating the purpose.

Re: LINQ Ruined My Favorite Interview Question

#183
post #13

I hate to be the arrogant know it all on Hacker News, but seriously, if you're writing c# and not using LINQ all the time, you need to catch up. I'm tired of seeing people answer interview questions with anything _other_ than LINQ.

The Linq answer is fine for an academic exercise, but it doesn't handle any of the myriad edge cases that pop up when such functionality meets a live product with actual users.

I say this as someone who has built a 'word counter' as part of a commercial product. We tried Linq, but had to abandon it due to speed issues as well the sheer number of edge cases.

"var words = s.Split(' ');"

Example edge case: Words separated by an em, en, or ordinary dash. In the first and second cases, they're two words, in the third, a single word. The basic Split by space function is insufficient for all but the most basic interpretation of a 'word'.

In terms of speed, that 20% becomes significant when you're processing millions of words, not a short Wikipedia article.

Expand the 'word counter' function to become a unique 'phrase counter' and you have to toss Linq out the window.

Re: LINQ Ruined My Favorite Interview Question

#184

Earlier quoted context omitted.

Though that will result in a lot more calls to Count(). A let would fix that.

I really really really should learn more Linq. Where would I use let, and why?

    from word in text.Split(' ') 
    group word by word into g
    let count = g.Count()
    orderby count descending 
    select g.Key
Under the covers when it is compiled it's turned into a select. That way it it only does N counts, not potentially N log N (for each comparison in the orderby), where N is the number of items you enumerate (so if you .Take() just 10 it doesn't really matter).

Re: LINQ Ruined My Favorite Interview Question

#185
So, aside from the Clojure, Mathematica, Python, Ruby, Bourne Shell, Haskell, and Scala solutions posted in the other comments, all of which are simpler than the C++, C#, and JS solutions, presented here with some minor cleanups:

    (take 10 (reverse (sort-by (comp first rest) (frequencies (string/split ... #"\+s")))) ; llambda Clojure

    // haakon Scala
    s.split(' ').groupBy(identity).mapValues(_.size).toList.sortBy(-_._2).take(10).map(_._1)

    (->> (string/split s #"\s+") frequencies (sort-by val) reverse (take 10)) ; aphyr Clojure

    var top = (from w in text.Split(' ')  // louthy C# LINQ
               group w by w into g 
               orderby g.Count() descending 
               select g.Key).Take(10);

    collections.Counter(s1.split()).most_common(10) # shill Python

    d = {}  # shill Python without collections library
    for word in s1.split(): d[word] = d.get(word, 0) + 1
    print [(x, d[x]) for x in sorted(d, key=d.get, reverse=True)][:10]

    words = s.split()    # spenuke and abecedarius probably O(N²) Python
    sorted(set(words), key=words.count, reverse=True)[:10]

    d3.entries((s.split(" ").reduce(function(p, v){  // 1wheel JS with d3
        v in p ? p[v]++ : p[v] = 1;
        return p;}, {})))
      .sort(function(a, b){ return a.value > b.value; })
      .map(function(d){ return d.key;})
      .slice(-10)

    # kenuke O(N²) Ruby:
    str.split.sort_by{|word| str.split.count(word)}.uniq.reverse.take(10)

    counts = Hash.new { 0 } # my Ruby
    str.split.each { |w| counts[w] += 1; }
    counts.keys.sort_by { |w| -counts[w] }.take 10

    # aaronbrethorst ruby
    str.split(/\W+/).inject(Hash.new(0)) {|acc, w| acc[w] += 1; acc}.sort {|a,b| b.last  a.last }[0,10]

    Commonest[StringSplit[string], 10]  # carlob Mathematica

    Reverse[SortBy[Tally[StringSplit[#]], #[[2]] &]][[;; 10, 1]] &  # superfx old Mathematica

    $a = array_count_values(preg_split('/\b\s+/', $s)); arsort($a); array_slice($a, 0, 10) // Myrth PHP

    tr -cs a-zA-Z '\n' | sort | uniq -c | sort -nr | head  # mzs and me sh

    -- lelf in Haskell
    take 10 . map head . reverse . sortBy (comparing length) . group . sort . words

    # prakashk Perl6
    .say for (bag($text.words) ==> sort {-*.value})[^10]

    # navinp1912 C++
     string s,f;
     map M;
     set > S;
     while(cin >> s) {
             M[s]++;
             int x=M[s];
             if(x>1) S.erase(make_pair(x-1,s));
             S.insert(make_pair(x,s));
     }
     set >::reverse_iterator it=S.rbegin();
     int topK=10;
     while(topK-- && (it!=S.rend())) {
             cout secondfirst
I thought I'd maybe take a look at Afterquery: http://afterquery.appspot.com/help

Although I haven't tested it, I think the Afterquery program to solve this, assuming you first had something to tokenize your text into one word per row, would be something like

    &group=word;count(*)
    &order=-count(*)
    &limit=10
which, though perhaps less readable, is simpler still, except for Mathematica. More details at http://apenwarr.ca/log/?m=201212.

Perl 5, perhaps surprisingly, is not simpler:

    perl -wle 'local $/; $_ = ; $, = " "; $w{$_}++ for split; print @{[sort {$w{$b}  $w{$a}} keys %w]}[0..9]'
And neither is this, although it uses less code and less RAM:

    perl -wlne '$w{$_}++ for split; END { $, = " "; print @{[sort {$w{$b}  $w{$a}} keys %w]}[0..9]}'
I was surprised, attempting to solve this in Common Lisp, that there's no equivalent of string/split in ANSI Common Lisp, and although SPLIT-SEQUENCE is standardized, it's not included in SBCL's default install, at least on Debian; and counting the duplicate words involves an explicit loop. So basically in unvarnished CL you end up doing more or less what you'd do in C, but without writing your own hash table. Lua and Scheme too, I think, except that in Scheme you don't even have hash tables.

Re: LINQ Ruined My Favorite Interview Question

#186
post #120
post #117

Earlier quoted context omitted.

Think you also need a (map key) there to be equivalent to the C# code. At which point it's starting to look very similar to well written C# code solving the same problem. Much as I love Clojure, it's worth pointing out that Clojure's lazy sequences (include intermediate sequences) get cached, while LINQ evaluates more like Clojure's reducers. (You can even get it to do so in parallel.)

You're right; it does need a (map key) line to be equivalent. I was thinking of http://www.leancrew.com/all-this/2011/12/more-shell-less-egg... , which wants the counts too. :) On a side note, I'd like to point out a key difference between this Clojure example and corresponding variants in C#, Python, &c: the Clojure variant has no variables . This isn't just a matter of concision: coming up with descriptive names is…

In C# that could be:

  return new Swimmer("foo")
  { Style = "butterfly"}
  .swim();
It is unidiomatic to have Swimmer.swim return 'this', though.

  var result = new Swimmer("foo")
  { Style = "butterfly"};
  result.swim();
  return result;

Re: LINQ Ruined My Favorite Interview Question

#187

(his name is Jon Skeet) (sorry)

string s,f; map M; set > S; while(cin >> s) { M[s]++; int x=M[s]; if(x>1) S.erase(make_pair(x-1,s)); S.insert(make_pair(x,s)); } set >::reverse_iterator it=S.rbegin(); int topK=10; while(topK-- && (it!=S.rend())) { cout second first

[deleted]

Re: LINQ Ruined My Favorite Interview Question

#188

(his name is Jon Skeet) (sorry)

string s,f; map M; set > S; while(cin >> s) { M[s]++; int x=M[s]; if(x>1) S.erase(make_pair(x-1,s)); S.insert(make_pair(x,s)); } set >::reverse_iterator it=S.rbegin(); int topK=10; while(topK-- && (it!=S.rend())) { cout second first

Not bad. I think it would be a little simpler and faster with:

    while (cin >> s) M[s]++;
    for (map::iterator i = M.begin(); i != M.end(); i++) {
             S.insert(make_pair(i->second, i->first));
    }
But maybe there's a downside to that approach that isn't obvious to me?

Re: LINQ Ruined My Favorite Interview Question

#190

(his name is Jon Skeet) (sorry)

string s,f; map M; set > S; while(cin >> s) { M[s]++; int x=M[s]; if(x>1) S.erase(make_pair(x-1,s)); S.insert(make_pair(x,s)); } set >::reverse_iterator it=S.rbegin(); int topK=10; while(topK-- && (it!=S.rend())) { cout second first

[deleted]
Post reply on HN