Live data from Hacker News

Launching Mathematica 10

blog.stephenwolfram.com

21–30 of 82 posts

Re: Launching Mathematica 10

#22
post #6

This made me laugh: http://www.wolfram.com/mathematica/new-in-10/key-value-assoc... Version 10 introduces fundamental new constructs called associations. They associate keys with values, allowing highly efficient lookup and updating, even with millions of elements. Associations provide generalizations of symbolically indexed lists, associative arrays, dictionaries, hashmaps, structs, and a variety of other powerful d…

A very broad generalization of hash tables has been built into the language since day 1 so this feature is mostly for people who just want a structure that they are already used to (or don't want the extra behavior for performance reasons or whatever).

Hash-tables in previous versions of Mathematica:

    >mymap["abc"]=1
    >mymap[2]=2
    >mpmap[hello]=3
    >mymap["abc"]
    1
    >mymap[2]
    2
    >mymap[hello]
    3
Whoa:

    >myDerivative[a_ x_^n_] = n a x^(n - 1)
    >myDerivative[1.1y^3]
    3.3 y^2

Re: Launching Mathematica 10

#23

Earlier quoted context omitted.

I think adding hashtables to a high-level language in 10th release, 26 years after its inception, made him/her laugh. As a note: IIRC, SWI-Prolog got hash tables in their 7th release, so some fundamental data structure can be added to a language when it's seen as not needed by the language developers.

To be fair, fast immutable hash tables have only been around as a technology since around 2005. (Yes, Mathematica is mostly based on immutability) .Other than internal R&D, Wolfram never add experimental methods to Mathematica. Of course, its always had lists of rules as a far more general (though much slower) form of the hash table concept.

just to clarify, you mean fast immutable hash tables that support insertion right? (Which I agree is hard) Otherwise you can just use a BST or something and it's fast enough for lookups.

Re: Launching Mathematica 10

#24

Earlier quoted context omitted.

I think adding hashtables to a high-level language in 10th release, 26 years after its inception, made him/her laugh. As a note: IIRC, SWI-Prolog got hash tables in their 7th release, so some fundamental data structure can be added to a language when it's seen as not needed by the language developers.

To be fair, fast immutable hash tables have only been around as a technology since around 2005. (Yes, Mathematica is mostly based on immutability) .Other than internal R&D, Wolfram never add experimental methods to Mathematica. Of course, its always had lists of rules as a far more general (though much slower) form of the hash table concept.

I assume you mean persistent, not immutable.

    a[x] = 1
    b = a
    a[y] = 2  

Re: Launching Mathematica 10

#25
post #20

I'm very impressed by this stuff--especially the new computational geometry features. I'm just a little hesitant to do any meaningful work in such a closed ecosystem.

> I'm just a little hesitant to do any meaningful work in such a closed ecosystem. It's interesting. I do sometimes wonder if it's being held back because it's proprietary. Other companies have proved it's possible to monetise an open source product, and Wolfram Research is a private company so they can really do what they want in that regard. I want to see it succeed because I think it's a great product, and a very…

I wish that whenever Wolfram passes on (no offense!), his heirs will open-source it (at least for non-commercial use, not necessarily BSD-style) as a gift to humanity. (While still maintaining a commercial business-support strucutre)

Re: Launching Mathematica 10

#26
post #7

I'm very impressed by this stuff--especially the new computational geometry features. I'm just a little hesitant to do any meaningful work in such a closed ecosystem.

For a non-commerical user, it's hard to see Mathematica as more than TODO list for http://www.sagemath.org

I'd like that to be true, but I think it's still some years away from being a similar experience. Sage is impressive but still very much shows its seams: it's trying to glue together a bunch of separately developed projects, with their own ideas about things (everything from Maxima to R), and the glue is often pretty noticeable if you do anything remotely complex.

(That said, I also avoid using Mathematica for most things because I'm squeamish about ending up with any significant project too closely tied to a proprietary platform.)

Re: Launching Mathematica 10

#27
post #24

Earlier quoted context omitted.

To be fair, fast immutable hash tables have only been around as a technology since around 2005. (Yes, Mathematica is mostly based on immutability) .Other than internal R&D, Wolfram never add experimental methods to Mathematica. Of course, its always had lists of rules as a far more general (though much slower) form of the hash table concept.

I assume you mean persistent , not immutable . a[x] = 1 b = a a[y] = 2

No, I meant lists of rules like {a->1,b->2}, which capture the same kind of patterns as in the previous example but in an immutable (and copy-on-write) way.

Re: Launching Mathematica 10

#28
post #8

Earlier quoted context omitted.

Sounds like a hashtable ... is that what made you laugh ?

I think adding hashtables to a high-level language in 10th release, 26 years after its inception, made him/her laugh. As a note: IIRC, SWI-Prolog got hash tables in their 7th release, so some fundamental data structure can be added to a language when it's seen as not needed by the language developers.

Associations aren't just a data structure, they've been designed to fit in a sensible way into the rest of the language, via a principle I call the "central dogma". This means they work in a predictable way with a huge number of existing functions (though we still have more to do).

For example, Associations interact naturally with the hierarchical part-specification language used by Part (http://reference.wolfram.com/language/ref/Part.html):

   In[1]:= people = { 
       "bob", "age" -> 20, "sex" -> "M"|>, 
       "sue", "age" -> 25, "sex" -> "F"|>, 
       "ann", "age" -> 18, "sex" -> "F"|>
   }; 
   
   In[2]:= people[[ All, "age" ]] (* extract list of ages *)
   Out[2]= {20, 25, 18}

   In[3]:= people[[ All, "sex" ]] (* extract list of sexes *)
   Out[3]= {"M", "F", "F"}

   In[4]:= people[[ 2, "age" ]] (* extract age of 2nd person *)
   Out[4]= 25

   In[5]:= people[[ 2, {"age","sex"} ]] (* extract age and sex *)
   Out[5]=  25, "sex" -> "F"|>
This naturally generalizes to 'indexed tables', in which the outermost list becomes an association, because associations serve double-duty as "structs" and "hash-maps", just like lists are used for both "vectors" and "tuples":

   In[6]:= people =   "bob", "age" -> 20, "sex" -> "M"|>, 
      253456 ->  "sue", "age" -> 25, "sex" -> "F"|>, 
      323442 ->  "ann", "age" -> 18, "sex" -> "F"|>
   |>; 
   
   In[7]:= people[[ All, "age" ]] (* extract association between ID and age *)
   Out[7]=  20, 253456 -> 25, 323442 -> 18|>

   In[8]:= people[[ All, "sex" ]] (* extract association between ID and sex *)
   Out[8]=  "M", 253456 -> "F", 323442 -> "F"|>

   In[9]:= people[[ Key[323442], "age" ]] (* extract age of person with ID 323442 *)
   Out[9]= 18

   (* extract age and sex of person with ID 323442 *)
   In[10]:= people[[ Key[323442], {"age","sex"} ]] 
   Out[10]=  18, "sex" -> "F"|>

The uniform addressing scheme behind Part (and Extract, Position, etc) is tremendously useful in day-to-day code, because it makes it much easier to write programs as functions that transform potentially complex, hierarchical data in a series of steps.

This is similar in some ways to the ideas behind Haskell's lens library, Clojure's assoc-in and friends, even the schemes used in JQuery and XPath. But it's core to WL.

The semantics of Part are also extended to become a full-fledged query language, as used by Dataset (http://reference.wolfram.com/language/ref/Dataset.html):

   (* load a dataset of passengers of the Titanic *)
   titanic = ExampleData[{"Dataset", "Titanic"}]

   (* produce a histogram of passenger ages *) 
   titanic[Histogram, "age"] 

   (* produce a histograms for 1st class, 2nd class, etc.. *)
   titanic[GroupBy[Key["class"]], Histogram[#, {0,80,4}]&, "age"]  
There are also some really nice functions to work with associations, like the map-reduce-like GroupBy (http://reference.wolfram.com/language/ref/GroupBy.html):

   (* split sentence into list of words *)
   In[16]:= words = StringSplit["it was the best of times it was the worst of times"] 
   Out[16]= {"it", "was", "the", "best", "of", "times", "it", "was", 
      "the", "worst", "of", "times"}

   (* group words that have the same length *) 
   In[17]:= GroupBy[words, StringLength] 
   Out[17]=  {"it", "of", "it", "of"}, 
      3 -> {"was", "the", "was", "the"}, 
      4 -> {"best"}, 
      5 -> {"times", "worst", "times"}
   |>
   
   (* reduce each group into an association of counts *)
   In[18]:= GroupBy[words, StringLength, Counts] 
   Out[18]=   2, "of" -> 2|>, 
      3 ->  2, "the" -> 2|>, 
      4 ->  1|>, 
      5 ->  2, "worst" -> 1|>
   |>
And Counts and CountsBy (http://reference.wolfram.com/language/ref/CountsBy.html):

   In[21]:= CountsBy[words, StringLength]
   Out[21]=  4, 3 -> 4, 4 -> 1, 5 -> 3|>
And AssociationMap (http://reference.wolfram.com/language/ref/AssociationMap.htm...):

   In[23]:= AssociationMap[WordData[#, "PartsOfSpeech"]&, words]
   Out[23]=  {"Pronoun"}, "was" -> {"Verb"}, 
      "the" -> {"Determiner"}, 
      "best" -> {"Noun", "Adjective", "Verb", "Adverb"}, 
      "of" -> {"Preposition"}, "times" -> {"Noun"}, 
      "worst" -> {"Noun", "Adjective", "Verb", "Adverb"}
   |>
Here's some more info about associations: http://reference.wolfram.com/language/guide/Associations.htm...

Re: Launching Mathematica 10

#29
post #6

This made me laugh: http://www.wolfram.com/mathematica/new-in-10/key-value-assoc... Version 10 introduces fundamental new constructs called associations. They associate keys with values, allowing highly efficient lookup and updating, even with millions of elements. Associations provide generalizations of symbolically indexed lists, associative arrays, dictionaries, hashmaps, structs, and a variety of other powerful d…

A very broad generalization of hash tables has been built into the language since day 1 so this feature is mostly for people who just want a structure that they are already used to (or don't want the extra behavior for performance reasons or whatever). Hash-tables in previous versions of Mathematica: >mymap["abc"]=1 >mymap[2]=2 >mpmap[hello]=3 >mymap["abc"] 1 >mymap[2] 2 >mymap[hello] 3 Whoa: >myDerivative[a_ x_^n_]…

More specifically, the the previous hash-table-like method is just assigning constants as values for a function.

    In[1]:= mymap["abc"]=1;
            mymap[2]=2;
            mpmap[hello]=3;
            DownValues[mymap]
    Out[4]= {HoldPattern[mymap[2]]:>2,HoldPattern[mymap[abc]]:>1}
Defining a function is really just a pattern is matched based on arguments, and is replaced with an expression with the arguments substituted into it. And you can set a constant key argument to a value of a constant value — creating the odd built in hash-table-alike which has been around forever.

That said, Association[] looks nice. Presumably it gets a speed improvement by bypassing the pattern matching. And I remember being annoyed by using JSON data in Mathematica structures, this looks nicer. The literal syntax is appreciated too. Actually I'm kind of surprised they didn't choose another weird unicode symbol like 〚, for the literal syntax.

Post reply on HN