Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Upgrade to SyntaxNet, New Models and a Parsing Competition

research.googleblog.com

Post reply on HN
Print view View original

61–70 of 93 posts

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#61

Post by peterbonney »

jorgemf wrote: post #57
peterbonney wrote: post #48

Earlier quoted context omitted.

I disagree - I think you are well-served to compile on your own unless you know you don't need it, e.g. if you are just trying it out to learn how it works. The standard build uses a "least common denominator" Intel instruction set (SSE4), but the odds are extremely high that the machine on which you're running tensorflow supports 4.2. Building from source allows you to use the most up-to-date instruction set (the de…

>The standard build uses a "least common denominator" Intel instruction set (SSE4) TensorFlow is not meant for the CPU. If you want to do something serious you have to move to the GPU, which it is at least 8 times faster than your optimized CPU with an average GPU.

For those of us waiting on OpenCL support because we don't have an Nvidia GPU, CPU will have to do for now.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#62

Post by akinalci »

devy wrote: post #18
RussianCow wrote: post #13

Earlier quoted context omitted.

Probably because Google still mostly uses Python 2.7 internally.

Yikes! Not being to run on Py3k is a deal breaker for me.

It's definitely an unfortunate situation. The community has been coalescing around Python 3 in the last couple years, but Google is obviously encumbered by all its legacy Python 2.7 code. Their SyntaxNet library still doesn't have Python 3 support a year after release. I'm wondering what their long-term plans are given 2.7 EOL in 2020.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#63

Post by peterbonney »

matt4077 wrote: post #56
peterbonney wrote: post #48

Earlier quoted context omitted.

I disagree - I think you are well-served to compile on your own unless you know you don't need it, e.g. if you are just trying it out to learn how it works. The standard build uses a "least common denominator" Intel instruction set (SSE4), but the odds are extremely high that the machine on which you're running tensorflow supports 4.2. Building from source allows you to use the most up-to-date instruction set (the de…

I don't quite understand how you performance-obsessed enough to care about the compiler options they use – but not run Tensorflow on a GPU. Even mobile GPUs offer a 5x speedup at least.

As I said above, if you are constrained to run on a machine without a supported GPU then every little bit helps.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#64

Post by jorgemf »

minimaxir wrote: post #24
jacquesm wrote: post #15

I've been fighting Tensorflow in the last couple of days to try an application on it, never before have I seen such a convoluted build process and a maze of dependencies. The best manual on getting tensorflow with CUDA support up and running is here: http://www.nvidia.com/object/gpu-accelerated-applications-te... But it is a little bit out of date when it comes to version numbers. If you're going to try TensorBox ( h…

sudo pip install tensorflow works fine as of the recent v1.0, and works out of the box for CPU training. The annoying thing for GPU training is handling the cudNN dependency, which Google's guides are annoyingly lacking.

It is not Google's fault. You need to agree the terms and conditions of Nvidia in order to download cuDNN. So you have to do it manually.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#65

Post by syntaxing »

minimaxir wrote: post #5

See also: spaCy, which is an open-source NLP framework that has some integration with Keras as well: https://news.ycombinator.com/item?id=13874787 ...and apparently will release a major version update today . Ouch.

This is super awesome! Thank you for mentioning about them because their announcement did not show up on my feed on HN. Even being somewhat comfortable with Tensorflow, I always find some of Google's announcements kind of overwhelming and convoluted. There's something about packages like spaCy that seems comforting and less intimating.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#66

Post by jorgemf »

peterbonney wrote: post #61
jorgemf wrote: post #57

Earlier quoted context omitted.

>The standard build uses a "least common denominator" Intel instruction set (SSE4) TensorFlow is not meant for the CPU. If you want to do something serious you have to move to the GPU, which it is at least 8 times faster than your optimized CPU with an average GPU.

For those of us waiting on OpenCL support because we don't have an Nvidia GPU, CPU will have to do for now.

Buy an Nvidia GPU with 8Gb of ram and you are good to go by now. Or you can use amazon instances or google cloud. For toy things I wouldn't bother to suffer the hell that is to compile it (I do it for my job and it is a pain to suffer its unstability everytime I need to compile it)

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#67

Post by PeterisP »

webmaven wrote: post #2

Very interesting release. The bit about guessing the part of speech, stem, etc. for previously unseen words should (I think) make it much more useful in contexts that succumb to neologizing, verbing nouns, nouning verbs, and so on (such as business writing, technical writing, academic papers, science fiction & fantasy, slang, etc.). I wonder how well it would do at parsing something that seems deliberately impenetrab…

It's much more useful in all contexts - every problem/task has a 100 words that are very common and important there while being rare and unknown in general; the problem is that for every niche that's different 100 terms.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#68

Post by PeterisP »

camdenlock wrote: post #50

> and to allow neural-network architectures to be created dynamically during processing of a sentence or document. Oh lord, is this the spark that lights the google skynet powder keg

Not yet, you'd need to apply the same approach to agentive (decisions on how to act) problems as opposed to classification tasks; then you'd have the spark that lights the google skynet powder keg.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#69

Post by liviosoares »

dang wrote: post #58

We changed the title from "Google open-sources Tensorflow-based framework for NLP", which appears misleading, given that it happened last May: https://news.ycombinator.com/item?id=11686029 . On HN the idea is to rewrite titles only to make them less misleading (or less baity). Please see https://news.ycombinator.com/newsguidelines.html .

dang, sorry if this seemed misleading. In my humble opinion, the blog title does not do full justice to the new release, primarily since it carries a new framework within SyntaxNet:

https://github.com/tensorflow/models/blob/master/syntaxnet/g...

This new DRAGNN framework is what I thought the folks here would want to know. Perhaps I should have linked to the github page, rather than the blog announcement.

Re: An Upgrade to SyntaxNet, New Models and a Parsing Competition

#70

Post by syllogism »

plusepsilon wrote: post #53
minimaxir wrote: post #5

See also: spaCy, which is an open-source NLP framework that has some integration with Keras as well: https://news.ycombinator.com/item?id=13874787 ...and apparently will release a major version update today . Ouch.

I think spaCy uses perceptrons (essentially a shallow neural network) so it should be faster. Accuracy is pretty similar with SyntaxNet at least on the training data but I'm guessing SyntaxNet works better on long range dependencies. I wonder if the spaCy update will go deep :)

The current update uses the linear model. I've also been working on neural network models, and more generally, better integration into deep learning workflows. That'll be the 2.0 release.

I've learned a lot while doing the neural network models, though. The 1.7 model takes advantage of this by having a more sophisticated optimizer. Specifically, I use an online L1 penalty and the Adam optimizer with averaged parameters. The L1 penalty allows control of size/accuracy trade-off.

This means we're finally shipping a small model: 50mb in total, compared to the current 1gb. The small model makes about 15-20% more errors.

Post reply on HN