Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python, Machine Learning, and Language Wars (2015)

sebastianraschka.com

Post reply on HN
Print view View original

51–60 of 62 posts

Re: Python, Machine Learning, and Language Wars (2015)

#51

Post by a3n »

spdustin wrote: post #50
a3n wrote: post #49

Earlier quoted context omitted.

I have anaconda installed at work/windows and home/linux. One minor annoyance is that it doesn't play well (at all?) with virtualenv. They worked so hard to get a huge suite of off the shelf third party libraries integrated, it's curious that they would come up with a replacement for virtualenv. I'm halfway hoping that I don't know what I'm talking about.

http://conda.pydata.org/docs/using/envs.html Glad to help :)

Right, I know about that. What I'm pointing out is that anaconda doesn't support http://docs.python-guide.org/en/latest/dev/virtualenvs/

It's nice that they have an env tool, but why didn't they just use the existing virtualenv?

Re: Python, Machine Learning, and Language Wars (2015)

#52

Post by spdustin »

a3n wrote: post #51
spdustin wrote: post #50

Earlier quoted context omitted.

http://conda.pydata.org/docs/using/envs.html Glad to help :)

Right, I know about that. What I'm pointing out is that anaconda doesn't support http://docs.python-guide.org/en/latest/dev/virtualenvs/ It's nice that they have an env tool, but why didn't they just use the existing virtualenv?

Because using theirs means not having to recompile numpy (for example) with each environment.

Why doesn't anaconda's env work for you?

Re: Python, Machine Learning, and Language Wars (2015)

#53

Post by a3n »

spdustin wrote: post #52
a3n wrote: post #51

Earlier quoted context omitted.

Right, I know about that. What I'm pointing out is that anaconda doesn't support http://docs.python-guide.org/en/latest/dev/virtualenvs/ It's nice that they have an env tool, but why didn't they just use the existing virtualenv?

Because using theirs means not having to recompile numpy (for example) with each environment. Why doesn't anaconda's env work for you?

Not saying it doesn't work, just wondering why the duplication of functionality. Which you answered, thanks.

Re: Python, Machine Learning, and Language Wars (2015)

#54

Post by x1798DE »

CalRobert wrote: post #5
danso wrote: post #3

As someone who's switched from Ruby to Python (for now, because the latter is far easier to teach, IMO) and also put significant time into learning R, because of how strong ggplot2 is...I was really surprised at the lack of Google results for "switching from python to r" -- or similarly phrased queries to find guides on how to go from Python to R...in fact, that particular query will bring up more results for R -> Py…

You know, I remember when I was trying my first language other than BASIC (VB6, perhaps? or maybe 1995 era JS?) and it bugged me that "x = y" wasn't the same as "y = x". Remembering it as "LET x = y" was helpful.

Heh, that brings me back to when I was a kid, thinking, "x = x + 1? No it doesn't."

Re: Python, Machine Learning, and Language Wars (2015)

#55

Post by spdustin »

a3n wrote: post #53
spdustin wrote: post #52

Earlier quoted context omitted.

Because using theirs means not having to recompile numpy (for example) with each environment. Why doesn't anaconda's env work for you?

Not saying it doesn't work, just wondering why the duplication of functionality. Which you answered, thanks.

conda env also installs other dependencies for code. Really, the whole point of it all is that conda replaces pip and virtualenv.

Conda manages more than just Python packages, but other dependencies as well. For example, here's list of packages in a new environment I created with

  $ conda create -n hn python
  $ source activate hn
  $ conda list -c

  openssl-1.0.2g-0
  pip-8.1.1-py35_0
  python-3.5.1-0
  readline-6.2-2
  setuptools-20.3-py35_0
  sqlite-3.9.2-0
  tk-8.5.18-0
  wheel-0.29.0-py35_0
  xz-5.0.5-1
  zlib-1.2.8-0
setuptools and wheel are the only Python packages installed. The others are non-Python packages that conda is managing for my environment.

For all the talk of vendoring in package manager tools these days, I really, really like the way conda manages this stuff.

Re: Python, Machine Learning, and Language Wars (2015)

#56

Post by techdragon »

takeda wrote: post #48
yeukhon wrote: post #32

Earlier quoted context omitted.

Is wheel not default? Or play it elementary :-) how would I avoid this issue in the future?

I did not write anything worthy to send to PIP so don't know exactly but looks like it is up to the developers[1] Anyway I just noticed that PyPI only supports binary packages for Windows and Mac OS X. Although, you could still generate wheels of packages that you use by using something like this: pip wheel -r requirements.txt You can then install them with pip install or (unfortunately I forgot the option, perhaps i…

the issue with Wheels on Linux, and this is very much a Linux problem, is that the 'pre built' nature of libraries in wheels doesnt play nice with the raw chaos of Linux's package management + distro + kernel ecosystem. I raised the question of FreeBSD and Solaris based wheels at a PyCon when I was in a face to face discussion with someone more knowledgeable, and the answer was 'in theory that should work like Windows & OSX, no one has done the hard work yet.'

So yeah Linux is not the most friendly environment for Python Wheels.

Re: Python, Machine Learning, and Language Wars (2015)

#57

Post by spot »

zmmmmm wrote: post #47
spot wrote: post #28

why pick just one language? with the polyglot Beaker Notebook, you can work with many languages, even in the same notebook, and your data is automatically translated between them. each of these languages has its strong point. there is always some library you want to use in some other language. or you want to collaborate with someone. or next year you change your mind and Julia is finally good enough. http://BeakerNot…

I really like Beaker. However Jupyter also caters to a wide range of languages, albeit one language per notebook. So the main differentiator for Beaker is that it can auto translate data between languages. However there is a giant catch to that: it is done by serialising the data which intrinsically means it is extremely expensive for any significant sized data set. As a result, in most cases I end up writing my data…

thanks. indeed autotranslation currently only works for small data. fixing it with shared memory and hdf5 is on the agenda for later this year: https://github.com/twosigma/beaker-notebook/wiki/Roadmap

I would really like to hear what usability and convenience advantages RStudio has for you...

Re: Python, Machine Learning, and Language Wars (2015)

#58

Post by p4wnc6 »

TheLogothete wrote: post #23
p4wnc6 wrote: post #22

Earlier quoted context omitted.

That is in fact an exact situation I've been in, where it had previously been organized as a bunch do ad hoc scripts, and it caused huge problems. It is exactly all of these fractured cases where you're running analyses in some bespoke corner of a business that it is most critical that the analysis tool is productionized and unified. Plenty of businesses don't do it this way, but their reliance on manual scripts, man…

Vendors are standardizing on R, so I don't think the situation is as severe as you make it out to be.

Vendors tend to standardize on crap. How many vending machines do you see with salad or tofu instead of Doritos or soda?

Big multinational IT vendors are even worse than the vending machine example. They cater to situations where a firm has some internal power struggle over IT and the struggle is won by one side who gets the certified approval of a statusy external vendor.

For instance, most big, showy technology re-orgs around Hadoop are unequivocally bullshit. Most firms do not face data problems that are appropriate for Hadoop, not even when their primary enterprise data store approaches the raw size at which Hadoop becomes plausible, because the types of analytical workflows still tend to involve much smaller data.

These firms hardly adopt Hadoop for pragmatic engineering reasons, and engineers may even resist the re-org and offer compelling evidence that it is a waste of money, yet are ignored.

Vendor offerings are much more about catering to the CTO or some other manager level employees, and focusing on b.s. on-paper "milestones" that the product can help them achieve. Many times, merely just the installation and integration of these tools can result in a multi-hundred-thousand dollar bonus, or more, for people high up the IT food chain -- even before anyone has seen whether it will bear any fruit or even come close to being cost effective.

Vendors will do this with anything they can, so it's not surprising they would do it with R technology and they also absolutely do it with Scala, Python, Spark, etc. Big vendors peddling all of that truly are the hotel lobby snack machines of the software world.

I'm not saying it's bad to have R as part of a professional computing environment. R is a great tool. I'm saying that for most business situations, people who tend to make the assumptions necessary to justify using R are often very wrong, and the clunky and inconvenient support for professional computer science and software design in R bites them, making Python a much more pragmatic choice.

Whatever vendors do is pretty irrelevant to this.

Re: Python, Machine Learning, and Language Wars (2015)

#59

Post by anon4 »

CalRobert wrote: post #5
danso wrote: post #3

As someone who's switched from Ruby to Python (for now, because the latter is far easier to teach, IMO) and also put significant time into learning R, because of how strong ggplot2 is...I was really surprised at the lack of Google results for "switching from python to r" -- or similarly phrased queries to find guides on how to go from Python to R...in fact, that particular query will bring up more results for R -> Py…

You know, I remember when I was trying my first language other than BASIC (VB6, perhaps? or maybe 1995 era JS?) and it bugged me that "x = y" wasn't the same as "y = x". Remembering it as "LET x = y" was helpful.

I bet that '=' is now used more in programming than in math. In that case, isn't its original meaning the wrong one today?

Re: Python, Machine Learning, and Language Wars (2015)

#60

Post by TheLogothete »

p4wnc6 wrote: post #58
TheLogothete wrote: post #23

Earlier quoted context omitted.

Vendors are standardizing on R, so I don't think the situation is as severe as you make it out to be.

Vendors tend to standardize on crap. How many vending machines do you see with salad or tofu instead of Doritos or soda? Big multinational IT vendors are even worse than the vending machine example. They cater to situations where a firm has some internal power struggle over IT and the struggle is won by one side who gets the certified approval of a statusy external vendor. For instance, most big, showy technology re-…

I guess we have to agree to disagree.
Post reply on HN