Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MS Python: How to deal with the pain of “unable to find vcvarsall.bat”

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

1–10 of 25 posts

Re: MS Python: How to deal with the pain of “unable to find vcvarsall.bat”

#3

Post by wyldfire »

I guess I knew in general about wheels but didn't see how it would solve "usability" problems like this one.

Does `pip install lxml` fail in a similarly perplexing way if you don't have `build-essential` (`gcc` et al) installed? If so maybe the difference is that the posix/linux crowd is more likely to have installed a compiler at this point.

Here's hoping MSFT's Linux ABI support evolves into the norm for a port target.

Re: MS Python: How to deal with the pain of “unable to find vcvarsall.bat”

#4

Post by dsp1234 »

wyldfire wrote: post #3

I guess I knew in general about wheels but didn't see how it would solve "usability" problems like this one. Does `pip install lxml` fail in a similarly perplexing way if you don't have `build-essential` (`gcc` et al) installed? If so maybe the difference is that the posix/linux crowd is more likely to have installed a compiler at this point. Here's hoping MSFT's Linux ABI support evolves into the norm for a port tar…

  compilation terminated.
  
  *********************************************************************************
  
  Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
  
  *********************************************************************************
  
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  
  ----------------------------------------
  Cleaning up...
  Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_test/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-QEyvN6-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_test/lxml
  Storing debug log for failure in /home/test/.pip/pip.log
There is some stuff above in the log, but the nothing about build-essentials. And a new dev would likely have to hit google/stackoverflow.

Re: MS Python: How to deal with the pain of “unable to find vcvarsall.bat”

#6

Post by petetnt »

wyldfire wrote: post #3

I guess I knew in general about wheels but didn't see how it would solve "usability" problems like this one. Does `pip install lxml` fail in a similarly perplexing way if you don't have `build-essential` (`gcc` et al) installed? If so maybe the difference is that the posix/linux crowd is more likely to have installed a compiler at this point. Here's hoping MSFT's Linux ABI support evolves into the norm for a port tar…

> If so maybe the difference is that the posix/linux crowd is more likely to have installed a compiler at this point.

The long running issue with Windows was that there weren't any compilers available (AFAIK) outside of installing Visual Studio first. When Visual Studio started offering free versions, the problem persisted: it's rather hard to justify downloading and installing a +4gb program just to make an dependency or extension to work (even if it wasn't directly required for the said dep).

For example, this was a huge issue with node-gyp[1], which prompted Microsoft to release standalone Build Tools[2] to alleviate the problem. I think it's the same kind of issue with Python here, right?

[1] https://github.com/nodejs/node-gyp/issues/629 [2] https://www.microsoft.com/en-us/download/details.aspx?id=499...

Re: MS Python: How to deal with the pain of “unable to find vcvarsall.bat”

#7

Post by tdicola »

This is a very odd blog post in my opinion coming from MS. It acknowledges the pain of dealing with source distributions on Windows and gives excellent advice on how to overcome the issues. However I think it really misses the forest through the trees.

If MS was really serious about Python being a first class citizen on Windows then fix the root problem which the post acknowledges, ".... Because Windows has a different culture, most people do not have (or need) a compiler." Why doesn't Visual C++ express come installed or as a super easy to install dependency on Windows? Howabout MS pushing to change the culture of Windows to one where compiling from source isn't a big deal and is as easy as it is on Linux?

Also it's super helpful that this post lays out where to find precompiled versions of Python packages for Windows, but again it's something if MS were really serious about they would step up their game significantly. Why is it that some random person, Christoph Gohlke (and as a side note many thanks to them, they have undoubtedly helped thousands of people), is the maintainer of one of the most important resources for Python users on Windows? How about MS steps up and dedicates resources to ensuring precompiled and tested python binary packages are available for all packages?

Finally I think the message that people should bug Python library owners to get windows wheels up is very poor advice. Library maintainers have enough to deal with and piling on more frivolous issues won't help. What if the maintainers don't even have Windows machines to build binary packages with (and again, why doesn't MS step up to provide them with said Windows machines/licenses)? Badgering the maintainers isn't going to make things better. If MS is serious about python on Windows they need to step up their support and make the platform better.

Re: MS Python: How to deal with the pain of “unable to find vcvarsall.bat”

#8

Post by xenophonf » 

  If none of these options is available, you will need
  to consider building the extension yourself.  In many
  cases this is not difficult...
This is horseshit.

I spent a few weeks at the beginning of the year building MSIs for SaltStack and its dependencies, as I wanted a fully native way to bootstrap a Windows client installation via GPO using a native Python installation instead of SaltStack's NSIS executable installer, which uses an esky build. In short:

* The compiler included in Microsoft Visual C++ Compiler for Python 2.7 is not compatible with distutils.

* The compiler included in version 7.0 of the Microsoft Windows SDK is not compatible with distutils.

* The compiler included in version 7.1 of the Windows SDK is not compatible with Python, nor is it compatible with distutils.

The only combination of compiler plus SDK that I could get to work properly was Visual Studio 2008 and Windows SDK 7.1, and that still didn't work quite right due to bugs in the code underlying bdist_msi, source distributions not including everything used to actually build a given package, incorrect package build instructions, and so forth. (For example, I seriously doubt that the pywin32 developers even follow their own package build instructions.)

In case you're curious:

https://gist.github.com/xenophonf/c2351f8af80d3231b547

Ultimately, I've given up and switched back to SaltStack's executable installer, run out of a computer startup script (a page out of my own book on 90s-era NetWare sysadmin, yuck).

Re: MS Python: How to deal with the pain of “unable to find vcvarsall.bat”

#10

Post by staticautomatic »

wyldfire wrote: post #3

I guess I knew in general about wheels but didn't see how it would solve "usability" problems like this one. Does `pip install lxml` fail in a similarly perplexing way if you don't have `build-essential` (`gcc` et al) installed? If so maybe the difference is that the posix/linux crowd is more likely to have installed a compiler at this point. Here's hoping MSFT's Linux ABI support evolves into the norm for a port tar…

`pip install lxml` will indeed fail without VC++ for Python. In fact, sometimes it fails even with VC++ (e.g. installing lxml from within PyCharm). However, executing the binary installer or using `pip install lxml_package_whatever.whl` works just fine.
Post reply on HN