Live data from Hacker News

Introducing PackageManagement in Windows 10

blogs.technet.com

171–180 of 221 posts

Re: Introducing PackageManagement in Windows 10

#171
post #92

Earlier quoted context omitted.

> These features would have to be thrown away if it unified pip, npm, etc. because they make no such guarantees. Again I disagree. For example in this case, it would just mean that when installing from nix repos you get the reproducibility etc guarantees, installing from upstream repos would get you vanilla version and installing from OS repos would get you an integrated/patched version. But the installation process…

If we've any hope to unify package management, we need to get to the essence of what package management is. It's really quite simple though - it's the ability to say that one piece of software depends upon another, and to have a piece of software which can automatically resolve the dependencies (which form a DAG). To construct our DAG we need a list of nodes (the packages), and a list of edges (the dependencies of a…

I think you are massively overthinking this. Dependency tracking and all that jazz are just implementation details of individual backends, the unifying tool does not need to be aware of that at all.

Let's say we define basic API with two verbs: SEARCH and INSTALL. When the user wants to install a package the unifying tool first queries backends with SEARCH to see if they have such package, and after resolving which source/backend it wants to install it from (a process that might involve user interaction), and then invokes INSTALL to that backend. Nowhere in this process the unifying tool needs to know what black magic the backend needed to do to get the package installed.

Re: Introducing PackageManagement in Windows 10

#172
post #171

Earlier quoted context omitted.

If we've any hope to unify package management, we need to get to the essence of what package management is. It's really quite simple though - it's the ability to say that one piece of software depends upon another, and to have a piece of software which can automatically resolve the dependencies (which form a DAG). To construct our DAG we need a list of nodes (the packages), and a list of edges (the dependencies of a…

I think you are massively overthinking this. Dependency tracking and all that jazz are just implementation details of individual backends, the unifying tool does not need to be aware of that at all. Let's say we define basic API with two verbs: SEARCH and INSTALL. When the user wants to install a package the unifying tool first queries backends with SEARCH to see if they have such package, and after resolving which s…

SEARCH is the reason package management is so broken. If a developer intends a particular piece of software to be installed as a dependency, he should convey that to the users, rather than a vague criteria for which the user might hopefully get the right thing. The search needs to return only one result - the right one. If several package managers keep a package of the same name and version, we shouldn't need to keep adding criteria to narrow down our search until we get the right one (and someone could later add a package which meets all those criteria after you've published).

The heart of the problem is making a SEARCH which will always return one result, and to do this we either need to add N criteria which are collectively guaranteed to be unique - or to just stick a unique identifier there to begin with and treat the rest as information queryable from the content.

The use of a hash as an identifier solves several other problems such as having multiple repositories whereby each one could host packages of the same name and such. While we continue to rely on search without some means of uniquely identifying packages, we continue to have the social burden of making sure our repositories don't conflict with other peoples (usually the "official distro" repo - but when we're talking of unifying packages, that's a lot of repositories which could conflict. Unless we add our "vendor" flag into the mix, etc)

Re: Introducing PackageManagement in Windows 10

#173

Earlier quoted context omitted.

In that scenario I'd never have just a single server, be it Linux, Windows, BSD, or anything else. Being in a situation where you're completely unable to restart a machine without losing money, is being in a situation where you're living on a knife edge regardless. A single hardware failure and you're in deep trouble. I've been in exactly that situation (the entire company rested on a single Ubuntu server), it was re…

his point remains. being forced to run 2 servers so your server doesn't go down once/month is a bit ridiculous. No, it isn't untenable, yes you can work around it, but that's a problem that shouldn't need to be worked around.

Is it normal to update server software like PostgreSQL or Redis on Linux without stopping the running process during the update?

Maybe Linux doesn't need to restart the OS as often, but you're still going to have down time, right?

Re: Introducing PackageManagement in Windows 10

#174
post #150

Earlier quoted context omitted.

> Is application dependency management in windows normally a problem? It has its own name. http://en.wikipedia.org/wiki/DLL_Hell

DLL hell certainly used to be a big issue in Windows. But I disagree that it remains a major factor; Windows apps are increasingly bundled and Windows system functions that used to be the cause of these issues, like DirectX, are packaged as redistributables that allow side-by-side installs.

How do they handle security updates? Does every app have to update all its own libraries in an e.g. heartbleed scenario?

Re: Introducing PackageManagement in Windows 10

#175
Well, one small step.

I first encountered package management in SGI IRIX. IRIX had a "mini root", which was an installer, subset of IRIX, running single user mode. Just know it's there for now, and hold that thought.

Most of the time, people launched "swmgr", which put a nice GUI on top of "inst", which was the actual tool doing the work. Inst eventually could pull from an http repository, and of course, worked with file based ones, mounted however made best sense.

That thing was seriously potent! One time, I needed to setup Alias renderer software on a bunch of machines to push out a little movie. So I just did it, remote displaying the gui on my O2 after mounting a shared image to work from.

Asked them all to load it, and about half reported out of space. Right MID INSTALL, I was given the option to do some removals, so I did, at which point the primary task could continue. Short story was it took about 30 minutes to blast some renderer software onto about 10 IRIX boxes, all with users logged in, doing stuff.

(Yes, I looked at what they were running, and pulled some sub-systems that wouldn't impact them, and no, they didn't ever know it was done, but for some disk activity)

After authoring a little script to push frames around, I set 'em all to render with every free cycle, and as users left the building, added a renderer process and escalated the priority of both. It all got done, I pulled the software, and on the machines that I had pruned from, put that all back, and never heard a peep.

Awesome. No reboots.

The other notable thing was taking my very first IRIX filesystem from a modest Indy, all the way through to a multi-CPU Origin system. Each time, I would clone it, setup on a new box, launch that mini-root, have it evaluate what was there, issue a couple of commands to keep configuration, update system, go. And it would update all the libraries, drivers, and assorted bits to run on the new box.

From time to time, I would need to do a little cleanup, maybe touching an unmanaged application and a setting or two, but I really did carry one file system through about 10 years of computing, using a package manager to keep it sorted with basically zero issues.

This was mid 90's.

Linux has most of those features today, (hello apt get!) and it's a bigger mess of packages, but then again, there is a whole lot more managed now than IRIX had. Some companies would produce managed packages and sync up with SGI, and those were great. Others were tarballs. Oh well. One could make packages and go that way, and I did a time or two, but mostly didn't.

The thing I liked the most about that era of software, and about IRIX in particular, was how just about every single thing was written to be effective and useful on the command line, and in scripted form.

Additionally, most everything had a --gui, or --verbose option, or both that would improve on the console I/O to the point where a gui could just be a wrapper, able to do what it needs to do without there being anything separate needed. When in single user mode, or remote console, terminal, whatever, really didn't matter. One could script, run a gui, command line, whatever, and it all just worked fine.

Back then, I saw Linux growing, and also later, saw Linux start to get the better bits of IRIX as SGI moved off IRIX and MIPS and onto Linux Itanium. That caused me to pick up a copy of Redhat 5.2 and begin switching over. One of the best moves I ever made actually.

So those days on IRIX are long gone, but I really do miss the fantastic systems engineering, package management, and documentation they shipped. It was complete, and if you went digging, what you needed to know actually shipped on the box with very, very few exceptions.

Windows... Yeah, it's better now. I run it all the time, and that's due to some software and the niche I'm in. No worries. But it really never did compare to the work SGI did in the 90's. Few things have.

I'm glad to see this. Just like I'm glad to see lots of things in Microsoft land, many way overdue.

I just wish it would launch a bit more complete and be a bit less painful before it really settles in and works well.

Maybe somebody should give these guys a tour of computing outside the bubble. There were a lot of really great things done. Seems to me, getting them done again, now, unabashedly is the right move. No shame. Just do it.

Lots of us would be a lot happier, and we really don't care where it comes from. Just nail it for those of us who have to continue to run the OS.

Thanks.

Re: Introducing PackageManagement in Windows 10

#176
post #92

Earlier quoted context omitted.

> These features would have to be thrown away if it unified pip, npm, etc. because they make no such guarantees. Again I disagree. For example in this case, it would just mean that when installing from nix repos you get the reproducibility etc guarantees, installing from upstream repos would get you vanilla version and installing from OS repos would get you an integrated/patched version. But the installation process…

I just think it's a mistake to believe that the solution to the proliferation of package managers is to write a package manager manager.

I'm not sure that anybody is advocating taking that route as the way forward...

Re: Introducing PackageManagement in Windows 10

#177

Perhaps I missed it, but the article doesn't mention this in any way: Windows ABSOLUTELY needs a way to go from 'Fresh Install' to 'Fully upgraded' with minimal interaction. Linux can do this: run `apt-get dist-upgrade` in Ubuntu, and you get all of the most recent updates. However, in Windows, you install 3 updates, restart, install 5 updates, restart, install 110 updates, restart....

A lot of the time, restarts are required to remove old versions of files that are still in use. AFAIK, there is no way to remove files that are 'in-use' (and not locked) without having to make an entry in the registry to schedule a deletion on the next reboot.

I always wondered about that.

Shouldn't a real OS and filesystem be capable of removing something in use?

IRIX always could. So can Linux a whole lot of the time, if not all the time, as can BSD and Mac OS.

Here's something:

Once I upgraded a pile of IRIX boxes to a new version of some CAD software. A few of them had the older version running...

Didn't matter!

The files got cached, and those versions continued running the next day, until shut down. Next launch, the new version popped up.

One could just run a program, then delete the program right out from under the running process, and that process didn't know. IRIX just cached it all, and when the process was done, cleaned up the cache.

Re: Introducing PackageManagement in Windows 10

#178
post #171

Earlier quoted context omitted.

I think you are massively overthinking this. Dependency tracking and all that jazz are just implementation details of individual backends, the unifying tool does not need to be aware of that at all. Let's say we define basic API with two verbs: SEARCH and INSTALL. When the user wants to install a package the unifying tool first queries backends with SEARCH to see if they have such package, and after resolving which s…

SEARCH is the reason package management is so broken. If a developer intends a particular piece of software to be installed as a dependency, he should convey that to the users, rather than a vague criteria for which the user might hopefully get the right thing. The search needs to return only one result - the right one. If several package managers keep a package of the same name and version, we shouldn't need to keep…

That's all fine and well but the thing is -

Is it possible to unify package management un-intrusively or not? Call the un-intrusive path A nd the intrusive path B.

A) If we were to to it in such a way that did not require us to modify all the existing package managers out there then how do we do that? I think the solutions (a database of unique hashes and so on) you are coming up with might be a way forward. But don't we still have the problem of communicating with the underlying package managers?

B) What sort of intrusion? Would package managers have to adhere to some kind of standard or API or expose a minimum amount of surface area? How would you get package manager maintainers to sign up to something like that? A summit? Who would fund such a summit? Then we'd still need something to implement that API. Somebody elsewhere mentioned PackageKit, would this fit the bill? If not, is there something else that would? And would we still need to track installs and so on with something like what you're proposing.

Remember this would ideally would on Debian-like (.deb apt-get) and RPM-like (.rpm yum) and Gentoo (.pkg emerge) and so on and so forth ad nauseum ... I mean, I'm running Ubuntu so after the Great Request for the Unification of Managing Packages Summit (GRUMPS) I want to still be using Synaptic but I want to be able drill down into sub-package managers. See what I mean?

Re: Introducing PackageManagement in Windows 10

#179
post #71

I recently blogged about the Linux world needing something like this. http://leto.electropoiesis.org/propaganda/plugins-and-packag... I hate having to individually update my Wordpress install, my Rubies, my system packages, my IDE (be it Eclipse or Android Studio), a separate TeXLive install from the OS packages, even Vim now has its own package management with Pathogen... There must be some way to _unify_ this proli…

> There must be some way to _unify_ this proliferation of software update mechanisms. There are many ways. The problem is there are many different environments who all use different methods for different reasons. To a solo developer with personal control over the entire stack, the Operating System is merely one more tool in the toolbox. That dev can pick any distro he wants and then install and configure anything he…

Aha! Lightbulb moment. I thought that Puppet and co. (chef and so on) were about pure configuration across multiple machines. I didn't realise they were about deployment as well. In that case I guess the problem is sort of solved but then I need to start thinking a layer higher than I have been.

It would be nice if the solution to this proliferation still allowed me to think at the level of [synaptic/apt-get/dpkg] on Debian/Ubuntu/... or [yum/rpm] on Redhat/CentOS/Suse/... and so on. Do you think this is unreasonable of me?

Re: Introducing PackageManagement in Windows 10

#180

Earlier quoted context omitted.

That's a false distinction. bash has a (primitive, idiosyncratic) language built-in. So does PS. They also are both designed to glue together independent programs via the pipe abstraction. The difference is that PS has much more modern language constructs and capabilities. The fact that it hooks into the .NET API as well is only a convenience.

> They also are both designed to glue together independent programs via the pipe abstraction PowerShell and its object pipeline are focused on cmdlets which are not arbitrary programs, they are .NET classes, and PowerShell objects are .NET objects. Unix shells like sh and bash are focused on tying together separate programs and only have a very minimal set of built-in commands. The focus of each one is very different…

> PowerShell and its object pipeline are focused on cmdlets which are not arbitrary programs, they are .NET classes, and PowerShell objects are .NET objects.

Both correct and horribly wrong.

PowerShell is indeed focused on cmdlets (and functions) and it's pipeline concept is focused on objects. It is also correct that cmdlets are not arbitrary programs.

However, cmdlets are not .NET classes. It is true that you can implement a cmdlet through a .NET class that derives from the base Cmdlet class. But that is just one way to create a cmdlet that can be recognized by PowerShell. You can build cmdlets using PowerShell itself. There is nothing in the PowerShell specification (http://www.microsoft.com/en-us/download/confirmation.aspx?id...) that require a cmdlet to be a .NET class.

If you use PowerShells (rich) introspection function to enquire about a cmdlet, there is nothing objective to give it away as either .NET class based or advanced function based (advanced functions are cmdlets). Yes, you can see if a module (distribution unit of cmdlets, providers, functions etc) are "binary" or "script". That merely reinforces the argument that cmdlets are not classes.

Indeed, PowerShell will now dynamically create a new type of cmdlets based on an Open Management Infrastructure (OMI) implementation. Think managed routers, switches, Linux boxes etc. If they expose a REST service that complies with OMI, PowerShell can query the capabilities and create cmdlets in-session for each individual type of equipment. PowerShell has since v3 been able to create WMI/CIM cmdlets.

The point is that your assertion that cmdlets are .NET classes is both factually wrong, and grossly misleading. Saying that PowerShell cmdlets are .NET classes is like saying that Yum packages are just archives. Sure, you can pick one apart and say "look! it's a .NET class". I can pick a Yum package apart and say: "look! it's just an archive with some files in it!".

That said, it is true that PowerShell builds upon .NET. It is implemented using .NET and the object model is based on .NET. PowerShell's type system is a superset of .NETs - because an administrators tool needs more ad-hoc typing.

We have gotten so used to think that sh shells were developed with pipelines like that (streams stdin/stdout) because the commands are like that. In reality, the commands and the (first) shells were developed in tandem so that the commands would fit the shell and vice versa.

What Microsoft is doing with PowerShell is challenge that commands must be "like that". If we make the commands "richer" and allow them to use a higher level protocol (objects) for inter-command communication, an even smarter shell can be built. Yes, it does create some friction with traditional commands. PowerShells solution is to regard those as "cmdlets" that just consume sequences of strings (which are objects) and produce sequences of strings.

> Unix shells like sh and bash are focused on tying together separate programs and only have a very minimal set of built-in commands.

PowerShell is focused on tying together separate cmdlets and only has a very minimal set of build-on commands. See?

(Aside: Actually, PowerShell is more terse than any sh shells: It has 0 (zero) "built-in" commands. What it is has is a module concept with auto-discovery. The seemingly "built-in" are not built in. Those cmdlets comes from modules that are distributed alongside the PowerShell binary, and PowerShell auto-loads them as it will any module that is placed in the correct location. sh shells needs commands like e.g. "cd" that manipulate the environment to be built-in. You could never implement "cd" as an external command. In PowerShell "cd" (alias for the Set-Location cmdlet) is distributed through the Microsoft.PowerShell.Management module)

> The focus of each one is very different, despite PowerShell's syntax. Piping objects between cmdlets in PowerShell is analogous to working with objects in Python, Ruby, Perl, etc, not like Unix pipes.

No it is not. A pipeline is exactly that: A temporary communications channel that is set up for the duration of the (compound) command, and through which multiple objects flow. In PowerShell the individual cmdlets - like commands in a sh pipeline - are active a the same time. From the PowerShell spec: "A pipeline is a series of one or more commands each separated by the pipe operator | (U+007C). Each command receives input from its predecessor and writes output to its successor."

Re: Python, Ruby, Perl, etc: You can create a pipeline concept for those languages as well. See for instance pypi (https://code.google.com/p/python-pipelines/). But it is like you want to portrait PowerShell pipelines as mere method invocations. They are considerably more.

Post reply on HN