Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The largest Git repo

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

251–260 of 416 posts

Re: The largest Git repo

#251

Post by e40 »

manyoso wrote: post #199
garyclarke27 wrote: post #45

Linus must be very proud - his favourite software Windows - now depends on GIT.

I am very certain that Linus would read this article and curse the jaw dropping stupidity of the whole endeavor. They've basically taken a tool he wrote to do real distributed source control that can scale and turned it into a central server. Git was never meant to be used this way and I know he'd be horrified+amused in the extreme.

centralization has nothing to do with this feature. It's about big repos, centralized or not.

Re: The largest Git repo

#252

Post by deepsun »

amag wrote: post #227
eridius wrote: post #176

Earlier quoted context omitted.

I just looked at git-mediate and I'm very confused. It appears that all it does is remove the conflict markers from the file after you've already manually fixed the conflict. Except you need to do more work than normal, because you need to apply one of the changes not only to the other branch's version but also to the base. What am I missing here, why would I actually want to use git-mediate when I'm already doing al…

It looks like git-mediate does one more important thing; it checks that the conflict is actually solved. In my experience it's very easy to miss something when manually resolving a conflict and often the choices the merge tools give you are not the ones you want.

IntelliJ IDEA default merger knows when all conflicts in a file are handled and shows you a handy notification on top "Save changes and finish merging".

Re: The largest Git repo

#253

Post by skybrian »

cubano wrote: post #237
skybrian wrote: post #228

Earlier quoted context omitted.

No, not really. When it scaled enough for the Linux kernel he lost interest in scaling any further, and that's nowhere near what's needed for a monorepo (or a Linux distribution). And I recall that there are was fairly infamous tech talk at Google where he basically dismissed Google's concerns in an arrogant way. So I'd say it's mostly a validation of git and Github's popularity with developers, that other people wer…

Well by "validating git" aren't you by proxy validating its design by its creator? Or am I missing something key here? Yes of course it's been extended by very smart people has new features etc etc...but a weekend project that was robust enough to handle the Linux kernel codebase for him? That's is no small feat. Oh yes perhaps its the new meme among developers that there are no real genius architects and coders and…

Sure, that weekend project was extremely successful, and git has a pretty clean design.

My point is that it didn't scale to monorepo size, and was never intended to do anything like that. (How many years did it take before Microsoft came along and decided to do it?) Making a claim like that is basically hero worship.

When software is popular enough, people will jump through all sorts of hoops to extend it and maintain compatibility with it. (For example, POSIX, JavaScript, Java, or even PHP.) These technologies all have their good parts and bad parts. The quality of the design is less important than whether it fulfilled a need successfully and appeared at the right time.

Re: The largest Git repo

#254

Post by nikital »

cryptonector wrote: post #194

At Sun Microsystems, Inc., (RIP) we have many "gates" (repos) that made up Solaris. Cross-gate development was somewhat more involved, but still not bad. Basically: you installed the latest build of all of Solaris, then updated the bits from your clones of the gates in question. Still, a single repo is great if it can scale, and GVFS sounds great! But that's not what I came in to say. I came in to describe the rebase…

A pain I have with rebase workflow is that it creates untested commits (because diffs were blindly applied to a new version of the code). If I rebase 100 commits, some of the commits will be subtly broken.

How do you deal with that?

Re: The largest Git repo

#255

Post by ethomson »

comex wrote: post #173
lloeki wrote: post #125

Coming from the days of CVS and SVN, git was a freaking miracle in terms of performance, so I have to just put things into perspective here when the topmost issue of git is performance . It's just a testament how huge are the codebases we're dealing with (Windows over there, but also Android, and surely countless others), the staggering amount of code we're wrangling around these days and the level of collaboration i…

Broadly speaking this is true, but note that in some ways CVS and SVN are better at scaling than Git. - They support checking out a subdirectory without downloading the rest of the repo, as well as omitting directories in a checkout. Indeed, in SVN, branches are just subdirectories, so almost all checkouts are of subdirectories. You can't really do this in Git; you can do sparse checkouts (i.e. omitting things when c…

CVS and SVN are probably a bit better at scaling than (stock) Git. Perforce and TFVC _are certainly_ better at scaling than (again, stock, out-of-the-box) Git. That was their entire goal: handle very large source trees (Windows-sized source trees) effectively. That's why they have checkout/edit/checkin semantics, which is also one of the reasons that everybody hates using them.

GVFS intends to add the ability to scale to Git, through patches to Git itself and a custom driver. I don't think this is a hack - by no means is it the first version control system to introduce a filesystem level component. Git with GVFS works wonderfully while offline for any file that you already have fetched from the server.

If this sounds like a limitation, then remember that these systems like Perforce and TFVC _also_ have limitations when you're offline: you can continue to edit any file that you've checked out but you can't check out new files.

You can of course _force_ the issue with a checkout/edit/checkin but then you'll need to run some command to reconcile your changes once you return online. This seems increasingly less important as internet becomes ever more prevalent. I had wifi on my most recent trans-Atlantic flight.

I'm not sure what determines when something is "a hack" or not, but I'd certainly rather use Git with GVFS than a heavyweight centralized version control system if I could. Your mileage, as always, may vary.

Re: The largest Git repo

#256

Post by ethomson »

qznc wrote: post #175
CiPHPerCoder wrote: post #3

Earlier quoted context omitted.

Sure, a couple of questions: 1. How do you measure "largeness" of a git repo? 2. How are you confident that you have the largest? 3. How much technical debt does that translate to?

I recently heard a story that someone tried to push a 1TB repo to our university Gitlab which then ran out of disk space. Sure, that might have been not be a usable repo but only an experiment. Still, I would bet against the claim that 300GB is the largest one.

300 GB is not the size of _the repository_. It's the size of the code base - the checked out tree of source, tests, build tools, etc - without history.

It's certainly possible that somebody created a 1 TB source tree in Git, but what we've never heard of is somebody actually _using_ such a source tree, with 4000 or more developers, for their daily work in producing a product.

I say this with some certainty because if somebody had succeeded, they would have needed to make similar changes to Git to be successful, though of course they could have kept such changes secret.

Re: The largest Git repo

#257

Post by happycube »

wfunction wrote: post #222
cubano wrote: post #211

Isn't this perhaps the greatest validation of Linus's design genius that what was initially a weekend project[0] has successfully scaled to this? They could no longer use their revision control system BitKeeper and no other Source Control Management (SCMs) met their needs for a distributed system. Linus Torvalds, the creator of Linux, took the challenge into his own hands and disappeared over the weekend to emerge th…

> Isn't this perhaps the greatest validation of Linus's design genius that what was initially a weekend project has successfully scaled to this? I thought the entire point of the article was to show how git didn't scale, and how they're basically rewriting the project and changing it as much as necessary to make it scale. It's not like Linus designed git to scale as O(modified).

On the other hand, becoming something bigger out of his hands is validation in it's own right...

Re: The largest Git repo

#258

Post by ern »

colechristensen wrote: post #212
Goronmon wrote: post #153

Earlier quoted context omitted.

No, it should be hard to make arbitrary changes. If you change the word "arbitrary" to "necessary" (implying a different bias than the one you went with) then all of a sudden this attitude sounds less helpful. Similarly "easy to limp along with a bad architecture" could be re-written as "easy to work with the existing architecture". At the end of the day, it's about getting work done, not making decisions that are th…

You have to balance getting work done vs. purity, and Microsoft has spent years trying to fix a bad balance. Windows ME/Vista/8 were terrible and widely hated pieces of software because of "getting things done" instead of making good decisions. They made billions of dollars doing it, don't get me wrong, but they've also lost a lot of market share too and have been piling on bad sentiment for years. They've been pivot…

I assumed that Windows 8 was hated because it broke the Start Menu and tried to force users onto Metro.

Re: The largest Git repo

#259

Post by nxc18 »

andrey_utkin wrote: post #236
nxc18 wrote: post #159

Earlier quoted context omitted.

The NT kernel is surprisingly small and well-factored to begin with - it is a lot closer to a 'pure' philosophy (e.g. Microkernel) than something like Linux to begin with. If you have a problem with Windows being overcomplicated or in need of refactor it is almost certainly something to do with not-the-kernel. If you look at something like the Linux kernel its actually much larger than Windows. It needs to have every…

Could you please share a link to NT kernel sources so that I take a look? It is not true that internal architecture of Linux drivers is not clearly defined. It is just a practical approach to maintenance of drivers (as an author of one Linux hardware driver I'm pretty sure the best possible). Reasoning is outlined in famous "Stable API nonsense" document http://elixir.free-electrons.com/linux/latest/source/Documen...…

If you are sufficiently motivated, NT 4 leaked many years ago and you could find it; it even has interesting things like DEC Alpha support & various subsystems still included IIRC. Perhaps you could find a newer version like 5.2 on GitHub or another site, but beware, as a Linux dev/contributor, you probably don't want to have access to that.

FWIW, I've stumbled upon both of those things in my personal research while I had legitimate access to the 5.2 sources as a student. It turns out Bing will link you directly to the Windows source code if you search for .

Yes, I'm in the process of reporting this to Microsoft and cleansing myself of that poison apple.

Re: The largest Git repo

#260

Post by graycat »

Sounds like a lot of good work.

But, in "Git repo", what the heck is a "repo"? A repossession as in repossessing a car?

In the OP with "Everything you want to know about Visual Studio ALM and Farming", what is ALM -- air launched missile? What do air launched missiles and "farming" have to do with Visual Studio?

To Bill Gates and Microsoft: For my startup, I downloaded, read, indexed, and abstracted 5000+ Web pages from the Microsoft Web site MSDN. That took many months. Then I typed in the software for my startup, 24,000 programming language statements in Visual Basic .NET 4 and ADO.NET (Active Data Objects, for getting to the relational data base management system SQL Server) and ASP.NET (Active Server Pages, for building Web pages) in 100,000 lines of typing. For that work, all of it that was unique to me and my startup was fast, fun, and easy.

Far and away the worst problem in my startup, that delayed my work for YEARS, was the poor quality of the technical writing in the Microsoft documentation.

Some of the worst of the documentation was for SQL Server: Gee, I read the J. Ullman book on data base quickly and easily while eating dinner at the Mount Kisco Diner. But the Microsoft documentation was clear as mud. Just installing SQL Server ruined my boot partition: SQL Server would not run, repair, reinstall, or uninstall, and I had to reinstall all of Windows and all my applications and try again, more than once.

Quickly I discovered that documentation of logins, users, etc. were a mess: Basically the ideas seemed to be old capabilities, attributes, authentication, and access control lists, but nothing from Microsoft was any help at all. Eventually via Google searches I discovered some simple SQL statements, I could type into a simple file and run with the SQL Server utility SQLCMD.EXE; that way I got some commands that worked for much of what I needed. Now those little files are well documented and what I use. For getting a connection string that worked, again the documentation was useless, and I tried over and over with every variation I could think of until, for no good reason, I got a connection string to work. Once I tried to get a new installation of SQL Server to recognize, connect to, and use a SQL Server database from the previous installation of that version of SQL Server, but the result just killed the installation of SQL Server.

Again, once again, over again, yet again, one more time, far and away the worst problem in my startup is making sense out of Microsoft's documentation. I found W. Rudin, Real and Complex Analysis fast, fun, and easy reading; Microsoft's documentation was an unanesthetized root canal procedure -- OUCH!

So, again, once again, over again, yet again, one more time, please, Please, PLEASE, for the sake of my work, Microsoft, and computing, PLEASE get rid of undefined terms and acronyms in your technical writing. Get them out. Drive them out. Out. Out of your writing. Out of your company. Out of computing. No more undefined terms and acronyms, none, no more. I can't do it. You have to do it. Then, DO IT.

Post reply on HN