Live data from Hacker News

"We ran out of columns"

jimmyhmiller.github.io

321–330 of 588 posts

Re: "We ran out of columns"

#321
In case anyone is curious about what the proper solution to some of these problems is in modern SQL Server:

1) Many columns can happen with per-customer customisations to a shared table. The common way is to have a "customcolumns" table with "ID,ColumnName,ColumnValue" linked to the base table that has an "ID" key, but SQL Server also supports this natively now with Sparse Columns: https://learn.microsoft.com/en-us/sql/relational-databases/t...

2) Shared identity or globally sequential numbers have a built-in schema object type now: https://learn.microsoft.com/en-us/sql/t-sql/statements/creat...

3) Manually populated calendar tables are actually a common schema design pattern, especially in manufacturing, shipping, and OLAP reporting systems. This is not that bizarre, it's just a bit weird that it would break logins! These tables can let you define all sorts of things such as international holidays, manufacturing calendars, tax years, finance reporting schedules, etc...

4) Dropping and recreating tables is also common, albeit usually done in a transaction. The fancy way to do this is with partition switching, where a new table is populated from whatever (external data, business logic, etc...) and then instantly swapped for the original without any long-running operations like truncate & insert would. See: https://pragmaticworks.com/blog/table-partitioning-in-sql-se...

5) Delayed reporting replicas ("here was a copy of the database. Data in this copy was about 10 minutes out of date.") is also a common pattern. At this point, the blog author is just complaining about the realities of business databases. Typically you'd have a bunch of read only replicas with different delays: Synchronous for HA failover, Asynchronous for DR failover and real-time reporting, and deliberately delayed on a schedule ETL copies for "point in time" consistent reporting. These would typically be done at 3am or something to minimise inconsistencies in the data. The modern way to do this is a SQL Server Always On readable secondary: https://learn.microsoft.com/en-us/sql/database-engine/availa...

6) "The main codebase I worked in was half VB, half C#." this is also surprisingly common. It's often not worth rewriting old code, there's not enough return on the invested money. These days there are automated conversion tools for VB to C#, such as: https://marketplace.visualstudio.com/items?itemName=SharpDev...

7) The Gilfoyle character is honestly the only thing that stands out here as an actual problem, not just a typical thing that happens in large enterprise bespoke software development.

Re: "We ran out of columns"

#322

When I started at my first company, they had a very complex VB application running on dozens of customers around the country, each having some particular needs of course. There was a LOT of global variables (seemingly random 4 uppercase letters) controlling everything. At some point, the application had some bugs which were not appearing when the application was run in debug mode in Visual Studio. The solution was ob…

As much as this stuff is nuts to think of today and there's tons to hate, I am kinda nostalgic for some aspects of my experience of working at a place where software is maybe needed and/or valued but isn't a core competency. Or maybe a time when software was a new fangled thing that hadn't fully been integrated into corporate structure yet: - No one having any preconception of how you're /supposed to/ do things or wh…

Don’t think so. My first job was kind of like that. I don’t even know how they thought that little old me just out of university could be left alone to successfully build applications on my own, but I think people trusted a lot more during that era because eternal september hadn’t arrived yet.

Working directly for the users without any weird BA/PM/TA shit in between is glorious, both because you can always walk up to get immediate feedback (people generally like to see you are actively working on their issue), and in a place like that you can likely deploy it in the middle of the day and immediately improve their workflow.

It still amuses me that IT was located together with finance, because we did reports xD

Re: "We ran out of columns"

#323
post #316

Earlier quoted context omitted.

You don’t think if the company behind GitHub went all-in on mercurial that might have emerged as the winner? There were plenty of companies using either. Git wasn’t the only distributed game in town. I definitely think GitHub had a lot to do with it.

Before there was GitHub there was Sourceforge. Sourceforge supported several different protocols and git still won. Gits internals can be confusing at first, but once you understand them there's really nothing you can't do. Being distributed by design also helps.

yeah, git is just the right layer of indirection to support everyone's weird take on how the workflow should go, so you can just use it however you want.

Re: "We ran out of columns"

#324
post #211

Earlier quoted context omitted.

This is amazing. I can so well imagine a bright young hire joining that team, helpfully offering to "setup this thing called git" only to be laughed out of the meeting by all the "senior" staff.

Astonishingly, It took a long time for revision control to become widespread. Around 1991 when Cygnus had 6-7 employees and was based in the apartment complex where I lived, none of the GNU codebase was hosted in any sort of revision control. Everything was FTPed around as obscurely named tarballs. We had gathered something like 27 different forks of gdb floating around the net, for example. This was back when forkin…

My first job out of college was on a university administrative system where all the source files were managed by RCS. I came on in '97, but the system had been operation in at least that state to 1995. Obnoxiously, the vendor layered some secret sauce on top of it in Makefiles for the actual checkins and outs, so you did something like `make co F=foo.c` which totally messed up filename completion. I had it aliased in tcsh within minutes to preserve my own sanity.

When my consultant contract finally came to an end in 2003 they were still using it.

Re: "We ran out of columns"

#325
post #277

Earlier quoted context omitted.

Why is it astonishingly? Git is not a natural development at all. Obviously, it is a standard right now. But I as a hobby coder at my teens I started out with FileZilla and copying over index2.php, index3.php, index_final.php, index_final_2.php and all of it worked well enough when at that point. I took a little break from that hobby, and it still took me a lot of time to build intuition around Git when I started out…

> Git is not a natural development at all. Obviously, it is a standard right now. Git is actually an unnatural development. Its UI is atrocious. And it worked like crap on Windows for forever. Over time, I taught non-computer people who used Windows all of CVS, Subversion, and Mercurial. They got each one and why things were better. The first time they had to recover something, they really got it. Source control got…

GitHub won because they built a great product. It was better than all the alternatives and people flocked to it for that reason.

Git itself is mediocre, and some other DVCS like Mercurial could have won out, although HgHub really doesn't have the same ring to it. The halo effect from the Linux kernel was also a factor. But the VC cash dump and Microsoft buyout came later, people used GitHub because it was better than the rest.

Re: "We ran out of columns"

#326
This made me think of my first job. I was the sole developer on a project because the old developer left. Nothing was documented and nobody knew why things were designed the way they were.

We had no code reviews, no design docs, no tests, nothing. We made the changes the way we thought they were right and would git pull them onto the production server.

After I struggled to get productive for the first four months, my manager went on a four-week Christmas vacation. In a moment of frustration, I seized the opportunity and rewrote the whole project from scratch. I don’t remember if my manager ever noticed, but that was the moment I finally got productive.

Re: "We ran out of columns"

#327
post #77

Probably some of the worst code I ever worked on was a 12k+ line single file Perl script for dealing with Human Genome Project data, at Bristol-Myers Squibb, in the late 1990s. The primary author of it didn't know about arrays. I'm not sure if he didn't know about them being something that had already been invented, or whether he just didn't know Perl supported them, but either way, he reimplemented them himself on t…

> and it still passed all the tests All his sins can be forgiven because he made it so easy to refactor by writing comprehensive tests.

And lo, on the first day god created the test

Re: "We ran out of columns"

#328
post #206

Earlier quoted context omitted.

Visual Basic was a smash hit released in 1991. For reference, Perforce was released in 1995, Subversion in 2000, git in 2005. RCS existed, but only in the Unix world.

Don't forget Visual Sourcesafe, that came out around 1995 as well and was the standard source control package for most of the Windows shops in the 90s and even up to the mid 2000s (at least in my experience)

Until about 2014 or so in my department, when we started using SVN. We were more concerned about file locking and a "single version of the truth" of MS Word files stored centrally but accessed locally than we were of fine-grained version control, and Git didn't have the kind of file locking we needed (as I understood it).

Re: "We ran out of columns"

#329

Earlier quoted context omitted.

> people would shove entire CSV rows into a single value So you invented a “no-sql” document database in Oracle.

We got rid of most of the DBAs because they got overfond of saying 'no' and they became a major source of friction for doing reasonable architecture. People started figuring out ways to work around the DBAs like stuffing more shit into existing columns or adding secondary tables that established new relationships between the sacrosanct tables. A lot of the worst sins happened on Oracle databases, mostly because they…

I have to work with a DBA who has decided that nothing new gets developed using Postgres and is confident that our use cases are best served with a document db… all without knowing any of our use cases, requirements or constraints.

Now I just don’t involve him in anything unless forced.

Re: "We ran out of columns"

#330
post #155

My first day at my first full-time programming gig, I was asked to look at some reporting job that had been failing to run for months. I logged in, found the error logs, found that it needed a bit more memory assigned to the script (just a tweak in the php.ini) and let the team lead know it should run fine that night. He was shocked, "Dude, if you just fixed that report you probably just got a promotion, no one has b…

I know things have gotten somewhat better, but the amount of wasted time and latency of using RDP and Windows UI for development, testing and production maintenance is insane. Throw in some security requirements of RDP into host 1 to the RDP jump to host 2 and companies are just wasting money on latency. There is, often, not an appreciation of the administrative costs of the delivery. Not necessarily system admin cos…

Windows programming has a natural job security dependency that is often overlooked.
Post reply on HN