Live data from Hacker News

Ask HN: What's your "it's not stupid if it works" story?

news.ycombinator.com

181–190 of 510 posts

Re: Ask HN: What's your "it's not stupid if it works" story?

#181
Scraping output meant for a printer from old software/systems for "ETL" purposes has been very useful. A few of the uses I can think of during my career:

- Apple II-based database used by a choir teacher to track the school music library

- MS-DOS accounting software payroll reports to generate W-2 forms

- Patient records from a pediatric office

- Receipt printer on a gas station pump controller

- Customer transactions and balances from a home heating propane supplier

- Customer transactions from unattended fueling site controllers

You might think this only applies to old software, but often "printing" to a Generic/text-only printer in Windows gives good results.

Re: Ask HN: What's your "it's not stupid if it works" story?

#182
post #131

I created the most popular Turkish social platform, Eksi Sozluk, using a single plaintext file as its content database back in 1999. It had taken me only three hours to get it up and running without any web frameworks or anything. It was just an EXE written in Delphi. The platform's still up albeit running on .NET/MySQL now and getting banned by Erdogan government for baseless or false reasons (like "national securit…

Oh, I did something similar. I built quite popular local (non-english language) gaming forum with an Access file hosted in a Windows server and a VBScript ASP file, which had just been released. That's the original version, before ASP.NET. I was 13 or 14 years old at the time and didn't know better. It was no SQLite, so I had some weird concurrency problems. On top of that I ran into the some size limit (was it 2GB?) pretty quickly, but at this point it was time to look for a bigger server and figure out real databases anyway.

It eventually stopped being popular under my administration, so I transferred the domain to some people around 1999. It was rebuilt with PHPBB or something and got a new life. It's still on, surprisingly.

Re: Ask HN: What's your "it's not stupid if it works" story?

#183
I'm in the middle of such a story right now. I'm doing research on a data set of COVID-19 hospital patients with multiple blood samples over time from each patient. The obvious thing we want to do with this data is to line up all the samples on a single timeline so we can see how the data changes over the course of COVID-19 from infection to resolution. Unfortunately, as with most infectious diseases, no one knows exactly when they were actually infected, which means we can't just sort them by time since infection and be done with it.

So, we set out to find some way of inferring the timeline from the data itself (RNA-seq and other molecular assays from the blood, in this case). The first thing we tried was to apply some standard methods for "pseudo-time" analysis, but these methods are designed for a different kind of data (single-cell RNA-seq) and turned out not to work on our data: for any given patient, these methods were only slightly better than a coin flip at telling whether Sample 2 should come after Sample 1.

Eventually, we gave up on that and tried to come up with our own method. I can't give the details yet since we're currently in the process of writing the paper, but suffice it to say that the method we landed on was the result of repeatedly applying the principle of "try the stupidest thing that works" at every step: assuming linearity, assuming independence, etc. with no real justification. As an example, we wanted an unbiased estimate of a parameter, and we found one way that consistently overestimated it in simulations and another that consistently underestimated it. So what did we use as our final estimate? Well, the mean of the overestimate and the underestimate, obviously!

All the while I was implementing this method, I was convinced it couldn't possibly work. My boss encouraged me to keep going, and I did. And it's a good thing he did, because this "stupidest possible" method has stood up to every test we've thrown at it. When I first saw the numbers, I was sure I had made an error somewhere, and I went bug hunting. But it works in extensive simulations. It works in in vitro data. It works in our COVID-19 data set. It works in other COVID-19 data sets. It works in data sets for other diseases. All the statisticians we've talked to agree that the results look solid. After slicing and dicing the simulation data, we even have some intuition for why it works (and when it doesn't).

And like I said, now we're preparing to publish it in the next few months. As far as we're aware (and we've done a lot of searching), there's no published method for doing what ours does: taking a bunch of small sample timelines from individual patients and assembling them into one big timeline, so you can analyze your whole data set on one big timeline of disease progression.

Re: Ask HN: What's your "it's not stupid if it works" story?

#184

We have a production service running for years that just mmaps an entire SSD and casts the pointer to the desired C++ data structure. That SSD doesn't even have a file system on it, instead it directly stores one monstrous struct array filled with data. There's also no recovery, if the SSD breaks you need to recover all data from a backup. But it works and it's mind-boggingly fast and cheap.

LMDB has a mode designed to do something similar, if anyone wants something like this with just a bit more structure to it like transactional updates via CoW and garbage collection of old versions. It's single writer via a lock but readers are lock/coordination free. A long running read transaction can delay garbage collection however.

Re: Ask HN: What's your "it's not stupid if it works" story?

#185
post #114

Earlier quoted context omitted.

Wow. How do design decisions get made that result in these types of situations in the first place?

Honestly it’s not too far off from what many databases do if they can. They manage one giant file as if it’s their own personal drive of memory and ignore the concept of a filesystem completely. Obviously that breaks down when you need to span multiple disks, but conceptually it really is quite simple. A lot of the other stuff file systems do are to help keep things consistent. But if there’s only one “file“ and you…

Yeah, a lot of database storage engines use O_DIRECT because the OS's general purpose cache heuristics are inferior vs them doing their own buffer pool management. That said if you try this naively you're likely to end up doing something a lot worse than the Linux kernel.

Re: Ask HN: What's your "it's not stupid if it works" story?

#186

This is a little different than what you are asking but when I was a kid my C64 would crash if it was on too long and I would lose all of my work. If I ran my mother's vacuum cleaner plugged into the same outlet as the C64 then it would not crash. No I cannot explain it but it worked so that vacuum cleaner would get turned on before I started to save my work to tape.

The vacuum probably lowered the voltage, preventing a component from overheating.

Re: Ask HN: What's your "it's not stupid if it works" story?

#187
post #158

Back when I was a junior engineer I made a small system for dealing with industrial printers that took either direct printer language commands or pdfs and exported the printer driver output to a file via the Redmon printer driver output file exporter and then ftped it to the printer over the network. This avoided manual driver installation of hundreds of printers and may still be sold in a new project. It's hacky af…

I love Redmon!

I had a Customer who used Zebra industrial label printers for labeling product. The print jobs came from one of their Customers' ERP servers, sent over a VPN directly to the production line.

The Customer might send hundreds of labels in a single job. If the roll of label stock ran out during the job their server would re-send the job from the beginning after the printer was reloaded. This meant somebody had to find and dispose of the duplicate labels (or risk re-using a serialized label).

The Customer said that they couldn't modify the ERP software that was composing the jobs.

A friend and I wrote a parser for the Zebra "ZPL" printer language to ingest the large jobs, split them into single label jobs, then shoot those single labels into the printer. We used Redmon to intercept the jobs coming from the Customer's ERP server into an LPR queue on a Windows server machine. Redmon would hand the job off to the label splitter.

Re: Ask HN: What's your "it's not stupid if it works" story?

#188

I'm in the middle of such a story right now. I'm doing research on a data set of COVID-19 hospital patients with multiple blood samples over time from each patient. The obvious thing we want to do with this data is to line up all the samples on a single timeline so we can see how the data changes over the course of COVID-19 from infection to resolution. Unfortunately, as with most infectious diseases, no one knows ex…

I'd love to read it when it comes out. Where should I look for it when its published? Ignore if its a stupid question.

Re: Ask HN: What's your "it's not stupid if it works" story?

#189
post #131

I created the most popular Turkish social platform, Eksi Sozluk, using a single plaintext file as its content database back in 1999. It had taken me only three hours to get it up and running without any web frameworks or anything. It was just an EXE written in Delphi. The platform's still up albeit running on .NET/MySQL now and getting banned by Erdogan government for baseless or false reasons (like "national securit…

Is there a reason why they are not taking the 1999 version of the domain down?

Re: Ask HN: What's your "it's not stupid if it works" story?

#190
In the early 2000s I worked at an on-campus lab that did 3D printing back when that required machines that were 10s to 100s of thousands of dollars. One of the machines built blocks of paper that were each laser cut and laminated to one another. The company that made it went out of business and there wasn’t any software support. Many of the supported printers had tools to email when various events happened and we used that to page employees to come in and remove finished prints or whatever needed to be done.

This machine didn’t have that capability nor any obvious extension points. I ended up writing a VB app that would pill the serial port, which the machine used to talk to the control system, and if the serial port was busy and then became free, send an email. Email was sent by writing a very simple SNMP client.

That program ended up working for another, lower cost 3D printer that we acquired later as well.

I ended up extending for a 3ᴿᴰ printer to tail its log and look for a message it emitted when prints finished.

We share it with a few places and got one of the printer companies to add email support later.

Post reply on HN