Live data from Hacker News

Ask HN: What is the most complex software that you have built/worked on?

news.ycombinator.com

21–30 of 43 posts

Re: Ask HN: What is the most complex software that you have built/worked on?

#21

Not the most complex software, but definitely the most needlessly complex software. I worked for a company where IT owned customer onboarding for historical reasons. Each customer had their own DB in the same server so that each site would be isolated. So onboarding was basically cloning a “gold DB” and templating some values. The script that cloned the DB was over 50k lines of Batch files (Windows .bat). There were…

There is no way you replaced 50,000 lines of batch files with 10 lines of a script.

Re: Ask HN: What is the most complex software that you have built/worked on?

#22
post #20

Earlier quoted context omitted.

Oh wait. I'm working on a project right now. Never mind the PDF-to-braille thingy. How about a proxy server with HTTP, HTTPS, Socks5 and websocket? Not complex enough? How about routing to Tor and I2P? DNS and DNS-over-HTTPS? It's a local proxy server so it supports local disk access from a web browser. Trapping requests to modify them or write them to disk? The purpose is to support browser apps to do things like re…

If it's something you want others to use, why not reuse audited libraries? For something for which security is important, no one wants to audit your homegrown HTTP parser/TOR implementation/etc.

It's not as extensive as you're suggesting. HTTP parsing is handled by Node.js and it merely connects to an existing Tor server running in the background. The thing is, if I use any kind of third-party software then I end up with much more than I'm needing. For example, what is involved in running a websocket? Mention websocket to anyone and they'll tell you that you need a package like ws or something. Fact is, basic websocket functionality can be had with a few dozen lines of code. Same with DNS. Think you need a package for that?

Besides, even a well-designed library can be fatal for security if it is not handled correctly.

Re: Ask HN: What is the most complex software that you have built/worked on?

#24

Not the most complex software, but definitely the most needlessly complex software. I worked for a company where IT owned customer onboarding for historical reasons. Each customer had their own DB in the same server so that each site would be isolated. So onboarding was basically cloning a “gold DB” and templating some values. The script that cloned the DB was over 50k lines of Batch files (Windows .bat). There were…

There is no way you replaced 50,000 lines of batch files with 10 lines of a script.

I'm positive there's some exaggeration for effect, but I've run into some Powershell scripts for that have grown into 1000+ lines and I'd guess roughly 2/3rds of them were in functions never called.

The call is removed, but the function remains. Fields in a form stay, unused and unpopulated, but the validation is disabled and left behind. Things like this.

No version control meant it was hard to go back, and people are afraid to break things.

Re: Ask HN: What is the most complex software that you have built/worked on?

#26
A tower defense game whose core concept was from a chance reading of a Game Programming Gems book in the university library. Large numbers of bad guys in screen with solid frame rates, home built engine, and an AI that always attempts to thwart you by trying new paths into your base to defeat you (but also runs in a separated thread that pathfinds without hurting your framerate on a dual core or better system). Did I mention I started it on the old XNA library from Microsoft back when they offered the $99 license to make games?

Hasn't made a cent yet, but it has been a LOT of fun to build from scratch and tinker with over the years.

Re: Ask HN: What is the most complex software that you have built/worked on?

#27
Currently working on module for ERP system that handles milk price calculations. The idea is to collect milk from farms and then based on some qualititive measurements, and fat/protein and lactose to calculate the price for the farms. There's so many factors that the price calculations include like the measurements but also contracts, quantities etc. I think its the most complex stuff so far for me.

Re: Ask HN: What is the most complex software that you have built/worked on?

#30

Well im about to release the alpha but basicly its a project i worked on ~4 years in privat time and consists of two parts. 1. A custom in-memory graph storage/database which is threadsafe and designed for fast multithreading purposes. It also comes with a custom query builder/language which can be transportet via json so viable for every language. It can either be used by directly importing it as dependency in your…

>This was a strongly simplified explaination for what it does You missed the part about what the end result is supposed to be. What is the purpose? Can I use it to run a website?

Well you could say its for "data driven processing" and probably best suited for any kind of data processing especially data gathering/enrichment. What you could build with it is only limited by your imagination. Tho i will give a simple example (the one i gonne use for the example project i gonne provide).

A webcrawler. What does a webcrawler do? It expects a domain (data) and crawls for more data - analyzes it and enriches your collected data. You may end up with writing multiple plugins like.

- resolveIpFromDomain (takes Domain returns Domain->IP)

- detectWebserver (takes ip uses for example nmap to scan ports+banner) returns ip->port->software->(banner,state)

- detectVhost (takes ip->port->(software[webserver],state[open]) || domain->ip->(software[webserver],state[open]) and returns ip->port->(software[webserver])->[]vhost[]->page[/] ) -> loadPage (takes page loads it with curl and return page->content)

- extractLinks(takes page->content return page->content->[]link)

- loadLink (takes vhost->page->link returns vhost->[]page )

- extractMedia (takes page->content return page->content->[]media)

- analyzeMedia (takes page->content return page->content->media->[]attribute)

..... So what you do is you provide a domain, which will trigger resolveIpFromDomain. This will map the data back to the datahive and based on the Ip in new data trigger detectWebserver. This will return found webservers which triggers the requirement of detectVhost. At this point you probably see how its going.

Due to how the architecture works it will always maximum parallelize the work, it will always map the data into one big structure without you having to care about it, it will only execute things that are necesary/usefull.

So the more your software should branch/parallelize the more gain you get.

Tho as i mentioned in my original post ill release the first alpha so there is still things that can be extended and improved. And right now im spending time in writing the docs which will probably take me some more weeks in orders to make them good enaugh for people to understand how to use it by themself.

I mostly will release it because i think its a great showcase of how you can do optimized data driven processing while havin an architecture that cares about the most painfull things like data mapping / parallelization / etc. I dont expect it to be the next "big thing" or even beeing used by alot of people, but if it inspires people or someone maybe write a even better version based on the idea i would be happy already .)

So to come back to your original question - can it host a website? Probably - but not really meant to do it and a nginx would serve u better.

Post reply on HN