Live data from Hacker News

"I've isolated the bug to a database query"

thedailywtf.com

11–20 of 173 posts

Re: "I've isolated the bug to a database query"

#11
I once worked on a site where the original developer clearly didn't know joins existed, so if he wanted data from two related tables, he'd get all the required results from table one, then loop through them, one by one, querying table two for the corresponding record. Sometimes this went 3 or 4 tables deep, the site would take nearly a minute to load a table of products.

Re: "I've isolated the bug to a database query"

#12
post #9

Earlier quoted context omitted.

The & and | operators weren't good enough for him? That stuff ends up in databases because on your average team the knowledge of databases drops dramatically when you get past selects and maybe left joins. I worked with a few developers who wrote and lived with queries that were taking 30 seconds each to run on their local machines because they couldn't imagine how to fix them and just waited for me to get assigned t…

So show your coworkers how to run the query-plan dumper for your DB brand. The curtain is torn away, and the underlying ISAM is revealed for all to see to allow the needed "hints", indices or restructuring to be understood. Don't lord it over them, explain.

They were disinterested. Under normal conditions you are correct and most people would be interested but they were not.

Re: "I've isolated the bug to a database query"

#13
post #6

One company I was at did a merger with another startup, and most of the other company's engineers quit. Amongst the piles of Visual Basic we found a stored procedure that was about 5 pages long. It took about 18 hours to run; its job was to do a daily report. I hate being afraid of code. I spent a day with it, got to understand it, then rewrote it as a couple of queries and some Java code, whereupon it took about fiv…

I had a digital logic class where we had to find minimum logic gate solutions to get truth tables. My classmates fiddled with karnough maps.

http://en.wikipedia.org/wiki/Karnaugh_maps

I wrote a program which constructed minimum gate solutions to all possible truth tables, eventually finding the minimum gate solution to the requested problem.

------------

LoseThos has it's own document format, like MS Word. I made a program to generate html from my document format. Sadly, my format supports overlaying columns of text, but my html conversion program does not. The columns of text thing is too inconvenient to be used in practice.

------------

I used table look-ups for my text font rendering. My text can be offset from 8x8 grid position by +-15 pixels to allow for markee or super/sub scripting.

Converting formulas to table look-ups can save time, unless the memory reference is not in cache.

I made tables for bitpattern reversal, number of set bits, whitespace characters, etc. It used to be faster to use tables than formulas, but it's conceivable you are correct in the age of cache-miss-penalties.

It's a sad day when floating-point is faster than fixed-point arithmetic.

------------------

God says... C:\LoseThos\www.losethos.com\text\BIBLE.TXT

ad.

16:1 In the seventeenth year of Pekah the son of Remaliah Ahaz the son of Jotham king of Judah began to reign.

16:2 Twenty years old was Ahaz when he began to reign, and reigned sixteen years in Jerusalem, and did not that which was right in the sight of the LORD his God, like David his father.

16:3 But he walked in the way of the kings of Israel, yea, and made his son to pass through the fire, according to the abominations of the heathen, whom the LORD cast out from before the children o

Re: "I've isolated the bug to a database query"

#14
post #9

Earlier quoted context omitted.

The & and | operators weren't good enough for him? That stuff ends up in databases because on your average team the knowledge of databases drops dramatically when you get past selects and maybe left joins. I worked with a few developers who wrote and lived with queries that were taking 30 seconds each to run on their local machines because they couldn't imagine how to fix them and just waited for me to get assigned t…

So show your coworkers how to run the query-plan dumper for your DB brand. The curtain is torn away, and the underlying ISAM is revealed for all to see to allow the needed "hints", indices or restructuring to be understood. Don't lord it over them, explain.

You're assuming the other people want to learn. Generally, if people want to learn something, they ask questions. Telling co-workers how to do something when they don't ask can be... tricky. It depends on the environment you're in, your relationship with the people, and the people themselves.

Re: "I've isolated the bug to a database query"

#15
post #10
post #8

While we're sharing horror stories... I once encountered a stored procedure that returned HTML in a result set. It literally created the UI of a webpage. It returned several columns of HTML that the app would place in strategic parts of the page. Well, as years went by, the app required a more innovative and web 2.0 UI. Rather than remove the HTML from the sproc, more columns were returned with more HTML, Javascript…

Sounds like Oracle's Portal product?

It was a custom built web app that had grown over the course of ten years or so. It was actually a fairly heavily trafficked web site, and this was a particularly old piece of it.

Re: "I've isolated the bug to a database query"

#16
Just so folks know, there are tools that will decompose queries and make nice little pictures out of them. With something like this, you'd have to use it just to get started.

Once you've visually decomposed it, you'd physically decompose it by splitting it inside-out. Then proceed to understand and debug inside-outwards.

Not fun, but not impossible. Just a huge pain in the ass. Making it more fun would be a database with bad RI, nulls, and duplicate data all over the place. Don't get me wrong -- from looking at the image it definitely looks like aspirin will be required. :)

Re: "I've isolated the bug to a database query"

#17
post #9

Earlier quoted context omitted.

The & and | operators weren't good enough for him? That stuff ends up in databases because on your average team the knowledge of databases drops dramatically when you get past selects and maybe left joins. I worked with a few developers who wrote and lived with queries that were taking 30 seconds each to run on their local machines because they couldn't imagine how to fix them and just waited for me to get assigned t…

So show your coworkers how to run the query-plan dumper for your DB brand. The curtain is torn away, and the underlying ISAM is revealed for all to see to allow the needed "hints", indices or restructuring to be understood. Don't lord it over them, explain.

As my prior boss used to say... "Give a man a fish, feed him for a day. Teach a man to fish, and you'll have to answer fucking fish questions the rest of your life."

Re: "I've isolated the bug to a database query"

#18

I once worked on a site where the original developer clearly didn't know joins existed, so if he wanted data from two related tables, he'd get all the required results from table one, then loop through them, one by one, querying table two for the corresponding record. Sometimes this went 3 or 4 tables deep, the site would take nearly a minute to load a table of products.

Mmmm... nested loop joins. Little more code and he would have his own database. :)

Re: "I've isolated the bug to a database query"

#19
post #6

One company I was at did a merger with another startup, and most of the other company's engineers quit. Amongst the piles of Visual Basic we found a stored procedure that was about 5 pages long. It took about 18 hours to run; its job was to do a daily report. I hate being afraid of code. I spent a day with it, got to understand it, then rewrote it as a couple of queries and some Java code, whereupon it took about fiv…

"Why do I find all the really howling bad stuff so close to the databases?"

Because the database and queries against were were coded as an afterthought.

Because the programmers ran all the db access tests against tables with 4 rows of data when the customers would windup with 100 million rows in production.

Because most undergrad education around databases is poor and antiquated. Plus, it is not SQL focused - much time is spent on talking about data modeling. Not wrong, but not often helpful in practice.

Because DBA's are (rightly) focused on making sure your db is properly backed up, replicated, and fault-tolerant. They may not even understand SQL queries beyond the basic one table select.

Because the users will force data into the systems that the developers never anticipated.

Because db jobs are segregated by product for development. If you've done SQL coding on Oracle for the last 10 years, good luck applying for a job doing SQL coding on MS SQL Server.

Because SQL hacking doesn't get much respect. I've been in job interviews where deep into the process they ask me a question with data in two tables that requires an outer join. And answering correctly differentiates you significantly from other applicants. I guess it is the "fizz buzz" of SQL these days . . .

Re: "I've isolated the bug to a database query"

#20
My first assignment in my first ever job working for a 'proper consultancy' was to babysit an overnight process which was a SQL Server stored procedure. Back in those days they had a 64k limit, so it was split into 3 or 4 sections. It took around 12 hours to run.

I'd like to say I rewrote it, but I didn't. I just left.

Post reply on HN