Ask HN: How do you solve hard problems if you can't make incremental progress?

#1

Post by ntoshev » Fri, Mar 26, 2010, 4:29 PM UTC

Ask HN: How do you solve hard problems if you can't make incremental progress?
ravimohan.blogspot.com

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#2

Post by vital101 » Fri, Mar 26, 2010, 4:36 PM UTC

I generally try to step away from the problem for awhile. Even if it's only for 10 minutes. I come back with a fresh perspective that usually helps me get through it.

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#3

Post by thecombjelly » Fri, Mar 26, 2010, 4:36 PM UTC

I try to attack the problem persistently and creatively. Often I will walk around and just ask random people what they think. Sometimes even describing the problem in non technical terms for the non techies to understand. People usually have some ideas that you haven't thought of. I've found that any idea can set you off on a path to solve the problem.

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#4

Post by kmak » Fri, Mar 26, 2010, 4:38 PM UTC

The part I don't understand is, for this particular problem, it is well-known enough that a simple visit to Wikipedia would've been enlightening.

I remember reading up on it awhile ago, and it seems like the Wikipedia entry already had the dancing links algorithm by then.

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#5

Post by ntoshev » Fri, Mar 26, 2010, 4:46 PM UTC

I have a programming problem, and I'm pretty sure I'm aware of the basic components of the solution... it's just that I get swamped in complexity pretty quickly when I try out different combinations.

I know this is a very meta question but this is not the only case. I'll solve it eventually but I'd like to ask for common strategies.

Edit: Some of my strategies are: to absorb more information about similar stuff while I put solving it on back burner, to try to describe the problem clearly, to bounce the problem off other people.

-------------------------

Okay, here is the specific question: I have a map of categories of things (a few words) and descriptions what these categories mean (longer text). I also have another list of category names that sometimes uses different words or abbreviations, and I need to match the second list to the first one precisely.

I know all about tf-idf, vector space similarity between a document and a term, wordnet as a source of semantic relations between words, and it's ok to have a human map some of these. The problem is I don't know when a matching score is good enough and when I need to fall back to a human.

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#6

Post by [deleted] » Fri, Mar 26, 2010, 5:00 PM UTC

ntoshev wrote: post #5

I have a programming problem, and I'm pretty sure I'm aware of the basic components of the solution... it's just that I get swamped in complexity pretty quickly when I try out different combinations. I know this is a very meta question but this is not the only case. I'll solve it eventually but I'd like to ask for common strategies. Edit: Some of my strategies are: to absorb more information about similar stuff while…

[deleted]

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#7

Post by NEPatriot » Fri, Mar 26, 2010, 5:00 PM UTC

I look for inspiration from the world around me and focus on trying to always find the simplest solution with the least resistance.

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#8

Post by dwwoelfel » Fri, Mar 26, 2010, 5:14 PM UTC

Use Poyla's methods of problem solving described in his book: How to Solve It (http://www.amazon.com/o/ASIN/0691023565)

He deals specifically with mathematical problems, but the same ideas apply to computer science.

For an outline of the method see this Wikipedia article:

http://en.wikipedia.org/wiki/How_to_Solve_It

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#9

Post by sandaru1 » Fri, Mar 26, 2010, 5:14 PM UTC

ntoshev wrote: post #5

I have a programming problem, and I'm pretty sure I'm aware of the basic components of the solution... it's just that I get swamped in complexity pretty quickly when I try out different combinations. I know this is a very meta question but this is not the only case. I'll solve it eventually but I'd like to ask for common strategies. Edit: Some of my strategies are: to absorb more information about similar stuff while…

Use Levenshtein distance - http://en.wikipedia.org/wiki/Levenshtein_distance

Re: Ask HN: How do you solve hard problems if you can't make incremental progress?

#10

Post by ntoshev » Fri, Mar 26, 2010, 5:29 PM UTC

sandaru1 wrote: post #9

ntoshev wrote: post #5

I have a programming problem, and I'm pretty sure I'm aware of the basic components of the solution... it's just that I get swamped in complexity pretty quickly when I try out different combinations. I know this is a very meta question but this is not the only case. I'll solve it eventually but I'd like to ask for common strategies. Edit: Some of my strategies are: to absorb more information about similar stuff while…

Use Levenshtein distance - http://en.wikipedia.org/wiki/Levenshtein_distance

Thanks, my boolean approximation for abbreviations so far is:

  def abbr(short, full):
      return re.match(''.join(c+'.*' for c in short), full)

Actually what I'm doing now seems to be working, so far I can't see any pattern in the things my algo can't match by itself.

Also thanks to ramanujan, who deleted his comment for some reason, but besides pointing out orgmode which I want to check, his proposition reminded me that I'm trying to deal with my dataset incrementally while a batch mode might work better.