I have all this data that I collect and it's very difficult to make sense of it or to break it down into a form that tells me what steps I should be taking next to improve my bottom line.
We have similar problems. For example, I'm fairly certain some sort of bayesian model could tell me valuable things about my data. Someone who could run with that provide me an automated way to extract probabilities of desirable outcomes from my data would get money very quickly.
It's kind of hard to put Bayesian Data Analysis into boxes and sell it :-)
I have the product you need sitting in version control. We never released it to market because when we went to start marketing it, it seemed too difficult to go up against the big guys so we scrapped it and integrated it into a product where it's in use to this day. How would you market a product like this?
The difficulty in this market is that each business has a different need. Each business has different data and requires different information. So I'd break this up into individual plugins that just work, and that handle a specific problem. So I send my data into your system, and you have different pipelines I can send the data into that provide me with different intelligence things. And then afterwards, just approach…
Yes, its so hard to target a product like this. I think we'll just leave it sitting in version control, as I'm not passionate about solving industry specific problems other than the ones we built it for! :P
I have all this data that I collect and it's very difficult to make sense of it or to break it down into a form that tells me what steps I should be taking next to improve my bottom line.
What kinds of data are you collecting?
I have sometimes wondered whether the concept of a profiler could be applied outside programming. I'll bet if you put all expenses of an average big company into a profiler (plug: such as sysprof), you'd find tons of things that could be cut.
Though I guess this is what operations research was supposed to do.
Because it is basically a glorified frontend (as far as pdf to other formats conversion is concerned) for pdftohtml. It treats every line as a paragraph, can't intelligently decide on what is a chapter heading etc. The problem I'm describing is AI hard, it can't be done be some rule based matching. I know that Calibre has some regex based rules for finding chapters etc. but that is way too hard and only solves part o…
We may have different goals in mind... I have a bunch of PDFs I need to get to my sony reader, for example academic papers. I want them to look basically OK, which I don't "think" is an AI hard problem. Detecting any text that represents a chapter certainly is... but I'm OK with writing regex to get my data out. I'll be posting in the mobileread forums about this at some point in the coming weeks if you're interested…
Academic papers especially are a PITA, at least those formatted in columns, so it depends on the journal I guess. Plus you need detection of paragraphs to get line breaks at the proper locations. If it's not in columns and if you are willing to tweak the algorithm parameters to each conversion (paper), then it won't be that hard (just detect increased line spacing, or indented first line), but a commercial application would have to work almost automatically. or otherwise have a very quick feedback cycle (preview/edit settings/redo conversion). That's another one of my gripes with Calibre, the epub/fb2 preview application is so so slow that it's almost unusable. Yeah let me know if you find a working solution, although I do have dozens of hours worth of scripts laying about to do custom conversions, so real one-off hackjobs I have so many of that I don't need any more :)