My workflow is far from exemplary, and I've often thought about ways to improve it, so good luck with your project. There could be a market there.
I have basically three modes of note-taking: at the computer, in a meeting, and offline. I work remotely, so most of my meetings actually are "at the computer" and on the phone; "in a meeting" means in-person.
At the computer I use a text editor. I don't think it matters which one, as long as it's one you know well. I use TextMate mostly, or Vim as a fallback. When something comes up an actionable task I put it into my personal task tracker, currently Wunderlist but again, that could be anything.
So what happens when I need to draw or diagram something?
I describe, briefly, what I will need to illustrate. Then I go back and illustrate it later if I still think it's worth the trouble.
This works surprisingly well because I learned to touch-type quite young and don't have much of a gap between my mind and the keyboard.
I usually use Markdown (with a bunch of common extensions) so it's easy to include code snippets and simple lists.
In a meeting, where it would be rude to type that much, I keep a pad of paper and a pen, and just write stuff down, occasionally with a diagram. However, these days most drawing-type content from meetings will be photographed and mailed around anyway, so I don't try to copy stuff off of whiteboards. I keep the computer open usually in case something requires immediate action, and I usually type up some notes quickly after the meeting -- but as often as not, somebody else mails out proper minutes so I probably don't need to.
Offline, I'm very old-school: lots of paper, pens, pencils. (I studied art.) A lot of it is just writing on paper -- I think it's important to keep your physical writing skills sharp, because your brain works differently with a pen than at a keyboard.
Some things end up scanned back into the computer, but most of it doesn't. I usually find that if I've got good notes on paper, they'll mostly still be in my brain the next time I'm taking notes at the computer.
I've tried Evernote, OmniOutliner, OmniFocus (!), rich-text, DevonThink, and a few other things, and everything was worse for me than what I use now.
However, I would happily try something else, as long as it has no lock-in. For me, and probably for a lot of other not-so-young hackers, putting my notes into a system that will be hard to extract them from is a show-stopper.
Here's my wish-list, off the top of my head:
* Everything stored in plan-text (Markdown, whatever) except binary assets.
* Binary assets stored together with that.
* Great sync between devices, and everything very secure.
(Maybe just use DropBox for that.)
* Really fast, great UX!
* Great search, tags, etc.
* Integrated TODO, calendar, etc.
I think the real opportunity is in providing a great user experience together with some impressive auto-organizing magic. The trick is to resist the temptation of lock-in. And the killer feature might be a smart way to deal with stuff that originates on paper.
Best of luck!