Ask HN: What weird or hard problems are you trying to solve? • Hacker News Board

Re: Ask HN: What weird or hard problems are you trying to solve?

#641

Post by kristopolous » Mon, Jul 06, 2020, 11:08 AM UTC

crabl wrote: post #513

Existing mediums for note-taking (Evernote, Notion, Roam Research) are not sufficient for doing knowledge work over long periods of time. The functions these incumbents serve are primarily as “stores of knowledge” that we save because it’s interesting in the moment but never read through or “scratchpads” that we use once and never get rid of, which end up cluttering our information space like a junk drawer full of sh…

I worked with someone in 2013 who I'm still convinced solved this problem. But he was an absolute terror to work with. A petty authoritarian who would look over every website I'd go to and have screen monitoring and key logging software.

I kid you not. I left after 6 weeks. I snuck out the work with me (I wasn't "allowed" to bring it home.) If you're interested I'd be more than happy to share the code.

I think it was really revolutionary.

Essentially it used grammatical structures to arrange text in a navigable 2d space which he needed because he was a highly visual and spatial learner.

But what it allowed for is a nonlinear and a nonsequential arrangement of ideas using Wikipedia text, not just some simple mindmap stuff.

I used Wikipedia as an example in a prototype engine I made.

Articles Flowed Into each other through a continuous navigable space. You could interact and engage to go cognitively deeper and expand a new path, as opposed to a series of documents. Wikipedia became one continuous thing that you could endlessly navigate through a 2d space of.

The content wasn't large blocks of text but broken up using a separate visual language so that there'd only be a few words then a relation to another group and so on. This kept the concepts spatially relative and made the distinction of pages disappear.

It did it all automatically. Really amazing stuff. I also worked with Ted Nelson, this guy's methods were better. No question.

He developed the techniques over about 20 years manually and had transferred textbooks to rolls of butcher paper that he kept in cabinets. He totally didn't understand the value of his process as something as transformative as Vannevar Bush's As We May Think.

Instead he wanted to make it a proprietary format with proprietary content under a private publishing company for childhood education. He wanted a kludgy editor to make new content with and then a kludgy viewer for single topic things. He wanted to dictate the interface, keystrokes...

Because once again he's an authoritarian pedant. Bah, he didn't see what he made.

Ideas need to be controlled at the right level of abstraction and liberated at the others. That's what Linus knows that RMS doesn't. That's what TBL knows that Nelson doesn't. That's what Jobs knew but Apple doesn't.

I wanted to run with the idea but yeah, 7 years ago and I've done nothing. I got everything still.

I should stop everything and finish it. It's really something radically different. I think it will change at least the way I personally learn things.

The campaign to convince others, yeah well, no guarantees there.

Re: Ask HN: What weird or hard problems are you trying to solve?

#642

Post by forgotmypw17 » Mon, Jul 06, 2020, 11:12 AM UTC

insomniacity wrote: post #42

forgotmypw17 wrote: post #41

I'm developing a web-based system which supports every browser in existence. So far, I have full baseline feature support for everything back to Mosaic, including Lynx, IE3, Netscape3, Opera3, and many others. At the same time, still including advanced features like client-side PGP for browsers which will support it. Every browser presents its own challenges, and it is not always the oldest ones which have the dumbes…

There are/were browsers with client-side PGP?

I use OpenPGP.js to integrate it.

It's not the most secure solution, but it's just one of the options.

There are browser add-ons that do integrate PGP.

Re: Ask HN: What weird or hard problems are you trying to solve?

#643

Post by sputr » Mon, Jul 06, 2020, 11:15 AM UTC

drallison wrote: post #538

sputr wrote: post #508

Earlier quoted context omitted.

Solutions to the stated problems are, from a technical and policy stand point very simple. Yes, we know how to solve all of this - it's called market regulation and we know it works reeeely well. But it's politically hard. What we need is an economically pragmatic green political movement, who will not keep shooting itself in the foot with left-wing ideological purity competitions. Profit seeking motive is only a pro…

I think you are begging the question. Market regulation and a more economically pragmatic green political movement, and a fettered profit-seeking motive won't respond quickly enough to save the day. We have a crisis here and have not much of anything which makes a difference. For example, when huge fires burn, dumping significant CO2 into the atmosphere, it would be to everyone's best interest to put them out. But we…

Why do you presume it won't respond quickly enough? I think that if the EU made any carbon fuel that was created from the atmospheric CO2 free of any and all tax ... we would see it on the market in a few years at max with plants cropping up very quickly. When that industry matures in a few years just force anyone polluting with CO2 to buy and sequester this carbon fuel.

Besides that's only half of the solution. The other is pragmatic policy. If the green movement was pragmatic they would have championed "EU builds 200 nuclear plants in And regarding the pandemic thing - the question you should ask: "wrong choice for whom?". The general population? Or the people in power? The biggest problem in our society is that we've fallen for the lie "in a democracy the general population rule and the system inherently works in their best interests". Democracy has nothing to do with taking care of the people. It can be used for that ... WHEN the general population realizes that it first has to BECOME a power player in the game of politics.

Re: Ask HN: What weird or hard problems are you trying to solve?

#644

Post by krspykrm » Mon, Jul 06, 2020, 11:24 AM UTC

nightcracker wrote: post #261

Improving the performance of fountain codes and applying them securely to peer 2 peer file sharing. A fountain code is an almost magical algorithm that can split a file of size n up into a (practically) infinite stream of blocks of size b, such that collecting any n/b blocks out of the stream can reconstruct the file. Applied to p2p file sharing it effectively can eliminate rare pieces as well as the need to communic…

Good topic. Have you seen this high-performance Reed-Solomon implementation (1) or Wirehair (2)? Thoughts?

1) https://github.com/klauspost/reedsolomon

2) https://github.com/catid/wirehair

Re: Ask HN: What weird or hard problems are you trying to solve?

#645

Post by silveroriole » Mon, Jul 06, 2020, 11:33 AM UTC

collyw wrote: post #632

silveroriole wrote: post #622

Earlier quoted context omitted.

So, let’s assume that everyone who got banned for saying there are two genders (and presumably strongly believes it, because they cared enough to be banned for it) is attracted to this platform. People who disagree won’t be attracted to it; they can already post their opinion on, say, Twitter without being banned, and they hold the opposite opinion anyway. Now Twitter is an echo chamber of one side of the debate, and…

The difference being that you can actually post / see both sides of the debate on one platform. And that isn't the one with censorship policies. You seem to be in agreement that the lack of moderation actually gets us closer to the form of debate that we are after.

I don’t really agree. Like I said, I don’t think you will ever get both sides of an argument there; one side will force the other out eventually. I’d rather have a proliferation of communities which are ‘censored’. Sure, have a forum for racists where non-racists get banned, whatever (although I won’t shed any tears if it gets closed down). If I want to see what racists think, I can go check it out. But I don’t want to be part of a space where I’ll regularly encounter racist stuff, and they don’t want to be part of a space where they’ll regularly encounter anti-racist stuff. Mashing me and the racists together in an anonymous arena where anything goes obviously won’t result in us all nicely debating and getting to know each other. It is not a step in the right direction.

Re: Ask HN: What weird or hard problems are you trying to solve?

#646

Post by uhhyeahdude » Mon, Jul 06, 2020, 11:35 AM UTC

kristopolous wrote: post #519

uhhyeahdude wrote: post #503

Earlier quoted context omitted.

What an ambitious and important project! I love the concept, and that it is meant to be a "long, slow project." It probably should be paced slowly due to the scope and complexity of material. I think a slower pace should also help "smooth" the exponential change rate of technological progress into a narrative listeners can follow, without sacrificing too much detail for the sake of listenability.* I'm going to sub ri…

It's all scripted. Google books and archive.org have been great resources. There's also a couple torrents of 17th-19th c. journal articles. I've made my own sqlite databases from the meta information and ran ocr over them to make things searchable. I'll be putting more of these tools in the repo and eventually put the search systems online for the general public. Google's navigation of annualised volumes is a joke so…

I'll just reiterate that I find it quite impressive. The above just reinforces that.

I mentioned that I listen to a lot of podcasts; have been, in fact since ~2006. I've watched the evolution of the medium over the years, have seen many pods come and go, and have witnessed many content creators experiment with different methods of attaining/maintaining/interacting with listeners.

Later on, as the form started to mature and gain more of an audience, the issue of monetization became more and more important and necessary; I've seen many experiments focused on that facet as well. This was still before "podcast" had entered the vernacular for most people I would classify as tech-adjacent--the general public was still not really aware of podcasting. Those who were didn't exactly advertise their listening habits yet, and podcasts were almost like an embarrassing thing one did in private.

You've done so much correctly, at least from my observations over the years, from the get-go.

If some of your ideas come to fruition, it would be a boon to those who'd like to make a podcast of their own, especially if technical or academic in nature. I love the work you've done, appreciate the open distribution of your expanding toolkit, and also want that auto-hyperlinked footnotes a whole lot.

I am doing what I can to encourage you for two reasons. One, I am personally interested in the subject(s) being covered, and I think projects like this are needed if we aren't going to lose a lot to the mists of history in the face of accelerating acceleration, to borrow a phrase. Two, I want the tools for my project and do not possess the requisite skills to make them myself, although I'm working on it.

So, selfish but genuine encouragement...

Re: Ask HN: What weird or hard problems are you trying to solve?

#647

Post by hazeii » Mon, Jul 06, 2020, 11:47 AM UTC

shezi wrote: post #500

hazeii wrote: post #137

Earlier quoted context omitted.

Good to get your attention :) but still got to print the gray code to 1 part in 8192 and fit it all on a bike. Along the lines of your idea, we've looked at using the CCD array out of desktop scanners with a spiral pattern printed on the crank. But bike bottom brackets are a hostile environment, plus (ideally) we'd actually be measuring the angle against local vertical (i.e. the direction in which gravity sucks). Our…

Indeed, manufacturing is hardly a problem in the software world. =) Maybe get it laser engraved into aluminium and mount it... somewhere? Grey code is essentially a more robust spiral pattern, so that's the same idea. What if you put a roll sensor on the chain instead and infer crank angle from that? Chain slip is rare enough not to worry about it, right? You'd also have more space and could use off the shelf compone…

Nice idea on the laser engraving, maybe we could use that inside the bottom bracket - thanks.

We have considered the chain, trouble is both chains and chainrings wear and often cranks/rings aren't terribly-well centred (as fixed gear bike riders know, from variations in chain slack).

For the gravity part we've tested a range of accelerometers; vibration is quite an issue though.

Thanks for the interest!

Re: Ask HN: What weird or hard problems are you trying to solve?

#648

Post by nightcracker » Mon, Jul 06, 2020, 11:50 AM UTC

sdinsn wrote: post #377

nightcracker wrote: post #261

Improving the performance of fountain codes and applying them securely to peer 2 peer file sharing. A fountain code is an almost magical algorithm that can split a file of size n up into a (practically) infinite stream of blocks of size b, such that collecting any n/b blocks out of the stream can reconstruct the file. Applied to p2p file sharing it effectively can eliminate rare pieces as well as the need to communic…

Similar to this? https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing

Shamir's Secret Sharing is roughly equivalent to Reed-Solomon erasure coding, using the same idea.

The fountain codes I'm interested in are built from the perspective of inverting boolean matrices, and XOR'ing the right blocks together.

Re: Ask HN: What weird or hard problems are you trying to solve?

#649

Post by hazeii » Mon, Jul 06, 2020, 11:51 AM UTC

DavidPeiffer wrote: post #349

hazeii wrote: post #149

Earlier quoted context omitted.

Thanks, looked at them. Can't get the resolution we need for any vaguely feasible price though.

Brainstorm here - could you put a number of lower resolution encoders firmly on the same shaft? The theory would be they'll all pulse on slightly different but consistent phases. Simple example, an encoder is capable of 60 degree resolution. I'll mount 5 of these which will all have some variation in phase, and I'd average out to having 12 degree resolution, with variation that could be modeled statistically. Concept…

Interesting, but I think we're run out of space. Sort of related, we've been considering how we can gut an optical mouse and use the sensor from them. The idea is broadly similar, basically learn the pattern around one revolution and work out where we are in it.

Re: Ask HN: What weird or hard problems are you trying to solve?

#650

Post by nightcracker » Mon, Jul 06, 2020, 11:54 AM UTC

krspykrm wrote: post #644

nightcracker wrote: post #261

Improving the performance of fountain codes and applying them securely to peer 2 peer file sharing. A fountain code is an almost magical algorithm that can split a file of size n up into a (practically) infinite stream of blocks of size b, such that collecting any n/b blocks out of the stream can reconstruct the file. Applied to p2p file sharing it effectively can eliminate rare pieces as well as the need to communic…

Good topic. Have you seen this high-performance Reed-Solomon implementation (1) or Wirehair (2)? Thoughts? 1) https://github.com/klauspost/reedsolomon 2) https://github.com/catid/wirehair

No and yes.

For fast Reed-Solomon I'm aware of this: https://github.com/Bulat-Ziganshin/FastECC. It's kind of amazing how far Reed-Solomon has come, thanks to the fast fourier transform. At the very large problem sizes it does slow down more and more though. If cleverly applied (e.g. layering erasure codes) you can make it go very far indeed. I don't like how hardware support is essentially necessary to make the larger field sizes necessary for bigger instances fast.

Wirehair is very interesting, and I hope to study it well and describe it more formally as part of my Master's thesis. I'm not aware of any academical analysis of it. I did look into it enough to diagnose it suffers from the same O(n sqrt n) issue that RaptorQ does, again for very large instances. The issue lies in having to do Gaussian Elimination for a submatrix (to solve the inactivated columns) near the end of decoding, and this submatrix can be on the order of O(sqrt n).

I'm interested in 'very large instances' because ideally I'd be able to create an efficient fountain code with block sizes on the order of the size of a UDP packet, disk page or QR code, which has some very interesting applications.