Viewing profile — agencies

What is state of the art currently?

You say that if someone has the chops to be a real mathlete they won't need Polya's _How To Solve It_ I'll say I went to college 25 years ago with people who had competed internati…

Is the code or expanded explanation available?

Here's a write-up from a relatively small/personal perspective https://blog.qwertyforce.dev/posts/similar_image_search

Yeah several threads on HN that have lists of tools and pros/cons.

Have you tried https://historio.us/ ??

Depends on required features like cross browser support, cross device support, handling pdfs, ocr images, etc. Some of the mentioned features already exist in the browser. Not sure…

How much would you be willing to pay for such a service?

I want to have a local device that will serve web apps to my Android browser over bluetooth or wifi *while my Android is still connected to normal wifi or mobile data*. I have thou…

I recently completed a small web crawl including only pages linked from HN stories. For example I would crawl the lwn article from https://news.ycombinator.com/item?id=32614633. I …

Interesting intro/overview in "What every software engineer should know about search" https://scribe.rip/p/what-every-software-engineer-should-kno...

There is definitely demand and folks are willing to pay: https://www.duckbillgroup.com/ https://www.vantage.sh/

If it were easy/cheap enough to host, can a model like shared game servers or web/email hosting work? People pay $20 a month without thinking for web hosting. What does it take to …

Where do folks keep up with and ask questions about sota performance code (e.g. simd, hashmap tuning, allocators, Dan Lemire, gaming code optimizations, etc.)? Stackoverflow? Parti…

Many domains have expired or content is no longer available.

If a HN story is a link to Wikipedia, the HN api serves the content of the Wikipedia page??

To clarify I'm not asking about HN itself but articles linked from HN. As you said the HN api is great and there are at least 2 existing published crawls of it that help a lot.

HN links to over 6 million urls in stories and comments. Many domains have expired or content is no longer available. Internet archive has much of the content but throttles request…

Nice! Have you considered publishing your crawl data?

HN continues to pay lip service to wanting to make this better but does NOTHING to move the ball forward. Simple things like providing barrier less access to HN content and harder …

How much does it cost to crawl all urls that are linked from hackernews submissions and comments? Ballpark at least 3.5 million story urls. Several services would likely do this fo…

Is there an updated dump of hackernews that picks up where pushshift data ends? Specifically asking about web hosted flat files (for much easier access than BigQuery). https://file…