Ask HN: What interesting problems are you working on? • Hacker News Board

Re: Ask HN: What interesting problems are you working on?

#651

Post by manishjhawar » Thu, Jan 30, 2020, 10:35 AM UTC

RobinL wrote: post #400

Building a library to deduplicate data at scale in Apache Spark, where there is no unique record identifier (i.e. fuzzy/probabilistic matching). https://github.com/moj-analytical-services/sparklink It's currently in alpha testing, but the goal is for it to: - Work at much greater scale than current open source implementations (the ambition is to work at 100 million records plus) - Get results much faster than current…

I'm currently working on a solution involving larger data sets to match a record with a binary score (0/1). I'm using Redis with the Bloom Filter module. This works in that the query results are sub-second, but the data ingestion/filter population part is quite slow comparatively (<100 MB/s). Another block for me is if having to use multiple filters to query across multiple sets which just multiplies all the resources needed. Does Spark have any advantages or specialized filters for this use case? (I have nil experience with Spark, but am ready to dig up if it would really help.)

Re: Ask HN: What interesting problems are you working on?

#652

Post by coolvision » Thu, Jan 30, 2020, 11:18 AM UTC

Working on computer vision & perception system that allows delivery robots to drive autonomously 99% of the time, with only occasional remote human assistance. Making sure robots need less and less assistance, and this is while having hundreds of robots in production doing commercial deliveries.

https://www.instagram.com/starshiprobots

Technology and business do work, so we probably will have thousands of robots within a year, and millions not long after that )

Re: Ask HN: What interesting problems are you working on?

#653

Post by solresol » Thu, Jan 30, 2020, 11:34 AM UTC

abathur wrote: post #115

tmamic wrote: post #43

Dental treatments, besides being very expensive, are often (up to 28%) unnecessary. This happens because no-one keeps dentists in check. I am trying to make dental treatment and diagnosis reviews easy, cheap, reliable and fast.

I suppose it won't save me, but <3 for working on this. I need work done, but I don't have dental insurance, will have to pay out of pocket, and have yet to overcome the analysis-paralysis problem of finding someone who'll charge a fair price, do good work, and won't add any more holes than I need.

If you have a lot of work that needs doing, it may well work out cheaper to find a dentist overseas, fly there, stay for a period of time, and then fly home again.

For some reason I keep hearing about people flying to Serbia to do this.

Re: Ask HN: What interesting problems are you working on?

#654

Post by ace_of_spades » Thu, Jan 30, 2020, 12:00 PM UTC

syllable_studio wrote: post #203

Solving the world's trillion-dollar energy storage crisis. (multi-trillion, actually.) https://www.terramenthq.com/ About a year ago, I started spending more time researching about climate change. I learned how important energy storage will be to enable renewable energy to displace fossil fuels. The more I read, the more fascinated I became with the idea of building underground pumped hydro energy storage. I found a…

If you are looking for an informed academic perspective on energy trading and renewables I can recommend to contact the following researchers [1]. Just write them an email and explain what you are working on, I could imagine they are going to be interested!

1: http://www.is3.uni-koeln.de/

Re: Ask HN: What interesting problems are you working on?

#655

Post by Chris2048 » Thu, Jan 30, 2020, 12:26 PM UTC

johnmorrison wrote: post #313

A while back, I posted a list of my long term focus problems here [1] Short list: * Pollution and the climate * Privation * Avoidable death * Interplanetary settlement * Liberty and communication * Transportation My primary focus is developing and commercializing reliable clean energy, because I believe that is the most effective way to further progress in the majority of the above problems. To that extent, I've come…

All those things are pretty much industries, let alone fields of study.

Also, consider that "keep ~100% of my net worth invested in these main problems" may not be the better strategy for funding those problems, versus "put net worth in growth investments" that can fund the same problems later on.

Re: Ask HN: What interesting problems are you working on?

#656

Post by Chris2048 » Thu, Jan 30, 2020, 12:29 PM UTC

atheiste wrote: post #360

I am working on a blogging platform that does not need any backend (in terms of an app listening for http connections). The overall architecture is an web app that is talking to WebDAV and then pages get build by a static site generator. I use getpelican.com but you can use Hugo or Jekyll based on your preference.

How would that work? surely something needs to serve the static content to a http request?

Re: Ask HN: What interesting problems are you working on?

#657

Post by Chris2048 » Thu, Jan 30, 2020, 12:31 PM UTC

geocrasher wrote: post #122

I'm working on figuring out four different ways for somebody with a light background in electronics (basic soldering really) and has a Technician or General class ham radio license to get on the air, from scratch, for $100. No added expenses. It's possible, and a fun challenge. Its research for some writing I'm doing.

What kind of band are you thinking?

Re: Ask HN: What interesting problems are you working on?

#658

Post by appleflaxen » Thu, Jan 30, 2020, 12:32 PM UTC

tmamic wrote: post #513

tyjen wrote: post #125

Earlier quoted context omitted.

May you succeed with this endeavor. An ex-dentist attempted to strong arm me into a receiving an occlusal adjustment because my TMJ popped during a single visit. I knew this permanent procedure is rarely the best solution for the scenario. The dentist subsequently became irate and told me, "You'll lose all your teeth and look like an AIDs patient!" You can probably guess what era he's from. I wanted to file a complai…

Our philosophy has to key rules: 1. Diagnostician shouldn't treat. 2. No-one should review themselves. This eliminates so much fraud and mistakes.

wow; interesting idea! but how do you prevent dentists from forming referral cabals and cheating the system?

Re: Ask HN: What interesting problems are you working on?

#659

Post by Chris2048 » Thu, Jan 30, 2020, 12:36 PM UTC

Jaruzel wrote: post #363

My 'paid job' is boring - Identity Management for Blue Chip companies. My fun stuff at the moment: 1. Learning Windows IoT on a Raspberry Pi 3B 2. Working on proof-of-concept Search Engine Indexers for specific datasets and/or local file-systems (on network servers). 3. Exploring a new paradigm of allowing people to easily publish train-of-thought type content without having to post a long series of tweets or silo it…

wrt #3, see https://thoughtstreams.io/

Re: Ask HN: What interesting problems are you working on?

#660

Post by peterwoerner » Thu, Jan 30, 2020, 12:51 PM UTC

tobmlt wrote: post #252

helltone wrote: post #217

Earlier quoted context omitted.

I'm interested about your project #2. As you mentioned B-splines do you deal with trimmed surfaces? Would you have any reading recommendations for someone learning about surface optimisation?

Hey, thanks for your interest! I've avoided trimmed surfaces, in part because I'm interested in doing one or another kinds of analysis on or with the parametric geometry, and trimmed surfaces are not so easy to work with for some of the finer control I want from my optimization tools. (They often cause comparability issues with export between programs as well, but that becomes more important only if somebody uses you…

I keep coming back to bother you :). One of the newer tricks to make parameter fitting less expensive which has recently been developed is active subspaces. I thought you might be interested in playing around with it.

Most of the research is being done out at the Colorado school of mines by Paul Constantine. The basic idea is that you reduce your parameter space to the eigenvectors of the sensitivity matrix with the largest eigenvalues. Some of the work I have seen in constitutive modeling (and UQ) has effectively reduced parameters spaces of a couple hundred DOF to about 5-6.