Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What problem are you close to solving and how can we help?

news.ycombinator.com

Post reply on HN
Print view View original

301–310 of 486 posts

Re: Ask HN: What problem are you close to solving and how can we help?

#301

Post by adammunich »

betwixthewires wrote: post #173
adammunich wrote: post #43

I am working on two problems: 1. I want to create way to generate electrical power without pollution. Basically, a closed cycle process that releases no pollutants, or electronic waste. 2. I want to do everything I can to eliminate gender bias in the world.

You sound like a wonderful person! What progress have you made in your work on either front? What sort of work do you do to solve these problems?

I worked for years studying how to use microcontrollers and after a lot of determination now I have a $750,000 grant to build solar systems that are fireproof so you can install them anywhere. It will be a few years of work to get something suitable made but I have full confidence it can be done.

I am also spending much time lately in the SF kink community to build a fundamental understanding of the biases people have experienced in life with respect to their gender identity, and am strongly considering HRT so I can live life on the other side and experience the prejudice first hand.

Re: Ask HN: What problem are you close to solving and how can we help?

#302

Post by adammunich »

znpy wrote: post #80
adammunich wrote: post #43

I am working on two problems: 1. I want to create way to generate electrical power without pollution. Basically, a closed cycle process that releases no pollutants, or electronic waste. 2. I want to do everything I can to eliminate gender bias in the world.

Regarding 1, if you can accept some pollution at the beginning, hydro-electrical can be a solution, albeit probably not for a global scale. We have a small hydro-electrical plant una River near my house and really it's no big deal, it fits very nicely in the surrounding environment and it produces clean energy. It's also educational because since the river is near the city small children classes can visit it and lear…

I love hydroelectric power and I want the circuits I design for solar to be capable of utilizing the raw power from a small turbine as well without any hardware modifications.

Re: Ask HN: What problem are you close to solving and how can we help?

#303

Post by staz »

this more like a tool suggestion request, because I haven't been able to find a solution in Google.

I'm looking for a more advanced duplicate files finder in Linux, specially one that can handle folders.

Most tool just return a list of duplicate files, but what I need is to also know is if whole folders are duplicate, or subset of others and have it presented in an elegant way t o resolve conflict. ofc everything is in a big folder and everything is a mess, bit of a hoarding collection of files that got accumulated over the years.

I could probably code some stupid script myself, if I ever got the time (spoiler I probably won't) but I have no idea on how to present the result elegantly. So it would be nice if such tool already existed.

Re: Ask HN: What problem are you close to solving and how can we help?

#304

Post by erikerikson »

slx26 wrote: post #262
erikerikson wrote: post #106

How do we scale social accountability and knowing? This is expected to enable us to solve distributed coordination problems. Also, it should facilitate richer more meaningful relationships between people. Expected outcomes include increased thriving and economic productivity. [edit: consider the limit on how many people you can know and the relationship between how deeply you come into relationship with that populati…

I have spent quite a lot of time thinking about coordination in general. Indeed, knowledge is a vital part of it. The problem that I see is that knowledge is too vague and lossy and changing and incomplete [as I mentioned in this comment https://news.ycombinator.com/item?id=26203718 ]. An hypothetical solution would be a system that talked a language similar to plain english, but that was determinist. You let people…

A combination of "all categories are fuzzy" and "all models are wrong but some are useful"? I too doubt the effectiveness of a symbolic AI approach. Although I studied that and other approaches in the field, you may note that my background is in biologically plausible methods for pursuing artificial intelligence.

I think the direct human input method is given too much focus although it and related interactions have their place. The fallible sensors directly reporting readings from reality already has sufficient noise related issues. I suspect more richly informing people will yield better results.

I am inspired by stories such as the fish farm pollution problem [0]. Consider how a reality based game theoretic analysis of agent choices might guide your selection of future work mates (or lakes) and facilitate a different friction in finding your next contribution to the world.

[0] search "3. the fish" on https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditation...

Re: Ask HN: What problem are you close to solving and how can we help?

#305

Post by minhmeoke »

pgroves wrote: post #218

How to make png encoding much faster? I'm working with large medical images and after a bit of work we can do all the needed processing in under a second (numpy/scipy methods). But then the encoding to png is taking 9-15secs. As a result we have to pre-render all possible configurations and put them on S3 b/c we can't do the processing on demand in a web request. Is there a way to use multiple threads or GPU to encod…

Are you required to use PNG or could you save the files in an alternative lossless format like TIFF [1]? If you're stuck with PNG, mtpng [2] mentioned earlier seems to be significantly faster with multithreading (>40% reduction in encoding times). If you're publishing for web, TIFF or cwebp might also be possibilities with -mt (multithreading) and -q 25 (lower compression and larger filesize but faster) flags, or an experimental GPU implementation [3].

[1] https://blender.stackexchange.com/questions/148231/what-imag...

[2] https://github.com/brion/mtpng

[3] https://emmaliu.info/15418-Final-Project/

Re: Ask HN: What problem are you close to solving and how can we help?

#306

Post by Radim »

udev wrote: post #265

I am blocked on finding a good (defined below) way to determine whether a product description A and product description B refer to the same product. Imagine that a product description is a n-dimensional vector like: ( manufacturerName, modelName, width, height, length, color, ...) Now imagine you have a file with m such vectors (where m is in millions), and that not all fields in the vectors are reliable info (typos,…

What's your cost matrix? How much does a false positive hurt? False negative?

I built a commercial system like that for Thermo Fisher, except their descriptions were encoded as natural language text on input, not vectors (for an extra complication).

Some observations:

1. Crude methods based on vector embeddings, cosine similarity, Levenshtein, etc – don't work, if you care at all about false positives.

I see sibling comments recommend this, but it's clear this cannot work if you think about it. Values like "black" and "white", or "I" and "II" (part numbers), "with" and "without", are typically close together in such crude representations, but may lead to products that are not interchangeable.

2. A hybrid approach worked. The SW produced suggestions for which products might be duplicates (along with a soft confidence score), then let a human domain expert accept / reject these suggestions. It also learned from these expert decisions as it went, to save human time.

What I quickly learned is that even as a human (programmer with a PhD in ML), I could not look at two product descriptions and make the decision myself. Are these the same product or not? One word, even one letter, could be absolutely vital. Or absolutely irrelevant. Sometimes even the same attribute / word, depending on the product category.

Hence the final interactive solution with a domain expert in the middle. It worked well and saved time, rather clever, but not in the "hooray NN training" way. A lot of work went into normalizing the surface features intelligently based on context: units, hyphens / tokenization, typos…, because that's a mess in product sheets. The "fancy" downstream ML and clustering part was relatively simple by comparison.

But YMMV, the Thermo Fisher products were fairly specialized and sophisticated (in their millions).

Re: Ask HN: What problem are you close to solving and how can we help?

#307

Post by sundarurfriend »

Rodeoclash wrote: post #5

I'm not sure if this is in the spirit of the thread but I've been working on a way to allow reviews of gameplay in video games. In short, you upload a video of you playing the game and someone who's an expert can review it. I currently have a UI with the comments down the side of the screen which looks like this: https://www.volt.school/videos/c980297a-417b-416f-947b-58a70... This is good because you can easily: - Se…

The default should be to show one comment at a time, because that's convenient and quick to get into, but also with an option (maybe just a scroll down) to view all comments. One, that helps the reviewer get an overall idea of what kind of things the submitter is looking for, if they want that, and two, some submitters are inevitably gonna screw up, posting at the wrong times or asking overall summary questions that should be asked at the end right at the beginning or somesuch. So an All Questions button or similar should be there as an escape hatch, but not the primary UI.

Re: Ask HN: What problem are you close to solving and how can we help?

#308

Post by minhmeoke »

staz wrote: post #303

this more like a tool suggestion request, because I haven't been able to find a solution in Google. I'm looking for a more advanced duplicate files finder in Linux, specially one that can handle folders. Most tool just return a list of duplicate files, but what I need is to also know is if whole folders are duplicate, or subset of others and have it presented in an elegant way t o resolve conflict. ofc everything is…

https://meldmerge.org/ might be what you are looking for. But if there are tools which can find duplicated files and directories which are named differently, I'd be interested in learning more about these as well.

Re: Ask HN: What problem are you close to solving and how can we help?

#309

Post by geitir »

sgk284 wrote: post #24
k1rcher wrote: post #2

We are having atrocious READ/WRITE latency with our PG database (api layer is django rest framework). The table that is the issue consists of multiple JSON BLOB fields, with quite a bit of data— I am convinced these need to be abstracted to their own relational tables. Is this a sound solution? I believe it is the deserialization in these fields of large nested JSON BLOBS that is causing latency. Note: this database…

Are you just doing primary key lookups? If so, a new index won’t do much as Postgres already has you covered there. If you have any foreign key columns, add indexes on them. And if you’re doing any joins, make sure the criteria have indexes. Similarly, if you’re filtering on any of the nested JSON fields, index them directly. This alone may be sufficient for your perf problems. If it isn’t, then here’s some tips for…

Unrelated to post, but as you seem well informed in the field, would you agree that if a schema is not likely to change and is controlled as you put it, there is no reason to attempt to store that data as denormalized document?

Or at least as you suggest if required for performance the data would still be stored denormalized and where needed materialized / document-ized?

At my current company, there seems to be a belief that everything should be moved to mongo / cosmo (as document store) for performance reasons and moved away from sql sever. But really I think the issue is the code is using an in house orm that requires code generation for schema changes and probably less than ideal performance query generation.

But then I am also aware of the ease of horizontal scaling with the more nosql orientated products, and trying to be aware of my bias as someone who did not write the original code base.

Re: Ask HN: What problem are you close to solving and how can we help?

#310

Post by BudaDude »

zanek wrote: post #287

I'm working on a different type of compression (for all file types). I am able to to get in the 10-20% range, but the speed to compress is to slow many times, or the compression doesnt complete at other times (I've been working on this for years). My personal website: http://danclark.org I'm also working on a conversational search engine (using NLP) at http://supersmart.ai

Have you looked into Middle Out compression?
Post reply on HN