Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What problem are you close to solving and how can we help?

news.ycombinator.com

Post reply on HN
Print view View original

441–450 of 486 posts

Re: Ask HN: What problem are you close to solving and how can we help?

#441

Post by jlokier »

myWindoonn wrote: post #127

I'm working on destroying proof-of-work blockchains. I have a plan for BTC, but I'm not sure how to approach ETH. Advice would be appreciated.

If you're interested in eliminating proof-of-work for ETH, you should really take a look at the proof-of-stake network in progress. Keywords: "Eth2", "proof of stake" and "the merge".

When proof-of-stake takes over, there won't be any miners. The block proposal process is done by stakers instead. Some of the incentive issues with miners still exist with stakers, but raw competitive power consumption isn't one of them.

It's true that proof-of-stake has been talked about for years, but it has picked up momentum since late last year, as the staking network was actually launched.

The proof-of-stake network has been staking real ETH since end of last year, but does not yet handle mainnet ETH contract transactions. It's called Eth2, but that's caused some confusion, because it's not really a second version to run alongside the first, it is the R&D branch into proof-of-stake and other technical improvements, with mainnet ETH expected to adopt it in due course.

So, the Eth1 components have been renamed "execution layer", Eth2 components renamed "consensus layer", and through a series of testnets and API developments which have been quite active this year, a big change called "The Merge" is being worked on by multiple funded groups (for client diversity) of core Eth developers at the moment.

The Eth2 staking network that already exists has demonstrated the viability, and the investment of real ETH in serious quantities has built up some cryptoeconomic stability prior to its deployment as the ETH consensus layer. The time lag is intentional - you don't want to suddenly switch all ETH over to a network with too few invested stakers.

Re: Ask HN: What problem are you close to solving and how can we help?

#442

Post by dmje »

lonk11 wrote: post #216
dmje wrote: post #164

Earlier quoted context omitted.

Really like this idea, something I've been thinking about for a while. Will join and give the tyres a kick :-)

Thanks! It looks like 13 people signed up, which is really encouraging. I wrote about the performance tuning I did in preparation to the Shown HN post: https://linklonk.com/item/277645707356438528 If you have any feedback please add a comment to that post.

Hey, site down?

Re: Ask HN: What problem are you close to solving and how can we help?

#443

Post by physicles »

pgroves wrote: post #218

How to make png encoding much faster? I'm working with large medical images and after a bit of work we can do all the needed processing in under a second (numpy/scipy methods). But then the encoding to png is taking 9-15secs. As a result we have to pre-render all possible configurations and put them on S3 b/c we can't do the processing on demand in a web request. Is there a way to use multiple threads or GPU to encod…

I solved a similar problem last year. As others have said, your bottleneck is the compression scheme that PNG uses. Turning down the level of compression will help. If you can build a custom intermediate format, you'll see huge gains.

Here's what that custom format might look like.

(I'm guessing these images are gray scale, so the "raw" format is uint16 or uint32)

First, take the raw data and delta encode it. This is similar to PNG's concept of "filters" -- little processors that massage the data a bit to make it more compressible. Then, since most of the compression algorithms operate on unsigned ints, you'll need to apply zigzag encoding (this is superior to allowing integer underflow, as benchmarks will show).

Then, take a look at some of the dedicated integer compression algorithms. Examples: FastPFor (or TurboPFor), BP32, snappy, simple8b, and good ol' run length encoding. These are blazing fast compared to gzip.

In my use case, I didn't care how slow compression was, so I wrote an adaptive compressor that would try all compression profiles and select the smallest one.

Of course, benchmark everything.

Re: Ask HN: What problem are you close to solving and how can we help?

#444

Post by physicles »

snerual wrote: post #147

Stateful, exaclty once, event processing without the operational capacity to run a proper Flink cluster. This thing needs to be dead simple, pragmatic and cheap/simple to operate and update. The only stateful part in our infra at the moment is a PG database. We are going to start work on this in a weeks, so I'm looking for some insights/shortcuts/existing projects that will make our lives easier. The goals is to proc…

PG is great for this. Should handle ~100 or more events per second without much work (but set up a retention policy, and watch out for tables growing to > ~1M rows, as that will kill you during autovacuum).

You can use txid_current_snapshot() and friends to track the last "timestamp". Proper use of locks will help you avoid the complexity associated with long-lived transactions.

Exactly-once semantics can be tricky to guarantee if you do it at the wrong layer of abstraction. Sometimes building exactly-once semantics on top of at-least-once semantics is the way to go.

Kafka and rabbit MQ are both overkill under 100 events/sec. The extra ops overhead isn't worth it. Besides, with PG it'll be nice to be able to always query a couple tables to completely discern the state of the system.

Re: Ask HN: What problem are you close to solving and how can we help?

#445

Post by loa_in_ »

palmtree3000 wrote: post #357

Json diffing. I haven't found any implementations I'd consider good. The problem as I see it is that there are tree based algorithms like https://webspace.science.uu.nl/~swier004/publications/2019-i... and array based algorithms like, well, text diffing, but nothing good that does both. The tree based approach struggles because there's no obvious way to choose the "good" tree encoding of an array. I've currently sett…

The problem is in the syntax of JSON itself. Use JSONL instead

Re: Ask HN: What problem are you close to solving and how can we help?

#446

Post by geitir »

sgk284 wrote: post #426
geitir wrote: post #309

Earlier quoted context omitted.

Unrelated to post, but as you seem well informed in the field, would you agree that if a schema is not likely to change and is controlled as you put it, there is no reason to attempt to store that data as denormalized document? Or at least as you suggest if required for performance the data would still be stored denormalized and where needed materialized / document-ized? At my current company, there seems to be a bel…

> would you agree that if a schema is not likely to change and is controlled as you put it, there is no reason to attempt to store that data as denormalized document As a general rule of thumb, yes. Starting with denormalization often opens you up to all sorts of data consistency issues and data anomalies. I like how the first sentence of the Wikipedia page on denormalization frames it ( https://en.wikipedia.org/wiki…

Thanks for your response. The comparison between infinite ttl cache entries and a denormalized doc is an insight I can't say I've had before and makes intuitive sense

Re: Ask HN: What problem are you close to solving and how can we help?

#447

Post by lonk11 »

dmje wrote: post #442
lonk11 wrote: post #216

Earlier quoted context omitted.

Thanks! It looks like 13 people signed up, which is really encouraging. I wrote about the performance tuning I did in preparation to the Shown HN post: https://linklonk.com/item/277645707356438528 If you have any feedback please add a comment to that post.

Hey, site down?

Yes, I managed to delete the "A" DNS record last night when I was adding records for mail hosting linklonk.com. Sorry! It is up now.

Re: Ask HN: What problem are you close to solving and how can we help?

#448

Post by wizzwizz4 »

lonk11 wrote: post #386
wizzwizz4 wrote: post #331

Earlier quoted context omitted.

When and how are you planning to change your Terms of Use / Privacy Policy? That kind of information should be in the documents. Your privacy policy is currently insufficient. > We only collect your information for the purpose of providing this service. Okay, but what information do you collect? “Your information” is too broad; if you're collecting my retinal scans or matching me to a behavioural analytics profile, y…

I share your concerns about user privacy. The information LinkLonk collects is what you explicitly provide (ratings, etc) and the regular server request logs (which include your ip address, user agent). I clarified this in https://linklonk.com/privacy I do want to add functionality to download your ratings. I'm thinking of exporting the ratings data in either the bookmarks format (ie, the format that browsers use to…

Yeah, that's great now. (You're collecting even less than I expected!) Thanks for making LinkLonk.

JSON would probably be easiest to start with, because it's easy to generate, easy to read and well-defined. Bookmarks would be a nice extra, though.

Re: Ask HN: What problem are you close to solving and how can we help?

#449

Post by ElectricMind »

I have to wake up and pee 2-3 times a night. I put pillow under bed on my leg side and make leg side up. It is working so far. Every 4-5 hours from every 3 hours. I also trying Kegel exercise too. I can't stop drinking water before bed because I have acidity too and I can not sleep with dry throat :)

Re: Ask HN: What problem are you close to solving and how can we help?

#450

Post by oebboe »

red0point wrote: post #428
oebboe wrote: post #367

Earlier quoted context omitted.

I'm currently thinking about starting a very similar project, would you like to talk about it and exchange some learnings?

Sure, how can I reach you?

claudioreiter on telegram or via email hn@dagobert.pw
Post reply on HN