Live data from Hacker News

Ask HN: What problem are you close to solving and how can we help?

news.ycombinator.com

41–50 of 486 posts

Re: Ask HN: What problem are you close to solving and how can we help?

#41

We are experiencing very high CPU load caused by tinc [0], which we use to ensure all communication between cloud VMs is encrypted. This is primarily affecting the highest traffic VMs, including the one hosting the master DB. I am starting to consider alternative tools such us wireguard to reduce load, but I am concerned of adding too much complexity. Tinc's mesh network makes setup and maintenance easy. The wireguar…

NoCode: fly.io with its 6pn (out-of-the-box private networking among clusters in the same org).

DIY: envoyproxy.io / HashiCorp Consul for app-space private networking over public interfaces.

LowCode: Mesh P2P VPN network among your clusters with FOSS/SaaS like WireTrustee / tailscale.io / Slack Nebula.

Re: Ask HN: What problem are you close to solving and how can we help?

#42
post #24
post #2

We are having atrocious READ/WRITE latency with our PG database (api layer is django rest framework). The table that is the issue consists of multiple JSON BLOB fields, with quite a bit of data— I am convinced these need to be abstracted to their own relational tables. Is this a sound solution? I believe it is the deserialization in these fields of large nested JSON BLOBS that is causing latency. Note: this database…

Are you just doing primary key lookups? If so, a new index won’t do much as Postgres already has you covered there. If you have any foreign key columns, add indexes on them. And if you’re doing any joins, make sure the criteria have indexes. Similarly, if you’re filtering on any of the nested JSON fields, index them directly. This alone may be sufficient for your perf problems. If it isn’t, then here’s some tips for…

This is super interesting stuff.

First of all, I think the caching layer (which we currently don’t have) is going to be a necessity in the coming weeks as we scale for an additional project (that will be relying on this architecture)

Second of all, it is just PK lookups. We don’t actually have a single fk (contractor did not set up any relations), which makes me think moving all of this replicated JSON data from fields to tables may help.

The queries that are currently causing issues are not filtering out any data but returning entire records. In ORM terms, it is Video.objects.all(), and from a URL param in our GET to the api, limiting the amount of entries returned. What’s interesting is this latency scales linearly, and at the point we ask for ~50 records we hit the maximum raw memory alloc for PG (1GB) causing the entire app to crash.

The solution you propose for s3 blob store is enormously fascinating. The one thing I’d mention is these JSON fields on the Video table have a defined schema that is replicated for each Video record (this is video/sensor metadata, including stuff like gps coords, temperature, and a lot more).

So retrieving a Video record will retrieve those JSON fields, but not just the values: the entire nested BLOB. And does so for each and every record if we are fetching >1

Would defining this schema with something like Marshmallow/JSON-Schema be a good idea when you mention JSON schemas we control? As well as explicitly migrating those JSON fields to their own tables, replaced with an FK on the Video table?

Re: Ask HN: What problem are you close to solving and how can we help?

#43
I am working on two problems:

1. I want to create way to generate electrical power without pollution. Basically, a closed cycle process that releases no pollutants, or electronic waste.

2. I want to do everything I can to eliminate gender bias in the world.

Re: Ask HN: What problem are you close to solving and how can we help?

#44

I'm looking for the best way to implement Multitenancy, Submultitenancy Impersonation with JwtTokens and IdentityServer4 ( dot net). I'm curious on how other people solved it ( by cookies, subdomain, ... ) and if you used a JwtToken for it.

Looking for something like authzed.com?

Re: Ask HN: What problem are you close to solving and how can we help?

#45
post #38

I try to find an agile project management tool that works for us. We run on what many would call Scrum (it’s not actually Scrum). We are on JIRA now, and it’s … JIRA. We tried basically any other tool, including Excel (yes, that is somewhat possible). My problem generally is that tools are slow, planning is cumbersome, visibility is limited and reporting for clients is often even more limited. Heck, I’d even write my…

Depends on workflow and team size. For small teams good fit could be some kanban based tools, for example Trello or GitHub projects.

You could also modern try agile tools, for example Linear. JIRA is good for 100+ teams and complex architectures.

Re: Ask HN: What problem are you close to solving and how can we help?

#48
I have posted this here before- hexafarms.com. I am trying to use ML to discover optimal phenotype for growing plants in vertical indoor farms to a. have the higest quality produce b. to lower the cost of producing leafy green/med plants, etc. within cities itself.

Basically, every leafy green (and herbs, and even mushrooms), can grow in a range of climatic condition (phenotype, roughly) ie temperature, humidity, water, CO2 level, pH, light (spectrum, duration and intensity) etc. As you might have seen around the world there is a rise in indoor vertical farms, but the truth is that 50% of those are not even profitable. My startup wants to discover the optimal parameters for each plant grown in our indoor vertical farm and eventually I would let our AI system control everything (something like alphaGo, but for growing plant X (lettuce, kale, chard, ). Think of it as reinforcement learning with live plants! I am betting on the fact that our startup will discover the 'plant recipes' and figure out the optimal parameters for the produce that we would grow. Then, the goal is that cities can grow food cheaper in more secure and sustainable way than our 'outsourced' approach in country side or far away lands.

So now I have secured some funding to be able to start working on optimizations, but I realized that *hardware* startups are such a different kind of beast (I am a good software product dev though, I think). Honestly, if anyone with experience in hardware related startups (or experience in the kind of venture I am in) would just want to meet me and advise me, I would take it any day. Being the star of the show, it's hard for me to handle market segmentation, tech dev, team, next round of funding, European tech landscape, etc. I am foreseeing so many ways that our decisions can kill my startup, all I need is advise from someone qualified/experienced enough. My email: david[at]hexafarms.com

Re: Ask HN: What problem are you close to solving and how can we help?

#49
post #2

We are having atrocious READ/WRITE latency with our PG database (api layer is django rest framework). The table that is the issue consists of multiple JSON BLOB fields, with quite a bit of data— I am convinced these need to be abstracted to their own relational tables. Is this a sound solution? I believe it is the deserialization in these fields of large nested JSON BLOBS that is causing latency. Note: this database…

I think your heart shouldn't quail at the thought of re-schemaing 5-6GB! I'm going to claim that the actual migration will be very quick.

This is an affirmation I’ve been longing to here, lol!

I’ve already done the legwork, cloning to the current prod DB locally and playing around with migrations, but the fear of applying anything potentially-production breaking is scary to a dev who has never had to work on a “critical” production system!

Re: Ask HN: What problem are you close to solving and how can we help?

#50
post #6

We are working on a totally new way to do cold fusion, our only problem is getting enough new fuel into the reactor without disturbing the running process. Any help would be greatly appreciated.

Disclaimer : I know nothing about anything.

Hawking radiation ?

Laser tunnel ?

Magnetic canon ?

Centrifugal launcher ?

Vacuum diffusion ?

Electrical beam lensing ?

Post reply on HN