Live data from Hacker News

Ask HN: What are the most important problems in your industry?

news.ycombinator.com

41–50 of 152 posts

Re: Ask HN: What are the most important problems in your industry?

#41

Intersection of trading and data mining. I've said this a few times but we're going through a growth period like AAA video games have over the past 20 years. I used to be that 2 guys could make a video game, then it went to 10, then 50, now its around 200 from what I've last heard. Hedge funds are going through a similar shift. It used to be that one person could manage data cleaning, and algo generation for a fund.…

I think here is an idea isn't unique to Quants/finance: curated datasets.

Hosted data-sets that are fully cleaned, verified and kept up to date. You pay a fee for the feed, which essentially covers initial and on-going curation. Fees would probably be based on usage (dev/test/commercial etc) but also the realistic market value of curation.

There's a mile of difference between a data-set that's been fed through a few cleaners and is 99% right, and one that is thoroughly checked, 99.99% right, and still updated as such with little delay. The former is the "one-man dev looking for easy passive income", the latter is the "quality datasets taken seriously".

Re: Ask HN: What are the most important problems in your industry?

#42

I may be reading between the lines too much and I apologize if I am... But every few months (weeks?) I see a post by a founder-type essentially trying to mine the Hacker News collective brains for startup ideas. It doesn't work that way. The best startups are ones that solve a pain point you yourself have experienced. The idea of a savior who comes in and solving the major problems of an industry they have never work…

> The idea of a savior who comes in and solving the major problems of an industry they have never worked in is not a myth but close to one.

I think Bezos would be a much better example than Musk. Musk is in the process of nudging the car industry a bit (or so SV tends to want to make the rest of the world believe); Bezos basically obliterated small-scale retail as we knew it.

Re: Ask HN: What are the most important problems in your industry?

#43
post #10

I work in the games industry. There are plenty of problems to go around, but I'll pick just one: Discoverability In the "good old days" where 2 people could make a video game, odds are that just shipping something guaranteed you'd make money. But that's no the case anymore now that 1000+ apps come out every day on iOS / Google Play. Of course most of those are crap. But you could be making a great game that caters we…

Also the games industry - I'd add soaring content costs for "AAA" games. Trees, crates, terrain, clothing, props from cups to cars etc. etc. these things take and cost a lot to make and the required fidelity only ever increases. It's unsustainable.

Interesting stuff with scanning real world items and photogrammery but these techniques aren't widely used yet.

Marketplaces for models don't tend to solve this because quality isn't consistent and unique content is preferred.

Re: Ask HN: What are the most important problems in your industry?

#44
post #4

1. Public Transport. 2. Funding/Capacity. 3. More money.

Hi, I'm both a public transport planner and a tech founder (www.podaris.com). I disagree with your 2nd and 3rd points.

Funding and capacity limitations are indeed the cause of problems, but they're not root causes. They're symptomatic of broken political systems, broken business models, and a moribund industry that often delivers shockingly poor value for money.

Similarly, more money is a solution, but it's not necessarily a good solution, if the majority of that money will be lost due to graft, mistakes, and inefficiencies. (McKinsey estimates that around $1 Trillion dollars is wasted every year, across all classes of infrastructure.[1])

(Note: this assessment has a lot of regional variance. It is mostly true in most of America, mostly false in Singapore, and has various levels of applicability in between.)

Anyhow, before simply throwing more money at the problem, I'd recommend taking a look at how the politics, business models, and industry workflows can be shaken up a bit.

1: http://www.mckinsey.com/industries/capital-projects-and-infr...

Re: Ask HN: What are the most important problems in your industry?

#45
1. Jobs/Work

2. Most people don't like their jobs but suck it up. The 9-5 grind, climb-ladder, can't switch careers, lack of meaning, social pressure to have job. Getting laid off, searching new job, financial downsides of being unemployed

3. Restructure the job model/market (flexible choices, live comfortably, security)

Unengaged workers- Gallup poll on American workforce trends http://www.gallup.com/reports/199961/state-american-workplac...

Re: Ask HN: What are the most important problems in your industry?

#46

I may be reading between the lines too much and I apologize if I am... But every few months (weeks?) I see a post by a founder-type essentially trying to mine the Hacker News collective brains for startup ideas. It doesn't work that way. The best startups are ones that solve a pain point you yourself have experienced. The idea of a savior who comes in and solving the major problems of an industry they have never work…

Yes, very true. In case you've started a company and you're fumbling, think about the problems you've encountered in starting a company. Some of the major issues I faced which seemed to me should not be problems ended up being huge companies e.g. several years ago why we had to have our own email server to send confirmation emails (there were no email apis and it's non-trivial to set up an email server) or why it was a pain/unreliable to get a taxi. It's even mystifying why it's so expensive just to start a company...if you start with some kind of reasonable investment, legal fees are between $7500 to $25k.

Re: Ask HN: What are the most important problems in your industry?

#47
post #6

Earlier quoted context omitted.

> The best startups are ones that have solved a pain point you yourself have experience. Yet there is zero money in writing development tools :(

Have you heard of companies like Atlassian and Gitlab?

Those companies are running platforms. Not really a fair comparison.

Besides, I'm not literally saying there is no money in development tools. I'm saying that it's a difficult market.

Re: Ask HN: What are the most important problems in your industry?

#48
1. Embedded (safety and criitical systems)

2. Knowledge sharing within companies/organizations. Formal methods (primarily their absence). Effective use of simulations in design, development, and V&V efforts. Requirements traceability (this is mundane and seems bureaucratic, but it's critical here).

3. Yes, to everything.

For the first, break down information silos and project fiefdoms. Allow for greater flexibility for staff to move across project boundaries so knowledge can be shared more equitably, and people can see other teams work (learn both good and bad things here). Training. Make it a recurring event. Not the crap training many organizations do. Have a seminar series where people come in and present on something, not always directly related to work. Encourage people to write up their lessons learned, and perform and publish post mortems on projects. Take the approach of avoiding blame, focus on correctable errors and faults along the way (these are primarily process faults, not technical ones; where technical they're typically design and not implementation errors).

Formal methods and simulations are much easier to get started with today than ever. I'm not even talking about making a full-blown simulation of the final system, just high level "is this protocol sound" models. Presently working on radios. I don't need to implement a simulation of every detail of the protocol, I just need to know things like: If we add this new message, that must be sent so often, can it actually get broadcast at the correct frequency within the physical constraints of the radio? This turned out to be NO on one project I saw (not worked on), but not discovered until it was implemented (several man-months wasted). A message was supposed to be sent out every X time slots, containing N bytes of data. Each slot allowed you to send MAX size. Other messages also had to be sent out, say every X4 slots with size M. N+M > MAX, meant something wasn't sent. Both were mandatory, by design the protocol couldn't function. Another similar issue, though requiring a more complete simulation/model, was that one of the processors handling some of the messages simply wasn't fast enough. It was required to (worst-case) process N messages within X microseconds, but could actually only process ~N0.75 messages. Admittedly, this was worst case behavior, but by the system's design (protocol requirements, selected hardware, selected data bus, selected program design) it could not achieve the required performance.

The more complete the simulation, though, the better off you are. Technical solutions already exist, it's primarily an issue of finding good case studies or getting an amenable manager to sign off on trying it to demonstrate the cost savings (versus the typical approaches, which in my experience are often significantly late and errorful). Also being at the right stage in a project. Being at the maintenance end, constructing these models/simulations is harder than when you're taking on a novel project.

But, simulations also aid V&V efforts. If you can construct a full(er) simulation of the radio network, your V&V team can start creating test cases, procedures, and models and verifying that they're reasonable. From a protocol perspective, this is relatively easy on our radios, setting aside timing. So ignore time (as a strict concept) and instead focus on time slots. Create a simulation where each tick corresponds to one time slot, let computation run as long as needed. At the end, you'll see what should show up from the radios given some inputs. Run these through your data analysis tools to exercise them, and when you have functioning radios you can use these tools to create simulated network peers (pre-generated network data played back to the radio being tested).

For requirements traceability, just stop using Word and Excel. Use an actual requirements database. I know DOORS sucks, but it's infinitely better than Word and Excel.

Re: Ask HN: What are the most important problems in your industry?

#49
post #8

Mobile phone repairs - lack of education and the ability to access quality replacement parts. You might think that there's no shortage of phone Repairer's out there and your right but you can bet that 90% of them are self taught or eventually taught by someone. Considering the amount of important information we store in phones and the price of the devices it has now become more important to ensure that your phone rep…

Phone manufacturers have an definite interest in stepping on your right to repair.

Re: Ask HN: What are the most important problems in your industry?

#50
1. Which industry do you work in?

In the publishing industry (where I have worked since 1997), the transition from print-only to print-plus-digital that began around 1999 and really got underway after Amazon released the Kindle in 2007 has finished. Now we have an industry in which print and digital co-exist (at different levels – 50/50 for fiction, but more like 80/20 for non-fiction, and even less of digital for more complex product types like Bibles). Currently the growth area is audiobooks, led (of course) by Audible.

2. What are the biggest problems stopping your industry from growing?

Publishers have not really solved these problems:

(a) How to distribute very small publications and receive very small payments? We're still reliant on credit cards for payments, which pushes us to a smallest payment size of about $1.99 or so.

(b) How to increase discoverability? Most publishers are reluctant to post all of their content in a web-searchable and social-shareable form (for somewhat obvious reasons). However, this means that it's hard for them to draw direct traffic to their books.

(c) How to reduce reliance on the behemoth of online retailing? As physical bookstores have died away, publishers have recognized that they are too reliant on one distributor, which is a dangerous position to be in (as that retailer has shown itself very ready to use monopsony powers to bully their suppliers). Most publishers have direct-to-consumer selling operations. But (a) and (b) and other factors mean that they find it extremely difficult to draw traffic to their sites.

3. Can something be done about it?

I have been working on some of these problems in my business (http://blackearthgroup.com). Here is a sketch outline of how I would encourage publishers to solve these problems:

(a) Micropayments are needed, and to do that we need an online currency that can be used to buy content without going through the credit card processing network. Publishers should invest in the development of an online token that they would support on their sites. Customers could then purchase a supply of tokens and use them on publishers' sites to buy content. There are a couple of projects like this in the works. The simplest approach would be to create a coin based on the Ethereum network, and then support that coin for all purchases. (The hardest part of this is probably that the value of the coin would not be completely stable, because Ethereum is not, which means that publishers would have to either adjust their token prices regularly, or would have to live with variability in revenues to sales – this problem is solvable, but it requires a lot of capital to create a value stabilization mechanism.)

(b) Publishers should put all their content online in excerpt chunks using non-discoverable public URLs, then submit it all to the search engines, and start sharing excerpts through social channels. It is true that some of the content would be given away, but that would be limited because each excerpt chunk would not be linked to the others in the same publication – access to one would not grant access to all. Using full-content search and sharing is one of the best ways for them to draw more organic traffic to their own sites. (They'll need to invest in better discovery mechanisms on their sites, too.)

(c) Publishers have a real chance of building a customer base in their own content niches, if they invest in developing a content discovery and purchase experience that is significantly better in that niche than what customers experience on Amazon.

(Cross-posted to my blog: http://blackearthgroup.com/2017/04/20/what-are-the-greatest-... .)

Post reply on HN