Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep-Fried Data

idlewords.com

Post reply on HN
Print view View original

81–90 of 149 posts

Re: Deep-Fried Data

#81

Post by intended »

keyle wrote: post #44
pimlottc wrote: post #19

> Many [programmers] work jobs that are intellectually stimulating, but ultimately leave nothing behind. There is a large population of technical people who would enjoy contributing to something lasting. This hits pretty close to home.

Same here. I'm battling with this thought a lot. Beyond jobs, I think there should be communities of developers, designers, producers, writers, getting together and figuring out this stuff. And I don't mean open source projects. Let's group together smart people wanting to make a difference and have a hit list of things we (people) actually need. A group that would organise people into mission driven development. I'm…

That used to be what the net and all of the coder based communities were about. Thats why I used to come to slashdot, or HN when I found the net.

And we've reached that point when the creatures (the firms and technologies built by those people) of that culture are diverging from ideals of the culture.

But the above comment is inherently empty - any successful system will eventually expand till it reaches a barrier of complexity which cannot be overcome on its own.

Figuring out what to do is the challenge.

The things that worked were coders having free time to spend on interesting projects. But I suspect, that we've better understood the value of coder time, and the major firms are now paying the correct amount to keep coders busy.

The market BS is a good thing for coders in the short and medium term. People who understand finance and strategy are willing to pay what it takes today, to own a chance at being richer tomorrow.

If you have a neat hobby? communities will help you get better at it. Maybe if its really good, you can convert that into a product/firm and possibly a good exit. If that happens you won't have to worry about it ever, and you'll be that thing which is respected among your peers - a succesful serial entrepreneur. You would have done the hard thing (product creation, team management, finance, successful exit).

In a group of people who respect ability and excellence, its hard not to think of the guy who did the harder job as meritorious.

In short: I don't think theres a market solution for a new market normal.

Its easier to figure out you are cut of a different cloth, recognize the market dynamics for what they are, and make time to build whatever it is you want to build.

Eventually a lot of other coders are going to come to similar realizations (provided the cultural bubble online isn't too distortive)

Re: Deep-Fried Data

#82

Post by big_surprise »

chewxy wrote: post #15

Regarding Maciej's fears about machine learning - I've written about this before, and even right now I'm not sure where I stand exactly, except that tweaking the algorithms to compensate for bias is definitely not the right answer: if you look at the mirror and don't like what you see, you don't draw on top of the mirror to accentuate the result! You go on a diet! I liked the idea of data gardening, but the thought o…

  So the question is how do we democratize that?
Commenter wtracy has already linked to the FSF's list of High Priority Free Software Projects... From there, look into what they have to say about free wifi (and in particular, but not limited to the OrangeMesh package): http://www.fsf.org/campaigns/priority-projects/free-software...

If a convincing case could be made that the benefits to National Security outweigh the costs to the copyright cartels, I'd be willing to bet that young secondary-schoolers would have a blast with a decently designed curriculum that includes a working student-to-student mesh-network as one of its goals.

Re: Deep-Fried Data

#83

Post by [deleted] »

keyle wrote: post #44
pimlottc wrote: post #19

> Many [programmers] work jobs that are intellectually stimulating, but ultimately leave nothing behind. There is a large population of technical people who would enjoy contributing to something lasting. This hits pretty close to home.

Same here. I'm battling with this thought a lot. Beyond jobs, I think there should be communities of developers, designers, producers, writers, getting together and figuring out this stuff. And I don't mean open source projects. Let's group together smart people wanting to make a difference and have a hit list of things we (people) actually need. A group that would organise people into mission driven development. I'm…

[deleted]

Re: Deep-Fried Data

#84

Post by chewxy »

Singletoned wrote: post #79
pcmaffey wrote: post #59

Machine learning does not have less bias than human researchers. It is simply magnified at scale. And that scale is exactly the state of the internet. There is so much data available to study and understand, that we absolutely need better tools, like machine learning or whatever we want to call it, to help us keep up. Shit's moving faster than our human perception can handle, especially for those who didn't grow up w…

> Machine learning does not have less bias than human researchers. You are right that machine learning gains bias from the humans that created it, but unless they managed to transfer 100% of their biases to it, it will always have less bias.

We impart our bias on ML algorithms by choosing what data to use to train the AI on.

The problem, I think is one of self-selection.

Consider two hypothetical social networking websites - Friendface and FaceSpace. Friendface's userbase are mostly white users, while FaceSpace catered mostly to urban, black populations. And it would make sense too - you would only join a social network if your friends are on it. If you're white, chances are the majority of your friends are also white. And vice versa.

So Friendface is a lot more active on their ML front. The problem is when Friendface releases their data - because they're more active on the ML front, and ML scientists love to not have to collect their data, what happens is more and more models are trained on the Friendface data and more and more models are being optimized based on Friendface data. Apparent "structural" racism happens. Tumblrinas all pounce on it as if it were the biggest oppressive struggles of their lives.

A very cute thing to imagine in this scenario would be to imagine FaceSpace suddenly got good at NLP, and open sources their statistical language model. Recall that FaceSpace users are more likely to use AAVE in their communication, so what do you think the statistical language model would be?

In the original article, Maciej mentions "going to the community" - using crowd wisdom to handle these sorts of thing, and preferring to use open standards as opposed to silo'd standards (like writing your blog post on facebook... why??!!). While that sounds like a good idea, like I've mentioned in my other comment, it also sounds tiring as hell.

Firms act rationally (more or less)... ML is driven by huge companies with huge datasets. Why would they need to prune external datasets when they could just do their ML research with a few SQL queries?

Re: Deep-Fried Data

#85

Post by chewxy »

big_surprise wrote: post #82
chewxy wrote: post #15

Regarding Maciej's fears about machine learning - I've written about this before, and even right now I'm not sure where I stand exactly, except that tweaking the algorithms to compensate for bias is definitely not the right answer: if you look at the mirror and don't like what you see, you don't draw on top of the mirror to accentuate the result! You go on a diet! I liked the idea of data gardening, but the thought o…

So the question is how do we democratize that? Commenter wtracy has already linked to the FSF's list of High Priority Free Software Projects ... From there, look into what they have to say about free wifi (and in particular, but not limited to the OrangeMesh package): http://www.fsf.org/campaigns/priority-projects/free-software... If a convincing case could be made that the benefits to National Security outweigh the…

I actually meant democratizing the network effect that Medium has. The "free marketing" bit.

I mean, right now one can pretty freely go write up a blog using self-hosted wordpress, octopress, pelican, hugo or whatever. But choosing that over Medium can sometimes mean a lot more work to put in. But if we can democratize the ease-of-use and the good bits of Medium/Facebook/Twitter... Maciej's end statement about "using open standards, write text in text, images in images" would have been achieved.

The problem is that corporations now create a significantly more compelling version (in most criteria - UX, UI, etc) of the Free and Open versions out there.

Re: Deep-Fried Data

#86

Post by coldtea »

wtracy wrote: post #76
coldtea wrote: post #66

Earlier quoted context omitted.

> You described exactly what some of the largest, most successful FOSS projects (Firefox, KDE, Gnome, Libre Office, FreeBSD) are already doing. They "make a difference"? How exactly? At best, I can understand that for Firefox.

By providing quality software that lets me and thousands of others get useful work done, and not forcing us to accept onerous licensing terms in the process? But hey, none of those can cure cancer, so what's the point, right?

>By providing quality software that lets me and thousands of others get useful work done, and not forcing us to accept onerous licensing terms in the process?

That software (desktop Linux/UNIX) comes for free, and yet, only 1% or less (from browser stats of major traffic points) seem to opt to use it as their desktop.

Is this the kind of difference the parent was describing? Letting a small minority of people avoid "onerous licensing terms" that billions of others are OK and can get "get useful work done" with?

In the server, of course, it's a whole different story.

Re: Deep-Fried Data

#87

Post by coldtea »

AndrewKemendo wrote: post #80
coldtea wrote: post #63

Earlier quoted context omitted.

> Machine learning does not have less bias than human researchers. It is simply magnified at scale. Scale differences can and often do lead to qualitative differences. Individual (or aggregate) human researchers are not hooked up in huge services to make inferences and deductions automatically about billions of people. Besides those machine learning tools, beside the huge data sets, are programmed in their general fr…

Individual (or aggregate) human researchers are not hooked up in huge services to make inferences and deductions automatically about billions of people Yes they (we) are. It's the same data set. TV, movies, papers, internet videos et al. is all the same biased, labeled data that is being fed (watched, listened to etc...) to machines. You automatically make inferences and deduce things about people based on labeling a…

>Yes they (we) are.

I didn't say researchers and/or individual people are not making such judgements about billions of others (e.g. "the Chinese suck/are great").

I said they are not "hooked up in huge services" to make them automatically for billions of individuals -- like an ML algorithm used by Google or Amazon or some government agency etc would do.

Re: Deep-Fried Data

#88

Post by big_surprise »

chewxy wrote: post #85
big_surprise wrote: post #82

Earlier quoted context omitted.

So the question is how do we democratize that? Commenter wtracy has already linked to the FSF's list of High Priority Free Software Projects ... From there, look into what they have to say about free wifi (and in particular, but not limited to the OrangeMesh package): http://www.fsf.org/campaigns/priority-projects/free-software... If a convincing case could be made that the benefits to National Security outweigh the…

I actually meant democratizing the network effect that Medium has. The "free marketing" bit. I mean, right now one can pretty freely go write up a blog using self-hosted wordpress, octopress, pelican, hugo or whatever. But choosing that over Medium can sometimes mean a lot more work to put in. But if we can democratize the ease-of-use and the good bits of Medium/Facebook/Twitter... Maciej's end statement about "using…

Right, I got that. I guess I meant to say- start young. Provide a compelling alternative to ad-driven consumerism... Otherwise, it seems that you're just talking about creating another new corporation with an more compelling (even if nonsensical) reason to adopt.

Re: Deep-Fried Data

#90

Post by pjc50 »

keyle wrote: post #44
pimlottc wrote: post #19

> Many [programmers] work jobs that are intellectually stimulating, but ultimately leave nothing behind. There is a large population of technical people who would enjoy contributing to something lasting. This hits pretty close to home.

Same here. I'm battling with this thought a lot. Beyond jobs, I think there should be communities of developers, designers, producers, writers, getting together and figuring out this stuff. And I don't mean open source projects. Let's group together smart people wanting to make a difference and have a hit list of things we (people) actually need. A group that would organise people into mission driven development. I'm…

Some friends of mine came up with https://www.mysociety.org/ as an answer to this.

> "We should be focused on solving problems, instead of being focused on escaping the rat race, to then be able to solve problems."

Sometimes one person's problem is another person's solution, and vice versa; this is where so much of politics comes from. A lot of people are invested in making sure certain problems stay unsolved, or even unacknowledged.

Post reply on HN