Data isn't just being collected from your phone, it's being used to score you • Hacker News Board

Re: Data isn't just being collected from your phone, it's being used to score you

#71

Post by api » Sat, Aug 01, 2020, 5:07 PM UTC

TeMPOraL wrote: post #69

icedistilled wrote: post #39

Earlier quoted context omitted.

I mean, it's already standard practice (All states except CA) to use credit score in things like pricing your auto insurance. It's only a tiny leap to incorporate a similar magic number some company comes up with. Actually due to competition, if it correlates to risk, all companies will literally be forced to use the magic numbers or go into an adverse selection death spiral. Unless there is regulation against it, li…

I wonder at which point it becomes a self-fulfilling property - at which point decisions based on data-driven pigeonholing actually lock people on the paths "discovered" in the numbers? E.g. if a young adult gets classified as "disorderly, drunk, unsuitable for reproduction, suitable only for low-skill work" based on their history of college partying, and then consequently denied work and social opportunities (as eve…

This is kind of the meta plot of Gattaca. You get what you measure, so if we try to measure "potential" we end up getting only the potential our measurements say we have.

Re: Data isn't just being collected from your phone, it's being used to score you

#72

Post by srveale » Sat, Aug 01, 2020, 5:08 PM UTC

rhn_mk1 wrote: post #50

xwdv wrote: post #42

Earlier quoted context omitted.

Well then these people are fucked. Much like how using no credit means you have no credit history and a low credit score, you will have a very hard time trying to buy a house. If you spend your whole life being anonymous, be prepared to be treated as a high risk consumer who cannot participate in certain things, because of a low social score.

That may well be, but that's not the spirit. Instead, "if we all resisted more, we'd be free of that crap". Granted, this is the tragedy of the commons. By acting in one's own interest: get a good score, people poison the well for everone: everyone must work and self-censor to keep a good score.

What percentage of people do you think are willing to take even one of the steps mentioned in OP's comment? I'm pessimistic.

Re: Data isn't just being collected from your phone, it's being used to score you

#73

Post by joejerryronnie » Sat, Aug 01, 2020, 5:08 PM UTC

TeMPOraL wrote: post #69

icedistilled wrote: post #39

Earlier quoted context omitted.

I mean, it's already standard practice (All states except CA) to use credit score in things like pricing your auto insurance. It's only a tiny leap to incorporate a similar magic number some company comes up with. Actually due to competition, if it correlates to risk, all companies will literally be forced to use the magic numbers or go into an adverse selection death spiral. Unless there is regulation against it, li…

I wonder at which point it becomes a self-fulfilling property - at which point decisions based on data-driven pigeonholing actually lock people on the paths "discovered" in the numbers? E.g. if a young adult gets classified as "disorderly, drunk, unsuitable for reproduction, suitable only for low-skill work" based on their history of college partying, and then consequently denied work and social opportunities (as eve…

A Brave New World did it easier, simply sort each person into a genetically pre-determined path prior to birth. No reason to go through all that messy data collection and analysis.

Re: Data isn't just being collected from your phone, it's being used to score you

#74

Post by annoyingnoob » Sat, Aug 01, 2020, 5:10 PM UTC

shuntress wrote: post #66

annoyingnoob wrote: post #61

Earlier quoted context omitted.

I was rear-ended in stop-and-go traffic. The same person was behind me for miles. He was looking at his phone and ran into me - totally and completely not my fault. He refused to give me his insurance info. I made a claim against my own insurance. I'm pretty sure the insurance company went after him and recovered costs. How exactly should that influence my rates? This is exactly why I pay for insurance, service I alr…

Your insurance company would prefer to cover people who notice the reckless driver then pull over to get out of their path. You did not do that, so they raise your rates. Edit: not that you were at fault or did anything wrong. My point is that the insurance companies strongest incentive is to pay out as little as possible. This includes payments made where their client was not at fault. If it were up to them, everyon…

They recovered their costs. I already pay for the service. Tracking is just another excuse to raise rates, which as you note is the goal (bring money in, don't let it out). When rates have nothing to do with liability is it really insurance?

To quote wikipedia: If the likelihood of an insured event is so high, or the cost of the event so large, that the resulting premium is large relative to the amount of protection offered, then it is not likely that the insurance will be purchased, even if on offer. Furthermore, as the accounting profession formally recognizes in financial accounting standards, the premium cannot be so large that there is not a reasonable chance of a significant loss to the insurer. If there is no such chance of loss, then the transaction may have the form of insurance, but not the substance (see the U.S. Financial Accounting Standards Board pronouncement number 113: "Accounting and Reporting for Reinsurance of Short-Duration and Long-Duration Contracts"). https://en.wikipedia.org/wiki/Insurance

Re: Data isn't just being collected from your phone, it's being used to score you

#75

Post by patmorgan23 » Sat, Aug 01, 2020, 5:11 PM UTC

justanotherc wrote: post #28

Karunamon wrote: post #13

Earlier quoted context omitted.

I think that people wind up worried about the wrong thing on these systems. The worst case scenario, where some all-knowing system negatively impacts you because it's rating your behavior on some opaque scale, isn't the one we should be worried about. That by itself is scary enough, but the much more likely case is the one where this system is rating you based on wrong information , and given Finagle's law.. Especial…

The justice system has built into it the requirement to justify allegations against you though. By design everything in the justice system is transparent or its not admissible. So an opaque "score" of any sort wouldn't ever be acceptable evidence.

Predictive policing

Re: Data isn't just being collected from your phone, it's being used to score you

#76

Post by joejerryronnie » Sat, Aug 01, 2020, 5:12 PM UTC

annoyingnoob wrote: post #20

cosmodisk wrote: post #5

People are complex. If some piece of code is used to predict my employability based on my facial expressions, there's not much to add. It's already as bad as it gets: you can't say/write anything non vanilla in public because 10 years later some HR snooping app will analyze the sentiment on the post and outright reject you. And we all kind of blindly walking into this no questions asked.

At least the Chinese system is out in the open. Many private companies surveilling everyone without anyone's knowledge is terrible. Companies using these scores are really only targeting a group of people that allow the collection to happen. My car insurance company would sure love to put a tracker in my car but there is no way in hell that is going to happen. I'm sure my driving style would qualify me for higher rat…

> My car insurance company would sure love to put a tracker in my car but there is no way in hell that is going to happen.

If you take your cell phone in the car with you, they are already tracking you.

Re: Data isn't just being collected from your phone, it's being used to score you

#77

Post by TeMPOraL » Sat, Aug 01, 2020, 5:14 PM UTC

amoshi wrote: post #34

cosmodisk wrote: post #5

People are complex. If some piece of code is used to predict my employability based on my facial expressions, there's not much to add. It's already as bad as it gets: you can't say/write anything non vanilla in public because 10 years later some HR snooping app will analyze the sentiment on the post and outright reject you. And we all kind of blindly walking into this no questions asked.

>And we all kind of blindly walking into this no questions asked. It's already happening, now, unfortunately. All your tweets (posted & liked) etc analyzed and flagged for bad language, drug/alcohol mentions, bigotry etc. and a report generated for HR. https://news.ycombinator.com/item?id=22211363 The company in question is used by Sterling and HireRight, so this isn't some unknown/niche company.

Holy shit.

The tweet thread under the link is well worth seeing.

Taken from the company in question's site (https://fama.io/product/):

> What Fama finds...

> Our machine learning technology has flagged hundreds of thousands of instances of misogyny, bigotry, racism, violence and criminal behavior in publicly available online content.

If you go and compare what the product actually seems to do - tracking down your Twitter account and grepping every Tweet you interacted with against a list of "thoughtcrime" words (like "hell" or "ass") - you can almost feel the next AI winter coming. How many more bullshit companies calling their fake, broken and trivial technology "AI" or "Machine Learning" will it take until the whole field of ML gets derailed by bad reputation? At least, (as much as I know history), the last AI winter involved companies trying but failing at AI. This time around, they're not even trying.

Re: Data isn't just being collected from your phone, it's being used to score you

#78

Post by joejerryronnie » Sat, Aug 01, 2020, 5:16 PM UTC

Swizec wrote: post #16

cosmodisk wrote: post #5

People are complex. If some piece of code is used to predict my employability based on my facial expressions, there's not much to add. It's already as bad as it gets: you can't say/write anything non vanilla in public because 10 years later some HR snooping app will analyze the sentiment on the post and outright reject you. And we all kind of blindly walking into this no questions asked.

> If some piece of code is used to predict my employability based on my facial expressions Funfact: when I first moved to USA I would practice emoting in the mirror so Americans could read my face and wouldn’t penalize me during interactions. Now people back home say I “grin like an american” > can’t say anything in public Ever noticed how Gen Z is switching back to private chatrooms and message groups? Public stuff…

Millennials just keep getting screwed - the generation that was born just in time to post every thought and action to the online public but also born to early to realize what a bad idea that was.

Re: Data isn't just being collected from your phone, it's being used to score you

#79

Post by LorenPechtel » Sat, Aug 01, 2020, 5:19 PM UTC

perl4ever wrote: post #55

rootusrootus wrote: post #52

Aside from being a mostly clickbait puff piece, the solution to this is regulation. All companies collecting data this way need to be subject to exactly the same requirements as the credit bureaus. Citizens need to be told what data is collected, why, when it gets used, and they need easy access to seeing their own data, as well as a well-regulated method for correcting it. And we could just dramatically limit how mu…

While we're at it, maybe we could enforce easier access to credit information. One report per year with no credit score is absurdly restrictive.

It was reasonable at the time it was put into place. It's now so much easier to deal with information, reform is needed but it's not going to happen unless the Democrats are in control.

What I would like to see is a rule that if a company collects data for more than in-house use (so it doesn't apply to a company who simply has records of it's customers) they must make a reasonable effort to notify you and allow you to examine the data and challenge anything you believe to be incorrect.

Re: Data isn't just being collected from your phone, it's being used to score you

#80

Post by zozbot234 » Sat, Aug 01, 2020, 5:19 PM UTC

TeMPOraL wrote: post #69

icedistilled wrote: post #39

Earlier quoted context omitted.

I mean, it's already standard practice (All states except CA) to use credit score in things like pricing your auto insurance. It's only a tiny leap to incorporate a similar magic number some company comes up with. Actually due to competition, if it correlates to risk, all companies will literally be forced to use the magic numbers or go into an adverse selection death spiral. Unless there is regulation against it, li…

I wonder at which point it becomes a self-fulfilling property - at which point decisions based on data-driven pigeonholing actually lock people on the paths "discovered" in the numbers? E.g. if a young adult gets classified as "disorderly, drunk, unsuitable for reproduction, suitable only for low-skill work" based on their history of college partying, and then consequently denied work and social opportunities (as eve…

Yup, this is what "structural" social prejudice and discrimination are all about, once you strip away all the pointless and meaningless rhetoric that's somehow supposed to be "about" these issues. It's a self-perpetuating equilibrium of basic social conditions and superstructure (viz. discourse, supporting ideas, commonly-held worldviews etc.) that create a nearly unescapable "trap" of invisible oppression.