Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

June 2023 Data Dump is missing

meta.stackexchange.com

Post reply on HN
Print view View original

21–30 of 268 posts

Re: June 2023 Data Dump is missing

#21

Post by josephcsible »

bioemerl wrote: post #20
CoastalCoder wrote: post #19

Earlier quoted context omitted.

IANAL, but I'm curious: Suppose that deep-pocketed AI companies were paying Reddit, Stack Overflow, etc. to make it harder for other AI upstarts to access those data. I.e., to build a mote by denying competitors access to previously accessible data sets. Would that violate antitrust laws in various major markets?

Given that this seems to happen all the time without antitrust issues it probably wouldn't, even though I feel like it should. What we need is a legal way for companies to keep the data open, but also require OpenAI and friends to pay them for it.

> What we need is a legal way for companies to keep the data open, but also require OpenAI and friends to pay them for it.

Couldn't that be accomplished by a law or ruling that using something for training AI doesn't exempt you from having to follow its license? OpenAI is already in blatant violation of both the "BY" and "SA" parts of the existing license.

Re: June 2023 Data Dump is missing

#22

Post by biorach »

qbasic_forever wrote: post #18
Kiro wrote: post #9

Earlier quoted context omitted.

Why is that surprising?

It's basically a wiki with well under a terrabyte of data total and a billion requests a month (modest load in the grand scheme of web apps). It runs on less than 10 servers ( https://www.datacenterdynamics.com/en/news/stack-overflow-st... ). It's kind of bonkers to have hundreds of engineers supporting a handful of servers.

500 employees, not 500 engineers.

Re: June 2023 Data Dump is missing

#24

Post by kiba »

lumb63 wrote: post #16

This, along with recent Reddit goings-on has made me realize a major risk with the current structure of online communication. Take either Reddit or Stack Exchange as examples. They build a platform, and users contribute their time, thought, energy, and knowledge to build a community on that platform. Those companies can then gatekeep and restrict access to all that the community built, when all they did is provide th…

Perhaps these sort of things shouldn't be for profit enterprise, given the inability of companies to not slaughter the goose that lay the golden eggs.

Re: June 2023 Data Dump is missing

#25

Post by arp242 »

mindcrime wrote: post #15
endisneigh wrote: post #10

Not surprising - why would any content driven business want all of their stuff to be vacuumed up for free?

It's not really their stuff though. The content comes from the users. And part of the reason users are willing to contribute to SE is because of the licensing model and the fact that the data is available outside of SE. Obviously this is more important to some users than others, and probably some percentage don't care about it at all. It's hard to say what those percentages actually look like though. IF (and it's def…

> they are fundamentally changing the basic understanding between users and SE

Given the way they've communicated in the last 2 weeks or so, this seems pretty clear. Before we had employees engaging as real human beings all over the place, and you were talking to Jon, Tim, Robert, Shog, etc. and not "Mr. Ericson, title such-and-such, representing Stack Exchange Inc."

Now all we have is a bunch of announcements, with no discussion, engagement, or even a recognition that anything is even being read. It feels like pissing in the wind – disagreement is one thing, reasonable people can disagree, but ignoring is so much worse; it's like you're not even taken serious.

Stack Exchange has gone through various phases (e.g. the "Jeff era" was different from the "stagnation era" that followed after he left), but the implied social contract was always that the community would offer their spare time and in return they would get a platform and some voice in how that platform is run. There have certainly been moments of friction in this relationship, but the basics of it never changed until now (not even with the whole debacle surrounding the firing of a moderator a few years back).

Re: June 2023 Data Dump is missing

#26

Post by semi-extrinsic »

CoastalCoder wrote: post #19
bioemerl wrote: post #12

It's unfortunate we are seeing all of these data platforms get locked off, because this is not going to affect AI development from big companies, it's only going to affect the ability for individuals to run AI development of any form in their home. I hope the data that has been found so far is going to big enough going forward, but it's incredibly unfortunate that this is happening. I hope all the people making these…

IANAL, but I'm curious: Suppose that deep-pocketed AI companies were paying Reddit, Stack Overflow, etc. to make it harder for other AI upstarts to access those data. I.e., to build a mote by denying competitors access to previously accessible data sets. Would that violate antitrust laws in various major markets?

Nitpick, also because the contrast is kind of funny:

mote: a small particle, speck, atom, "mote of dust"

moat: a deep ditch, often filled with water, as a first line of defence around a castle.

Re: June 2023 Data Dump is missing

#28

Post by endisneigh »

bioemerl wrote: post #20
CoastalCoder wrote: post #19

Earlier quoted context omitted.

IANAL, but I'm curious: Suppose that deep-pocketed AI companies were paying Reddit, Stack Overflow, etc. to make it harder for other AI upstarts to access those data. I.e., to build a mote by denying competitors access to previously accessible data sets. Would that violate antitrust laws in various major markets?

Given that this seems to happen all the time without antitrust issues it probably wouldn't, even though I feel like it should. What we need is a legal way for companies to keep the data open, but also require OpenAI and friends to pay them for it.

> What we need is a legal way for companies to keep the data open, but also require OpenAI and friends to pay them for it.

inherently not possible as then it would not be "open" to begin with.

Re: June 2023 Data Dump is missing

#29

Post by arp242 »

qbasic_forever wrote: post #18
Kiro wrote: post #9

Earlier quoted context omitted.

Why is that surprising?

It's basically a wiki with well under a terrabyte of data total and a billion requests a month (modest load in the grand scheme of web apps). It runs on less than 10 servers ( https://www.datacenterdynamics.com/en/news/stack-overflow-st... ). It's kind of bonkers to have hundreds of engineers supporting a handful of servers.

- Sales people, account managers, etc. for their ads business

- Sales people, account managers, etc. for their Teams product.

- Sales people, account managers, etc. for their Enterprise self-hosted product.

- Sales people, account managers, etc. for sponsored tags, collectives, etc.

- Support for the above (and the public Stack Exchange sites).

- Engineers for the above (and the public Stack Exchange sites).

- Community managers (who, among other things, fight abuse).

It all adds up. From what I remember most people working for SO weren't engineers, not even years ago (many were involved with the jobs site back then). There used to be a "Our Team" page which listed everyone who worked for SO, but it seems that's gone now.

Re: June 2023 Data Dump is missing

#30

Post by bioemerl »

endisneigh wrote: post #28
bioemerl wrote: post #20

Earlier quoted context omitted.

Given that this seems to happen all the time without antitrust issues it probably wouldn't, even though I feel like it should. What we need is a legal way for companies to keep the data open, but also require OpenAI and friends to pay them for it.

> What we need is a legal way for companies to keep the data open, but also require OpenAI and friends to pay them for it. inherently not possible as then it would not be "open" to begin with.

Open except you have to pay if it gets big enough seems perfectly reasonable to me.

I understand the idea, it's not truly open in that case, but so long as the ability to build new things on it and prosper from it is preserved im alright.

The key is that it's not doing something like trying to restrict you from using it in a certain way, only requiring you give a fair share of profits.

This was, fun fact, the original purpose of patents. They weren't designed to keep things closed and owned by individuals, they were designed to allow people to freely share and make a profit so that ideas could be built on by each other. The patent system is turned into this corrupted terrible mess where things are almost never shared through licensing or payment, and it's just a way to build monopolistic enterprises nowadays.

An open source system that allows for this sort of payment would also allow for many many more things to be open when currently the bad actors who will build and take that work and just never pay you back for it.

Post reply on HN