Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

June 2023 Data Dump is missing

meta.stackexchange.com

Post reply on HN
Print view View original

91–100 of 268 posts

Re: June 2023 Data Dump is missing

#91

Post by wolfgang42 »

albertzeyer wrote: post #58

Is this such a big problem? You could still scrape all the data, or not?

Yeah, this is a baffling part of this—you can still scrape (for now, I guess), if you have the time and effort to do so. Disabling the dump makes it harder only if you have e.g. a shoestring budget.

For example, my hobby search engine got started because I found out about these dumps and decided it would be an interesting challenge to try to work with them[1]. If I’d needed to build a scraper first the project would never have gotten off the ground.

[1]: https://search.feep.dev/blog/post/2021-09-04-stackexchange

Re: June 2023 Data Dump is missing

#92

Post by dylan604 »

"Just sorta stating the obvious here, but the timing of this is unbelievably terrible; I actually can't fathom a worse time for this call to be made than in light of this week. –zcoop98"

Or, it's exactly the best time to do it. Doing it now allows your news to get blended in with the Reddit news. Doing it later after Reddit chatter settles down means all of the chatter is directed squarely at you.

Re: June 2023 Data Dump is missing

#93

Post by [deleted] »

lumb63 wrote: post #16

This, along with recent Reddit goings-on has made me realize a major risk with the current structure of online communication. Take either Reddit or Stack Exchange as examples. They build a platform, and users contribute their time, thought, energy, and knowledge to build a community on that platform. Those companies can then gatekeep and restrict access to all that the community built, when all they did is provide th…

[deleted]

Re: June 2023 Data Dump is missing

#94

Post by ilyt »

EVa5I7bHFq9mnYK wrote: post #86
the_pwner224 wrote: post #33

Earlier quoted context omitted.

A decentralized system will never work because 99% of users do not care at all; the centralized systems are easier to sign up for and use. It's been demonstrated over and over and over again. Even if the underlying tech is decentralized, the community will settle around one or a few big instances (for example, Gmail and GitHub) which often end up having significant control over the trajectory of the entire ecosystem.…

Between gmail and your own email server, there are thousands of medium size email providers that work ok, so email is a bad example, decentralization still works there. As for GitHub, the coders of all people should have known better than to pile into one site and sell their s̶o̶u̶l̶s̶ code to devil for a little bit of convenience.

It does because e-mail is not a social network.

You don't "join" a gmail to see social streams of posts. There is no penalty or friction to start with some small e-mail provider.

Re: June 2023 Data Dump is missing

#95

Post by grumpymouse »

benrutter wrote: post #67
the_pwner224 wrote: post #33

Earlier quoted context omitted.

A decentralized system will never work because 99% of users do not care at all; the centralized systems are easier to sign up for and use. It's been demonstrated over and over and over again. Even if the underlying tech is decentralized, the community will settle around one or a few big instances (for example, Gmail and GitHub) which often end up having significant control over the trajectory of the entire ecosystem.…

I don't know that I agree. I think most people don't care about decentralization but they do care about the effects it brings. Email is a great example where most people wouldn't be interested in a version of email that only let's you email other @gmail.com users. Having a email address that can contact anyone, a phone number that can ring any other phone number etc instead of being locked into a single corporation n…

I think things like being able to contact anyone are important to people, but decentralisation doesn't necessarily provide that (e.g. if I sign up on a Mastodon instance will I be able to see the messages of everyone on every Mastodon instance, and will they be able to see mine? Will I even know if somebody I care about can see my messages or not?)

I think decentralisation is not a selling point to most people. It's an implementation detail that they're happy to go along with but it's a negative if it make the experience worse, makes everything more complicated, if they can't talk to the people they know IRL, etc.

Re: June 2023 Data Dump is missing

#96

Post by ethanbond »

ImPostingOnHN wrote: post #70
ethanbond wrote: post #64

Earlier quoted context omitted.

Really? The takeaway isn’t rather that rent-seeking AI models need to figure out a way to reimburse companies and communities who’ve stored up all this capital? Seems to me SO built and delivered huge, huge amounts of value and it’s now all at risk because multibillion dollar companies are free riding.

SO delivered some value, the users are the ones who delivered huge, huge amounts of value

Right, which SO (and not some other site) managed to entice.

Somehow users didn’t flock to my ethansusefulprogrammerquestionandanswers.com ¯\_(ツ)_/¯

Re: June 2023 Data Dump is missing

#97

Post by Zetice »

nukeman wrote: post #73
Zetice wrote: post #56

Earlier quoted context omitted.

The "value" of Reddit as a website is vastly overrated anyway. There's nothing on Reddit that can't be obtained elsewhere, folks just get stuck in patterns that are familiar and presume it's because options are limited. The world will continue to spin without Reddit, or if Reddit isn't popular anymore, or if Reddit kicks all of its current users out, and so on.

Reddit’s value these days is as a super-forum. Instead of needing five logins and five accounts for separate hobbies, you can have one login, one account, and if you occasionally want to comment or ask questions about something else, you can do so without having to create another account. Doesn’t hurt that Reddit has a very good network effect.

That's how you use it, but I'd argue that 90% of Reddit's users don't operate the same way, considering they don't even have accounts to begin with.

In fact, I would even argue that Reddit is actively trying to push users who use Reddit this way off of the platform, as they're not as easily monetized. Reddit is notorious for having low conversion rates on its ads and extremely high ad blocking rates.

Re: June 2023 Data Dump is missing

#98

Post by belter »

abetusk wrote: post #72

As a reminder, all the SE sites have content under a Creative Commons, By Attribution, Share Alike license, allowing for, among other things, commercial re-use [0] [1]. Yes, it sucks that the SE sites are getting more draconian about allowing access to their content but the SE sites are well insulated against it completely disappearing precisely because they're under a libre/free license. Note that Reddit [2], nor HN…

The change to 4.0 was done without permission according to many in the comunity.

"Stack Exchange doesn't have the right to unilaterally change the license of previously submitted content." - https://meta.stackexchange.com/questions/333089/stack-exchan...

Re: June 2023 Data Dump is missing

#99

Post by croes »

ImPostingOnHN wrote: post #70
ethanbond wrote: post #64

Earlier quoted context omitted.

Really? The takeaway isn’t rather that rent-seeking AI models need to figure out a way to reimburse companies and communities who’ve stored up all this capital? Seems to me SO built and delivered huge, huge amounts of value and it’s now all at risk because multibillion dollar companies are free riding.

SO delivered some value, the users are the ones who delivered huge, huge amounts of value

And companies like OpenAI will take the profit and even kill some of the users jobs.

Re: June 2023 Data Dump is missing

#100

Post by jethro_tell »

tedivm wrote: post #40
the_pwner224 wrote: post #33

Earlier quoted context omitted.

A decentralized system will never work because 99% of users do not care at all; the centralized systems are easier to sign up for and use. It's been demonstrated over and over and over again. Even if the underlying tech is decentralized, the community will settle around one or a few big instances (for example, Gmail and GitHub) which often end up having significant control over the trajectory of the entire ecosystem.…

>If you run your own email server and you get put onto Google's spam list - you're fucked. It's even worse than that- I ran my own email server, and for some reason gmail delayed any emails from their system to outside of their system. That meant that people would send me an email but I wouldn't get it for 20 minutes. These delays don't exist when using big email providers (it stopped being a problem when I switched…

Sounds like you need to allowlist google from your grey list. They have a long retry on their side. Once you tell a server to 'go away, come back later' it is really up to the server to decide when or if to retry. Additionally, If they use multiple sending IPs, you can end up grey listing again and again before they try back with a good ip.

You'd either need to allowlist the big providers sending blocks or just drop grey listing all together.

Post reply on HN