Live data from Hacker News

APIs for content sites must be free

somebits.com

151–160 of 300 posts

Re: APIs for content sites must be free

#151
post #13

I like vehement arguments even when I disagree with them. Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to download the posts created by the users? How about a comment section on my blog? How about a community Discord server? Why not Facebook posts? Or Instagram stories? I often get the feeling that in these circumstances people derive first principles from specific d…

>if I were to run a web forum then I must provide an API or data dump to download the posts created by the users

It's implicitly provided by hosting the forum, blog, etc. at a publicly accessible URL. HTTP is an API, HTML is a data interchange format. There's this weird idea (which I've seen parroted both in this article, and in u/spez's discourse on the topic) that "scraping" is somehow evil and bad, when it's actually an intentional feature of the web. Scraping is quite literally how Google built its search empire. It seems to me, now that the major players are sufficiently centralized and entrenched, they want to turn scraping into a boogeyman so nobody can follow in their wake.

The advantage of providing a "real" API is to (a) limit scope narrower than an account/session, and (b) eliminate the overhead implicit in laying out the content for human eyeballs; reducing processing costs for both the server and client.

I find it somewhat hypocritical for Reddit et al. to provide the former for free, and charge for the latter, when the latter is explicitly designed to optimize their costs.

Re: APIs for content sites must be free

#152
post #59

Earlier quoted context omitted.

I blocked reddit in my hosts file and have found my news through other means. Here and fark.com lol

Same on my local PiHole =D It's worked the couple times I instinctively sought reddit advice.

Same for me on NextDNS

Re: APIs for content sites must be free

#153
post #53

I've found myself resisting posting comments on Reddit recently - things like answers to simple questions people have about SQLite - partly because I don't want to cross the picket line but also because I realize that I was relying on a very light social contract that was in place there: I would share my knowledge for free, in exchange for which I knew that I was contributing to a larger dataset that myself and other…

Really? The average discussion on Reddit/StackOverflow/etc is full of wrong answers and it is hit or miss if the "accepted" answer is correct, never mind any of the other ones. These sites are a great model for convincing bullshit but if you wanted something worth training an A.I. you'd need to filter out 90% of the content.

My experience is that it is a good source for leads on things to do but not for convincing facts. And an LLM is good for that too. I still end up needing both.

An example of something is someone posting that their AKG headphones will only charge on the USB-C plug if the other end is USB-A. I couldn't get mine to charge, and that was the answer. That answer existed only on Reddit, until recently when someone had that same problem on another forum.

Re: APIs for content sites must be free

#154
post #129

Earlier quoted context omitted.

> People understand that a T-Mobile customer can send a text to an AT&T customer. They understand that a Samsung owner can call an iPhone owner. That was a hard-won victory: https://en.m.wikipedia.org/wiki/Kingsbury_Commitment A hundred years later, we've got the same story, except today's Big Tech had wised up.

The EU can do better than spamming us with cookie popups. They'd like to be relevant, why don't they try passing legislation for this?

They do try: https://news.ycombinator.com/item?id=36324118

But keep in mind that EU legislature (and "government", such as it is) is as dysfunctional as the US one, just in different ways, don't expect much enlightened lawmaking there either.

Re: APIs for content sites must be free

#155

Is there a reason web scrapers aren't used for public sites with content? It seems like this would avoid the pitfalls of APIs and changing terms and prices?

They were, in a much less constrained way, until companies running platforms started suing people for scraping. And investing heavily in making it difficult technically. The difference is advertising. The vast majority of people do not want to be advertised to, given the choice, nor do they want to have their contract with a platform downgraded (by having to use the UI designed by the company). This situation is some of the users taking a stand about it.

Re: APIs for content sites must be free

#156
post #147

Earlier quoted context omitted.

The funny thing to me is this sort of expectation that a site/app that accepts user contributed content must provide access to it other than the site/app it was submitted.

I assume that most sites won't do that - so I don't invest much of my time and effort in most sites. Reddit had 17 years of track record of providing access, which had eventually bought my trust. They're setting that trust on fire right now.

I stipulate that a 30 day notice for a major change from something that has been available for 17 years is pretty egregious. However, I'm still of the mind that when you build a company/product that 100% depends on a 3rd party existing or allowing one to play in their playground, these are the exact situations that can arise. You have to know that at some point, if the dependency is no longer available, neither is your product regardless of the reason for the lack of availability. That's just how it goes. Yes, it sucks to have the product of your blood, sweat, tears get invalidated, but that is where you chose to put that effort while knowing the caveats

Re: APIs for content sites must be free

#157
post #86

All of your labor should be free also. What do you mean you want a six figure salary? How incredibly greedy of you. It's fair to pay you only what you need to survive.

> All of your labor should be free also.

Yes, it is, and that's ultimately the contention here. Reddit has taken many millions of man hours of free labor. Now it wants to charge for the fruits of that free labor.

Legally, they are within their rights to do so. Socially, however, it's thought to be uncouth to exploit workers like that.

Re: APIs for content sites must be free

#158
post #13

I like vehement arguments even when I disagree with them. Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to download the posts created by the users? How about a comment section on my blog? How about a community Discord server? Why not Facebook posts? Or Instagram stories? I often get the feeling that in these circumstances people derive first principles from specific d…

> I often get the feeling that in these circumstances people derive first principles from specific desired outcomes.

That's a pretty normal way derive principles, isn't it? You start with an observation, and then you recursively ask "why?" It's possible that your answers are wrong, or even that your observation itself is flawed, but the concept of deriving principles from outcomes isn't really flawed. Where else are they going to come from? Should people just pick principles at random and hope they're not terrible?

Re: APIs for content sites must be free

#159
post #114
post #87

Earlier quoted context omitted.

Uhh... I really don't think you do know what a social contract is. A social contract is a temporary state of affairs, where both sides are benefitting from being able to implicitly hold the other side to it; where either side can freely breach it, but where this dissolves the social contract that held until that moment; and where dissolving the social contract will (obviously) cause the other side to rescind all bene…

> A ceasefire during a battle is an example of a social contract. This is absolutely false. A ceasefire is not a social contract. It's just bizarre to even claim this, as ceasefires are most often enacted in the context of ceasefire.. uh, treaties . > Not eating your coworkers' food in the office fridge is another example of a social contract. This is a better example (though still a very poor one, as we have laws ag…

> as ceasefires are most often enacted in the context of ceasefire.. uh, treaties.

Also known as "armistice treaties." (Where "armistice" is preferred, because otherwise "ceasefire" would have two very different meanings.)

Yes, an armistice usually causes a ceasing... of fire... at the site of any battles that happen to be going on at the time. But you wouldn't usually call what happens on the front lines as a result "a ceasefire." You'd call it "the battle being quit by both sides." During an armistice, both sides usually pack up and go back to their respective bases to be held for redeployment.

"A ceasefire", when the term actually comes up in practice in relation to "phenomena you can experience during a battle", does not usually refer to the ceasing-of-fire that exists as part of the enactment of an armistice. Rather, a ceasefire is used mostly to refer to a temporary, tactical Nash-equilibrium state that lasts a few hours or days; is local to a particular battleground; and is enacted bottom-up by the boots on the ground just not shooting anybody for a while.

(A ceasefire is usually considered an example of a principal-agent problem — during a battle, the known Commander's Intent is to drive your opponent to surrender. A ceasefire does not accomplish this; in fact, it gives both sides time to replenish materiel, and thereby "gives ground" on a war of attrition. But a ceasefire is a state that's better for everyone who's actually there at the site of the battle — so it can be held to by social contract of the people who are actually there.)

Mind you, the phenomena of "a ceasefire" doesn't exist much any more in its original usage (land wars), as land wars have evolved past the point where ceasefires are feasible, due to the introduction of over-the-top raids, guerilla warfare, precision drone strikes, and other approaches designed to punish stillness and allow top-down enforcement of bottom-level participation. Ceasefires still come up pretty often in naval combat, though! (And they can also still come up in land wars in edge-case situations when a particular land-war battle has the properties of a naval battle — e.g. if two strike forces of armored vehicles meet.)

---

I should mention, as well, that there can be ceasing-of-fire during an armistice, where both sides continue to just sit there guarding the lines they had established for the whole armistice period. This is the case mostly for wars of conquest being fought to claim scraps of land inch by inch. (There's a well-known one of those going on right now, in fact.)

During an armistice in this kind of war, the zone between the previous front lines of each side will stay watched, to make sure neither side "slips" the lines forward a bit in the other's absence. This creates a No Man's Land [or eventually a Demilitarized Zone], where both sides agree that no one from either side may enter as long as the armistice holds. (A good example of such a No Man's Land / DZ exists between North and South Korea, which have maintained such an armistice for decades now.)

The soldiers posted to guard a No Man's Land / DZ aren't usually the same soldiers used for combat — so for the soldiers who were on the front, they do still experience the armistice as "and then we were picked up and taken back home." The ceasing-of-fire is just one step in that higher-level "clocking out" process.

(The only time you'll have the same soldiers sitting there during an armistice that were originally delivered there to fight the battle, is if the armistice period isn't expected to last very long. And even then, only if both sides are willing to "tip their hand" to reveal their lack-of-confidence in how long the armistice will last.)

Re: APIs for content sites must be free

#160
post #68
post #66

Earlier quoted context omitted.

You're thinking of a legal contract. A social contract is what two sides do in practice in order to not piss the other off.

I know exactly what a social contract is, and it's always superseded by a legal contract (which the ToS sort-of is, though it would have to go through the courts if challenged). Given that the ToS explicitly says that Reddit et al. can do whatever they want with your data, the idea that there's a social contract there is wishful thinking.

You are thinking of a social agreement often called a “handshake agreement” or “verbal agreement”. Yes a verbal or social agreement at law is often superceded by a written agreement mainly because there’s a specific common clause in written agreements that say exactly this.

The above legal concept is distinct from the term “social contract” that is a sociological term and has a wider definition that a mere legal meaning. Most of our laws are intended to capture concepts that have wider social and cultural relevance to the society to which they apply.

“Marriage” by example is a term with both a distinct legal meaning in many legal jurisdictions as well as a wider sociological conception that can vary from culture to culture.

Post reply on HN