Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-Host All the Things?

tedium.co

Post reply on HN
Print view View original

51–60 of 295 posts

Re: Self-Host All the Things?

#51

Post by samstave »

MuffinFlavored wrote: post #3

> Email newsletter tools: Old or new, your pick Am I wrong to think that most businesses/people pay for Mailchimp because getting your e-mail actually delivered into the inboxes of your target audience/customers is non-trivial? aka, you're going to end up in "spam" otherwise? I find it hard to believe that you can "free-ly" send e-mail to, say, 100,000 e-mails and actually have it get delivered at a high rate? I woul…

The biggest aspect that used to be used in spam detection (from an OSI, not a content reading perspective) was source IP blocks.

Many people dont realize that spam was the original source for social networking...

I cant type up all the history I know quickly, but Friendster (who 'invented the social graph', HI5, Tagged, MySpace, were all started as an overlay to email harvesting mechanisms to --> spam....

They needed to create high value email-lists of valid emails.

Asking for such, was stupid as most people rejected it.

Then, they figured out that adding a service (chat and share with your friends, give us your email and their email so we can connect you by sending them invites etc) was the best social-engineering (the 'hacker' meaning) mechanism was to have people validate their personal email, offer a novel e-'service' to 'connect' with your friends within some context - and have you pre-validate the email list based on your invites and contacts... then parlay MLM structure to create better more validated email lists.

Then you sell the lists on the BM to spammers looking to avoid a high bounce rate based on real emails.

Then they started nefariously stealing your contacts with auto-opt-in agreements and such....

Then as the battle btwn spam and socially-interesting services ramped up the spam companies (such as Postini (which was bought by google) became the spam filters (selling their services to BigCorps) began to realize that filtering on the sending IPs was a good measure for determining spam (along with rate-limiting, and other aspects) - such that spammers were getting blocked based on delivery IP blocks.

This set-off a market incentive for spammers to buy up swaths of IPv4 blocks so they could swap out IPs...

Then there were many ranges, sources, tracrts etc used to determine senders and ID them as spammers etc....

So - the spammers invented VPN/Tunneling delivery routes such they could send to a number of various global relays so that they could send from a central source of machines, but be delivered to the endpoints from a variety of global IP blocks.

There was a market for IPv4 blocks all over the world and spammers were spending big bucks on all aspects, from paying for the IP blocks, relationships with ISP/VPN/etc tech....

All while attempting to provide what was a thin layer of utility service to the user to keep what was effectively continued access to the growing address books of their users and keep them engaged on the platform such that they could keep knowing if existing or new contacts were valid.

There were even back-room deals between spammers/tech/isp etc to allow access.

So, the "social networks" we know know of were birthed literally upon spam.

-

Have you ever wondered why as soon as tiktock came out, all of a sudden a fuck-ton of spam was hitting your gmail inbox (previously postini) Zuck literally stated that the entire revenue model for FB was "senator, we sell ads"

When in an interview with Google, they asked "what kind of company do you think google is "Well, most people think youre a search engine, but youre actually an advertisement correlation engine"

In an interview with Twitter (dont forget about the infamous ATT room 641A?) - what do you think twitter is: "Twitter is a global sentiment monitering engine" (this was ~2006?8? I cant recall)

--

Source: I know these founders and many of the original devops members from the above companies, and other more scary outcomes from the above statements.

And here we are today with the advanced learning all built upon "consumption" ad algos

Re: Self-Host All the Things?

#53

Post by samstave »

Nextgrid wrote: post #12
dijit wrote: post #6

Earlier quoted context omitted.

> I find it hard to believe that you can "free-ly" send e-mail to, say, 100,000 e-mails and actually have it get delivered at a high rate? I would love to learn if I'm wrong though. You can do this, I have done this, but honestly it's annoyingly painful and you're always one bad ad campaign away from being nuked to death by people marking your emails as spam. There's a lot of rules to follow and even when you follow…

To be fair, if people are marking your emails as spam frequently enough to get your IPs/domains blacklisted then it suggests the system is working as designed and you shouldn’t be sending whatever you’re sending to those people.

See my other comment below on how IP blocks for IPv4 went through the roof on price and availabilty...

The global spam market is what caused the hockey-stick rise in IPv4 "shortage"

Re: Self-Host All the Things?

#54

Post by [deleted] »

dijit wrote: post #6
MuffinFlavored wrote: post #3

> Email newsletter tools: Old or new, your pick Am I wrong to think that most businesses/people pay for Mailchimp because getting your e-mail actually delivered into the inboxes of your target audience/customers is non-trivial? aka, you're going to end up in "spam" otherwise? I find it hard to believe that you can "free-ly" send e-mail to, say, 100,000 e-mails and actually have it get delivered at a high rate? I woul…

> I find it hard to believe that you can "free-ly" send e-mail to, say, 100,000 e-mails and actually have it get delivered at a high rate? I would love to learn if I'm wrong though. You can do this, I have done this, but honestly it's annoyingly painful and you're always one bad ad campaign away from being nuked to death by people marking your emails as spam. There's a lot of rules to follow and even when you follow…

[deleted]

Re: Self-Host All the Things?

#55

Post by ghaff »

nkrisc wrote: post #48
dijit wrote: post #15

Earlier quoted context omitted.

Nah, about 2% of my "Thank you for ordering, here is your receipt" mails also get marked as spam. Some number of people just smack the "spam" button for nearly everything that is automated, and those "spam" buttons seem to work on absolute numbers not percentages; so if you have a high number of people in the pool then you will be false flagged eventually. We had a very explicit double opt-in system, made it super ea…

> I think a major issue is that people don't want to even check how to unsubscribe and they see the “mark as spam” button as a “just make this go away” button. You can thank unscrupulous actors for this. I get so much spam I’m not going to try to figure out what is actually spam or not, nor am I going to risk clicking “unsubscribe” links in emails I assume are malicious spam anyway. If it looks automated and I don’t…

Here's the thing. If companies never collected and used email addresses in exchange for providing free webinars, reports, developer seminars, books, reports, etc. they'd do far less of those things because digital marketing would be much more just shouting out into the void with often difficult to measure results. And they'd generally be way out-marketed (and out-sold because marketing brings in leads).

You may be fine with all that but remember that selling pays for engineering salaries.

Re: Self-Host All the Things?

#56

Post by nkrisc »

ghaff wrote: post #55
nkrisc wrote: post #48

Earlier quoted context omitted.

> I think a major issue is that people don't want to even check how to unsubscribe and they see the “mark as spam” button as a “just make this go away” button. You can thank unscrupulous actors for this. I get so much spam I’m not going to try to figure out what is actually spam or not, nor am I going to risk clicking “unsubscribe” links in emails I assume are malicious spam anyway. If it looks automated and I don’t…

Here's the thing. If companies never collected and used email addresses in exchange for providing free webinars, reports, developer seminars, books, reports, etc. they'd do far less of those things because digital marketing would be much more just shouting out into the void with often difficult to measure results. And they'd generally be way out-marketed (and out-sold because marketing brings in leads). You may be fi…

You’re right. And it’s lead us to a place where I mark most email I get as spam.

Re: Self-Host All the Things?

#57

Post by djbusby »

dijit wrote: post #6
MuffinFlavored wrote: post #3

> Email newsletter tools: Old or new, your pick Am I wrong to think that most businesses/people pay for Mailchimp because getting your e-mail actually delivered into the inboxes of your target audience/customers is non-trivial? aka, you're going to end up in "spam" otherwise? I find it hard to believe that you can "free-ly" send e-mail to, say, 100,000 e-mails and actually have it get delivered at a high rate? I woul…

> I find it hard to believe that you can "free-ly" send e-mail to, say, 100,000 e-mails and actually have it get delivered at a high rate? I would love to learn if I'm wrong though. You can do this, I have done this, but honestly it's annoyingly painful and you're always one bad ad campaign away from being nuked to death by people marking your emails as spam. There's a lot of rules to follow and even when you follow…

How does one even know that message are being tagged as spam?

Re: Self-Host All the Things?

#58

Post by navigate8310 »

nitnelave wrote: post #37
justin_oaks wrote: post #19

I thought this article would go into more than a handful of apps. What apps do you think work well for self-hosting, even if it limited to us tech folk? I've self-hosted Grafana and InfluxDB for monitoring and metrics and found them OK to self host. The authentication and TLS setups were the most annoying. I've self hosted a few kinds of wiki software, but I eventually settled on a combination of a single Tiddlywiki…

I got fed up installing OpenLDAP for user management, so I made LLDAP, targeting the Goldilocks zone of the article: simple to setup/manage, but powerful enough for most self-hosting needs.

Can something like this be used to host phone books for IP PBX?

Re: Self-Host All the Things?

#59

Post by dijit »

djbusby wrote: post #57
dijit wrote: post #6

Earlier quoted context omitted.

> I find it hard to believe that you can "free-ly" send e-mail to, say, 100,000 e-mails and actually have it get delivered at a high rate? I would love to learn if I'm wrong though. You can do this, I have done this, but honestly it's annoyingly painful and you're always one bad ad campaign away from being nuked to death by people marking your emails as spam. There's a lot of rules to follow and even when you follow…

How does one even know that message are being tagged as spam?

Depending on the provider you can receive a “bounce” response. Yahoo and Hotmail do this, Google was a little more opaque if memory serves.

Re: Self-Host All the Things?

#60

Post by nottathrowaway3 »

monsieurbanana wrote: post #16
Nextgrid wrote: post #12

Earlier quoted context omitted.

To be fair, if people are marking your emails as spam frequently enough to get your IPs/domains blacklisted then it suggests the system is working as designed and you shouldn’t be sending whatever you’re sending to those people.

Could be, or it could be that those systems are so aggressively tuned that newcomers have no chance to not be labeled spam while established players are whitelisted. (I truly don't know, but I don't think it's as simple as you're saying)

Email delivery is not purely a protection racket.

People use Gmail because they legitimately want to filter out the unsolicited spam, marketing, etc. To an anonymous attacker, there is no cost to send these emails. Middlemen like MailChimp and Sendgrid play the role of converting email from a free, publicly exploitable channel into a paid, KYC one.

Email fbfw is the de facto standard communication channel for almost everything, but by design a single computer can send an unlimited number of emails to other addresses. This maybe was a good enough design originally, but now the role of email has grown so much that, today, it should be a paid KYC channel.

What is the alternative to spam filtering? Everyone maintains their own allowlist of good senders?

Post reply on HN