Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku Kafka

heroku.com

Post reply on HN
Print view View original

81–90 of 120 posts

Re: Heroku Kafka

#81

Post by colinbartlett »

franciscop wrote: post #40

I love Heroku and everything they are doing, it's doubtless a push forward for the web as a whole. However, the pricing for hobby sites (including SSL) is crazy from a personal point of view so I'm slowly moving my projects out of it [1][2]. I wish they had some kind of "Hobby Bundle". [1] http://umbrellajs.com/ [2] http://picnicss.com/

I am a longtime Heroku customer on an Enterprise service plan at the moment. I brought this up to my sales rep last time he checked in.

I explained how we moved a bunch of smaller sites to S3, reluctantly because I really like having a unified platform for all our sites. But even though (or perhaps precisely because) we are spending thousands of dollars a month with Heroku, I find the $20/month SSL charge insulting. SSL is not an option anymore.

The good news is, the sales rep said this has come up a lot, they hear us, and to "stay tuned".

Re: Heroku Kafka

#82

Post by colinbartlett »

nickfrostatx wrote: post #68
mintplant wrote: post #66

Earlier quoted context omitted.

It looks like those are static sites. Why host them on Heroku? You could stick them on GitHub Pages [1] for free. [1] https://pages.github.com/

The S3 Free Tier can be a good alternative as well. https://aws.amazon.com/s3/

Even off the free tier, S3 + Cloudfront is now my static host of choice. Incredibly cheap for my low traffic side projects and free (and instant) SSL certificate setup.

Re: Heroku Kafka

#83

Post by okaram »

ashitlerferad wrote: post #52
superuser2 wrote: post #43

Earlier quoted context omitted.

You send a message (for example some JSON) to a Kafka topic. Any number of clients subscribe to that topic with a specific start time-stamp. Pluck a message off the queue, compute with it, send an acknowledgement. Kafka provides strong assurances that all readers get all the messages and report success (it retries otherwise), even if some participants come and go. Very useful if, say, you have some real world event a…

Sounds like a mailing list.

Yes, but for applications ; instead of doing a remote call to your other system to create an order or send an email, you just stick the data in a queue, and the other system does it when it feels like it

Re: Heroku Kafka

#84

Post by elcct »

kasey_junk wrote: post #78
elcct wrote: post #59

My impression of Kafka was that this thing is bloated. How it compares to something like NSQ?

Its a completely different use case. Many times people call Kafka a "message queue" but its not. It's a distributed log service. Its possible to build a message queue on top of a distributed log service but there are reasons not to. Its better to think of Kafka as a database for events, not as a transport mechanism for those events. As for being bloated, Kafka lives in a very empty space, that is it supports fully or…

Thanks for explanation. I didn't know those things.

Re: Heroku Kafka

#85

Post by mtw »

franciscop wrote: post #40

I love Heroku and everything they are doing, it's doubtless a push forward for the web as a whole. However, the pricing for hobby sites (including SSL) is crazy from a personal point of view so I'm slowly moving my projects out of it [1][2]. I wish they had some kind of "Hobby Bundle". [1] http://umbrellajs.com/ [2] http://picnicss.com/

I host most of my sites on Github pages (Free) and Ruby / Go / Elixir / nodejs projects on DigitalOcean. I also host various logging/analytics/mailing services on DigitalOcean without paying saas services. Total is very reasonable

Re: Heroku Kafka

#86

Post by sixwing »

ktamura wrote: post #64
mbseid wrote: post #37

As a former user of Kafka, this is awesome and it would have been a huge help for our company if this was available then. I'm glad to hear that a company is offering Kafka as opposed to other propriety versions(AWS Kinesis etc). One thing is odd though, there is no mention of disk space at all and only a configuration of retention time. One of Kafka's best features is the use of disk to store large amounts of message…

Don't forget that Heroku is the original multi-tenant shop. I wouldnt be surprised if a single Kafka instance stores multiple customers's messages and elastically scale as more customers/data is added.

I'm Rand Fitzpatrick, and this is one of the products I work on at Heroku. None of our current Kafka offerings are multi-tenant.

Re: Heroku Kafka

#87

Post by evgenyp »

balamaci wrote: post #60
yolesaber wrote: post #26

Earlier quoted context omitted.

Let's say you have a CMS which pushes content to your site. You also want to make the whole site searchable, so you index your content into (e.g) Elasticsearch. Kafka is great for this because you can put the content onto Kafka's message queue and then have a service reading from it which then put's it into Elasticsearch. It scales well, too. So let's say your site takes off and you have hundreds of articles publishe…

Hm but why would you not send it directly to ElasticSearch?

One key ability is to batch updates.

We just implemented Kinesis (AWS service similar to Kafka) to reduce load on our Elasticsearch database (~50GB) when running hundreds of individual jobs.

Individual tasks (implemented in Celery, actually running off Redis) push to a Kinesis stream which is then consumed in batches by a very simple processor.

Re: Heroku Kafka

#89

Post by redtuesday »

redtuesday wrote: post #62
franciscop wrote: post #40

I love Heroku and everything they are doing, it's doubtless a push forward for the web as a whole. However, the pricing for hobby sites (including SSL) is crazy from a personal point of view so I'm slowly moving my projects out of it [1][2]. I wish they had some kind of "Hobby Bundle". [1] http://umbrellajs.com/ [2] http://picnicss.com/

Check out Red Hat's Openshift Online [1] if you haven't already. They offer 3 Gears for free, each with 512 MB Ram, 1 GB Disk space (install e.g Postgres into one gear and you have a DB with 1 GB) and you can use Lets Encrypt with their Bronze plan (which is free if you only use the 3 free gears). Depending on what your hobby sites do this could be enough. [1] https://www.openshift.com/pricing/

btw: it seems you still can't automate the cert renewal for let's encrypt on openshift online.[1]

[1] https://openshift.uservoice.com/forums/258655-ideas/suggesti...

Post reply on HN