101–110 of 120 posts
I love Heroku and everything they are doing, it's doubtless a push forward for the web as a whole. However, the pricing for hobby sites (including SSL) is crazy from a personal point of view so I'm slowly moving my projects out of it [1][2]. I wish they had some kind of "Hobby Bundle". [1] http://umbrellajs.com/ [2] http://picnicss.com/
Earlier quoted context omitted.
You send a message (for example some JSON) to a Kafka topic. Any number of clients subscribe to that topic with a specific start time-stamp. Pluck a message off the queue, compute with it, send an acknowledgement. Kafka provides strong assurances that all readers get all the messages and report success (it retries otherwise), even if some participants come and go. Very useful if, say, you have some real world event a…
This is a really inaccurate description. Messages aren't indexed by timestamp in any meaningful way; that feature is currently under development. Messages don't need to be acknowledged, it's the client's responsibility to track what messages have been consumed. The server provides some facilities to make that easier, but ultimately clients can request whatever messages they want (repeating, skipping, whatever), as lo…
Earlier quoted context omitted.
because: * you can do ES indexing async * having articles index instantly is not critical (I guess)
Well ES is pretty fast by itself - lots of people use it to store log entries(ELK stack) and every log line triggers an indexing event in ES. Introducing Kafka into the mix just seems like an unnecessary complication.
If you are just dumping into ES, then yes, probably not the best tool (though it wouldn't necessarily hurt) - just use the HTTP API for that. However if you want to build a robust pipeline for multiple services or think you'll be needing to scale the feed into ES, Kafka is useful.
I love Heroku and everything they are doing, it's doubtless a push forward for the web as a whole. However, the pricing for hobby sites (including SSL) is crazy from a personal point of view so I'm slowly moving my projects out of it [1][2]. I wish they had some kind of "Hobby Bundle". [1] http://umbrellajs.com/ [2] http://picnicss.com/
Earlier quoted context omitted.
Check out Red Hat's Openshift Online [1] if you haven't already. They offer 3 Gears for free, each with 512 MB Ram, 1 GB Disk space (install e.g Postgres into one gear and you have a DB with 1 GB) and you can use Lets Encrypt with their Bronze plan (which is free if you only use the 3 free gears). Depending on what your hobby sites do this could be enough. [1] https://www.openshift.com/pricing/
If you are OK with running a >3 year old version of Postgres, that is. Other than that, openshift is nice, though, I agree.
Earlier quoted context omitted.
You should read this: https://engineering.linkedin.com/distributed-systems/log-wha... - it's long, but it's one of the most impactful essays I've read on software engineering in years.
I'm not able to find the one that made the light bulb go on in my head, but Martin Kleppman gives some good conf talks around this topic. This one looks promising - https://www.youtube.com/watch?v=GfJZ7duV_MM
Earlier quoted context omitted.
Sounds like a mailing list.
Yes, but for applications ; instead of doing a remote call to your other system to create an order or send an email, you just stick the data in a queue, and the other system does it when it feels like it
What kind of companies or startups usually use this service?
Lots of places also use it just as a message queue, some places for example write time series metrics to Kafka for monitoring.
Hey, what is Kafka?
"It's a distributed logging system, not a message queue"
Ok, what's the use case?
describes a case when its used as a message queue
Earlier quoted context omitted.
Hm but why would you not send it directly to ElasticSearch?
Kafka shines when you have multiple services that have data to publish and multiple services that need to read that data stream. If you have three services and they write to ES, publish metrics to some other store, and log events to the db, you could instead write that all to Kafka, and individual consumers can use the data (for instance, to put into ES). On the origin-service side, it has one integration point; it d…