Understanding Kafka with Factorio • Hacker News Board

Re: Understanding Kafka with Factorio

#71

Post by scotty79 » Sat, Jul 06, 2019, 1:23 PM UTC

ZeroGravitas wrote: post #67

dEnigma wrote: post #24

Earlier quoted context omitted.

Why? Isn't crack famously (or infamously) addictive?

I agree with the (apparently unpopular) notion and I avoid describing things as "x crack" and wince a bit when people use it as it seems a bit tone-deaf and childish, like referring to something you don't like that there's too much of as "programmer AIDS" or a bad design as a "UI Holocaust" or a failed project as an "abortion" or a bad sports defeat as being "raped", all of which make a similar kind of allegorical se…

I think that kind of word use is inevitable and actually a sign of something good. It for me shows that people know about these things but they are percieved to be so unbelievably horrible that they feel unreal and as such can serve as metaphors.

I know these things happened, happen and most likely will happen and such metaphors are unpleasant for people affected, but I think it's natural language evolution.

People still get murdered occasionally and families of murder victims might be uneasy when they hear about someone that "he killed her with this joke!" but it won't prevent people from using that phrase because murders are so rare for most people that the word gradually loses its literal meaning.

Re: Understanding Kafka with Factorio

#72

Post by commandlinefan » Sat, Jul 06, 2019, 3:00 PM UTC

pdpi wrote: post #17

chrischen wrote: post #14

The analogy helped me understand, but in factorio belts are required primarily because everything takes physical space. What are real world use cases for kafka? Is it primarily if your data is too much to simply store in a db?

Push architectures put all of the control on the producer, but become quite painful in the face of an unreliable consumer. Pull architectures put all of the control on the consumer, but become quite painful in the face of an unreliable producer. Message queues allow the producer to think it's participating in a push architecture, while the consumer believes it's a pull-based arch. Because the message queue itself doe…

It’s a good theory, and definitely something you should be aware of but... in my experience if the producer fails, you get nothing (of course), and if the consumer fails, the messages just back up until the queue overflows and of course if the queue itself fails - which can happen due to hardware or network as well as software issues - you have a new point of failure that’s unrelated to the producers or consumers. So the whole thing is only as good as the responsiveness of your monitoring team... which is effectively the case with synchronous message delivery, only with the added complexity of troubleshooting the queue in the middle.

Re: Understanding Kafka with Factorio

#73

Post by pul » Sat, Jul 06, 2019, 4:12 PM UTC

jand wrote: post #70

pul wrote: post #20

I wrote this article a couple of months ago and didn't expect it to appear on my news feed all of the sudden. Thanks for the love everyone. Any questions are welcome of course!

I really enjoyed reading this, thank you very much. Please check the picture with the label "All microservice instances consume all messages". From my understanding it is more of a depiction of a consumer group. As factorio really consumes the messages (they are gone once consumed) you cannot really implement "All microservice instances consume all messages". Maybe this is BS, i have no Kafka expertise.

You're absolutely right. There's no correct way to visualize this in Factorio.

Re: Understanding Kafka with Factorio

#74

Post by twic » Sat, Jul 06, 2019, 5:01 PM UTC

fredley wrote: post #61

Baeocystin wrote: post #33

Earlier quoted context omitted.

Pro-tip to others who think 'Programmer Crack' is hyperbole- it is not. Factorio does have a rather steep learning curve, and the UI is not the most intuitive. But. Once how it works clicks, it clicks, and suddenly morning. And I'm in my 40's, I'd thought my days of unexpected overnights while working on fascinating problems were mostly behind me. Highly recommended, but be aware.

I think the reason for this is that it pushes all the buttons your programmer brain wants pushed: designing building and debugging systems with a strong focus on optimisation. However it doesn't contain meetings, sprints, language oddities, politics or any of the stuff that makes normal programming less fun.

What if there was a package manager for Factorio?

Re: Understanding Kafka with Factorio

#75

Post by DonHopkins » Sat, Jul 06, 2019, 5:52 PM UTC

realityman wrote: post #4

A better way to understand Kafka is to fill a room with rakes, turn of the lights, and walk around.

Doesn't Kafka also support mouse traps, too?

Re: Understanding Kafka with Factorio

#76

Post by DonHopkins » Sat, Jul 06, 2019, 7:36 PM UTC

twic wrote: post #74

fredley wrote: post #61

Earlier quoted context omitted.

I think the reason for this is that it pushes all the buttons your programmer brain wants pushed: designing building and debugging systems with a strong focus on optimisation. However it doesn't contain meetings, sprints, language oddities, politics or any of the stuff that makes normal programming less fun.

What if there was a package manager for Factorio?

Factorio packages are Blueprints and Blueprint Books! And the equivalent of npmjs.com would be https://factorioprints.com .

https://wiki.factorio.com/Blueprint

https://wiki.factorio.com/Blueprint_book

https://factorioprints.com/

https://www.reddit.com/r/factorio/comments/7vnkze/blueprint_...

One of the stages of evolution of a Factorio programmer is learning how to make scalable modular blueprints that you can stamp down one next to another, with their inputs and output routed so they all plug together easily and run in parallel, without having to be manually wired together.

Re: Understanding Kafka with Factorio

#77

Post by DonHopkins » Sat, Jul 06, 2019, 7:44 PM UTC

root_axis wrote: post #6

gnulinux wrote: post #3

For those who don't know factorio is truly a masterpiece of a game. It's described as "programming the game". Strongly recommended to everyone here.

And high quality network multiplayer! "Pair programming" was never this much fun!

Katherine of Sky has some great "pair programming" videos where she teams up with other experienced players. It's amazing to see how diverse different people's styles can be! You really get insight into how their minds works.

I can't remember which ones are multi player, but here are some of her playlists -- awesome stuff:

Factorio Tutorials Playlist

https://www.youtube.com/playlist?list=PL4o6UvJIdPNpm9szThfI-...

Factorio 0.17: Entry Level to Megabase 2

https://www.youtube.com/watch?v=el9Ih1gg3i4&list=PL4o6UvJIdP...

Factorio 0.17: Pyanodon with Aven1017

https://www.youtube.com/playlist?list=PL4o6UvJIdPNpUZ230345G...

Factorio Landfill Avalanche! (MP with Aven!)

https://www.youtube.com/playlist?list=PL4o6UvJIdPNrEWhjZOzfC...

Factorio ASMR

https://www.youtube.com/watch?v=P7dasB1a5J0&list=PL4o6UvJIdP...

>Rabbi Tricky: OMG KoS, your voice, your choice of words, it makes me really relaxed. I saw it mentioned before, but its like Bob Ross. Happy little trees are now in my mind, together with happy little miners and such. Please don't make this a one time video, i really love this. Much love

>mithrane: KOS - You really are the Female Bob Ross of Factorio. Love your videos and love this concept.

Re: Understanding Kafka with Factorio

#78

Post by DonHopkins » Sat, Jul 06, 2019, 10:27 PM UTC

xmonkee wrote: post #51

That's crazy! I installed it yesterday for the first time to try out the steam proton thing and was literally up till 3 am. I didn't realise the programmer crack angle till I read this. I'm screwed.

Oh no, what have I wrought? Another addict. Well hopefully it served as a warning to more people than the number of people it tempted into getting hooked. It’s only because I haven’t set up my PC since I moved that I have had any time in my life not playing Factorio myself. I still have the dreams, though.

Re: Understanding Kafka with Factorio

#79

Post by NullPrefix » Sat, Jul 06, 2019, 10:44 PM UTC

rs23296008n1 wrote: post #64

DonHopkins wrote: post #32

Earlier quoted context omitted.

Dang suggested reposting it since it didn't get much attention the first time.

Wow, I ask a valid question about a old post and I'm punished with multiple downvotes. Lesson learnt.

You were downvoted because you did not RTF FAQ.

Re: Understanding Kafka with Factorio

#80

Post by X6S1x6Okd1st » Sun, Jul 07, 2019, 2:33 PM UTC

ZeroGravitas wrote: post #67

dEnigma wrote: post #24

Earlier quoted context omitted.

Why? Isn't crack famously (or infamously) addictive?

I agree with the (apparently unpopular) notion and I avoid describing things as "x crack" and wince a bit when people use it as it seems a bit tone-deaf and childish, like referring to something you don't like that there's too much of as "programmer AIDS" or a bad design as a "UI Holocaust" or a failed project as an "abortion" or a bad sports defeat as being "raped", all of which make a similar kind of allegorical se…

I was always kind of unsure why it made me feel weird until someone pointed out how weird it would be to say "OMG these fries are like meth". I think for many people there is a racial component to it as well.