Live data from Hacker News

Ask HN: What overlooked class of tools should a self-taught programmer look into

news.ycombinator.com

351–360 of 416 posts

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#351
> Googling, I encountered something I hadn't heard of called a message queue.

Former high frequency trader here. Messaging middleware is a god-send for distributed systems. It used to be quite commonplace in the late 1990s/early 2000s for creating trading systems.

By mid-to-late 2000s - and especially post GFC (great financial crisis), volatility lowered, technology improved and middleware systems were never again put in the critical path.

The important measure here is "tick-to-trade" - from the moment a market-data message comes off the wire to the moment you send an action to buy/sell/cancel back on the wire. A middleware system just slows this down considerably. As a result, you want "tick-to-trade" to be in the same process, preferably single-threaded and boost your thread affinity to minimize context switching.

To answer your actual question: I would say learn about the concepts that the big cloud providers are pushing. If you open the AWS app drop down - there are dozens of concepts that have been encapsulated in managed or serverless frameworks. They are all worth learning IMHO - as they define the current and next generation of computing.

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#352
post #351

> Googling, I encountered something I hadn't heard of called a message queue. Former high frequency trader here. Messaging middleware is a god-send for distributed systems. It used to be quite commonplace in the late 1990s/early 2000s for creating trading systems. By mid-to-late 2000s - and especially post GFC (great financial crisis), volatility lowered, technology improved and middleware systems were never again pu…

Message queues are really cool, but to paraphrase an old joke, "Some people see a distributed systems problem and think, "I know, I'll use a message queue." Now they have otw problems.

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#353

Earlier quoted context omitted.

I always had trouble reading makefiles because the control flow is not very linear. At least with shell files basically everything is explicit.

I often find the non linear way of working with Make an advantage. Since it allows you to break a big piece of procedural shell code with lots of control flow (if X is installed don't install again, etc) into small self contained functional pieces with clean input and output boundaries which can be run individually. It also greatly improves code reuse as every target/recipe can be considered a function.

> Since it allows you to break a big piece of procedural shell code with lots of control flow (if X is installed don't install again, etc) into small self contained functional pieces with clean input and output boundaries which can be run individually

At that point, I use a scripting -not shell- language (which is not as implicit)

I don't program in c anymore, so my major workflow is all in one language; I write my server in the same language that do s the compilation, which is in the same language that does utilities like creating network tunnels to my lab nodes

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#354
post #48

Unit testing, mocking, and various other testing techniques. Why? Any project of sufficient complexity is very hard to test. If all you're doing is code -> build -> run to debug your code, you can very easily break something that's not in your immediate attention. The problem is that good unit testing is hard, and time consuming. It can be so time consuming, that unless you can really plan in advance how you test, yo…

You are completely wrong.

Mocking is a huge design smell. The more mocks or integration tests your projects requires to get full coverage the less modular your program is. A program that uses many mocks is a sign of very very poor design. You will find the code more complex to reason about and much harder to reuse code without necessitating a lot of glue code to make things work together. Without proper knowledge you won't even know the program is poorly designed.

I will grant you that 90% of programmers out there don't know how to design programs in a truly modular way, so most engineering projects will require extensive mocking. In fact most engineers can go through their entire career without knowing that they are making their programs more complex and less modular then it needs to be. Following certain design principles I have seen incredibly complex projects require nearly zero mocking (very very rare though).

Mocking indicates a module is dependent on something. Dependency is different from composition.

     Dependencies                                Composition


           C                                        C
 +---------------------+
 |                     |       +----------------+       +-----------------+
 |     A               |       |                |       |                 |
 |                     |       |                |       |                 |
 |        +----------+ |       |                |       |                 |
 |        |          | |    in |                |       |                 |  out
 |        |          | |    -->+       A        +------>+         B       +-->
 |        |    B     | |       |                |       |                 |
 |        |          | |       |                |       |                 |
 |        |          | |       |                |       |                 |
 |        |          | |       |                |       |                 |
 |        +----------+ |       |                |       |                 |
 +---------------------+       +----------------+       +-----------------+
What's going on here? Both examples involve the creation of module C from A and B.

left: 'A' exists as wrapper code around B and is useless on its own. To unit test A you must mock B.

right: every module is reuseable on its own. Nothing needs to be mocked during unit testing. No dependencies.

The only exception to the right example where you MUST mock is a function that does IO. IO functions cannot be unit tested period, they can only be tested with integration tests.

There's a name for the left approach. It's called Object oriented programming using inheritance or composition(the oop version of composition; not functional composition) as a design pattern. (both are bad)

There's also a name for the right approach. It's called functional programming using function composition.

I don't advocate that you strictly follow either style. Just know that when you go left you lose modularity and when you go right you gain it. All functional programming does is force your entire program to be modular down to the smallest primitive unit. Extensive mocking in your program means you went too far to the left.

tangent: Another irony around this world is that a lot of functional programmers (javascript and react developers especially) don't even know about the primary benefit of functional programming. They harp about things like "immutability" or how its more convenient to write a map reduce rather than a for loop without truly ever knowing the real benefits of the style. They're just following the latest buzzword.

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#355

Makefiles. I always dismissed them as a C compiler thing. Something that could never be useful for Python programming. But nowadays every project I create has a Makefile to bind together all task involved on that project. From bootstrapping the dev environment, running checks/test, starting a devserver, building releases and container images. Makefiles are just such a nice place to put scripts for these common tasks…

Any good resources for learning about make files that you can recommend?

"GNU Make Book" by John Graham-Cumming

https://nostarch.com/gnumake

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#356

Read the curriculum of an undergraduate computer science course and read up on the things you haven't heard of. Some courses will even have lecture notes available. E.g. these four pages are the university of Cambridge masters in computer science: https://www.cl.cam.ac.uk/teaching/1819/part1a-75.html https://www.cl.cam.ac.uk/teaching/1819/part1b-75.html https://www.cl.cam.ac.uk/teaching/1819/part2-75.html https://www…

I support this 100%. I worked for years as a self-taught programmer. When I went back for my CS degree, I was shocked at how much I didn't know that I didn't know. Numerous times I'd be sitting in a class and we'd go over a solution to some theoretical problem, and I'd realize that this solved a problem that had taken me days to discover on my own (and this solution was usually better than what I'd come up with). If…

Could you give an example of one of the times something theoretical helped you solve a real world problem?

I've thought about going back for my CS degree a lot but can't really justify the cost and time investment vs self teaching. But it's something that's always been in the back of my mind.

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#357

Makefiles. I always dismissed them as a C compiler thing. Something that could never be useful for Python programming. But nowadays every project I create has a Makefile to bind together all task involved on that project. From bootstrapping the dev environment, running checks/test, starting a devserver, building releases and container images. Makefiles are just such a nice place to put scripts for these common tasks…

self taught dev here too. I have never used make files, but pretty sure I'm using NodeJs in a similar role. I use it to automate all my "scripting", including deploying of my SaaS product to the cloud and running unit tests.

If it sounds interesting, check out https://www.npmjs.com/package/shelljs

PS: I do my primary development on windows, but my production environment is ubuntu. node apps "just work" on both environments. truely cross platform.

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#358

Makefiles. I always dismissed them as a C compiler thing. Something that could never be useful for Python programming. But nowadays every project I create has a Makefile to bind together all task involved on that project. From bootstrapping the dev environment, running checks/test, starting a devserver, building releases and container images. Makefiles are just such a nice place to put scripts for these common tasks…

I also use similar Makefiles in my projects. I use "make release" to generate the docker container.

Re: Ask HN: What overlooked class of tools should a self-taught programmer look into

#359

1. Profiler. There's a standard tool that tells you what part of your code is slow. Over half the time it'll find something dumb and easy to fix instead of whatever you expected. 2. SQL / relational database schemas. Persistence opens up a lot of capabilities. And databases themselves are very well-optimized; if you do any nontrivial data manipulation it's likely that whatever the query planner comes up with will be…

strongly agree on profiler and SQL.

Horror story re SQL: in my SaaS I skipped sql and went with cloud datastore (NoSql) and regret it. basically (to simplify) you can't query your data without doing a full table scan (IE Slow).

Post reply on HN