They don't even know the fundamentals • Hacker News Board

Re: They don't even know the fundamentals

#191

Post by gameswithgo » Mon, Nov 01, 2021, 1:15 PM UTC

p2t2p wrote: post #100

smallerfish wrote: post #87

Earlier quoted context omitted.

Because sometimes the requirement is that the engineer being hired is able to come up with efficient algorithms. You don't have to be able to create the textbook binary tree algorithms, but you should be able to derive a reasonably performant algorithm with simple data structures if it matters to the position you're being hired for.

> requirement is that the engineer being hired is able to come up with efficient algorithms. Coding binary tree is not the way to discover that. Presenting a task and observing people solving it and explaining computation complexity behind their solution and the thinking behind their solution is how you do that.

If I give a candidate a truly real world difficult problem with an hour or even a day to solve it, some people who are not so smart will get the correct answer by luck, some people who are brilliant will start off with a bad approach and not get there.

If I ask if you can do something basic binary tree things, I will also not know if you are brilliant, but I can at least rule out if you are totally clueless or not.

Re: They don't even know the fundamentals

#192

Post by EdwardDiego » Mon, Nov 01, 2021, 1:17 PM UTC

markus_zhang wrote: post #164

EdwardDiego wrote: post #42

Earlier quoted context omitted.

Here's a question you can't properly answer with a web search for a data engineering role "I'm emitting 10TB of data a day with this schema {X} - it's produced in Avro. Currently we upload it into S3. But, instead, I want to stream into Apache Druid, but also want to make it available for ad-hoc analysis by BAs. What approach would you take?" Can't exactly google an answer to that.

I think it's a bit too specific. But for sure someone can relate to similar experience. Something like "For sure I haven't dealt with Druid specifically but in my job I tackled with a similar problem for a different set of tools...". BTW from my experience we are streaming a lot of data (similar to 10TB but definitely more) produced in JSON then converted to Avro using Spark and finally loaded into Vertica and the an…

> Because the wording seems to indicate that Druid is not particularly good with ad-hoc analysis

Bingo ;)

I was trying to phrase it like I've heard people do so in interviews.

And tbh, a candidate saying "I'm not familiar with Druid, but it sounds like you can't use it do ad-hoc queries with it? Can you tell me more?" would be a very good sign.

Re: They don't even know the fundamentals

#193

Post by kayodelycaon » Mon, Nov 01, 2021, 1:19 PM UTC

smallerfish wrote: post #128

p2t2p wrote: post #100

Earlier quoted context omitted.

> requirement is that the engineer being hired is able to come up with efficient algorithms. Coding binary tree is not the way to discover that. Presenting a task and observing people solving it and explaining computation complexity behind their solution and the thinking behind their solution is how you do that.

It's a fine way of doing it, although certainly it shouldn't be the only algorithms question. It's a simple data structures / algorithms question. Can you think in terms of how data is stored in memory? Do you understand the tradeoffs of storing data in one form or another? Can you work with recursive algorithms? Given a challenge of having to think about data in a form you haven't recently, can you wrap your mind ar…

> Can you think in terms of how data is stored in memory?

If you're using a modern OS, you don't know this. The kernel remaps everything and pretends to give you a linear chunk of memory. You don't know what's in the various caches and what's been paged out of ram completely.

Everything is built on layers of abstraction. C is an abstraction layer that fits poorly into what modern CPUs and operating systems actually do, but everything is programmed to cater to C's assumptions so people don't need to rewrite their programs.

Higher languages don't necessarily use the data structure you think they do and they can switch the implementation details based on the size.

"Simple data structures" are only simple because you pick one specific layer of abstraction to work with.

Re: They don't even know the fundamentals

#194

Post by Clubber » Mon, Nov 01, 2021, 1:19 PM UTC

rockbruno wrote: post #162

Oh for fucks sake, every month this topic comes back. It honestly doesn't matter if the fundamentals are useful or not. What gets me is that programmers want to be hired by top-tier companies and earn massive salaries without putting a single drop of effort into actually acquiring the knowledge that justifies that position. Can we just collectively swallow our privilege and stop whining that an employer is asking for…

>an employer is asking for proof that you've actually put the effort to be an engineer before they dump a river of money onto you?

But that's the rub, knowing the acronym to ACID isn't proof of anything other than you read about it recently. Asking about the ACID test is like asking what Chipper Jones' batting average was in 1994 when interviewing to be a baseball player in 2021.

20+ years ago, I could recite all that shit from memory, but after building countless systems from the ground up which are still in use today, I can only remember reading about it but I can't recite the words of the acronym.

Hiring in IT is fundamentally broken and has been for decades and probably will be for decades. This is an example of it, that is the discussion.

Re: They don't even know the fundamentals

#195

Post by jhgb » Mon, Nov 01, 2021, 1:21 PM UTC

sureglymop wrote: post #185

ascar wrote: post #137

Earlier quoted context omitted.

A binary tree is just a tree data structure where each node has at most two children. And that's dead simple and every programmer should be able to implement that when aksed. Note that it doesn't even have the search (i.e. sorted) requirement and much less the self-balancing one. You can work your way towards that with the right questions and that would be an acceptable way to structure an interview. Start with somet…

To add to that, a binary tree is often not really "implemented" but simply encoded in an array. That is, the children of any given node i are 2i and 2i+1 in the array. So depending on the actual exercise/interview question you might able to just show that, especially of you have to use pseudo code.

Correct me if I'm wrong, but isn't this exactly the case where you don't want an unbalanced tree? A degenerate tree is bad enough; a degenerate tree implemented as an array in this way is catastrophic. So it seems to me that if you're refining the topic further in some sort of talk, the topic of balancing should precede the topic of physically representing the tree as an array.

Re: They don't even know the fundamentals

#196

Post by EdwardDiego » Mon, Nov 01, 2021, 1:24 PM UTC

markus_zhang wrote: post #169

EdwardDiego wrote: post #65

Earlier quoted context omitted.

No it doesn't. The keywords there were "streaming" and "also ad-hoc analysis". Answering those questions requires no knowledge of Avro or Druid, but rather, how to build streaming pipelines, and also make the data available for a query engine. But that said, if you're in data engineering and are unaware of what Avro is, you're probably not at the level of knowledge I'd expect, and I'm happy to filter you out. Likewis…

But how can I be familiar with S3 without any data engineering work? Or maybe I should put it like this, what exactly does "familiar with S3" mean? Does it mean that I used it to store something (I mean even a non-programmer can do that), or that I should be able to manipulate (i/o) the store using Python with Boto (Again this can be done with a tutorial), or something else? I believe that "something else" usually in…

> But how can I be familiar with S3 without any data engineering work?

... you don't need to be in data engineering to use S3.

And can you be familiar with it? Read the docs, write some code.

Re: They don't even know the fundamentals

#197

Post by Clubber » Mon, Nov 01, 2021, 1:26 PM UTC

littlecranky67 wrote: post #174

eussg wrote: post #8

I like the Zoho strategy of just hiring high school kids and training them - https://m.timesofindia.com/india/this-man-is-also-driving-on...

I've worked with several companies that tried that, in the end once the juniors became experienced and skilled enough, they took offers from other companies with a much higher salary. The issue is that training/skilling juniors or fresh university graduates costs you money and slows you down (as you need to assign senior engineers/mentors to assist them). It is not unlikely that a newly joined junior employees only c…

>took offers from other companies with a much higher salary.

Seems like paying at least market rates once they are trained up would solve that problem. Once someone develops a skill, a company can't expect to keep them at intern or junior salary levels.

Re: They don't even know the fundamentals

#198

Post by kayodelycaon » Mon, Nov 01, 2021, 1:26 PM UTC

Koshkin wrote: post #95

Knowing fundamentals is important. It is simply part of the professional culture, or part of what they sometimes call a degree of professional “maturity.” A programmer who is not aware of the fundamentals will always find a way to write bad code.

It is... but the book knowledge version of fundamentals and their actual implementation are completely different when you start asking questions.

Even more so when those "fundamentals" are implementation details you don't think about much in your day to day job. Actively recalling knowledge that has become part of your intuition and understanding of the world is hard.

I know enough about many things to do basic implementations safely; everything else is paged out to google. The finer details of database transations? Usually not my job, but I used to know more about them and could relearn that information quickly if needed.

Re: They don't even know the fundamentals

#199

Post by helge9210 » Mon, Nov 01, 2021, 1:28 PM UTC

ayende wrote: post #85

helge9210 wrote: post #58

Earlier quoted context omitted.

> you will still end up somewhere in the CAP triangle Drop P and you'll end up with both consistency and availability, right? Right? ("This is a trade-off we are ready to accept")

How do you drop P?

You just assume network partition is never happening. If calls are synchronous, assume they always reach other side and finish. If async, also assume successful outcome immediately after initiating the call.

This "almost always" works.

Re: They don't even know the fundamentals

#200

Post by littlecranky67 » Mon, Nov 01, 2021, 1:29 PM UTC

Clubber wrote: post #197

littlecranky67 wrote: post #174

Earlier quoted context omitted.

I've worked with several companies that tried that, in the end once the juniors became experienced and skilled enough, they took offers from other companies with a much higher salary. The issue is that training/skilling juniors or fresh university graduates costs you money and slows you down (as you need to assign senior engineers/mentors to assist them). It is not unlikely that a newly joined junior employees only c…

>took offers from other companies with a much higher salary. Seems like paying at least market rates once they are trained up would solve that problem. Once someone develops a skill, a company can't expect to keep them at intern or junior salary levels.

That was exactly my point. But why hire & train a junior dev when there is no company loyalty and the money spent on training is basically lost. Instead management focus on directly hiring seniors. Of course that doesn't work as a whole, because if no one hires juniors there will be a point where there will be no seniors anymore. I also feel little to no pity when those companies complain, that they can't find seniors.