Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They don't even know the fundamentals

blog.royalsloth.eu

Post reply on HN
Print view View original

161–170 of 323 posts

Re: They don't even know the fundamentals

#161

Post by jhgb »

ascar wrote: post #137
jhgb wrote: post #98

Earlier quoted context omitted.

You mean a naive one, that is, a useless one...or a practical one? Such as, at least something like LLRB?

A binary tree is just a tree data structure where each node has at most two children. And that's dead simple and every programmer should be able to implement that when aksed. Note that it doesn't even have the search (i.e. sorted) requirement and much less the self-balancing one. You can work your way towards that with the right questions and that would be an acceptable way to structure an interview. Start with somet…

It is dead simple. What you're describing (two children, no constraints) is basically a cons cell. Yet I suspect that most people don't use cons cells as binary trees. Which is probably why I didn't understand that apparently rather nebulous topic as being this generic.

Re: They don't even know the fundamentals

#162

Post by rockbruno »

Oh for fucks sake, every month this topic comes back. It honestly doesn't matter if the fundamentals are useful or not. What gets me is that programmers want to be hired by top-tier companies and earn massive salaries without putting a single drop of effort into actually acquiring the knowledge that justifies that position.

Can we just collectively swallow our privilege and stop whining that an employer is asking for proof that you've actually put the effort to be an engineer before they dump a river of money onto you? Look at how lucky you're before whining that the world is unfair, and shut the hell up about fundamentals.

The interviews surely aren't perfect, but the constant whining is ridiculous. Why not just study it and be done with it already?

Re: They don't even know the fundamentals

#163

Post by qwerty456127 »

To me non-ACID means you can never use any logical reasoning to have an idea of what there is recorded in your database at any given moment. ACID means you execute a piece of code in a transaction and if something goes wrong it fails altogether returning you to the pre-transaction state. Am I wrong?

Re: They don't even know the fundamentals

#164

Post by markus_zhang »

EdwardDiego wrote: post #42
drclau wrote: post #36

Earlier quoted context omitted.

> Don’t ask questions that can be answered with a web search. These days, that’s equivalent to saying “don’t ask questions about what the day to day job requires one to know”. Sites like stackoverflow contain answers to problems that people encounter in their day to day jobs, and if you eliminate those as potential questions, then what should you ask? And what is the value in asking some obscure question, other than…

Here's a question you can't properly answer with a web search for a data engineering role "I'm emitting 10TB of data a day with this schema {X} - it's produced in Avro. Currently we upload it into S3. But, instead, I want to stream into Apache Druid, but also want to make it available for ad-hoc analysis by BAs. What approach would you take?" Can't exactly google an answer to that.

I think it's a bit too specific. But for sure someone can relate to similar experience. Something like "For sure I haven't dealt with Druid specifically but in my job I tackled with a similar problem for a different set of tools...".

BTW from my experience we are streaming a lot of data (similar to 10TB but definitely more) produced in JSON then converted to Avro using Spark and finally loaded into Vertica and the analysis pipeline. I have never used Druid but from Googling it is a column based data store which I guess is not too different from Vertica. Is there any specific difficulty you think would be encountered with Druid? Because the wording seems to indicate that Druid is not particularly good with ad-hoc analysis.

Re: They don't even know the fundamentals

#165

Post by lytefm »

gehatare wrote: post #92
pkilgore wrote: post #46

I mean sure, yes, there's also programmers out there who've been working for years and look at me like I'm speaking in tongues when I talk about closure. The author's casual arrogance about "pixel pushers" only caring about "hotdog stand colors" is just as damaging to the profession -- as it infects the hardest part of getting things right most of the time: teamwork.

I think you misunderstood his point, I think what he's saying is that even good back end devs are horrible at designing UI and you need different people with different specialisations on a team.

Yes the author was criticizing the kind of "I have deep CS knowledge, you are just pushing pixels" arrogance.

Re: They don't even know the fundamentals

#166

Post by throwaway9870 »

Koshkin wrote: post #111
throwaway9870 wrote: post #77

Earlier quoted context omitted.

My take is that too many people are telling others how to interview. It is their money, their company, their job, their life. Let them do it the way they want as long as they follow the legal requirements. If you think you have a better way, tell us about it without passing judgement on others. Tell us why it works, how great it is, give us some qualitative and quantitative data. Most of all, enjoy your competitive a…

While I don’t disagree, the biggest problem I think is that some or even many interviewers just don’t care or not motivated enough to care whether the candidate can indeed be a great help to the company or the project and approach their interviews in a shallow and formal way.

Then that is a great job for other unmotived people. For motivated people, the poor interview is a great signal to look for a different company.

Re: They don't even know the fundamentals

#167

Post by gpderetta »

siva7 wrote: post #157
ivanhoe wrote: post #43

> It turns out my day to day work doesn’t require deep knowledge about database internals and I can mostly treat them as a black box with an API 2. Because of such attitude of the previous dev team my client ended up with DB integrity ruined. Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening be…

I don't think that transactions count as deep database knowledge. Those are the absolute fundamentals that most developers need on a daily basis and not some dusty computer science stuff. If you have never heard of them it is fair to say that you should take a step back and learn the basics of your profession before being allowed to touch a production database. But the thing is of course - you don't know what you don…

Great point:

> you don't know what you don't know

aka knowledge is what you have after you forgot everything you learned.

Re: They don't even know the fundamentals

#169

Post by markus_zhang »

EdwardDiego wrote: post #65
jstx1 wrote: post #50

Earlier quoted context omitted.

Your question turns into a question of "have you worked with Avro, S3, Apache Druid..." which will filter out competent people that don't have experience with the specific list of tools.

No it doesn't. The keywords there were "streaming" and "also ad-hoc analysis". Answering those questions requires no knowledge of Avro or Druid, but rather, how to build streaming pipelines, and also make the data available for a query engine. But that said, if you're in data engineering and are unaware of what Avro is, you're probably not at the level of knowledge I'd expect, and I'm happy to filter you out. Likewis…

But how can I be familiar with S3 without any data engineering work? Or maybe I should put it like this, what exactly does "familiar with S3" mean? Does it mean that I used it to store something (I mean even a non-programmer can do that), or that I should be able to manipulate (i/o) the store using Python with Boto (Again this can be done with a tutorial), or something else?

I believe that "something else" usually indicates some previous data engineering work though, so it's an egg and chicken problem.

Re: They don't even know the fundamentals

#170

Post by EdwardDiego »

hannofcart wrote: post #67
ivanhoe wrote: post #43

> It turns out my day to day work doesn’t require deep knowledge about database internals and I can mostly treat them as a black box with an API 2. Because of such attitude of the previous dev team my client ended up with DB integrity ruined. Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening be…

I think it is possible that an engineer knows about the need to use transactions while doing certain kinds of DB updates while not knowing the exact definition of the ACID acronym. I suspect the author of this article would have no objections to probing if a candidate understands transactions and indices.

Exactly, I've spent years optimizing RDBMS and columnar databases and I couldn't at this moment fully explain the definition of ACID to you without having a wee google refresher.

I mean, I learned ACID's definition, nodded because it made sense, realised that MySQL's default (at the time) MyISAM storage engine wasn't ACID, it flat out failed at C, so encouraged my LAMP stack using friends to switch to InnoDB, then switched to Postgres myself.

Likewise, I use a lot of distributed systems, but I don't think in terms of the CAP theorem on the regular.

Rather I just think about all the fun ways each distributed system can die in, and while ZK is CP (although I'm dubious about the P), and Kafka is CA, the failure mode is roughly the same - you lose enough cluster members and shit breaks.

I know that my k-safety (k being how many nodes I can lose and still have a system) for ZK is N - (N +1)/2 (assuming truncating int division), and as for Kafka, well... it really depends on the failure.

But the usual conservative design for a Kafka cluster is n * 3 replicas on a 3 AZ stretch cluster with min.insync.replicas set to num.replicas - 1,leading to a k safety of 1.

I'm a fan of the 2.5 stretch cluster, if I have to do a stretch cluster, I generally prefer separate clusters replicating.

So for a 2.5, you'd go 2 AZs with M brokers and N ZK nodes, where M % 2 == 0 && N % 2 == 0, with one ZK in the third AZ as the tiebreaker.

You can now lose an entire AZ, and still have a quorum, while paying 33% less inter-AZ traffic from your Kafka clients.

Basically, theory underpins all my work, but the theory and the reality are quite different things.

Post reply on HN