They need UX/UI people
Etsy CTO: We Need Software Engineers, Not Developers
51–60 of 160 posts
Re: Etsy CTO: We Need Software Engineers, Not Developers
#52I am sympathetic to the distinction. I had a bunch of applicants for a position claiming to be "Distributed Systems Engineers" mostly because they'd stood-up, maintained, and or used a Hadoop or Spark cluster in their current or prior job. But to me that was more of an advanced (one hopes) end-user. Someone who could take a bunch of large, mostly-complete logical components that somebody else engineered and then use…
> I was looking for someone who could build something as good or better than HDFS from nothing if they had to. Well, for 90 percent of job offers, what a company claims to need is not what it actually needs (i.e. the typical "10 years experience with 5 year old technology" bullshit). If your company is not a very unique snowflake or in an academic setting, believing that it makes zero economic sense to completely rei…
Your characterization of the typical job marketplace in tech jives pretty closely with my experience. I've often said, in public speaking forums no less, that to me, "The overwhelming majority of the 'Big Data' marketplace seems predicated on selling to the hubris that software engineers want to believe they have problems bigger than they actually are." Despite being a denizen of the whole "NoSQL" thing personally.
That said, in my particular case it had a lot less to do with trying to necessarily rebuild HDFS from nothing, and more to do with a mindset of rigor and principles necessary to do so. Because being able to work all the way through that problem domain in both broad strokes and in meticulous detail would hopefully lend itself toward also considering ways to validate and attest the correctness of not only things like HDFS (rather than treating it like a solved problem ready built for use), but also applying that same level of rigor and principle to the stuff we actually do have to build from scratch.
Though to your point... a non-trivial amount of this concern and necessity is borne out of the market and regulatory regimes this stuff has to service and abide. That fact that it's not necessary for huge swaths of the marketplace is evidenced by the fact that things like property-based testing, mutation testing, chaos testing, and formal verification are fringe skills (at best) out there... yet the tech world continues to turn out totally awesome cool new stuff with none of that overhead all that time that still transforms all manner of life.
I actually think that "developers" and "engineers" are mostly pretty transparent about what sort they are. Or at the very least it's trivially easy to assess within just a few minutes with the right questions and conversation space. The harder part is getting non-technical people to understand that there's a distinction and that technical people aren't all just a fungible commodity. The weirdest part is that they get that on some level, especially when suddenly they're hit by a bus-factor problem, but that realization hasn't seemed to make a big impact in business/hiring process, practices, etc.
Re: Etsy CTO: We Need Software Engineers, Not Developers
#53In the paragraph on "Engineering vs. Development" I read what engineering is "Engineering, as a discipline and as an activity, is multi-disciplinary. It’s just messy. And that’s actually the best part of engineering. It’s not about everyone knowing everything. It’s about paying attention to the shared, mutual understanding. " But the author doesn't contrast that with what development is. I used the words interchangea…
Moreover, there seems to be a contradiction there: they are saying engineering is not inherently domain-specific, but to seek out domain experts when the need arises. Okay? That network engineer isn't REALLY an engineer until he wants to learn about databases? Would a "network engineer" be a non-sequitur in itself?
"Software engineering," as the term is usually used, is really a joke.
In my mind, engineering is about rigor: process, measurability and discipline. The hallmark of a well-engineered system, in my opinion, is reliability. Software is anything but reliable.
Too much of software development is throwing things at the wall and hoping that sticks, because there is not a good understanding (or willful neglect) of how the different parts of the stack may adversely affect your application. Add that to ever-expanding requirements scope, poorly designed/maintained/understood code artifacts, and developer churn, and the typical software project is rather frail.
There are considerations you can make for more reliable software: a testing regimen and release planning, conservative resource estimates and knowing your bottlenecks, strategies for degraded operating conditions, fallback and error mitigation, scaling, consistent documentation, and basically knowing the seams of your software, where things might break, under what conditions, and what corrective action could be done.
Most software projects either move too quickly or are simply not important enough to hit these points. There are exceptions, of course (most well-known software we use would qualify), but those are not the rule: most YC companies certainly would not qualify as doing "software engineering." In fact, that almost seems to be the antithesis of a fast-moving startup.
Imagine your civil engineer did not take shear, vibration, bedrock, joint and material strength, etc into account when designing a structure, or allowed a good design to be constructed with shoddy labor, duct-taped together. That is exactly what we see from most software "engineering" today. Move fast and break things, indeed.
Re: Etsy CTO: We Need Software Engineers, Not Developers
#54Joel Spolsky wrote an essay on "leaky abstractions," and from reading the interview with the Etsy CTO, the two seem related. Allspaw is making the distinction between engineers and developers this way. To him, an engineer is willing and able to get under the hood, if necessary; whereas a developer is limited to using libraries and frameworks as-is, and filing bug reports on them when things don't go as planned, or fe…
Oh man, the Law of Leaky Abstractions is an amazing read, and an incredibly useful general theory. It's so applicable that I've taught my non-technical friends and family as it related to university structures, the insurance industry, management, technical support, etc.
http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...
Re: Etsy CTO: We Need Software Engineers, Not Developers
#55I am sympathetic to the distinction. I had a bunch of applicants for a position claiming to be "Distributed Systems Engineers" mostly because they'd stood-up, maintained, and or used a Hadoop or Spark cluster in their current or prior job. But to me that was more of an advanced (one hopes) end-user. Someone who could take a bunch of large, mostly-complete logical components that somebody else engineered and then use…
I don't think the skills/abilities you're looking for would be denoted merely by looking for an "engineer" of some kind. In fact, I think looking for an "engineer" likely obfuscates your actual needs since it has become such a catchall term in computing fields.
Re: Etsy CTO: We Need Software Engineers, Not Developers
#56His distinction between 'engineer' and 'developer' is artificial and easily criticized. That isn't what is interesting in this article. There are a lot of things that are. Their method for avoiding the proliferation of different tools is interesting. His discussion of the kind of people they want, once you ignore the semantics, is interesting. They want people who take responsibility and are always pushing the bounda…
> His discussion of the kind of people they want, once you ignore the semantics, is interesting.
semantics == meaning. once you ignore what he's trying to say, the discussion ends, no?
edit: yes, i could care less about semantics of semantics, but i can't bring myself to it.
Re: Etsy CTO: We Need Software Engineers, Not Developers
#57Their focus on limited tooling resonates with me. Keeping a stack simple can really pay off. Etsy is actually a customer of ours at Cronitor, specifically their team in Europe. When we helped them on-board, their engineer mentioned that on their primary Etsy.com product they would likely opt for building their tooling in house -- something that is possible if you're as big as etsy.
Re: Etsy CTO: We Need Software Engineers, Not Developers
#58High-assurance software and security field have been doing real engineering for decades. Here's a nice presentation with a tie-in to another high-assurance discipline: http://web.cecs.pdx.edu/~hook/cs491sp08/AssuranceSp08.ppt Here's an old method, Cleanroom, that was cost-effective for business with low, defect rate even on first use: http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=E10... . Altran[-Praxis] i…
The 2nd and 3rd links are truncated and do not seem to go where they are supposed to go.
Didnt know this forum did that and Im already at work. Ill reply to that comment with new links soon if possible on mobile. Otherwise Ill add it when Im home after 10p.
Edit: added them in a reply via pastebin.
Re: Etsy CTO: We Need Software Engineers, Not Developers
#59I am sympathetic to the distinction. I had a bunch of applicants for a position claiming to be "Distributed Systems Engineers" mostly because they'd stood-up, maintained, and or used a Hadoop or Spark cluster in their current or prior job. But to me that was more of an advanced (one hopes) end-user. Someone who could take a bunch of large, mostly-complete logical components that somebody else engineered and then use…
> But to me that was more of an advanced (one hopes) end-user. Someone who could take a bunch of large, mostly-complete logical components that somebody else engineered and then use them to stitch together a solution by integrating these existing frameworks... I don't find this distinction very useful. We're all end-users at different layers in the stack. Building HDFS from scratch is also mostly taking others' compo…
I agree with you. On all counts. I also don't find my own distinction very useful, and I also agree that part of it is "level in the stack" related problems.
The attribution for me had a lot more to do with the balance of optimism and skepticism. In my head the "developer" sees HDFS and goes, "Sweet, somebody solved this problem, now let me go use that thing and it will give me all these wonderful solved-problem qualities I don't have to think about anymore. This is going to save me a ton of time." The "engineer" looks at HDFS and goes, "Hmmm. This thing seems interesting, but this feature over here must be an incredibly painful one to use despite the fact that it seems super useful and is plastered all over their docs as being awesome. Because there's no free lunch in this problem space. So what possible methods are there to have implemented this kind of thing and how exactly can I test and exploit just how weak these floorboards are before I decide to start building on it?"
Again, not a very useful distinction. Agreed.
Re: Etsy CTO: We Need Software Engineers, Not Developers
#60I am sympathetic to the distinction. I had a bunch of applicants for a position claiming to be "Distributed Systems Engineers" mostly because they'd stood-up, maintained, and or used a Hadoop or Spark cluster in their current or prior job. But to me that was more of an advanced (one hopes) end-user. Someone who could take a bunch of large, mostly-complete logical components that somebody else engineered and then use…
I don't think the skills/abilities you're looking for would be denoted merely by looking for an "engineer" of some kind. In fact, I think looking for an "engineer" likely obfuscates your actual needs since it has become such a catchall term in computing fields.
Completely agreed.