Earlier quoted context omitted.
The ACM's insistence on clearly delineated roles and specialties seems so bizarre and misguided. Having defined roles necessarily implies some rigidity in allowed process and organizational structure, which seems out of scope for an engineering body of knowledge. If you insist on defined roles then you end up with something like Scaled Agile Framework (SAFe) or Large Scale Scrum (LeSS). Which aren't necessarily bad m…
This is actually a general problem with the SWEBOK: it isn't an engineering body of knowledge. It's a set of management practices like the ones you describe. It doesn't steer clear of those issues, at all.
Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
121–130 of 170 posts
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#122There appears to be a lot of hate towards this in the comments (because it's not perfect?), but I feel strongly that we need explicit bodies of knowledge, along with certifications for having been trained on it. Every company I go to, the base of knowledge of all the engineers is a complete crapshoot. Most of them lack fundamental knowledge about software engineering. And they all lack fundamental knowledge about the…
You are under this illusion about other fields like architects because you don't work there and you can't tell. You don't know how the sausage is made. Historically I have tended to learn about a new field WAY too much before I tried to hire people in these fields. The truth is, that makes it hard to hire people (but for good reason - depending on your needs, you need to pass on a lot of people). More recently I have…
But I am also aware what the lack of it does. It leads to buildings falling down or burning up [with people in them]. This was a common occurrence 100+ years ago. You know what made it less common? Standardization. Building codes. Minimum standards for engineers and the trades. Independent studies have all concluded that real world outcomes improved across the board because of these things.
No formal certification or standard will lead to perfection. That is obvious. But what is also obvious, from actually looking at outcomes before and after their introduction, is that having them leads to better outcomes.
You have to stop thinking about individual engineers, and start thinking about the much, much larger picture. What changes will have a positive effect on the larger picture? You can only have an effect on the larger picture if you enforce a change across the board, and then look at the aggregate results.
That can not happen without a mechanism to enforce the change. We can't pray our way to better results, or just sit around hoping people magically get better at their jobs, because that clearly has not happened for the last few decades that I've been working.
The more we depend on technology, the more we see the failures of a lack of rigor. Probably every single person with an address and social security number in the United States has had their personal information leaked, multiple times over, by now. Lives are ruined by systems that do not take into consideration the consequences of a lack of safety, or the bias of its creators. Entire global transportation systems are shut down because nobody added basic tests or fail-safes to critical software infrastructure.
This shit isn't rocket science, man. It was all preventable. And just like with building codes, standards, licenses, etc, we can put things in place to actually teach people the right way to do things, and actually check for the preventable things, by law. If we don't, it's going to keep happening, and keep happening, and keep happening, and keep happening, forever.
We can do something to stop it. But we have to pound our fist on the desk and say, enough is enough. We have to put something imperfect in place to stem the tide of enshittification. Because there are consequences if we don't.
We have seen some of them globally in the form of warfare, but nothing compared to the devastation when the gloves come off. We have not yet seen an entire country's hacker resources attack the water, power, sanitation, food, and other systems of its enemy, all at once. But it's going to happen. And it's going to be devastating. Millions of people are going to die because some asshole set a default password on some SCADA systems. But it should have been impossible, because no SCADA system should be allowed to be sold with default passwords. That's the kind of thing we can prevent, just like you can't build a building today without a fire exit.
That's the really big obvious impact. The impact nobody sees are from tiny decisions all the time, that slowly affect a few people at a time, but on the scale of millions of businesses and billions of people, add up to really big effects. We can make a huge difference here too, which will only be visible in aggregate later on. Like public sanitation, clean water, or hand-washing with soap, nobody thinks about the dramatic effect on public health and longevity until it's clear after decades what kind of impact it made. Technology is everywhere, in every home, affecting every life. The more we improve it [as a standard], the more we will see huge positive impacts later.
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#123Earlier quoted context omitted.
You are under this illusion about other fields like architects because you don't work there and you can't tell. You don't know how the sausage is made. Historically I have tended to learn about a new field WAY too much before I tried to hire people in these fields. The truth is, that makes it hard to hire people (but for good reason - depending on your needs, you need to pass on a lot of people). More recently I have…
On the contrary, I am fully aware that there exists no field where a test or piece of paper guarantees excellence. But I am also aware what the lack of it does. It leads to buildings falling down or burning up [with people in them]. This was a common occurrence 100+ years ago. You know what made it less common? Standardization. Building codes. Minimum standards for engineers and the trades. Independent studies have a…
To me, this is a more interesting comparison. Is it PE certification and contractor licenses that led to this or is it building codes, construction inspectors, occupancy permits? I will argue that it's inspectors, NOT PE or contractors. And I will argue that the buildings codes have major negative consequences also. We all know of constructions methods that would have great benefits but have to be abandonned because they don't easily fit the current code. We all know of buildings that are to-code and yet ridiculously noisy and cheaply built.
I will also argue that there are building code equivalents already in software and system architecture. There are several for "certifying" system or site security and systems that host credit card payments. And we all know how well they work. So I agree with you that there is room for progress there, but I will also argue that the approach NEEDS to be different. The current security or payment checklists are bureaucratic, CYA nonsense which discourage thinking and encourage bureaucracy and CYA specifically in place of actual security. The only thinking they encourage is creative writing to twist reality into the proper buzzwords.
There may be a way to specify practices and security but we sure have not discovered it yet. So, a research question rather than already a standardization question? I will point out also that there WERE directions that did work in the past. For example, Dan Farmer and Wietse Venema's SATAN (and the several descendants since then) was bureaucracy-free: the test showed specific rubber-meets-the-road issues with your system that you could either fix or defend. No bullshit about using a firewall(tm) "because that's best practice".
I also don't say that it's bad to publish books. I will say that it is bad to push "best practice". "Best practice" is precisely bureaucracy and CYA in place of thinking. To the point of site owners defending their lapses in the name of "best practices".
What else currently goes in the right direction? Pen testing. Bug rewards. Code reviews.
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#124Earlier quoted context omitted.
Why do you think such wrong things about civil and mechanical engineering. Tell me about all the on time and under budget civil/mechanical engineering projects that are happening. Do you think that just because they have physics to lean on that they can just like press solve and have accurate estimates spit out? Edit: I totally agree that more long-lived battle tested software toolchains and libraries would be great…
How do you know things wouldn’t be much much worse if there were no standards for being a civil/structural engineer or architect that have been refined over long periods of time? Imagine municipalities taking the lowest bids by far thrown out there by any rando that decided they can make a few bucks by welding together the supports for a bridge or designing a really interesting building that will just cave in on itse…
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#125Runtime errors surface when a program runs into an unexpected condition or situation such as dividing by zero, memory overflow, or addressing a wrong or unauthorized memory location or device, or when a program tries to perform an illegitimate or unauthorized operation or tries to access a library, for example. The programs must be thoroughly tested for various types of inputs (valid data sets, invalid data sets and…
What exactly is wrong with it? That is a definition fit for someone who does not have prior knowledge of what a runtime error is. It might be boring to us, and I might word it a little different, but it's fine.
> Runtime errors surface when a program runs into an unexpected condition or situation such as dividing by zero,
This implicitly asserts that dividing by zero is always unexpected, that all runtime errors are unexpected, and that dividing by zero always causes a runtime error. None of these are true. Dividing by zero is very frequently not unexpected (it's utterly commonplace in 3-D graphics, for example, when a vertex happens to be in the camera's focal plane) and in floating-point math (standardized by, ironically, the IEEE) it doesn't generate runtime errors by default. Moreover, there are certain applications where runtime errors are also expected, such as in fuzzers. Some languages frequently use their runtime error mechanism to convey perfectly mundane conditions such as not finding a key in a hash table or coming to the end of a sequence. And it's fairly often a useful mechanism for terminating a recursive search procedure.
> memory overflow,
There's no such thing as a "memory overflow" on any platform I'm familiar with. This might have been intended to reference any of five things: stack overflows, buffer overflows, heap allocation failures, resource limit exhaustion, or OOM kills.
The stack is an area of memory, and a "stack overflow" is an attempt to put too much data in it, usually due to an infinite recursive loop. Stack overflows do often get detected as runtime errors, but many platforms fail to detect them, causing incorrect program behavior instead.
A buffer is also an area of memory (an array) and a "buffer overflow" is an attempt to put too much data in it, usually due to an omitted check on input data size. Buffer overflows do sometimes get detected as runtime errors, but often do not, because the most popular implementations of many popular programming languages (notably C, C++, and Pascal) do not do automatic bounds checking.
Heap allocation failures are attempts to dynamically allocate memory that are denied, often because too much memory is already allocated, so in a sense your heap memory has "overflowed". Some programming languages handle this as a runtime error, but the most popular implementations of many popular programming languages (notably C, C++, and Pascal) do not.
Resource limit exhaustion is a condition where a program exceeds operating-system-imposed limits on its usage of resources such as CPU time, output file size, or memory usage, in response to which the OS raises a runtime error. On Unix this manifests as killing the process with a signal.
OOM kills are a Linux thing where the OS starts killing resource-intensive processes to attempt to recover from a memory overload condition that is degrading service badly. In a sense the computer's memory has "overflowed", so Unix's runtime error mechanism is pressed into service. (Something very similar happens if you press ^C, though, so describing this as a "runtime error" is maybe not defensible.)
> or addressing a wrong or unauthorized memory location or device,
The standard verb here would be "accessing", not "addressing", but I don't want to make too much of the use of nonstandard terminology; it's not always an indicator of unfamiliarity with the field. And there are indeed machines where there are kinds of "wrong" other than "unauthorized"—that's why we have SIGBUS, for unaligned access exceptions on CPUs that don't support unaligned access.
But "device" is, as far as I know, wrong; I don't know of any platform which raises a runtime error when you try to access the wrong device. There are a number of platforms that have I/O instructions which will raise a runtime error if you try to use them in unprivileged code, but that doesn't depend on which device you're accessing. Usually in small microcontrollers there isn't any kind of runtime-error mechanism, and when there is, it generally isn't invoked for I/O instructions. Sometimes you can set up the MPU so that certain code can't access devices—when they're memory-mapped, in which case "memory location" would have covered the case.
Even if there is some platform where it's possible to get a runtime error accessing the wrong device, it's far from the kind of common case you'd want to include in such an abbreviated list of typical causes of runtime errors "for someone who does not have prior knowledge of what a runtime error is". Much more typical would be illegal instructions, attempts to access privileged state from unprivileged code, null pointer dereferences, runtime type errors such as ClassCastException, and array bounds violations, none of which are mentioned.
It seems like someone who doesn't know anything about software was trying to imagine what kinds of things might, in theory, cause runtime errors, but the kinds of protection mechanisms they imagined were completely different from the ones used in actual existing computers.
> or when a program tries to perform an illegitimate or unauthorized operation
As a simple matter of logic, this subsumes the previous item, and "illegitimate" in this context means the same thing as "unauthorized". So this list is not just comprehensively wrong at a technical level, but also so badly written as to be incoherent.
> or tries to access a library, for example.
This is completely wrong. Of course a program can get a runtime error when it tries to load a library, just as it can get a runtime error when it undertakes any other task, but there is no platform that raises a runtime error whenever you try to access a library.
> The programs must be thoroughly tested for various types of inputs (valid data sets, invalid data sets and boundary value data sets) and conditions to identify these errors.
This statement, in itself, only contains a couple of slight factual errors.
First, it says that the inputs to programs are data sets. This is probably something that someone who doesn't know anything about software copied from a 60-year-old textbook, because at that time, the inputs to programs were data sets (now more commonly known as files). In fact, the more important inputs to programs nowadays are things such as user interactions, data received over a network, and data received in real time from sensors such as microphones, oxygen sensors, and accelerometers. But this statement is stuck in punched-card land, where programs punched into card decks ran in batch mode to process some input data sets and produce some output data sets.
The second factual error is that it implies that runtime errors result primarily from inputs and how the program responds to them. And, of course, there are runtime errors that are determined by the program's handling of inputs; typically type errors, null pointer exceptions, array bounds violations, etc., can be reproduced if you run the program again with the same input. But large classes of runtime errors do not fit this profile. Some are errors that depend on internal nondeterminism in the program, such as nondeterministic interactions between different threads (we would say "scheduler interleaving order" back in the single-core days), or nondeterministic timing. Others depend on runtime conditions like how much memory is available to allocate or how full the disk is (writing to a full disk in Python raises an exception, though not in many other languages). Others result from hardware problems such as overheating. The statement does, in passing, say "and conditions", but these are given short shrift.
The much bigger problem with this statement, though, is that it's in the middle of a paragraph about runtime errors. But everything it says is equally applicable to logic errors, the subject of the next paragraph; indeed, much of it is more applicable to logic errors. Putting it in the middle of this paragraph misleadingly implies that things like testing with invalid input files (or the other, more important, invalid inputs it failed to mention) are uniquely or primarily applicable to runtime errors.
> Once identified, runtime errors are easy to fix.
This statement is also wrong in a multidimensional way.
First, although this is in the "debugging" section, many runtime errors aren't bugs at all. A program crashing with a MemoryError or OutOfMemoryError when it is run with insufficient memory available to allocate is, in general, desired behavior. It's also often the desired behavior when it's applied to a too-large input, though obviously that's not acceptable for games or avionics software. The same is true of permission errors, out-of-disk-space errors, etc. These are specifically the kind of runtime errors that most commonly result from the "and conditions" in the previous sentence.
Second, while it's usually easier to debug a segfault or exception than silently incorrect output, it is often far from trivial to track down the runtime error to its ultimate cause.
Third, even when you have figured out what the ultimate cause is, fixing it is sometimes not easy, for example because other people's deployed software depends on interfaces you have to change, or more generally because it is difficult to understand what other problems could be caused by the possible fixes you are considering.
By my count, that's 12 significant factual errors in 86 words, an average of 7.2 words per error. This is an error density that would be impressive in any contexts, although I've occasionally seen expert liars exceed it somewhat. In the context of an official publication purporting to establish standards for professional competence, it's utterly horrifying.
So this is not a definition fit for anyone, least of all someone who does not have prior knowledge of what a runtime error is; it is very far from being fine. It's embarrassing horseshit.
And it's not just this one paragraph. Virtually the entire document is like this. Occasionally you'll find a paragraph that's mostly correct from beginning to end, but then it'll be followed by the cringiest idiocy you can possibly imagine in the very next paragraph.
With respect to this paragraph in particular, an even worse problem than its unreliability is emphasis. This is from §4.6, "Debugging", which is 189 words, about a quarter of p. 16-13 (p. 326/413). Debugging is arguably the central activity of software engineering; about a quarter of the effort† spent on software engineering is spent on debugging, an amount which would be larger if we didn't spend so much effort on avoiding debugging (for example, with type systems, unit tests, version control, and documentation). Any actual software engineering body of knowledge would be largely concerned with knowledge about debugging. But the IEEE relegates it to 0.2% of their document.
______
† in https://www.quora.com/In-a-typical-software-engineering-comp... Raja Nagendra Kumar says 80% in IT services and typically 20–50% in "product companies", and Gene Sewell says ⅓ of the time; in https://www.quora.com/How-much-time-does-a-programmer-spend-... Ben Gamble says 5–70% of each day and Stephen Bear says ideally about 10% if you're doing everything else right; https://old.reddit.com/r/learnprogramming/comments/1eclw4l/i... complains about spending 60–70% of their time debugging; https://old.reddit.com/r/computerscience/comments/lwemi5/is_... quotes the Embedded Market Survey as saying 20%, while other people report numbers as high as 80%; https://craftbettersoftware.com/p/debugging-the-biggest-wast... says 50%, referencing https://www.researchgate.net/publication/345843594_Reversibl.... So I think "a quarter" is a pretty reasonable ballpark.
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#126Earlier quoted context omitted.
How do you know things wouldn’t be much much worse if there were no standards for being a civil/structural engineer or architect that have been refined over long periods of time? Imagine municipalities taking the lowest bids by far thrown out there by any rando that decided they can make a few bucks by welding together the supports for a bridge or designing a really interesting building that will just cave in on itse…
There are tons of physical engineers working on safety critical hardware that are not required to have some BS piece of paper that says they're safe. You do not need a credential to work on EV charging infrastructure, rockets, crew capsules to ferry astronauts to the ISS, or many, many other things. That's how you know, because those fields are not less safe. It's an easy comparison.
Could you expand on that? Are you saying that you don’t need a licensed electrician to connect a new EV charging terminal at installation time?
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#127Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#128SWEBOK 4 adds a dedicated section for security, but it's painfully 2012 (testing, for instance, centers on the old industry-driven "SAST" vs. "DAST" distinction). It also promotes stuff like Common Criteria and CVSS. The "domain-specific" security section could have been pulled out of the OWASP wiki from 2012 as well: "cloud", "IOT", "machine learning".
Believe it is supposed to be slow-moving, keeping to settled matters. Have those fallen out of favor, or shown to be wrong already?
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#129Earlier quoted context omitted.
This is actually a general problem with the SWEBOK: it isn't an engineering body of knowledge. It's a set of management practices like the ones you describe. It doesn't steer clear of those issues, at all.
Only a small fraction of the SWEBOK covers management practices and it doesn't dictate any particular methodology. Competent engineers might not need to do management but they have to understand at least the basics of the management context in which they operate.
>>> import random
>>> r = random.SystemRandom()
>>> [(r.randrange(1, 414), r.randrange(1, 3)) for i in range(10)]
[(299, 1), (164, 2), (292, 1), (246, 2), (205, 2), (113, 1), (167, 2), (393, 2), (16, 1), (129, 2)]
Page 299/413 column 1 contains: part of a confused description of mathematical optimization in the sense of finding infima, incorrectly conflating it with space-time tradeoffs in software, which is at least a software engineering topic; and the beginning of a section about "multiple-attribute decision making", which is almost entirely about the kinds of decision-making done by corporate management. Though software design is given lip service, if you dig into the two particular "design" approaches they mention, they turn out to be about corporate management again, with concerns such as brainstorming sessions, identifying business cost drivers, staff headcount, presenting ideas to committees, etc. Conclusion: project management, not software engineering.Page 164/413 (6-12) column 2 is about corporate operations (for which telemetry can be used), corporate operational risk management, and automating operational tasks to improve corporate efficiency. Conclusion: project management, not software engineering.
Page 292/413 (15-3) column 1 is about software engineering economics, specifically proposals and cash flow. Project management, not software engineering.
Page 246/413 (11-12) is a table summarizing chapter 11, which contains both project management and software engineering elements. I'm going to eliminate this point from the sample as being too much work to summarize fully and too hard to avoid interpretation bias.
Page 205/413 (9-13) column 2 is about software engineering management issues such as the difficulty of estimation, the project risks posed by the rate of change of the underlying technology, metrics for managing software, and what software organizational engineering managers should know. Project management, not software engineering.
Page 113/413 (4-14) column 1 is about what a platform standard is, TDD, and DevOps. Mostly software engineering, not project management.
Page 113/413 (6-15) is another summary table page similar to page 246, so I'm eliminating it too.
Page 393/413 (A-5) column 2 is about the SWEBOK itself and the documents it draws from and contains no information about either project management or software engineering.
Page 16/413 (xv) is part of the table of contents, so I'm eliminating it as well.
Page 129/413 (5-12) column 2 is about random testing (software engineering), "evidence-based software engineering" (an utterly vapid section which contains no information about software engineering, project management, or anything else, as far as I can tell), and test cases that force exceptions to happen (software engineering). Conclusion: software engineering, not project management.
So of the seven non-eliminated randomly sampled half-pages in the document, four are about project management, two are about software engineering, and the seventh is just about the SWEBOK. I guess my declaration that it's just a set of management practices was incorrect. It's only mostly a set of management practices. It's not at all only a small fraction.
Re: Software Engineering Body of Knowledge (SWEBOK) v4.0 is out [pdf]
#130Earlier quoted context omitted.
Only a small fraction of the SWEBOK covers management practices and it doesn't dictate any particular methodology. Competent engineers might not need to do management but they have to understand at least the basics of the management context in which they operate.
I agree with your second sentence, but your first sentence is pretty profoundly incorrect. Each of its 413 pages is divided into two columns. I generated a random sample of 10 page numbers associated with column numbers as follows: >>> import random >>> r = random.SystemRandom() >>> [(r.randrange(1, 414), r.randrange(1, 3)) for i in range(10)] [(299, 1), (164, 2), (292, 1), (246, 2), (205, 2), (113, 1), (167, 2), (39…
I don't think you even understand what software engineering is. If we want to limit the discussion to just software development as a craft and take out the engineering aspects then you might have a point, but that's not what the SWEBOK is about.
And in fairness, most real world software projects can produce good enough results without applying real engineering practices. If you're just building yet another CRUD web app then rigorous engineering is hardly required or even economically justified.