Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 'internet of things' is sending us back to the Middle Ages

theconversation.com

Post reply on HN
Print view View original

41–50 of 54 posts

Re: The 'internet of things' is sending us back to the Middle Ages

#41

Post by jeffdavis »

bsder wrote: post #22
jeffdavis wrote: post #12

The problem is software complexity. You can't really own anything you don't understand, and you can't be free in a world you don't understand. And we don't understand software. Imagine a naive young king who doesn't understand how to do anything for himself. He will easily be controlled by those around him that do understand the world. Of course, nobody before ever understood everything about the world. But different…

> And we don't understand software. People are perfectly capable of understanding very large programs even from reverse engineering. The issue is that things change so quickly that generating such knowledge is economically useless. I have this discussion with my CTO all the time: "I'd like you to go do ". "No, please give that to ." "But he's junior. I want someone senior to do it." "The half-life on the knowledge to…

"The issue is that things change so quickly that generating such knowledge is economically useless."

That's a good point, and certainly part of the problem, but it does not detract from my point.

The bottom line is that we don't understand the software in the world around us and that it would be hopeless to try. Whether that's because of static complexity or dynamic complexity seems like a separate discussion.

Changing software on the scale that humans do is kind of like changing the laws of physics every year. Great to know that we understand some past snapshot of the software stack that we built, but not relevant to living in today's world.

Re: The 'internet of things' is sending us back to the Middle Ages

#42

Post by jeffdavis »

golemotron wrote: post #21
jeffdavis wrote: post #12

The problem is software complexity. You can't really own anything you don't understand, and you can't be free in a world you don't understand. And we don't understand software. Imagine a naive young king who doesn't understand how to do anything for himself. He will easily be controlled by those around him that do understand the world. Of course, nobody before ever understood everything about the world. But different…

> The problem is software complexity. You can't really own anything you don't understand, and you can't be free in a world you don't understand. And we don't understand software. What this means is that for all practical purposes AI is already here. AI, or rather the fear of AI, is about software making decisions that impact us outside of our knowledge or control. Prior to software this was done by bureaucracy. We ca…

+1. The AI apocalypse looks more like a Kafka-esque bureaucracy and not so much like I, Robot with machines shooting at us.

Re: The 'internet of things' is sending us back to the Middle Ages

#43

Post by jeffdavis »

zeep wrote: post #38
digi_owl wrote: post #13

Earlier quoted context omitted.

and much of that complexity, at least on personal computers, gets added in the name of "user friendliness"...

also in the name of saving money/time (by using large libraries/frameworks and such)

And it does! That's what's maddening about it. The complexity always seems to justify itself, but the end results are probably not where we want to be.

Re: The 'internet of things' is sending us back to the Middle Ages

#44

Post by jeffdavis »

anmorgan wrote: post #30
jeffdavis wrote: post #12

The problem is software complexity. You can't really own anything you don't understand, and you can't be free in a world you don't understand. And we don't understand software. Imagine a naive young king who doesn't understand how to do anything for himself. He will easily be controlled by those around him that do understand the world. Of course, nobody before ever understood everything about the world. But different…

> Software is so complex that nobody really understands it. Unless this statement is trying to imply something deeper than what it states, I would disagree. I work with embedded systems and connected devices. To me it's not a problem of complexity, but a problem of not having a strong standard of practices on both the device side and the facility side. Have sensitive data on your network? Separate it from the rest of…

"don't put devices on the network that don't meet your security needs"

Does it not disturb you that the mere presence of a device on your network could compromise everything else? Why is that?

I argue it's because of complexity. We don't really know how these devices behave outside of very controlled circumstances.

And the vulnerability could be anywhere in the stack. I remember this bug from a while back:

https://lcamtuf.blogspot.com/2014/10/psa-dont-run-strings-on...

The gnu strings utility was vulnerable to untrusted input! Who would possibly imagine that would ever be the case!

Honestly, that bug just made me give up. Software cannot be reasoned about any longer. I used to believe that solid components strung carefully together could add up to something understandable. But no, we are beyond that.

I'm not talking about crazy James Bond hackers that somehow infected your compiler or something. I mean that, by accident, a basic utility does something crazy.

This is not math or science or engineering any more. It's wizardry, witchcraft, and alchemy.

(I'm exaggerating a bit, but it really is discouraging to me.)

Re: The 'internet of things' is sending us back to the Middle Ages

#45

Post by joshtch »

jeffdavis wrote: post #44
anmorgan wrote: post #30

Earlier quoted context omitted.

> Software is so complex that nobody really understands it. Unless this statement is trying to imply something deeper than what it states, I would disagree. I work with embedded systems and connected devices. To me it's not a problem of complexity, but a problem of not having a strong standard of practices on both the device side and the facility side. Have sensitive data on your network? Separate it from the rest of…

"don't put devices on the network that don't meet your security needs" Does it not disturb you that the mere presence of a device on your network could compromise everything else? Why is that? I argue it's because of complexity. We don't really know how these devices behave outside of very controlled circumstances. And the vulnerability could be anywhere in the stack. I remember this bug from a while back: https://lc…

I agree, and this is why "safe" languages like Rust, OCaml, Erlang, etc. are important. It's not possible to guarantee safety under all circumstances, but as software becomes more complex, anything we can do ensure some degree of safety is pretty crucial.

Re: The 'internet of things' is sending us back to the Middle Ages

#46

Post by kogepathic »

dozzie wrote: post #16
kogepathic wrote: post #3

On the whole I agree with the author, but they have glossed over some things I feel should be addressed: > The computer manufacturer Lenovo, for instance, used to sell its computers with a program called “Superfish” preinstalled. Most "serious" users reformat the machine and re-install the OS to get rid of the OEM included crapware. Not something the average user does, sure, but you're still allowed to re-install the…

>> The computer manufacturer Lenovo, for instance, used to sell its computers with a program called “Superfish” preinstalled. > Most "serious" users reformat the machine and re-install the OS to get rid of the OEM included crapware. [...] It's a matter of inconvenience, not impossibility. And this is why said Lenovo included the crapware and spyware in non-volatile memory that is executed by Windows boot automaticall…

> And this is why said Lenovo included the crapware and spyware in non-volatile memory that is executed by Windows boot automatically by design, so your reinstallation doesn't work as intended

Could you please provide a citation for this claim?

I've never heard of Lenovo bundling parts of Superfish into UEFI to be installed again if Windows is reinstalled.

Re: The 'internet of things' is sending us back to the Middle Ages

#47

Post by flukus »

kogepathic wrote: post #3

On the whole I agree with the author, but they have glossed over some things I feel should be addressed: > The computer manufacturer Lenovo, for instance, used to sell its computers with a program called “Superfish” preinstalled. Most "serious" users reformat the machine and re-install the OS to get rid of the OEM included crapware. Not something the average user does, sure, but you're still allowed to re-install the…

> Most "serious" users reformat the machine and re-install the OS to get rid of the OEM included crapware.

And then you get all the fun of running around the web for random drivers, navigating the OEM's and third parties websites to find the multi hundred GB driver package that reinstalls the crapware. That's if the OEM site will let you download the drivers of course, I've run into a few where it wasn't possible.

Re: The 'internet of things' is sending us back to the Middle Ages

#48

Post by pmontra »

wepple wrote: post #35
jwmullally wrote: post #19

I agree. Copy+pasting an old comment of mine from this old thread https://news.ycombinator.com/item?id=12683924 > A general solution to IoT security would be for all IoT devices to only communicate to your personally owned home gateway, which would run open-source drivers for each device to provide the networking/external communication functionality. The IoT device could even be assigned its own isolated network link…

This is a great solution to the ownership problem, but now each individual is responsible for patching and updating their 25 devices? Doesn't solve the security problem.

The home gateway could download the patches and send them to the devices. Its firewall will prevent them to phone home or somewhere else no matter what into those patches.

Re: The 'internet of things' is sending us back to the Middle Ages

#49

Post by pmontra »

jeffdavis wrote: post #12

The problem is software complexity. You can't really own anything you don't understand, and you can't be free in a world you don't understand. And we don't understand software. Imagine a naive young king who doesn't understand how to do anything for himself. He will easily be controlled by those around him that do understand the world. Of course, nobody before ever understood everything about the world. But different…

I don't know how to make clothes (I guess very few of us do) but I do own them and I can somewhat fix them. Imagine if we had to go back to the manufacturer to sew on a button. We can even mod clothes and copy designs without being sued.

A couple of links about that

https://www.techdirt.com/articles/20100526/0039459578.shtml

https://www.ted.com/talks/johanna_blakley_lessons_from_fashi...

Re: The 'internet of things' is sending us back to the Middle Ages

#50

Post by blub »

jeffdavis wrote: post #44
anmorgan wrote: post #30

Earlier quoted context omitted.

> Software is so complex that nobody really understands it. Unless this statement is trying to imply something deeper than what it states, I would disagree. I work with embedded systems and connected devices. To me it's not a problem of complexity, but a problem of not having a strong standard of practices on both the device side and the facility side. Have sensitive data on your network? Separate it from the rest of…

"don't put devices on the network that don't meet your security needs" Does it not disturb you that the mere presence of a device on your network could compromise everything else? Why is that? I argue it's because of complexity. We don't really know how these devices behave outside of very controlled circumstances. And the vulnerability could be anywhere in the stack. I remember this bug from a while back: https://lc…

You might find Normal Accidents by Perrow interesting. He describes accidents caused by systems (e.g. nuclear power plants) becoming so complex that they are incomprehensible to humans.

"I used to believe that solid components strung carefully together could add up to something understandable. But no, we are beyond that."

I think that's true, but it turns out that we don't have any solid components. None.

Post reply on HN