Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unfashionably secure: why we use isolated VMs

blog.thinkst.com

Post reply on HN
Print view View original

171–180 of 256 posts

Re: Unfashionably secure: why we use isolated VMs

#171

Post by b112 »

compsciphd wrote: post #148
XorNot wrote: post #133

Earlier quoted context omitted.

What I really want is a "magic" shell on a VM - i.e. the ability using introspection calls to launch a process on the VM which gives me stdin/stdout, and is running bash or something - but is just magically there via an out-of-band mechanism.

Not really "out of band", but many VMs allow you to setup a serial console, which is sort of that, albeit with a login, but in reality, could create one without one, still have to go through hypervisor auth to access it in all cases, so perhaps good enough for your case?

You can launch KVM/qemu with screen + text console, and just log in there. You can also configure KVM to have a VNC session on launch, and that ... while graphical, is another eye into the console + login.

(Just mentioning two ways without serial console to handle this, although serial console would be fine.)

Re: Unfashionably secure: why we use isolated VMs

#172

Post by ianpurton »

I've solved the same problem but used Kubernetes namespaces instead.

Each customer gets their own namespace and a namespace is locked down in terms of networking and I deploy Postgres in each namespace using the Postgres operator.

I've built an operator for my app, so deploying the app into a namespace is as simple as deploying the manifest.

Re: Unfashionably secure: why we use isolated VMs

#173

Post by thinkst »

jmprspret wrote: post #146
ploxiln wrote: post #75

> we operate in networks where outbound MQTT and HTTPS is simply not allowed (which is why we rely on encrypted DNS traffic for device-to-Console communication) HTTPS is not allowed (locked down for security!), so communication is smuggled over DNS? uhh ... I suspect that a lot of what the customer "security" departments do, doesn't really make sense ...

DNS tunneling, or smuggling through DNS requests, is like a known malware C2 method. Seems really weird to (ab)use it for ""security""

Product builders can learn loads from malware in terms of deployment and operational ease. Malware needs to operate without any assistance in unknown environments. Nobody is allowing outbound comms deliberately for malware, so tunnel methods were developed.

Networks have these capabilities, inherently they're part of the specs. But only malware seems to realise that and use it. We love reusing offensive techniques for defence (see our Canarytokens stuff), and DNS comms fits that perfectly. Our customers get an actual 2-minute install, not a 2-minute-and-then-wait-a-week-for-the-firewall-rules install.

Re: Unfashionably secure: why we use isolated VMs

#174

Post by b112 »

kkfx wrote: post #164
secondcoming wrote: post #103

Earlier quoted context omitted.

I'm not sure what you're referring to here? Our cloud machines are largely VMs. Deployments mean building a new image and telling GCP to deploy that as machines come and go due to scaling. The software is up to date, dependencies are managed via ansible. Maybe you think VMs means monoliths? That doesn't have to be the case.

That's precisely the case: instead of owning hw, witch per-machine it's a kind-of monolith (even counting blades and other modular solution), you deploy a full OS or half-full to run just a single service, on top of another "OS". Of course yes, this is the cloud model, and is also the ancient and deprecated mainframe model, with much more added complexity and no unique ownership with an enormously big attack surface.…

(note .. use 'which' not 'witch', quite different words)

Not sure if you mentioned it, but cost and scaling is an absurd trick of AWS and others. AWS is literally 1000s, and in some usage cases even millions of times more expensive than your own hardware. Some believe that employee cost savings help here, but that's not even remotely close.

Scaling is absurd. You can buy one server worth $10k, that can handle the equivalent of thousands upon thousands of AWS instances' workload. You can buy far cheaper servers ($2k each), colo them yourself, have failover capability, and even have multi-datacentre redundancy, immensely cheaper than AWS. 1000 of times cheaper. All with more power than you'd ever, ever, ever scale at AWS.

All that engineering to scale, all that effort to containerize, all that reliance upon AWS and their support system.. unneeded. You can still run docker locally, or VMs, or just pound it out to raw hardware.

So on top of your "run it on bare metal" concept, there's the whole "why are you wasting time and spending money" for AWS, argument. It's so insanely expensive. I cannot repeat enough how insanely expensive AWS is. I cannot repeat enough how AWS scaling is a lie, when you don't NEED to scale using local hardware. You just have so much more power.

Now.. there is one caveat, and you touch on this. Skill. Expertise. As in, you have to actually not do Really Dumb Things, like write code that uses 1000s of times CPU to do the same task, or write DB queries or schema that eat up endless resources. But of course, if you do those things on your own hardware, in DEV, you can see them and fix.

If you do those in AWS, people just shrug, and pay immense sums of money and never figure it out.

I wonder, how many startups have failed due to AWS costs?

Re: Unfashionably secure: why we use isolated VMs

#176

Post by khana »

PedroBatista wrote: post #2

As a permanent "out of style" curmudgeon in the last ~15 years, I like that people are discovering that maybe VMs are in fact the best approach for a lot of workloads and the LXC cottage industry and Docker industrial complex that developed around solving problems created by themselves or solved decades ago might need to take a hike. Modern "containers" were invented to make things more reproducible ( check ) and sim…

[dead]

Re: Unfashionably secure: why we use isolated VMs

#177

Post by blipvert »

compsciphd wrote: post #148
XorNot wrote: post #133

Earlier quoted context omitted.

What I really want is a "magic" shell on a VM - i.e. the ability using introspection calls to launch a process on the VM which gives me stdin/stdout, and is running bash or something - but is just magically there via an out-of-band mechanism.

Not really "out of band", but many VMs allow you to setup a serial console, which is sort of that, albeit with a login, but in reality, could create one without one, still have to go through hypervisor auth to access it in all cases, so perhaps good enough for your case?

Indeed, easy enough to get a serial device on Xen.

Another possibility could be to implement a simple protocol which uses the xenstore key/value interface to pass messages between host and guest?

Re: Unfashionably secure: why we use isolated VMs

#178

Post by jmprspret »

thinkst wrote: post #173
jmprspret wrote: post #146

Earlier quoted context omitted.

DNS tunneling, or smuggling through DNS requests, is like a known malware C2 method. Seems really weird to (ab)use it for ""security""

Product builders can learn loads from malware in terms of deployment and operational ease. Malware needs to operate without any assistance in unknown environments. Nobody is allowing outbound comms deliberately for malware, so tunnel methods were developed. Networks have these capabilities, inherently they're part of the specs. But only malware seems to realise that and use it. We love reusing offensive techniques fo…

I didn't mean to imply that so for security was a bad thing. Now I read back my comment I see that is exactly how it sounds.

I agree with you

Re: Unfashionably secure: why we use isolated VMs

#179

Post by Veserv »

tptacek wrote: post #159
Veserv wrote: post #125

Earlier quoted context omitted.

You mean your argument that it is hard to find a vulnerability, despite the fact that commercial systems in Unix kernel lineage have historically been easy to hack and have never once demonstrated high robustness? You have not even established what level of security you are arguing has been achieved. This is not even moving the goalposts, this is Calvinball. I contend that a major cloud service, that runs trillions o…

Do you feel like this is going well for you?

Yes. You have been unable to provide any objective or quantitative means of evaluating security and reject any attempt to do so. You even reject attempts to characterize what would constitute an objective minimum bar for adequacy.

"I can not evaluate my work, I demand you do not evaluate my work, and I do not even know what my goal is, but I can tell you hot or cold."

What you have there is not engineering, it is art and is why commercial IT software security is a joke.

Re: Unfashionably secure: why we use isolated VMs

#180

Post by fsflover »

tptacek wrote: post #91
Veserv wrote: post #87

Earlier quoted context omitted.

VT-x. You should get the name of the technology right before defending it. VT-d is the I/O virtualization technology. When did it become customary to defend people making claims of security instead of laughing in their face even though history shows them such claims to be a endless clown parade? How about you present the extraordinary evidence needed to support the extraordinary claim that there are no vulnerabilitie…

Not an especially impressive flex, but I'm not above trying to dunk on people for misspelling things either, so I'm not going to high-horse you about it (obviously i am). The history of KVM and hardware virtualization is not an endless clown parade. Find a vulnerability researcher to talk to about OpenBSD sometime, though. https://isopenbsdsecu.re/

I did mean VT-d, not VT-x. This is what Qubes OS uses for virtualization; my daily driver. Last escape was in 2006 by the Qubes founder.
Post reply on HN