Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Cloud Computer

oxide.computer

Post reply on HN
Print view View original

821–830 of 994 posts

Re: The Cloud Computer

#821

Post by bcantrill »

Always_Anon wrote: post #654
jjav wrote: post #641

Earlier quoted context omitted.

> Dell is a single vendor that will diagnose and fix all of your hardware issues. I've been a Dell customer at a previous company. I know for a fact that's not true. I had a support ticket for a weird firmware bug open for two years, they could never figure it out. I left that job but for all I know the case is still open many years later. Dell doesn't know how to fix things like that because they don't design and en…

I've been a Dell customer for decades at this rate and I know for a fact it's true. I've had support tickets open for all kinda of weird firmware, hardware, etc. bugs and they've been well resolved, even if it meant Dell just replaced the part with something comparable (NIC swap). >Dell doesn't know how to fix things like that because they don't design and engineer the systems they sell. Of course they do. That's lik…

Ironically, it was Dell's total inability to resolve a pathological rash of uncorrectable memory errors very much is part of the origin story of Oxide: this issue was very important to my employer (who was a galactic Dell customer) and as the issue endured and Dell escalated internally, it became increasingly clear that there was in fact no one at Dell who could help us -- Dell did not understand how their own systems work.

At Oxide, we have been deliberate at every step, designing from first principles whenever possible. (We -- unlike essentially everyone else -- did not simply iterate from a reference design.)

To make this concrete with respect to the CPU in particular, we have done our own lowest-level platform enablement software[0] -- we have no BIOS. No one -- not the hyperscalers, not the ODMs and certainly not Dell -- has done this, and even AMD didn't think we could pull it off. Why did we do it this way? Because all along our lodestar was that problem that Dell was useless to us on -- that we wanted to understand these systems from first principles, because we have felt that that is essential to deliver the product that we ourselves wanted to by.

There are plenty of valid criticisms of Oxide -- but that we don't understand our system simply isn't one of them.

[0] https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-...

Re: The Cloud Computer

#822

Post by goldinfra »

mlindner wrote: post #671
goldinfra wrote: post #403

Earlier quoted context omitted.

Coupling vs Decoupling is not some one-sided thing. It's a major trade-off. One of the most obvious examples of the problem with this approach is that they're shipping previous generation servers on Day 1. One can easily buy current generation AMD servers from a number of vendors. They will also likely charge a significant premium over decoupled vendors that are forced to compete head-to-head for a specific role (ser…

> One of the most obvious examples of the problem with this approach is that they're shipping previous generation servers on Day 1. One can easily buy current generation AMD servers from a number of vendors. > Their coupling approach will most likely leave them perpetually behind This is a startup that took years to get their initial hardware developed. The time between this version and the version using the next ver…

Coupling requires more integration work, including writing and testing custom firmware. Oxide will be a tiny market player for a long time, even if things go very well. Are AMD and Broadcom really going to spend as much time helping Oxide as they do helping Dell? Of course not, Oxide's order volume will be a rounding error.

I'm sure they'll improve their processes over time but the lag will probably always be a non-zero value. Hopefully they'll be able to keep it low enough that it's not an important factor but as a customer it's certainly something one should consider.

It would be surprising if they don't run into some nasty issue that leaves their customers 6+ months behind on servers or switches at some point.

Re: The Cloud Computer

#823

Post by goldinfra »

SoftTalker wrote: post #772
xp84 wrote: post #743

Earlier quoted context omitted.

> will probably never be ideal for more savvy customers IDK about every use case, but slightly older generations of CPUs would affect me roughly zero. I'm sure there are things so compute-intensive that one would care very much, but a lot of people probably wouldn't bat an eye about that, and not because they're unsavvy. To the extent that these things are supported as a whole by the vendor rather than a bunch of fin…

Indeed, I'm still using a cluster of Haswell processors to run VMs for appropriate workloads and it's all fine.

If you're not rapidly scaling it probably doesn't matter. But if you're still buying (and maybe even using) Haswell CPUs in 2023, you may be missing out in a big way.

A moderately large Haswell cluster is equivalent in power to a moderately powerful modern server.

Re: The Cloud Computer

#824

Post by Kranar »

PTOB wrote: post #780
Kranar wrote: post #576

Earlier quoted context omitted.

Let's just say I hope I am interpreting something incorrectly, because if 500k is the minimum price and you match it to the minimum configuration found here: https://oxide.computer/product/specifications Then yeah, it's ridiculously expensive. That said compared to competitors it's in the right ballpark, but I have no idea how companies manage to spend so much money for this stuff. I am the founder of my own tech sta…

Tell me more!

Yeah sure, my process to learn was literally I went down to a local computer store, I bought a cheap desktop computer and 6 cheap hard drives, an HBA, a RAID controller, a bunch of cables, went back to the office and installed Ubuntu Server onto it and practiced several "skills", like how to setup automated backups using rsync, how to physically install these components, how to install mdadm for software RAID etc... comparing software RAID to using a RAID controller. I setup several drills for each task involving various types of failures and how to manage them to the point that it was part of muscle memory.

Some drills I practiced were setting up RAID 10 and on hardware failure having an email sent to me, so I went through the process of getting RAID 10 working using 4 of the drives, and then I would physically pull a hard drive out of the system as it was running to simulate a failure, and then I swapped in a new hard drive and ensured that the RAID rebuild process took place.

Once I was confident enough, I went to thinkmate.com and bought two JBOD expansion chassis each of which supported 78 drives and filled each of them up with 5 TB Seagate drives to get a total of 390 TB. The JBODs are managed by a 1U server running Ubuntu Server and software RAID using mdadm. I also bought 8 compute servers that were considered high density and could fit in 4U. I got a Cisco router for Internet and I networked everything within the data center together using a used Infiniband switch that I bought off of ebay. I also got a KVM so I could remotely access all of the systems. If there's one thing that I would change today, it would be to use ZFS.

I remember comparing the cost of doing this DIY setup to some other premade solutions like EMC and the cost was astronomical, like 3-4x what I ended up having to pay. I also even remember watching Linus Tech Tips and Level1Techs on Youtube and they both had good content about how they managed their storage that was fairly reasonable but still slightly on the pricey side, nevertheless I learned quite a bit from it.

At any rate, the bottom line is that it's not trivial to learn all this stuff by any means and I remember having some serious frustrations due to just how bad and demoralizing some error messages can be, but it's not thaaaat difficult either and in my situation my company is self funded, no venture capital or outside investors of any kind so every dollar my company spends is a dollar out of my own pocket. You better believe when I'm starting my business I'm not going to just blow 100s of thousands of dollars extra unless I absolutely have to.

About 5 years ago I managed to use my own storage system to ditch Dropbox in favor of Nextcloud, which is just leaps and bounds superior. I remember I got so frustrated with Dropbox because I wanted to just do something as simple as create a sub-directory and grant only read only permissions to some accounts. I also remember wanting to do some simple things like create a fake account with very limited permissions that could be used in our conference room for presentations or demos but the only way to do that with Dropbox would be to create an actual account and have to pay the full price for it.

Nextcloud works amazingly well, has all kinds of cool plugins, and gives our organization a lot of flexibility that Dropbox doesn't and I can make as many accounts as I feel like for whatever reason I feel like.

Re: The Cloud Computer

#825

Post by lulznews »

bigironcto wrote: post #241

I am CTO of a large global data center provider posting with throwaway account. As a technologist, I really appreciate what they have done. Impressive work, high quality, however I don't understand who this is for. The meaningful market for Data Center hardware is pretty well defined in two clusters. People that build/make custom gear (such as Hyperscalers) and people that buys HP/Cisco/IBM/Dell... (blades or hyper-c…

It’s an acquihire play. It might be possible to beat the hypers with $100 billion and some really good engineers. But building custom racks ain’t the angle to do it.

Re: The Cloud Computer

#826

Post by Fnoord »

dusted wrote: post #50

> core beliefs as a company: > Cloud computing is the future of all computing infrastructure. Oh god no! Anywhere I can donate to have this not happen? I want my computing on premise, preferably under my table. But well, if it _HAS_ to happen, 0xide is probably a lesser evil.

No worries, it is just their sales pitch. They'll co-exist.

Re: The Cloud Computer

#827

Post by eep_social »

bigironcto wrote: post #241

I am CTO of a large global data center provider posting with throwaway account. As a technologist, I really appreciate what they have done. Impressive work, high quality, however I don't understand who this is for. The meaningful market for Data Center hardware is pretty well defined in two clusters. People that build/make custom gear (such as Hyperscalers) and people that buys HP/Cisco/IBM/Dell... (blades or hyper-c…

I think you actually said it without recognizing.

The current state of the art is a fucking train wreck. Choose any layer of abstraction and start picking at it and you’ll find mostly gaffer tape. Scaling up sucks. Rewriting old monoliths to micro services wasn’t a panacea except maybe for cloud vendor profits. You said the hardware market is well defined, which is another way of saying ossified. Particularly when you start comparing it to the expected pace of software. How’s Open19 going? How much liquid cooling do you have in customer racks?

In my opinion this is ultimately a software offering that happens to come on vertically integrated hardware. It offers a complete, highly polished API to a minuscule-scale DC. If they can find market fit and make a little bit of money, the next step is to start making deep improvements to things behind the abstraction. From where you sit, you are well aware that you could make huge improvements inside the DC demarc if you could do it without disrupting customers. But you’re probably limited by the terrible, terrible APIs you would be forced to use, that don’t offer the capabilities you need, from vendors who would be happy to chat but would provide timeframes in the 6-18 month range for even a modest improvement.

So this, IMO, is about defining that interface to a DC on a single hardware platform with vertical integration, and then scaling up from there. The current hyperscalers will adopt the suite of APIs and capabilities directly, make lower quality copies, or die.

Of course all assuming they’re successful enough to get the revenue machine going in the first place which seems likely to me, given the absolute dogshit state of the cloud world today, the trend towards multi cloud, and the business case for moving certain loads back on prem or to a bare metal colo++ offering.

Re: The Cloud Computer

#828

Post by grappler »

bobthecowboy wrote: post #657
EvanAnderson wrote: post #169

Earlier quoted context omitted.

I'm talking shops who spend $200-$500K on servers and storage, not north of $1M (which is where this Oxide gear lives). Something like a 1/4 scale Oxide rack, perhaps.

I work at SoftIron, another startup in this space. Our HyperCloud product might be interesting for you. I'm not in sales, so I can't comment on the prices, but I'd guess we're much more competitive since you don't actually need to buy an entire rack of our gear at a time. That said, where this product-space gets tough is actually scaling it down. It's pretty challenging to create something that is remotely stable/fun…

This all has me wondering, if I just want to play with stuff in this space as an individual homelabber who earns a tech salary and wants a nicely designed rack-mounted alternative to a mess of unorganized NUCs and cables and whatnot, what are my best options?

Re: The Cloud Computer

#829

Post by bcantrill »

latchkey wrote: post #521
cryptonector wrote: post #459

Earlier quoted context omitted.

Bryan Cantrill is famously against vendor lock-in. He wrote a[n in]famous blog about the "FYO point" while at Sun. Oxide may be going for customers that also have the same aversion to vendor lock-in. One thing that Bryan understands is that you can "lock" the customer in with great products and services, as well as continuing development, while also making the customer feel secure in having a way out should you turn…

Oxide is the definition of vendor lock in. All of their hardware is unique... even down to the choice of fans. Fan burns out? Now you've got to buy another one... from them. One of the amazing shifts in the last 20 years was realizing that commodity hardware, when deployed correctly, could do the job.

Not sure if we're talking metaphorical fans or literal ones but assuming the latter: replacement is covered under warranty. And for me personally, the "amazing shift" that you describe was in fact a decade-long experiment that left me with an inescapable conclusion: commodity hardware cannot, in fact, do the job -- and not for lack of trying!

Re: The Cloud Computer

#830

Post by fuzzylightbulb »

cryptonector wrote: post #511
cpursley wrote: post #3

Can somebody explain in a sentence or two what this actually is and what the benefits are?

Low risk of vendor lock-in.

This thing is the definition of vendor lock in. It's a rack full of non-commodity hardware using non-standard connectors and a bunch of custom software.

The industry moved to commodity x86 for compute for a reason.

Post reply on HN