Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geocities Cage at the Exodus Datacenter

detritus.org

Post reply on HN
Print view View original

61–70 of 89 posts

Re: Geocities Cage at the Exodus Datacenter

#61

Post by vidarh »

acveilleux wrote: post #8

So much memories from that era. I saw my first 1TB storage array back then, it was an EMC Clariion fibre channel raid box, it cost nearly 1m$ and it was a full rack of 9GB dual-ported 15k RPM disks.

We had an IBM ESS "Shark" back then. A fantastic piece of kit but totally overkill for us in the end. It was basically the size of two or three standard racks, full of shelves for drives, and two RS/6000 AIX servers, and trays of redundant controllers.

You could pull trays of drives, one of the RS/6000's, or any of the controllers, and it'd keep on humming along thanks to redundancy pretty much everywhere. If any components went bad it'd call home to IBM via a modem.

And it could do things like automatic mirroring to a remote site.

1.5TB was a common configuration, but you could connect multiple boxes to increase capacity.

Re: Geocities Cage at the Exodus Datacenter

#62

Post by vidarh »

freeone3000 wrote: post #17
biofox wrote: post #13

Earlier quoted context omitted.

Which makes me wonder: what would it take to match the capabilities of this setup with modern hardware?

Could be done with a moderate-spec single server. Replace the JBOD enclosures with on-server 8xSamsung EVOs in hardware RAID6, and gain a bit in speed because we don't have drives that slow anymore. Xeon E5v4 could replace the computational power. Network multihoming would be done the same, except with 10000baseT instead of multiple cables. Run nginx on it and you're good. Or, one could host the entire thing on S3 wi…

> Or, one could host the entire thing on S3 with CloudFront at a fraction of the cost.

Or, you could rent capacity somewhere with decent bandwidth prices and pay 2%-20% of CloudFront/S3 bandwidth prices...

Re: Geocities Cage at the Exodus Datacenter

#63

Post by vidarh »

jsjohnst wrote: post #45
acveilleux wrote: post #23

Earlier quoted context omitted.

A few decent things I could find online that jive with what I remember: https://www.patton.com/technotes/build_yourself_an_isp.pdf http://www.gwi.net/behind-the-scenes-of-a-90s-internet-start... https://news.ycombinator.com/item?id=8352432 The absolute minimum, and representative of the very first dial-up ISPs: http://www.linuxjournal.com/article/2025 http://www.datamation.com/erp/article.php/615281/My-own-priv... Th…

In my smaller regional ISP we started off using USR courier and sportster modems with Cyclades cards before moving over to Livingston Port masters. I ended up selling the company before moving to all digital incoming lines (needed to support 56k modems).

Did exactly the same.... We had sporters hanging off the wall for a while (couldn't pack them tightly due to the crappy ventilation on the Sporters, but they were light, so clamping the cables to a suitable plate and fitting it to the wall worked great....)

The local telco had problems supplying enough lines from the nearest exchange, so they ended up hanging a thick cable bundle in the trees for several hundred meters, make a hole in one of our windows and put a large multiplexer cabinet in our office... Then three months later we moved to different offices - they were not pleased.

Re: Geocities Cage at the Exodus Datacenter

#64

Post by vidarh »

halbritt wrote: post #31
acveilleux wrote: post #23

Earlier quoted context omitted.

A few decent things I could find online that jive with what I remember: https://www.patton.com/technotes/build_yourself_an_isp.pdf http://www.gwi.net/behind-the-scenes-of-a-90s-internet-start... https://news.ycombinator.com/item?id=8352432 The absolute minimum, and representative of the very first dial-up ISPs: http://www.linuxjournal.com/article/2025 http://www.datamation.com/erp/article.php/615281/My-own-priv... Th…

Lots of people used the US Robotics Total Control boxes in the early 90s. http://www.kmj.com/tcont.html

They were also way too expensive for a lot of smaller ISPs.

Re: Geocities Cage at the Exodus Datacenter

#65

Post by mitchty »

acveilleux wrote: post #34
roganp wrote: post #21

Earlier quoted context omitted.

Veritas was definitely better than the alternative. I was a dba at the time and it was a godsend.

I still can't believe a top-tier UNIX vendor required an expensive 3rd party software to have halfway decent file-system. Or in the case of HP-UX, partitioning. I get that Veritas also did logical volume, raid and so on and that was a guenuine value added at the time. But so many box I saw had veritas so they could partition and get a file system that was OK with 36GB+ monster SCSI disks.

From having to use it on solaris, it was more the flexibility of resizing and changing the layout without downtime that was useful.

When you have a 100k+ machine it makes more sense, when you can just rebuild a new instance it makes less sense. I do NOT miss using it, but it was nice for its niche.

Re: Geocities Cage at the Exodus Datacenter

#66

Post by SSLy »

setq wrote: post #30
tyingq wrote: post #24

Earlier quoted context omitted.

>>Must have cost as much as a house when it was new. I do remember some $100k - $300k invoices for larger SMP servers from HP, Sun, and the like. For machines that probably had less overall horsepower than my current cell phone :)

Yeah pricing was awful. I remember someone playing £12k for a single PA RISC CPU option and when they cracked it open to have a look it was 95% heatsink. Cue the "bloody expensive heatsink" comments. It had about as much go as one of those "big slab" xeon slot CPUs at the time which was 1/10th of the cost.

Well, I've seen appliances over $50k a piece ( you need at least two to be useful.) So systems like that are still here.

Re: Geocities Cage at the Exodus Datacenter

#67

Post by technofiend »

beachstartup wrote: post #55
alexhawdon wrote: post #42

Earlier quoted context omitted.

That's a great story. It'd be interesting to know whether that was all they could afford, or if they were being very forward-thinking in using COTS hardware and a failure-tolerant architecture.

probably a combination of both. i worked at a startup in the late 90's and the price of sun microsystems gear was mind-blowingly high. from memory: ultra 5 workstation with a scsi card and disk array was $15,000+. an ultra e450 server was on the order of $100,000 and went up from there depending on how you wanted to build. of course, that was exactly the time people started switching to linux on x86 en masse. pentium…

I was the technical architect for an internet-based commodity trading system that was also hosted at Exodus in Austin before the .com bust. As I recall we spent $6 million on our two Sun clusters "Coke" and "Pepsi" but that was almost 15 years ago so they might have been $6 million each. Then the bottom fell out of energy trading thanks to Enron and others and that was that. Those systems were probably sold for pennies on the dollar and would easily be replaced today with a small AWS presence for even less.

Frankly that's part of what makes working now such fun: the fact these systems just keep getting better and better.

Re: Geocities Cage at the Exodus Datacenter

#68

Post by technofiend »

kyledrake wrote: post #57
beachstartup wrote: post #55

Earlier quoted context omitted.

probably a combination of both. i worked at a startup in the late 90's and the price of sun microsystems gear was mind-blowingly high. from memory: ultra 5 workstation with a scsi card and disk array was $15,000+. an ultra e450 server was on the order of $100,000 and went up from there depending on how you wanted to build. of course, that was exactly the time people started switching to linux on x86 en masse. pentium…

I got the impression "hardware dogma" still happens when I was planning our infrastructure. The datacenter world is still a place where you can spend $36k for something you can get for $1000 if you're not careful. I didn't use Cisco for our switches and some IT guys I talked to acted like I was a heretic. But the decision saved us several thousand dollars and we ended up with a design that was IMHO more reliable than…

Unfortunately storage dogma still seems to be a thing for EMC shops. Just try suggesting Ceph over Scale I/O. :-)

Re: Geocities Cage at the Exodus Datacenter

#69

Post by jsjohnst »

aidenn0 wrote: post #51
jsjohnst wrote: post #49

Earlier quoted context omitted.

Your edit mostly got it right with the exception that it wasn't commonly called a T-1 or E-1 and the numbers are slightly off. * ISDN BRI = two 64k B channels and one 16k D channel * ISDN PRI over T1 = twenty three 64K B channels and one 64k D channel * ISDN PRI over E1 = thirty 64k B channels and one 64k D channel (time slot 0 on the E1 was used as a sync channel and wasn't considered a B or D) T1/E1 was considered…

In the 90s in my little corner of the mid Atlantic US at least, it was very commonly called a T1 or DS1 and the terms were used interchangeably even though they refer to slightly different things (at least by sysadmins; use the terms wrong around a telecom equipment engineer and you'd likely get corrected).

I'm aware the misnaming was common, but it's a bad idea to do so. There's a pretty big difference between a T1 and a PRI in practice when used as a data only circuit. For one, the T-1 is a fair bit faster.

Re: Geocities Cage at the Exodus Datacenter

#70

Post by icedchai »

Overtonwindow wrote: post #7

Does anyone know of an article like this about the dial in modem systems and infrastructure for some of the early dialup services?

I worked at a couple of small ISPs in the mid 90's, and built the dialup infrastructure of one from scratch. I remember using Xylogics Annex terminal servers connected to either 16 or 32 individual USR modems. There were stacks of modems, each with its own power brick, serial, and RJ-11 connection.

This was a few years before everything went digital with PRI lines (T1s that let you do digital modems, basically.)

Post reply on HN