Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Add 10 GbE to your system with an M.2 2280 module

cnx-software.com

Post reply on HN
Print view View original

141–150 of 259 posts

Re: Add 10 GbE to your system with an M.2 2280 module

#141

Post by jabl »

tomohawk wrote: post #73
dtech wrote: post #64

Earlier quoted context omitted.

I've noticed 2.5 becoming a bit more common on enthusiast hardware, so it'll be a while yet before 10G becomes mainstream, but 2.5 and 5 might be the standard for new hardware a decade from now.

2.5 and 5 can use existing cat5e or better cabling. There is no solution for 10GigE that uses that cabling.

10Gbase-T does 55m with Cat6, or the full 100m with Cat6A, both of which are ubiquitous and cheap. It's probably more to do with the expense and power consumption of 10Gbase-T PHY's.

Re: Add 10 GbE to your system with an M.2 2280 module

#142

Post by _flux »

kortilla wrote: post #108
jiggawatts wrote: post #80

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

A lot of that sounds pretty incredibly wasteful and I don’t think it should be something to strive for more of. Docker images are layered for a reason. If you’re having to download multi gigabyte docker layers multiple times, something has gone very wrong. Azure charges (like the other major cloud providers) by egress data. I feel bad for whoever is paying the bill for 20gbps going out. In general, people in ML shoul…

> Docker images are layered for a reason. If you’re having to download multi gigabyte docker layers multiple times, something has gone very wrong.

Well the images need to be in some order, and if you are fine tuning the very initial parts of the image, this can happen.

But it seems a better transfer algorithm could be used for those individual layers. It rather seems like if they don't match, everything's transferred, while there could be a lot better options as well. If we consider that they are structured like file systems then it opens even more options.

---

So I took a peek on my local Docker installation. It seems the layer files on the drive are gzipped json.

That's right, .json.gz, _that includes the files BASE64-encoded_.

Yes, I believe there is room for improvement, if this is also the format they are transferred in.

Re: Add 10 GbE to your system with an M.2 2280 module

#143

Post by ckdarby »

ClumsyPilot wrote: post #122
kortilla wrote: post #108

Earlier quoted context omitted.

A lot of that sounds pretty incredibly wasteful and I don’t think it should be something to strive for more of. Docker images are layered for a reason. If you’re having to download multi gigabyte docker layers multiple times, something has gone very wrong. Azure charges (like the other major cloud providers) by egress data. I feel bad for whoever is paying the bill for 20gbps going out. In general, people in ML shoul…

"You should be able to work from a stable coffee shop WiFi connection." I think there are different philosophies, just like 'no comments allowed in code', this is a matter of judgement. I saw 5 different corporations setup centralised workflow, and usually you end up with a massive cloud bill, incorrectly configured admin rights, under-resourced VDI, and a ticketing system that takes a week to resolve the smallest is…

5? Are you a consultant who goes in to fix these situations?

Re: Add 10 GbE to your system with an M.2 2280 module

#144

Post by l30n4da5 »

ckdarby wrote: post #138
jiggawatts wrote: post #80

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

How do you regularly pull 50-200 GB? This sounds like terrible practices or implementation at work. I've never seen docker images of that size that can't be shrunk with multistage, or docker shrink. I can see how someone could get to that size with just baking in all their models to the image itself but yuck.

windows docker images are incredibly bloated. I don't think I've seen one that size, but the base image for a windows container is like 17GB by itself, if I remember correctly. Basically just pulling down an entire Hyper-V image.

Re: Add 10 GbE to your system with an M.2 2280 module

#145

Post by jjoonathan »

YXNjaGVyZWdlbgo wrote: post #99
pdimitar wrote: post #93

Earlier quoted context omitted.

In my case I'd like to transfer from/to my NAS where I centralize a number of data pieces. I work inside a workstation but occasionally on a laptop as well. Having 10GbE will help me when I'm on my workstation. It's not fatal, definitely, but it adds up to lost productivity.

Is your NAS NVME based if you are using spinning metal you won't reach speeds to saturate 10GbE especially with small file sizes?

If somebody knows why they should pay a premium for 10GbE, they also know how bad spinning rust is.

Re: Add 10 GbE to your system with an M.2 2280 module

#146

Post by jcelerier »

jiggawatts wrote: post #80

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

> As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port.

same, I live in a rural village in france where I get 2GB fiber. Yet all my computers aren't able to leverage that.. quite frustrating to see speeds being limited at 110/120 megabytes/s where my drives are easily able to handle gigabytes.

Re: Add 10 GbE to your system with an M.2 2280 module

#147

Post by kkielhofner »

junon wrote: post #115
kortilla wrote: post #108

Earlier quoted context omitted.

A lot of that sounds pretty incredibly wasteful and I don’t think it should be something to strive for more of. Docker images are layered for a reason. If you’re having to download multi gigabyte docker layers multiple times, something has gone very wrong. Azure charges (like the other major cloud providers) by egress data. I feel bad for whoever is paying the bill for 20gbps going out. In general, people in ML shoul…

> If you’re having to download multi gigabyte docker layers multiple times, something has gone very wrong When I worked at Vercel during the days we supported Docker hosting (back when it was still ZEIT), we saw that even among many tenants on the same machine, the "layers" thing wasn't really that beneficial - we still had to fetch a ton of stuff over and over again. So this is great in theory, but it doesn't really…

I believe Docker has even throttled/blocked corporate users doing excessive layer pulls from Hub (and suggested they sign up for X plan).

Reasonable, IMHO.

Re: Add 10 GbE to your system with an M.2 2280 module

#148

Post by kkielhofner »

PragmaticPulp wrote: post #140
jiggawatts wrote: post #80

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

As a counterpoint: I had 10G networking and went back to 1G last time I reorganized. Haven’t really missed the 10G for now . I ditched the 10G because the 10G switch was hot and noisy and had a fan, whereas my 1G equipment was silent and low power. This could be overcome by putting the switch in another room, obviously, but I haven’t missed it enough to go through the trouble yet. > During the 2 years of the pandemic…

This is exactly where I ended up recently. I went from noisy and power hungry Cisco/Juniper 10G switches to a silent Ubiquiti GigE switch with PoE and haven’t been happier.

10G was cool but I can’t recall a single instance where I really wish I had it back.

Re: Add 10 GbE to your system with an M.2 2280 module

#149

Post by ericd »

Hamuko wrote: post #86
ericd wrote: post #77

Earlier quoted context omitted.

I'd say it's worth it as soon as you have a home NAS, it lets you treat it almost as a local drive for any computer that also has 10G.

I have a home NAS, but I think I need at least two, maybe three 10G switches in order to get everything hooked up properly. And then I need gear to actually get 10G on my computers. Sounds a bit expensive especially since the NAS is unfortunately limited to 2x1GbE.

Yeah, I’ve been a bit surprised at how few of the home tier NASes can get 10G. A bit of a chicken and egg, I guess.

You don’t need to go fully 10G, though, I mainly prioritized my workstation for example, and if you don’t want to splash out for switches, you can do a direct connection on 10G to a computer that needs it, and use the 1G links to the rest of the LAN.

Re: Add 10 GbE to your system with an M.2 2280 module

#150

Post by kkielhofner »

manuel_w wrote: post #44

A bit unrelated but still: Can someone recommend quality USB(-C) ethernet adapters brand? I'm building an embedded system in a professional context and need to connect some of our own custom in-house built embedded devices (usb-c only, no ethernet) to LAN. Right now, when some device goes offline, I don't know wether it's our product or the cheapish usb-ethernet-adapter which is at fault. Would like to have something…

I’d be more concerned with the chip the adapter uses. While I don’t have experience with their USB-C variants I’ve had good experiences otherwise with Axis.

As far as I can tell they seem to be the “go-to” in the USB-Ethernet game and are well supported on Linux and anything else I’ve used them on.

Post reply on HN