Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Libcu++: Nvidia C++ Standard Library

github.com

Post reply on HN
Print view View original

101–110 of 139 posts

Re: Libcu++: Nvidia C++ Standard Library

#101

Post by blelbach »

einpoklum wrote: post #74

1. How do we know what parts of the library are usable on CUDA devices, and which are only usable in host-side code? 2. How compatible is this with libstdc++ and/or libcu++, when used independently? I'm somewhat suspicious of the presumption of us using NVIDIA's version of the standard library for our host-side work. Finally, I'm not sure that, for device-side work, libc++ is a better base to start off of than, say,…

> apparently only a small bit of the library is actually implemented.

Yep. It's an incremental project. But stay tuned.

> I'm somewhat suspicious of the presumption of us using NVIDIA's version of the standard library for our host-side work.

Today, when using libcu++ with NVCC, it's opt-in and doesn't interfere with your host standard library.

I get your concern, but a lot of the restrictions of today's GPU toolchains comes from the desire to continue using your host toolchain of choice.

Our other compiler, NVC++, is a unified stack; there is no host compiler. Yes, that takes away some user control, but it lets us build things we couldn't build otherwise. The same logic applies for the standard library.

https://developer.nvidia.com/blog/accelerating-standard-c-wi...

> Finally, I'm not sure that, for device-side work, libc++ is a better base to start off of than, say, EASTL (which I used for my tuple class: https://github.com/eyalroz/cuda-kat/blob/master/src/kat/tupl... ).

We wanted an implementation that intended to conform to the standard and had deployment experience with a major C++ implementation. EASTL doesn't have that, so it never entered our consideration; perhaps we should have looked at it, though.

At the time we started this project, Microsoft's Standard Library wasn't open source. Our choices were libstdc++ or libc++. We immediately ruled libstdc++ out; GPL licensing wouldn't work for us, especially as we knew this project had to exchange code with some of our other existing libraries that are under Apache- or MIT-style licenses (Thrust, CUB, RAPIDS).

So, our options were pretty clear; build it from scratch, or use libc++. I have a strict policy of strategic laziness, so we went with libc++.

Re: Libcu++: Nvidia C++ Standard Library

#102

Post by einpoklum »

pjmlp wrote: post #79
einpoklum wrote: post #75

Earlier quoted context omitted.

1. printf-style debugging is what we use on NVIDIA hardware too. 2. OpenCL 2.x allows for C++(ish) source code. Not sure how good the AMD support is though.

1. Ever heard of Nights and Visual Studio plugins? 2. OpenCL 2.0 was a failure, so OpenCL 1.2 got renamed as OpenCL 3.0. C++ bindings were dropped and SYSCL is now backend agnostic.

> 1. Ever heard of Nights and Visual Studio plugins?

Those are apples and oranges... also, you forget cuda-gdb.

> OpenCL 1.2 got renamed as OpenCL 3.0. C++ bindings were dropped

Well, yes, but also no. They were made optional, and transitioned to some other C++-cum-OpenCL initiative:

https://github.com/KhronosGroup/Khronosdotorg/blob/master/ap...

I'm not exactly sure how this differs and what's usable in practice though.

Re: Libcu++: Nvidia C++ Standard Library

#103

Post by gj_78 »

blelbach wrote: post #91
gj_78 wrote: post #26

Earlier quoted context omitted.

Agree++. They are good at hardware and should stay that way.

We employ more software engineers than hardware engineers. Our hardware doesn't really do much in isolation, software is part of the product.

The question is not about the head count. How many software engineers at nvidia produce software that is expected run/compile on the host CPU of the customer, like this library ? I expect not too much.

Re: Libcu++: Nvidia C++ Standard Library

#104

Post by blelbach »

gj_78 wrote: post #61
dahart wrote: post #50

Earlier quoted context omitted.

You can use this library to write code that runs on both risc-v and a GPU! You seem to be pretty confused about what this library is. It’s not exerting any control. It’s open source! It’s strictly optional, and it only allows developers to do something they actually want, to write code that will compile for any type of processor that a modern c++ compiler can target.

Again, I see what you mean. I am even against nvidia advising the developers to use such or such C++ library (be it GNU). It is not their role to do that. We need smarter and more shining GPUs from nvidia, not software. I would say .... The hardware must be sold independently of the software ... but it is a bit too complex, I know.

> It is not their role to do that.

You are incorrect.

NVIDIA employs more software engineers than hardware engineers.

> We need smarter and more shining GPUs from nvidia, not software.

Software is a part of the GPU. You get better GPUs by having hardware and software engineers collaborate together.

It is extremely expensive to put features into hardware. It costs a lot of money and takes a very long time. It takes 2-4 years at a minimum to put features into hardware. And there are physical constraints; we only have so many transistors.

If we make a mistake in hardware, how are we supposed to fix it? At NVIDIA we have a status for hardware bugs called "Fix in Next Chip". The "Next Chip" is 2-4 years away.

So what do we do? We solve problems in software whenever possible. It's cheaper to do so, it has a quicker turnaround time, and most importantly, we can make changes after the product has shipped.

> I would say .... The hardware must be sold independently of the software ... but it is a bit too complex, I know.

We don't sell hardware and you don't want to buy hardware. Trust me, you wouldn't know what to do with it. It's full of bugs and complexity.

We sell a platform that consists of hardware and software. The product doesn't work without software.

If we tried to make the same product purely in hardware, the die would be the size of your laptop and would cost a million dollars.

Re: Libcu++: Nvidia C++ Standard Library

#105

Post by blelbach »

gj_78 wrote: post #48
dahart wrote: post #45

Earlier quoted context omitted.

What do you mean about running in the same memory space? Your operating system doesn’t allow that. Is your concern about using host memory? This open source library doesn’t automatically use host memory, users of the library can write code that uses host memory, if they choose to. How would a firmware help me write heterogeneous bits of c++ code that can run on either cpu or gpu?

IMHO, the question is not that we need code to run on CPUs and GPUs , we do need that, The question is whether the GPU seller has to control both sides. Until I buy a CPU from nvidia I want to keep some kind of independence. When will we be able to use a future riscv-64 CPU with an nvidia GPU ? we will let the answer to nvidia ?

> IMHO, the question is not that we need code to run on CPUs and GPUs , we do need that, The question is whether the GPU seller has to control both sides.

The question is not about running code on CPUs, or running code on GPUs. It's about running code on both CPUs and GPUs at the same time. It's about enabling the code on the CPU and the code on the GPU to seamlessly interoperate with each other, communicate with each other, move objects and data to and from each other.

Who do you expect to make that happen?

> Until I buy a CPU from nvidia I want to keep some kind of independence

You can buy a CPU from NVIDIA, check out our Tegra systems. We also sell full systems, like DGX platforms, which use a 3rd party CPU.

> When will we be able to use a future riscv-64 CPU with an nvidia GPU ? we will let the answer to nvidia ?

Who else would answer this question?

Okay, you want to use with our GPU.

Who is going to design and build the interconnect between the CPU and the GPU?

Who is going to provide the GPU driver?

The CPU manufacturer? Why would they do that? They don't make any money from selling NVIDIA products. Why should they invest effort in enabling that?

Re: Libcu++: Nvidia C++ Standard Library

#106

Post by blelbach »

gj_78 wrote: post #71
dahart wrote: post #70

Earlier quoted context omitted.

I'm not understanding your point at all. You don't think developers should be able to write C++ code for the GPU? What do you even mean about 'it is not their role to do that.' and 'hardware must be sold independently of the software'?? Why are you saying this? Software interfaces are critical for all GPUs and all CPUs, just ask AMD & Intel. There is no such thing as CPU or GPU hardware independent of software. Plus,…

> You don't think developers should be able to write C++ code for the GPU? To be clear, I don't think nvidia-paid developers should be able to write C++ Code for a nvidia-sold GPU. The world will be better if any developer (paid by nivida or not) is able to write code for any GPU (sold by nvidia or not). It is not nvidia role to say how or when software will be written. Their hardware is good and that's more than OK.…

> To be clear, I don't think nvidia-paid developers should be able to write C++ Code for a nvidia-sold GPU.

I'm not sure what you're saying here? You think another company or organization should write all the software for our hardware?

I don't think you understand the semiconductor industry.

Our business model relies on hardware and software engineers working closely together, as I've described in other replies.

We would not be able to produce a viable product that is solely raw hardware.

Also, what motivation does this other organization or company have to create software for our hardware?

> The world will be better if any developer (paid by nivida or not) is able to write code for any GPU (sold by nvidia or not).

This library is something that is designed to help you write Standard C++ code that runs on our GPU. Standard C++ runs everywhere.

> It is not nvidia role to say how or when software will be written.

Providing the SDKs and toolchains to program our platform is definitely part of our role in the ecosystem.

> Their hardware is good and that's more than OK.

Our hardware is useless without our software.

> AI/CUDA code written specifically for nvidia is useless/deprecated in the long term. A lot of brain waste.

I expect CUDA will be around for a while.

Re: Libcu++: Nvidia C++ Standard Library

#107

Post by blelbach »

gj_78 wrote: post #36
my123 wrote: post #30

Earlier quoted context omitted.

The thing is: that hardware isn't very usable without good software, and an easy to use software stack at that. That's what NVIDIA understood and made them what they are today.

A lot of hardware has builtin software, either inside a firmware or as a driver. Keeping the software part in firmware lets customer free to use any kind of OS. Using host cpu and memory is bad design IMHO.

> A lot of hardware has builtin software, either inside a firmware or as a driver.

Correct.

> Keeping the software part in firmware lets customer free to use any kind of OS.

Do you mean firmware, or firmware and driver?

You can't do everything in firmware.

> Using host cpu and memory is bad design IMHO.

How do you propose that you program the GPU then?

The CPU has to interact with the GPU. Some software has to manage that interaction.

That said, we are not talking about either a driver or firmware. This is a part of our toolchain. It is a library that you use when writing a heterogeneous program.

Re: Libcu++: Nvidia C++ Standard Library

#108

Post by blelbach »

gj_78 wrote: post #73
dahart wrote: post #69

Earlier quoted context omitted.

This is gcc or whatever compiler, it is not nvidia's compiler. This library does not give nvidia any "control" over host operations, it gives developers another tool. They did publish the specs, it's open source . BTW, Nvidia's acquisition of ARM means that it will be selling CPUs. P.P.S., the driver runs on the host, so your proposed alternative doesn't address the point you think you're making.

I did not say the library controls anything, Nvidia controls the library : its features, its roadmap, its bugs corrections, development efforts (people) etc. All these choices are made by Nvidia. It is not just another tool , it is the tool that is closest to hardware evolution. Nvidia buying ARM is not a good news for me. The same way I don't like them making software, I also don't like them selling CPUs or seafood.…

> I did not say the library controls anything, Nvidia controls the library : its features, its roadmap, its bugs corrections, development efforts (people) etc. All these choices are made by Nvidia.

libcu++ is a fork of LLVM's libc++, which we do not control. We contribute upstream and engage with that community.

libcu++ is an implementation of the C++ Standard Library, which is controlled by an ISO standardization committee, which has about ~300 members, 10 of whom work at NVIDIA.

> The same way I don't like them making software, I also don't like them selling CPUs or seafood. They are good at GPUs and that's OK.

We employ more software engineers than hardware engineers. We don't sell hardware, we sell software + hardware.

We manufacture and sell CPUs today. I like to think we're quite good at it.

> The drivers are usually running in the kernel space and do not involve much of interaction with users.

Incorrect. The core part of the driver, called the Resource Manager (RM), runs in the kernel space. Each different SDK (CUDA, OpenGL, Vulkan, etc) has its own "user mode driver", which is a shared library that interacts with RM. It's hard to say what the split is, but I'd say roughly half of what you think of as "the driver" is in user mode.

> Firmware, on the other side, is hardware-close software and can be gradually replaced by specific hardware continuous improvements without the user/software or the OS noticing.

Firmware runs on the GPU. You can't do everything from the GPU.

But neither firmware nor drivers have anything to do with the toolchain that you use to write heterogeneous programs. That's what this is a part of.

Re: Libcu++: Nvidia C++ Standard Library

#109

Post by blelbach »

gj_78 wrote: post #103
blelbach wrote: post #91

Earlier quoted context omitted.

We employ more software engineers than hardware engineers. Our hardware doesn't really do much in isolation, software is part of the product.

The question is not about the head count. How many software engineers at nvidia produce software that is expected run/compile on the host CPU of the customer, like this library ? I expect not too much.

The majority of software engineers at NVIDIA write software that runs on the host CPU.

The majority of software written at NVIDIA (by any metric, lines of code, number of projects, etc) runs either solely on the CPU, or on both the CPU and the GPU.

Re: Libcu++: Nvidia C++ Standard Library

#110

Post by blelbach »

shaklee3 wrote: post #28
Mr_lavos wrote: post #11

Does this mean you can do operations on struct's that live on the GPU hardware?

You have been able to do that for a long time with UVA.

Since Unified Memory. UVA, or Unified Virtual Addressing, just ensured that a GPU-private object wouldn't have the same address as a CPU-private object.
Post reply on HN