Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Embedded Ring Buffer C++98

github.com

Post reply on HN
Print view View original

31–40 of 41 posts

Re: Show HN: Embedded Ring Buffer C++98

#31

Post by ncmncm »

tialaramex wrote: post #30
klodolph wrote: post #13

Earlier quoted context omitted.

The use of volatile is typical here. It allows the ring buffer to be used from interrupts, as long as you have one reader and one writer at a time. I haven't checked the code for correctness, but in a typical ring buffer implementation intended to be used in interrupts, you would make the read and write pos volatile. To write, you put the value in the array, and then advance the write position. To read, you copy a va…

> The use of volatile is typical here. This is definitely true. C++ programmers typically do this. > It allows the ring buffer to be used from interrupts Unfortunately "allows" here is telling us about the programmers not the hardware. The programmers see this and figure eh, I don't really understand this but somebody wrote "volatile" so I guess they knew what they were doing. > Volatile ensures that if a read is int…

Right. Another reason not to confine ourselves to C++98.

While the abstract machine is not allowed to reorder volatile writes, the compiler is NOT obliged to emit instructions forcing the actual hardware not to reorder writes. Thus, regular cache behavior can turn your carefully ordered sequence of volatile writes into a bunch of local cache operations followed by a single writeback bus transaction.

If you are coding to a microcontroller, its cache hardware might be simple enough that this can't happen. Or, you might be able (and need!) to initialize a memory controller, at startup, to give a chosen memory address range simpler write semantics, e.g. "write-through".

But atomics are the cleanest way to express things at the source level.

Re: Show HN: Embedded Ring Buffer C++98

#32

Post by ncmncm »

gpderetta wrote: post #15
klodolph wrote: post #13

Earlier quoted context omitted.

The use of volatile is typical here. It allows the ring buffer to be used from interrupts, as long as you have one reader and one writer at a time. I haven't checked the code for correctness, but in a typical ring buffer implementation intended to be used in interrupts, you would make the read and write pos volatile. To write, you put the value in the array, and then advance the write position. To read, you copy a va…

Unfortunately the compiler can still reorder nonvolatile operations across volatile ones, so unless the buffer is also volatile, it is not as useful as one would expect.

Right. Volatile very rarely means what peoole using it imagine it means. It very much does not mean Do What I Mean.

E.g. compilers routinely elide writes to stack variables declared volatile—or even atomic—that they "know" are not aliased. So e.g. if an interrupt routine might look at things in your stack frame, you need to use asm to force it not to fool with writes there. Volatile and atomic don't help you, there, because the stack frame is special.

LTO builds can expose what had been invisible operations in other TUs (".o" files) to the optimizer. So that might demand especial care.

Re: Show HN: Embedded Ring Buffer C++98

#33

Post by gpderetta »

ncmncm wrote: post #32
gpderetta wrote: post #15

Earlier quoted context omitted.

Unfortunately the compiler can still reorder nonvolatile operations across volatile ones, so unless the buffer is also volatile, it is not as useful as one would expect.

Right. Volatile very rarely means what peoole using it imagine it means. It very much does not mean Do What I Mean. E.g. compilers routinely elide writes to stack variables declared volatile— or even atomic —that they "know" are not aliased. So e.g. if an interrupt routine might look at things in your stack frame, you need to use asm to force it not to fool with writes there. Volatile and atomic don't help you, there…

To be pedantic, technically they can't elide, reorder or coalesce a accesses to volatile objects any more they can do any other form of I/O, even if it is a local variable whose address is never taken.

Of course how accesses on the abstract machine maps to the actual hardware is implementation defined, although most will document to translate them to plain movs.

And implementations have bugs of course; here [1] gcc removing a volatile access to an otherwise unused volatile parameter is considered a bug, even when the parameter is actually passed via register (in this case I would say the standard is underspecified).

You are absolutely correct regarding non-volatile atomics though.

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71793

Re: Show HN: Embedded Ring Buffer C++98

#34

Post by ncmncm »

gpderetta wrote: post #33
ncmncm wrote: post #32

Earlier quoted context omitted.

Right. Volatile very rarely means what peoole using it imagine it means. It very much does not mean Do What I Mean. E.g. compilers routinely elide writes to stack variables declared volatile— or even atomic —that they "know" are not aliased. So e.g. if an interrupt routine might look at things in your stack frame, you need to use asm to force it not to fool with writes there. Volatile and atomic don't help you, there…

To be pedantic, technically they can't elide, reorder or coalesce a accesses to volatile objects any more they can do any other form of I/O, even if it is a local variable whose address is never taken. Of course how accesses on the abstract machine maps to the actual hardware is implementation defined, although most will document to translate them to plain movs. And implementations have bugs of course; here [1] gcc r…

It is notable that this report is against gcc-5, not assigned, and has not been looked at in five years.

There certainly are people who think volatile should mean something for stack variables, but the compiler people very much Do Not Care. So, by the technical wording of Standards, volatile means the same for all variables, in actual compilers (with certain exceptions) it does not.

Re: Show HN: Embedded Ring Buffer C++98

#35

Post by gpderetta »

ncmncm wrote: post #34
gpderetta wrote: post #33

Earlier quoted context omitted.

To be pedantic, technically they can't elide, reorder or coalesce a accesses to volatile objects any more they can do any other form of I/O, even if it is a local variable whose address is never taken. Of course how accesses on the abstract machine maps to the actual hardware is implementation defined, although most will document to translate them to plain movs. And implementations have bugs of course; here [1] gcc r…

It is notable that this report is against gcc-5, not assigned, and has not been looked at in five years. There certainly are people who think volatile should mean something for stack variables, but the compiler people very much Do Not Care. So, by the technical wording of Standards, volatile means the same for all variables, in actual compilers (with certain exceptions) it does not.

Well, Richard Biener certainly count as a compiler person (they are heavily involved with the GCC middle end), and they seem to care, although not enough to fix it as the issue if fairly academical in this case.

Re: Show HN: Embedded Ring Buffer C++98

#36

Post by gpderetta »

ghhhhhk8899jj wrote: post #23

volatile does not imply compiler fence on gcc, so this code has a race condition when updating read/write_position, you need compiler barriers here >data[write_position] = value; >COMPILER_BARRIER(); // __asm__ volatile("":::"memory"); >write_position = (write_position + 1U) % LENGTH; and same in Skip()

exactly. The fence is required even when communicating with a signal handler (or interrupt handler) on the same thread. For for the multithreaded case, depending on the architecture, the compiler barrier is likely not sufficient and an actual hardware #StoreStore fence might be required.

A specular barrier is needed on the reader side.

Re: Show HN: Embedded Ring Buffer C++98

#37

Post by klodolph »

tialaramex wrote: post #30
klodolph wrote: post #13

Earlier quoted context omitted.

The use of volatile is typical here. It allows the ring buffer to be used from interrupts, as long as you have one reader and one writer at a time. I haven't checked the code for correctness, but in a typical ring buffer implementation intended to be used in interrupts, you would make the read and write pos volatile. To write, you put the value in the array, and then advance the write position. To read, you copy a va…

> The use of volatile is typical here. This is definitely true. C++ programmers typically do this. > It allows the ring buffer to be used from interrupts Unfortunately "allows" here is telling us about the programmers not the hardware. The programmers see this and figure eh, I don't really understand this but somebody wrote "volatile" so I guess they knew what they were doing. > Volatile ensures that if a read is int…

> Unfortunately "allows" here is telling us about the programmers not the hardware. The programmers see this and figure eh, I don't really understand this but somebody wrote "volatile" so I guess they knew what they were doing.

So... you're saying that whether a piece of code is correct is something that's more complicated than just noticing that it has the word "volatile" written somewhere? This is super obvious.

> C++ 98 doesn't provide standardized atomics, so you would need to find out on each target what (if anything) you're required to do to get atomic behaviour.

Yes.

But you don't need all of your operations to be atomic in order to get atomic behavior for an operation. It turns out that for a common ring buffer, with one producer and one consumer, you only need the read/write position to be read/written atomically, and operations on buffer data must be ordered wrt. operations on the read/write positions.

This is commonly achieved with "volatile" on single-core systems.

> This keyword does not mean "I want a Sequentially Consistent memory model across all my code, except somehow still very fast". That's not a thing.

You get ordering of the volatile operations with respect to other volatile operations, from the perspective of one CPU core. That is sometimes all you need.

The point that "some people don't understand what volatile means" is not germane.

> Volatile accesses for things that are clearly just RAM are a code smell. As a result the volatile keyword in C and C++ is usually a code smell.

This is a quite extreme viewpoint. I can't agree with it.

Volatile is, yes, overused and abused. Or at least it was. However, if you want to write a ring buffer and use it on a single-core processor in an embedded environment, you can write the whole thing in old-school C90 or C++98, and the only real question you have about your environment is whether the operations on read/write pointers will tear.

It is rare, at the very least, to find a CPU where reads and writes to an int will tear.

Re: Show HN: Embedded Ring Buffer C++98

#38

Post by klodolph »

gpderetta wrote: post #33
ncmncm wrote: post #32

Earlier quoted context omitted.

Right. Volatile very rarely means what peoole using it imagine it means. It very much does not mean Do What I Mean. E.g. compilers routinely elide writes to stack variables declared volatile— or even atomic —that they "know" are not aliased. So e.g. if an interrupt routine might look at things in your stack frame, you need to use asm to force it not to fool with writes there. Volatile and atomic don't help you, there…

To be pedantic, technically they can't elide, reorder or coalesce a accesses to volatile objects any more they can do any other form of I/O, even if it is a local variable whose address is never taken. Of course how accesses on the abstract machine maps to the actual hardware is implementation defined, although most will document to translate them to plain movs. And implementations have bugs of course; here [1] gcc r…

This is why volatile function parameters are deprecated in C++20. It's basically nonsensical to begin with.

It sounds like the author of the code expected that the argument would get a memory location on the stack somewhere, but this behavior isn't actually mandated by the standard. If a variable is placed in a register, then "reading" it is a no-op.

Re: Show HN: Embedded Ring Buffer C++98

#39

Post by gpderetta »

klodolph wrote: post #38
gpderetta wrote: post #33

Earlier quoted context omitted.

To be pedantic, technically they can't elide, reorder or coalesce a accesses to volatile objects any more they can do any other form of I/O, even if it is a local variable whose address is never taken. Of course how accesses on the abstract machine maps to the actual hardware is implementation defined, although most will document to translate them to plain movs. And implementations have bugs of course; here [1] gcc r…

This is why volatile function parameters are deprecated in C++20. It's basically nonsensical to begin with. It sounds like the author of the code expected that the argument would get a memory location on the stack somewhere, but this behavior isn't actually mandated by the standard. If a variable is placed in a register, then "reading" it is a no-op.

Interesting that they are deprecated because volatile locals have specific guarantees with regard to longjmp.

Re: Show HN: Embedded Ring Buffer C++98

#40

Post by tialaramex »

klodolph wrote: post #37
tialaramex wrote: post #30

Earlier quoted context omitted.

> The use of volatile is typical here. This is definitely true. C++ programmers typically do this. > It allows the ring buffer to be used from interrupts Unfortunately "allows" here is telling us about the programmers not the hardware. The programmers see this and figure eh, I don't really understand this but somebody wrote "volatile" so I guess they knew what they were doing. > Volatile ensures that if a read is int…

> Unfortunately "allows" here is telling us about the programmers not the hardware. The programmers see this and figure eh, I don't really understand this but somebody wrote "volatile" so I guess they knew what they were doing. So... you're saying that whether a piece of code is correct is something that's more complicated than just noticing that it has the word "volatile" written somewhere? This is super obvious. >…

> It turns out that for a common ring buffer, with one producer and one consumer, you only need the read/write position to be read/written atomically, and operations on buffer data must be ordered wrt. operations on the read/write positions.

But again, putting a volatile modifier on the read/write position did not achieve this.

Most of what you thought you wanted "volatile" for here is either just always the behaviour for aligned primitive types anyway (on platforms like x86 with a relatively strong consistency guarantee at a low level) or still unsafe even after you sprinkled volatile around (on ARM platforms with relaxed memory access rules, unless your compiler specifically gives different behaviour for volatile)

> This is a quite extreme viewpoint. I can't agree with it.

It's the conventional viewpoint by now, even the C++ Core Guidelines explicitly tell you not to do this. e.g. CP200

But if you want an extreme viewpoint not yet widely accepted by say the C++ Standards Committee, try this:

Neither volatile nor atomic should exist as type qualifiers. A "volatile 32-bit integer" isn't a thing, and neither is an "atomic 32-bit integer". The operations exist and they're important in low-level programming, but the types are a fiction‡.

Rust's model here is much closer. You can do volatile memory access in Rust, but you can't have "volatile" types, they're not a thing. You must explicitly call an (inlined) function to read whatever primitive data you need out of a memory address or write it to the address. As a result, programmers who need volatile memory access are far more likely to remain clear about what they're doing and why, not just sprinkling "volatile" keywords and hoping it'll do what they meant.

‡ If you want some real C++ horror, why is volatile constexpr a thing? Literally the committee's excuse is maybe this could be useful. They don't have a concrete example of use, but hey, why not throw things we don't need into the standard, it's not as though C++ is a bloated language with far too many edge cases already...

Post reply on HN