Live data from Hacker News

Peredvizhnikov Engine: Lock-free game engine written in C++20

github.com

171–180 of 183 posts

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#171
post #100

Earlier quoted context omitted.

What is special about msvc's `std::deque`?

It's block size is something stupid like 16 bytes, so it effectively becomes a slower linked list.

See https://devblogs.microsoft.com/oldnewthing/20230810-00/?p=10... which includes confirmation & a brief commentary of the tiny block size.

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#173
post #14

Earlier quoted context omitted.

Never mind that a lot of Volga boats were very single-threaded. https://www.amusingplanet.com/2021/12/belyana-russias-giant-...

It's a logging framework! :D

Streaming logging framework

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#174
post #138

Where is the game demo? Also to be considered gaming engine nowadays, you need actual tools, exporters (Maya, 3DSMax, etc.) + who knows what else (collaboration tooling, metrics, alerts)

I disagree—"game engine" does not necessarily mean "thing I can replace Unity or Unreal with".

Is there any "game engine" here though currently? Seems like 'just' a lock-free work scheduler

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#175

Earlier quoted context omitted.

bullshit, consoles/mobile are bigger markets than PC/Windows PC is just less than 1/3 of the whole picture https://www.data.ai/en/insights/mobile-gaming/2022-gaming-sp...

A bit more than that since Xbox runs Windows too.

A special flavour of Windows.

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#176

Does anyone have experience debugging/profiling highly contended critical sections of STM vs a more traditional mutex implementation? At the end of the day something has to mediate concurrent access to shared memory, there’s no free lunches, and mutexes are so well optimized, profiled, and understood. I’m unclear if the same applies to STM where a transaction may need to be retried an unbounded(?!) number of times.

Actually, for starvation-free STMs, transactions will retried a _bounded_ number of times. One example is 2PLSF, but there are several others https://zenodo.org/record/7886718

Interesting, thanks! It’s been years since I’ve read up on STM.

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#177

[flagged]

Breaking the site guidelines like this will eventually get your main account banned as well, so please don't.

HN is no place for nationalistic flamewar, including when there's an actual war going on—in fact, especially not then:

"Comments should get more thoughtful and substantive, not less, as a topic gets more divisive."

https://news.ycombinator.com/newsguidelines.html

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#178

I have an Actor framework which uses a vanilla std::deque for method pointers, and to add messages to the queue, the locking technique is a Benaphore (the original Futex, which uses an atomic and a locking primitive, with the twist that my locking primitive is a combo of spinlock/mutex based on retry count). Nothing special. Benchmarks show that very rarely does the message push function block, and the chance of an O…

When you protect an std::deque with a mutex you would need at least two atomic operations: to lock the queue before pushing, and to unlock the queue after pushing. Because you're using an std::deque it may need to allocate memory during a push, which would happen under the lock, which makes it more likely for a thread to suspend with the lock taken. While the queue is locked other threads will have to wait, possibly even suspend on a futex, and then the unlocking thread would have to wake another thread up.

The most expensive part of any mutex/futex is not locking, it's waking other threads up when the lock is contended. I'm actually surprised you only get 10 million messages per second, is that for a contended or an uncontended case? I would expect more, but it probably depends on the hardware a lot, these numbers are hard to compare.

My actor framework currently uses a lockfree intrusive mailbox [1]_, which consists of exactly two atomic exchange operations, so pushing a node is probably cheaper than with a mutex. But the nicest part about it is how I found a way to make it "edge triggered". A currently unowned (empty) queue is locked by the first push (almost for free, compared to a classic intrusive mpsc queue [2]_ the second part of push uses an exchange instead of a store), which may start dequeueing nodes or schedule it to an executor. The mailbox will stay locked until it is drained completely, after which it is guaranteed that a concurrent (or some future) push will lock it. This enables very efficient wakeups (or even eliding them completely when performing symmetric transfer between actors).

I actually get ~10 million requests/s in a single-threaded uncontended case (that's at least one allocation per request and two actor context switches: a push into the target mailbox, and a push into the requester mailbox on the way back, plus a couple of steady_clock::now() calls when measuring latency of each request and checking for soft preemption during context switches). Even when heavily contended (thousands of actors call the same actor from multiple threads) I still get ~3 million requests/s. These numbers may vary depending on hardware though, so like I said it's hard to compare.

In conclusion it very much depends on how lockfree queues are actually used, and how they are implemented, they can be faster and more scalable than a mutex (mutex is a lockfree data structure underneath anyway).

I'd agree with you in that mutexes are better when protecting complex logic or data structures however, because using lockfree interactions to make it "scalable" often makes the base performance so low, that you'd maybe need thousands of cores to justify the resulting overhead.

.. [1] https://github.com/snaury/coroactors/blob/a599cc061d754eefea... .. [2] https://www.1024cores.net/home/lock-free-algorithms/queues/i...

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#179

Earlier quoted context omitted.

bullshit, consoles/mobile are bigger markets than PC/Windows PC is just less than 1/3 of the whole picture https://www.data.ai/en/insights/mobile-gaming/2022-gaming-sp...

Got bad news for you about what the console SDKs run on (Also, the consoles don't run Linux)

From what I know:

XBOX runs a variant of Windows.

PS3/4/5 OSes have all been based on FreeBSD.

Apparently Nintendo Switch runs a proprietary kernel, which is interesting (https://en.wikipedia.org/wiki/Nintendo_Switch_system_softwar...).

Re: Peredvizhnikov Engine: Lock-free game engine written in C++20

#180
post #162
post #120

Earlier quoted context omitted.

Lock free is kind of an overloaded term that can mean a variety of things depending on what the user is thinking. Usually the goal is being able to make forward progress even if one thread is context switched out by the OS scheduler, which might be called wait-free or non-blocking. Using mutexes (unless you can prevent OS scheduling, interrupts, etc) makes this property impossible to achieve. In general, MPSC queues…

> In general, MPSC queues are super fast and there's no real reason to prefer a locked queue. There's one significant advantage that a locked vector or deque has over MPSC/MPMC queues: the consumers can dequeue all messages in a single operation by locking the vector, swapping it with an empty vector (typically, that's just 3 words), and locking it again. That's such a simple operation that it will typically be as fa…

All true, although I would quibble with:

> That's such a simple operation that it will typically be as fast or even faster than a single pop-one-message operation in an MPSC/MPMP.

Not if the lock is blocked because one of the writers context switched out! The typical case is good, but the worst case is pretty bad.

Post reply on HN