Earlier quoted context omitted.
Honestly, I've made over 15 games in my career and safety with C++ really just isn't an issue with decent developers. The line between what's a programmer for games and what's a designer is narrowing, most designers are competent programmers. Furthermore, there's some great tools out there to help prevent things like memory leaks. Combine that with good company practice, like code reviews, and it becomes a non-issue.
I guess, then, that you weren't part of the Battlefield 4 team [1]. I've discussed the issue of the "no decent programmer" fallacy in the past; yes, in theory if programmers were careful and alert, they could create flawless software, yet this never happens in practice because humans are prone to errors (i.e. not understanding a subtlety of the language or library, thinking that a validation is done at a different le…
Why C++ for Unreal 4
121–130 of 178 posts
Re: Why C++ for Unreal 4
#122Earlier quoted context omitted.
The biggest advantage of scripting for me has been 1. Co-routines Co-routines (co-operative multi-tasking?) mean you can do stuff like while (isWalking()) { advance(); yield(); } This is effectively 'yield' from Python, C#, etc.. You can implement this in C++ by swapping stacks and calling setjmp but there's usually issues. 2. Iteration time You can usually swap script code live. This project aims to fix that for C++…
The slightly obscure music programming language SuperCollider [1] added co-routines about 10 years ago and they became one of my beloved techniques. Was very glad to see them come to python and soon to mainstream javascript. Boost has a c++ implementation but it looks quite different: http://www.boost.org/doc/libs/1_55_0/libs/coroutine/doc/html... [1] http://supercollider.github.io edit: pythons new asyncio stuff loo…
I really need to get around to writing a blog post to explain this in detail since this misapprehension is endemic. Python and JavaScript do not have coroutines, they have generators. Lua has actual coroutines.
The latter is dramatically more expressive than what you can do with what Python, JavaScript, and C# offer. This mistake drives me crazy because it means people don't know what they're missing.
Here's a quick example. Let's say we've got a little Python class for binary trees:
class Tree:
def __init__(self, left, data, right):
self.left = left
self.data = data
self.right = right
We'll add a method to do an in-order traversal. It takes a callback and invokes the callback for every data value in the tree, like so: def walk(self, callback):
"""Traverse the tree in order, invoking `callback` on each node."""
if self.left:
self.left.walk(callback)
callback(self.data)
if self.right:
self.right.walk(callback)
We can create a little tree and then print the data items in order like so: tree = Tree(Tree(Tree(None, 1, None), 2, Tree(None, 3, None)), 4, None)
tree.walk(print)
(This works in Python 3, in Python 2, you'll have to make a little fn for print.) Swell, right?Later, we decide we want to iterate over the items in a tree. Easy-peasy, Python has generators! We can just make a function that takes a Tree and returns a generator. We already have a method to walk the nodes, so we just need to call that and then yield the items, like so:
def iterateTree(tree):
def callback(data):
yield data
tree.walk(callback)
Then you can just use it like so: for x in iterateTree(tree):
print(x)
Perfect, right?Actually, no. This doesn't work at all. You can't yield from the callback passed to walk. That's because walk() itself doesn't know that the callback is a generator.
This is the problem with generators: they divide all functions into two categories: regular functions and generators. You run a regular function by calling it. You run a generator by iterating over it. The caller must use it in the correct way.
At its simplest level it means you have to be careful when refactoring. If you have a generator function that gets too big and you want to split it up, you have to remember that the functions you split out are also special generator functions if they contain a yield. You have to remember to flatten it when you "invoke it".
It's more than just annoying though: it means it's impossible to write code that works generically with both kinds of functions. In other words, all of your higher-order functions like map, filter, etc. now only work with some of your functions. (Or, I suppose, you could explicitly implement them to support both but that's more work and I don't think most languages do.)
In languages like Lua, the above code just works. You can yield from anywhere in the callstack and the entire stack is suspended. It's fantastic.
(If I can be forgiven a bit of self-promotion, I'll note that my programming language Wren[1] can not only express full coroutines like Lua, but also supports symmetric coroutines which can express some things Lua cannot. They are roughly like the equivalent of tail call elimination for coroutines.)
Re: Why C++ for Unreal 4
#123We've also been there done that (about 10 years ago though), we had a very powerful scripting approach integrated into our game engine which gave direct access to game play systems in order to let our level and game designers build scripted behaviour into the game. In the end we ended up with a terribly huge mess of script code (I think it was about a third of the actual C/C++ code) and the majority of the per-frame…
An ideal scripting language:
- Allows designers to build complex gameplay elements, define complex ai behaviors, create gameflow with minimal work from the software engineers.
- Handles memory allocation/destruction behind the scenes
- Does not crash the game when an error occurs
- Handles multithreading and/or events
- Does not allow designers to shoot themselves in the foot
- Has a simple and clean syntax
- Allows software engineers to expose their APIs to it easily
- Can be reloaded on the fly
If you control the feature set of the scripting language then it becomes a crucial tool for rapid development of your game - empowering the creative team to make their game and allowing the software engineers to concentrate on all the other stuff that needs their attention.
In my opinion visual scripting and component systems can be a useful addition to an engine, but I have always seen a usefulness in having a scripting language layer.
(I have shipped multiple high profile console games with more lines of script code than of game code and design teams matching the size of the programming teams)
Re: Why C++ for Unreal 4
#124Earlier quoted context omitted.
I guess, then, that you weren't part of the Battlefield 4 team [1]. I've discussed the issue of the "no decent programmer" fallacy in the past; yes, in theory if programmers were careful and alert, they could create flawless software, yet this never happens in practice because humans are prone to errors (i.e. not understanding a subtlety of the language or library, thinking that a validation is done at a different le…
If BF4 was written in Csharp, or java (or rust or go?? I'm sure it would still have just as many bugs. One of peoples biggest complaint is the kill shots that you don't see, but that's a design choice (client side hit detection).
Re: Why C++ for Unreal 4
#125It's important to note that what is being talked about in this post is not, "why we wrote the Unreal engine in C++", because it already was in C++. Many games, older Unreals included, had a separation between "code" and "scripting", where stuff like animations, weapon firing, etc. was written in scripts, in the belief that this would be easier to update as required vs. C or C++ code. Doom 3 and previous Unreal engine…
Brief rant: Quake 1 ran a virtual machine, which QuakeC compiled down to. Quake 2 ran native code via DLLs. Quake 3 ran either VM code or native code--depending on how clever you wanted to be, you might need to break into the native code. There wasn't some "mistake" about using scripts-vs-code, because they would actually compile down to executable bytecode. This made it much easier to load mods over the network if y…
Re: Why C++ for Unreal 4
#126Earlier quoted context omitted.
Exactly. What's most distressing is people (as normal) are completely ignoring the garbage collection overhead, which is mostly where the advantage of having complete control is, in terms of micro-managing your memory allocation, e.g. using slab allocators, memory pools, pre-allocation, etc. C# code in theory (ignoring things like intrinsics support and inline asm) can be as fast as C++ for tight loops, but in my exp…
C# has support for stack-allocated "struct" objects that avoid the GC. They have their limitations and gotchas, being somewhere between simple C structs and C# classes, but they exist. GC-based languages run games on many, many platforms. The problem, imho, is that you have to leave 90% of the language features on the shelf when you're doing your main loops in order to avoid triggering the GC. The gaming industry is…
Though in this context it's relevant to point out pretty much any implementation will stack allocate them, it's not accurate to say C# has stack allocated objects.
[1] http://blogs.msdn.com/b/ericlippert/archive/2009/04/27/the-s...
Re: Why C++ for Unreal 4
#127We've also been there done that (about 10 years ago though), we had a very powerful scripting approach integrated into our game engine which gave direct access to game play systems in order to let our level and game designers build scripted behaviour into the game. In the end we ended up with a terribly huge mess of script code (I think it was about a third of the actual C/C++ code) and the majority of the per-frame…
> The main problem with scripting layers is that you are basically handing programming tasks over to team members who's job is not to solve programming tasks, and thus getting a lot of beginner's code quality and performance problems which are almost impossible to debug and profile I disagree that it has to follow that scripting -> bad code. I think of the C scripting integration that I do as a Judo secret weapon. I…
Re: Why C++ for Unreal 4
#128We've also been there done that (about 10 years ago though), we had a very powerful scripting approach integrated into our game engine which gave direct access to game play systems in order to let our level and game designers build scripted behaviour into the game. In the end we ended up with a terribly huge mess of script code (I think it was about a third of the actual C/C++ code) and the majority of the per-frame…
By removing the power from your design team you are creating more work for the software engineers and removing tools from the creative team. An ideal scripting language: - Allows designers to build complex gameplay elements, define complex ai behaviors, create gameflow with minimal work from the software engineers. - Handles memory allocation/destruction behind the scenes - Does not crash the game when an error occur…
Re: Why C++ for Unreal 4
#129Earlier quoted context omitted.
No, modern C++ is not even close to memory safe. This is my favorite meme to destroy over and over on HN. :) Consider iterator invalidation, null pointer dereference (which is undefined behavior, not a segfault -- and you can't get away from pointers because of "this" and move semantics), dangling references, destruction of the unique owner of the "this" pointer, use after move, etc. etc.
Extraordinary claim- please elaborate. I'm working on 100's of thousands of lines of C++ code with a medium-sized team; memory issues are almost non-existent because of disciplines described above.
* Iterator invalidation: if you destroy the contents of a container that you're iterating over, undefined behavior. This has resulted in actual security bugs in Firefox.
std::vector v;
v.push_back(MyObject);
for (auto x : v) {
v.clear();
x->whatever(); // UB
}
* "this" pointer invalidation: if you call a method on an object that is a unique_ptr or shared_ptr holds the only reference to, there are ways for the object to cause the smart pointer holding onto it to let go of it, causing the "this" pointer to go dangling. The simplest way is to have the object be stored in a global variable and to have the method overwrite the contents of that global. std::enable_shared_from_this can fix it, but only if you use it everywhere and use shared_ptr for all your objects that you plan to call methods on. (Nobody does this in practice because the overhead, both syntactic and at runtime, is far too high, and it doesn't help for the STL classes, which don't do this.) class Foo;
unique_ptr inst;
class Foo {
public:
virtual void f();
void kaboom() {
inst = NULL;
f(); // UB if this == inst
}
};
* Dangling references: similar to the above, but with arbitrary references. (To see this, refactor the code above into a static method with an explicit reference parameter: observe that the problem remains.) No references in C++ are actually safe.* Use after move: obvious. Undefined behavior.
* Null pointer dereference: contrary to popular belief, null pointer dereference is undefined behavior, not a segfault. This means that the compiler is free to, for example, make you fall off the end of the function if you dereference a null pointer. In practice compilers don't do this, because people dereference null pointers all the time, but they do assume that pointers that have been successfully dereferenced once cannot be null and remove those null checks. The latter optimization has caused at least one vulnerability in the Linux kernel.
Why does use after free matter? See the page here: https://www.owasp.org/index.php/Using_freed_memory
In particular, note this: "If the newly allocated data chances to hold a class, in C++ for example, various function pointers may be scattered within the heap data. If one of these function pointers is overwritten with an address to valid shellcode, execution of arbitrary code can be achieved." This happens a lot—not all use-after-free is exploitable, of course, but it happened often enough that all browsers had to start hacking in special allocators to try to reduce the possibility of exploitation of use-after-frees (search for "frame poisoning").
Obligatory disclaimer: these are small code samples. Of course nobody would write exactly these code examples in practice. But we do see these issues in practice a lot when the programs get big and the call chains get deep and suddenly you discover that it's possible to call function foo() in one module from function bar() in another module and foo() stomps all over the container that bar() was iterating over. At this point claiming that C++ is memory safe is the extraordinary claim; C++ is neither memory safe in theory (as these examples show) nor in practice (as the litany of memory safety problems in C++ apps shows).
Re: Why C++ for Unreal 4
#130Earlier quoted context omitted.
C# has support for stack-allocated "struct" objects that avoid the GC. They have their limitations and gotchas, being somewhere between simple C structs and C# classes, but they exist. GC-based languages run games on many, many platforms. The problem, imho, is that you have to leave 90% of the language features on the shelf when you're doing your main loops in order to avoid triggering the GC. The gaming industry is…
I feel like I should point out that the stack is an implementation detail [1] Though in this context it's relevant to point out pretty much any implementation will stack allocate them, it's not accurate to say C# has stack allocated objects. [1] http://blogs.msdn.com/b/ericlippert/archive/2009/04/27/the-s...