Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ex Valve dev on CS:GO’s codebase

twitter.com

Post reply on HN
Print view View original

91–100 of 129 posts

Re: Ex Valve dev on CS:GO’s codebase

#91

Post by kllrnohj »

Arelius wrote: post #87
kllrnohj wrote: post #80

Earlier quoted context omitted.

Also in this case talking about the graphics stack it's not like OpenGL or DirectX are setup to be testable. You can't really "unit test" shader code, the best you can do is render it in some test scenes and screenshot the results. Which ends up flaky & noisy due to valid-per-spec differences in GPU & driver behaviors.

Every language has valid-per-spec differences. That's exactly why you test. For sure OpenGL/DX requires more infrastructure to run unit tests than a generic block of C code. But it's absolutely possible to "unit test" shader code, with buffer read-back and/or vertex stream out, among other options. It's more the game engines themselves that aren't setup for unit tests rather than the graphics stack

> For sure OpenGL/DX requires more infrastructure to run unit tests than a generic block of C code. But it's absolutely possible to "unit test" shader code, with buffer read-back and/or vertex stream out, among other options.

Which is what I said, you can screenshot & compare. But it becomes a fuzzy compare due to acceptable precision differences.

And it ends up being more of an integration test and not a unit test.

> Every language has valid-per-spec differences.

They really don't, but that's not entirely what I'm talking about. I'm talking about valid hardware behavior differences, which doesn't exist broadly. How a float performs in Java is well-defined and never changes. How numbers perform in most languages is well-defined and does not vary.

GPU shaders are completely different. Numbers do not have consistent behavior across differing hardware & drivers. This is a highly unique situation. Even in languages where things are claimed to be variable (like the size of int in C & C++), end up not actually varying, because things don't cope well with it. Shaders don't play any such similar games.

Re: Ex Valve dev on CS:GO’s codebase

#92

Post by Jugurtha »

leetcrew wrote: post #72
arkitaip wrote: post #17

Earlier quoted context omitted.

Are there any drawbacks of using a plugin centric approach? Typically there is loss of expressiveness in code, loss of performance or disconnect between core and plugin development.

you need to have someone who is very comfortable saying "no" to be in charge of maintaining the interface. otherwise, and especially if the plugin devs have access to the core code, they will say stuff like "hey, I see you have a very convenient function in the core, can you expose it for my plugin?". once you open the door to this, your encapsulation suffers death by a thousand cuts. you can end up with a de facto m…

The core pretty much does nothing, except load the plugins and a few functions that we will extract into their components. This is what we did for the other parts.

One reason we did this was because we built custom, turn-key, ML products for large enterprise. Complete applications, from data acquisition and model training to "the JavaScript", admin interface, user management, etc.

Now... these large enterprise clients were in a sector. We could hardly sell the product to other similar clients because we couldn't just pick and choose which component or features to put on a skeleton.

It took us a lot of time, because these projects were both "software engineering" and "machine learning". In other words, we were toast. The worst of both worlds, as we were doing complete applications that even allowed their people to train models themselves.

It took a toll on morale. At some point working on eight different projects with different code bases and subsets of the team. We were fed up with this. We wanted to do things differently. We wanted to be able to get the time it took to ship the project the closest possible to the time it took to train models, which we historically did rapidly. It was all the rest that took time.

Total time = time to define problem + time to get data + time to produce models + time to write application + a big hairy epsilon

We wanted to bring "Total time" to its irreducible form. We didn't want to keep writing different applications for clients. We knew how to do it, but we did it enough times for several clients to notice patterns we wanted to extract into components. We also were losing time with the ML project lifecycle (experiment tracking, model management, collaboration, etc). We didn't want to ask the question "Which model is deployed again? What data produced that model? I tried your notebook on my machine, it doesn't work! DS: Hey, Jugurtha... Can you deploy my model? Jugurtha: I'm busy right now. I'll do it as soon as possible".

So we started building our ML platform[0] to remove as much overhead as possible, while being flexible. For example, one of our design goals is that everything one can do on the web app, they should be able to do with an API call.

- [0]: https://iko.ai

Re: Ex Valve dev on CS:GO’s codebase

#93

Post by an_opabinia »

Scuds wrote: post #73

richgel999 has posted before about Valve's toxic culture - so bikeshedding about "well why don't you just... " is beside the point in the larger context of perverse incentives. https://twitter.com/richgel999/status/1330765701037101057?s=...

It's a refreshing perspective. It is still not that much of a knock on Valve - after all, they hired him, seems like they made the right choice, he is really articulate, voted with his feet, etc. etc. It actually makes him and Valve look pretty good, especially since it's such engineering and strategy focused and honest criticism, and none of it concerns illegal stuff either, like retribution.

Re: Ex Valve dev on CS:GO’s codebase

#94

Post by dehora »

arkitaip wrote: post #17
Jugurtha wrote: post #4

One of the best, and first, things we did when starting our machine learning platform was to design it using a plugin architecture. There's a lot of scar tissue and horrible experience through our previous ML products we built for enterprise. Namely, it was extremely hard to onboard new developers to work on the product. They had to understand the whole thing in order to contribute. Changing something was also hard,…

Are there any drawbacks of using a plugin centric approach? Typically there is loss of expressiveness in code, loss of performance or disconnect between core and plugin development.

> Are there any drawbacks of using a plugin centric approach?

Here are some:

1. You're perhaps more subject to Hyrum's law. If plugin devs can see it, they will use it. The general observation here is that it's harder to control the visible interfaces and implicit dependencies you export than the dependencies and interfaces you rely on. As one example, semantic versioning doesn't cater for this at all. Plus, most of the practice knowledge in software is on managing relied on dependencies.

2. Dog follows tail. It can happen that a plugin becomes so successful the overall system evolution slows down. The core system can upgrade, but adoption/deployment can be constrained when a particularly valuable plugin doesn't move up to the latest and the customer base sees more value in the plugin than its core platform. This can compound poorly over time, and in extreme cases the desirable plugin can become its own platform/system (something I think business savvy tech leaders are increasingly aware, and wary, of).

3. Operational complexity. It can be harder to run and maintain a plugin based system than a closed one. 2 is a consideration here, but so are other concerns, such as security and resource isolation. Strategies vary, but who pays this cost on a relative basis is one of the more (and perhaps the most) interesting aspect of working on or using plugin systems. As one example of this, think about allocating responsibility for bugs.

4. R&D complexity. It may take more time to design and build a plugin system than a closed one. Incrementally evolving to a plugin system can be difficult if you didn't start there to begin to with. So you usually need a clear opening motivation to delay reward (or avoid over engineering) to invest in a system design where functionality can be extended by non-core developers.

Re: Ex Valve dev on CS:GO’s codebase

#95

Post by Bayart »

kllrnohj wrote: post #86
eertami wrote: post #35

Earlier quoted context omitted.

>Valve has no excuse, they make crazy amounts of money, they can fund the development of a new engine from scratch easily. They just choose not to. It admittedly gets a bit more difficult when these hacks and quirks are part of what create the unique feel of your game engine. People have played CS at such high levels for so long that switching engine at all is likely going to introduce some difference in feeling, eve…

> It admittedly gets a bit more difficult when these hacks and quirks are part of what create the unique feel of your game engine. An example of this in the Quake 3 engine (and now permanent behavior in the CS series) is air strafing. It's a glitch in how Quake 3 handles motion vectors. But it's now also enshrined behavior, complete with entire game modes in CS built around it (KZ & surf maps). If you went and made a…

A more famous example of that same class of bugs is bunny-hopping, why has become an FPS staple far beyond the scope of the Quake engine.

Re: Ex Valve dev on CS:GO’s codebase

#96

Post by andor »

moron4hire wrote: post #84
matsemann wrote: post #45

Earlier quoted context omitted.

Game code can be hard to unit test, it needs integration tests on actual hardware. Lots of weird stuff on all chips that need to be taken care of.

There's also no easy way to test for things like "do the shadows render correctly". About the only thing you can do is take before/after screenshots and compute a signal-to-noise ration on a diff between the images. Which makes for an extremely fragile test definition. What if you change the default FOV of the camera? Now all your tests fail for no good reason.

What if you change the default FOV of the camera? Now all your tests fail for no good reason.

Which is completely fine, because you probably wouldn't want to accidentally change the FOV, would you?

High confidence tests fail on unexpected results. If only some aspects of the results are checked, the tests have obvious blind spots.

Re: Ex Valve dev on CS:GO’s codebase

#97

Post by speeder »

mam2 wrote: post #14
wccrawford wrote: post #8

"Also, if you touched the renderer, even in a simple way, and a team later encountered a rendering bug, you would be blamed and have to fix it. Even if the bug had nothing to do with your change. This taught programmers to not change anything unless absolutely necessary." I've seen this effect in code that wasn't nearly this bad, and I've even felt this way... But in the end, I've decided to do it anyhow. The end res…

No unit test ?

Source engine is actually a fork of Quake engine.

Back then people often had no idea what they were doing, often because they were doing stuff nobody did before, ever.

Re: Ex Valve dev on CS:GO’s codebase

#98

Post by enragedcacti »

kllrnohj wrote: post #86
eertami wrote: post #35

Earlier quoted context omitted.

>Valve has no excuse, they make crazy amounts of money, they can fund the development of a new engine from scratch easily. They just choose not to. It admittedly gets a bit more difficult when these hacks and quirks are part of what create the unique feel of your game engine. People have played CS at such high levels for so long that switching engine at all is likely going to introduce some difference in feeling, eve…

> It admittedly gets a bit more difficult when these hacks and quirks are part of what create the unique feel of your game engine. An example of this in the Quake 3 engine (and now permanent behavior in the CS series) is air strafing. It's a glitch in how Quake 3 handles motion vectors. But it's now also enshrined behavior, complete with entire game modes in CS built around it (KZ & surf maps). If you went and made a…

To add on, players like this feel enough that this behaviour is very convincingly replicated by Riot in Valorant, which is a CS-like game on Unreal engine.

Re: Ex Valve dev on CS:GO’s codebase

#99

Post by jrockway »

flohofwoe wrote: post #41
mam2 wrote: post #14

Earlier quoted context omitted.

No unit test ?

The Source engine hails from the 90s, testing hadn't been invented back then ;) But on a more serious note, writing automated tests for game engines involves a lot more than just "duh, unit tests" (especially when testability wasn't a concern in the original design).

I am surprised that game engines aren't set up to test simple scenarios programmatically. I play a lot of Overwatch and the bugs / patch notes about fixing those bugs amaze me every time; they tell me a lot about how the software is designed and tested.

There was one bug where a character has a deployable ability that doubles the damage and healing of all projectiles that pass through it. One day, the patch notes read "fixed an issue where healing was not amplified when passing through the amplification matrix". And, I totally get it... every conference talk I've seen out of Blizzard goes into details about all the infrastructure they've made for play testing their games. It sounds easy to get your coworkers into a build of your latest PR and try it out. But things like these subtle numbers adjustments just don't translate well to play testing -- sometimes the enemy is doing so much damage that you can't really be sure that the problem is the Amp Matrix isn't multiplying the healing by the right number. So, from time to time, refactors break it!

But, in a world where you could easily write integration tests, this problem would never happen. You'd write a simple scenario like "create empty room. place baptiste at position 0,0. deploy amp matrix at position 10,0 with orientation 90 degress. place sombra at position 20,0. set her health to 80. make baptiste fire a healing grenade along vector 1,0 at an angle of 45 degrees. wait 10 ticks. ensure that sombra's health is now 200." The framework to be able to write tests like this is not difficult (you can do it in their "workshop"), and it's not difficult to write a test like this for every ability, and even every combination of abilities. And, it would mean that play testers never ever need to be suspicious of numbers; the automated tests already check that. You'd make developers more productive (the computer can check the basics like this), and play testers more productive (they don't need to test simple stuff anymore). But... I don't think they do it. The buggiest releases are when the team is under time pressure to hit a deadline (Overwatch has seasonal events; the patch that introduces a seasonal event always has some weird bugs), and I don't think automated tests miss things under time pressure -- but humans sure do.

The one thing I'll give Blizzard credit for is that their games are fun. All that playtesting is certainly a good idea. I'd supplement it with some gameplay-focused integration tests, though. They have the money and the tools teams, and their games last longer than a few months, so it just seems like a smart investment to me. So it just baffles me what bugs ship to production.

Re: Ex Valve dev on CS:GO’s codebase

#100

Post by tecleandor »

kllrnohj wrote: post #88
dx87 wrote: post #6

Earlier quoted context omitted.

They're on Source 2 now, but not all of their games have been ported to it. A former employee said that Source 2 is pretty much just Source 1 with some extra phsyics bolted on, not a completely new engine.

Source 2 is Source 1 with most of the key systems replaced. They may have started with physics, but they didn't stop there. Many game engines are a collection of modules, Source included. So it becomes a fuzzy line when it becomes a "new" engine. Does replacing one module make a new engine or not? How about 2? 3? And I very much do mean "replaced" there. Physics, since you mentioned that, was switched from Havok to t…

It's the Engine of Theseus! ;)
Post reply on HN