My manager doesn't do almost anything. I wonder why they keep him around. He has like two reports and we're both fully self-directed. He can't understand technical issues, and whenever he proposes something, it's completely untenable due to his lack of understanding what's useful or possible.
well look, I already told you, I deal with the customers so the engineers don't have to. I have people skills. I'm good at dealing with people, can't you understand that? What the hell is wrong with you people?
My manager spent $1M on a backup server that I never used
161–170 of 244 posts
Re: My manager spent $1M on a backup server that I never used
#162This rant starts of pretty badly, with "Windows bad, Unix good, Windows especially bad because it doesn't have Unix tools". Reverse that statement. Is Unix bad because it doesn't Windows tools? > but simultaneously there was nothing to lean back on: no shell, no Unix tools like sed/awk, no SSH. I He joined in 2014, when Windows had a superior shell and remoting system that eliminates the requirement for low-level str…
What are you talking about - Windows was always terrible for an admin. Unix was and is vastly superior when it comes to remote admin. Having used both (Powershell is powerful yes, but it's nowhere as fluid as a Unix shell).
I have as devops managed both a small fleet of windows boxes (~20 machines) an a similar amount of Unix (Linx-based) machines.
Remote management of Windows was initially a pain to set up but once it was running it was fantastic. Being able to manage the entire application layer with properly tested, versioned code written in C# is an absolute killer. Things like debugging/logging etc are also "just there", and SQL Server is a fantastic database.
The Linux alternative on the other hand was easier to update and had some benefits to the developers (you can sort-of recreate a production-like environment in containers locally). The management after that was a huge pain because everything seems to have been built McGuyver style. I like Postgres but prefer using SQL Server (probably personal preference at this point tbh).
Things are different nowadays: systems like Kubernates apparently simplify the Linux side and make it affectively trivial to deploy and do cool stuff such as dynamically scale systems. It's also more efficient as you can remove the overhead of the Virtual Machines. It doesn't take much imagination to see that that would make a Unix-based system better.
Re: My manager spent $1M on a backup server that I never used
#163This rant starts of pretty badly, with "Windows bad, Unix good, Windows especially bad because it doesn't have Unix tools". Reverse that statement. Is Unix bad because it doesn't Windows tools? > but simultaneously there was nothing to lean back on: no shell, no Unix tools like sed/awk, no SSH. I He joined in 2014, when Windows had a superior shell and remoting system that eliminates the requirement for low-level str…
What are you talking about - Windows was always terrible for an admin. Unix was and is vastly superior when it comes to remote admin. Having used both (Powershell is powerful yes, but it's nowhere as fluid as a Unix shell).
Let me quote the article directly: "no SSH"
WinRM is roughly equivalent and superior in many ways. E.g.: It takes quite the effort to produce a report of "some setting or flag" across a fleet of servers with SSH, Bash, Sed, Awk, etc..
Now try this with PowerShell:
Invoke-Command -ComputerName $varWithListOfServers { Get-Process 'ubiserver.exe' } | Export-Csv 'report.csv'
This will connect in parallel to all the servers in the list (could be thousands!), 32 at a time by default. It'll get all of the process info (memory, cpu time, etc...) for a game server, and then add the source server computer name as an extra column. Dumping this to a valid CSV is a no-brainer.Put this in a scheduled task and you've got something Excel or Power BI or whatever can pick up and chart, graph, or send to a manager.
Try this with Linux. Seriously. Sit down and do the equivalent: many servers in parallel, batched to prevent overload, server name as an extra column, valid CSV output with proper quoting of quote characters, etc...
Oh, did you notice the lack of authentication rigmarole? The whole thing is secured and encrypted via Kerberos automatically. Copy the equivalent of that too. (Assume you have 10K servers that you haven't manually added the SSH keys of into your local workstation config.)
Put the script in the reply.
PS: I know some Unix people think that brevity is more important than time-to-solution, so to head off that whole argument:
PS C:\> Get-Alias | ? ResolvedCommandName -in ( 'Invoke-Command', 'Get-Process', 'Export-Csv' )
CommandType Name
----------- ----
Alias epcsv -> Export-Csv
Alias gps -> Get-Process
Alias icm -> Invoke-Command
Alias ps -> Get-Process
Ta-da, now it's just as unreadable as Unix: icm -CN $s { ps 'ubiserver.exe' } | epcsv 'report.csv'Re: My manager spent $1M on a backup server that I never used
#164This rant starts of pretty badly, with "Windows bad, Unix good, Windows especially bad because it doesn't have Unix tools". Reverse that statement. Is Unix bad because it doesn't Windows tools? > but simultaneously there was nothing to lean back on: no shell, no Unix tools like sed/awk, no SSH. I He joined in 2014, when Windows had a superior shell and remoting system that eliminates the requirement for low-level str…
Yes, Bash has a lot of warts (so much so that I wrote my own shell) but Powershell creates lots of new warts of its own.
As for automation, Windows doesn’t even come close to UNIX-like systems for east of scripting and automation. There’s just no contest.
Bash might be ugly at times but at least its ugliness is consistent. I’ve lost count of the number of times I’ve found parameters parsed differently between applications (because Windows passes parameters as a single string rather than an array of arguments like POSIX systems), or that so called headless installers still have GUI prompts and/or spin off a non-blocking process so it’s challenging figuring out when it’s complete. Or Powershell routines come back with an unexpected type thus breaking your entire pipeline. Or don’t even support basic help flags so you cannot discover how to use the damn routine to begin with. Or the utterly ridiculous numeric error code that MS returns rather than descriptive error messages. Or its over engineered approach to logging that makes quick inspections of the systems state far more involved than it should be, or even just managing simple things like software updates requires while other expensive 3rd party solutions because the whole application and OS package management situation is fundamentally broken at its foundations …I could go on. But suffice to say there’s so many painful edge cases to consider with Windows.
Windows does get somethings right though: RDP is a great protocol and its backwards compatibility story is second to none. But for server administration, Windows feels like a toy OS compared to most UNIX-like systems. As in it has support for pretty much anything you’d want to do With modern server management processes but yet everything it does support it does so in a really awkward and immature (technologically speaking) way.
I’m sure it’ll get there though. But likely not before I retire
Re: My manager spent $1M on a backup server that I never used
#165Earlier quoted context omitted.
Agreed. This reads like the $1 million system did what the company needed it to do (safely archive code to prevent more loss of old games) but it didn’t do exactly what this developer wanted. There are various good points scattered in the article for the author’s specific use case, but it’s written as if the entire company was mistaken to not make this decision revolve around this one developer.
Hi, Author here. thanks for reading my poorly written rant! The issue was that nobody ever said what the system was designed for and equally nobody was open to the idea of doing things differently until things went pop. The main thesis of the article is supposed to be that it doesn't matter what something costs or how much money is invested if it doesn't solve your needs. I wrote it a long time ago in a fit of aggrav…
Maybe a different country thing - devs in the US might be expected to be a lot more positive, in other countries we find complaining cathartic + this comes across as standard chat about work.
Re: My manager spent $1M on a backup server that I never used
#166This rant starts of pretty badly, with "Windows bad, Unix good, Windows especially bad because it doesn't have Unix tools". Reverse that statement. Is Unix bad because it doesn't Windows tools? > but simultaneously there was nothing to lean back on: no shell, no Unix tools like sed/awk, no SSH. I He joined in 2014, when Windows had a superior shell and remoting system that eliminates the requirement for low-level str…
Re: My manager spent $1M on a backup server that I never used
#167>all of this, written by hand in C++, nothing off the shelf, very minimal dependencies (OpenSSL being the only one of note), everything running on Windows and completely bespoke Great engineering but this is the time you could have spent building your core competency. re-inventing the wheel is a good recipe for failure
There's a reason Google had Borg and all the other big companies had their internal systems. Those were not reinventing the wheel, they were just building wheels round enough to hit the highway.
By the time our product launched Kubernetes had come out and we could migrate to that eventually.
Re: My manager spent $1M on a backup server that I never used
#168Enterprise architecture can commission a project to review options and select the one true enterprise backup solution. After sufficient peer review / diligent testing / bribery from vendors, an enterprise backup solution is chosen, wrangled into production, and then inflicted upon the org.
Let's be pragmatic. For existing production systems, it may not make sense to migrate them over to the new enterprise backup solution. Much risk and cost, limited payoff. Perhaps it's simplest to just keep them running their existing legacy backup solution until the entire production system that depends on it is finally decommissioned, which is planned for only 3 years away. We'll definitely decommission it after 3 years.
But for any new teams trying to deliver a new service into production. Are you storing data? Well, you must have a backup strategy before you can go to prod, and further, your backup design must align with the enterprise backup strategy -- you must integrate with the enterprise backup solution for your design to be stamped approved by architecture, unless there is an exception granted by appeal and/or ritual sacrifice to your manager's manager's manager's manager's manager.
Re: My manager spent $1M on a backup server that I never used
#169Earlier quoted context omitted.
What are you talking about - Windows was always terrible for an admin. Unix was and is vastly superior when it comes to remote admin. Having used both (Powershell is powerful yes, but it's nowhere as fluid as a Unix shell).
It depends on exactly what and how you're doing it. I have as devops managed both a small fleet of windows boxes (~20 machines) an a similar amount of Unix (Linx-based) machines. Remote management of Windows was initially a pain to set up but once it was running it was fantastic. Being able to manage the entire application layer with properly tested, versioned code written in C# is an absolute killer. Things like deb…
A single "small business" server is easier to run on Windows and it'll do everything you need it to do with minimal fuss. Easy GUI admin consoles are available, etc...
For 2-3 servers it swings back to Linux because some things are just easier at that scale.
For 10-1000 servers that aren't all identical, I prefer Windows because of things like Active Directory, GPOs, DSC, etc...
Above 10K and especially above 100K scale the Windows licensing costs become absurd and you're likely deploying thousands of identical nodes (web farms, HPC clusters, etc...) and then Linux becomes very compelling. This is why all of the FAANGs run primarily Linux, because at the multi-million-node scale it's the best option by far.
Interestingly, container technology is maturing in Windows as well, especially with Server 2022 and Windows 11.
However, it still has a few rough edges, so I'm waiting to see how it all pans out...
Re: My manager spent $1M on a backup server that I never used
#170> The games industry is weird: It simultaneously lags behind the rest of the tech industry by half-a-decade in some areas and yet it can be years ahead in others. I'd love to see examples of "ahead" coz I literally never saw it. Game dev studios act like automated testing is a new thing... > PostgresSQL performed much better and had the additional benefit of being able to cleanly split write-ahead logs (which are lar…
The “years ahead” was described in the article. We had what amounts to a very well oiled kubernetes installation with mTLS, but on Windows, in C++ and 10 years ago (before Kubernetes was a thing). Everything else you say is true, I could have chosen another solution for backups but the tradeoff between backup speed (replaying WAL can be time consuming) vs database load (full backups delay replication to replicas) was…
The revolutionary thing about Kubernetes was not what it did, but that it was an enterprise supported open source version of what everyone was doing internally. And not tailored to a specific domain but fully generic.
Both you, I and the other person replying to your comment were running a bespoke system like this in 2014. That's just 3 people semi-randomly meeting in a thread. If you imagine how much engineering work was saved by Kubernetes since then it's mind boggling.
Of course not all of them were as fancy as you're describing yours was, mine was just a collection of bash scripts, but they solving that set of problems was definitely a thing for many projects in 2014.