41–50 of 621 posts
Earlier quoted context omitted.
Good lesson on making command prompts on machines always tell you exactly what machine you're working on.
I like to color code my terminal. Production systems are always red. Dev are blue/green. Staging is yellow.
Earlier quoted context omitted.
Yes, those are the initials of an employee here. Sorry for the confusion!
As much as I appreciate GitLabs extreme openness, that's maybe something that by policy shouldn't be part of published reports. Internal process is one thing, if something goes really bad customers might not be so good at "blameless postmortems" if they have a name to blame.
"So in other words, out of 5 backup/replication techniques deployed none are working reliably or set up in the first place."
Does this mean whatever was in that database is gone, with no available backups? Is this an SOA where important data might lie in another service or data store, or is this a monolithic app and DB that is responsible for many (or all) things? What was stored in that database? Does this affect user data? Code?
23:00-ish YP thinks that perhaps pg_basebackup is being super pedantic about there being an empty data directory, decides to remove the directory. After a second or two he notices he ran it on db1.cluster.gitlab.com, instead of db2.cluster.gitlab.com Good lesson on the risks of working on a live production system late at night when you're tired and/or frustrated.
Good lesson on making command prompts on machines always tell you exactly what machine you're working on.
Earlier quoted context omitted.
Good lesson on making command prompts on machines always tell you exactly what machine you're working on.
I like to color code my terminal. Production systems are always red. Dev are blue/green. Staging is yellow.
i2-badge ()
{
printf "\e]1337;SetBadgeFormat=%s\a" $(echo -n "$1" | base64)
}
23:00-ish YP thinks that perhaps pg_basebackup is being super pedantic about there being an empty data directory, decides to remove the directory. After a second or two he notices he ran it on db1.cluster.gitlab.com, instead of db2.cluster.gitlab.com Good lesson on the risks of working on a live production system late at night when you're tired and/or frustrated.
Also, as a safety net, sometimes you don't need to run `rm -rf` (a command which should always be prefaced with 5 minutes of contemplation on a production system). In this case, `rmdir` would have been much safer, as it errors on non-empty directories.
Earlier quoted context omitted.
I think it's a staff member. Can't remember first name, Yuri maybe, who is fairly active with the project.
Nope, that would be me.
I noticed the issue when I was pushing code earlier today. Hopefully this gets resolved soon. You guys are doing a great job. Keep up the good work!
I noticed the issue when I was pushing code earlier today. Hopefully this gets resolved soon. You guys are doing a great job. Keep up the good work!
"So in other words, out of 5 backup/replication techniques deployed none are working reliably or set up in the first place."
Does this mean whatever was in that database is gone, with no available backups? Is this an SOA where important data might lie in another service or data store, or is this a monolithic app and DB that is responsible for many (or all) things? What was stored in that database? Does this affect user data? Code?
Regarding what's gone: The production PostgreSQL database. This suggests that the code itself is fine, but the mappings to the users are gone. But git is a distributed VCS after all, so all the code should be on the developer's machines as well.