171–180 of 189 posts
Earlier quoted context omitted.
Renames aren't atomic on crash. https://danluu.com/file-consistency/
They are if you perform the correct sequence of fsync operations on both the file and the directories, and use a file system which is correctly implemented.
Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…
Only downside is you need to format ext4 with type small otherwise you run out of inodes before you run out of disk!
I went further, to log the logical changes of state as json files. (aka command sourcing) To reduce the IO overhead, I batch multiple json values into a larger file. I don't need random access because I'll replay all the changes when the server start. Going to open source the library soon.
However we are using Postgres as a backend.
Our code is already technically Open Source, but it's not relatively stable and not doc'd yet, so I won't link.
Earlier quoted context omitted.
> So the next problem was that you couldn't query quickly for things that weren't the primary key. So new columns had to be added called "opt_key1" and "opt_key2" where certain rows could put key values, and indexes could be added on those columns, so you could quickly query by it's first optional key, or it's second optional key. It's all fun and games until you realize DynamoDB works more or less the same way: http…
"webscale"
Earlier quoted context omitted.
Doesn't every git repo have an origin branch? What is the alternative to creating a feature branch for developing something you don't want in production until it's ready?
Yep! Sorry, I meant that the only developed branch on origin is `master` (or whatever it’s called at your org). You can create branches locally, but pushing a local branch is strongly discouraged. The workflow looks something like this. git pull master; git checkout -b my-feature; ... ; git add -A; git commit At this point, you submit the code for review, and upon approval the branch is merged into master and pushed.…
Encouraging smaller changes isn't nearly as useful if those changes aren't isolated - if it's just half the picture then you can't accurately review it.
I hit a similar sort of issue recently - I've been incrementally developing a complex data migration, each change to the migration has worked on its own and been reviewed separately but I'm still going to go in and request a full review once the piece of logic is fully assembled. This is also happening on an integration branch on origin - we do try and keep these to a minimum but we're making a backwards incompatible change that would be quite expensive to do in a fully backwards compatible manner.
There are things that are infeasible to reasonably do without an integration branch (nothing is impossible technically, but it might be a huge waste of time) but even those things are pretty few and far between. If integration branches are common place at your company it might be good to examine coding practices and see if you can slice up tickets to be smaller.
Earlier quoted context omitted.
> And the first time I used couch, my programming "knowledge" was very basic HTML (no JS). That probably explains a lot: you have a hammer and everything looks like a nail. CouchDB makes no sense whatsoever for the requirements described.
> you have a hammer and everything looks like a nail. Actually, that comment made me see things differently on a project that had me scratch my head for the last couple of weeks, so thanks a lot! Don't know if it's related but I tend to feature creep.
Earlier quoted context omitted.
There is nothing bad about writing your own solutions. What is bad is putting them in production when you don't have a clue about the domain.
Being ignorant didn't make you a prima donna tho, as above says. Also, they have to have some clue about the domain, because the domain is their own problem and they're writing a solution for it. So I don't think we can really just someone as not having any clue about their own engineering challenges.... especially if they're working solutions to them.... Antirez said literally he didn't know about existing solutions…
They can be lifelong experts on their problem, yet have no clue about writing a database engine and low-level programming in general.
> Antirez said literally he didn't know about existing solutions when he went to write redis
Nobody is born with knowledge. The difference is that Antirez studied previous solutions, studied how to do it, and then applied that knowledge right.
Instead, that person did the equivalent of building a bridge disregarding everything humans learnt about it since the Roman empire. It will not be a surprise if the bridge ends up collapsing.
Earlier quoted context omitted.
Yep! Sorry, I meant that the only developed branch on origin is `master` (or whatever it’s called at your org). You can create branches locally, but pushing a local branch is strongly discouraged. The workflow looks something like this. git pull master; git checkout -b my-feature; ... ; git add -A; git commit At this point, you submit the code for review, and upon approval the branch is merged into master and pushed.…
Ouf that last bit makes me hurt. Encouraging smaller changes isn't nearly as useful if those changes aren't isolated - if it's just half the picture then you can't accurately review it. I hit a similar sort of issue recently - I've been incrementally developing a complex data migration, each change to the migration has worked on its own and been reviewed separately but I'm still going to go in and request a full revi…
But, I do believe it pays off in the form of a higher quality end-product (fewer bugs, more testable/legible components, more extensible), which saves you time in the long run.
Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…
https://www.cs.ait.ac.th/~on/O/oreilly/perl/cookbook/ch07_09...