351–360 of 369 posts
Earlier quoted context omitted.
The article mentions connection, so it’s the connection the DB rather than the data. Ie the thing you get after you open the socket.
Hence the batching.
Earlier quoted context omitted.
> Most other programming language package managers don't, see Maven for Java. So it’s not able to express native extensions or even codegen then? I suppose that’s OK for something that’s not the only available solution, but I don’t think I’d love it, either. > But then again, NONE of the scripting languages ever wanted to learn stuff from Java, especially regarding packaging[.] Both Java and its docs are just extreme…
Well, they didn't need to read much about Java in this case, frankly. Just creating a simple Java project with Maven would have showed the groupId concept (namespaces preventing top level squatting), for example. https://maven.apache.org/guides/getting-started/index.html#h... Anyway, too late now, now Python & co. are finding their own, alternative, ways to retrofit stuff like this.
I think that is extremely short sighted, and that all packages including standard ones should be namespaced.
Earlier quoted context omitted.
> Small teams. And ones that work collaboratively, not ones that than carve the code up into bailiwicks so that they can mostly work in mini-silos. Well, that may explain some of the difference between our points of view. Most of my experience is with medium ( 500 developers) applications. At some point, no matter how cooperative the team is, the amount of complexity that you can hold in your head is not sufficient t…
Sure but at that point we're probably not on the first iteration of code anyway. Even at a big tech company, I find it most effective to make a POC first-iteration that you prove out in a development or staging environment that uses the map-of-heterogeneous-types style development. Once you get the PMs and Designers onboard, you'll iterate through it until the POC is in an okay state, and then you turn that into a fi…
1. Iterate on early prototype.
2. Show prototype to stakeholders.
3. Stakeholders want more features. At best, one dev has a little time to tighten a few bolts here and there while working on second prototype.
4. Show second prototype to stakeholders.
5. Stakeholders want more features. At best, one dev has a little time to tighten a few bolts here and there while working on third prototype.
6. etc.
Of course, productivity decreases with each iteration because as things progress (and new developers join or historical developers leave), people lose sight of what every line of code means.
In the best case, at some point, a senior enough developer gets hired and has enough clout to warrant some time to tighten things a bit further. But that attempt never finishes, because stakeholders insist that new features are needed, and refactoring a codebase while everybody else is busy hacking through it is a burnout-inducing task.
Earlier quoted context omitted.
I realize that this is the common wisdom these days: write code that works just well enough that we have a chance to fix before it does too much damage whenever it breaks. I suspect that this approach is strongly fueled by the unlimited VC money available in tech, since it means that any company can employ an unlimited number of full-time developers (and PR) just to handle catastrophes. We'll see how that wisdom hold…
> I suspect that this approach is strongly fueled by the unlimited VC money available in tech, Well, I suppose we could trade anecdotes and counter-examples, but my position largely comes from my own experience rather than received wisdom (though there's plenty of that). Instead I'll just say that I disagree, largely because because a business is a complex and shifting arrangement of various factors competing for lim…
Earlier quoted context omitted.
On the one side, yes Python is incredibly versatile. On the other side, I have worked on many Python projects, some of them fairly high profile, and I have seen exactly two kinds of Python codebases: 1. a few were written by extreme professionals, plugging at every single hole, with ~100% coverage, plus considerable maintenance because every dependency upgrade tends to break something; 2. many that feel cobbled toget…
>Option 1. feels like "writing Python like it's Rust", but of course without any of the benefits of Rust either on performance or on safety. ...but with the benefits of Python. You might not see them/need them but they are there :)
Tooling has come a long way in the past few years to help support better Python. Nowadays I can just throw on Pydantic V2 (still in Alpha, but stable to code against for basic use cases), ruff and mypy. Turn strict mode on in mypy and install the relevant vscode extensions and you get an experience that rivals a compiled language in both speed and usability.
In terms of usability, I feel that getting a Python codebase to a point where typing can actually be relied upon requires much, much, much more work than writing the same code in Rust or any other language with strong, static typing. And mypy is fragile enough that you can fairly easily sneak code that will break the guarantees without any warning, so even when I write "typing can actually be relied upon", I feel I'm…
In rust you need an external dependency for anything. And of course external dependencies can be unmaintained, have vulnerabilities, change API every week…
Earlier quoted context omitted.
Sure but at that point we're probably not on the first iteration of code anyway. Even at a big tech company, I find it most effective to make a POC first-iteration that you prove out in a development or staging environment that uses the map-of-heterogeneous-types style development. Once you get the PMs and Designers onboard, you'll iterate through it until the POC is in an okay state, and then you turn that into a fi…
Do you work in an organization that does this? Because most organizations I've seen who don't pick the approach of "write it like it's Rust" rather have the following workflow. 1. Iterate on early prototype. 2. Show prototype to stakeholders. 3. Stakeholders want more features. At best, one dev has a little time to tighten a few bolts here and there while working on second prototype. 4. Show second prototype to stake…
Yup! I'm at a company that used to be a startup and ended up becoming Big Tech (over many years, I'm a dinosaur here.) Our initial phase involved building lots of quick-and-dirty services as we were iterating very quickly. These services were bad and unreliable but were quick to write and throwaway.
From there we had a "medium" phase where we built a lot of services in more strictly typed languages that we intended on living longer. The problem we encountered in this phase was that no matter the type safety or performance, we started hitting issues from the way our services were architected. We started putting too much load on our DBs, we didn't think through our service semantics properly and started encountering consistency issues/high network chatter, our caches started having hotspotting issues, our queues would block on too much shared state, etc, etc.
We decided to move to a model that's pretty common across Big Tech of having senior engineers/architects develop a PoC and using that PoC to shop around the service. For purely internal services with constrained problem domains and infrequent changes, we'd usually skip this step and move directly to a strictly typed, high performance language (for us that's Java or Go because we find them able to deal with ===
I have friends who work/have worked in places like you describe but a lot of them tell me that those shops end up in a morass of tech debt over time anyway and eventually find it very difficult to hire due to the huge amount of tech debt and end up mandating huge rewrites anyway.
Earlier quoted context omitted.
https://dropbox.tech/infrastructure/rewriting-the-heart-of-o...
And the fact that they're spending so much time on it and were afraid to rewrite it tells you all you need to know. Rewriting it in C++, leveraging the platform-specific APIs for disk I/O (sync_file_range on Linux, not even wrapped by a Python module) would yield not only much better performance, but also much higher reliability. As it is this daemon is a recipe for page thrashing. Not that I would expect a Python de…
You ship C code which is extensively tested by Python code.
Python is simple to use, got good C interop and the performance of your test frameworks hardly matters.
Earlier quoted context omitted.
> The whole point of Python is the duck typing. Duck typing implies that "Should this work?" has one true answer: "Did it work?" I am at work to work, not to play guess-and-check with upstream libraries.
When you couple that with an interpreted language with no compile time, finding the answer to did it work is often faster than should it work. Using Duck typing in a scripting language is the correct choice.
"Did it work?" is a point-in-time observation. It is always situational. To find out, I have to construct esoteric objects with unobservable internal state and hope it is the state that I actually care about.
"Should it work?" has an answer of infinite duration. I can recall what I learned years later without needing to re-run some experiment.
Earlier quoted context omitted.
When you couple that with an interpreted language with no compile time, finding the answer to did it work is often faster than should it work. Using Duck typing in a scripting language is the correct choice.
(Sure, no compile time. But how much time do you think humanity collectively wastes waiting for CPython to import things? On. Every. Run.) "Did it work?" is a point-in-time observation. It is always situational. To find out, I have to construct esoteric objects with unobservable internal state and hope it is the state that I actually care about. "Should it work?" has an answer of infinite duration. I can recall what…