721–730 of 816 posts
Earlier quoted context omitted.
I've almost entirely replaced Pandas with DuckDB in my day-to-day work. I wonder if it would be an easier lift for someone who is familiar with SQL, and doesnt want to pick up Pandas/Polars.
This comment might finally push me to try DuckDB, thanks. I’m quite proficient with scientific python in general, but Pandas API is just alien (and slow). SQL is more natural for its domain.
Earlier quoted context omitted.
At least for Python dependency management, I'd hardly call it a mess these days and yes it was a horrible 10 years ago. The de-facto standard of venv and pip (using a requirements.txt file) is generally painless these days. To the extent that moving between MacOS, Linux and Windows is feasible for most (of my) Python work. And the Python 2/3 pain stopped being an issue a few years back. I haven't been forced because…
I started using Dagster a few days ago, which runs on Python. As far as I can tell, I have to install the packages with pip. Wait - don't forget venv first. Oh but I'm not supposed to commit the venv, so how is anybody to know what I did? Okay I have to setup a requirements.txt... but now what did I install in the venv? Okay there's a way to dump the venv.. but it's giving me all the dependencies, not the root packag…
People always under-estimate the cost of properly learning a language. At any given time I tend to have a "main go-to language". I typically spend 2-4 years getting to the point where I can say I "know" a language. Then I try to stick to it long enough for the investment to pay off. Usually 8-10 years. A surprising number of people think this is a very long time. It isn't. This is typically the time it takes to under…
And then you have the ever changing ecosystem, that can take years so sort out on it's own and must be constantly studied. E.G: if you arrive in Python right now, numpy is in version 2 and polars is stable. uv is all the rage, pydantic gained so much perf it's not even funny, toml is part of the stdlib and textual is looking very good. Type hints are much better than 2 years ago, htmx is used a lot in the web departm…
The author feels happy and contented and you all are getting defensive. He’s a solo dev, just recognize that that scopes his projects and stop trying to shoot him down. Use what you want.
Earlier quoted context omitted.
In which way do you believe the JVM/language has been mismanaged?
Oracle requiring commercial licensing for their JVM has made it so radioactive that my workplace firewalls "oracle.com" to prevent anyone from accidentally using it.
Earlier quoted context omitted.
I personally think that writing so called "terse, clever" (misnomer) code, is not an issue with the language, rather the user. Do we really want to have worse tools, just because some people are writing bad code? Clearly it's an issue with the software engineering process rather than language itself. A good language should allow a skilled user to write code as clear as day, while properly modelling the problem domain…
> Do we really want to have worse tools, just because some people are writing bad code? People tend to write bad code. It's a fact of life. Tools forcing people who write bad code to write better code can't be worse tools by definition. They are better tools.
Earlier quoted context omitted.
When do you ship notebooks to production? Jupiter was never meant for external clients.
While terrifying, it is not uncommon to see python notebooks make it to production.
Earlier quoted context omitted.
When do you ship notebooks to production? Jupiter was never meant for external clients.
Usually what happens is: 1. "We got it all working in the notebook" 2. "Great, ship it" 3. (data scientist takes the notebook code almost verbatim, wraps it in a basic CLI or HTTP API and it gets shipped off in a docker container for other services to consume)
If you ever want to work in teams that kind of setup works extremely well.
Earlier quoted context omitted.
> What are Go programmers doing that they don’t feel the need for map/filter/etc? As a refugee from a scala project that went badly (we eventually ported the entire thing to go), it's not so bad when you're just using map and filter and friends. But eventually there's so many of those little methods each with their own nuances and I don't want to have to remember them all (`sliding` comes to mind), and it's just exha…
Thank you for sharing your experience and I am sure it is right for you. And I agree that Scala can be too much. I just wanted to point out that there is a fundamental reason that for loops are inferior to map/reduce when working with data. For loops is "how" where map/reduce is "what" and that puts the burden on you when you need to parallelize your job. Joel Spolsky described it here (long ago) https://www.joelonso…
> For loops is "how" where map/reduce is "what" and that puts the burden on you when you need to parallelize your job.
That's not inferior. That's one advantage that map/reduce has in the set of trade-offs between them. I'm aware of that argument. It's part of what I was trying to get at with the "less powerful" comment (the other side of the coin of being less powerful/more constrained is that it's easier to parallelize).
But ~95% of code never makes it to that step. And I've found that defaulting to it leads to worse code bases, instead of defaulting to the simple intuitive thing and accepting the burden for the other 5% of the code.
Because the burden of that 5% of the code is never the biggest part of scaling. It's usually something like getting the business to understand the implications of CAP theorem and figuring out what trade-offs are best for them. The code is pretty easy, by comparison. Even if I have to chuck out 5% of the files and re-write them over a period of time.
I guess my recommendation is basically to do things that don't scale in your code, because 95% of code won't need to, and the other 5% has a good chance of being re-written anyway during the process of scaling.
Earlier quoted context omitted.
If you want to progress your career you'll need to take on hard problems at some point. Go isn't particularly unique in excelling at easy problems.
Go doesn't excel at easy problems. Go is fine at pretty much everything. Do you think Kubernetes is an easy problem? The thing is just that Go is very opiniated in its feature set. That's you see people here writing about complex projects using "wild" or even "god awful" things, and lament the inability to properly map API boundaries in the language. The truth is obviously that all of these is not particularly wild.…
Kubernetes is an easy problem made hard by doing a bunch of things that don't need to be done. I've used small bash scripts to deploy software for most of my freelance career, and the few times I've been forced to use a containerization tool, it has been far more difficult, for no discernible benefit.
> The thing is just that Go is very opiniated in its feature set. That's you see people here writing about complex projects using "wild" or even "god awful" things, and lament the inability to properly map API boundaries in the language.
The problem isn't that Go is opinionated--I often wish Python was more opinionated. The problem is that Go started off with the wrong opinion on generics, and took two iterations (first casts, then `go generate`) to arrive at generics, resulting in a system that isn't opinionated on this issue, because all three ways work for reverse compatibility. And this is a) a very important issue to be opinionated on, and b) extremely forseeable given the languages that came before.
> The truth is obviously that all of these is not particularly wild. It's just things that the commenter considers inelegant but is perfectly able to follow which is Go strength and why it's so code. Want it or not, you will have to write code that someone else can follow.
The lack of abstractions means it's easy to follow on the line by line level, but that falls apart as context grows. Lines in more powerful languages are harder to follow because they do more. If you want to do the same amount of work in Go, you have to write more lines of code. You're going to be implementing the same abstractions ultimately, but because you're writing it custom you're going to do it a little differently every time. As a result, any few lines of code in Go is easy to understand, but the big picture is much harder to understand, because you're caught up in minutia which is slightly different every time instead of using a standard abstraction.
EDIT: There's no way the first person who downvoted this had time to read it.