751–760 of 782 posts
Earlier quoted context omitted.
I'd like to hear why you think Python is as bad
Horrible package management story. Cross platform is a headache. Slow.
Yeah, it's a bit slow. It's faster than Javascript, though. The best benefit is flexible number size, so it's good for some types of computation.
What improvements in package management would you want? At least it has some form of management, unlike C/C++...
Earlier quoted context omitted.
Lol, of course you need a general understanding of how your database works on the inside. There are a million different ways you can store your data and I would argue that choosing your data storage is the most important and tricky decision we have to make as software engineers. Back at my first big tech company, I remember reading the best document I have ever read related to software engineering. It was entirely de…
>Lol, of course you need a general understanding of how your database works on the inside. There are a million different ways you can store your data and I would argue that choosing your data storage is the most important and tricky decision we have to make as software engineers. You aren't responding to my argument, everything you said is something I already know. So lol to you. You're making a remark and extending…
> For "SELECT * FROM TABLE", I have to understand implementation
This is not even remotely an apples-to-apples comparison. One is a fairly simple code construct that executes locally. The other is a call to a remote service.
It doesn't matter if the language you use to write it is XML, JSON, protocol buffers or SQL, any and all calls across an RPC boundary are going to have unknown performance characteristics if you don't understand how the remote service is implemented. If you are the implementer, and you still choose not to understand how it works, that's your choice, not the tool's. Every serious RDBMS comes with a TFM that you can R at any time. And there are quite a few well-known and product-agnostic resources out there, too, such as Use the Index Luke.
Alternatively, feel free to write your own alternative in C++ so that you can understand how it works in detail without having to read any manuals. It was quite a vogue for software vendors to sink a few person-years into such ventures back in the 90s. Some of them were used to build pretty neat products, too. Granted, they've all long since either migrated to a commodity DBMS or disappeared from the market, so perhaps we are due for a new generation to re-learn that lesson the hard way all over again.
Earlier quoted context omitted.
The job of these services is to transform their internal event format to 140 different output formats. You can imagine that there is a lot of duplication in the functionality that these services need to do. Are you suggesting that they avoid any shared libraries and just rewrite the same code over and over hundreds of times and update them independently?
If making a change to a shared library breaks half the services, should it have been shared in the first place? It still smells odd to me there’s so much interdependence amongst the individual services break at will.
Earlier quoted context omitted.
I wouldn't. React makes it hard to FUBAR a front end. JQuery makes it stupidly easy to write a spaghetti code unmaintainable mess.
React makes it easier for bad developers to do complicated things. They still end up writing spaghetti code.
Earlier quoted context omitted.
You’re oversimplifying. Architecturally speaking, there are some similarities between micro services and libraries because they’re both forms of modularization and usually have an API, but there are some stark differences beyond the “dispatch mechanism”. The main difference is that a service’s deployment lifecycle is completely up to the service admin. Microservices are like websites - they can continuously evolve (w…
You are assuming that microservices must be deployed on the internet, however they can be deployed on another type of network or on a single OS installation, which would prevent them from being completely transparently updated. Shared libraries can be an implementation of micro services. On a platform (e.g. Android) it can be that a shared library is updated and then all consumers are forced to update.
Services, whether SOA, web services, messaging services, or network services etc, as in SOA, describe a client/server architecture with an API. Usually these APIs are designed by the principles of Domain Driven Design, where different teams map to bounded contexts that have their own published API.
Microservices are a form of SOA where each API also runs in its own process and thus has independent deployment lifecycle. Many in the SOA world advocated for this 10-15 years ago (and often ignored), and now the industry has gone back and coined a term for this practice.
In an organization that does microservices properly at scale, like Amazon, you have teams that build and run and upgrade their service autonomously from others. Read the Steve Yegge rant about Amazon and Platforms to understand this. It allows tremendous parallelisation of effort and allows for thousands of deploys to production daily without breakage. This is hard to pull off though, and a new initiative doesn’t generally won’t more than a small handful of services.
If it’s a shared library, then call it a shared library. It has a completely different lifecycle from a microservice. In your Android example, shared libraries by definition are controlled by whomever can dictate OS updates, not the app developer.
Earlier quoted context omitted.
Interesting perspective. I think that seeking and naming patterns "microservices", "agile", etc. is useful. It provides something like a domain specific language that allows a higher level conversation to take place. The problem, as your identify, is that once a pattern has been identified people too easily line up behind it and denigrate the "contrasting" pattern. The abstraction becomes opaque. We're used to simpli…
I agree that it's useful, I even think more people should do it more often. Creating your own language (and learning other people's) is a way of having deep thoughts, not just expressing. Words for patterns (or abstract ions generally) are a quanta of language. I just think there can be downsides to them. These are theories as well as terms and they become parts of our worldview, even identity. This can engage our se…
Concepts like microservices synthesize a bunch of tradeoffs and patterns that have been worked on for decades. They’re boiled down to an architecture fad, but have applicability in many contexts if you understand them.
Similarly with Agile, it synthesizes a lot of what we know about planning under uncertainty, continuous learning, feedback, flow, etc. But it’s often repackaged into cliche tepid forms by charlatans to sell consulting deals or Scrum black belts.
Alan Kay called this one out in an old interview: https://queue.acm.org/detail.cfm?id=1039523
“computing spread out much, much faster than educating unsophisticated people can happen. In the last 25 years or so, we actually got something like a pop culture, similar to what happened when television came on the scene and some of its inventors thought it would be a way of getting Shakespeare to the masses. But they forgot that you have to be more sophisticated and have more perspective to understand Shakespeare. What television was able to do was to capture people as they were.
So I think the lack of a real computer science today, and the lack of real software engineering today, is partly due to this pop culture.”
Earlier quoted context omitted.
>Lol, of course you need a general understanding of how your database works on the inside. There are a million different ways you can store your data and I would argue that choosing your data storage is the most important and tricky decision we have to make as software engineers. You aren't responding to my argument, everything you said is something I already know. So lol to you. You're making a remark and extending…
> I'm saying that the fact that you need "a general understanding of how a database works on the inside" is a design flaw. It's a leaky abstraction. And I just said that "choosing a database is all about tradeoffs" which you need to understand (aka: the leaky abstractions). > A C++ for loop has virtually the same performance across all systems/implementations > For "SELECT * FROM TABLE", I have to understand implemen…
And I'm saying the tradeoff of using a leaky abstraction is entirely the wrong choice. A hammer vs a screwdriver each have tradeoffs but when dealing with a nail, use a hammer, when dealing with a screw use a screw driver. SQL is a hammer to a database screw.
>No you don't, it has the same performance: a for loop. However, by grouping all of your data onto 1 server, for loops are much more costly than the likely orders of magnitude more regular servers you have than a database.
See you don't even know what algorithm most SQL implementations use when doing a SELECT call. It really depends on the index but usually it uses an algorithm similar to binary search off of an index that is basically a binary search tree. It's possible to index by a hash map as well, but you don't know any of this because SQL is such a high level language. All you know is that you add an index and everything magically scales.
>Fortunately, your SQL database supports indexes which speed up those queries. Granted, I'm no database expert, but adding the right indexes and making sure your queries utilize them have solved pretty much every scaling problem I have thrown at them.
Ever deal with big data analytics? A typical SQL DB can't handle the million row multi dimensional group bys. Not even your indexes can save you here.
>It is optimizable and 90% of those optimizations I have made simply involve adding an index and then running a few explains/tests to make sure you are using them properly.
I don't have to run an EXPLAIN on any other language that I have ever used. Literally. There is no other language on the face of this planet where I had to regularly go down into the lower level abstraction to optimize it. When I do it's for a rare off case. For SQL it's a regular thing... and given that SQL exists at the bottleneck of all web development this is not just a minor flaw, but a huge flaw.
>If you'll only answer me this though, what database would you recommend than? I'm dying to know since you think you know better and google, a company that probably has more scaling problems than anyone else, doubled down on SQL with spanner which from what I have read, requires even more actual fine tuning.
I don't know if you're aware of CSS or javascript and the glaring flaws everybody complains about front end web development but it's a good analogy to what you're addressing here. Javascript and CSS are universally known to have some really stupid flaws yet both technologies are ubiquitous. No one can recommend any alternative because none exists. SQL is kind of similar. The database implementations and domain knowledge have been around so long that even alternative NO-SQL technologies have a hard time over taking SQL.
Which brings me full circle back to the front end. WASM is currently an emerging contender with javascript for the front end. Yet despite the fact that WASM has a better design then JS (not made in a week) current benchmarks against googles V8 javascript engine indicate that WASM is slower then JS. This is exactly what's going on with SQL and NO-SQL. Google hiring a crack team of genius engineers to optimize V8 for years turning a potato into a potato with a rocket booster has made the potato faster then a formula one race car (WASM)
Earlier quoted context omitted.
Re your last paragraph, you're describing Lua. I don't know the state of its coroutines though as I've never used them.
I should have mentioned that "async coroutines" (i.e., goroutines) are also a requirement. Also, I understand that Lua doesn't have much of a standard library and isn't really used as a general purpose programming language (compared to Python, for example)?
People hack the `continue` statement into it all the time, it wouldn't be terribly surprising.
Earlier quoted context omitted.
parameters could have default values or default heuristically chosen values if a default value isn't specified explicitly. Joins can be done imperatively as well. This leads to a language that is more clear and optimize-able by hand. x = binary_search(column_name=id, value=56, show_all_columns=True) y = dictionary_search(column_name=id, value=56, show_all_columns=True) z = join(x, y, joinFunc=func(a,b)(a==b)) The exa…
This is still a declarative language, just with the algorithms spelled out explicitly. (or some algorithms, as you didn't specify it for the join). However, I think specifying the algorithms in the queries is really not a good idea. Your performance characteristics can change over time (or you might now know them at all yet when you start the project). With your solution, if you, e.g. realize later that it makes sens…
Declarative yes, but unlike SQL my example is imperative. An imperative language is easier to optimize then a functional or even a expression based language (SQL) because computers are basically machines that execute imperative assembly instructions. This means the abstractions are less costly and have a better mapping to the underlying machine code.
>However, I think specifying the algorithms in the queries is really not a good idea. Your performance characteristics can change over time (or you might now know them at all yet when you start the project). With your solution, if you, e.g. realize later that it makes sense to add a new index, you'd have to rewrite every single query to use that index. With SQL, you simply add the index and are done.
Because the DB sits over a bottleneck in web development you need to have explicit control over this area of technology. If I need to do minute optimizations then am api should provide full explicit control over every detail. You should have the power to specify algorithms and the language itself should never hide from you what it's doing unless you tell choose to abstract that decision away...
What I mean by "choose to abstract that decision away" is that the language should also offer along with "binary_search" a generic "search" function as well, which can automatically choose the algorithm and index to use... That's right, by changing the nature of the API you can still preserve the high level abstractions while giving the user explicit access to lower level optimizations.
Or you can memorize a bunch of SQL hacks and gotchas and use EXPLAIN to decompile your query. I know of no other language that forces you to decompile an expression on a regular basis just to optimize it. Also unlike any other language I have seen literally postgres provides a language keyword EXPLAIN that allows users to execute this decompilation process as if they already knew SQL has this flaw. If that doesn't stand out like a red flag to you I don't know what will.
Earlier quoted context omitted.
The test of time isn't always a good metric. html, css and javascript is a huge hack, yet it has withstood the test of time. SQL to me is a huge design flaw despite it's ubiquity. On the web bottlenecks happen at IO and algorithmic searches. Databases are essentially the bottlenecks of the web and how do we handle such bottlenecks? SQL; A high level almost functional language that is further away from the metal than…
SQL is declarative and functional. It's only a matter of time before it's the new hotness.