121–130 of 259 posts
> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…
Earlier quoted context omitted.
OFFTOPIC: "Today, I want to share some recent learnings from performance tuning this new application stack." The word you want is "lessons". Jeez, not only is your comment offtopic and pedantic, it's also wrong.
I flagged it as Offtopic. But it is correct. "Learnings" is at best a pointless affectation, like saying "utilised" instead of "used". At worst, according to the OED, it's not even a word. http://english.stackexchange.com/questions/19227/plural-of-l... Also: http://www.merriam-webster.com/dictionary/learning No plural option.
No, at best it's a deliberately wonky usage for comic effect [1], like "Internets". It's quite possible the author is being gamesome. [2]
[1] https://simple.wikipedia.org/wiki/Borat:_Cultural_Learnings_...
> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…
Of course there's a faster way! Combine all the routes into a DFA, then run the DFA over the URL. It's guaranteed to run in constant space and O(n) (n=URL length) time! The union of any set of regular languages is itself a regular language. You can use Ragel[1] to build your automaton. [1] http://www.colm.net/open-source/ragel/
Nice idea.
Edit: I did consider rewriting the runtime as a forth style interpreter as well but the time never appeared.
Earlier quoted context omitted.
Running (uncompiled?) regular expressions, it seems.
V8 compiles regular expressions to machine code.
(To explain: obviously C++ churns out machine code, the parent was talking about compiled vs uncompiled regexes – if I'm not terribly wrong, the compilation step is turning the regex into a finite automaton.)
Earlier quoted context omitted.
> "not good when your bottleneck is CPU" 99% of cases, your bottleneck will be I/O. In the 1% of cases where your webserver's bottleneck is the CPU, you have much bigger problems than using a hipster ;) language: you're doing it wrong (tm) on an architectural level: (a) Your processor-intensive/long-running tasks need to be in seperate worker processes and (b) you need more webserver instances. I usually try to avoid…
I shouldn't have tried to be funny on HN* . For this I apologize. Can I get a rebuttal to my points along with the downvotes, though? Is someone disputing that CPU intensive tasks should be moved off the webserver? Have you personally had experiences where your honest-to-$deity webserver bottleneck was the CPU? I'd be incredibly surprised. In most cases I'd guess you're underprovisioned and/or badly architected. * Th…
1. Your networking pipe into the server is slow.
2. You don't have a reverse proxy in front of the webserver to buffer the slow connections.
3. You don't make decisions about what information to show the user in the webserver; your UI isn't tailored for the particular user whose request you're servicing.
4. You haven't made productivity trade-offs to move work from the developer to the computer.
Google's webservers (for Search) are CPU-bound, and I can guarantee that they are neither underprovisioned nor badly architected, and CPU-intensive tasks are moved off the webserver (the total amount of CPUs for webservers is miniscule compared to the number of CPUs used for the indexing & serving system). But Google has the fastest fiber networks in the world, a massive load-balancing infrastructure, its philosophy is to serve each user only the HTML/JS that they need (to cut latency), it relies heavily on A/B tests & experimentation, and it invests significantly in developer-productivity tools so that all the bookkeeping for the last two points are done by computers rather than humans.
OFFTOPIC: "Today, I want to share some recent learnings from performance tuning this new application stack." The word you want is "lessons".
Earlier quoted context omitted.
> It it is non-trivial (not possible?) to do this in O(1) for routes that use matching / wildcards I'd be impressed if they did it consistently in O(1) for static routes. I think they were looking for O(log(number of different routes)) instead of O(n).
If the paths have some kind of non crazy Regex leading up to what gets munched for path variables (eg /path1 versus /path2) then you could at least build a tree of maps at each level, which would be roughly constant time for the most common cases.
For small values of n ;)
A tree of maps would be consistently log(n), like any map. Even a hashtable would hash to buckets eventually, and are log(n).
an unfortunate title. Ha ha "flames" ha ha "Node.js" but the article is really about express. Not so "ha ha"
For the audience, this was originally titled: "Node.JS in Flames"
"We made incorrect assumptions about the Express.js API without digging further into its code base. As a result, our misuse of the Express.js API was the ultimate root cause of our performance issue."
This situation is my biggest challenge with software these days. The advice to "just use FooMumbleAPI!" is rampant and yet the quality of the implemented APIs and the amount of review they have had varies all over the map. Consequently any decision to use such an API seems to require one first read and review the entire implementation of the API, otherwise you get the experience that NetFlix had. That is made worse by good APIs where you spend all that time reviewing them only to note they are well written, but each version which could have not so clued in people committing changes might need another review. So you can't just leave it there. And when you find the 'bad' ones, you can send a note to the project (which can respond anywhere from "great, thanks for the review!" to "if you don't like it why not send us a pull request with what you think is a better version.")
What this means in practice is that companies that use open source extensively in their operation, become slower and slower to innovate as they are carrying the weight of a thousand different systems of checks on code quality and robustness, which people using closed source will start delivering faster and faster as they effectively partition the review/quality question to the person selling them the software and they focus on their product innovation.
There was an interesting, if unwitting, simulation of this going on inside Google when I left, where people could check-in changes to the code base that would have huge impacts across the company causing other projects to slow to a halt (in terms of their own goals) while they ported to the new way of doing things. In this future world changes, like the recently hotly debated systemd change, will incur costs while the users of the systems stop to re-implement in the new context, and there isn't anything to prevent them from paying this cost again and again. A particularly Machievellan proprietary source vendor might fund programmers to create disruptive changes to expressly inflict such costs on their non-customers.
I know, too tin hat, but it is what I see coming.