21–30 of 54 posts
Earlier quoted context omitted.
I doubt that exactly that will work. There are tens of thousands of different UAs (maybe 100K). Perhaps some kind of tiny (few CPU cache lines) cache for most popular UAs could help. But again: measure, measure, measure :)
Publish your test set and I can look at it :)
A lot of manual work with various perf tools.
What's a bit missing is some production performance monitoring (APM) that gives you such data, with no manual interaction.
Earlier quoted context omitted.
I have that feeling too. But I found it harder to implement, especially with fallback failure references.
I'd look for a library and not write one from scratch. Lucene [1] and OpenFST [2] are great implementations. I haven't used C#, so I don't know if bindings exist or not. Also you may find this talk useful [3] (Particularly slide 11). Great write up by the way. Really thorough on the benchmarking! [1] https://lucene.apache.org/core/4_1_0/core/org/apache/lucene/... [2] http://www.openfst.org/twiki/bin/view/FST/WebHome…
Excellent post showing how to correctly improve code w.r.t. performance using the scientific method: hypothesis, measuring the baseline, change, measuring effect with real tools, real code. Thanks for sharing!
I also found it an interesting post in that it kind of inadvertently proves that for most situations you shouldn't optimise to this extent. Meaning, yes, the OP got impressive performance improvements but the code is also completely unreadable and utilises unsafe code sections which could expose you to security problems/memory leaks/memory corruption. Not to mention they've recreated and will need to maintain an in-h…
I cannot believe that you are honestly saying a 2x increase to throughput in production is something you "shouldn't take seriously" because the code isn't as readable as it was before.
Programmers are expensive. Hardware is cheap. That doesn't justify completely throwing out the window any performance increasing changes just because a fresh college grad won't be able to understand what's going on within 10 minutes.
Good post! A lot of manual work with various perf tools. What's a bit missing is some production performance monitoring (APM) that gives you such data, with no manual interaction.
Blocking access based on arbitrary user agent strings is a really bad idea. Every single bad bot will avoid known user agent strings or pretend to be Google, so you're only blocking well behaved ones. Plus there's thousands of browser versions out there, so there's a very good chance you're blocking some users for no reason. The proper way to do this is to block by IP, based on behavior. Block IPs slowing down the si…
Disclaimer: Getting rid of those idiots^wmisguides poor souls is part of my job description.
Excellent post showing how to correctly improve code w.r.t. performance using the scientific method: hypothesis, measuring the baseline, change, measuring effect with real tools, real code. Thanks for sharing!
I also found it an interesting post in that it kind of inadvertently proves that for most situations you shouldn't optimise to this extent. Meaning, yes, the OP got impressive performance improvements but the code is also completely unreadable and utilises unsafe code sections which could expose you to security problems/memory leaks/memory corruption. Not to mention they've recreated and will need to maintain an in-h…
what if the "grey" traffic came from residential IP addresses using a normally distributed range of user agents? How would you reliable distinguish them from regular traffic?
Technical : if the UserAgent claim to be a regular browser (let say Chrome 43) we will check on network level if the client implement http protocol like Chrome 43 usually do and on the JS side if the Javascript render is correct for Chrome. In case it's a real Chrome, we will check if the Browser is controlled by automation Tool.
Behavior : we will check if the path of requests is regular according to the website usage.
Disclaimer: I'm working at https://datadome.co, a bot protection tool.
what if the "grey" traffic came from residential IP addresses using a normally distributed range of user agents? How would you reliable distinguish them from regular traffic?