Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

100M-Row Challenge with PHP

github.com

Post reply on HN
Print view View original

31–40 of 104 posts

Re: 100M-Row Challenge with PHP

#31

Post by Joel_Mckay »

gib444 wrote: post #5
brentroose wrote: post #2

A month ago, I went on a performance quest trying to optimize a PHP script that took 5 days to run. Together with the help of many talented developers, I eventually got it to run in under 30 seconds. This optimization process with so much fun, and so many people pitched in with their ideas; so I eventually decided I wanted to do something more. That's why I built a performance challenge for the PHP community The goal…

> A month ago, I went on a performance quest trying to optimize a PHP script that took 5 days to run. Together with the help of many talented developers, I eventually got it to run in under 30 seconds That's a huge improvement! How much was low hanging fruit unrelated to the PHP interpreter itself, out of curiosity? (E.g. parallelism, faster SQL queries etc)

In general, it is bad practice to touch transaction datasets in php script space. Like all foot-guns it leads to Read-modify-write bugs eventually.

Depending on the SQL engine, there are many PHP Cursor optimizations that save moving around large chunks of data.

Clean cached PHP can be fast for REST transactional data parsing, but it is also often used as a bodge language by amateurs. PHP is not slow by default or meant to run persistently (low memory use is nice), but it still gets a lot of justified criticism.

Erlang and Elixir are much better for clients/host budgets, but less intuitive than PHP =3

Re: 100M-Row Challenge with PHP

#32

Post by Xeoncross »

codegeek wrote: post #25
Xeoncross wrote: post #23

This is why I jumped from PHP to Go, then why I jumped from Go to Rust. Go is the most battery-included language I've ever used. Instant compile times means I can run tests bound to ctrl/cmd+s every time I save the file. It's more performant (way less memory, similar CPU time) than C# or Java (and certainly all the scripting languages) and contains a massive stdlib for anything you could want to do. It's what scripti…

I am not that smart to use Rust so take it with a grain of salt. However, its syntax just makes me go crazy. Go/Golang on the other hand is a breath of fresh air. I think unless you really need that additional 20% improvement that Rust provides, Go should be the default for most projects between the 2.

I hear you, advanced generics (for complex unions and such) with TypeScript and Rust are honestly unreadable. It's code you spend a day getting right and then no one touches it.

I'm just glad modern languages stopped throwing and catching exceptions at random levels in their call chain. PHP, JavaScript and Java can (not always) have unreadable error handling paths not to mention hardly augmenting the error with any useful information and you're left relying on the stack trace to try to piece together what happened.

Re: 100M-Row Challenge with PHP

#33

Post by thinkingtoilet »

Xeoncross wrote: post #23

This is why I jumped from PHP to Go, then why I jumped from Go to Rust. Go is the most battery-included language I've ever used. Instant compile times means I can run tests bound to ctrl/cmd+s every time I save the file. It's more performant (way less memory, similar CPU time) than C# or Java (and certainly all the scripting languages) and contains a massive stdlib for anything you could want to do. It's what scripti…

It's almost comical how often bring up Rust. "Here's a fun PHP challange!" "Let's talk about Rust..."

Re: 100M-Row Challenge with PHP

#34

Post by Xeoncross »

thinkingtoilet wrote: post #33
Xeoncross wrote: post #23

This is why I jumped from PHP to Go, then why I jumped from Go to Rust. Go is the most battery-included language I've ever used. Instant compile times means I can run tests bound to ctrl/cmd+s every time I save the file. It's more performant (way less memory, similar CPU time) than C# or Java (and certainly all the scripting languages) and contains a massive stdlib for anything you could want to do. It's what scripti…

It's almost comical how often bring up Rust. "Here's a fun PHP challange!" "Let's talk about Rust..."

Sorry, but it's honestly just a lot of our journeys. Started on scripting languages like PHP/Ruby/Lua (self-taught) or Java/VB/C#/Python (collage) and then slowly expanded to other languages as we realized we were being held back by our own tools. Each new language/relationship makes you kick yourself for putting up with things so long.

Re: 100M-Row Challenge with PHP

#35

Post by daviddoran »

semiquaver wrote: post #26

Are they just confused about what characters require escaping in JSON strings or is PHP weirder than I remember? { "\/blog\/11-million-rows-in-seconds": { "2025-01-24": 1, "2026-01-24": 2 }, "\/blog\/php-enums": { "2024-01-24": 1 } }

PHP has always escaped forward slashes to help prevent malicious JSON from injecting tags into JavaScript I believe. Because it was common for PHP users to json_encode some data and then to write it out into the HTML in a script tag. A malicious actor could include a closing script tag, and then could inject their own HTML tags and scripts etc.

Re: 100M-Row Challenge with PHP

#36

Post by thinkingtoilet »

Xeoncross wrote: post #34
thinkingtoilet wrote: post #33

Earlier quoted context omitted.

It's almost comical how often bring up Rust. "Here's a fun PHP challange!" "Let's talk about Rust..."

Sorry, but it's honestly just a lot of our journeys. Started on scripting languages like PHP/Ruby/Lua (self-taught) or Java/VB/C#/Python (collage) and then slowly expanded to other languages as we realized we were being held back by our own tools. Each new language/relationship makes you kick yourself for putting up with things so long.

I understand that but there's a time and a place. Rust has nothing to do with this. 100% of the people on this site understand that this challenge can be done faster in C, or Rust, or whatever. This is a PHP challenge. Perhaps we could discuss the actual submission as opposed to immediately derailing it.

Re: 100M-Row Challenge with PHP

#37

Post by tosti »

swasheck wrote: post #19
tiffanyh wrote: post #15

Earlier quoted context omitted.

Aren’t these optimizations less about PHP, and more about optimizing how your using the database.

in all my years doing database tuning/admin/reliability/etc, performance have overwhelmingly been in the bad query/bad data pattern categories. the data platform is rarely the issue

The worst offenders I've seen were looping over a shitty ORM

Re: 100M-Row Challenge with PHP

#38

Post by onion2k »

brentroose wrote: post #2

A month ago, I went on a performance quest trying to optimize a PHP script that took 5 days to run. Together with the help of many talented developers, I eventually got it to run in under 30 seconds. This optimization process with so much fun, and so many people pitched in with their ideas; so I eventually decided I wanted to do something more. That's why I built a performance challenge for the PHP community The goal…

A month ago, I went on a performance quest trying to optimize a PHP script that took 5 days to run. Together with the help of many talented developers, I eventually got it to run in under 30 seconds.

When people say leetcode interviews are pointless I might share a link to this post. If that sort of optimization is possible there is a structures and algorithms problem in the background somewhere.

Re: 100M-Row Challenge with PHP

#40

Post by kpcyrd »

olmo23 wrote: post #22
kpcyrd wrote: post #20

Earlier quoted context omitted.

What about using the filesystem as an optimized dict implementation?

this is never going to be faster because it requires syscalls

The time you lose at the syscall boundary you may be able to win back during much shorter GC pauses.
Post reply on HN