Live data from Hacker News

How big are PHP arrays (and values) really?

nikic.github.com

71–80 of 90 posts

Re: How big are PHP arrays (and values) really?

#71
post #66

Earlier quoted context omitted.

Considering PHP's scope and its limitations (lacks things like threads for example) I would say that is an extremely rare scenario. If you're storing 100.000 integers on a PHP data structure you're most likely doing it wrong.

> Considering PHP's scope and its limitations (lacks things like threads for example) Please look up PHP-FPM. I use it in production, with great success. > If you're storing 100.000 integers on a PHP data structure you're most likely doing it wrong. What if I had a ton of price points and I needed to do statistical analysis? Well, those would be floats, but you get my point. PHP is capable of doing the work efficient…

How would you evoke such code? Squeezing it into an http reply? Calling it from PHP CLI?

If doing it on your webapp, don't! You're doing it wrong. Otherwise why would you need php specifically? I mean I guess you could, but what advantage does it poses comparing to other languages available in pretty much every *nix system nowadays (perl, python)?

For the record, I have a nice collection of downvotes for defending PHP on point-and-laugh-at-PHP threads here at HN. I just don't see how the use case you're presnting is realistic.

Re: How big are PHP arrays (and values) really?

#72
post #15
post #9

Earlier quoted context omitted.

It's funny. I don't have much experience with php, and I had to actually run your last example to know that it appended the values (instead of recreating the array with a single value each time). Is there an append operator, or something more explicit ? '+=' seems to do something strange..

[] IS the append operator. And += only works right if both sides are arrays. If not the right side will be converted. += is really intended for hashmaps, not integer indexes, for those use [].

+= does union-by-key: any keys present in the right array, and not in the left, are appended to the left.

`array_merge` will append all numeric keys from the right array to the left, under new key values, as if you used [] one-by-one; for string keys, all keys from the right are copied into the left, possibly overwriting what was there.

In real code, I either have all-numeric or all-string keys, and `array_merge` does what I want in bulk operations: it's essentially equal to Python list.extend and dict.update, respectively. `+` on arrays is basically useless for me.

Re: How big are PHP arrays (and values) really?

#73
post #66

Earlier quoted context omitted.

> Considering PHP's scope and its limitations (lacks things like threads for example) Please look up PHP-FPM. I use it in production, with great success. > If you're storing 100.000 integers on a PHP data structure you're most likely doing it wrong. What if I had a ton of price points and I needed to do statistical analysis? Well, those would be floats, but you get my point. PHP is capable of doing the work efficient…

Isn't PHP-FPM a fastcgi process manager? I don't see any references to threading in it's documentation. I also recall the php documentation saying php is unsafe with multiple threads due to a large number of libraries that are not coded to be thread-safe.

> Isn't PHP-FPM a fastcgi process manager

It is.

> I don't see any references to threading

Wow. I always thought it was threaded. My mistake. It seems it launches multiple child processes. There is some form of memory sharing going on, though. Looks like it's some sort of a hybrid. Now I'm just thoroughly confused ...

Re: How big are PHP arrays (and values) really?

#74
post #20

Earlier quoted context omitted.

I think they are a little more complex than regular hash maps. 1) They keys can be integers or unicode strings (ordering works differently for these two cases) 2) The keys are not necessarily ordered (see ksort and krsort functions, for example).

They are ordered by insertion. That's different from for instance objects in Javascript.

Not really

    $arr = array();
    $arr[0] = 'cat';
    $arr[2] = 'dog';
    $arr[1] = 'fish';

    krsort($arr);

    var_dump($arr);
Outputs:

    array(3) 
    {
        [2]=> string(3) "dog" 
        [1]=> string(4) "fish" 
        [0]=> string(3) "cat" 
    }

Re: How big are PHP arrays (and values) really?

#75
post #66

Earlier quoted context omitted.

> Considering PHP's scope and its limitations (lacks things like threads for example) Please look up PHP-FPM. I use it in production, with great success. > If you're storing 100.000 integers on a PHP data structure you're most likely doing it wrong. What if I had a ton of price points and I needed to do statistical analysis? Well, those would be floats, but you get my point. PHP is capable of doing the work efficient…

How would you evoke such code? Squeezing it into an http reply? Calling it from PHP CLI? If doing it on your webapp, don't! You're doing it wrong. Otherwise why would you need php specifically? I mean I guess you could, but what advantage does it poses comparing to other languages available in pretty much every *nix system nowadays (perl, python)? For the record, I have a nice collection of downvotes for defending PH…

> If doing it on your webapp, don't!

That's the only place it's useful. Once the request comes in, I would split it off to a separate PHP CLI process. When the analysis is complete, I can transfer it to the user via web sockets.

So they would request a report. I would tell them great, it'll take a bit, keep working on something else and we'll let you know when it's ready. When it's ready, a notice shows up telling them to check it out.

> I mean I guess you could, but what advantage does it poses comparing to other languages available in pretty much every *nix system nowadays (perl, python)?

For something like this, the advantage would only be that you're sticking to the same language as the rest of your code base. It would make things simpler. So if you're using PHP, stick with that. If Perl, Python, etc. stick with that. It doesn't really matter. They can all do the work.

Re: How big are PHP arrays (and values) really?

#76
post #46

Earlier quoted context omitted.

It is the same for Javascript: even though you have an Array object that you can instantiate instead of the more generic Object, they are much the same thing under the hood with both arrays and objects being hash maps [you can use the two interchangably to an extent (if you don't need the extra functinos defined by the array prototype) - references object properies in an hash-like manner or array contents the same wa…

It is the same for Javascript: even though you have an Array object that you can instantiate instead of the more generic Object, they are much the same thing under the hood with both arrays and objects being hash maps While this is true if you only consider language semantics, implementions do use actual arrays to store dense properties with integer names, ie arrays are backed by both flat and sparse storage with som…

Good point. Presumably that is one of the differences in array performance between modern browsers and their eldery relatives.

Re: How big are PHP arrays (and values) really?

#77
post #68

What does memory_get_usage() actually do ? Does it report the "heap" size assigned to the process, or does it use PHP internal counters for the allocated user data/variables ? A C malloc subsystem will assign a whole lot of virtual memory, in steps of pages, or more if it decides to attach a piece using mmap(). In order to make this test case relevant, I'd say one have to know what memory_get_usage() does - it's at l…

This is easy to check in the source. memory_get_usage() calls zend_memory_usage(), which accesses the size field on a global structure mm_heap, which is updated by PHP's memory allocation system (e.g. if you call *_zend_mm_alloc_int)

I was debugging a php memory issue yesterday, and noticed that at some point my get_memeory_usage() value became much, much smaller than the memory footprint recorded by top (20 MB in get_memory_usage(), 500 MB in top RES). It was a sudden jump during a loop execution according to top. Does top give accurate memory estimates for php scripts?

Re: How big are PHP arrays (and values) really?

#78
post #11

Earlier quoted context omitted.

Since PHP arrays are within a factor of two for the theoretical optimum for a dynamically typed language that unifies arrays and hashes, all these languages (add javascript to the list) should have roughly comparable behaviour unless they do some ugly special casing for hashes where all keys are integers. [edit: see the comment by InfernalH for a measurement that indicates that ruby seems to do this.] You have to loo…

As I see it, doing the ugly special casing for you is exactly what any competent dynamic language implementation should do.

Ugly special casing requires code that would detect those special cases. This adds overhead in terms of processing cycles and code. Most arrays, majority of the time, store a tiny amount of information. Less than a dozen entries. Doing that sort of detection is massive overkill and would actually hurt performance. You want to speed things up for the real world. Not artificial benchmarks. How many times in your code are you really going to need 100,000 integers in memory? Probably once in your entire lifetime, if that.

Re: How big are PHP arrays (and values) really?

#80
post #78

Earlier quoted context omitted.

As I see it, doing the ugly special casing for you is exactly what any competent dynamic language implementation should do.

Ugly special casing requires code that would detect those special cases. This adds overhead in terms of processing cycles and code. Most arrays, majority of the time, store a tiny amount of information. Less than a dozen entries. Doing that sort of detection is massive overkill and would actually hurt performance. You want to speed things up for the real world. Not artificial benchmarks. How many times in your code a…

I use much larger integer arrays all the time but that's beside the point. The question is whether or not it makes sense to make dynamic language runtimes as efficient as possible. I think it does, not because it necessarily helps all or even most existing programs, but because it enables people to create a much wider range of software with a language they already know. Many modern websites wouldn't be possible with JavaScript engines from 1999.

And if you optimize at all, memory usage must be a top priority. Lack of memory is a lot more costly in terms of performance than a couple of C based heuristic checks. Creating and then garbage collecting an array full of individual integer objects doesn't just use a lot more memory, it's also orders of magnitude slower. The array doesn't have to be very large for that to matter as there may be large numbers of arrays.

Post reply on HN