PHP 7's new hashtable implementation • Hacker News Board

Re: PHP 7's new hashtable implementation

#51

Post by mootothemax » Tue, Dec 23, 2014, 11:45 AM UTC

mmaunder wrote: post #38

Awesome, but I still don't think it's enough. In benchmarks we did the memory usage of PHP array() was horrific. Sorry I don't have actual numbers to post, but we ended up using pack() and unpack() to store stuff that should have been in an array because it would grow to 100's of megs using PHP's array() and using a binary structure it stays under 10 megs. I just don't think a 2.5X improvement is going to come close…

we ended up using pack() and unpack() to store stuff

Out of interest, can you give a few more details of what you were storing, and how?

I've seen serialise() used surprisingly often (WordPress comes to mind!), which is always going to be pretty verbose:

http://uk.php.net/serialize

Re: PHP 7's new hashtable implementation

#52

Post by orf » Tue, Dec 23, 2014, 11:59 AM UTC

skrebbel wrote: post #41

This is probably not a popular opinion, but I believe that PHP's associative array is one of the best-designed data structures in programming languages. Its main distinguishing property, as mentioned in this article, is that values can be indexed by key, but are still iterated in the order they were set. This is "do what I want" in so many cases that it's just nuts. Sure, just as often it's just needless overhead, bu…

Maybe it's just me but I rarely need the ordering, seems a big waste to carry this unneeded overhead.

Re: PHP 7's new hashtable implementation

#53

Post by nly » Tue, Dec 23, 2014, 12:01 PM UTC

ape4 wrote: post #28

I assumed they used C++ std::map

PHP isn't written in C++. It's written in some pretty macro dense C. Even if it were, isn't a hash table.

Re: PHP 7's new hashtable implementation

#54

Post by nikic » Tue, Dec 23, 2014, 12:07 PM UTC

gopalv wrote: post #12

This is neat. Looking through it, looks like it makes regular numeric arrays faster as well via the flags. I wonder if the ->pDataPtr vs ->pData confusion has been resolved. I'm probably a few years behind, but a lot of my confusion working with hashes has been that pair of void* pointers.

Yes, pData and pDataPtr are no more. They have always been pretty pointless - both could have been dropped even retaining the rest of the previous implementation by using a struct hack layout. In the new implementation they aren't needed because it's specialized to zvals (our 99%-or-so use case).

Re: PHP 7's new hashtable implementation

#55

Post by fleetfox » Tue, Dec 23, 2014, 12:17 PM UTC

cranklin wrote: post #46

fleetfox wrote: post #18

Earlier quoted context omitted.

No primitive types: tuples, lists, sets. Having only arrays to work with and being weak typed results in a lot of headache.

Those aren't primitive types

What are those called then? Wikipedia definition of primitive type is pretty vague. I don't think collections/containers can't be called primitive.

Re: PHP 7's new hashtable implementation

#56

Post by gopalv » Tue, Dec 23, 2014, 12:23 PM UTC

nikic wrote: post #54

gopalv wrote: post #12

This is neat. Looking through it, looks like it makes regular numeric arrays faster as well via the flags. I wonder if the ->pDataPtr vs ->pData confusion has been resolved. I'm probably a few years behind, but a lot of my confusion working with hashes has been that pair of void* pointers.

Yes, pData and pDataPtr are no more. They have always been pretty pointless - both could have been dropped even retaining the rest of the previous implementation by using a struct hack layout. In the new implementation they aren't needed because it's specialized to zvals (our 99%-or-so use case).

Nice work. I read deeper into the changeset and I like.

The ->pDataPtr was the one thing behind 90% of the bugs I caused with exts (obviously stuff like the frozen_array hashtable handling was a completely odd-ball case).

I read deeper and found that you also fixed the "void * *" in zend_hash_find(), which is another pain point in the old API - you cannot rely on the compiler type-checking at all.

I no longer work with PHP, but avenge me for the hair I've lost over the IS_REF madness (copy_ctor vs separate_zval) :)

Re: PHP 7's new hashtable implementation

#57

Post by stdbrouw » Tue, Dec 23, 2014, 12:28 PM UTC

joeyo wrote: post #37

stdbrouw wrote: post #31

Earlier quoted context omitted.

... though I wouldn't necessarily call Lua's way of doing things "sane". E.g. if Lua encounters a `nil` somewhere in your table, it will stop iterating.

It's certainly sane in the sense that it was an intentional, if perhaps unusual, design decision. I personally like it since it results in an efficient implementation of sparse arrays. In any event, Lua 5.2 will respect the __len, __pairs and __ipairs metamethods, so you can tweak this behavior if you need to store nils in your tables and iterate over them.

The world would be a beautiful place if everything intentional were sane :-) It's bitten me a couple of times (e.g. when unpacking arguments in a function to feed them to another function, where one of the arguments is nil) but I can see how it could be the desired behavior in some cases.

Re: PHP 7's new hashtable implementation

#58

Post by [deleted] » Tue, Dec 23, 2014, 12:32 PM UTC

orf wrote: post #52

skrebbel wrote: post #41

This is probably not a popular opinion, but I believe that PHP's associative array is one of the best-designed data structures in programming languages. Its main distinguishing property, as mentioned in this article, is that values can be indexed by key, but are still iterated in the order they were set. This is "do what I want" in so many cases that it's just nuts. Sure, just as often it's just needless overhead, bu…

Maybe it's just me but I rarely need the ordering, seems a big waste to carry this unneeded overhead.

[deleted]

Re: PHP 7's new hashtable implementation

#59

Post by acmd » Tue, Dec 23, 2014, 12:36 PM UTC

nly wrote: post #53

ape4 wrote: post #28

I assumed they used C++ std::map

PHP isn't written in C++. It's written in some pretty macro dense C. Even if it were, isn't a hash table.

But std::unordered_map is. I wonder what would be the actual C++ memory consumption using that.

Re: PHP 7's new hashtable implementation

#60

Post by pornel » Tue, Dec 23, 2014, 12:40 PM UTC

anonymoushn wrote: post #43

skrebbel wrote: post #41

This is probably not a popular opinion, but I believe that PHP's associative array is one of the best-designed data structures in programming languages. Its main distinguishing property, as mentioned in this article, is that values can be indexed by key, but are still iterated in the order they were set. This is "do what I want" in so many cases that it's just nuts. Sure, just as often it's just needless overhead, bu…

Python has https://docs.python.org/2/library/collections.html#collectio... I've never wanted this though. I just discovered OrderedDict when I was looking for something like std::map.

> I've never wanted this though.

When it's not the default normal thing, you don't build solutions around it, so you never see what you're missing :)

In PHP I make lots of tiny uses of it in many places. I really missed it when I switched to Python (sure, there's OrderedDict, but it's a second-class citizen: there's no syntax for literals and standard APIs don't explicitly take advantage of it).

* It's very useful for deduplicating things without losing order (especially when you have a bigger algorithm that collects data from multiple sources or a tree structure into one array).

* It's neat for sorting objects without having to mutate them to add a key or wrap them in a key/value object.

* It's great for configuration with JSON-like structures, but key order gives extra flexibility in the design, e.g. instead of [{id:"foo"},{id:"bar"}] you can use ["foo"=>[],"bar"=>[]].

It's not a major feature, but it makes things nicer. I've never had arrays accidentally randomized in PHP, but had bugs due to careless list->dict->list conversions in Python.