Interesting. I was looking at the comments hoping for some more technical background, but unfortunately they seem to have been run over by the animated gif crowd. Any more details on this?
The pull request is better:
Composer – Disable GC when computing deps and refs
51–60 of 135 posts
Re: Composer – Disable GC when computing deps and refs
#52Earlier quoted context omitted.
[deleted]
No. Concern trolling and tone policing needs to fuck right the fuck off. Do you find you have much success motivating people when you use this tone of voice? I reported this issue in February. Others have reported it before I did. What is Composer's response? Silence and negligence while its maintainers go to cons and drink beer. Their priorities are totally fucking backwards. I would strongly advise using less infla…
[deleted]
Re: Composer – Disable GC when computing deps and refs
#53Interesting. I was looking at the comments hoping for some more technical background, but unfortunately they seem to have been run over by the animated gif crowd. Any more details on this?
I think the article on this, explains why a memory reduction by garbage collection also increases execution speed: http://derickrethans.nl/collecting-garbage-performance-consi...
Re: Composer – Disable GC when computing deps and refs
#54As far as i understand composer is roughly the same thing as the cpan client. And they just simply disabled the garbage collector for it. What is this guy doing that he needs gigabytes of memory to install a bunch of php libraries? Before: Memory usage: 2194.78MB (peak: 3077.39MB), time: 1324.69s After: Memory usage: 4542.54MB (peak: 4856.12MB), time: 232.66s
That user is using PEAR, which is the old shitty PHP "CPAN" That's the reason for the huge memory usage. We're slowly moving away from PEAR, but since it works for now not everyone has/will transition. Edit: I should also point out that there are a few packages that almost everyone uses (PHPMD, PHPCS, phpUnit) that are still mostly pulled from PEAR, though I think phpUnit has a composer option.
PHPUnit stopped updating PEAR in April, and actively destroyed PHPUnit on their PEAR repo at the start of this month (i.e. a patch version update that consisted just of a bash script printing a migration message...).
Re: Composer – Disable GC when computing deps and refs
#55For those looking for a technical explanation, the PHP garbage collector in this case is probably wasting a ton of CPU cycles trying to collect thousands of objects (a LOT of objects are created to represent all the inter-package rules when solving dependencies) during the solving process. It keeps trying and trying as objects are allocated and it can not collect anything but still has to check them all every time it triggers.
Disabling GC just kills the advanced GC but leaves the basic reference counting approach to freeing memory, so Composer can keep trucking without using much more memory as the GC wasn't really collecting anything. The memory reduction many people report is rather due to some other improvements we have made yesterday.
As to why the problem went unnoticed for so long, it seems that the GC is not able to be observed by profilers, so whenever we looked at profiles to improve things we obviously did not spot the issue. In most cases though this isn't an issue and I would NOT recommend everyone disables GC on their project :) GC is very useful in many cases especially long running workers, but the Composer solver falls out of the use cases it's made for.
Re: Composer – Disable GC when computing deps and refs
#56The commit is great. I love that the comments have spiraled completely out of control. At this point, 30 minutes after the link was posted, the comment thread is now a competition to see who can post the best gif. I know we're serious here, but stuff like this reminds me why I love the internet so much. It's fun to cut loose once in a while.
Agreed. It's such a shame that HackerNews doesn't let you post animated GIFs - I think it'd really add a lot of value to the discussions here.
Technically, it will let you post them as links, it just won't upload and embed them. But a plugin or a userscript would fix that, provided they're posted somewhere with an easy to deal with API like imgur.
Re: Composer – Disable GC when computing deps and refs
#57The commit is great. I love that the comments have spiraled completely out of control. At this point, 30 minutes after the link was posted, the comment thread is now a competition to see who can post the best gif. I know we're serious here, but stuff like this reminds me why I love the internet so much. It's fun to cut loose once in a while.
That's what I figured as well, and since https://twitter.com/seldaek/status/539773104835555329 the rate of comments went nuts :)
I really felt bad seeing how we missed that improvement for so long, so turning it all in a gif-fest is more productive than self-deprecation!
Re: Composer – Disable GC when computing deps and refs
#58The commit is great. I love that the comments have spiraled completely out of control. At this point, 30 minutes after the link was posted, the comment thread is now a competition to see who can post the best gif. I know we're serious here, but stuff like this reminds me why I love the internet so much. It's fun to cut loose once in a while.
I think it's great too, but I shudder to think what it might look like 5 years from now. I can only figure that there will be dead gifs everywhere
Looks like github hosts most/all of them so should be ok.
Re: Composer – Disable GC when computing deps and refs
#59I had the same issue in Python recently. The project runs as a server that loads a huge amount of objects from the database, and could use as much as 10GB memory! Python's reference counting works great, but every so often, the full-heap-scanning cycle collector would run, and it took quite a lot of time to scan a mutli-GB heap.
We noticed the issue happened most often when deserializing objects (loading them from Redis to memory). As it turns out, Python would schedule a collection every time the object_created counter was sufficiently higher than object_destroyed counter. In general, this makes sense, because that way you can be sure that objects are being created and not being freed, which most likely means a resource leak or a reference cycle. However, the same thing happens during deserialization - many new objects are created, and none are freed. Coupled with Python's low threshold (700), GC was triggered many many times in every serialization loop (usually in vain, as no new objects became recyclable). Disabling GC and running full collections manually solved the problem
Re: Composer – Disable GC when computing deps and refs
#60Wait, when did Github become the new 4chan?
[deleted]