Become a Patron!

My Amazon wishlist can be found here.

Life Line

Xdebug Update: December 2019

Another month, another monthly update where I explain what happened with Xdebug development in this past month. It will be published on the first Tuesday after the 5th of each month. Patreon supporters will get it earlier, on the first of each month. You can become a patron here to support my work on Xdebug. If you are leading a team or company, then it is also possible to support Xdebug through a subscription.

In December, I worked on Xdebug for near 50 hours, on the following things:

Xdebug 2.9.0

After releasing Xdebug 2.8.1, which I mentioned in last month's update, at the start of the month, more users noticed that although I had improved code coverage speed compared to Xdebug 2.8.0, it was still annoyingly slow. Nikita Popov, one of the PHP developers, provided me with a new idea on how to approach trying to find out which classes and functions still had to be analysed. He mentioned that classes and functions are always added to the end of the class/function tables, and that they are never removed either. This resulted in a patch, where the algorithm to find out whether a class/function still needs to be analysed changed from from O(n²) to approximately O(n). You can read more about this in the article that I wrote about it. A few other issues were addressed in Xdebug 2.9.0 as well.

Breakpoint Resolving

In the May update I wrote about resolving breakpoints. This feature will try to make sure that whenever you set a breakpoint, Xdebug makes sure that it also breaks. However, there are currently two issues with this: 1. breaks happen more often than expected, and 2. the algorithm to find lines is really slow. I am addressing both these problems by using a similar trick to the one Nikita suggested for speeding up code coverage analysis. This work requires quite a bit of rewrites of the breakpoint resolving function, and hence this is ongoing. I expect this to cumulate in an Xdebug 2.9.1 release during January.

debugclient and DBGp Proxy

I have wanted to learn Go for a while, and in order to get my feet wet I started implementing Xdebug's bundled debugclient in Go, and at the same time create a library to handle the DBGp protocol.

The main reason why a rewrite is useful, is that the debugclient as bundled with Xdebug no longer seems to work with libedit any more. This makes using debugclient really annoying, as I can't simply use the up and down arrows to scroll through my command history. I primarily use the debugclient to test the DBGp protocol, without an IDE "in the way".

The reason to write a DGBp library is that there are several implementations of a DBGp proxy. It is unclear as to whether they actually implement the protocol, or just do something that "works". I will try to make the DBGp proxy that I will be working on stick to the protocol exactly, which might require changes to IDEs who implement it against an non compliant one (Komodo's pydbgpproxy seems to be one of these).

This code is currently not yet open source, mostly because I am still finding my feet with Go. I expect to release parts of this on the way to Xdebug 3.0.

Business Supporter Scheme and Funding

Support through the Business Supporter Scheme continues to trickle in.

This month's new supporter is Strategery. Thanks!

If you, or your company, would also like to support Xdebug, head over to the support page!

Besides business support, I also maintain a Patreon page and a profile on GitHub sponsors.

Podcast

The PHP Internals News has been on hiatus since the PHP 7.4 release, but it will return in January.

Shortlink

This article has a short URL available: https://drck.me/xdebug-19nov-fg6

Comments

No comments yet

Xdebug Update: November 2019

Another month, another monthly update where I explain what happened with Xdebug development in this past month. It will be published on the first Tuesday after the 5th of each month. Patreon supporters will get it earlier, on the first of each month. You can become a patron here to support my work on Xdebug. If you are leading a team or company, then it is also possible to support Xdebug through a subscription.

In November, I worked on Xdebug for about 30 hours, on the following things:

Website Redesign

Matt Brown contacted me a few months ago, suggesting that I should consider cleaning up the design and content of Xdebug's website. He spend countless hours both redoing my atrocious (and old!) code that powers the website, and creating a new design for it. During November we put this new design online, and I upgraded the server to run PHP 7.4 too. There are still a few rough edges, and there are a few thins I still want to improve, but I believe that the new design (and code!) are much cleaner. Thanks Matt!

Xdebug 3 development

I only spent a little time on Xdebug 3 this month, mostly due to travel to speak at conferences. I did finished the modularizing of the Xdebug code base, and now have moved on to cleaning code up and refactoring it even more to continue to make it more maintainable.

Beyond that, I have started to remove a few things from Xdebug as well. I removed the aggregated profiler feature, which was never documented, and prepared an uncommitted patch to remove the xdebug.remote_handler setting. This setting could only ever have one value (dbgp), and it seems very unlikely that in the future Xdebug will support other debugging protocols. The underlying code for being able to have more protocols continues to exist. This is mainly because it enforces better design and less coupling between the different parts of Xdebug.

Xdebug 2.8.1 Release

I was right to think last month that it would be likely to have to make a bug fix release. A user commented on Twitter that the code coverage functionality was drastically slower. In Xdebug 2.8 I changed how the coverage functionality remembers which classes and their methods it had already analysed. In 2.7 and earlier, it sets a specific flag on the class entry, but that was always a hack, which stopped working (again) with PHP 7.4. Instead of using that flag, I now use a hash table to do so.

However, I had inadvertently negated the check, so instead of only analysing classes and their methods on the first visit, Xdebug ended up analysing it every single time. The fix for this was therefore small (and embarrassing).

During the testing of this new "fix", I noticed that code coverage was still a lot slower than in Xdebug 2.7.2, so I did some more research to improve this. Instead of allocating memory to create the hash key, I use stack memory instead.

For Xdebug 3 I have a few further ideas to speed up code coverage.

The bug fix for the performance degradation is the only ticket that made it into Xdebug 2.8.1.

Update: Xdebug 2.8.1 was released on December 2nd (so not actually in November).

Business Supporter Scheme and Funding

I have had further, but minimal, interest for the Business Supporter Scheme that I launched in September.

This month's new supporter is e3N GmbH. Thanks!

If you, or your company, would also like to support Xdebug, head over to the support page!

Besides business support, I also maintain a Patreon page and a profile on GitHub sponsors.

Podcast

I have continued with the PHP Internals News podcast. In this weekly podcast, I discuss in 15-30 minutes, proposed new features to the PHP language with fellow PHP internals developers. It is available on Spotify and iTunes, and through an RSS Feed. Let me know if you are a listener! The podcast will be on hiatus until early next year.

Shortlink

This article has a short URL available: https://drck.me/xdebug-19nov-f1q

Comments

No comments yet

Crafty Code Coverage

Xdebug's code coverage functionality has had dead code analysis for years. It is used to be able to mark lines of as having executable code on it, as well as lines which can not be reached. In order to provide this functionality it runs an algorithm code after each file has been compiled. For each class and function it checks whether the algorithm to analyse executable lines and dead code has already been run, as it makes no sense to check it for the same class, method, or function twice.

Until PHP 7.4, Xdebug stored this information on whether it has seen a class with a special flag on the class entry of a class, PHP's internal structure that contains all the information the engine needs to be able to instantiate objects and run methods.

PHP 7.4 changes the values of these flags, which prompted me to ask Nikita whether it was actually safe to use a flag like this as a marker of whether Xdebug has already analysed that class (and its methods). He said no and suggested that instead I should use a hash table to store this information instead. I implemented that for Xdebug 2.8, that was released just before PHP 7.4 came out.

Soon after PHP 7.4 came out, I received a bug report that code coverage was now now significantly slower. A specific run went from 8 minutes to more than 3.5 hours. I tried this out for myself with the test suite of the Document package of Zeta Components, and indeed, with PHP 7.3 and Xdebug 2.7.2 it took about 2.83 minutes, and with PHP 7.3 and Xdebug 2.8.0 46 minutes — a slow down of about 16 times.

I quickly discovered that I had forgotten a ! in the code, which meant that the analyses would run for every class after a new file was loaded/compiled. I fixed that in Xdebug 2.8.1. This did improve the code coverage timings, but it still took 22.26 minutes, instead of the original 8 minutes.

I started talking to twitter user Anthony to try out a few more speed improvements, and although we were making improvements, I was not getting anywhere near the original timings of Xdebug 2.7. At this point I referred back to Nikita to ask whether he had a good idea to improve on this. He mentioned that classes and functions are always added to the end of the class/function tables, and that they are never removed either. He also hinted at a method to only loop over the newly added classes and functions after each PHP file was compiled: loop over the table backwards up to the point where the size of the list was the previous time that we looped. This resulted in a patch that did exactly that. Xdebug now longer needs the hash table mechanism to check whether we have analysed classes with their methods, and functions already, and also no longer loops over the whole list of classes after each file. As most people use one class per file, the algorithm went from O(n²) to O(n) approximately.

This cut down the time to run the test suite with code coverage for the Document package to 1.21 minutes. About 2½ times faster as Xdebug 2.7 and earlier. Anthony was also pleased and surprised:

Although I tend to make an Xdebug release only once a month, in this case I thought it warranted to expedite this. So here is your end-of-year present: Xdebug 2.9.

🔔 📣 ⛄ ❄️ 📣 🎄 🎇 🎆 🎃 🎄 ✨ 🎉 🎊

Shortlink

This article has a short URL available: https://drck.me/ccc-f1p

Comments

Thank you for all your hard work! Community appreciates!

Good job. Even though only a few say "thank you", you should know you're doing great!