Graphing the recent VM changes

Samuel Greear graphed the performance differences for Postgres and MySQL on DragonFly, before and after the recent VM changes.  Note that 1: this was done a little while ago, so I think the performance difference would be even greater now, and 2: this was graphed versus the already-performing-better 2.12, not the current stable release of 2.10.

Lazy Reading for 2011/10/23

Not a lot of links this week, for some reason.

Your unrelated comics link for the week: Oglaf.  This week’s OK, but it’s frequently NSFW, and frequently hilarious.

The next release and what’s needed

There’s a rare crash in DragonFly 2.10, where applications would segfault.  The system would run find.  This is apparently more likely to happen in 2.12, though reports on this vary.  It’s real, though.

Matthew Dillon went looking for this bug, and happened to roll back vm_token, the last lock in DragonFly that presented a serious impediment to multiprocessing.  It’s a big patch.  It fixes the problem, which is great!  It also happens to make DragonFly buildworlds almost twice as fast depending on the number of cores in the system.

Holy crap we want to get that out…  but it makes some significant changes to the system and needs to be tested.  So, the next release probably won’t be for a few weeks.

If you want to help, build master and do something with it – move data, run server programs, whatever.  Report crashes.  This performance improvement is worth working for.

Bulk build results for 2011Q3

I have some pkgsrc-2011Q3 builds done, for x86_64 and i386.  I performed them on DragonFly 2.11, but they should work fine for 2.12/2.13.  They’re uploading to the pkgsrc-2011Q3 folder on mirror-master, so you’ll need to set PKG_PATH correctly to use them via pkg_radd.

PKG_PATH=http://mirror-master.dragonflybsd.org/packages/x86_64/DragonFly-2.13/stable/

The x86_64 package upload is done, and I anticipate the i386 one will be done within the next 24 hours.

The last MP step

There’s only one multiprocessing bottleneck left in DragonFly: vm_token.  Matthew Dillon’s working on removing it, and he’s been testing his initial results on a 4-core machine and a 48-core machine, using heavily parallelized buildworlds to test concurrency.  He’s posted the results, showing an initial speedup of up to 30%.  This definitely isn’t going to make it into 2.12, but it’s looking good already.  Keep in mind these are improvements on top of the performance graphed here yesterday.

More performance graphing

Venkatesh Srinivas sent along a graph of his nmalloc testing that shows mysql threading performance on DragonFly, from slightly over a year ago.  Both graphs were done on a 4-core system, though I don’t know if the specs are comparable, so the curve is important.  Look at the just-posted curve for comparison.  That’s how much things have improved.

In fact, here’s a cheesy overlay, cropping the more recent results and laying the old ones on top of it.  The black lines are the year-ago performance, and the colored lines are the performance now.

Improved DragonFly threading performance

Performance curve for DragonFly 2.12

Samuel Greear has graphed out the performance of both MySQL and Postgres on DragonFly 2.12 as you add threads.  There’s a very nice correlation on performance and number of cores.  For comparison, there’s this old test from 2007 which shows uniprocessor performance to be good but not improved by adding cores.  The tests were on completely different hardware, so the actual curve of the graph is the telling point.

As he points out in his post, excellent multiprocessor performance is arriving on DragonFly, without any catastrophic shifts or destabilizing changes.