Posted on
Summer of Code DragonFly projects announced

Google’s announced the accepted projects for 2011.  DragonFly has 6 slots!

We had a large number of interesting project proposals; far more than than the slots available.  If you’re one of the students who did not get in, please consider working on your project as time allows.  I know it won’t be lucrative, but I’d still like to see them happen.

Here’s the list of accepted projects:

  1. Implementing a mirror target for device mapper: Adam Hoka, mentored by Joe Talbott
  2. Improve dsched interfaces and implement BFQ disk scheduling policy: Brills Peng, mentored by Alex Hornung
  3. Make vkernels checkpointable: Irina Presa, mentored by Venkatesh Srinivas
  4. Port PUFFS from NetBSD/FreeBSD: nickprok, mentored by Nathaniel Filardo
  5. Bring kernel event notification in DragonFly BSD to its logical conclusion: Samuel J. Greear, mentored by Sascha Wildner
  6. Porting Virtio Drivers from NetBSD to DragonFly BSD to speed up DragonFly BSD as a KVM guest: Stéphanie Ouillon, mentored by Pratyush Kshirsagar

 

Posted on
Sysbench and DragonFly releases

I did some comparative benchmarking between the 2.6, 2.8, and upcoming 2.10 release for DragonFly.   As several people have guessed, performance has improved significantly, and the difference would probably be even more pronounced if I was using more modern hardware,  e.g. swapcache or a system with AHCI.  I have a mailing list post with details, and here’s the graph that sums it up:

Shorter bars are better(Sorry, no Lazy Reading this week.  Life didn’t co-operate.  At least there’s a pretty graph!)

 

Posted on
Zombie shirts

Not shirts with zombies on them, but DragonFly shirts that don’t have a seller.  I had a random Google search turn up a store selling DragonFly T-shirts, among other things.  It was essentially a spam store.  The seller wasn’t producing anything but instead reselling other people’s material for a commission, similar to the splogs out there that recopy material from other blogs or Wikipedia and slap ads on it.  (I’ve seen Digest material pop up that way.)

Following the link back shows that the shirt is sold through a Cafepress store called ossgear.  It looks like the original store owner asked permission to use the logo back in 2006.  ossgear.org is no longer a functioning domain, and I can’t find any other reference to this seller; they appear to have stopped doing business 5 years ago.

The moral of this story: Sites like Cafepress will try to profit from your work long after you’ve stopped using them.  The frustrating part is that the logo isn’t even right!

Posted on
RAM vs. deduplication

Tomas Bodzar asked about RAM usage with Hammer and deduplication, pointing at this example that shows ZFS requiring…  I’m not sure.  Lots?  Anyway, Matthew Dillon noted that offline deduplication in Hammer would use available RAM/swap for CRCs on all files, but only a limited subset for ‘live’ dedup.  For a real-world example, Venkatesh Srinivas described deduplicating about 600G down to 400G, with a machine having only 256M of RAM. Yes, only 256M.

Posted on
Add package-clean to your list

The usual way for building pkgsrc packages from source is ‘bmake install clean’, to build and install the package, and then clean the work files from building it.  Since the recent change to DESTDIR, where a binary package is built before installation, you may want to add ‘package-clean’ to the list, so that the binary package is also removed after installation.

Posted on
Lazy Reading for 2011/04/17

I hope I can get this together.

  • This article asks “Does anyone in Silicon Valley care about Windows anymore?”   It’s an inflammatory title, to get you to read it, and it’s based on anecdotal ideas, but I think there’s some truth to it.
  • Something similar, in hardware: I see people who care about what they run either getting a Macbook or a Thinkpad these days.  (I’ve owned both, and they are nice laptops…)  Let’s run with that idea, in fact: Macbook is to Thinkpad running BSD as is… iPhone is to Android phone running custom ROM?  This is turning into a “levels of nerditry” sort of comparison.
  • Community is your best feature, a talk about how to encourage the growth of an open source group.  I link to it because it’s useful and well done, but also because it lets me feel a bit self-congratulatory; we already use many of the listed concepts in DragonFly.
  • Zero knowledge user identification is interesting, though it’s not something you could apply to a lot of users.  (via)
  • Things found via Google: A DragonFly 2.8.2 x86_64 VMWare image on Sourceforge.  Don’t know who put it there.
  • This article about passwords says multiple common words make more secure passwords than adding upper/lower case and numbers to passwords.  An interesting contention, though I don’t think it works as well as it’s described.  (Adding ” ” into the list of possible characters isn’t as effective as having to double the list for case, for instance.)
  • It’s been a while since I posted a roguelike link.  Well, how about “How Rogue Ended Up On The Sofa“?  (via)  It very nicely draws a line connecting rogue and a whole lot of modern games.
Posted on
How much swapcache can help (with graphs)

I love graphs.  Jan Lentfer made some! Both of these show recent speed improvements in DragonFly – especially some spectacular results from swapcache(8) and the recent NCQ tagging improvements.  (Note that only the third graph represents the NCQ improvements; the first two graphs were done before.)

The first one is a comparison of pgbench running on the same hardware twice – once with the 2.8 release of DragonFly, and once with a recent 2.9 version.  2.9 is definitely looking to be faster than 2.8.

Next up is a 2.9 system run with and without swapcache, showing an astounding difference between the two.  It’s pretty clear just how much performance improvement you can get from swapcache…  (see Jan’s notes on the setup after the graphic.)

 

Jan’s notes, from EFNet #dragonflybsd on IRC:

15:08 < lentferj> these are SELECT-Only tests
15:09 < lentferj> JustinS: it’S important to note, that the database is 2,5x
bigger than RAM on the swapchache test
15:11 < lentferj> JustinS: I did a Select-Only ramp-up of 30 minutes to get
caches and swapcache filled
15:12 < lentferj> JustinS: and then I ran
15:13 < lentferj> for i in 1 2 3 4 6 8 12 16 24 32; do
/usr/lib/postgresql/8.4/bin/pgbench -U pgsql -h atom -s 400
-S -c “$i” -T 600 pgbench; done
15:13 < lentferj> so, select only pgbench for 10 minutes each
15:13 < lentferj> with increasing numbers of client
15:14 < lentferj> pgbench on another box, 100MBit switched network
15:15 < lentferj> JustinS: the first graph (2.8.2 vs current) is the same w/ a
database that fits in RAM entirely
15:15 < lentferj> so measuring concurrency performance (w/o I/O)
15:17 < lentferj> the swapcache comparison was on a 2GB box with a 5GB database
and 16GB swapcache (INTEL) attached to a sili card
15:17 < lentferj> on a atom 330 :)

Now, here’s testing with the recent NCQ tagging update for AHCI:

These results are astonishing.  Please, someone compare with other operating systems!

Here’s the stats for this last test:

 

  • 5.6GB database, system w/ 2GB RAM –> io benchmark
  • pgbench with increasing no of client 1->32, SELECT-Only Mode
  • sili controller Dawicontrol DC-3410 SATA PCI controller which is using a Silicon Image 3124-2 chip
  • 2 Seagate Barracuda ES.2 250GB SATA II disks
  • lvm stripe over those disks
  • postgresql.conf is default, except shared_buffers set to 512MB and effective_cache_size to 1024MB
  • atom330 on a Foxconn mobo
  • SSD is SATA INTEL SSDSA2M040 2CV1

 

Posted on
Lazy Reading for 2011/04/13

Get out your wallet!  I encourage purchasing here.

  • You should buy a SSD.  Not necessarily news to you, but that article does a good job of summarizing why.
  • On the other hand, SSD prices are already on their way up/availability is way down.  Japan’s disasters are having a ripple effect through the high-tech supply chain.  Either buy immediately or get ready to wait for a while…
  • Introduction to Architecting Systems for Scale – you either don’t care, or find scaling questions immediately engaging.  I am one of the latter, so here’s the link.
  • I’ve been watching pkgsrc-changes@netbsd.org for a little while.  One thing I’ve discovered: there’s a lot of updates going on!  Another thing that’s nice to see: DragonFlyupdates, including ones that help with our move to gcc 4.4.
  • Aw, no more Kermit.  (via)  Not that I have a use for it at this point, but still: aww.  I bet in about 10 years I’ll say the same thing about… gopher?  Remember that?  It’s not even supported in Firefox 4 now, which kinda makes me feel sad.  And old.
  • Server plans: Facebook vs. Google.  (warning: Facebook article is somewhat giddy.)
  • The infinite hard drive.  (via I lost it, sorry)

Here’s an extra little thing: next time you’re dealing with dusty computer equipment, remember this picture:

That is what happens to an exposed RJ45 port after a few years in a salt mine (my employer).  This was inside an enclosed, mostly-sealed  structure, too.