Sascha Wildner has been using a new-to-me tool called coccinelle (no, not that) to scan for a number of problems. Patches for this tool may be useful for anyone else using coccinelle for bug-finding in other software.
Two tips for working with pkgsrc, derived in part from this mailing list post on users@ (follow the thread) and from my own experience. If you put WRKOBJDIR=/usr/obj/pkgsrc into /usr/pkg/etc/mk.conf :
- You can clean up any leftover package building files by deleting the files in that directory and leave your pkgsrc files untouched.
- You can have a read-only /usr/pkgsrc, which means it can be shared over NFS (or SMB?) between multiple machines, DragonFly or otherwise.
I’ve had a bunch of posts like this so far, but that’s not a bad thing. Sepherosa Ziehau has a pair of optimizations that appear to make performance with big pipes (1G) and tiny packets (18b, if I read correctly) reach near the physical maximum for 1000-base-T Ethernet.
I think a lot of people don’t realize rcrun exists, and run files in /etc/rc.d directly. If you’re one of those people, read the rcrun(8) manpage. Then, notice that Sascha Wildner has extended rcrun to work with the etc/rc.d in /usr/local and /usr/pkg too.
Francois Tigeot has been working for quite a while on a VFS accounting system. It doesn’t restrict to a quota (yet), but it will give you byte totals for each mounted filesystem. It has been committed, so it looks like a good way to tell which PFS is eating your disk.
Update: Francois pointed out he’s still adding parts for this. So it’s not quite done yet, but soon.
Buildworlds are now much faster, because they can run themselves in parallel. Invoke it using the -j option to make. Matthew Dillon saw a 25% reduction in time when using ‘make -j 12 buildworld’ on a 4-core system. You may need to manually update xinstall and mkdir:
cd /usr/src/usr.bin/xinstall make clean; make obj; make all install cd /usr/src/bin/mkdir make clean; make obj; make all install
It’ll also use more memory than a non-parallel build, but heck, that’s cheap these days.
Venkatesh Srinivas made a minor change to a ddb backtrace – it now prints the raw instruction pointers. On x86_64, a backtrace would not print the correct objects out, so this is better. It’s a minor change, but I’m pointing it out because it totally helped solve a problem for me on a package-building machine.
The general rule of thumb is that if you have a function written in an interpreted language (Perl, Python, etc.), it’ll be faster in C. If you need it faster than that, you go to assembly. Prepare to have your world rocked: Venkatesh Srinivas found that strlen() in libc was actually slower written in assembly than in C. His commit message has numbers to back that up.
Adrian Chadd showed up on the DragonFly kernel@ mailing list, offering some help in keeping things compatible with FreeBSD and 802.11 networking. That’s quite neighborly of him, especially since his hands are already pretty full.
It’s another throughput tweak from Sepherosa Ziehau: soaccept is run differently when pulling in network data from a socket. The commit message once again shows the results of the change using httperf.
Siju George asked about how he could figure out which serial number (in /dev/serno) maps to which disk. Tim Darby posted a script he used for it, or you can just use devattr(8). There’s also a linking trick described by Chris Turner to remember how the names map.
Binutils in DragonFly is now up to version 2.22 – the commit linked is one of several.
Some time ago, Matthew Dillon worked on a bulk build system that built as much of pkgsrc in parallel as possible. It’s in the tree now as ‘fastbulk‘, for anyone wanting to try it out. I used it a bit; I didn’t measure the degree of speed increase, but was able to get about 70% of the packages built.
Sepherosa Ziehau has implemented another networking speedup. Read the commit message for details on what he changed, since it’s rather in-depth. He shows an 18% improvement in netperf results.
Matthew Dillon has written a contiguous memory mapper, which is designed to fix problems with video cards and USB drives that need a big chunk of memory to keep. This can affect booting or later on, when disconnecting/reconnecting a USB drive. If this still doesn’t fix the problem for you, try adjusting the sysctl ‘vm.dma_reserved’ to something bigger, like 64M. It defaults to 16M.
(Normal mailarchive isn’t updating because of an ongoing upgrade to crater.dragonflybsd.org – sorry!)
When building world and kernel on DragonFly, /usr/obj is where the work files get placed. This can eat a bit of space, but it can be safely deleted. If you keep the files around, subsequent rebuilds can be done faster with a quickwork/quickkernel, but this may not matter to you.
(This was answered on the mailing lists by Max Herrgaard, but I don’t have a link to his reply – sorry!)
Remember the Postgres benchmark I described here a few days ago? Francois Tigeot has updated it with numbers from Scientific Linux running the same pgbench procedure. (see page 2) If you’re too lazy to look at the PDF, his summary is this: Linux is fastest of all, and also crashes the most.
Sepherosa Ziehau has implemented an asynchronous pru_send in sendfile. The results are a 70-90% increase in performance, as shown in his netperf localhost test.