The latest quarterly release of pkgsrc, 2011Q2, has been branched. There’s no formal announcement yet to describe the highlights, but I’ll link it when it shows up. I’ve already started building binary packages for DragonFly 2.10 and 2.11.
Two completely separate and unrelated changes:
First, Alex Hornung has added a check to look for certain lines in a commit message, and add a MFC reminder note to the commit message if they are found. MFC, if you haven’t heard it, means ‘merge from current’, or moving a change from dragonfly-current to the last release version.
Second, with the next quarterly release of pkgsrc coming up, there’s some old packages that will get dropped. Speak up if you need them to stick around.
Pkgsrc bmake bootstrap, that is. There’s a new version of bmake, and it needs to be tested on every platform possible.
The pkgsrc ‘freeze’ in preparation for the pkgsrc-2011Q2 branch is coming up, starting this Sunday the 19th. This means the quarterly release will be tagged in about 2 weeks, and I’ll probably have binary packages built for DragonFly about a week or so after that.
There’s still a few packages in pkgsrc that don’t support DESTDIR (e.g. being built by someone other than root). If you want to help out, here’s a list of those 60 packages.
I moved to DragonFly 2.10 over the past few days, and I tried out deduplication, to see what kind of results I would get. The procedure is outlined below. I’m using /home here as an example, just to reduce the amount of text pasted in.
/pfs/@@-1:00004 966000640 566434576 399566064 59% /home
Move my various Hammer pseudo-file systems to version 5, which supports deduplication.
# hammer version-upgrade /home 5
Issue a deduplication simulate command, to see what it guesses will be the savings:
# hammer dedup-simulate /home
Dedup-simulate /home: objspace 8000000000000000:0000 7fffffffffffffff:ffff pfs_id 4
Dedup-simulate /home succeeded
Simulated dedup ratio = 1.22
That ratio turned out to be pretty accurate for the actual deduplication. I didn’t time it, unfortunately. I don’t know if the time taken is proportional to the amount of deduplication or the total volume of data, though I suspect the latter.
# hammer dedup /home
Dedup /home: objspace 8000000000000000:0000 7fffffffffffffff:ffff pfs_id 4
Dedup /home succeeded
Dedup ratio = 1.22
462 GB referenced
378 GB allocated
14 MB skipped
6869 CRC collisions
0 SHA collisions
0 bigblock underflows
The end result?
/pfs/@@-1:00004 966000640 505887504 460113136 52% /home
That data space is shared across all file systems, and it’s a 1TB disk, so it’s 7%, or 70GB. I was hoping for more, but I don’t have any obviously duplicated data (no local mail store, no on-disk backups), so perhaps this is normal. 70GB that I didn’t have before is no bad thing, though.
Incidentally, I was able to upgrade my installed software from pkgsrc-2009Q4 to pkgsrc-2011Q1 entirely using pkg_radd -u <pkgname>. Remarkably quick and painless, though pkgin may have been able to do it even faster since it would pull from the same place.
This new build is on x86_64, pkgsrc-2011Q1. It’s already uploaded, if you want to update. i386 coming soon. Several packages freeze up during build, so it’s been turning into a manual process.
One of the Google Summer of Code projects that will be valuable for DragonFly even though it isn’t a DragonFly project: “Add other package formats to pkgsrc”, where pkgsrc can interpret rpm, dpkg, and FreeBSD Ports files. Anyway, the project has a Sourceforge site.
I have them, for DragonFly 2.11/i386 and DragonFly 2.11/x86_64 (see pkgsrc-bulk), on pkgsrc-2011Q1. I think I even uploaded them correctly this time.
This week: lots more reading!
- Michael Lucas describes an extra layer of protection for when you can’t force public key usage on every SSH user.
- Cool, but obscure Unix tools (via) The screenshots are all from a Mac… How many of the 24 tools listed are in pkgsrc/pkgsrc-wip? Almost all of them. (tpp sounds entertaining.)
- NYCBUG, in addition to having a really fun convention, has been regularly posting audio of the presentations they host. The most recent is “William Baxter’s NYCBUG presentation on The Unix Method of Development Management”. See the BSD Events tweet for the download.
- What Ubuntu means. (via)
- Here’s a nice explanation of Intel’s new Tri-Gate design and with it, an incidental explanation of the processor market.
- This ycombinator post about Hammer2 work has an in-depth comment from Venkatesh Srinivas about DragonFly’s network setup, memory allocator, and token use. (Ignore the trolling in other comments.)
- Michael Lucas’s next No Starch Press book is Absolute OpenBSD, second edition.
- Pictures and video are starting to show up from the just-passed BSDCan 2011. (via this and also thesjg on EFNet #dragonflybsd)
- My first experience of The Internet was very similar to this. It should be bizarrely unfamiliar to anyone under 20 or so. (via) Get this: I typed ‘exit’ instead of just closing the browser window when I was done messing with it, because some habits cannot be broken.
I finished two builds of pkgsrc-2011Q1 on DragonFly; the reports are available. The only real breakage is security/heimdal. There’s a patch for that, though.
Rui-Xiang Guo is looking for testers for wip/chromium. That’s the zippy Google browser. He especially wants DragonFly users – this would be useful, especially since I think Firefox 4 does not build on DragonFly right now.
We’ll have a full set of pkgsrc packages for the upcoming DragonFly 2.10 release, built from the most recent quarterly release of pkgsrc: 2011Q1. For the curious, here’s the build reports for i386 and for x86_64 architectures.
The usual way for building pkgsrc packages from source is ‘bmake install clean’, to build and install the package, and then clean the work files from building it. Since the recent change to DESTDIR, where a binary package is built before installation, you may want to add ‘package-clean’ to the list, so that the binary package is also removed after installation.
No, 2.10 is not out. I built packages for pkgsrc-2011Q1 on 2.9, and set it to think 2.10 so that the pkgsrc tools wouldn’t complain. We’re close enough to release that this shouldn’t be a problem. The packages are available for x86_64; i386 packages coming “soon”. See my note to users@dragonflybsd.org for details on accessing these packages.
Get out your wallet! I encourage purchasing here.
- You should buy a SSD. Not necessarily news to you, but that article does a good job of summarizing why.
- On the other hand, SSD prices are already on their way up/availability is way down. Japan’s disasters are having a ripple effect through the high-tech supply chain. Either buy immediately or get ready to wait for a while…
- Introduction to Architecting Systems for Scale – you either don’t care, or find scaling questions immediately engaging. I am one of the latter, so here’s the link.
- I’ve been watching pkgsrc-changes@netbsd.org for a little while. One thing I’ve discovered: there’s a lot of updates going on! Another thing that’s nice to see: DragonFlyupdates, including ones that help with our move to gcc 4.4.
- Aw, no more Kermit. (via) Not that I have a use for it at this point, but still: aww. I bet in about 10 years I’ll say the same thing about… gopher? Remember that? It’s not even supported in Firefox 4 now, which kinda makes me feel sad. And old.
- Server plans: Facebook vs. Google. (warning: Facebook article is somewhat giddy.)
- The infinite hard drive. (via I lost it, sorry)
Here’s an extra little thing: next time you’re dealing with dusty computer equipment, remember this picture:
That is what happens to an exposed RJ45 port after a few years in a salt mine (my employer). This was inside an enclosed, mostly-sealed structure, too.
Branch tomorrow, release in 2 weeks. There’s a ton of new features for 2.10, so this will be a fun release. I’m trying to get pkgsrc-2011Q1 packages built for 2.10 ahead of time, too.
I already noted that the quarterly release is out, but the pkgsrc-2011Q1 release announcement is available now. There’s good reasons to link to it – the list of updated packages, new packages, and credits for the work people have been doing. Here’s the part I really want to pick out:
We’re aiming to make this the last branch to support non-DESTDIR packages. We have almost finished the transition to DESTDIR installation, where a staging directory is used to make a binary package, which is then managed by the pkg_install tools.
The reason I’m highlighting this is: it’s good news! One of the long-term complaints with pkgsrc is that the upgrade process is painful. If you try to build an upgrade and the build processfails after uninstalling the existing package, not only are you not getting the upgrade, but you’ve lost the existing package. Binary packages for download helps with this (and generally is faster), but only so many packages can be built separately and made available for download.
Building a package separately and then installing from there removes these issues. No binary redistribution issues, actual downtime is minimal, and the package is known to work when an upgrade happens. This removes most of the problems I’ve heard raised about pkgsrc over the years.
It’s tagged, though there’s no release announcement yet. Working on building binaries starting tonight…
There’s a number of pkgsrc packages that have a combination of security vulnerabilites and lack of updates for more than a year which is placing them on the chopping block. (Follow the discussion to see which ones make it off the list.) The removals will happen after the next branch, pkgsrc-2011Q1, which is itself due in two days.