31 Replies to “Hammer and FreeBSD”

  1. could be openbsd. I really don’t care which bsd it’s going to be.

  3. OpenBSD has actually really wanted (and needs) a modern filesystem. I think they would receive it with truly open arms. The NetBSD guys have been talking about it recently as well, at least the user “medfly” is already trying to port it.

  4. yeah that’s true.

    i could try on openbsd if you or anyone wants to see it, though i can’t guarantee when it’ll be done (or even when i’ll start, but i really like to start working on this).

  5. I would gladly contribute 100 usd to help start development in openbsd.

  9. Nothing for now.

    I have another porting stuff I need to get done (which was asked by someone else a while ago), but this shouldn’t take too much time considering the code size and complexity. After that I can start hammer1 (hopefully on openbsd) probably sometime in may.

    The problem is dragonflybsd isn’t a fork of openbsd and I’ve never written openbsd kernel code. I’ll probably need to understand how similar these two are before I ever try to write something.

  10. Above gsoc and github links are for hammer2. hammer2 isn’t finished even on dragonfly. What’s been discussed here is hammer1 which is basically a different filesystem. hammer1 has been default fs of dragonflybsd since (i think) 2009 or somewhere around.

  11. I’ve understood from previous discussions that HAMMER relies extensively on DragonflyBSD’s VFS layer. Partly because of this previous attempts at porting have failed.

    Also, are we discussing HAMMER 1 now because HAMMER 2 isn’t going to be finished?

  12. honestly, I think the respective foundation should be paying market rate to port something like HAMMER 1. As previously noted, OpenBSD (and NetBSD) have a more critical need because FreeBSD has ZFS.

    I respect the “I’ll do it because I like the challenge” approach, but it’s a real project that provides real value and I think that’s worth real money. :)

  13. Has nothing to do with the status of hammer2 since I don’t know the status of hammer2.

    Any kernel fs more or less relies on vfs and/or page cache interface that kernel has, but people somehow (try to) port them whether that’s ntfs or xfs or reiserfs.

  14. I am not sure tkusumi asked for our votes but if I could cast mine it would be for porting HAMMER to OpenBSD. OpenBSD would benefit tremendously from a modern file system. There is some preliminary work done by Walter Neto on porting WAPBL to OpenBSD which in hold due to Neto’s day job. https://marc.info/?l=openbsd-misc&m=145917025913348&w=2. However WAPBL targeted use cases are mostly related to embedded. Having HAMMER1 in OpenBSD will essentially mean that I could finally have supper secure file server too. It would be interesting to look at DragonFly NFSv3 implementation which seems to be second to none in terms of speed and try to improve OpenBSD NFS implementation.

  16. HAMMER seems to be the best candidate to be ported, also when factoring in required support from the core OpenBSD devs. HAMMER2 is still far from ready.

    tkusumi, how do you see the timing/planning of the HAMMER port? And you are confident that you will be able to complete the port? I would really look forward to it, and many other users as well.

  17. Can’t guarantee anything, but say starting from mid to late May which is a month from now, I want it to be able to do regular filesystem operations (mount/umount/open/read/write/close/etc) by early 2017 if not too stable. I now have plenty of time till August.

    The initial target will be what dillon (Matt Dillon) has done on DragonFly by 2009 or so. There are some other features added by other devs in 2009-2011 time frame, but these are either not enabled by default, or incomplete, or unstable. It also has some design issues because no such thing was initially designed to be implemented. People are basically only using HAMMER code written by Matt, so ignoring the rest won’t be a problem anyway.

    Technically speaking the VFS difference obviously is difficulty of porting any filesystem (not specific to HAMMER). As far as I know Matt has rewritten namei, and some incompatibility in VOP operations compared to FreeBSD. It also has a large filesystem buffer, plus there is an idea of “layers” within a buffer which is used by HAMMER. These are something that may or may not be an issue on porting HAMMER.

    Locking probably needs to be rewritten or reconsidered since lwkt tokens in DragonFly behaves quite differently from ordinary kernel mutex as well as design difference from OpenBSD kernel.

  18. I read that transparent compression was planned for HAMMER1 was this ever enabled?

  19. HAMMER(v1) would totally make an excellent addition to OpenBSD! A modern filesystem is really one of the things missing from this OS and ZFS will obviously not happen as it is overly complex. I guess that most people would agree that OpenBSD could make use of a newer fs (Henning Brauer is another developer who mentioned this about a year ago: http://quigon.bsws.de/papers/2015/asiabsdcon/mgp00022.html).

    Thanks for considering some serious port, tkusumi! You definitely won’t have a lack of volunteers to help test this. If you start this project, please use some means of communicating progress and keeping people informed. Too many interesting projects unnecessarily die a silent and lonesome death.

    BTW: Does anybody have any information about how the GSOC project around porting HAMMER(v2) went? It was announced on Undeadly once – only to never be heared from again… Even if the project did not succeed it would have been nice to know a bit of the background (how far did it get? what where the problems? etc…)

  20. Anonymous – compression was worked on as a project but I don’t remember the final implementation.

    https://www.dragonflydigest.com/2013/06/27/12033.html

    Deduplication is also available and works very well:

    https://www.dragonflydigest.com/2013/08/28/12369.html

    Kraileth – the person who was working on that Hammer2 port got a job offer too good to refuse right at the start of the GSoC period – and also too much to work on both projects at once, unfortunately.

  21. If anything, the hammer2 efforts might lead to the developers on the OpenBSD team who are most willing to help out.

  22. Justin Sherrill: Ah, thanks for pointing that out. That’s definitely a reason to stop the project and while it’s a pitty for most of us it’s probably good for the student.

  23. A port to FreeBSD was considered but it’s too complex:
    https://wiki.freebsd.org/PortingHAMMERFS

    The recommendation from Matt was to wait for hammer2.

    Porting to OpenBSD will likely be tougher but I agree OpenBSD needs it the most.
    I wonder what ever happened to the GSoC effort: perhaps it was not desired effect but OpenBSD tends to keep those projects closed, or perhaps the student dropped the ball completely before starting.

  24. Pedro – see my answer to Kraileth above – the student got suddenly employed before he could get very far in GSoC. He’s still around in #dragonflybsd EFNet IRC, but very, very busy. (and better-paid)

  25. Anonymous: thanks for the explanation.

    I am somewhat concerned that a port to any non-dragonfly OS, will be inefficient and will make hammerfs look bad. Still it would be nice to see it ported to the other BSDs.

  26. Hi Pedro,

    I am curious, why would a port to the other BSDs be inefficient? Wouldn’t this be the case for any other filesystem? I.e. ZFS on FreeBSD has come a very long way from when it was first ported. But its quite ok now after a good bit of work.

  27. Hi K,
    Read the FreeBSD wiki discussion: Matt heavily modified all the key components in the kernel to make hammer perform well. The changes are a complete evolution of the OS, and Matt obviously did it with expertise but with no thought for backward compatibility. Curiously, I think the namecache is more similar to linux than to BSD now.

    ZFS is revolutionary but it still meant for the unmodified Solaris VFS and while there were differences with FreeBSDs VFS, they were manageable.

    I may be proved wrong of course, and that would make me very happy ;).

  28. Hammer2 hasn’t had any commits in like 6 months…does it mean its dead?

  29. It’s software; it can’t be dead. Or alive.

    Also, Matt Dillon got married; most people don’t work on filesystem commits while on their honeymoon.

  30. Hammer2 is taking a bit of a break for now. The clustering bits are quite a complex, and Dillon was a bit burnt out. He mentioned some time ago that he’s keen on doing this right. It is shaping up to be quite an impressive filesystem.

  31. Whoa. I just had a chance to read the HAMMER2 design document. Really interesting. On paper it seems to be what I need in my lab. Hopefully it gets out soon. Best of luck Dragonfly team!

Comments are closed.