ReiserFS

I haven't done it myself, but I think you should just be able to define a disk image as raw/unformatted, and then create whatever filesystem you want on it.

Reiserfs kernel support appears enabled, at least in the various Linode kernels I'm running, though you may or may not need to install user level tools depending on distribution.

Oh, and I think this will eliminate the possibility of that disk image participating in backups, as I seem to recall that only support ext2/3 images at the moment.

– David

@xtractly:

Is there anyway to use reiserfs instead of ext3 on a linode. We ran into problems trying to store a few million files. The issues are known ext3 issues.

Have you considered ext4 instead?

Reiserfs is a well designed filesystem but the chef developer is now locked up and everyone else has moved away from it.

Maybe it's possible to look at what your application is doing? It looks like it should split its storage over multiple directories like BackupPC or Cyrus does instead of dumping too many in one directory. That's better regardless of filesystem.

With Reiser in jail for murder, moving to ReiserFS is not a terribly good idea…

My understanding is that ext4 should be a bit faster with many small files, and XFS should be significantly faster. You might want to give XFS a try before ReiserFS.

And finally, this might be a good time to point out that if you're storing millions of small files, a database makes a heck of a lot more sense than a file system.

ReiserFS is a stable and mature filesystem. I don't think it really matters that the original creator murdered his wife, as far as ReiserFS's performance and stability are concerned. AFAIK there aren't any major issues that only Hans Reiser himself would be able to fix. I've been using ReiserFS on a number of VMs (not Linode) without any issues.

Having said that, I second @Guspaz's recommendation that you check out alternatives, including the database option. It is usually considered a bad idea to store blobs in a database, but if you literally have millions of little pieces of data, something like PostgreSQL might give you much better performance and less headache overall.

Besides XFS, another option would be btrfs, which incorporates some of the better ideas from ReiserFS. It's been in the mainline kernel for while, so it's pretty stable these days. I'm not sure if Linode kernels support it, though.

I have only good things to say about XFS. Give it a try.

Thanks for all the feedback.

A database wouldn't work for what we are trying to do.

The RAW partition formatted with reiserfs was the simplest solution. It is working well for now, and is easy to resize as needed.

Perhaps XFS or ext4 would work well also, but I am very familiar with reiserfs and it handles millions of files/directories with no difficulty.

Eventually we are migrating to a CDN for storage, so this is a temporary situation either way.