MongoDB and logrotate

December 22, 2011 12:26 pm    Posted by Viktor Petersson    Comments (4)

I’ve been using MongoDB now for some time in production. It’s great, and I really love how easy it is to set up and scale with replica sets etc.

There is one thing that bugs me with MongoDB though. Instead of following the praxis of using Logrotate, they’ve decided to re-invent the wheel by building in a log rotation-feature (that is less powerful than Logrotate). The documentation on MongoDB’s log rotation-feature can be found here, but the gist of it is that you send ‘kill -SIGUSR1 PID‘ to MongoDB and it will automatically rotate and rename the old logfile to something like ‘mongod.log.2011-12-22T17-05-50‘.

There are some issues with this. If we were to only rely on the built-in log rotation-feature, we would need to address the issues of vacuuming old files (as this is not supported). Moreover, the built-in logrotation feature doesn’t support compression of old logs, which means your old log files will take up a lot more space.

There is however a way to use MongoDB with logrotate. It’s just a bit more messy than most other applications. If we assume that MongoDB is configured to spit out log-files to ‘/var/log/mongodb/‘, we could simply use the following logrotate script:

/var/log/mongodb/mongod.log {
	daily
	missingok
	rotate 7
	compress
	delaycompress
	notifempty
	create 640 mongodb mongodb
	sharedscripts
	postrotate
		killall -SIGUSR1 mongod
		find /var/log/mongodb/ -type f -regex ".*\.\(log.[0-9].*-[0-9].*\)" -exec rm {} \;
	endscript
}

The real ‘hack’ here is the somewhat ugly find-command that deletes the log-file generated by MongoDB. It uses a Regular Expression that only matches the MongoDB-generated files, and leaves the logrotate-files alone (as they are named mongodb.log.[0-9].gz).

4 Responses to “MongoDB and logrotate”

  1. sath says:

    Thanks, It works.

  2. Adrian Nye says:

    If you are using a replica set, then the postrotate command causes the primary to step down, and there is a short period of unavailability before the secondary takes over. If you use Mongo’s own rotation, this doesn’t happen.

  3. Mark says:

    you can probably skip the “-SIGUSR1″ and old log cleanup with the user of the copytruncate option in logrotate.

    copytruncate
    Truncate the original log file in place after creating a copy,
    instead of moving the old log file and optionally creating a new
    one, It can be used when some program can not be told to close
    its logfile and thus might continue writing (appending) to the
    previous log file forever. Note that there is a very small time
    slice between copying the file and truncating it, so some log-
    ging data might be lost. When this option is used, the create
    option will have no effect, as the old log file stays in place.

    http://linuxcommand.org/man_pages/logrotate8.html

    So your config would look something like this:

    /var/log/mongodb/mongod.log {
    daily
    missingok
    rotate 7
    compress
    delaycompress
    notifempty
    sharedscripts
    copytruncate
    }


Leave a Reply