collectd v3 to v4 migration guide

Since collectd v4 is a new major version it is not backwards compatible to old v3 installations. This doesn't mean you cannot use v4 if you've already used v3, but you will need to migrate your data and configfile. This document is here to guide you along the necessary steps.

There are two major steps: First, you should migrate your configuration. Once that's done your RRD-files need to be migrated, possibly deleting the data that was collected when the first step was tested.

Migrating the configuration

The layout of the configfile hasn't changed dramatically, but the name of options may have changed and in some cases functionality has been moved into plugins. We recommend you use the vanilla configfile installed with make install, open your working configfile next to it, and transfer your changes "by hand".

You will need to make the following changes:

• Quote strings. All strings need to be quoted in double quotes, ‘"’, even paths and IP-addresses. The only exception are "words", i. e. strings that only consist of the characters 'A' to 'Z' and 'a' to 'z'.
• Replace the "modes". Version 3 used to have the "modes" Local, Client, Server and Log, all of which don't exist anymore.
  If you used the Local or Log modes you will need to load the rrdtool-plugin (if you used Local) or the csv-plugin (if you used Log). You can, of course, load both if that is any use for you.
  If you used the Client or Server modes you will need to load the network-plugin and configure the Server (if you used Client) or Listen (if you used Server) option. Since the ex-server is supposed to save the data to somewhere, you will need to load the rrdtool-plugin there, too.
• Enable logging. Since even logging is done by plugins in version 4 you will need to load one of them if you want to have diagnostic or informative output. We recommend you load the syslog-plugin and set the log-level to at least warning. An alternative is the logfile-plugin which writes the output to a given file or STDOUT.
• Remove unused plugins. When running a daemon in mode Server it usually loaded next to all plugins in order to be able to save the values submitted by these plugins. This is not true anymore. The description of the various "data sets" (usually called "types") is now loaded from a file called "TypesDB". So if you load the network-plugin and configure it to listen on an interface, only load those plugins the daemon is supposed to collect from the host it's running on.
  If, for example, the "server instance" instance shall receive the MySQL-statistics of your database server, you do not need to load the mysql-plugin. But if you want to see the interface-traffic of the machine that's running the "server instance" you need to load the interface-plugin (formerly called traffic) on that machine.

Once you have changed your config, let the daemon run for a couple of minutes. If you get warnings like read-function of plugin `foobar' failed. you have either made a mistake when configuring that plugin or that plugin isn't appropriate for your system, e. g. if you have loaded the cpufreq-plugin on a desktop machine. Check if all the data you collected using version 3 is being collected with version 4, too. If all looks good, you can start migrating the data.

Migrating existing RRD-files

Migrating existing RRD-files is not an easy task, because there are many of them, they're hard to change and given their nature it's kind of time-critical. There are two scripts in the contrib/ directory which should assist you with that task.

The script migrate-3-4.px scans your data directory and prints a bash-script to STDOUT that is supposed to do the migration for you. It does not execute the commands right away to give you the possibility to look over the commands and make your own adjustments.

The script extractDS.px is used by the code generated by migrate-3-4.px to split one RRD-file with many DSes into multiple RRD-files with only one or two DSes. You don't have to worry about it's usage much, but it must be found when the migration-script is being executed.

We recommend to use those scripts along this lines:

When the script is done start the (version 4) daemon and see if all files that were updated before are still being updated. If some filenames didn't get translated correctly (the disk-plugin is a likely candidate for this) make the appropriate changes to the /tmp/migration.sh (or however you named it) and try again.

Explore the possibilities

We put much effort into the creation of collectd 4, so this new major version provides many new and, hopefully, interesting possibilities. Here are some examples to get you started:

• The rrdtool-plugin provides caching to improve IO-performance. This hopefully allows very large setups.
• The network-plugin allows you to forward the data is has received. This allows for multicast-aggregation or multilayered network layouts.
• The perl-plugin provides the internal API to Perl-modules, so you can write your own plugins in Perl if C isn't flexible enough for you.
• The exec- and unixsock-plugins provide more means of custom interaction with the daemon.

Now that you're done: Welcome to the next generation of statistics collection. Enjoy! :)