Difference between revisions of "Roadmap"
(→Gearman: added some notes about a mod-gearman worker plugin)
(→Store to RDBMS: mention write support in PostgreSQL available since 5.2)
|Line 179:||Line 179:|
:''See also: http://github.com/bcotton/collectd_dbstore/tree/master''
:''See also: http://github.com/bcotton/collectd_dbstore/tree/master''
=== Query wprobe ===
=== Query wprobe ===
Revision as of 12:47, 9 January 2013
- 1 Description
- 2 Implemented features
- 3 Planned features
- 3.1 Plans for 5.0
- 3.2 perfmon output
- 3.3 Library for binary network protocol
- 3.4 FreeSWITCH plugin
- 3.5 Match and target for meta data
- 3.6 Event infrastructure
- 3.7 NetCmd plugin
- 3.8 Erlang plugin
- 3.9 InnoDB addon to MySQL plugin
- 3.10 SNORT plugin
- 3.11 WPAR plugin
- 3.12 MongoDB plugin
- 3.13 Write MongoDB plugin
- 3.14 ZeroMQ plugin
- 3.15 Lua plugin
- 3.16 Modbus plugin
- 3.17 Gearman
- 3.18 LXC plugin
- 4 Wishlist / Ideas
- 4.1 RRDtool plugin: Type-specific configuration
- 4.2 Distributed setups
- 4.3 Embed more scripting languages
- 4.4 Store to RDBMS
- 4.5 Query wprobe
- 4.6 output_text plugin
- 4.7 Apache plugin: Per virtualhost stats
- 4.8 pf (packetfilter) plugin
- 4.9 Per-plugin interval configuration
- 4.10 Drop privileges
- 4.11 Summarize CPU information
- 4.12 CPU percent usage
- 4.13 apcupsd
- 4.14 Swap plugin
- 4.15 collectd-nagios
- 4.16 NTPd
- 4.17 Reverse DNS Check
Life goes on. Which features are planned in future releases? This page is more a list of ideas than a fixed plan for the future, but it may give you an idea where the journey is going. If you want to contribute but don't know what is to be done, simply drop us a short mail and we'll get you started.
This file is not automatically generated and may thus be out of date from time to time. To give you an idea how fresh this information is please take a look at its history.
The following features have been implemented and will be released with the next minor version. You can check out our Git repository to test these features. They can be found in the
master branch. Any feedback and bug reports are welcome!
A new Redis plugin has been written by Andrés J. Díaz. It connects to one or more Redis databases and queries statistics such as commands received, memory allocated and connected clients. The code was merged into the
master branch on August 17.
Andrés J. Díaz has written a patch that introduces a global
timeout_g variable rather than the hard-coded default (two). The patch has been sent to the mailing list on December 17, 2009.
- I've rebase the code to the master branch and contacted Sebastien for some feedback. —octo 13:26, 4 August 2010 (UTC)
- Sending and receiving data via AMQP is now possible. Possible future todos:
- Use the parsetree interface of YAJL to parse received JSON data. Currently, the parsetree code has not yet been merged into YAJL's master branch.
- Possibly rename the AMQP plugin to Write AMQP. For consistency's sake it might be best to rename the CSV plugin, RRDtool plugin and RRDCacheD plugin, too. —octo 06:55, 18 August 2010 (UTC)
- No, since the AMQP plugin can send and receive values, renaming the plugin is not appropriate. —octo 20:43, 4 September 2010 (UTC)
- Merged to the
masterbranch. Parsing JSON still not possible, though. —octo 08:42, 6 November 2010 (UTC)
Aurélien Reynaud has sent a patch for an Logical Partitions (LPAR) plugin to the mailing list on August 10. It collects information about the (a?) virtualization infrastructure of AIX. —octo 07:23, 18 August 2010 (UTC)
The following features are actively being worked upon. They are in various stages of completion and may or may not make it into the next release. They should have a contact (name) associated with them, so people who are interested can contact the person working on a feature. Please don't add email addresses of other people without their permission. If you want to contact a person below, please look for the email address using the mailing list archives.
Backwards incompatible changes have been collected on the Version 5.0/Plans page. Make sure every point raised there has either been addressed or has actively been dismissed (i.e. not forgotten about).
Dave McLellan is looking into a text output plugin which creates perfmon compatible output (last contact: April 15th, 2009). See also: #output_text_plugin
Library for binary network protocol
Write a library for the binary network protocol, so that external applications have an easier job interfacing with collectd. The new authentication and encryption feature (added in version 4.7) makes this even more desirable. Octo is working on it.
- Sending plain and signed network packets is now possible. The code is available from the
ff/netlibbranch. --octo 06:51, 18 August 2010 (UTC)
- Sending encrypted packages is now possible, too. —octo 13:37, 19 August 2010 (UTC)
- See also:
Plugin which connects to FreeSWITCH, a free IP-telephony software, issues commands periodically and parses the returned output. The plugin was originally written by Leon de Rooij and is currently available from the
Match and target for meta data
It'd be nice to match values by the meta data attached to it and to attach or re-set meta data for values. At least something simple (see the
MARK match and target in ip_tables) would be nice.
EVENT command to the UnixSock plugin. This command can be used to count events, such as page loads and the like. The goal is to make it possible for applications to implement statistics collection with the minimum of overhead. powdahound is looking into it.
The UnixSock plugin is limited to UNIX domain sockets. It'd be great if it was possible to access this infrastructure from remote hosts, for example for receiving values. Some initial code lives in the
ff/netcmd branch but isn't ready yet. The idea is basically to take the UnixSock plugin and make it possible to use network sockets, too. Several people have expressed interest in picking that up, but so far no progress has been made. It's safest to keep development cycles short and provide many small patches as soon as possible.
InnoDB addon to MySQL plugin
Morten Hanshaugen is working on adding InnoDB statistics to the MySQL plugin. Last contact was on January 27th via email.
Benjamin Petrin is working on a plugin that will gather statistics from SNORT, an open source network intrusion prevention and detection system.
Julien Ammous is working on a communication plugin which uses ØMQ (aka. ZeroMQ) as transport mechanism. Apparently it works, but it'd be great if it was merged with the the
ff/netlib branch (see: #Library for binary network protocol) before including it in master. The code lives in the zeromq branch on Github. —octo 08:12, 5 November 2010 (UTC)
Julien Ammous started writing a plugin which will eventually allow users to write plugins in the scripting language Lua. The code is available from the
ja/lua branch and
schmurfy/collectd/lua. —octo 08:02, 26 November 2010 (UTC)
Ivo De Decker has implemented a grouping feature for the
<Data /> blocks of the Modbus plugin. The code is available from the
id/modbus branch. Last contact in a private email on January 12th, 2011. —octo 10:25, 14 January 2011 (UTC)
Find out what this does and expand the page. —octo 22:32, 11 March 2011 (UTC)
Gearman allows to distribute work-loads to multiple workers. A (Gearman) client dispatches jobs to the Gearman job server which then distributes them to Gearman workers. The worker executes the requested task and sends back the result to the client (thru the Gearman job server). I'm not sure how collectd might benefit from that. -- tokkee 11:58, 7 July 2011 (UTC)
It might make (some) sense to implement a gearman worker plugin which receives Nagios perfdata sent by the Nagios mod-gearman add-on and dispatches that data to collectd. I think the one challenging question is how to do the data mapping. The following approaches come to my mind:
- Use the same approach as in most generic plugins, i.e. let the user specify each and every data set (check command and/or service description, perfdata label) plus how to convert that info collectd types.
- Use a similar approach to PNP4Nagios, i.e. use default values (e.g. host = Nagios hostname, plugin = mod_gearman, plugin instance = check command or service description, type = gauge, type instance = perfdata label) and let the user overwrite (some of those) information, c.f. PNP's custom templates.
- Combine the two approaches by defining a default data-mapping and (maybe) let other mappings "inherit" from that.
tokkee 10:40, 9 January 2013 (UTC)
François-Xavier Bourlet is working on a LXC plugin. LXC is a linux-specific containers implementation (à la OpenVZ) based on kernel cgroups and namespaces. The code is available from
dotcloud/collectd on Github.
Wishlist / Ideas
The following is a list of ideas we had but so far nobody has stepped up to implement them. Feel free to add your own “cool stuff” :)
RRDtool plugin: Type-specific configuration
This would allow to keep interesting data longer and less interesting data only for a short time. Also seasonal data could be added. It's not reasonable to add seasonal data to all RRD files, because it operates on unconsolidated primary datapoints (thus taking up a lot of disk space) and you can't specify one seasonal period that fits all kinds of data. A discussion of this feature can be found in the mailinglist's archive.
Maybe an option that can be passed to write functions by the Chains mechanism would be a good idea for implementing this.
Make it possible to have several "servers" with shared storage. When one server dies another one takes its place, so that you don't have gaps in your statistics. This is probably best implemented as a match that matches only those values the host is responsible for. A really simple first step is available in form of the Hashed match as of version 4.9. Adding automatic fail-over would be a nice second step.
A solution for non-shared storage is probably even harder to implement and likely not worth the effort.
Embed more scripting languages
- ECMA Common Language Infrastructure
- Work in progress, for information contact octo.
- Best known implementation is likely Microsoft .NET; for UNIX there's a free implementation called Mono.
- Target language of a variety of programming languages, for example C#.
- Work in progress, see #Lua plugin above.
- It is now (November 2010) at version 2.10 and provides a stable API. Whether the interpreter has become thread- and reentrant-safe has to be determined again.
Store to RDBMS
Being able to store values in an RDBMS would be nice. Bob Cotton has put together some SQL for PostgreSQL and a Perl based plugin to store values into it. This could serve as a starting point as a C based plugin with the same SQL backend.
Starting with version 5.2, the PostgreSQL plugin supports storing data in PostgreSQL. This has been done using a generic approach by letting the user specify the query that is used for storing a value-list and, thus, allowing arbitrary database layouts. See the manpage for details.
wprobe is a interface to query/manage wireless links, written by the guys over at OpenWrt.
Apache plugin: Per virtualhost stats
mod_watch gives access to stats like:
domain.com 8190 539056 20965428 4813 4813 0 14.520 0 4286618408 domain2.com 8190 539056 20965428 4813 4813 0 14.520 0 4286618408
Why not to use it?
- Couldn't find a hopepage for this project. Where does it live? Is it still alive? -- octo 21:16, 22 September 2009 (UTC)
pf (packetfilter) plugin
pf is a firewall available on OpenBSD and FreeBSD. There are quite a lot of counters which should be available (packets/seconds, octets/seconds, by states, tables, ...). Some munin plugins for reference : pf_bytes, pf
Per-plugin interval configuration
It'd be great to be able to update the interval for each plugin separately. The syntax could be something like the following:
# foobar is updated every 42 seconds. <LoadPlugin "foobar"> Interval 42 </LoadPlugin> # qux is updated with the global default interval. LoadPlugin "qux"
This is in the works, see 
A couple of plugins require special privileges. It'd be great if we could find a way to allow these plugins to do their job without requiring collectd to run with privileges.
Under Linux, a solution might be to use capabilities(7) to deal with this problem: Assign the collectd binary the permission to a certain capability, for example to open a raw socket. The plugin can then active this capability if needed. For example:
setcap cap_net_raw=p $prefix/bin/collectd
For other UNIXes, this solution is not applicable. In order to drop privileges in the traditional UNIX way, plugins such as the Ping plugin would need to live in a separate process space, since it needs to open new sockets whenever the address of a pinged hostname changes.
Summarize CPU information
Implement a configuration option in the CPU plugin, which allows to dispatch the sum of all CPUs, rather than information about individual CPUs. —octo 06:31, 7 December 2010 (UTC) Please see patch http://mailman.verplant.org/pipermail/collectd/2011-March/004415.html
CPU percent usage
It will be nice in FreeBSD (may be other unixes) divide values by stathz and multiply by 100 to get usage more like percent usage (munin does like this, see sysctl -n kern.clockrate). In linux it can be achieved dividing to HZ value. Please see patch http://mailman.verplant.org/pipermail/collectd/2011-March/004415.html
In many cases on one server runned many apcupsd on different tcp ports. Plugin should be able to poll many apcupsd.
The ReportByDevice option has been added to the Solaris code of the Swap plugin in version 5.0. Similar information can be read from
/proc/swaps under Linux. Make this option available there, too. —octo 07:55, 2 January 2011 (UTC)
FreeBSD and OpenBSD
The prototype of the swapctl(2) function on BSD systems differs from the same function on SunOS. Its prototype is:
int swapctl(int cmd, const void *arg, int misc);
The configure script can already check for this second version, but the Swap plugin can not yet handle it. Implement this.
Support querying multiple data-sets; apply consolidation to selected data-sets. --tokkee 14:46, 5 January 2011 (UTC)
It'd be great to be able to see only the declared peer instead of all peer (declared and candidate). Actually we can't see which server is declared as peer.
Reverse DNS Check
XANi suggested to ignore values if the reverse DNS of the hostname submitted over the network does not match the sending IP. This could be implemented by adding the remote IP as meta-data to received value-lists in the network plugin and then add some match plugin to check that kind of information. --tokkee 06:44, 16 August 2011 (UTC)