Whilst debugging our CollectD and Graphite setup, I found that CollectD was generating hundreds of repeated errors.
Jun 13 17:11:14 hmon collectd[15336]: rrdtool plugin: rrd_update_r (/var/lib/collectd/rrd/hmon/processes/ps_state-running.rrd) failed: /var/lib/collectd/rrd/hmon/processes/ps_state-running.rrd: illegal attempt to update using time 1371136272 when last update time is 1371136272 (minimum one second step)
Jun 13 17:11:14 hmon collectd[15336]: rrdtool plugin: rrd_update_r (/var/lib/collectd/rrd/hmon/processes/ps_state-sleeping.rrd) failed: /var/lib/collectd/rrd/hmon/processes/ps_state-sleeping.rrd: illegal attempt to update using time 1371136272 when last update time is 1371136272 (minimum one second step)
Jun 13 17:11:14 hmon collectd[15336]: rrdtool plugin: rrd_update_r (/var/lib/collectd/rrd/hmon/processes/ps_state-zombies.rrd) failed: /var/lib/collectd/rrd/hmon/processes/ps_state-zombies.rrd: illegal attempt to update using time 1371136272 when last update time is 1371136272 (minimum one second step)
These messages were occurring every minute or so and filling up /var/log/messages.
A quick search of the Internet shows that unfortunately this is a known issue which will be fixed in a further release of the CollectD 5.1 branch.
We were currently running
collectd-5.1.0-1.el6.rft.x86_64
collectd-rrdtool-5.1.0-1.el6.rft.x86_64
An upgrade to 5.3.0 fixed it!