Minutes from 2020-07-20

Attendees: Slawomir Strehlau, Octo, Pawel Zak, Matthias Runge, Pawel T, Piotr Dresler, Sunku, Barbara Kaczorowska, Elene Margalitadze, Jabir Kanhira Kadavathu, Svetlana Schmidt

Collectd 6.0 Design Aspects

• Having label based metrics, plugins converted are important
• Bunch of backward compatibility items in the code, for example, libvirt would transparently get converted into virt however the term libvirt has been hardocded multiple places; little hacks like this are in many places in the code, should be cleaned up
• Tree wide removal in absolute types, which is merged
  • https://github.com/collectd/collectd/pull/3507
• Not necessary for all plugins, go with subset of plugins
• Features based in doc discussed during meetup during Feb 2020:
  • https://docs.google.com/presentation/d/1MEIusZJEpgUCegFKP6Qfup7kV04ketvH5M6YnPNaObI/edit#slide=id.g7db70ed343_0_7
  • Ability to add/remove plugins without reloading Collectd daemon: is relatively complicated - no one is working on it
  • Dynamically change plugin configuration without restarting Collectd daemon - whats needed is someone to come up with design, propose concrete path forward. Its not a heavily contested feature but its hard to implement. Open to contributions
  • Ability to customize/add metadata/labels to the metrics and Ability to control level of metadata/labels exposed by read plugins for easier consumption - this is WIP
    • convention similar to Prometheus for example labels with starting and ending '_' are for write plugins,
    • could probably get rid of metadata but rather have info available via label set
    • Minimal CPU footprint - could consider this at a later time. This isnt a blocker. Plan to run regression test as things approach to beta stage so 6.0 doesn’t occupy lot of CPU
    • One item that might reduce CPU footprint is modify looking up a string in a binary search tree, not converting to a string might result in more string comparisons
  • Ability to collect metrics at sub-second level - already have this. Currently use weird time format interanally inside collectd, value 2 power 30 is considered 1 second, considering changing this to counting microseconds, as its easy to convert to human readable. Its purely internal thing, externally expose seconds with 3 digits of precision after a comma. Can convert to something human readable. Its internal data structure, little bit unusual to use. Changing it mean touching a lot of places, its not obvious that this will improve situation much, lot of work but little gain. It would make it easier for new contributors however
• Container packaging, better alignment with Kubernetes - not a release tiems
• Ability to produce/consume on-wire metric streams in a standardized format - plan to have a new write plugin that is exporting opentelemetry protobuf stream, ideally would like it to be much more for example to have ascii export format that opencensus use; we already have write_prometheus which uses prometheus format
• Support for distribution metrics - interns working on it
• Florian is updating a design doc -> will publish a doc. There is a page in wiki, new plugin maintainers guide, it’s a starting point -> will look to expand this