View source for Minutes from 2020-08-31

Attendees: Dago, Barbara Kaczorowska, Elene Margalitadze, Emma Foley, Erick, [[User:Octo|Octo]], Jabir, Kamil, Matthias, Pawel Zak, Piotr Wysocki, Piotr Dresler, Svetlana Shmidt, Waldek Wojtkie, [[User:Sranganath|Sunku]]

== 5.12 release ==
* Fixes from 5.11 will be merged into 5.12 (issue 3542), found a new PR merged into master (not main)
** Florian will merge everything to 5.12, and then 5.12 will be merged to main
* Sept 2nd release is still the plan

== CI OS versions ==
* Solaris tests https://github.com/collectd/collectd/pull/3552
** Dago and Florian are looking into it
* Issue3539 -> old CI versions
** MarkF had originally set this up
** 2 repos configured for this, first one is a docker file for all arch with each arch with different branch (not all in main branch), then the built docker containers for all of these based on debian labels, which are in turn used to build collectd
* We need to copy what we have in old debian distros into new branch/base image for new debian images
* Then we need to change Collectd CI to use newer images
* Florian will try this for atleast one new image and document it
* CI docker is open for all trusted contributors
* Regarding 3539 -> not sure what happened. Possible reason is that "ldd: src/.libs/lvm.so: No such file or directory".
** Florian is to follow up in the PR
** https://github.com/collectd/ci-docker/blob/el6_x86_64/checks/check-built-plugins.sh that causes a failure
* Volunteers welcome

== 6.0 updates ==
* Florian spent time into dividing code into smaller chunks but ended up spending lot of time and not being successful
* Rather to submit all the code and give time till end of the year
* Will allow more time for people to migrate plugins
* Alternative is spend serious time and review 7000 loc
* Matthias: it makes sense to merge this and handle any failures afterwards
* Florian's changes are failing on Solaris (need to be looked into). Dago found why buildbot is not building solaris builds, looking to resolve
** Will do a follow up PR on resolving solaris builds
* With this merge, we could move onto take away currently disabled plugins
** Fair amount of API exposed to plugins has been changed
** Bunch of plugins use restricted subset of API and can make use of upgraded mechanism ( convert value list to metrics family) but bunch of plugins use counters to look up metadata, etc. which end up being disabled
** Out 174, 84 compile good, 74 fail compilation and few with CI issues
** Added exclusions in travis/cirrus yaml files, with plugins not compiled are not tested
** Workflow basically is remove plugins from this test and try resolving them separately 
** The translation layer between valueList and metricsFamily has negligable performance impact (depends on magnitude of optimization, for example with mallocs) 
** Version 6 is not bound by backward compatibility, so this is one chance to have good naming schema for the metrics without being bound by data structure conversion
** Ideally we would pick naming schema for popular plugins in par with other collectors
* Conversion:
** Not all plugins needs to be ported but recommended to
** Need to check what makes sense to port
* Florian will Call to Action and write doc on how best to port (plugin upgrade guide)
** Design doc is yet to be written
* Collectd 6.0 project in github -> we get a kanban board/project plan 
** Create tasks for all plugins that fail and see if they are being updated
** https://github.com/orgs/collectd/projects/1

== Intern updates ==
* Implementing distribution metrics data structures, each intern implemented different approach 
** Non-cumulative approach  - uses binary search for individual updates to be searched
** Distribution metric with cumulative buckets, prometheus also uses this approach. Time complexity function is also in there as we want to use cumulative counters. Can use binary search approach to find proper percentile
** Distribution metrics using segment tree -> links in segment tree are buckets themselves, meaning children are merged with every node. Disadvantage is it needs twice the memory than array approach
* Created a benchmark to compare them -> picked up signatory approach. More details will be at final presentation
* Integrating this DS into collectd project. Documented new behaviors in metrics header file
* Next step is to provide support to cache file and structure. Can see each metric is stored in the cache, so added a new type to cache entry, verifying the functionality works
* Working on stackdriver format (json) to send to GCP. Also updated to provide histograms to text formats from json objects
* Updated to calculate standard deviation. Need to implement unit tests
* In config file, user can specify type of distribution and formatting for it
* Example is provided using write_http plugin to provide distribution
* Changes has been implemented already, need to test and contribute to main

== Terminology updates: ==
* Collectd is open to move away from using sensitive terminology such as blacklist/whitelist and master/slave.
* Following files need to be updated:
**	https://github.com/collectd/collectd/search?q=master+slave&unscoped_q=master+slave
**	https://github.com/collectd/collectd/search?q=whitelist&unscoped_q=whitelist
**	https://github.com/collectd/collectd/search?q=blacklist&unscoped_q=blacklist
* Some SQL commands are used that couldnt be changed.
* Few changes that mention slave details, etc. could easily be changed
* For blacklist/whitelist change to allow/deny list
* It would be easy for us to push the changes with these updates