Plugin:Intel PMU/tests

From collectd Wiki
< Plugin:Intel PMU
Revision as of 22:51, 13 March 2020 by Efoley (talk | contribs) (Add testplan for Intel PMU plugin)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

This page contains tests instructions for the Intel_PMU plugin. This page is meant to aid in testing at release time and be a guide for developing automated tests. Please add any tests that you have run when testing this plugin.

OPNFV Barometer

These tests are specified by the high level design guide in Barometer. They are run manually.

Summary collectd.conf Steps Results
1 Verify intel_pmu compilation is skipped and collectd is successfully compiled and installed when jevents is installed without cFPIC flag n/a
  • Build jevents without specifying cflag option.
   cd jevents
   make
   make install
  • Build collectd without --with-libjevents argument.
 ./build.sh
 ./configure --enable-syslog --enable-logfile --prefix=/root/orest/install --enable-debug
 make
 make install
  • Make sure that collectd installation is completed and intel_pmu compilation skipped.
 Collectd compilation fails and intel_pmu plugin compilation is not skipped.
2 Verify intel_pmu compilation is not skipped and collectd is successfully compiled and installed when jevents is installed with cFPIC flag - n/a
  • Build jevents with cflag option.
   cd jevents 
   make CFLAGS+='-fPIC'
   make install CFLAGS+='-fPIC'
  • Build collectd without --with-libjevents argument.
   ./build.sh
   ./configure --enable-syslog --enable-logfile --prefix=/root/orest/install --enable-debug
   make
   make install
  • Make sure that collectd installation is completed and intel_pmu is compiled.
 Collectd installation is completed and intel_pmu is compiled successfully.
3 Verify only HWCacheEvents is collected when related field is set to True. 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Make sure that events related only to HWCacheEvents are collected.
 HWCacheEvents are collected.
4 Verify only KernelPMUEvents is collected when related field is set to True. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents false
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Make sure that events related only to KernelPMUEvents are collected.
 KernelPMUEvents are collectd.
5 Verify only SWEvents is collected when related field is set to True. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Make sure that events related only to SWEvents are collected.
 SWEvents are collected.
6 Verify only HWSpecificEvents is collected when related field is set to specific values. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT"
</Plugin>
  • Make sure that events are downloaded using pmu-tool event_download.py
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Make sure that events related only to HWSpecificEvents are collected.
7 Verify that all metrics are collected when all fields have enabled flags. 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents true
 SWEvents true
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"
</Plugin>
  • Make sure that events are downloaded using pmu-tool event_download.py
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
 Make sure that events related to HWCacheEvents , KernelPMUEvents, SWEvents, HWSpecificEvents are collected.
8 Verify number of counters for HWCacheEvents matches number of events defined in OPNFV metrics page. 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Verify number of counters for HWCacheEvents matches number of events defined in OPNFV metrics page.
 Number of counters for HWCacheEvents matches number of events defined in OPNFV metrics page.
9 Verify number of counters for KernelPMUEvents matches number of events defined in OPNFV metrics page. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Verify number of counters for KernelPMUEvents matches number of events defined in OPNFV metrics page.
 Number of counters for KernelPMUEvents matches number of events defined in OPNFV metrics page.
10 Verify number of counters for SWEvents matches number of events defined in OPNFV metrics page. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 2 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Verify number of counters for SWEvents matches number of events defined in OPNFV metrics page.
 Number of counters for SWEvents matches number of events defined in OPNFV metrics page. PASS
11 Verify that HWCacheEvents counters are incremented. 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 10 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Verify counters related to HWCacheEvents are incremented.
12 Verify that KernelPMUEvents counters are incremented. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 10 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Verify counters related to KernelPMUEvents are incremented.
13 Verify that SWEvents counters are incremented. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 10 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Verify counters related to SWEvents are incremented.
14 Verify that HWSpecificEvents counters are incremented. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"
</Plugin>
  • Run collectd with write plugin enabled as well.
  • Wait 10 read intervals.
  • Stop collectd
  • Get intel_pmu statistics
  • Verify counters related to HWSpecificEvents are incremented.
15 Verify values reported for HWCacheEvents are in range of actual values reported by perf tool. 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  • Run collectd and perf tool for HWCacheEvents simultaneously.(interval is set to 1 second)
  • Wait 20 seconds.
  • Stop collectd.
  • Get last 5 values of HWCacheEvents and perf data.
  • Verify perf values are in range of collectd values.
16 Verify values reported for KernelPMUEvents are in range of actual values reported by perf tool. 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  • Run collectd and perf tool for KernelPMUEvents simultaneously.(interval is set to 1 second)
perf stat -A -C 1,5,20 -e <events> sleep 20
  • Wait 20 seconds.
  • Stop collectd.
  • Get last 5 values of KernelPMUEvents and perf data.
  • Verify perf values are in range of collectd values.
17 Verify values reported for SWEvents are in range of actual values reported by perf tool 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  • Run collectd and perf tool for SWEvents simultaneously.(interval is set to 1 second)
 perf stat -A -C 1,5,20 -e <events> sleep 20
  • Wait 20 seconds.
  • Stop collectd.
  • Get last 5 values of SWEvents and perf data.
  • Verify perf values are in range of collectd values.
18 Verify values reported for HWSpecificEvents are in range of actual values reported by perf tool 
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"
</Plugin>
  • Run collectd and perf tool for HWSpecificEvents simultaneously.(interval is set to 1 second)
 perf stat -A -C 1,5,20 -e <events> sleep 20
  • Wait 20 seconds.
  • Stop collectd.
  • Get last 5 values of HWSpecificEvents and perf data.
  • Verify perf values are in range of collectd values.
19 Verify intel_pmu counters are reported per each core and per all cores 
<Plugin intel_pmu>
 HWCacheEvents {}
 KernelPMUEvents {}
 SWEvents {}
 HWSpecificEvents {}
</Plugin>
  • Set HWCacheEvents to true in collectd.conf
  • Wait 10 seconds.
  • Stop collectd.
  • Verify that values are reported per core and for all cores.
  • Repeat 1-4 steps for KernelPMUEvents, SWEvents, HWSpecificEvents.
20 Verify intel_pmu counters are not reported for unsupported events per core and per all cores. 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  • Set HWCacheEvents to true in collectd.conf
  • Make sure that perf list doesn't have some event that is in the list of collectd events.
  • Wait 10 seconds.
  • Stop collectd.
  • Verify that unsupported events are not collected.
 Unsupported events are collected per all cores.
21 Config-less testing - invalid HWCacheEvents field 
<Plugin intel_pmu>
 HWCacheEvents incorrect
 KernelPMUEvents true
 SWEvents false
 HWSpecificEvents ""
</Plugin>

Values to test = ['None', '', '1', 'incorrect']

  • Set HWCacheEvents to incorrect value in collectd.conf
  • Start collectd
  • Verify collectd is not started.
22 Config-less testing - invalid KernelPMUEventsfield 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents invalid_value
 SWEvents false
 HWSpecificEvents ""
</Plugin>

Values to test = ['None', '', '1', 'incorrect']

  • Set KernelPMUEvents to incorrect value in collectd.conf
  • Start collectd
  • Verify collectd is not started.
23 Config-less testing - invalid SWEventsfield 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents invalid_value
 HWSpecificEvents ""
</Plugin>

Values to test = ['None', '', '1', 'incorrect']

  • Set SWEvents to incorrect value in collectd.conf
  • Start collectd
  • Verify collectd is not started.
24 Config-less testing - invalid HWSpecificEventsfield 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents 
</Plugin>

Values to test = ['']

  • Set HWSpecificEvents to incorrect value in collectd.conf
  • Start collectd
  • Verify collectd is not started.
25 Config-less testing - invalid field name 
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
 IncorrectField true
</Plugin>

Values to test = ['None', '', '1', 'true']

  • Set invalid field name in collectd.conf
  • Start collectd
  • Verify collectd is not started.


See also

Retrieved from "https://collectd.org/wiki/index.php?title=Plugin:Intel_PMU/tests&oldid=4923"