Performance
TESTS OVERVIEW
Important Notes
-
There are two different TAF used by Performance Team
-
Some tests still exist only in old "testrunner"
-
Some tests are already implemented in new "perfrunner"
WHY DOES NEW TAF MATTER?
-
New workload generator based on libcouchbase - more realistic and fair workload
-
Tight integration with cbmonitor - better reporting and stats visualization
-
Clean, well-structured, Pythonistic framework speeds up tests development a lot
KPI vs. EXTENDED REPORTS
-
Key metrics are defined for every performance test
- There are multiple dashboards to represent them
- We collect other internal and system metrics as well
- There is cbmonitor to deal with them
showfast.sc.couchbase.com
showfast.sc.couchbase.com/compare/2.0.1-170/2.1.0-717
COMPACTION TESTS
-
KPI: bucket compaction time
-
KPI: index compaction time
Indexing tests
-
KPI: initial indexing time
-
KPI: incremental indexing time
QUERY PERFORMANCE TESTS
-
KPI: query latency (90th percentile)
STATIC REBALANCE TESTS
-
KPI: rebalance time without front-end workload
-
Options: with and without secondary index
-
Options: HDD, DGM, IN/OUT
DYNAMIC REBALANCE TESTS
-
KPI: rebalance time with front-end workload
-
Options: with and without secondary index
-
Options: HDD, DGM, IN/OUT
-
To-do: migrate to perfrunner
INITIAL XDC REPLICATION
-
KPI: rate of initial replication/cluster synchronization
-
Options: HDD, DGM, number of buckets, bi-directional/uni-directional
GENERAL XDCR PERFORMANCE
KPI: maximum replication lag
KPI: maximum replication queue length
Options: HDD, DGM, bi-directional/uni-directional
Other 2.2 Todo
-
Test execution time optimization
-
ep-engine related tests (BgFetcher, Flusher, TAP)
-
Complete transition to perfrunner
-
Several cbmonitor and showfast improvements
