JMeter + Ruby =

Do performance test awesomely

What are we talking about?

JMeter

Performance tool

JMeter

  • Analyze and measure performance
  • Well tested
  • Long standing (10+)
  • Broad community
  • Support for everything
  • Apache project

Typical JMeter

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.7" jmeter="2.12 r1636949">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="TestPlan" enabled="true">
      <stringProp name="TestPlan.comments"/>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="TestPlan" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"/>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="ThreadGroup" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="ThreadGroup" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <intProp name="LoopController.loops">-1</intProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">100</stringProp>
        <stringProp name="ThreadGroup.ramp_time">50</stringProp>
        <longProp name="ThreadGroup.start_time">1416747107000</longProp>
        <longProp name="ThreadGroup.end_time">1416747107000</longProp>
        <boolProp name="ThreadGroup.scheduler">true</boolProp>
        <stringProp name="ThreadGroup.duration">120</stringProp>
        <stringProp name="ThreadGroup.delay"/>
        <boolProp name="ThreadGroup.delayedStart">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="HttpRequestDefaults" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="HttpRequestDefaults" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">typeform.com</stringProp>
          <stringProp name="HTTPSampler.port"/>
          <stringProp name="HTTPSampler.connect_timeout"/>
          <stringProp name="HTTPSampler.response_timeout"/>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.contentEncoding"/>
          <stringProp name="HTTPSampler.path">/</stringProp>
          <stringProp name="HTTPSampler.implementation">HttpClient4</stringProp>
          <boolProp name="HTTPSampler.image_parser">true</boolProp>
          <boolProp name="HTTPSampler.concurrentDwn">true</boolProp>
          <stringProp name="HTTPSampler.concurrentPool">6</stringProp>
          <stringProp name="HTTPSampler.embedded_url_re"/>
        </ConfigTestElement>
        <hashTree/>
        <CookieManager guiclass="CookiePanel" testclass="CookieManager" testname="HttpCookieManager" enabled="true">
          <collectionProp name="CookieManager.cookies"/>
          <boolProp name="CookieManager.clearEachIteration">false</boolProp>
          <stringProp name="CookieManager.policy">default</stringProp>
          <stringProp name="CookieManager.implementation">org.apache.jmeter.protocol.http.control.HC4CookieHandler</stringProp>
        </CookieManager>
        <hashTree/>
        <CacheManager guiclass="CacheManagerGui" testclass="CacheManager" testname="HttpCacheManager" enabled="true">
          <boolProp name="clearEachIteration">true</boolProp>
          <boolProp name="useExpires">false</boolProp>
        </CacheManager>
        <hashTree/>
        <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="User-Agent" enabled="true">
          <collectionProp name="HeaderManager.headers">
            <elementProp name="" elementType="Header">
              <stringProp name="Header.name">User-Agent</stringProp>
              <stringProp name="Header.value">Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.46 Safari/536.5</stringProp>
            </elementProp>
          </collectionProp>
        </HeaderManager>
        <hashTree/>
        <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="Accept-Encoding" enabled="true">
          <collectionProp name="HeaderManager.headers">
            <elementProp name="" elementType="Header">
              <stringProp name="Header.name">Accept-Encoding</stringProp>
              <stringProp name="Header.value">gzip, deflate</stringProp>
            </elementProp>
          </collectionProp>
        </HeaderManager>
        <hashTree/>
        <TestAction guiclass="TestActionGui" testclass="TestAction" testname="Think Time" enabled="true">
          <intProp name="ActionProcessor.action">1</intProp>
          <intProp name="ActionProcessor.target">0</intProp>
          <stringProp name="ActionProcessor.duration">0</stringProp>
        </TestAction>
        <hashTree>
          <GaussianRandomTimer guiclass="GaussianRandomTimerGui" testclass="GaussianRandomTimer" testname="GaussianRandomTimer" enabled="true">
            <stringProp name="ConstantTimer.delay">1000</stringProp>
            <stringProp name="RandomTimer.range">0</stringProp>
          </GaussianRandomTimer>
          <hashTree/>
        </hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Visit Typeform" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="Visit Typeform" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">admin.typeform.com</stringProp>
          <stringProp name="HTTPSampler.port">443</stringProp>
          <stringProp name="HTTPSampler.connect_timeout"/>
          <stringProp name="HTTPSampler.response_timeout"/>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.contentEncoding"/>
          <stringProp name="HTTPSampler.path">/to/mUKRFF</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
          <stringProp name="HTTPSampler.implementation"/>
          <boolProp name="HTTPSampler.monitor">false</boolProp>
          <stringProp name="HTTPSampler.embedded_url_re"/>
          <stringProp name="TestPlan.comments"/>
        </HTTPSamplerProxy>
        <hashTree/>
        <TestAction guiclass="TestActionGui" testclass="TestAction" testname="Think Time" enabled="true">
          <intProp name="ActionProcessor.action">1</intProp>
          <intProp name="ActionProcessor.target">0</intProp>
          <stringProp name="ActionProcessor.duration">0</stringProp>
        </TestAction>
        <hashTree>
          <GaussianRandomTimer guiclass="GaussianRandomTimerGui" testclass="GaussianRandomTimer" testname="GaussianRandomTimer" enabled="true">
            <stringProp name="ConstantTimer.delay">30000</stringProp>
            <stringProp name="RandomTimer.range">5000</stringProp>
          </GaussianRandomTimer>
          <hashTree/>
        </hashTree>
      </hashTree>
      <kg.apc.jmeter.vizualizers.CorrectedResultCollector guiclass="kg.apc.jmeter.vizualizers.ResponseTimesOverTimeGui" testclass="kg.apc.jmeter.vizualizers.CorrectedResultCollector" testname="Response Times Over Time" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>true</message>
            <threadName>true</threadName>
            <dataType>true</dataType>
            <encoding>false</encoding>
            <assertions>true</assertions>
            <subresults>true</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>true</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
          </value>
        </objProp>
        <stringProp name="filename"/>
        <longProp name="interval_grouping">500</longProp>
        <boolProp name="graph_aggregated">false</boolProp>
        <stringProp name="include_sample_labels"/>
        <stringProp name="exclude_sample_labels"/>
      </kg.apc.jmeter.vizualizers.CorrectedResultCollector>
      <hashTree/>
      <kg.apc.jmeter.vizualizers.CorrectedResultCollector guiclass="kg.apc.jmeter.vizualizers.ThreadsStateOverTimeGui" testclass="kg.apc.jmeter.vizualizers.CorrectedResultCollector" testname="Active Threads Over Time" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>false</message>
            <threadName>true</threadName>
            <dataType>false</dataType>
            <encoding>false</encoding>
            <assertions>false</assertions>
            <subresults>false</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>false</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
            <threadCounts>true</threadCounts>
            <sampleCount>true</sampleCount>
          </value>
        </objProp>
        <stringProp name="filename"/>
        <longProp name="interval_grouping">500</longProp>
        <boolProp name="graph_aggregated">false</boolProp>
        <stringProp name="include_sample_labels"/>
        <stringProp name="exclude_sample_labels"/>
        <stringProp name="start_offset"/>
        <stringProp name="end_offset"/>
        <boolProp name="include_checkbox_state">false</boolProp>
        <boolProp name="exclude_checkbox_state">false</boolProp>
      </kg.apc.jmeter.vizualizers.CorrectedResultCollector>
      <hashTree/>
      <kg.apc.jmeter.vizualizers.CompositeResultCollector guiclass="kg.apc.jmeter.vizualizers.CompositeGraphGui" testclass="kg.apc.jmeter.vizualizers.CompositeResultCollector" testname="Composite Graph" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>false</message>
            <threadName>true</threadName>
            <dataType>false</dataType>
            <encoding>false</encoding>
            <assertions>false</assertions>
            <subresults>false</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>false</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
            <threadCounts>true</threadCounts>
            <sampleCount>true</sampleCount>
          </value>
        </objProp>
        <stringProp name="filename"/>
        <longProp name="interval_grouping">500</longProp>
        <boolProp name="graph_aggregated">false</boolProp>
        <stringProp name="include_sample_labels"/>
        <stringProp name="exclude_sample_labels"/>
        <stringProp name="start_offset"/>
        <stringProp name="end_offset"/>
        <boolProp name="include_checkbox_state">false</boolProp>
        <boolProp name="exclude_checkbox_state">false</boolProp>
        <collectionProp name="COMPOSITE_CFG">
          <collectionProp name="">
            <stringProp name="">Response Times Over Time</stringProp>
            <stringProp name="">Active Threads Over Time</stringProp>
          </collectionProp>
          <collectionProp name="">
            <stringProp name="">Visit Typeform</stringProp>
            <stringProp name="">Overall Active Threads</stringProp>
          </collectionProp>
        </collectionProp>
      </kg.apc.jmeter.vizualizers.CompositeResultCollector>
      <hashTree/>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

Ruby

Object Goddess

Ruby

  • It Just Works™
  • Everything is an object
  • Support for everything
  • Huge community
  • Even older than JMeter (20+)

Typical Ruby

require 'ruby-jmeter'
require 'require_all'
require_all 'lib/users'

test do
  threads count: 100, ramup: 60, duration: 120 do
    user = Users::Respondent.new(self)
    user.pause 1000,0
    user.visit_typeform('mUKRFF')
  end

  threads count: 1 do
    visit name: 'Get API results', url: 'https://api.typeform.com/v0/form/mUKRFF?key=542227ce4c86525e1faea153ab19d96f2bd1d576'
  end

  response_times_over_time 'Response Times Over Time'
  active_threads_over_time 'Active Threads Over Time'

  composite_graph('Composite Graph', [
       {
           graph: 'Response Times Over Time',
           metric: 'Visit Typeform'
       },
       {
           graph: 'Active Threads Over Time',
           metric: 'Overall Active Threads'
       }
   ])
end.run(gui: true)

Why not use only one of them?

JMeter

  • Editing JMX (JMeter configuration) sucks
  • Diffing JMX sucks
  • Clicking around in a GUI sucks

Ruby

  • Not created for performance testing
  • Doesn't create as pretty graphs as JMeter
  • Doesn't support Jenkins et al out of the box

Introducing ruby-jmeter

Ruby DSL for JMeter

require 'ruby-jmeter'
require 'require_all'
require_all 'lib/users'

test do
  threads count: 100, ramup: 60, duration: 120 do
    user = Users::Respondent.new(self)
    user.pause 1000,0
    user.visit_typeform('mUKRFF')
  end

  threads count: 1 do
    visit name: 'Get API results', url: 'https://api.typeform.com/v0/form/mUKRFF?key=542227ce4c86525e1faea153ab19d96f2bd1d576'
  end

  response_times_over_time 'Response Times Over Time'
  active_threads_over_time 'Active Threads Over Time'

  composite_graph('Composite Graph', [
       {
           graph: 'Response Times Over Time',
           metric: 'Visit Typeform'
       },
       {
           graph: 'Active Threads Over Time',
           metric: 'Overall Active Threads'
       }
   ])
end.run(gui: true)
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.7" jmeter="2.12 r1636949">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="TestPlan" enabled="true">
      <stringProp name="TestPlan.comments"/>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="TestPlan" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"/>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="ThreadGroup" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="ThreadGroup" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <intProp name="LoopController.loops">-1</intProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">100</stringProp>
        <stringProp name="ThreadGroup.ramp_time">50</stringProp>
        <longProp name="ThreadGroup.start_time">1416747107000</longProp>
        <longProp name="ThreadGroup.end_time">1416747107000</longProp>
        <boolProp name="ThreadGroup.scheduler">true</boolProp>
        <stringProp name="ThreadGroup.duration">120</stringProp>
        <stringProp name="ThreadGroup.delay"/>
        <boolProp name="ThreadGroup.delayedStart">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="HttpRequestDefaults" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="HttpRequestDefaults" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">typeform.com</stringProp>
          <stringProp name="HTTPSampler.port"/>
          <stringProp name="HTTPSampler.connect_timeout"/>
          <stringProp name="HTTPSampler.response_timeout"/>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.contentEncoding"/>
          <stringProp name="HTTPSampler.path">/</stringProp>
          <stringProp name="HTTPSampler.implementation">HttpClient4</stringProp>
          <boolProp name="HTTPSampler.image_parser">true</boolProp>
          <boolProp name="HTTPSampler.concurrentDwn">true</boolProp>
          <stringProp name="HTTPSampler.concurrentPool">6</stringProp>
          <stringProp name="HTTPSampler.embedded_url_re"/>
        </ConfigTestElement>
        <hashTree/>
        <CookieManager guiclass="CookiePanel" testclass="CookieManager" testname="HttpCookieManager" enabled="true">
          <collectionProp name="CookieManager.cookies"/>
          <boolProp name="CookieManager.clearEachIteration">false</boolProp>
          <stringProp name="CookieManager.policy">default</stringProp>
          <stringProp name="CookieManager.implementation">org.apache.jmeter.protocol.http.control.HC4CookieHandler</stringProp>
        </CookieManager>
        <hashTree/>
        <CacheManager guiclass="CacheManagerGui" testclass="CacheManager" testname="HttpCacheManager" enabled="true">
          <boolProp name="clearEachIteration">true</boolProp>
          <boolProp name="useExpires">false</boolProp>
        </CacheManager>
        <hashTree/>
        <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="User-Agent" enabled="true">
          <collectionProp name="HeaderManager.headers">
            <elementProp name="" elementType="Header">
              <stringProp name="Header.name">User-Agent</stringProp>
              <stringProp name="Header.value">Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.46 Safari/536.5</stringProp>
            </elementProp>
          </collectionProp>
        </HeaderManager>
        <hashTree/>
        <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="Accept-Encoding" enabled="true">
          <collectionProp name="HeaderManager.headers">
            <elementProp name="" elementType="Header">
              <stringProp name="Header.name">Accept-Encoding</stringProp>
              <stringProp name="Header.value">gzip, deflate</stringProp>
            </elementProp>
          </collectionProp>
        </HeaderManager>
        <hashTree/>
        <TestAction guiclass="TestActionGui" testclass="TestAction" testname="Think Time" enabled="true">
          <intProp name="ActionProcessor.action">1</intProp>
          <intProp name="ActionProcessor.target">0</intProp>
          <stringProp name="ActionProcessor.duration">0</stringProp>
        </TestAction>
        <hashTree>
          <GaussianRandomTimer guiclass="GaussianRandomTimerGui" testclass="GaussianRandomTimer" testname="GaussianRandomTimer" enabled="true">
            <stringProp name="ConstantTimer.delay">1000</stringProp>
            <stringProp name="RandomTimer.range">0</stringProp>
          </GaussianRandomTimer>
          <hashTree/>
        </hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Visit Typeform" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="Visit Typeform" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">admin.typeform.com</stringProp>
          <stringProp name="HTTPSampler.port">443</stringProp>
          <stringProp name="HTTPSampler.connect_timeout"/>
          <stringProp name="HTTPSampler.response_timeout"/>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.contentEncoding"/>
          <stringProp name="HTTPSampler.path">/to/mUKRFF</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
          <stringProp name="HTTPSampler.implementation"/>
          <boolProp name="HTTPSampler.monitor">false</boolProp>
          <stringProp name="HTTPSampler.embedded_url_re"/>
          <stringProp name="TestPlan.comments"/>
        </HTTPSamplerProxy>
        <hashTree/>
        <TestAction guiclass="TestActionGui" testclass="TestAction" testname="Think Time" enabled="true">
          <intProp name="ActionProcessor.action">1</intProp>
          <intProp name="ActionProcessor.target">0</intProp>
          <stringProp name="ActionProcessor.duration">0</stringProp>
        </TestAction>
        <hashTree>
          <GaussianRandomTimer guiclass="GaussianRandomTimerGui" testclass="GaussianRandomTimer" testname="GaussianRandomTimer" enabled="true">
            <stringProp name="ConstantTimer.delay">30000</stringProp>
            <stringProp name="RandomTimer.range">5000</stringProp>
          </GaussianRandomTimer>
          <hashTree/>
        </hashTree>
      </hashTree>
      <kg.apc.jmeter.vizualizers.CorrectedResultCollector guiclass="kg.apc.jmeter.vizualizers.ResponseTimesOverTimeGui" testclass="kg.apc.jmeter.vizualizers.CorrectedResultCollector" testname="Response Times Over Time" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>true</message>
            <threadName>true</threadName>
            <dataType>true</dataType>
            <encoding>false</encoding>
            <assertions>true</assertions>
            <subresults>true</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>true</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
          </value>
        </objProp>
        <stringProp name="filename"/>
        <longProp name="interval_grouping">500</longProp>
        <boolProp name="graph_aggregated">false</boolProp>
        <stringProp name="include_sample_labels"/>
        <stringProp name="exclude_sample_labels"/>
      </kg.apc.jmeter.vizualizers.CorrectedResultCollector>
      <hashTree/>
      <kg.apc.jmeter.vizualizers.CorrectedResultCollector guiclass="kg.apc.jmeter.vizualizers.ThreadsStateOverTimeGui" testclass="kg.apc.jmeter.vizualizers.CorrectedResultCollector" testname="Active Threads Over Time" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>false</message>
            <threadName>true</threadName>
            <dataType>false</dataType>
            <encoding>false</encoding>
            <assertions>false</assertions>
            <subresults>false</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>false</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
            <threadCounts>true</threadCounts>
            <sampleCount>true</sampleCount>
          </value>
        </objProp>
        <stringProp name="filename"/>
        <longProp name="interval_grouping">500</longProp>
        <boolProp name="graph_aggregated">false</boolProp>
        <stringProp name="include_sample_labels"/>
        <stringProp name="exclude_sample_labels"/>
        <stringProp name="start_offset"/>
        <stringProp name="end_offset"/>
        <boolProp name="include_checkbox_state">false</boolProp>
        <boolProp name="exclude_checkbox_state">false</boolProp>
      </kg.apc.jmeter.vizualizers.CorrectedResultCollector>
      <hashTree/>
      <kg.apc.jmeter.vizualizers.CompositeResultCollector guiclass="kg.apc.jmeter.vizualizers.CompositeGraphGui" testclass="kg.apc.jmeter.vizualizers.CompositeResultCollector" testname="Composite Graph" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>false</message>
            <threadName>true</threadName>
            <dataType>false</dataType>
            <encoding>false</encoding>
            <assertions>false</assertions>
            <subresults>false</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>false</xml>
            <fieldNames>false</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
            <threadCounts>true</threadCounts>
            <sampleCount>true</sampleCount>
          </value>
        </objProp>
        <stringProp name="filename"/>
        <longProp name="interval_grouping">500</longProp>
        <boolProp name="graph_aggregated">false</boolProp>
        <stringProp name="include_sample_labels"/>
        <stringProp name="exclude_sample_labels"/>
        <stringProp name="start_offset"/>
        <stringProp name="end_offset"/>
        <boolProp name="include_checkbox_state">false</boolProp>
        <boolProp name="exclude_checkbox_state">false</boolProp>
        <collectionProp name="COMPOSITE_CFG">
          <collectionProp name="">
            <stringProp name="">Response Times Over Time</stringProp>
            <stringProp name="">Active Threads Over Time</stringProp>
          </collectionProp>
          <collectionProp name="">
            <stringProp name="">Visit Typeform</stringProp>
            <stringProp name="">Overall Active Threads</stringProp>
          </collectionProp>
        </collectionProp>
      </kg.apc.jmeter.vizualizers.CompositeResultCollector>
      <hashTree/>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

ruby-jmeter still a young project

  • Not everything is supported out of the box
  • Easy to extend
  • Maintainer very open to PR and acting fast

Extending ruby-jmeter

Adding support for ActiveThreads in JMeter

module RubyJmeter
  module Plugins
    class ActiveThreadsOverTime
      attr_accessor :doc
      include Helper
      def initialize(name, params={})
        @doc = Nokogiri::XML(<<-XML.strip_heredoc)
          <kg.apc.jmeter.vizualizers.CorrectedResultCollector guiclass="kg.apc.jmeter.vizualizers.ThreadsStateOverTimeGui" testclass="kg.apc.jmeter.vizualizers.CorrectedResultCollector" testname="#{name}" enabled="true">
            <boolProp name="ResultCollector.error_logging">false</boolProp>
            <objProp>
              <name>saveConfig</name>
              <value class="SampleSaveConfiguration">
                <time>true</time>
                <latency>true</latency>
                <timestamp>true</timestamp>
                <success>true</success>
                <label>true</label>
                <code>true</code>
                <message>false</message>
                <threadName>true</threadName>
                <dataType>false</dataType>
                <encoding>false</encoding>
                <assertions>false</assertions>
                <subresults>false</subresults>
                <responseData>false</responseData>
                <samplerData>false</samplerData>
                <xml>false</xml>
                <fieldNames>false</fieldNames>
                <responseHeaders>false</responseHeaders>
                <requestHeaders>false</requestHeaders>
                <responseDataOnError>false</responseDataOnError>
                <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
                <assertionsResultsToSave>0</assertionsResultsToSave>
                <bytes>true</bytes>
                <threadCounts>true</threadCounts>
                <sampleCount>true</sampleCount>
              </value>
            </objProp>
            <stringProp name="filename"></stringProp>
            <longProp name="interval_grouping">500</longProp>
            <boolProp name="graph_aggregated">false</boolProp>
            <stringProp name="include_sample_labels"></stringProp>
            <stringProp name="exclude_sample_labels"></stringProp>
            <stringProp name="start_offset"></stringProp>
            <stringProp name="end_offset"></stringProp>
            <boolProp name="include_checkbox_state">false</boolProp>
            <boolProp name="exclude_checkbox_state">false</boolProp>
          </kg.apc.jmeter.vizualizers.CorrectedResultCollector>
        XML
        update params
      end
    end
  end
end



def active_threads_over_time(params={}, &block)
    node = RubyJmeter::Plugins::ActiveThreadsOverTime.new(params)
    attach_node(node, &block)
end

alias_method :active_threads, :active_threads_over_time

Adding DSL method

Usage

active_threads_over_time 'Active Threads Over Time'

Writing test file

What we want to test?

Let's test form landings

  • Respondents lands on form during two minutes (spike test)
  • User loads results from API once every second
  • We want a graph with three values
    • Respondent load time for form landing with resources, mimick browser behaviour
    • User load time for requests to API
    • The number of threads (users) that we have

Project (TypeMeter) structure

├── Gemfile
├── Gemfile.lock
├── lib
│   └── users
│       ├── base_user.rb
│       ├── respondent.rb
│       └── user.rb
├── readme.md
└── tests
    ├── file_download_healthcheck.rb
    ├── test.rb
    ├── visit_typeform.rb
    └── workers.rb

3 directories, 10 files

BaseUser

module Users
  class BaseUser
    def initialize(dsl)
      @dsl = dsl
      defaults domain: 'typeform.com', # default domain if not specified in URL
               protocol: 'https', # default protocol if not specified in URL
               download_resources: true, # download images/CSS/JS
               use_concurrent_pool: 6 # mimic Chrome/IE parallelism for images/CSS/JS

      cookies # om nom nom
      cache clear_each_iteration: true # each thread iteration mimics a new user with an empty browser cache
      with_user_agent 'chrome_osx' # send Chrome OSX browser headers
      with_gzip # add HTTP request headers to allow GZIP compression. Most sites support this nowadays.
    end
    
    def pause(delay=30000, deviation=5000)
      test_action name: 'Think Time', action: 1, target: 0, duration: 0 do
        think_time delay, deviation
      end
    end

    private
    # Passes method calls through to the underlying DSL.
    def method_missing method, *args, &block
      @dsl.__send__ method, *args, &block
    end
  end
end

Respondent

module Users
  class Respondent < BaseUser
    def visit_typeform(name, uid)
      visit name: name, url: "https://admin.typeform.com/to/#{uid}"
    end
  end
end

User (Admin user)

module Users
  class User < BaseUser
    def get_api_results(params = {})
      visit name: params[:name], url: "https://api.typeform.com/v0/form/#{params[:uid]}?key=#{params[:api_key]}"
    end
  end
end

Writing the

actual test

Full test-file

Require-statements

Test itself

Respondents visit

User using API

Loops, one call per second

Samplers, what data we want

Composite Graph = Many graphs in one graph

Response time for "Visit Typeform"

Response time for API results

Active Threads (Active users)

DONE

Using JMeter to run the test

Running test

JMeter GUI

Test results

Questions?

Made with Slides.com