What's Wrong with Existing Cucumber?

Because we realized what we loved was...

Huge, messy codebase

MADNESS in steps

Always global all the code

Hard to maintain and reuse

Coupled scenarios/step definitions

Complex code in step definitions

Step description not corresponding to actions

Hard to read and understand features for business

Just to be clear,

take a look on this...

When(/^I (?:create|add)\s?\w* payment\s((?:with|to) conversion)?\s?using options:$/) do |with_conversion, options|

  @options_for_pay_create = options.hashes.first.symbolize_keys
  params_for_removal = @options_for_pay_create.reject{|k,v| v != 'remove'}.keys
  @options_for_pay_create[:conversion_id] = @conversion_id if @conversion_id

  beneficiary_options = {}
   if @options_for_pay_create.has_key?(:on_behalf_of)
    if @options_for_pay_create[:on_behalf_of] =~ (/\A[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]*\Z/)
      @on_behalf_of = @options_for_pay_create[:on_behalf_of]
    else
      client_type = @options_for_pay_create[:on_behalf_of]
      @on_behalf_of = (TCC_ENV[:users][client_type]).contact_id
    end
    @options_for_pay_create[:on_behalf_of] = @on_behalf_of
    beneficiary_options[:on_behalf_of] = @on_behalf_of unless @beneficiary_id
   end
  unless @beneficiary_id
    beneficiary_options.merge!({:currency => @options_for_pay_create[:currency], :bank_country => 'GB'})
    @beneficiary_id = V2ApiServiceHelper.create_beneficiary(OptionsHelper.options_for_beneficiary_api_v2(beneficiary_options))['id']
  end
  @options_for_pay_create.merge!(:beneficiary_id => @beneficiary_id) if @beneficiary_id
  @options_for_pay_create = OptionsHelper.options_for_payment(@options_for_pay_create)
  if with_conversion
    @conversion_id = @options_for_pay_create.fetch(:conversion_id, V2ApiServiceHelper.create_conversion(OptionsHelper.options_for_conversion)['id'])
    @options_for_pay_create.merge!({:conversion_id => @conversion_id})
  end
  @options_for_pay_create.each do |k, v|
    @options_for_pay_create.delete(k) if  v == 'remove'
    @options_for_pay_create[k] = SecureRandom.random_number(999999).to_s if v == 'random'
    @options_for_pay_create[k] = SecureRandom.hex(10).to_s if v == 'random_string'
  end
  params = OptionsHelper.options_for_payer(@options_for_pay_create).merge(@options_for_pay_create)
  if params['payer_entity_type']
    params.delete('payer_type')
  else
    params['payer_entity_type'] = params.delete('payer_type') if params['payer_type']
  end

  params.delete_if{|k,_| params_for_removal.include?(k.to_sym)}
  params["payment_date"] = (Date.today).strftime('%Y-%m-%d') if params["payment_date"] == 'today'
  conversion_date = @conversion ? Date.parse(@conversion["conversion_date"]) : nil
  params["payment_date"] = (StaticDataServiceHelper.get_next_date_of('Saturday', conversion_date)).strftime('%Y-%m-%d') if params["payment_date"] == 'Saturday'
  @v2_response = V2ApiServiceHelper.create_payment(params)
  @options_for_pay_create.delete(:on_behalf_of)
  @payment = @v2_response
  @payment_id = @v2_response['id']
  @empty_timestamps = ['transferred_at']
  @payer_id = @v2_response['payer_id']
  @payments_for_submissions ||= []
  @payments_for_submissions << @payment_id
  @response = @v2_response
end

More problems in details ... 

Test author has to provide around 15!!! parameters to create beneficiary, payments etc.

Most of them are not “interesting” for particular scenario

Existing code for generation of required options is total mess and  is not maintainable at all

Hard to read big tables in scenarios

  And I create payment using options:
    | currency   | amount | reason | reference       | payment_type | payer_entity_type | payer_address       | payer_city |..
    | <currency> | 200    | Test   | paymentgroup 1  | regular      | individual        | 221b, backer street | London     |..

  ..| payer_country | payer_first_name | payer_last_name | payer_date_of_birth | payer_identification_type | payer_identification_value |
  ..| GB            | John             | Smith           | 1990-11-11          | passport                  | 123456                     |

They are:

Hard to maintain

Hard to find method you want (if one is not found duplicate methods are created)

They are not only "transport" classes including only methods with calls, but also helper methods that should not be there:

Service helpers include too many methods

Example:

PaymentsServiceHelper class contains 136 methods

They are randomly added without any grouping

Many of those methods are not used at all

def self.create_settlement_in_status_part_paid(principal_identifier)
 conversions = []
 options_for_conversion = OptionsHelper.options_for_conversion
 conversions << V2ApiServiceHelper.create_conversion(options_for_conversion)
 options_for_conversion = options_for_conversion.merge({:buy_currency => options_for_conversion[:sell_currency],
                                                        :sell_currency => options_for_conversion[:buy_currency]})
 conversions << V2ApiServiceHelper.create_conversion(options_for_conversion)
 settlement = create_settlement_in_status_funds_arrived(conversions)
 conversions.each do |conversion|
   TransactronServiceHelper.close_hedges_for_trade(conversion['short_reference'], conversion['currency_pair'], conversion['buy_currency'], conversion['client_buy_amount'])
 end
 currency_entry = TradingServiceHelper.find_settlement_entry(settlement['short_reference'], conversions.first['sell_currency'])
 TradingServiceHelper.make_entry_payment({:entry_id=>currency_entry["settlement_entry"]["uuid"], :currency => conversions.first['sell_currency'], :override_status_check => "true"}, principal_identifier)
 V2ApiServiceHelper.get_settlement(settlement['id'])
end

Usage of instance variables to maintain state

No registry objects that store all the created/retrieved entities during the test

 Most of step definitions include statements that  share and set state in instance variables
All step definitions are very dependent on each other

Find suitable existing step definition that uses proper instance variable takes too much time

Examples:

When(/^I (?:create|add)\s?\w* payment\s((?:with|to) conversion)?\s?using options:$/) do |with_conversion, options|

 :
 :
 @v2_response = V2ApiServiceHelper.create_payment(params)
 @options_for_pay_create.delete(:on_behalf_of)
 @payment = @v2_response
 @payment_id = @v2_response['id']
 @empty_timestamps = ['transferred_at']
 @payer_id = @v2_response['payer_id']
 @payments_for_submissions ||= []
 @payments_for_submissions << @payment_id
 @response = @v2_response
end

Then(/^I should receive the success response$/) do
 @response.code.should == 200
end

Then(/^I should get successful response with created payment from API v2$/) do
 @v2_response['id'].should_not be_nil
end

New reusable step definitions must be prepared

• Many Conjunctive Steps as result of using instance variables for sharing state
• Often steps are duplicated in order to use another instance variable in it
• Many step definitions have description that does not correspond to actions
• There are steps that do a lot of additional actions  
• In many “action” steps missing verification that action was successful. As result harder to debug failed tests 
• A lot of technical steps

Mess with existing step definitions

• Hard to find wanted step definitions for new tests
• Many duplicate steps that do the same actions 
• No grouping of steps by business objects, model
• Some files containing step definitions are really huge, find something there is really hard. 
• Some files contain step definitions that do not correspond to the file name.
  
  Example : payments_steps.rb” with 1280 lines of code has huge number of steps related to trades creation, response validations, beneficiaries, legal entities etc

Mess with existing feature files

• We do not have proper structure of feature files
• Hard find tests for particular feature
• Sometimes scenarios for one feature are located in different directories
• Sometimes one test covers two diffrent features
• Huge feature file with too many scenarios(e.g. ....)

References

• Document describing Cucumber problems and proposed solutions in more details - Link
• Document with cucumber probles actualized in 2016 year - Link
• JIRA ticket - TCC-7624