A Short IntroDUCTION to 


(the dark Side of) QA 



presented and adapted by Xavier Val

BASED ON Nishant Pandey SLIDES

Agenda



  • What QA does
  • What TESTING really means
  • Aplication Vs Products
  • V&V
  • Bugs 
  • Web Test 

What a QA does


is ...

What They want us to do:


What they Think we Do:

What we really do:



When you test a program, you want to add 
some value to it

 value through testing means raising the quality or reliability of the program.


Raising the reliability means
 Finding and removing errors.

QA VS QC 


Quality Control 
is the process which usually includes inspection and audit operations to segregate bad from good 


Quality Assurance
is  a process that focuses upon each and every role and their responsibilities in the dev process and checks whether they are being accomplished as per the guidelines

QUALITY CONTROL        QUALITY ASSURANCE

Concentrates on product                  Concentrates on process 


Done after product is  built              Done throughout life-cycle 


Defect Detection &  correction    Defect Prevention Method    

It is a line function                                     It is staff function 

 What Testing REALLY means

One of the primary causes of poor  testing is the fact that people begin with a false definition of the term. 

“Testing is the process of demonstrating that errors are not
present."

“The purpose of testing is to show that a program performs its intended functions correctly.”

“Testing is the process of establishing confidence that a program does what it is supposed to do.”


“TESTING IS THE PROCESS OF EXECUTING A PROGRAM/APP WITH THE INTENTION OF FINDING ERRORS.”  

- G J  MYERS


THE ART OF SOFTWARE TESTING. WILEY, 1979.


TESTING CAN SHOW THE PRESENCE OF BUGS BUT NEVER THEIR ABSENCE.”

EDSGER W. DIJKSTRA


Dijkstra received the 1972 ACM Turing Award for fundamental contributions in the area of programming languages 

Application Vs Products 


Application 


Developed for specific client or  user  group 

Users are Highly involved  in Dev 

Business Knowledge and Domain Expertise 

Dev is focused on fixed time and price 




PRODUCT


Developed for  specific market 


Solving some Business  Problem 


End user are not involved in Dev 

v & v


VERIFICATION 

is also known as In-process testing and qa

 Reviews & meeting to evaluate Doc , plans , code 
,requirements and specification

checks  "Whether we are  building  the product right  "
 
 



VALIDATION 

Validation typically involves actual testing and takes place after the verification are complete. It is  Quality control. 

It determines the correctness of a build with the respect to it's requirement .

checks "Whether we are building the right product"?

Two things contribute to unhappy customer , Late Delivery & 



A bug is generally referred to as a feature in the application that does not work according to customer's expectation .  (Business def.)

 BS 7925-1: Bug--generic term for fault, failure, error, human action that produces an incorrect result.


Bug is Basically a terminology for testers 

 Why :  Due to an unspecified or misunderstood requirements  or a mistake while developing the Application 

SOME DEFINITIONS

DEFECT, ERROR, BUG, FAILURE AND FAULT 

ºº

A mistake in coding is called Error  


Error found by tester is called Defect  


Defect accepted by development team then it is called bug 

 
Build does not meet the requirements then it Is Failure.” 

COST OF BUG




In 1999 when NASA engineers lost contact with the Mars Climate Orbiter as they attempted to place it in Mars's orbit for research. 













Reason : One Team was using a program that calculated distance in English Units such as 'Inches, feet & pounds  ,While the other team  used metric units. This discrepancy led to faulty measurements of the Spacecraft's trajectory  

Loss : Operation which cost more than  $655 million  

Bug Lifecycle

Bug life cycle...continued.

 In the Life cycle will result in a higher return on investment (ROI)


CATCH THEM EARLY !

The cost of fixing a bug differs depending on the stage of development it is caught in ...

  • Requirement Stage 
  • Coding
  • Integration Testing 
  • Testing Stage 
  • Production Stage 

Key takeaway

 


Finding a bug later in the life cycle of the
software / Web App costs more. 



The greatest ROI is to find bugs early.


    Web Testing  

    •  Functionality Testing

    •  Usability testing

    •  Compatibility testing

    •  Performance testing

    •  Security testing


    Functional

    Functional testing verifies that each function of the software application operates in conformance with the requirement specification. This testing mainly involves black box testing and it is not concerned about the source code of the application.

    Test all the links  Outgoing / Internal /Anchor / MailTo link

    Test  your HTML and CSS  Check for syntax Errors , Validation , Standard Compliance with  W3C. 

     Test Forms ;  working as expected , Check default values being populated , Mandatory fields Error message.  check whether Data is populated to DB . 

     cont'd..

    Database consistency , Data integrity  are the most important part of any Software / Web App  Perform  basic CRUD operation  to test . 


    Testing  With DB  beyond CRUD  comes  in Performance & Load Testing .

    Test Business Workflow.... includes Testing the end -to -end workflow/ business scenarios. Negative Scenarios  . 
     

    Ex.


     when I call the validate_country_code() function and pass it the country code 'ZZ' it should return false is a unit test .


    Functional test is  when I fill out the shipping form with a country code of ZZ, I should be redirected to a help page which allows me to pick my country code out of a menu.

    Tool 




    Usability 


    Usability testing generally involves setting a series of task for people to complete and noting any problems they encounter                                     -It's as simple as that !  


    Don't make me think! 

    Steve Krug’s first law of usability


    Usability is defined by Five Quality Component 

    Learnability
    How easy to  accomplish basic task the first time they encounter  design ?
    Efficiency  
    One the user have learned design , How quickly can they perform task  ?

    Memorability 
     When users return to the design after a period of not using it  How easily can  
    they re-establish proficiency ?

      Cont'd

      Errors 
      How many Errors do users make , How severe are these errors & How easily they recover from the errors


       Satisfaction : 
      How pleasant is it to use the design 

      Usability testing What to test ?  


      Four Focal Points  of Design 

      Navigation  | Content  | Presentation  | Interaction 

      UT are  carried out by testers like you or a small focus group similar to the target audience of the web application.

      Site Navigation Menus , buttons or Links to different pages on your site should be consistent Test the Content, no spelling or grammatical errors. 

      Usability Testing Methods 

      • Card Sorting 
        • Card sorting is a structured exercise that asks volunteer  users to sort a list of sample of content into logical groupings.
        • Card sorting comes in two flavours: 
          • a) open card sorting b) closed card sorting. 

      • Tree Testing 
        • Tree-testing is a lesser known UX method but can substantially help with improving problems in navigation. 
        • Success rate | Speed rate | Directness rate 

      Usability Tools





      CrazyEgg - Heat Map 

      Compatibility 


      Compatibility is nothing but capability of existing or living together. In normal life, Oil is not compatible with water, but milk can be easily combined with water.

       It's a type of non functional Testing  used to ensure compatibility of the system/web App 
        

      CT find out how well a system performs in a particular environment that includes  different hardware, network, operating system  .

      Tools 







      Performance 


      A performance test is a technical investigation done to determine or validate the responsiveness, speed, scalability, and/or stability characteristics of the Product UT / AUT.

      Performance testing is a means of quality assurance (QA).

      The goal of performance testing is not to find bugs but to eliminate performance bottlenecks

      IDEA BEHIND PERFORMANCE TESTING IS TO CHECK AN APPLICATION'S

      Speed 
      To determine and increase the response time in an application

      Stability 
      To check the stability and reduce failures under increased load in an application

      Scalability 

      To determine the maximum user/transaction load that application can sustain and to increase its capacity if required

       A study at Yahoo found that on average only 10-20% of total page loading time is spent on the back-end and other 80-90% time is spent on the front-end.

      COMMON TYPE OF PERFORMANCE teST 

      Single user performance tests 
      This is usually done at app development level using HTTP debugging  profiling tools.Fix performance issues at a single user level if the response time is high.
      Load tests 
      load testing is done to check application's performance at a predetermined load . The load that the application is subjected upon is based on the peak load that the application witnessed in the last release. As per planning standards, performance testers prefer to go for a value that is 125% of the last release load value.
      Benchmarking test 
      Benchmarking tests are usually the initial multi-user tests done on applications with minimal load . The transactions’response times are benchmarked in these tests and made better in subsequent releases.  

      Cont'd Performance Testing 


      Stress tests 
      is done to determine the breaking point of an application. The user load is increased gradually till the time the application breaks.

      Capacity determination tests 
      SLAs are set for each transaction in terms of response time values and failure rates. The user load upon the application is increased in an incremental manner and the point is determined from where SLAs are violated. That point is considered as the capacity of the application

      Metrics to monitor (SS) 

      Processor Usage 
      Percentage of the time that the processor consumes while executing threads.

      Memory  
      %  of memory available on a computer that can be used to process requests.

      Disk time 
      Page faults/sec 
      %  of time for which the disk remains busy while executing read and write requests.
      Rate at which fault pages are processed by the processor is measured by this counter .

        Cont'd

        Total Network bytes per second 
         speed at which data is sent and received on the interface  

        Hits/second 
        No. of hits on a application servers per unit time 

        Active sessions  
        Total number of active sessions on the server.

        CPU Interrupts/sec 
        Average number of H/W  interrupts a processor is receiving & processing 

        Performance Bottlenecks (CS)


        • Page size | No. of request 
        •  Rendering time | Parsing Time
           
        • Load  time  calculated as 

          T total = Tn_end  - Tn_start 

        front-end  Performance Tools 

        PERFORMANCE Tools 






        It  helps to make Client side  web  faster.

        performance Test scenarios


        •  if page load time is within acceptable range
        •  page load on slow connections
        • Response time for any action under light, normal, moderate and heavy load conditions
        • performance of database stored procedures and triggers
        •  database query execution time
        • for load testing of application
        • Check for stress testing of application
        •  CPU and memory usage under peak load condition


        security 


        Security testing is a broad effort that requires domains of expertise beyond traditional software testing.


        OWASP Top -1o


        *The Open Web Application Security  Project 

        Some Security Testing Terms


        “Vulnerability”?
        This is a weakness in the web application. The cause of such a “weakness” can be bugs in the application, an injection (SQL/ script code) or the presence of viruses.

        “SQL injection”?
        is the process of inserting SQL statements through the web application user interface into query that is then executed by the server.

        “URL manipulation”?
        Web applications communicate additional information between the client (browser) and the server in the URL. Changing "information" in the URL may sometimes lead to unintended behavior by the server.


        cont'd

        "XSS (Cross Site Scripting)?”
        When a user inserts HTML/ client-side script in the user interface of a web application and this insertion is visible to other users, it is called XSS.


        In order to perform a security test , tester should have an understanding how client (browser ) and server communicates using HTTP . Basics of Sql injection & XSS 

        Security test Scenarios

         SQL injection attacks

        Secure pages should use HTTPS protocol

        Page crash should not reveal application or server info. 

        Escape special characters in input

        Error messages should not reveal any sensitive information


         All credentials should be transferred over an encrypted channel


        Test password security and password policy enforcement

        cont'd


         Password should not be stored in cookies

        Check if directory listing is prohibited

        Verify CAPTCHA functionality

         Important events should be logged in log files

        Check if access privileges are implemented correctly

        security testing tool





        Mozilla addons 

        SQL Inject Me: for testing SQL injection vulnerabilities

        XSS-Me: for testing XSS vulnerabilities

        QA Overview Workshop

        By xavierval

        Source: Nishant Pandey

        QA Overview Workshop

        Intro to test methods & tools

        • 323

        More from xavierval