A Short IntroDUCTION to
(the dark Side of) QA
presented and adapted by Xavier Val
BASED ON Nishant Pandey SLIDES
Agenda
- What QA does
- What TESTING really means
- Aplication Vs Products
- V&V
- Bugs
- Web Test
What a QA does
is ...
What They want us to do:
What they Think we Do:
What we really do:
When you test a program, you want to add
some value to it
value through testing means raising the quality or reliability of the program.
Raising the reliability means
Finding and removing errors.
Finding and removing errors.
QA VS QC
Quality Control
is the process which usually includes inspection and audit operations to segregate bad from good
Quality Assurance
is a process that focuses upon each and every role and their responsibilities in the dev process and checks whether they are being accomplished as per the guidelines
QUALITY CONTROL QUALITY ASSURANCE
Concentrates on product Concentrates on process
Done after product is built Done throughout life-cycle
Defect Detection & correction Defect Prevention Method
It is a line function It is staff function
What Testing REALLY means
One of the primary causes of poor testing is the fact that people begin with a false definition of the term.
“Testing is the process of demonstrating that errors are not
present."
“The purpose of testing is to show that a program performs its intended functions correctly.”
“Testing is the process of establishing confidence that a program does what it is supposed to do.”
“TESTING IS THE PROCESS OF EXECUTING A PROGRAM/APP WITH THE INTENTION OF FINDING ERRORS.”
TESTING CAN SHOW THE PRESENCE OF BUGS BUT NEVER THEIR ABSENCE.”
EDSGER W. DIJKSTRA
Dijkstra received the 1972 ACM Turing Award for fundamental contributions in the area of programming languages
Dijkstra received the 1972 ACM Turing Award for fundamental contributions in the area of programming languages
Application Vs Products
Application
Developed for specific client or user group
Users are Highly involved in Dev
Business Knowledge and Domain Expertise
Dev is focused on fixed time and price
PRODUCT
Developed for specific market
Solving some Business Problem
End user are not involved in Dev
v & v
VERIFICATION
is also known as In-process testing and qa
Reviews & meeting to evaluate Doc , plans , code
,requirements and specification
checks "Whether we are building the product right "
VALIDATION
Validation typically involves actual testing and takes place after the verification are complete. It is Quality control.
It determines the correctness of a build with the respect to it's requirement .
checks "Whether we are building the right product"?
Two things contribute to unhappy customer , Late Delivery &
A bug is generally referred to as a feature in the application that does not work according to customer's expectation . (Business def.)
BS 7925-1: Bug--generic term for fault, failure, error, human action that produces an incorrect result.
Bug is Basically a terminology for testers
Why : Due to an unspecified or misunderstood requirements or a mistake while developing the Application
SOME DEFINITIONS
DEFECT, ERROR, BUG, FAILURE AND FAULT
ºº
A mistake in coding is called Error
Error found by tester is called Defect
Defect accepted by development team then it is called bug
Build does not meet the requirements then it Is Failure.”
COST OF BUG
In 1999 when NASA engineers lost contact with the Mars Climate Orbiter as they attempted to place it in Mars's orbit for research.
Reason : One Team was using a program that calculated distance in English Units such as 'Inches, feet & pounds ,While the other team used metric units. This discrepancy led to faulty measurements of the Spacecraft's trajectory
Loss : Operation which cost more than $655 million
Bug Lifecycle
Bug life cycle...continued.
In the Life cycle will result in a higher return on investment (ROI)
CATCH THEM EARLY !
The cost of fixing a bug differs depending on the stage of development it is caught in ...
- Requirement Stage
- Coding
- Integration Testing
- Testing Stage
- Production Stage
Key takeaway
Finding a bug later in the life cycle of the
software / Web App costs more.
The greatest ROI is to find bugs early.
Web Testing
-
Functionality Testing
- Usability testing
-
Compatibility testing
-
Performance testing
-
Security testing
Functional
Functional testing verifies that each function of the software application operates in conformance with the requirement specification. This testing mainly involves black box testing and it is not concerned about the source code of the application. Test all the links Outgoing / Internal /Anchor / MailTo link
Test your HTML and CSS Check for syntax Errors , Validation , Standard Compliance with W3C.
Test Forms ; working as expected , Check default values being populated , Mandatory fields Error message. check whether Data is populated to DB .
cont'd..
Database consistency , Data integrity are the most important part of any Software / Web App Perform basic CRUD operation to test .
Testing With DB beyond CRUD comes in Performance & Load Testing .
Test Business Workflow.... includes Testing the end -to -end workflow/ business scenarios. Negative Scenarios .
Ex.
when I call the validate_country_code() function and pass it the country code 'ZZ' it should return false is a unit test .
Functional test is when I fill out the shipping form with a country code of ZZ, I should be redirected to a help page which allows me to pick my country code out of a menu.
Tool
Usability
Usability testing generally involves setting a series of task for people to complete and noting any problems they encounter -It's as simple as that !
Don't make me think!
Steve Krug’s first law of usability
Usability is defined by Five Quality Component
Learnability
How easy to accomplish basic task the first time they encounter design ?
How easy to accomplish basic task the first time they encounter design ?
Efficiency
One the user have learned design , How quickly can they perform task ?
Memorability
When users return to the design after a period of not using it How easily can
they re-establish proficiency ?
Cont'd
Errors
How many Errors do users make , How severe are these errors & How easily they recover from the errors
Satisfaction :
How pleasant is it to use the design
Usability testing What to test ?
Four Focal Points of Design
Navigation | Content | Presentation | Interaction
UT are carried out by testers like you or a small focus group similar to the target audience of the web application.
Site Navigation Menus , buttons or Links to different pages on your site should be consistent Test the Content, no spelling or grammatical errors.
Usability Testing Methods
- Card Sorting
- Card sorting is a structured exercise that asks volunteer users to sort a list of sample of content into logical groupings.
- Card sorting comes in two flavours:
- a) open card sorting b) closed card sorting.
- Tree Testing
- Tree-testing is a lesser known UX method but can substantially help with improving problems in navigation.
- Success rate | Speed rate | Directness rate
Usability Tools
CrazyEgg - Heat Map
Compatibility
Compatibility is nothing but capability of existing or living together. In normal life, Oil is not compatible with water, but milk can be easily combined with water.
It's a type of non functional Testing used to ensure compatibility of the system/web App
CT find out how well a system performs in a particular environment that includes different hardware, network, operating system .
Tools
Performance
A performance test is a technical investigation done to determine or validate the responsiveness, speed, scalability, and/or stability characteristics of the Product UT / AUT.
Performance testing is a means of quality assurance (QA).
The goal of performance testing is not to find bugs but to eliminate performance bottlenecks
IDEA BEHIND PERFORMANCE TESTING IS TO CHECK AN APPLICATION'S
Speed
To determine and increase the response time in an application
Stability
To check the stability and reduce failures under increased load in an application
Scalability
To determine the maximum user/transaction load that application can sustain and to increase its capacity if required
A study at Yahoo found that on average only 10-20% of total page loading time is spent on the back-end and other 80-90% time is spent on the front-end.
COMMON TYPE OF PERFORMANCE teST
Single user performance tests
This is usually done at app development level using HTTP debugging profiling tools.Fix performance issues at a single user level if the response time is high. Load tests
load testing is done to check application's performance at a predetermined load . The load that the application is subjected upon is based on the peak load that the application witnessed in the last release. As per planning standards, performance testers prefer to go for a value that is 125% of the last release load value.
Benchmarking test
Benchmarking tests are usually the initial multi-user tests done on applications with minimal load . The transactions’response times are benchmarked in these tests and made better in subsequent releases. Cont'd Performance Testing
Stress tests
is done to determine the breaking point of an application. The user load is increased gradually till the time the application breaks.
Capacity determination tests
SLAs are set for each transaction in terms of response time values and failure rates. The user load upon the application is increased in an incremental manner and the point is determined from where SLAs are violated. That point is considered as the capacity of the application
Metrics to monitor (SS)
Processor Usage
Percentage of the time that the processor consumes while executing threads.
Memory
% of memory available on a computer that can be used to process requests.
Disk time
Page faults/sec
% of time for which the disk remains busy while executing read and write requests.
Rate at which fault pages are processed by the processor is measured by this counter .
Cont'd
Total Network bytes per second
speed at which data is sent and received on the interface
speed at which data is sent and received on the interface
Hits/second
No. of hits on a application servers per unit time
Active sessions
Total number of active sessions on the server.
CPU Interrupts/sec
Average number of H/W interrupts a processor is receiving & processing
Performance Bottlenecks (CS)
-
Page size | No. of request
-
Rendering time | Parsing Time
-
Load time calculated asT total = Tn_end - Tn_start
front-end Performance Tools
PERFORMANCE Tools
at last remember Yahoo's performance guideline for web
It helps to make Client side web faster.
performance Test scenarios
- if page load time is within acceptable range
- page load on slow connections
- Response time for any action under light, normal, moderate and heavy load conditions
- performance of database stored procedures and triggers
- database query execution time
- for load testing of application
-
Check for stress testing of application
-
CPU and memory usage under peak load condition
security
Security testing is a broad effort that requires domains of expertise beyond traditional software testing.
OWASP Top -1o
*The Open Web Application Security Project
Some Security Testing Terms
“Vulnerability”?
This is a weakness in the web application. The cause of such a “weakness” can be bugs in the application, an injection (SQL/ script code) or the presence of viruses.
“SQL injection”?
is the process of inserting SQL statements through the web application user interface into query that is then executed by the server.
“URL manipulation”?
Web applications communicate additional information between the client (browser) and the server in the URL. Changing "information" in the URL may sometimes lead to unintended behavior by the server.
cont'd
"XSS (Cross Site Scripting)?”
When a user inserts HTML/ client-side script in the user interface of a web application and this insertion is visible to other users, it is called XSS.
In order to perform a security test , tester should have an understanding how client (browser ) and server communicates using HTTP . Basics of Sql injection & XSS
Security test Scenarios
SQL injection attacks
Secure pages should use HTTPS protocol
Page crash should not reveal application or server info.
Escape special characters in input
Error messages should not reveal any sensitive information
All credentials should be transferred over an encrypted channel
Test password security and password policy enforcement
cont'd
Password should not be stored in cookies
Check if directory listing is prohibited
Verify CAPTCHA functionality
Important events should be logged in log files
Check if access privileges are implemented correctly
security testing tool
Mozilla addons
SQL Inject Me: for testing SQL injection vulnerabilities
XSS-Me: for testing XSS vulnerabilities