Why Should You Automate Your Regression Testing Process As Soon As Possible?

What is regression testing?

Regression testing, which is also known as repeated testing, is the process to ensure all the old functionalities still correctly works with the new changes. In other words, handling regression testing is to test an already tested application to find defects as a result of the changes. This is a usual step of any software development process and done by testing specialists. Testers do regression testing by re-executing the tests against the modified application to evaluate whether the revised code breaks anything which was working earlier. The only reason regression testing might not work is that changing or adding new code to a program can easily introduce errors into code that is not intended to be changed.

Why should regression testing be automated?

Let’s consider an example and how to perform regression testing in that situation:
At the beginning of a project, assuming we have six developers and two testers. The Agile model is set up every two weeks for a Sprint. Everything started.

In the first Sprint, we start with the basic features (e.g., about ten functions), the testers begin designing testing scenarios for testing (e.g. about 100 scenarios). The very first Sprint receives a good rating from customers.

In the second sprint, the developers continue to create new features - about 10, and testers also do things like in the first Sprint - with 100 new scripts plus 100 old scenarios that need to be retested. Well, only 200 scripts, everything is still in control.

In the next Sprint, the developers need to make eight new features and update the two old features due to new customer requirements. At this point, the two testers not only have to design test scripts for the 8 new features but also have to test and update 200 old scenarios. The whole implementation is about 300 scenarios. Did you feel there is something wrong?

Over the next few Sprints, the three developers still meet the number of features and the changing requirements, but with two testers, the number of scripts to create and update are much more. Tiredness begins to spread. The lack of time and the risk of miss bug is higher and higher. Too many problems arise.
Therefore, when regression testing is automated, it enables testers to check into a variety of changes and unusual cases in the production environment. Not all regressions are caused by new features or the consequences of routine bug fixes; database updates or new browser versions can also cause them. Regression can also be an issue with efficiency and speed. Automating those cases which are stable and repeatable allows manual testers to spend more time testing various environments and for merging more complicated cases at a higher level.
And what's more, regression analysis is the key to success. It needs to deal with intelligence rather than hard work.

1. Highest Return: Execute tests that contribute to high coverage of the requirements, then any others…
2. Quickly Lower Risk: Execute tests for the most critical requirements, then any others…
3. Practically Safe: Execute tests for all the critical requirements, then any others…
   Especially since often ~20% of the test cases are covering ~80% of the business value

-------------------------------------------------

Source: https://testautomationresources.com/software-testing-basics/automated-regression-testing/

What is Exhaustive Testing? And Why It Is Impossible?

What is Exhaustive Testing?

Exhaustive testing, which is also known as complete testing, occurs when all the testers in your team are exhausted and when all the planned tests have been executed. It is a quality assurance testing technique in which all scenarios or data is tested for testing. In a more understandable way, exhaustive testing means ensuring there are no undiscovered faults at the end of the test phase. Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. As testers, we often say “well I just never have enough time for testing”. Even if you had all the time in this world, you still wouldn’t have enough time to test all the possible input combinations and output combinations.

Why Exhaustive Testing is impractical and impossible?

It is not possible to perform complete testing or exhaustive testing. For most of the systems, it is near impossible because of the following reasons:

• The domain of possible inputs of a program is too large to be completely used in testing a system. There are both valid inputs and invalid inputs.
• The program may have a large number of states. There may be timing constraints on the inputs, that is, an input may be valid at a certain time and invalid at other times. An input value which is valid but is not properly timed is called an inopportune input.
• The input domain of a system can be very large to be completely used in testing a program.
• The design issues may be too complex to completely test. The design may have included implicit design decisions and assumptions. For example, a programmer may use a global variable or a static variable to control program execution.
• It may not be possible to create all possible execution environments of the system. This becomes more significant when the behavior of the software system depends on the real, outside world, such as weather, temperature, altitude, pressure, and so on.

Exhaustive Testing Examples

Example 1:

The IE Tools > Advanced Options window

53 binary conditions

1 condition with 3 options

1 condition with 4 options

2^53 = 9,007,199,254,740,992

       x                   12

       = 108,086,391,056,891,904 possible combinations of conditions

At one second per test execution:

108,086,391,056,891,904 / 360 = 300,239,975,158,033.067 hours (12,509,998,964,918.04 days or 34,273,969,766.9 years) to test all possible combinations.

Example 2:

Let's take an e-commerce site which has the following functionalities:

• Login
• Choose a product
  • Filter product with color
  • Filter a product with price.
• Add to Cart
• Buy the product (payment portal)

On the basis of risk identifying parameters, users can create a matrix to include in the test plan. Each parameter can be given scores so that we can have a correct way to identify high-risk areas.

• Impact on Business: 1-10
• Probability of Failure: 1-10
• Regression: 1-5
• Recovery: 1-5

Let’s create a matrix for the above example:

Functionality	Impact on Business	Probability of Failure	Regression	Recovery
Login	10	3	1	1=15
Choose a product with color filter	5	5	2	2=14
Choose a product with price filter	8	5	2	2=17
Add to Cart	10	8	3	4=25
Buy the product	10	7	2	2=21

So as per the score we have “Add to cart” functionality as the foremost contender for the "highest risk area" so now we can prioritize the testing. We can also determine for which functionalities QA team needs to do a near-exhaustive testing.
QA team can streamline the risk mitigation plan by looking at their scores

• Scores 1-5:- Unit testing and reviews.
• Scores 5-10:-Unit testing + black-box testing(regression and high business impact areas)
• Scores 10-15:-Typical test types with limited depth.
• Scores 15-25:-Typical test types with depth in certain test types.
• Scores 25-30:- High-risk areas. Complete coverage and in-depth testing for all test types.

As per the above-mentioned method areas with the score above 25 should be considered as an extremely high-risk area and a near-exhaustive testing needs to be done.So for the above example "Add to Cart" should implement in-depth testing for all test types and exhaustive testing should be done.

---

Source: https://testautomationresources.com/software-testing-basics/exhaustive-testing-fundamentals/

Top 10 common API testing interview questions (for Beginner and Intermediate level)

API testing has been considered an indispensable part of the software testing in recent years. It was reported by many testers that the REST API testing interview questions and SOAP API questions could be a huge challenge for them while applying for a test professional position.

Below are the collection of top 10 popular interview questions and answers which will be useful for your upcoming interviews.

1. What and why is API testing?

This is normally the very first and the most basic questions that an interviewer will provide before any other SOAP API questions or REST API testing interview questions.

API testing is a kind of software testing which focuses on deciding whether the built APIs meet expectations about the functionality, performance, security, and reliability for an application.

API testing provides a lot of benefits, particularly in four main aspects:

-       Test for core functionality:

-       Time effective

-       Language-independent

-       Easy integration with GUI

2. Which principles need to be considered when creating an API test design?

-       Setup: Create objects, start services, and initialize the data

-       Execution: Steps to apply API or the scenario, including logging

-       Verification: Oracles to evaluate the result of the execution

-       Reporting: Pass, failed or blocked

-       Clean up: Pre-test state

3. How many types of API testing?

There is no exact number for this API testing interview question, but you can list out the nine main categories below as it almost covers the API testing:

-       Validation Testing

-       Functional Testing

-       UI testing

-       Load testing

-       Runtime/ Error Detection

-       Security testing

-       Penetration testing

-       Fuzz testing

-       Interoperability and WS Compliance testing

4. Name some of the API testing tools

The more API testing is popular, the more tools are available in the market. You can refer to this list for the reference. Generally, there are some common tools that can be your safe answer for the API testing interview question: Postman, Katalon Studio, SoapUI, Tricentis Tosca.

5. Describe the difference between API testing and UI testing

API testing enables the communication between the two separate software systems, where one system implementing an API contains functions or subroutines that may be executed by another software system.

UI  testing is mainly about testing the graphical interface, for example: how users interact with application, testing application elements like fonts, images, layouts etc. In other words, UI testing focuses more on look and feel of an application.

6. What are some common API errors?

In the list of API interview questions, this seems to be more advanced question. Here are some suggested answers:

-       Missing module errors

-       Documentation errors

-       Parameter validation errors

-       Some other standard error expectations:  if the result is not so predicted then the occurrence of errors can be seen and for the same warnings are specified in the form of a message. There can be one or more warnings within an individual module.

7. What is an API documentation:

API documentation refers to complete and accurate technical instructions on how to effectively use and integrate with an API. It includes all the information needed to work with the API, and helps project members understand all the API testing questions including details on functions, classes, return types, arguments, and also examples and tutorials.

8. How to write a good API documentation?

You can refer to some templates which are considered simple to use: Swagger, Miredot, Slate, FlatDoc, RestDoc, so on.

When writing an API document, it is also essential to consider some of the below items:

-       Source of the content

-       Document plan or sketch

-       Delivery layout

-       Information needed for every function in the document

-       Automatic document creation programs

9. Describe your understanding about RESTful web service?

RESTful web service is one of the two types of website besides SOAP. RESTful web services use the HTTP methods to implement the concept of REST architecture. A RESTful web service usually defines a URI, Uniform Resource Identifier a service, provides resource representation like JSON and a set of HTTP methods.

REST contains some major characteristics:

-       The SERVER has no status (or session data) because REST is stateless. With a well-applied REST API, the server could be restarted between two calls, since all data is transferred to the server

-       Web service uses POST method primarily to perform operations, while REST uses GET for accessing resources.

You can get more detailed questions about REST API testing interview questions  here.

10.  Define the SOAP Web service and when to apply it?

The SOAP (Simple Object Access Protocol) refers to an XML-based protocol. It is specialized for designing and developing web services together with enabling communication between applications developed on different platforms using various programming languages over the Internet. It is both platform and language independent.

SOAP API can be used to create, retrieve, update or delete records, like accounts, leads, and user-defined objects. With more than 20 different calls, people can also use the SOAP API to manage passwords, perform searches, etc. by using the SOAP API in any language that supports web services.

More SOAP interview questions are available here for the deep learning about this area.

I hope the 10 questions above could provide you a overview picture about what will happen at the API section of the interview. Of course, it is just the beginning! Try to learn more and prepare yourself with as many SOAP and REST API interview questions as possible. Last but not least, good luck, and do not forget to suggest me with any interesting piece of information collected from your testing journey!

Top 5 API Testing Tools for REST & SOAP Services

API testing (Application Programming Interface Testing) is a software testing type which focuses on the determination if the developed APIs meet expectations regarding the functionality, reliability, performance, and security of the application.
This article will provide the summary of the top API testing tools cover both open-source and commercial solutions that testing teams can select to suit their needs.

1. SoapUI
SoapUI is a headless functional testing tool dedicated to API testing, allowing users to test REST and SOAP APIs and Web Services easily.
Website: https://www.soapui.org/
Pricing: Free - $659/year

2. Postman
Postman is a good choice for API testing for those who don’t want to deal with coding in an integrated development environment using the same language as the developers.
Website: https://www.getpostman.com/
Pricing: Free - $21/user/month

3. Katalon Studio
Katalon Studio is a free automation test tool provide a common environment to create and execute UI functional, API/Web services and mobile testing.
Website: https://www.katalon.com
Pricing: Free

4. Tricentis Tosca
Tricentis Tosca is a continuous testing platform for Agile and DevOps which supports many array of protocols: HTTP(s) JMS, AMQP, Rabbit MQ, TIBCO EMS, SOAP, REST, IBM MQ,NET TCP.
Website: https://www.tricentis.com/
Pricing: Contact Sales

5. Apigee
Apigee is a cross-cloud API testing tool, allowing users to measure and test API performance, supports and build API using other editors like Swagger.
Website: https://apigee.com/api-management/
Pricing: Free trial - $2,500/month
The rest 5 tools could be found at Top API Testing Tools

There is no one-size-fits-all tool
It hurts, but true!
We believe the list above nominates the best solution available out there if you are planning to adopt API automation testing. However, like most of the solutions in this industry, finding the ideal-one-tool to do it all is almost impossible.
Some may find the features of the commercial players (Postman, Tricentis Tosca,…) are sufficient but the costs of ownership will be the show-stop factor. Open source solutions (Rest-Assured, Karate DSL,…) are affordable but require skilled resources and effort to implement the right frameworks. Tools which seem to be a relative balance between cost and other factors (Katalon Studio, Postman) might have drawbacks to specific project types that need to be considered.


API testing established its trend in automation testing, and more tools will be developed to serve the growing demands of the software development teams. Finding the perfect tool is still tough, but we have the good news that you have way more choices than before. Carefully considering your requirements, pros and cons of each solution — try not too ambitious at the early stage and trialing the top 5 relevant candidates from the list above. With the POC for these solutions created, you will have a better knowledge of your project’s critical factors and fine-tune your shortlist. This approach gives you a good chance to identify the suitable tool for the current status and information of the next choice when your project is more mature.
I would love to hear your feedback and let me know if you have other tools to record for the others’ reference.
Source: https://testautomationresources.com/testing-tool/api-testing-tools/