MVP: Three Mainstream Methods of Software Testing

Image Source: g2.com

In the product concept stage, the "Elevator Test" is carried out to determine the positioning strategy and transform product features into significant customer benefits; in the product design stage, there is only a product model, and the purpose of the test is to optimize the product's attributes and characteristics, to attract more customers.

When the product is finally completed but not yet introduced into the market, product testing is implemented to control product quality and maintain product life. Before the product is officially launched, we conduct a small-scale market test to identify the strength and weaknesses of competitors (if the product has the potential for further improvement, we can also conduct an improvement test) and determine the product's position in the target market.

The purpose of product testing varies with the development of the tested product or different stages of the life cycle. The decision to use which test research method is based on the purpose of the research, so no test can be called the best.

α (Alpha), β (Beta), and γ (Gamma) tests are commonly used to represent three stages in the software testing process and are used to test new products during the development process and before listing:

• α is the first stage, which is generally only used for internal testing;
• β is the second stage, which has eliminated most of the imperfections in the software, but there may still be defects and loopholes, which are generally only provided to specific user groups for testing and use;
• γ is the third stage. At this time, the product is quite mature and needs to be further optimized in some places before it can be released on the market.

Due to the lack of a statistical basis for sample selection, this market research method does not provide specific statistical confidence. That is, this method is not strictly quantitative analysis, but it can indeed provide detailed feedback from customers after using the product; here, the customer is facing the final product or a product that is very close to the final form and function and the testing helps to verify and improve the product.

The following is a detailed introduction to the three mainstream test methods of α, β, and γ:

I. Alpha Testing

Image Source: Clever Tap

Alpha testing is similar to usability testing (Software testing in the software world), Which is usually done by internal testers; in rare cases, alpha testing is done by customers or outsiders. The version released by the alpha test is called the alpha version (often called the DAT development test in the software field).

Alpha testing is a test conducted by a user in a development environment. It can be a controlled test conducted by users within the company in a simulated actual operating environment, trying to find errors and correct them. Programmers or testers do alpha testing.

The key to alpha testing is to simulate the actual operating environment and the user's operation of the software product as realistically as possible and try our best to cover all possible user operation modes.

Alpha testing aims to evaluate the FLURPS (function, localization, usability, reliability, performance, and support) of a software product, with a particular focus on the interface and features of the product.

Alpha testing can start at the end of software product coding, after completing module (subsystem) testing, or after confirming that the product has reached a certain degree of stability and reliability during the testing process. The relevant "test cases" should be prepared before alpha testing.

Software testing is the final review of software requirements specification, design specification, and coding before the software is delivered to users or put into operation. It is a key step in software quality assurance. Software testing is the process of executing programs to find errors.

Software testing spans two phases in the software life cycle: usually, after each module is written, it needs to be tested (called unit testing), and coding and unit testing belong to the same phase in the software life cycle; After this stage is over, various comprehensive tests will be carried out on the software system, such as integration test, system test, performance test, and configuration test, etc. Another independent stage of the software life cycle is the test stage.

Beta testing can only be started when the alpha coefficient test reaches a certain level of reliability. The alpha test is an informal acceptance test, and the software product adjusted by the alpha test is called a beta version.

II. Beta Testing

Image Source: Clever Tap

Beta testing is acceptance testing. The so-called acceptance testing is a software testing activity carried out after the software product has completed functional and system testing and before the product is released. It is the last stage of technical testing.

After passing the acceptance test, the product will enter the release stage. The version released after the beta test is called the beta version (the UAT user test in some companies), and it can be said that the beta test is a "pre-release test."

A beta version of the software will be released online and made available to a broad audience, allowing the program to be tested in the "real world" and provide a partial preview of the next release. The main purpose of beta testing is to obtain feedback from different customer groups and check the compatibility of products under different types of networks and hardware.

End users of the software perform beta testing at one or more customer sites. Unlike alpha testing, developers are usually not present at the site of beta testing since beta testing is the "real" application of the software in an environment beyond the developer's control.

Any problems (real or imagined) that users encounter during beta testing, and they regularly report these issues to the developers. After receiving the issue reported during beta testing, the developer makes necessary changes to the software product and prepares to release the final product to all customers.

In a B2B environment, beta testing usually includes the following four aspects：

We should identify a small group of "seed" customers, often called lead customers or lead users.

• We should build a test plan and identify key roles and responsibilities in product development, marketing, sales, and product management. The test plan shall include the duration of the test and the post-test disposition of the results.
• The contract between the customer and the product company should include a project plan so that the customer understands the goals, duration, and compensation for delays. In addition, a confidentiality clause should also be included.
• Customers should understand what to test and how to report back on the results. The team needs to ensure that customer data can be collected, any final interviews can be organized, and the product can be returned to the lab.
• Acceptance testing goes through a series of black-box tests. The acceptance test generally checks the product strictly according to the product specification. It compares the requirements of the software product in the specification line by line to ensure that the developed software product meets the user's requirements.

After passing the comprehensive test, the software has been fully assembled, the interface errors have been eliminated, and the last step of software testing - acceptance testing can start; acceptance testing should check whether the software can work according to the contract requirements, that is, whether it meets the software requirements Confirmation criteria in the instructions.

Acceptance testing also needs to formulate a test plan and process. The test plan should specify the test type and progress, and the test process should define some special test cases to illustrate whether the software is consistent with the requirements.

Whether it is a plan or a process, we should focus on whether the software meets all the functions and performances stipulated in the contract. Is the documentation complete and accurate? Are the human-machine interface and other aspects (for example, portability, compatibility, error resilience, maintainability, etc.) satisfactory to users?

There are two possibilities for the results of the acceptance test. One is that the function and performance indicators meet the requirements of the software requirements specification, and the user can accept it; The other is that the software does not meet the requirements of the software requirements specification, and the user cannot accept it. Serious errors and deviations are found only at this project stage, and it is generally difficult to correct them within the scheduled construction period. Therefore, it is necessary to negotiate with the user to find a proper solution to the problem.

Before the official release of large-scale general-purpose software, Alpha and Beta tests are usually required. The purpose is to test the functions and performance of the software from the perspective of actual end-users and to find errors that end users may only find.

III. Gamma Testing

Image Source: QATestLab

Gamma testing is the final level of testing, after which the software is almost the final version available on the market; at this point, we do not perform any further functional development or improvement of the software.

The only possible changes at this stage are limited code errors. Once the software is ready for release and can meet the various requirements, we begin gamma testing, which does not require any other internal testing.

Except in the high-pressure situation where the development cycle time is extremely short, and the speed to market is required to be extremely fast (since gamma testing is not common, I am not going into too much detail here).