Oracle Tips
Oracle regression testing tips
Deploying complex applications has always been a challenge. Most ERP databases have thousands of tables and millions of lines of related code.
Oracle Applications patch management has always been a super complex task because of the recursive many-to-many relationship between application patches. For example, a patch for a Accounts Receivable screen may have many prerequisite patches. Immediately after this patch is released, it may become a prerequisite for a collection of subsequent patches.
Remember, to an application, a "patch" may take many forms, and it's not always a modified application executable. The changes could be PL/SQL stored procedure code changes, schema changes or external application changes (i.e. changes to a host language like C++ or Java).
Managing QA testing for Oracle-based Applications
The concept of "regression testing" came about because people noticed that some fixes became "un-done" as a result of later patches. In most applications we see a complex network of related tables and a corresponding complex network of interrelated application code.
When re-certifying a database after a change to the application layer, most people use a regression testing approach whereby the system behavior is checked using a known solution set of "proper" results. The system then undergoes a regression test using several approaches:
Workload Simulation testing - An artificial simulation of end-user interaction is applied to the new software.
Workload Benchmark testing - Existing workloads are "replayed" to stress the new software. There are many approaches to Oracle benchmarking.
The "impossible" certification challenge of complete regression testing
In some software, the complex interaction of the code and the sheer number of variables presents a serious issue of "technological feasibility". In these cases, there are billions of possible testing permutations and it's technologically unfeasible to thoroughly run the whole universe of possible test-case usage combinations.
For example, a medical treatment machine once had a bug which killed someone, an obscure combination of settings caused a lethal dose of radiation. When challenged, the software company said that they had only done spot-check testing because testing every possible combination would have taken many years, unjustly delaying the release of a life-saving medical treatment.
In the real-world the DBA must choose a statistically-reliable sample size for the regression test, one that does the deepest test of system functionality, while falling within reasonable time constraints. While it may not be possible to test all of the obscure functional features, a workload of the most popular functionality can be sampled from a working system.
Regression Testing and Oracle
Oracle has many tools to facilitate regression testing in complex environments, tools to help ensure a production-like environment, with production data and a typical workload, long enough to capture all "popular" system functionality"
Data Quality features - In Oracle11g we now see a methods to automate regression tests using the popular Oracle Data Guard replication product. The new standby snapshot feature allows you to encapsulate a snapshot for regression testing. You can collect a standby snapshot and move it into your QA database, ensuring that your regression test uses real production data.
Workload Quality Features - Oracle 11g also provides an exciting new workload capture and replay feature that promises to allow the DBA to record and playback a set of "popular" transactions.