Open

Introduction

The purpose of this document is to outline the User Acceptance Testing (UAT) process for the Implementation of Adonis for any customer both in Office and specifically in the Pilot Ship ([Client Vessel]). 

UAT is to be completed by the Functional Team with the Customer’s and Adonis’s other project members’ assistance as required. After go-live, the functional members will be utilizing the software, and the IT project support members will provide support. The testing is conducted to enable a user to validate that the software meets the agreed-upon acceptance criteria.

Project Overview

See Project Charter as to Background, Goals and Objectives, Scope Definition, Project Milestones, Assumptions, Constraints, Dependencies, Project Risks, Communication Plan, Project Budget, and Project Organizational Structure.

Objectives of the Test Plan

This test plan is intended to communicate to all stakeholders the detailed plan for running the UAT tests as to:

1. What is to be done in UAT.

2. Define the Scope of what will be tested.

3. Estimate the people and other resources required.

4. Organize the activities and timescales.

5. Specify the approach taken to testing.

6. Define the deliverables expected.

7. Specify how the testing results will be evaluated.

8. Estimate the risks to the testing plan and how to mitigate them.

9. Use as a basis for agreement by key stakeholders that the plan is acceptable.

Objectives of UAT

The UAT is designed to:

• Check that the functionality that is delivered works for relevant business scenarios.

• Check that all functionality required for business scenarios has been delivered.

• Check that the delivered functionality works to specification.

• Check that any process gaps are identified and addressed accordingly, e.g., change process, etc.

• Check that any functionality not delivered during UAT will be delivered within the agreed time before go-live. If not, a solution must be in place to address the gap. Such a solution should be evaluated as to acceptability before the sign-off of UAT.

• Decide whether the functionalities that have been tested have met the Customer’s overall requirements to roll out Adonis to the relevant business users.

Scope of UAT

In Scope

All business scenarios, functionalities, and features within the relevant Phase is In Scope.

Out of Scope

Other Phases - business scenarios, functionalities and features, and testing business continuity plan.

References

The following are reference documents the Customer can use for assistance with the UAT plan:

Test Items

For each Phase, the customer should outline all modules to be tested or not.

Features To Be Tested

Features Not To Be Tested

This is necessary to avoid later confusion when stakeholders thought something would be tested but was not.

User Acceptance Test Methodology

These sections define the QUALITY wanted from the UAT testing. These sections do not specify the quality expected from the system.

The section will define the techniques, the approach, and the completion criteria to enable the testing tasks to be identified and the amount of time they will require.

User Acceptance Testing will be conducted primarily by the end-users. Users will execute all approved test scripts as per the above section.  Users may also perform additional tests not detailed in the plan but remain relevant and within the Project’s Scope.  The UAT progress will be reported based on the percentage of executed test cases and other appropriate testing activities.

Users will report issues and defects to the assigned analysts or personnel assigned for documentation and escalation.  These incidents will be described, prioritized, and tracked by using screen captures, vocabulary, and steps necessary for the Customer and Adonis IT support group to reproduce the defect.  Any issues not within the Scope of this Project will go through the change process to further determine its impact on cost and the project duration (time) and resources involved. However, a resolution to change requests is not expected to be provided by Adonis AS within the UAT period, and such will not affect the UAT sign-off or go-live date of this UAT Phase.

Develop Tests

Testing will be done in Phases as outlined in the Project Charter. At the end of a Phase, the Customer will perform a UAT containing only test cases applicable to the specific Phase.

The Test Acceptance Criteria will be determined by the Customer and reviewed by Adonis AS. Test Cases, and related Test Scripts will be based on business processes. E.g., creating a new person, promoting a person, adding an activity to a person, scheduling a person to attend training, etc. The Customer’s functional resources will prepare such cases based on the company’s specific processes and use of the Adonis systems.

If reports testing is to be done, the specifications signed-off by the Customer will be used as the basis of the test cases to determine the Acceptance Criteria.

Samples of Test Phases (refer to Project Charter)

Phase 1: 

Adonis Personnel Manager

• Organizational Structure

• General Codes

• Relevant Data Groups

• Competence

• Requirements Profile

Adonis Control Centre

APM Modules

• Rotation Planning

• Crew Change

• Payroll Module

• Report Module for Phase 1

Phase 2: 

Adonis Crew Portal

• Time Clock

• Work & Rest Hours

• Other ship-related modules

• Manning Agent-related Access/Modules

• Report Module for Phase 2

Phase 3: 

Adonis Personnel Manager/Adonis Crew Portal

• Course Scheduler

• Cabin Allocation

• Crew Station Bill

• Module for Phase 3

Phase 4: 

Adonis Web Recruitment

• Flight scheduling/booking

• Web Recruitment

• Report Module for Phase 4

Prepare to Test

In addition to developing the User Acceptance Tests, other activities should take place to allow the UAT to proceed without significant issues.

• All applications required to test Adonis seamlessly will be checked to ensure UAT is executed as planned. This may include relevant Microsoft Office Applications and other Operating systems.

• A Performance Test should be done to make sure the performance or connectivity is within an acceptable range. The performance may adversely affect UAT results if performance issues are left unresolved before UAT.

• The Customer should coordinate with other vendors and make sure that a contact person is made available for testing of any interfaces or files.

Run Tests

Run Tests deal with the process of running the tests that have been developed. Steps include:

1. Based on the Test case, mock data or transactions will be entered in Adonis.

2. The Customer will check the samples to determine if the test has passed or failed based on pre-set success criteria.

3. Issues encountered should be documented and escalated.

4. Defects within the Scope should be corrected.

5. Re-test any issues/defects that have been fixed.

6. Document result of re-test.

7. Resolve if test and re-test results are acceptable.

8. Sign-off on UAT results.

Review Test Results

After the testing has been completed and the faults are fixed and re-tested, the test results need to be reviewed to make a decision about whether to accept or reject the current configuration and solutions.

Plan Testing

This section will be planned during the Project together with the Customer. It will state what plans will be created and how they will be reported.

Change Management

Anything out of scope will go through the Change Management process to analyze the impact of costs, resources, time, or project duration.

Item Pass/Fail Criteria

The Customer should describe the process and overall standards for evaluating the test results.

UAT Defects

Defects will be entered and tracked via spreadsheet by the assigned resource during the UAT process.  Each entry will include detailed information about each defect.

UAT Defect Tracking

Team members will be provided with instruction on effectively executing test scripts and identifying capture and report defects.  Utilization of Microsoft Office applications and screen capture programs (e.g., SnagIt) will be required to document defects for escalation. Team members will be expected to present findings on regularly scheduled touchpoint meetings if additional details are required.

UAT Defect Prioritization

Defects found in UAT can be assigned one of four (4) levels of severity:

• Critical – Testing defects that due to the complexity of the function or the scheduled dates are putting the implementation date at risk.  No workaround exists.

• High – Testing defects occurring in a less complex function of the application with sufficient time to resolve before the implementation date – but must be implemented as scheduled.  A workaround has been identified.

• Medium/Low – Testing defects occur in a simple function and should be reported but will not affect the scheduled implementation date.

UAT Defect Lifecycle

Defects must be captured and escalated to ensure prompt resolution by development.  Each fault submitted by UAT will be assigned a priority. Critical and High Priority issues will be resolved, and re-tested by UAT prior to closure

Acceptance Criteria

TBD

Suspension Criteria and Resumption Requirements

This section outlines the RISK processes followed during UAT testing. It specifically deals with the process used to agree when to suspend and later resume testing.

UAT Activities

All core UAT activities are defined below:

• Identify the UAT Team

• Create the UAT Plan
  A strategy-based document defining test methodology and criteria is distributed to the team.

• UAT Plan Team Review
  Hold a session with the business stakeholders to review the plan and provide feedback and sign-off.

• UA Test Cases
  This document details each specific test case that will be performed during the UAT process.

• Test Data Acquisition
  Receipt of accounts and test environment data from QA required to execute test scripts. Note: one full week of lead time is needed to acquire test accounts from QA.

• UA Test Cases
  This document details a step-by-step breakdown of each individual test case.

• UA Test Case Review
  Approval from business team and/or third parties on completed scripts.

• Desktop Validation
  A validation of installed applications and configuration that is necessary for the users to complete the testing.

• UAT Environment Validation
  A validation of connectivity and expected results in the test environment for each end user that participates in the testing.

• Test Case Execution
  Completion of all test scripts by test team.

• Defect Tracking
  Defects will be entered and tracked via a spreadsheet by the assigned resource. Each entry will include detailed information about each defect.

• UAT TouchPoint
  There will be regularly scheduled meetings to evaluate UAT progress and outstanding defects.

• UAT Sign-Off
  A formal sign-off indicates that the system satisfies the business’s needs as specified in the functional requirements and provides confidence in its use.

Responsibilities

UAT is entirely a collaborative process, and test results must be analyzed/verified from different perspectives and by team members with various levels of expertise across the business to ensure success.

The Customer is responsible for creating their own UAT test cases and plan based on their use of the system and their processes. Adonis will assist with sample UAT test cases for most modules and functions.

The testers will enter the mock data or transactions relevant to each business case. Testers may come from different departments so that a scenario can be tested – end-to-end. The verifiers/supervisors will check the test result if acceptable within the success criteria, as defined before UAT. The analysts will document any issues encountered and monitor the status of each. The Customer and Adonis project team members will check the completeness and accuracy of UAT results and discuss any relevant concerns.

Resource and Training needs

The test team must be comprised of members who possess a thorough knowledge of the current systems and processing methods.

These team members will be better able to initiate test input, review the results, and be more intuitively familiar with the impact on other related business issues and staff activities.  Members should be detail-oriented and be diligent in collecting proper documentation to support the test results.  Team members are selected based, in part, on the ability of management to reassign the daily duties they need to delegate to others (not part of the UAT team) while performing the testing.

All team members must be presented with an overview of the test process and their specific role in UAT. 

A Test Manager or a Business Analyst should be assigned to oversee testing by assigning scripts, providing general support, and serving as the primary UAT contact point throughout the test cycle.  The project managers will coordinate with the analyst. The analyst will be expected to filter out any duplicate defects found and escalate high priority issues to the team sensitively.

Schedule

Risks and Contingencies

List any issues that could go wrong with the testing process.

Approvals