APP Evaluation module

Introduction

The Evaluation module is a tool designed to automate the crew assessment process onboard. The module provides you with a customizable workflow that enables team or department leaders, who act as Evaluators, to communicate and share evaluation forms. With this customizable workflow, the system captures evaluations from Evaluators and Evaluee. This feature allows approvers and shore-based supervisors to compare the evaluator's assessment with the crew member's self-assessment.

Workflow

The Evaluation module allows circulating evaluations between seafarers based on the predefined chain of approval. The involved participants are then represented as:

  • Evaluator refers to Coordinator or Approver directly above Subordinate(s) in the Chain of Approval.

  • Evaluees include all the positions at the subordinate level in the Chain of Approval.

 

It is the Evaluator who initiates the evaluation process.

 

The evaluation is carried out until it reaches the final Approved stage. After that, the evaluation form is sent to an approver responsible for completing the evaluation.

The evaluation workflow outlines different stages that an evaluation document undergoes during the assessment process. For instance, an evaluation might undergo multiple rounds of assessments before the Evaluator gives their approval. Once approved, the evaluation process progresses to the subsequent level in the Chain of Approval (CoA). The crew member's position determines the choice of appraisal template. Consequently, a single CoA can support multiple workflows.

 

Module Peculiarities

Evaluation Form Template

An evaluation form is based on a template containing one or more sections. A section describes the evaluation topic and exists out of several questions. The answer type defines the format of the question and can be a free text field, multiple choice, or a set of values.
The evaluation form is divided into two: the left contains the crew members' assessment, while the evaluator uses the right.  Whether Evaluator and /or Evaluee section is available is determined by the workflow stage.

Weighing Result

You can determine the evaluation form's score by considering the importance of answers, questions, and sections. Once done, the system calculates the weighted score for each question, which involves multiplying the actual percentage of the score by the question's weight. Then, it aggregates these question scores to calculate the overall score for the section. Similarly, the system computes the weighted score for each section, and by summing the scores for all sections, we arrive at the overall calculated score for the Evaluation form.

Qw=Question Weight

Qsw = sum of the weight of all questions in the section

Qs= Question score (Defined by the answer type)

Qcw= Question calculated weight= 100/Qsw* Qw

Qcs  =Question calculated score =   (Qcw/100) *Qs

 

  • Qw (Question Weight): It represents the weight assigned to a particular question. This weight indicates the relative importance or significance of the question within its section.

  • Qsw (Sum of Question Weights): It refers to the sum of the weights of all questions present in a specific section. It provides the total weight assigned to all the questions in that section.

  • Qs (Question score): It represents the score associated with a particular question. This value reflects the response or assessment given to the question.

  • Qsc (Question calculated score): This variable calculates the real value or adjusted score for a question. It is calculated using the formula: (100/Qsw) * Qw * Qv. It considers the question weight, question value, and the total weight of questions in the section.

Example:
A section exist out 5 questions Q1, Q2, Q3,Q4, Q5. They are weighted as follows

Q1 :Qw=10, Qs=5
Q2: Qw=20, Qs=5
Q3;Qw=5, Qs=1
Q4:Qw=25,Qs=10
Q5:Qw=10, Qs=7
The following 4 steps calculate the overall score for the section:

Step 1: Calculate Qsw (Sum of Question Weights)
Qsw = QWq1 + QWq2 + Qwq3 + Qwq4 + Qwq5
Qsw = 10 + 20 + 5 + 25 + 10
Qsw = 70

Step 2: Calculate Qcw (Question Calculated Weight) for Each Question
The formula for calculating the Question Calculated Weight (Qcw) is:
Qcw = 100/Qsw x Qw

Q1
QcwQ1 = 100/70 × 10
QcwQ1 =1000/70
QcwQ1 ~ 14.29

Q2
QcwQ2 = 100/70 × 20
QcwQ2 =2000/70
QcwQ2 ~ 28.57

Q3
QcwQ3 = 100/70 × 5
QcwQ3 =500/70
QcwQ3 ~ 7.14

Q4
QcwQ4 = 100/70 × 25
QcwQ4 =2500/70
QcwQ4 ~ 35.71

Q5
QcwQ5 = 100/70 × 10
QcwQ5 =1000/70
QcwQ5 ~ 14.29

Step 3: Calculate Qcs (Question Calculated Score) for Each Question
The formula for calculating the Question Calculated Score (Qcs) is:
Qcs = Qcw/100 xQs

Q1.
QcsQ1 = 14.29/100 × 5
QcsQ1 =0.7143 x5
QcsQ1 ~ 3.57

Q2.
QcsQ2 = 28.57/100 × 5
QcsQ2 =0.2857 x5
QcsQ2 ~ 1.43

Q3.
QcsQ3 = 7.14/100 × 1
QcsQ3 =0.0714 x1
QcsQ3 ~ 0.07

Q4.
QcsQ4 = 35.71/100 × 10
QcsQ4 = 0.3571 x10
QcsQ4 ~ 3.57

 Q5.
QcsQ5 = 14.29/100 × 7
QcsQ5 =0.1429 x7
QcsQ5 ~ 1.00

 Step 4: Calculate the Overall Score for the Section
The overall score for the section is the sum of all the Question Calculated Scores (Ocs).
Overall Section Score = Qcsq1 + Qcsq2 + Qcsq3+ Qcsq4 + Qcsq5
Overall Section Score = 3.57 + 1.43 + 0.07 + 3.57 + 1.00
Overall Section Score ~ 9.64
So the overall calculated score for the evaluation form would be approximately 9.64

Ss= Section score is the sum of Qcs

Sw=Section Weight

Ssw=sum of weights of all sections in the template

Scw= Section calculated weight= (100/Ssw)*Sw

Scs =Section calculated score (Scw/100) *Ss

 

  • Ss (Section Score): It represents the cumulative value or score for a section. It is the sum of the Qsc values for all questions within that section.

  • Sw (Section Weight): It refers to the weight assigned to a particular section. Similar to question weight, it indicates the relative importance or significance of the section within the template.

  • Ssw (Sum of Section Weights): It represents the sum of the weights of all sections in the template. It provides the total weight assigned to all sections.

  • Scw (Section Real Value): This variable calculates the real value or adjusted score for a section. It is calculated using the formula: ((100/Ssw) * Sw) * Sv. It considers the section weight, section value (sum of Qrv values), and the total weight of all sections in the template.

  • Scs (Section calculated score)

  • Template Value: It represents the overall value or score for the entire template. It is the sum of the Scs values for all sections within the template.

Similarly we are calculating the section scores
Example

Section weighted
S1 : Sw =25, Sc =10
S2 : Sw =50, Sc =75

Step 1: Calculate Section (Sum of Section Weights)

Ssw = 25+50 = 75

Step 2: Calculate Scw (Section Calculated Weight) for Each Question

Scw = (100/Ssw) x Sw

S1.
Scws1 = (100/75)*25=33.33

S2.
Scws2 = (100/75)*50=33.33

Step 3: Calculate Scs (Section Calculated Score) for Each Question

Scs= (Scw/100)*Sc

S1.

Scss1 = 33.33/100 × 10
Scss1 = 3.3

S2.

Scss2 = 66.66/100 × 75
Scss2 = 49.99

Step 4: Calculate the Overall Score for the Form

Overall Form Score = Scss1 +Scss2

Overall Form Score = 3.3+49.99 = 53.29

Overall Form Score ~ 9.64

 

 

 

Functionality

This section contains a description of the Evaluation features. By adhering to these guidelines, you will be on the right path to effectively utilizing the module.

To get started, log in to the Adonis Crew Portal. On the Home page, click the Evaluation tile:

On clicking, you are then redirected to the page:

On the page, there are two operation tiles:

  • My Evaluation (available to Coordinators and Subordinates). The tile displays a crew member’s ongoing and processed evaluations. For more information, see Evaluee Workflow.

  • Co-Workers Evaluations (available to Coordinators and Approvers). The tile allows the creation and management of evaluation forms. For more information, see Evaluator Workflow.

 

Evaluator Workflow

The Co-workers Evaluation tile is only available to Evaluators predefined in the Chain of Approval. For more information, see Define Chain of Approval.

As Evaluator, you are entitled to:

  • Send evaluation forms to Evaluees and schedule meetings to discuss details. For more information, see Send Evaluation Form.

  • Assess Evaluee's skills based on the evaluation forms filled out by Evaluee. For more information, see Execute Evaluation.

  • Check the status of evaluation forms sent to Approver. For more information, see Check Evaluation Status.

 

Send Evaluation Form

As Evaluator, you can send out the evaluation forms to Evaluee. For this, on the toolbar, click Create Evaluation:

The page displays the evaluation form(s) precompiled in the Compile Evaluation Form.

To send the evaluation form, proceed as described below:

  1. Click Create next to the evaluation form you want to send out to your teammates. This displays the list of crew members matching the position(s) preselected for the current evaluation form:

  2. In the list of crew members, click Edit next to the crew member to whom the evaluation form is to be sent. This lets you schedule a meeting with the selected crew member to discuss details. For this, in the Schedule meeting start column, set the meeting date and time.

  1. Click Create to proceed.

As a result, the evaluation form is sent to Evaluee. The evaluation form status is set to Created:

 

Send Evaluation Forms in Batch

To make the workflow more efficient, you have the possibility to send out the evaluation forms in batch.

For this, follow the steps below:

  1. Navigate to Create Evaluation on the toolbar and expand the list of crew members matching the position(s) preselected for the current evaluation form:

  2. Select the Evaluees whom you want to send out the evaluation forms to and click CREATE. This opens the Confirm dialog:

  3. On the Confirm dialog, click CREATE to proceed.

As a result, the evaluation form is sent to the selected Evaluees. The affected evaluation forms statuses are set to Created:

 

Execute Evaluation

On sending the evaluation form to a crew member, as Evaluator, you are waiting until the evaluation form is filled out and sent back to you for review. To review and process the evaluation form, navigate to Evaluation > Co-workers Evaluation > Execute Evaluation:

On the page, proceed as described below:

  1. In the Select the workflow drop-down list, select the workflow to be used for evaluation. This reveals the stages defined for the active workflow.

  2. On the table, select the stage to view the evaluation form of a teammate.

  3. Open the evaluation form by clicking View. Clicking the command displays the evaluation form:

     

  4. Click SUBMIT to proceed.

As a result, the evaluation form is sent to a teammate for further processing.

The following commands are accessible to Evaluator:

  • REJECT sends the evaluation form back to a teammate. The evaluation form moves back to the previous stage.

  • CANCEL discards the changes made in the form.

  • APPROVE (available to only Approver) confirms that Approver verified the form and agreed with the answers in the form. The status of the evaluation form is Approved.

  • DISMISS cancels the evaluation form. The status of the evaluation form is Dismissed. The evaluation form is not subject for further processing.

  • SUBMIT sends the evaluation form to an approver for further processing. The status of the evaluation form is Submitted.

 

Check List of Co-Workers

Switch to the List of Co-Workers page listing your teammates based on the predefined chain of approval. For more information, see Define Chain of Approval.

In the list, you can check for whom you can create or execute evaluation forms.

 

Check Evaluation Status

As Evaluator (Coordinator or Approver), you can check the evaluation form status. For this, switch to the Status of Evaluation page that shows your teammates' evaluation forms and their current status:

As Approver, you can choose to view, approve, or reject the evaluation form by clicking the respective command.

 

Stand in for Evaluator

In case Coordinator leaves their position during an ongoing activity, Approver can take over the responsibility and manage the Evaluation forms submitted by Evaluee(s). This functionality ensures a smooth transition and continuity in managing ongoing evaluation forms even in case of urgent or unexpected Coordinator turnovers.

For this, it is enough to provide Approver with respective access rights. For more information on how to grant access, see Set Roles and Permission Levels.

 

Evaluee Workflow

The My Evaluation tile is intended for Evaluees required to pass an evaluation of their skills and/or performance.

To access the evaluation form, proceed with the following steps:

  1. Locate the Evaluation tile on the dashboard.

  1. Navigate to Evaluation > My Evaluation to view evaluation form(s) pending for processing:

 

Process Evaluation

On accessing the evaluation form, you can now proceed to fill it out. For this, first, make sure you are on the My Evaluation page displaying the evaluation form thumbnails:

Thumbnails may contain the following information:

  • Stage of the evaluation form: Created, Rejected, Approved, etc.

  • Instructions to be fulfilled.

  • Number of days within which the form is to be completed.

  • Date when the evaluation form was sent.

Now, click the thumbnail to open the evaluation form:

Fill in the fields in the form and click SUBMIT to send it back to your Evaluator for review.

The following commands are available:

  • SAVE stores changes made.

  • SUBMIT saves applied changes and sends a form back to an Evaluator.

  • CANCEL discards changes made in the form.

 

Housekeeping

This section enables you to perform a one-time setup of the module both in the Adonis Administrator Portal and Adonis Crew Portal.

Adonis Crew Portal Setup

To get started, log in to the APP Crew Portal. When the Home page appears, click the Evaluation tile:

On clicking, you are then redirected to the page:

On this page, click the Housekeeping tile to start configuring the module:

 

Set up Evaluation Workflow

On defining the chain of approval in the Adonis Administrator Portal (see Define Chain of Approval), you need to separately define the workflow between Coordinator, Subordinate, and Approver. For this, navigate to the Evaluation module > Housekeeping > Setup Workflow:

On the page, click NEW EVALUATION WORKFLOW. This evolves the Create New Evaluation Workflow dialog:

Fill in the following fields:

  • Code. Enter a unique alpha-numeric code.

  • Name. Enter the evaluation workflow name.

Click SAVE to proceed.

On saving the created workflow, you can see the predefined workflow stages:

 

To add stages to the evaluation flow, click NEW STAGE in the Create new Evaluation Workflow dialog:

 

In the Add details dialog, fill out the fields:

  • Step is a predefined sequential number of the step.

  • Evaluator timeline text. Enter a brief instruction/message that appears in the evaluator timeline.

Evaluation → Co-workers Evaluation → Execute Evaluation

 

 

  • Status. From the drop-down list, select the current stage status: Assign, Submitted, Review, Processed, Ready to process.

Evaluation → My Evaluation

 

  • Code. Enter a unique alpha-numeric code of the evaluation workflow stage.

  • Name of the Stage. Enter the evaluation stage name. The name must be unique.

Evaluation → My Evaluation

 

  • Evaluee timeline text. Enter a brief instruction/message that appears in the Evaluee timeline.

Evaluation → My Evaluation

 

  • In the Access Rights For section, define the read/write for both Evaluator and Evaluee. There are three options: Read, Write and Hide.

    • Evaluee side
      The access rights on this site are displayed in the drop-down menu. They are: read, write and hide the side. Evaluator cannot edit the form side designed for Evaluee, but only read and hide it.

    • Evaluator side
      Evaluator’s form side is totally opposite to the Evaluee’s. On this side, Evaluee can only read the information or, otherwise, the Evaluator’s side might be unavailable to Evaluee.

  • Submit by Evaluee. Select in the case at the current stage the form must be submitted by Evaluee. Selecting the option enables the Enable Reject button for the Evaluee option which might be unchecked.

  • Submit by Evaluator. Select in case in the current stage the form must be submitted by an evaluator.

  • Duration to submit form. Enter the maximum amount of days within which the form must be submitted.

  • Show Scores on the form. In the section, select one of the options that allows displaying scores in the evaluation form.

When completed, click SAVE to proceed.

As a result, a new stage appears in the Create New Evaluation Workflow dialog.

 

Create Template

You can create different templates for electronic forms intended for evaluating crew members' performance.

A template is a customizable set of sections which may contain predefined:

  • Answer types: Yes/No, TRUE/FALSE, Multiple Choice, etc. For more information, see Add Form Answers.

  • Question(s) to be used in a form. For more information, see Add Form Questions.

  • Section(s) to be available in a form. For more information, see Add Form Sections.

On adding all the template components, refer to Compose Template to get to know how to create a template.

 

Add Form Answers

To add form answers, first, navigate to Evaluation module > Housekeeping > Template. The Answer tab then opens up:

On the Answer tab, click NEW ANSWER STRUCTURE. This opens the Create new Answer Structure dialog:

In the dialog, fill in the following fields:

  • Code. Enter a unique alpha-numeric code.

  • Answer. Enter the answer type name.

  • Description. Provide a brief description of the answer type.

  • Control type. From the drop-down menu, select one of the options: Text box, Radion button*, Drop down list*, Numbering, Check box, Date.

When completed, click SAVE at the bottom of the Create New Answer Structure dialog.

 

*Selecting the Radio button or Drop down list control type reveals the section where you can set the number of choices to be displayed in the form for the current answer type. To do this, click the NEW MULTIPLE CHOICE button and fill in the table rows:

  1. In the Code column, enter a unique alpha-numeric code.

  2. In the Description column, enter the control name.

  3. In the Score column, specify the score to be counted.

  4. Is Passive, if selected, makes the record inactive. The option is primarily used to avoid deleting the records in use.

As a result, a new record becomes available on the Answer tab.

 

Add Form Questions

To add form questions, first, navigate to the Evaluation module > Housekeeping > Template > the Question tab:

On the Question tab, click NEW QUESTION. This opens the Create new Question dialog:

In the dialogue, fill in the following fields:

  • Code. Enter a unique alpha-numeric code.

  • Question. Type a question to be used in the form.

  • Description. Provide a brief description of the question.

  • Weight. Enter a numeric value specifying the importance of the question.

  • Answer type. From the drop-down list, select the answer type predefined in Add Form Answers.

When completed, click SAVE.

As a result, a new record becomes available on the Question tab.

 

Add Form Sections

To add form sections, first, navigate to Evaluation module > Housekeeping > Template > the Section tab:

On the Section tab, click NEW SECTION. This opens the Create new Section dialog:

In the dialog, fill in the following fields:

  • Code. Enter a unique alpha-numeric code.

  • Name. Enter the section name.

  • Weight. Enter a numeric value specifying the importance of the section.

  • Description. Provide a brief description of the section.

On filling in the fields, select the questions to be added to the section. The listed questions are fetched from those predefined on the Question tab. For more information on how to add form questions, see Add Form Questions.

When done, click SAVE to add a new form section to the table.

 

Compose Template

To create a form template, first, navigate to Evaluation module > Housekeeping > Template > the Template tab:

On the Template tab, click NEW TEMPLATE. This opens the Create new Template dialog:

In the dialog, fill in the following fields:

  • Code. Enter a unique alpha-numeric code.

  • Name. Enter the template name.

  • Description. Provide a brief description of the template.

On filling in the fields, select the sections to be added to the template. The listed sections are fetched from those predefined on the Section tab. For more information on how to add form sections, see Add Form Sections.

When done, click SAVE to add a new template to the table.

 

Compile Evaluation Form

After defining various workflows and composing templates, you can now specify a workflow and a template based on which crew members are to be evaluated. For this, navigate to Evaluation module > Housekeeping > Selection Criteria:

To add a new evaluation rule, click NEW SELECTION CRITERIA. This opens the Create new selection criteria dialog:

In the Details tab, fill in the following fields:

  • Evaluation Rule Code. Enter a unique alpha-numeric code.

  • Evaluation Template. From the drop-down list, select the evaluation template to be used. The list of evaluation workflows is fetched from those predefined in the Compose Template. You can only use a Template once for one Selection Criteria.

  • Evaluation Workflow. From the drop-down list, select the evaluation workflow to be used. The list of evaluation workflows is fetched from those predefined in the Set up Evaluation Workflow. It is possible to use the same Workflow for multiple Selection Criteria.

  • Scenario. Select one option from the drop-down list: After Sign-on, Before Sign-off. By selecting the option, define when the evaluation is applicable.

  • Days. Enter the number of days within which the evaluation form is to be completed and submitted.

Now, switch to the Ranks tab to define the position(s) for which the evaluation form is applicable:

To add a position, first, click NEW RANK. This enables editing the table fields:

  • Code. Enter a unique alpha-numeric code for the position.

  • Name. From the drop-down list, select the position for which the evaluation form is to be applicable.

When completed, click SAVE to add a new evaluation rule to the table:

 

Adonis Administrator Portal Setup

This section will walk you through the setup required in the Adonis Administrator Portal.

Define Chain of Approval

The Chain of Approval functionality allows defining the levels (along with positions) between which the documents will be circulating.

To get started, access the chain of approval levels by using one of the tips below:

  • Navigate to Chain of Approval > Levels.

  • On the main menu, select Chain of Approval > Chain Levels.

To create a hierarchical tree structure, select Evaluation in the listed module names and proceed as described below:

  1. Right-click the Evaluation module name and select New Chain in the menu. The following dialog appears:

     

  2. In the Chain dialog, enter the name of a new chain.

  3. Click SAVE to proceed.

As a result, a new chain will become available under the selected module:

Now, set an approver for the created chain. To do this, follow the steps below:

Right-click the newly created chain and select New Approver in the menu. This opens the Approver dialog:

  1. From the Position drop-down, select the position you want to assign to the chain.

  2. Click SAVE to proceed.

As a result, an approver level is added to the tree-view structure of the chain levels:

To define a coordinator:

  1. Under the selected module, right-click the added approver position and select Add BelowNew Coordinator from the menu. This opens the Member dialog:

  2. From the Position drop-down list, select the position you want to assign. On selecting, you can enable the options below:

    • Is Passive makes a position unavailable in the tree-view chain of approval. The option is primarily used to avoid deleting levels in use. Otherwise, all linked documents will be lost and unavailable for submitting or editing.

    • Is Coordinator sets the coordinator level in the chain of approval.

    • Subordinate creating and submitting the document gives a subordinate the permission to create and submit time registrations.

  3. Click SAVE to proceed.

As a result, a coordinator level is added to the tree-view structure of the chain levels:

To add a subordinate, follow the steps below:

  1. Right-click the added coordinator and select Add BelowNew Subordinate from the menu. This opens the Member dialog:

  2. From the Position drop-down list, select the position you want to assign. On selecting, you can enable the options below:

    • Is Passive makes a position unavailable in the tree-view chain of approval. The option is primarily used to avoid deleting levels in use. Otherwise, all linked documents will be lost.

    • Is Coordinator set the coordinator level in the chain of approval.

    • Subordinate creating and submitting the document gives a subordinate the permission to create and submit time registrations.

  3. Click SAVE to proceed.

As a result, a subordinate level is added to the tree-view structure of the chain levels.