Approving Scripts for Clinical Use

This chapter describes how you can use script approval in Eclipse to support the script development at your clinic.

Script approval makes sure that all ESAPI scripts used clinically have been validated and approved by senior clinical personnel.

Approve a Script for Clinical Use

Write-enabled scripts must be approved before they can be used in a clinical system. A responsible senior level staff member can approve them in Eclipse (External Beam Planning and Plan Evaluation) and BrachyVision (Brachytherapy Planning and Brachytherapy 2D Entry). After the script has been properly coded and tested according to your script development process, do the following:

  1. In Eclipse or BrachyVision, choose Tools > Script Approvals to open the ESAPI Script Administration window.

  2. Copy the script to the clinical system and click Register New Script to register it in the system.

  3. To approve for a limited evaluation, choose Approve for Evaluation Testing.

  4. To approve for full clinical use for all clinical users, choose Approve.

  5. Close the Script Approvals dialog box.

  6. Select File > Save All to save changes.

Script Development Process

Scripts are developed and tested in a non-clinical development system that allows running both read-only and write-enabled scripts without any approval process. When development is completed, scripts are moved to the clinical system, and to allow their use, scripts must be approved.

Script approval is mandatory for write-enabled scripts that use the Eclipse Automation features. For read-only scripts, approval is optional. Script approval is possible only for binary plug-in scripts and stand-alone executables. Any write-enabled custom action pack used in a visual script must be approved separately.

It is possible to first approve the script for evaluation use so that a group of users with specific evaluation rights can use the script for a limited time period in a clinical environment.

The following diagram illustrates the suggested steps in script development:

The system provides logging so that the QA responsible person can always find all plans and structure sets that have been modified by a write-enabled script.

Modifications made by a script to plans and structure sets are shown in the Plan Approval Wizard. The planner must acknowledge and approve the changes during the plan approval process.

For more information on configuration, see Chapter Approving Scripts for Clinical Use Section Configuring a Non-Clinical Development System, and Section "Configure a Clinical System to Require Approval for All Scripts".

Example Script Development Process

The following sections illustrate how script approval can be used to support the script development process in a radiotherapy clinic.

Initial Phase

  • The lead Clinical Physicist in the Radiotherapy Department asks Programmer to develop a new script.

  • Programmer starts creating the script following professional software engineering practices and the software development lifecycle in use at his institution.

Development in a Non-Clinical System

To develop the script, Programmer does the following work on the non-clinical development system:

  • Works with Clinical Physicist to develop the user story and document the intended clinical use for the script.

  • Writes testable user and system requirements for the script and reviews those with Clinical Physicist and other stakeholders.

  • Analyzes potential risks and hazards with colleagues and documents a risk and hazards analysis for the script along with mitigators for identified potential hazards.

  • Creates and documents a design for the script, if the script is large enough to warrant a design, and reviews that with stakeholders.

  • Programs the script source code and reviews the source code with other programmers.

  • Uses a software configuration management system to version and archive source code.

  • Creates automated unit tests and verifies that near 100% code coverage is possible.

  • Releases the script to Clinical Physicist and copies it to the release directory on the non-clinical development system.

Commissioning and Validation

Clinical Physicist does the following:

  • Works with Programmer to create a commissioning and validation plan for the script.

  • Validates and commissions the script for its intended clinical use and environment.

  • After confirming that the script satisfies requirements and works as intended, creates a validation and commissioning report.

  • Copies the script to the System Scripts directory on the clinical system.

Evaluation in a Clinical System

Clinical Physicist does the following:

  • Signs in to the clinical system and opens the ESAPI Script Administration window.

  • Chooses the newly developed script and sets the status to Approved for Evaluation.

  • In Varian Service Portal, gives Run Script in Evaluation State user rights to two senior Treatment Planners, and notifies them that the script is available for evaluation.

Treatment Planners do the following:

  • At Clinical Physicist's instruction, evaluate the script in the clinical system by shadowing actual clinical cases they are working on.

  • Create evaluation courses with evaluation plans that are created with the script, and compare those evaluation plans against the actual clinical plans they have manually created.

  • Complete the evaluation phase of the script, when they have successfully planned a number of different types of cases for different sites as defined in the validation and commission plan.

Clinical Physicist does the following:

  • Reviews the results of the evaluation phase, and then finalizes the validation and commissioning report.

Approval for Clinical Use

Clinical Physicist does the following:

  • Signs in to the clinical system and opens the ESAPI Script Administration window. Chooses the script and sets its status to Approved.

  • The script is now available for routine clinical use by all treatment planners in the department.

Differences between Clinical and Non-Clinical Environments

Non-clinical environments (also called research environments) are different from clinical environments in that:

  • Users cannot treatment-approve plans, which prevents treating from a research environment database.

  • The title bars in Eclipse workspaces note that the system is running in a research environment.

  • All Eclipse printouts generated from this environment note that they were created in a research environment.

  • The Beam Configuration application warns that changes should only be made if the DCF environment is dedicated for research.

  • All plans created or modified through the Eclipse Scripting API have their intent set to "Research".

  • Users can execute unapproved scripts for debugging and testing purposes.

Configuring a Non-Clinical Development System

When Eclipse is configured as a non-clinical development system (also called research system), additional API features are available for non-clinical research and development use.

A development environment has an Eclipse Scripting API research license installed, and the Varian System database has been configured for research use. The configuration is typically done by Varian service personnel at time of installation.

Configure Eclipse for Non-Clinical Use

To configure an Eclipse system for non-clinical development use (assuming you have the proper user rights and the Eclipse Scripting API for Research Users license):

  1. Open RT Administration using a system administrator account.

  2. Click System and Facilities.

  3. Click System Properties.

  4. Select the Database in Research Mode check box.

  5. Select File > Save All to save changes.

Tip

You can use the following script to test whether write-enabled scripts can be run on the system.

using System;
using VMS.TPS.Common.Model.API;
using VMS.TPS.Common.Model.Types;
using System.Windows;
[assembly: ESAPIScript(IsWriteable = true)]

namespace VMS.TPS

{
    public class Script
    {
        public void Execute(ScriptContext context)
        {
            if (context.Patient == null)
            {
                MessageBox.Show("Please load a patient before running this script.");
                return;
            }
            try
            {
                // throws an exception if writable scripting not enabled.
                context.Patient.BeginModifications();
                MessageBox.Show("SUCCESS! Writable scripting is enabled.");
            }
            catch (Exception e)
            {
                string Message = "Test FAILED! Writable scripting is not enabled.\\n";
                Message += "Message:\\n" + e.Message;
                MessageBox.Show(Message);
            }
        }
    }
}

Configure a Clinical System to Require Approval for All Scripts

The Eclipse system can be configured to enforce script approvals for all script types, read-only and write-enabled. By system design, approval is mandatory for write-enabled scripts. For read-only scripts you can decide whether to make the approval mandatory or not.

To configure the system to require approvals for all script types, both read-only and write-enabled:

  1. Open RT Administration using a system administrator account.
  2. Click System and Facilities.
  3. Click System Properties.
  4. Select the Approval required for read-only scripts check box.
  5. Select File > Save All to save changes.

Release a New Version of a Script that Is in Clinical Use

Before releasing a new version of a script that is in clinical use, retire the current version of the script:

  1. In the Script Approvals dialog box, select the current version of the script and click Retire.
  2. Enter your user name and password, and click Retire.
  3. Approve the new version of the script following instructions in the prior section.
  4. Close the Script Approvals dialog box.
  5. Select File > Save All to save changes.

Find All Plans and Structure Sets Changed by a Script

You may find, for example, that a write-enabled script you created and approved produces erroneous results due to a coding defect, and the script has been approved and used clinically for some time.

To evaluate the impact of the defect, you can create a stand-alone executable script that uses the API methods PlanSetup.ApplicationScriptLogs and StructureSet.ApplicationScriptLogs to find all plans and structure sets that the script has changed in the system. The following code sample shows an implementation of a stand-alone executable Execute method that could do this. Note that a stand-alone executable script like this opens every patient in the database and should only be run after-hours when no other users are working.

static void Execute(Application app)
{
    const string MyScript = "MyScript.esapi";
    //
    // look for objects created since April 1st 2016 since script was
    // approved then.
    //
    DateTime searchSince = new DateTime(2016, 4, 1);
    double searchDays = (DateTime.Now - searchSince).TotalDays;
    foreach (PatientSummary summary in app.PatientSummaries)
    {
        Patient p = app.OpenPatient(summary);
        //
        // find all plans touched by the script
        //
        foreach (Course course in p.Courses)
        {
            foreach (PlanSetup plan in course.PlanSetups.Where.
            (ps => (DateTime.Now - ps.HistoryDateTime).TotalDays <= searchDays))
            {
                var scriptLogs = plan.ApplicationScriptLogs;
                foreach (ApplicationScriptLog log in scriptLogs.
                    Where(s => s.ScriptFullName == MyScript))
                {
                    Console.WriteLine("Plan { 0}/{ 1}/{ 2} touched by script {3} on {4}.",
                        p.Id, course.Id, plan.Id, log.ScriptFullName, log.HistoryDateTime);
                }
            }
        }
        //
        // find all structure sets touched by the script
        //
        foreach (StructureSet structSet in p.StructureSets.
            Where(ss => (DateTime.Now - ss.HistoryDateTime).TotalDays <= searchDays))
        {
            var scriptLogs = structSet.ApplicationScriptLogs;
            foreach (ApplicationScriptLog log in scriptLogs.
                Where(s => s.ScriptFullName == MyScript))
            {
                Console.WriteLine("Structure set { 0}/{ 1} touched by script {2} on {3}.",
                    p.Id, structSet.Id, log.ScriptFullName, log.HistoryDateTime);
            }
        }
        app.ClosePatient();
    }
}

Information about Used Scripts in Plan Approval

The Plan Approval Wizard contains the following information about the script that has been used to modify the plan or its structure set:

  • Name of the script.
  • Version of the script.
  • Time when the plan or structure set was first saved after the modifications.

If you copy a plan or structure set that has been modified by a script, the same modification information is also transferred to the copied plan or structure set, and shown in the Plan Approval Wizard.

However, in this case, the time when you first saved the copied plan or structure set is shown instead of the time when the original plan or structure set was saved.