PILOTAGE COMMISSIONERS

[Filed November 21, 2023, 8:10 a.m.]

Preproposal statement of inquiry was filed as WSR 23-15-058.

Title of Rule and Other Identifying Information: WAC 363-116-076 Examination for pilot applicants and 363-116-077 Simulator evaluation for pilot applicants.

Hearing Location(s): On January 18, 2024, at 10:00 a.m., at 2901 3rd Avenue, Agate Conference Room, Seattle, WA 98121; and via Microsoft Teams. Please contact Jolene Hamel at HamelJ@wsdot.wa.gov for a meeting link.

Date of Intended Adoption: January 18, 2023.

Submit Written Comments to: Jaimie Bever, Executive Director, 2901 3rd Avenue, Suite 500, Seattle, WA 98121, email BeverJ@wsdot.wa.gov, fax 206-515-3906, by January 11, 2024.

Assistance for Persons with Disabilities: Contact Jolene Hamel, phone 206-515-3904, fax 206-515-3906, email HamelJ@wsdot.wa.gov, by January 11, 2024.

Purpose of the Proposal and Its Anticipated Effects, Including Any Changes in Existing Rules: In preparation for the 2024 Washington state marine pilot exam, the board of pilotage commissioners (BPC) is reviewing and updating these two WAC as needed for continued psychometric validation, diversity, equity and inclusion, clarity for applicants, as well as other housekeeping items.

The proposed changes to WAC 363-116-076 will update the psychometric validation requirement of the written exam process to better reflect current industry standards. It includes a description of the process to develop a job task analysis and the updated list of areas pilot applicants are to be tested on according to the job task analysis. It also further clarifies application requirement deadlines and contains minor housekeeping for clarity.

The proposed changes to WAC 363-116-077 will update the list of areas pilot applicants are to be tested on during the simulator evaluation according to the job task analysis. The proposed language also contains minor housekeeping for clarity.

Reasons Supporting Proposal: These proposed changes were vetted through BPC's exam psychometrician and will reflect the marine pilot exam process currently under development and to be administered in April 2024. Psychometric validation is crucial for a fair, equitable, and successful exam process, and for developing a ranked list of qualified pilot candidates to be called into the training program when a position is available.

Statutory Authority for Adoption: Chapter 88.16 RCW, Pilotage Act.

Statute Being Implemented: Chapter 88.16 RCW, Pilotage Act.

Rule is not necessitated by federal law, federal or state court decision.

Agency Comments or Recommendations, if any, as to Statutory Language, Implementation, Enforcement, and Fiscal Matters: BPC received a recommendation from the pilot exam committee after working closely with the exam psychometrician and BPC's assistant attorney general favoring implementation of the proposed language.

Name of Proponent: BPC, governmental.

Name of Agency Personnel Responsible for Drafting: Jaimie Bever, Executive Director, 2901 3rd Avenue, Seattle, WA 98121, 206-515-3887; Implementation and Enforcement: BPC, 2901 3rd Avenue, Seattle, WA 98121, 206-515-3904.

A school district fiscal impact statement is not required under RCW 28A.305.135.

A cost-benefit analysis is not required under RCW 34.05.328. RCW 34.05.328 does not apply to the adoption of these rules. BPC is not a listed agency in RCW 34.05.328 (5)(a)(i).

This rule proposal, or portions of the proposal, is exempt from requirements of the Regulatory Fairness Act because the proposal:

Is exempt under RCW 19.85.025(3) as the rule content is explicitly and specifically dictated by statute.

Scope of exemption for rule proposal:

AMENDATORY SECTION(Amending WSR 12-05-064, filed 2/15/12, effective 3/17/12)

WAC 363-116-076Examination for pilot applicants.

(1) Pilot applicants must pass a written examination ((given))administered and ((graded))scored by the board or the board's designated contracting entity. The board, in consultation with its ((designated contracting entity))contracted psychometrician, will develop the written examination and set the minimum passing or "cut" score in conformance with ((a psychometrically validated process))psychometric standards as put forth by The Standards for Educational and Psychological Testing (American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME), 2014). Notice of the examination shall be published at least four months in advance by one paid advertisement in a major marine industry publication and written notice to any party who has requested notice of such examinations. The board may publish additional notices in such publications or in other media ((at such times as it deems appropriate)). Applications will be accepted by the board immediately following the publication of the notice of the examination, and the application must be received by the board before the close of business on the first business day of the month preceding the exam month. The board may, in an emergency, call for an examination on less than four months' notice.

The notice shall indicate which pilotage district or districts the examination is for and, if for both districts, the notice shall make it clear that applicants can elect to apply for a license in one or both of the districts. If an exam is given for both the Grays Harbor and Puget Sound pilotage districts, the applicants shall be ((graded))scored and ((evaluated))ranked as one applicant pool.

(2) The examination may be taken by all pilot applicants who the board has determined have met the qualifications of WAC 363-116-0751 and who:

(a) Have ((had an))complied with the application ((on file with the board for at least one month prior to the examination))deadline provided in subsection (1) of this section. This requirement may be waived by the chairperson of the board upon the showing of good cause. The application shall specify whether the applicant is applying for the Puget Sound pilotage district, the Grays Harbor pilotage district or both.

(b) Have tendered with the application a nonrefundable examination administration fee in such amount as may be set by the board ((from time))prior to ((time))each administration period. The board may, at its discretion, refund all or part of the examination administration fee for a pilot applicant who is unable to sit for the written examination.

(3) A comprehensive Job Task Analysis (JTA), consisting of a workshop with a Subject Matter Expert (SME) panel, on-the-job observation, and validation survey, must take place at least every other exam administration period, and at a minimum must occur every five years. The SME panel will be chosen by the board and shall consist of at least five members, of whom at least three members shall be active Washington state-licensed pilots.

For exam administration periods conducted between comprehensive JTAs, a SME panel will conduct a formal review to validate the exam blueprint and content outline prior to examination development. The JTA must be conducted in consultation with the contracted psychometrician and within guidelines set forth by AERA, APA, and NCME (2014).

The validation survey will yield the weightings for the exam blueprint, as established by the contracted psychometrician.

(4) The written examination shall be in compliance with RCW 88.16.090 and ((may consist of questions covering, but not limited to, the following subjects:

(a) Rules of the Road then applicable to the pilotage district for which the pilot applicant is applying and accompanying information set forth in United States government publications on the subject;

(b) Meaning and understanding of the aids to navigation;

(c) Seamanship, including piloting and ship handling, docking and undocking problems, use of ship assist tugs and anchors;

(d) Vessel traffic system regulations;

(e) Engine and rudder order commands for United States and foreign merchant vessels and United States naval vessels;

(f) Operation and use of marine radar and automatic radar plotting aids (ARPA);

(g) Ability to calculate currents and tides;

(h) Federal laws affecting mariners and pilots including environmental laws;

(i) Use of vessel navigational equipment;

(j) Duties of a pilot;

(k) Relationship between pilot and master;

(l) Bridge resource management;

(m) United States government public health quarantine regulations;

(n) Marine VHF radio usage and phraseology, including bridge-to-bridge communications regulations;

(o) Federal navigation safety and security regulations;

(p) International distress signals;

(q) Nonlocal chart knowledge, including chart symbols and abbreviations as set forth in the latest U.S. Department of Commerce, NOS (National Ocean Survey) Chart No. 1;

(r) Maneuvering behavior for different vessel types; and

(s) Impact of propulsion and maneuvering machinery on vessel navigation.

(4)))must consist of multiple-choice items. Each question on the exam will be aligned with the tasks and knowledge statements on the blueprint resulting from the JTA process. The number of questions that are mapped to each content domain will be determined by the exam blueprint.

The written exam shall be designed to test a pilot applicant's capabilities in the following areas:

(a) Prevoyage planning;

(b) Master pilot-exchange;

(d) Docking and undocking;

(e) Use of anchors;

(f) Safe navigation;

(g) Shiphandling;

(h) Restricted water transit; and

(i) Use of tugs.

(5) The written exam must be administered in a proctored setting. This must be a live proctor, either in-person or via a proctoring platform chosen in consultation with the contracted psychometrician. It shall not be AI-based nor record-and-review.

(6) A preliminary cut score will be derived via psychometric practices (e.g., a modified-Angoff methodology) prior to the administration of the exam. Following an item analysis (conducted by the contracted psychometrician) and subsequent item revisions or removal recommendations, a secondary cut score will be recommended to the board by the psychometrician. The board will then approve a final cut score, taking into account the preliminary cut score, the item analysis results, and the recommendations of the psychometrician.

(7) The board may require that the cost of the written examination will be at the expense of the pilot applicant.

AMENDATORY SECTION(Amending WSR 12-05-064, filed 2/15/12, effective 3/17/12)

WAC 363-116-077Simulator evaluation for pilot applicants.

(1) Pilot applicants who pass a written examination and whose scores are among the top ((twenty))20 (or such other number as may be set by the board) of those taking the written examination (plus any pilot applicants who tie a qualifying score) shall be eligible to take the simulator evaluation set forth in this section.

(2) The simulator evaluation shall take place at a marine simulator facility designated by the board and ((may))shall be recorded. In this evaluation pilot applicants shall be observed by available board members but shall be evaluated only by those available board members who hold, or have held a minimum U.S. Coast Guard license as master of steam or motor vessels of not more than 1600 gross tons. The board, with guidance from its contracted psychometrician, shall ((also)) appoint a minimum of two additional evaluators who hold, or have held within ((ten))10 years of the examination date, a state pilot license issued by Washington or another state. ((If the evaluation is for applicants to both the))Applicants for either Puget Sound or Grays Harbor ((and Puget Sound pilotage district, the applicants)) shall be evaluated as one applicant pool.

(3) ((All pilot applicants will be evaluated in writing based on some or all of the following factors:

(a) Fundamental piloting and ship handling ability;

(b) Ability to assimilate and prioritize all data necessary to safely maneuver the ship;

(d) Ability to respond appropriately in emergency or nonroutine situations;

(e) Ability to communicate well and project the proper bridge presence;

(f) Understanding of bridge resource management; and

(g) Understanding and command of the Rules of the Road then applicable to the pilotage district for which the pilot applicant is applying.

(4) The board, in consultation with its designated contracting entity, shall develop this simulator examination, determine the scoring method, the minimum passing or "cut" score, and the relative weight of this score to the whole examination in conformance with a psychometrically validated process.

(5) The board may require that the cost of the simulator evaluation will be at the expense of the pilot applicant.))The simulator evaluation shall be designed to assess an applicant's competency in:

(a) Docking and undocking;

(b) Safe navigation;

(d) Restricted waterway transit; and

(e) Other tasks deemed by the board, in consultation with its contracted psychometrician and a subject matter expert (SME) panel, to be necessary to the performance of safe, efficient, and competent pilotage service.

(4) The design of the exercise shall be conducted by a SME panel, consisting of at least two pilots. The board may appoint additional SMEs to the panel. The panel will work closely with the psychometrician to minimize bias as well as to maximize measurement efficiency. Subsequently, the simulator facility development team shall work with the panel and the psychometrician to develop, beta test, and finalize the exercise with the accompanying scoring rubric. All SMEs will sign confidentiality agreements and will not disclose any content of the simulator evaluation.

(5) All pilot applicants will be graded on the simulator exercise based on a validated rubric created by a SME simulation development panel, with assistance from the psychometrician. The simulator rubric will provide objective measurement points that map to the shiphandling tasks described in subsection (3)(e) of this section.

(6) A preliminary cut score will be derived via psychometric practices (e.g., a modified-Angoff methodology) prior to the administration of the simulator evaluation. Following an item analysis (conducted by the contracted psychometrician) and subsequent item revisions or removal recommendations, a secondary cut score will be recommended to the board by the psychometrician. The board will then approve a final cut score, taking into account the preliminary cut score, the item analysis results, and the recommendations of the psychometrician.

(7) The board may require that the cost of the simulator evaluation be at the expense of the pilot applicant.