SENATE BILL REPORT
SB 5116
As of January 20, 2021
Title: An act relating to establishing guidelines for government procurement and use of automated decision systems in order to protect consumers, improve transparency, and create more market predictability.
Brief Description: Establishing guidelines for government procurement and use of automated decision systems in order to protect consumers, improve transparency, and create more market predictability.
Sponsors: Senators Hasegawa, Hunt, Kuderer and Wilson, C..
Brief History:
Committee Activity: State Government & Elections: 1/20/21.
Brief Summary of Bill
  • Requires public agencies to develop an algorithmic accountability report that meet certain requirements on the use of automated decision systems (ADS).
  • Specifies minimum requirements when an agency develops, procures, or uses an ADS.
  • Prohibits an agency from using ADS or equipment incorporating artificial intelligence-enabled profiling under certain conditions.
  • Requires the Office of the Chief Information Officer to adopt rules, inventory, review algorithmic accountability reports, and conduct audits for ADS.
  • Expands Washington's Law Against Discrimination to prohibit discrimination by automated decision systems.
  • Delays facial recognition statutory provisions to July 1, 2026.
SENATE COMMITTEE ON STATE GOVERNMENT & ELECTIONS
Staff: Melissa Van Gorkom (786-7491)
Background:

Automated Decision Systems in General.  Automated decision systems (ADS) are data-driven algorithmic tools that may be used to analyze and support decision-making in a variety of settings. 
 
Office of the Chief Information Officer.  The Office of the Chief Information Officer (OCIO) is housed within the Consolidated Technology Services agency, commonly referred to as WaTech.  The OCIO has certain primary duties related to state government information technology, which include establishing statewide enterprise architecture, and standards for consistent and efficient operation. 
 
Facial Recognition.  Effective July 1, 2021, unless a state or legal government agency is mandated to use specific facial recognition services pursuant to a federal regulation or order, agencies using facial recognition services must meet certain requirements regarding public notification, testing prior to deployment, independent testing for accuracy across distinct subpopulations, and service operator training.  State and local government agencies must maintain records of their use of facial recognition services, develop an accountability report for that service, and allow for public comment on the accountability report.  The final accountability report must be communicated to the public at least 90 days prior to use of the facial recognition service, and updated and submitted to a legislative authority every two years.

 

Washington's Law Against Discrimination.  Washington's Law Against Discrimination (WLAD) prohibits discrimination based on race, creed, color, national origin, sex, marital status, and other enumerated factors.  Discriminatory practices are prohibited in the areas of employment, commerce, credit and insurance transactions, access to public places, and real property transactions.

Summary of Bill:

Automated Decision Systems.  ADS is defined as any electronic software, system, or process designed to automate, aid, or replace a decision-making process that impacts the welfare or rights of any Washington resident, and that would otherwise be performed by humans.

 

Public agencies may not:

  • develop, procure, or use ADS that discriminates against an individual, or treats an individual less favorably than another, in whole or in part, on the basis of one or more factors enumerated in WLAD;
  • develop, procure, or use an automated final decision system to make a decision impacting the constitutional or legal rights, duties, or privileges of any Washington resident, or to deploy or trigger any weapon; or
  • operate, install, or commission the operation or installation of equipment incorporating artificial intelligence-enabled profiling in any place of public resort, accommodation, assemblage, or amusement or use artificial intelligence-enabled profiling to make decisions that produce legal effects or similarly significant effects concerning individuals.

 

Public agencies using or intending to develop, procure, or use an ADS must complete an algorithmic accountability report for that ADS which must be submitted to the director of the OCIO for approval.  Each algorithmic accountability report must include certain statements such as a description of the proposed ADS and its use, types of decisions the ADS will make, information on whether the ADS has been tested for bias, and data management policy that includes protocols for deployment, security, training, community engagement, and fiscal impact.

 

Agencies already using an ADS as of the effective date of this act have until January 1, 2022 to comply with the requirements or immediately cease use of the ADS until compliance is achieved.
 
The director of OCIO must post the algorithmic accountability report online and invite public comment for no less than 30 days.  The director of OCIO must determine whether the intended use of the ADS meets the minimum standards and may approve, deny, or make changes to rules and procedures set forth in the algorithmic accountability report prior to approval.
 
A public agency that develops, procures, or uses an ADS must follow any conditions set forth in the relevant approved algorithmic accountability report and the agency must, at minimum:

  • give clear notice in plain language to an individual impacted by the ADS, providing certain statements such as a description of the ADS, the proposed use of the ADS, information on the human verification process for ADS decisions, policies for deployment, and procedures regarding contesting an ADS decision;
  • ensure the ADS and the data used to develop the system are made freely available by the vendor before, during, and after deployment for agency or independent third-party testing, auditing, or research to understand its impacts;
  • ensure that any decision made or informed by the ADS is subject to appeal; immediate suspension if a legal right, duty, or privilege is impacted by the decision; and potential reversal by a human decision maker through a timely process not to exceed 20 days, and clearly described and accessible to an individual impacted by the decision; and
  • ensure the agency can explain the basis for its decision to any impacted individual in terms understandable to a layperson.

 

Public Agencies must also ensure that a procurement contract for ADS includes provisions to effectuate the minimum standards. 
 
Beginning January 1, 2022, agencies using ADS must publish online annual metrics regarding the number of requests for human review of a decision rendered by the ADS and the outcome of that review.
 
A person injured by a public agency's material violations of these provisions may institute proceedings for injunctive relief, declaratory relief, or a writ of mandate to enforce these provisions.
 
Office of the Chief Information Officer.  The OCIO must:

  • by December 1, 2021, make available on its website a comprehensive inventory of all ADS that have been proposed for or are being used, developed, or procured by state agencies, including all algorithmic accountability reports relating to such systems;
  • by January 1, 2022, in consultation with disproportionately impacted communities, adopt rules regarding the development, procurement, and use of ADS by public agencies;
  • beginning January 1, 2022, make available on its website metrics and written explanations for all decisions made regarding agency proposals to develop, procure, or use ADS; and
  • beginning January 1, 2022, conduct annual audits that include whether agencies have complied with the terms of approved algorithmic accountability reports, and descriptions of violations and any systematic issues raised by use of ADS, and make the first audit publicly available on their website by March 1, 2022, and annually thereafter.

 

Washington's Law Against Discrimination.  WLAD is expanded to prohibit discrimination by ADS against any individual on the basis of one or more factors enumerated in the WLAD.
 
Facial Recognition.  The effective date for facial recognition requirements is extended from July 1, 2021, to July 1, 2026.

Appropriation: None.
Fiscal Note: Available.
Creates Committee/Commission/Task Force that includes Legislative members: No.
Effective Date: The bill contains an emergency clause and takes effect immediately.
Staff Summary of Public Testimony:

PRO:  Racial disparities have risen to the forefront on issues and ADS or artificial intelligence are at the center of that, making decisions regarding how we do everyday things.  Artificial intelligence is intended to streamline processes but these tools impact the vulnerable communities at an alarming rate.  ADS cannot read social context.  There is no shortage of data that shows that there are disparities in how systems treat people of color.   These programs are the definition of institutional racism due to imbedded disparities or internalized racism that writers may have when writing the programs or analyzing datasets and even life or death decisions about people's lives.  There are many cases across the United States of agencies adopting faulty and bias algorithms.  This bill bans certain dangerous uses and provides transparency with regard to the use of these systems.  This would be the first bill of its kind in the United States and would set a sound standard for transparency and fairness.
 
OTHER:  Still trying to understand what systems would qualify as an ADS under the bill.   There are many legitimate public uses for this technology and so there needs to be more conversation regarding what is included.  Some examples of what might be included under the bill are use of:  polygraph machines for screening of qualified applicants; crime lab DNA, fingerprint, and firearm analysis; crime reports that use algorithms to suggest where to allocate resources; red light cameras; speed zones; and employee screening data such as years of services.  Agree with the goals of the legislation and the need but want to make sure that there are not unintended consequences on systems that are standard.

Persons Testifying: PRO: Senator Bob Hasegawa, Prime Sponsor; Jennifer Lee, ACLU-Washington; Hillary Haden, Washington Fair Trade Coalition; Jay Cunningham, PhD Student & Ethical Artificial Intellegence Researcher at University of Washington; Ben Winters, Electronic Privacy Information Center.
OTHER: James McMahan, Washington Association of Sheriffs and Police Chiefs; Vicki Christophersen, Internet Association.
Persons Signed In To Testify But Not Testifying: No one.