Validation

Following the stages of Testing and Discovery and IIS Assessment, Validation is a summary step to acknowledge IIS and IIS vendor products that are progressing toward or achieving alignment with community-selected measures and tests. A Validation designation is automatically achieved when an IIS or IIS vendor product meets the designated measures and tests in a specific content area (e.g., Transport), as specified in the Aggregate Analysis Reporting Tool (AART).

Measures for each content area of Validation will be drawn from published IIS Assessment measures and tests, where results are already available to individual IIS and IIS vendor products via AART. No new measures or tests will be introduced in the Validation stage that are not already measured and visible in the IIS Assessment stage. All measures and tests throughout the Measurement and Improvement Initiative have been and will continue to be developed by the Measurement for Assessment and Certification Advisory Workgroup (MACAW), vetted for comment with the full IIS community via a Town Hall meeting format, and reviewed and approved by the AIRA Board prior to implementation.

As mentioned above, a “Validation” designation is automatically achieved when an IIS or IIS vendor product meets the designated measures and tests in a specific content area (e.g., Transport, Submission, Query, etc.), but interim steps/statuses toward Validation are also recognized. Statuses include:

  • Validated
  • Validated with Minor Differences
  • Major Differences
  • Not Measured
  • Opted Out

 

Although the definitions for IIS Vendor Product Validation are the same as for individual IIS, the graphics are slightly different to distinguish IIS vendor products from individual IIS within AART. Please also note that due to IIS vendor products being base products that are not “production like,” some content areas will not be applicable, and testing will not occur (e.g. Data at Rest).

Both individual IIS and IIS vendors have expressed a desire to use results from independent testing in the contracting and procurement processes. Recently within the IIS community, more Response for Information (RFI) and Response for Procurement (RFP) contracting language is requesting alignment with IIS Functional Standards. All M&I content areas are driven by the IIS Functional Standards and provide results that can be shared to show alignment with standards. Participation in this process will allow individual IIS and IIS vendors to support each other in alignment with national standards as testing results may differ between the same IIS software depending on application configuration settings, hosting and production environmental factors. Shared independent testing results to prioritize enhancements and collaborative work between individual IIS and IIS vendors, will in turn build confidence in IIS across other communities and will help prioritize essential improvements and community-wide adoption of standards across both IIS and Electronic Health Record (EHR) systems. Over time, adoption of standards will reduce costs and resources needed for onboarding and maintaining interfaces both now and in the future.

Validation by Content Area

Below are measures and tests developed to date. Baseline and quarterly re-measurement reports are also available for the completed IIS Validation content areas via the AIRA repository found here.

Transport

This content area examines the level of alignment with the formally defined transport specification, SOAP/Web Services and the CDC WSDL. When any two systems connect to exchange data, they must use an agreed upon transport layer to connect; otherwise, communication and the exchange of data will be impossible. An expert panel made up of Subject Matter Experts (SMEs) from across the IIS and stakeholder community selected the SOAP/Web Services and CDC WSDL as the IIS Standard in 2012.

Helpful resources for Transport Validation:

Message Submission (VXU) and Acknowledgment (ACK)

This content area focuses on Submission and Acknowledgment messages to an IIS. VXU, or vaccine update, is an HL7 message that contains demographic and vaccination data which is submitted to an IIS by an outside electronic health record system (EHR). ACK, or acknowledgment, is an HL7 message used to relay information back to the EHR regarding the status of the VXU.

Helpful resources for the Submission and Acknowledgment Validation:

  • Submission and Acknowledgment Validation Reports for the Basic level and the Complete level are available.
  • Finalized and approved measures and tests for Submission and Acknowledgment are available here.
  • Information on the community wide Town Hall meeting for Submission and Query Validation, along with a recording and presentation slides are available.

Message Query (QBP) and Response (RSP)

This content area focuses on Query by Parameter (QBP) HL7 message submissions that are sent to an IIS and Response (RSP) HL7 message submissions that are returned by an IIS in response to a querying system. The QBP is used to query data from the IIS and contains basic demographic data about a patient. The sending system wants to know if the IIS has any demographic and/or vaccination data on the said patient, so it therefore “asks” the IIS to respond to the QBP message. Once the IIS receives a QBP, the IIS attempts to match the demographic information contained in the QBP message with data in the IIS. It then sends back an appropriate response HL7 message (RSP) telling the sending system if the patient was found in the IIS and what information (if any) was found. 

Helpful resources for the Query and Response content area:

Clinical Decision Support (CDS)

This content area focuses on Clinical Decision Support (CDS) tools designed to automatically determine the recommended immunizations needed when a patient presents for vaccination. These recommendations are developed by the Advisory Committee on Immunization Practices (ACIP). The measures for this content area focus on three CDS concepts which can be returned via HL7 messages: Evaluation Status, Earliest Date, and Recommended Date. Each IIS is assessed on capability to return a concept and accuracy if the concept is returned based on age of the patient.

Helpful resources for the CDS content area:

  • CDS Validation Reports for Basic and Complete level will be available soon.
  • The finalized and approved measures and tests for CDS are available here.
  • Information on the community wide Town Hall meeting, along with a recording and presentation slides, are available.

Additional Resources

Immunization Information System (IIS) Measurement and Improvement: Testing and Discovery, Assessment and Validation Overview

IIS Vendor Product Validation Summary

IIS Vendor Product Validation FAQ

Information on Participation and Sharing Settings