IIS Assessment

Immunization information system (IIS) Assessment, the second stage of the Measurement and Improvement Initiative, involves formal testing to assess individual IIS using community-selected measures and tests. Launching a formal IIS Assessment process to drive quality improvement and alignment with standards, while acknowledging there will always be local law and policy and flexibility within the standard that can add additional requirements and contrasts between IIS, provides information to plan and prioritize improvements while also fine-tuning standards and measures. This creates an iterative process of improvement of both IIS programs and the metrics used to assess them. Test results are shared with individual IIS users via the Aggregate Analysis Reporting Tool (AART). IIS programs and vendors can receive technical assistance to accelerate improvement.

All measures and tests used for IIS Assessment are directly related to the IIS Functional Standards and Operational Guidance Statements. Measures and tests are developed by the Measurement for Assessment and Certification Advisory Workgroup (MACAW), vetted for comment with the full IIS community via a town hall meeting format, and reviewed and approved by the AIRA Board prior to implementation.

There are five content areas (i.e., functional areas) currently available for IIS Assessment:

  • Transport
  • Submission/Acknowledgment
  • Query/Response
  • Incoming/Ongoing Data
  • Clinical Decision Support (CDS)

For questions regarding IIS Assessment, please contact Kristi Siahaya.

Measures and Tests by Content Area

Below are all measures and tests developed to date. Baseline and quarterly re-measurement reports are also available for the completed IIS Assessment content areas via the AIRA repository.

Transport

This content area examines the level of alignment with the formally defined transport specification: SOAP/Web Services and the CDC Web Services Definition Language (WSDL). When any two systems connect to exchange data, they must use an agreed upon transport layer to connect. Otherwise, communication and the exchange of data will be impossible. An expert panel made up of subject matter experts from across the IIS and stakeholder community selected SOAP/Web Services and CDC WSDL as the IIS Standard in 2012.

Helpful resources for the Transport content area:

Message Submission (VXU) and Acknowledgment (ACK)

This content area focuses on Submission and Acknowledgment messages to an IIS. VXU, or vaccine update, is an HL7 message that contains demographic and vaccination data that is submitted to an IIS by an outside electronic health record system (EHR). ACK, or acknowledgment, is an HL7 message used to relay information back to the EHR regarding the status of the VXU.

Helpful resources for the Submission and Acknowledgment content area:

Message Query (QBP) and Response (RSP)

This content area focuses on Query (QBP) HL7 message submissions that are sent to an IIS, and Response (RSP) HL7 message submissions that are returned by an IIS in response to a querying system. The QBP is used to query data from the IIS and contains basic demographic data about a patient. The sending system wants to know if the IIS has any demographic and/or vaccination data on said patient, so it “asks” the IIS to respond to the QBP message. Once the IIS receives a QBP, the IIS attempts to match the demographic information contained in the QBP message with data in the IIS. It then sends back an appropriate response (RSP) HL7 message telling the sending system if the patient was found in the IIS and what information (if any) was found. 

Helpful resources for the Query and Response content area:

Data Quality Incoming/Ongoing

This content area focuses on the IIS ability to detect data quality issues on a per-message basis. The IIS will be presented with messages that contain intentional data quality errors (e.g., vaccination date before date of birth) and the IIS is expected to detect these errors. The IIS will also be assessed on the ability to retain (and return) vaccination events from submission to query without changing the meaning (e.g., NDC mapped to CVX, return key information about a vaccination event).

Helpful resources for the Data Quality Incoming/Ongoing content area:

Clinical Decision Support (CDS)

This content area focuses on Clinical Decision Support (CDS) tools designed to automatically determine the recommended immunizations needed when a patient presents for vaccination. These recommendations are developed by the Advisory Committee on Immunization Practices. The measures for this content area focus on three CDS concepts which can be returned via HL7 messages: Evaluation Status, Earliest Date, and Recommended Date. Each IIS is assessed on capability to return a concept and accuracy if the concept is returned based on age of the patient.

Helpful resources for the CDS content area:

Data Quality Data at Rest (DAR)

MACAW is currently piloting data quality measures related to data at rest and plans to move these measures into Testing & Discovery in 2021.

Additional Resources