IIS Assessment

IIS Assessment, the second stage of the Measurement and Improvement Initiative, involves formal testing to assess individual IIS systems using community-selected measures and tests. Launching a formal IIS Assessment process to drive quality improvement and alignment with standards provides information to plan and prioritize improvements while also fine-tuning standards and measures. This creates an iterative process of improvement of both IIS programs and the metrics used to assess them. Test results are shared with individual IIS users via the Aggregate Analysis Reporting Tool (AART). IIS programs and vendors can receive technical assistance to accelerate improvement.

All measures and tests used for IIS Assessment are directly related to the IIS Functional Standards and Operational Guidance Statements. Measures and tests are developed by the Measurement for Assessment and Certification Advisory Workgroup (MACAW), vetted for comment with the full IIS community via a Town Hall meeting format, and reviewed and approved by the AIRA Board prior to implementation.

There are four content areas (i.e., functional areas) currently available for IIS Assessment:

  • Transport
  • Submission/Acknowledgment
  • Query/Response
  • Clinical Decision Support (CDS)

For questions regarding IIS Assessment, please contact Kristi Siahaya.

Measures and Tests by Content Area

Below are all measures and tests developed to date. Baseline and quarterly re-measurement reports are also available for the completed IIS Assessment content areas via the AIRA repository found here.

Transport

This content area examines the level of alignment with the formally defined transport specification: SOAP/Web Services and the CDC WSDL. When any two systems connect to exchange data, they must use an agreed upon transport layer to connect; otherwise, communication and the exchange of data will be impossible. An expert panel made up of subject matter experts from across the IIS and stakeholder community selected SOAP/Web Services and CDC WSDL as the IIS Standard in 2012.

Helpful resources for the Transport content area:

  • Finalized and approved measures for Transport are available here.
  • Information on the community-wide Town Hall meeting, along with a recording and presentation slides are available.
  • Transport training videos are available here.
  • Transport Assessment Aggregate Reports are available here.

Message Submission (VXU) and Acknowledgment (ACK)

This content area focuses on Submission and Acknowledgment messages to an IIS. VXU, or vaccine update, is an HL7 message that contains demographic and vaccination data which is submitted to an IIS by an outside electronic health record system (EHR). ACK, or acknowledgment, is an HL7 message used to relay information back to the EHR regarding the status of the VXU.

Helpful resources for the Submission and Acknowledgment content area:

  • The finalized and approved measures for Submission and Acknowledgment are available here.
  • Information on the community-wide Town Hall meeting, along with a recording and presentation slides are available.
  • Information on the additional community-wide Town Hall meeting to add measures to Submission and Acknowledgment Assessment, along with a recording and presentation slides are available.
  • Submission and Acknowledgment Assessment Aggregate Reports are available here.

Message Query (QBP) and Response (RSP)

This content area focuses on Query (QBP) HL7 message submissions that are sent to an IIS and Response (RSP) HL7 message submissions that are returned by an IIS in response to a querying system. The QBP is used to query data from the IIS and contains basic demographic data about a patient. The sending system wants to know if the IIS has any demographic and/or vaccination data on the said patient, so it “asks” the IIS to respond to the QBP message. Once the IIS receives a QBP, the IIS attempts to match the demographic information contained in the QBP message with data in the IIS. It then sends back an appropriate response HL7 message (RSP) telling the sending system if the patient was found in the IIS and what information (if any) was found. 

Helpful resources for the Query and Response content area:

  • The finalized and approved measures for Query and Response are available here.
  • Information on the community-wide Town Hall meeting, along with a recording and presentation slides, are available.
  • Query and Response Assessment Aggregate Reports are available here.

Clinical Decision Support (CDS)

This content area focuses on Clinical Decision Support (CDS) tools designed to automatically determine the recommended immunizations needed when a patient presents for vaccination. These recommendations are developed by the Advisory Committee on Immunization Practices (ACIP). The measures for this content area focus on three CDS concepts which can be returned via HL7 messages: Evaluation Status, Earliest Date, and Recommended Date. Each IIS is assessed on capability to return a concept and accuracy if the concept is returned based on age of the patient.

Helpful resources for the CDS content area:

  • The finalized and approved measures for CDS are available here.
  • Information on the community-wide Town Hall meeting, along with a recording and presentation slides, are available.
  • CDS Aggregate Reports are available here.

Data Quality (DQ)

MACAW is currently discussing the development of data quality measures, particularly focused on data at rest and incoming/ongoing data.

Additional Resources

Immunization Information System (IIS) Measurement and Improvement: Testing and Discovery, Assessment and Validation Overview