Evidence-Based Best Practice Toolkit for Competency-Based Orientation in Integrating an Adult Pneumococcal Protocol to Improve Vaccine Rate: A Program Evaluation Review

Practice Problem: Low pneumococcal vaccine uptake among high-risk adults is partly due to underutilized or lack of nurse-driven vaccine protocols, limited understanding of vaccine intervals and indications by nurses, and lack of proper training. Currently, physician-driven orders are the only avenue for high-risk adults to obtain this vaccine in ambulatory sites. PICOT: In adults 65 years or older within an ambulatory setting, does the use of an adult pneumococcal vaccine protocol, compared to a physician order (no vaccine protocol) impact the rate of vaccination over 2 months? Evidence: Utilization of two or more interventions provided higher immunization rates. Interventions with stronger evidence rate include nurse-driven vaccine protocols, vaccine reminders, and use of electronic health system alerts. Review of programs and toolkits proved efficient interventions of vaccine programs based on CDC program evaluation framework. Intervention: The program evaluation yielded best practices for adult pneumococcal vaccine compliance utilizing nurse-driven protocols based on CDC recommendations and a competencybased orientation toolkit to support staff members when working in ambulatory settings. Outcome: A CBO toolkit was created to facilitate the ordering and administration of pneumococcal vaccines based on approved protocol to increase vaccine uptake. CBO toolkits offer nurses autonomy and increased competency with safe injection practices. Conclusion: Adult vaccine programs with nurse-driven protocols in combination with other modalities, prove effective to increase pneumococcal vaccine rates among high-risk groups and expands access to preventive health services provided by nursing staff. A CBO toolkit increases competency of injection practices to nurses in ambulatory settings. PROGRAM EVALUATION CBO TOOLKIT 4 Evidence-Based Best Practice Toolkit for Competency-Based Orientation in Integrating an Adult Pneumococcal Protocol to Improve Vaccine Rate: A Program Review Vaccine-preventable diseases (VPD) account for complications and infectious diseases with low vaccine rates among high-risk adults (Tan et al., 2020). Negative outcomes for preventable disease due to low immunization rates are strongly linked to hospitalizations or readmissions, disease complications, disabilities, income loss, and demise (Ozawa et al., 2016). According to public health institutions, 80% of the economic burden correlates to the unvaccinated population and longer life expectancy, especially among adults (Sauer, et al., 2021; Tan et al., 2020; Williams et al., 2016). Integration of effective adult immunization programs will support healthy aging and decrease the mortality and morbidity rates (Sauer et al., 2021). Evaluation of vaccine programs allow quality improvement committees to improve ineffective practices or develop new ones and improve adherence to vaccine guidelines (Pennant et al., 2015). Limited access to immunizations and underutilization of vaccines places communities at risk for susceptible diseases. To close the vaccine gap among this aging population, this program evaluation and development of a competency-based orientation (CBO) toolkit, will identify best practices on adult vaccine programs and a tool to orient nurses when on the assessment and administration of pneumococcal vaccines. Significance of the Practice Problem Low pneumococcal vaccine uptake among adults in the ambulatory setting yields poor health outcomes. Managing and treating infectious diseases among high-risk adults, especially those 65 years and older becomes challenging while dealing with comorbidities that potentially be exacerbated. Efforts to inoculate the population have been a burden by physicians alone in the PROGRAM EVALUATION CBO TOOLKIT 5 current vaccine era leading to innovative ways to improve the vaccination rate. The nursing staff must understand vaccine schedules and indication, disease prevention and treatment, as well as preventive measures and integrate them into the plan of care to recognize early signs of illness or deterioration (Jump et al., 2018). Lower respiratory infections like pneumonia and bronchitis are the leading cause of hospitalization among the geriatric population in the United States (Jump et al., 2018). Strong and consistent vaccine campaigns, hand hygiene protocols, surveillance, and control of disease outbreaks will help control infectious diseases (Center for Disease Control,


Competency-Based Orientation in Integrating an Adult Pneumococcal Protocol to Improve Vaccine Rate: A Program Review
Vaccine-preventable diseases (VPD) account for complications and infectious diseases with low vaccine rates among high-risk adults (Tan et al., 2020). Negative outcomes for preventable disease due to low immunization rates are strongly linked to hospitalizations or readmissions, disease complications, disabilities, income loss, and demise (Ozawa et al., 2016).
According to public health institutions, 80% of the economic burden correlates to the unvaccinated population and longer life expectancy, especially among adults (Sauer, et al., 2021;Tan et al., 2020;Williams et al., 2016). Integration of effective adult immunization programs will support healthy aging and decrease the mortality and morbidity rates (Sauer et al., 2021).
Evaluation of vaccine programs allow quality improvement committees to improve ineffective practices or develop new ones and improve adherence to vaccine guidelines (Pennant et al., 2015). Limited access to immunizations and underutilization of vaccines places communities at risk for susceptible diseases. To close the vaccine gap among this aging population, this program evaluation and development of a competency-based orientation (CBO) toolkit, will identify best practices on adult vaccine programs and a tool to orient nurses when on the assessment and administration of pneumococcal vaccines.

Significance of the Practice Problem
Low pneumococcal vaccine uptake among adults in the ambulatory setting yields poor health outcomes. Managing and treating infectious diseases among high-risk adults, especially those 65 years and older becomes challenging while dealing with comorbidities that potentially be exacerbated. Efforts to inoculate the population have been a burden by physicians alone in the current vaccine era leading to innovative ways to improve the vaccination rate. The nursing staff must understand vaccine schedules and indication, disease prevention and treatment, as well as preventive measures and integrate them into the plan of care to recognize early signs of illness or deterioration (Jump et al., 2018). Lower respiratory infections like pneumonia and bronchitis are the leading cause of hospitalization among the geriatric population in the United States (Jump et al., 2018). Strong and consistent vaccine campaigns, hand hygiene protocols, surveillance, and control of disease outbreaks will help control infectious diseases (Center for Disease Control, 2016).
Currently, one clinical site for the organization revealed that only 88% (774/884) of eligible adults 65-years and older met the pneumococcal vaccine schedule, however, this percentage fell below the established organizational goal of 90% as recommended by the Healthy People 2030 benchmark. Low vaccine uptake in part is due to lack of nurse-driven protocols, limited understanding of vaccine intervals and indications, fear about the safety of vaccines, underutilization of best practice alerts in electronic health records (EHR) and access to services (Colmegna et al., 2021). Harris (2021) argued the importance of improving pneumococcal vaccination rates among immunosuppressed patients in ambulatory settings that are at a greater risk of developing respiratory tract infections by integrating best practices in the clinical setting.

Global and Regional Statistics
Vaccine-preventable diseases continue to impact populations worldwide. Over the last three decades, a cumulative disease burden of 25 million pneumococcal cases in the United States has been reported and a projected cost of 653 billion over the next 30 years to treat influenza and pneumococcal disease (Tailbird et al., 2021). The World Health Organization (WHO) reported 1.5 million deaths each year because of vaccine-preventable diseases (2015).
According to Healthy People 2030, an estimated 713.9 hospitalizations related to pneumonia infections per 1000,000 adults 65-years and older were recorded in 2016 (Healthy People 2030, n.d.). Approximately "…320,000 people get pneumococcal pneumonia every year, leading to 150,000 hospitalizations and 5,000 deaths, mostly among the elderly" (Centers for Disease Control, 2016, para. 1). Despite public health recommendations of two pneumococcal vaccine doses among adults 65 years and older, the pneumonia prevention vaccine rate was 61 % in 2014 in comparison to the Healthy People 2020 goal of 90% (Hughes et al., 2018). In 2018, California reported an estimate of 6,917 deaths related to lower respiratory infections among adults (National Center for Health Statistics, 2020).

Society and Population Impact
High-risk populations are the most vulnerable and account for most serious lower respiratory tract numbers leading to a financial burden on their retirement funds and overall healthcare costs of $1.8 billion annually (Huang et al., 2011). Inoculation of different populations prevents millions of vaccine-preventable diseases annually, interrupts disease transmission domestically and abroad, and reduces financial impact on communities (Vanderslott, 2018). Overall, unvaccinated adults account for $7.1 billion of economic burden and add up to approximately 2.3 million hospital days (CDC, 2014;Huang et al., 2011).
Integrating adult vaccine programs in primary care and specialty settings will mitigate pneumococcal infections and complications among high-risk adults. These programs will improve uptake of vaccines and avoid wasting costly vaccines due to underutilization. Over the years vaccines have proven to be effective with mild to moderate side effects, adverse effects to vaccines are rare.

Purpose of the Program Evaluation Project
The purpose of this program review was to gather evidence to support a proposed Doctor of Nursing Practice (DNP) project. The guiding objectives of this program evaluation and subsequent competency-based orientation toolkit development were to: 1) Apply the CDC program evaluation framework in the appraisal of an adult pneumococcal vaccine program to identify best practice recommendations to increase vaccination rates in ambulatory settings within 10-weeks and 2) develop a competency-based orientation toolkit for nurse-driven adult pneumococcal vaccination programs for community-based practice settings to increase competency-based skillsets and improve vaccine administration practices within 10-weeks.
This program evaluation focused on best practices for an adult pneumococcal vaccine program utilizing nurse-driven protocols based on CDC recommendations and to elaborate a competency-based orientation toolkit to support staff members when working in ambulatory settings. Nurse-driven protocols facilitate the ordering and administration of pneumococcal vaccines based on approved policies of a health entity. Protocols can be embedded in electronic health systems to support clinical decision making. Knowledge-deficit among staff members for vaccine guidelines is among other barriers to vaccine protocols according to the U.S. Community Preventive Services Task Force [CPSTF] (2016). Other concerns that interfere with the implementation of vaccine protocols is the lack of training, concern for additional workload, team members hesitancy to administer vaccines without a physician's order, and concern that vaccines would interfere with other care services (CPSTF, 2016).
The use of a vaccine protocol is an evidence-based intervention adaptable to inpatient and ambulatory clinical settings and supports onboarding of the nursing team during orientation. A qualitative study revealed that a major set-back to a vaccine protocol integration is provider acceptance of this process (Dempsey et al., 2015). During an informational session, clinicians would be provided the rational for the impact of integrating a vaccine protocol with workflow improvements, enhance access to preventive health services, and the collaboration of other team members like nurses to support them with the assessment and administration of vaccines to highrisk populations. In addition, the project leader provided periodic updates of the vaccine uptake numbers once the protocol in place compliance was ensured with the process. By the end of the PowerPoint session, staff had a better understanding of the pneumococcal vaccine criteria by providing feedback via a survey following the presentation. At the end of a two-week preceptorship period, the orientee was able to properly assess, recommend, administer, and document inoculation of the vaccine referencing the available toolkit to support use of the nurse driven vaccine protocol via direct observation of the preceptor. Volunteer preceptors within the clinical site were randomly assigned.

Program Problem Statement
The following clinical question has been formulated to guide this DNP project. In adults 65 years or older within an ambulatory setting, does the use of an adult pneumococcal vaccine protocol, compared to physician order (no standing order protocol) impact the rate of vaccination over 2 months? The population was focused on adults 65 years and older since they are considered a high-risk population. An intervention of a competency-based orientation toolkit on the use of a pneumococcal vaccine order protocol improved staff compliance on usage and improve access to preventive health services without the need to schedule a visit with a physician. The comparison to the intervention is a physician order needed to activate the vaccine in the system or missed opportunities to inoculate. The outcome of this evidence-based intervention has proven to be effective in increasing vaccine uptake among under-immunized adults and familiarize staff members with underutilized vaccine protocols (Hurley et al., 2020).
A CBO toolkit was developed to guide adult pneumococcal vaccination programs and reduce the utilization gap of immunization protocols, familiarize nurses with vaccine guidelines, and increase job satisfaction. The evaluation of current adult vaccine programs identified best practices available that supported the uptake of immunizations to mitigate vaccine preventable diseases among high-risk populations, especially in adults 65 years and older.

Utility of Program Review
Integration of a nurse-driven adult pneumococcal vaccine program will contribute to enhance access to preventive health services for high-risk populations, increase vaccine uptake and mitigate missed opportunities, relieve burden on physicians alone of ordering the vaccine, maximize nurses' scope of practice, increase compliance with practice guidelines and organizational policy. The evaluation of existing adult immunization programs utilizing the CDC program performance and evaluation framework allowed insight on the effectiveness of such programs, the interventions, and analysis of their outcomes. Evaluating best practices through the scrutiny of available programs supported this project leader with the development of an effective adult pneumococcal vaccine program and CBO toolkit. Both will increase protocol utilization, improve vaccine rate, and ease the transitioning orientation for nurses new to the ambulatory setting. Potential for workflow improvements will enhance immunization programs and decrease the probability of VPD in high-risk groups in an array of settings like hospitals, long-term care facilities, and ambulatory settings. This objective will increase the rate of vaccination, reducing missed opportunities to inoculate, reduce hospitalizations rates and complications, and increasing staff adherence to guidelines and institutional policy (CDC, 2019;Quinn et al., 2020;Sheth et al., 2021).
Supporting new hires to ambulatory practices will facilitate comfort when managing nurse schedules and improve their immunization assessment and administration skills. The utilization of AVP can easily be applied in settings that manage high-risk clients like rheumatology, endocrinology, and infusion centers, as well as other settings like pharmacies.

Analytical Framework
The following analytical model will guide this program evaluation. The Centers for Disease Control and Prevention (CDC) Program Performance and Evaluation Office (PPEO) is an effective framework offered to appraise immunization programs. This framework offered a systematic approach to effectively implement or improve best practices available while evaluating adult pneumococcal vaccine programs and its relation to key stakeholders. This program evaluation framework consists of six key steps focused on stakeholder engagement, program description, evaluation design, evidence credibility or strength, justification of conclusions, and sharing utilization of findings by ensuring use and sharing lessons learned (CDC, 2017). The John's Hopkins Evidence-Based Practice (JHNEBP) framework comprised of three steps focused on practice question, evidence, and translation, served as a foundational basis since its application follows a systematic approach like the PPEO. Both frameworks highlighted the integration of all team members vital to the development of a project through identification of a clinical inquiry or population of interest, planning or description of a program or intervention, appraisal and summary of best evidence gathered, and sharing findings with stakeholders (CDC, 2017;Dang & Dearholt, 2017). For this DNP project, the PPEO framework was utilized.
Step one and two of the CDCs PPEO framework aims to identify stakeholders invested and description of the program. Key team members identified are the nurses involved with the utilization of an adult pneumococcal vaccine program/protocol, physicians willing to share acceptance of nurses involvement with application of program, practice managers support of program implementation within clinical site, regulatory team involved with the results of program outcome to meet organizational quality metrics, informatics or analytics team to integrate other electronic health system functions to support program integration and data collection, and other ambulatory nurses compliance with program participation. For stakeholders to buy in, an informational session will be provided on the relevance of best practice applications to improve vaccination rates and integrate utilization of a nurse-driven pneumococcal vaccine protocol/program and review the workflow that integrate best practices. An explanation of the proposed workflow will include purpose of the program which aims to improve vaccination rates and facilitate the orientation or training for nurses with use the CDCs pneumococcal vaccine guideline road map and utilization of a CBO toolkit. The toolkit will provide access to institution policy, vaccination assessment checklist based on CDC guidelines, and handouts with related vaccine administration guidelines.
Step three of the program evaluation framework evaluates the design of the project. An outcome of this program evaluation was the creation of a CBO toolkit to guide onboarding nurses in an array of settings working with vaccine administration. Analysis of the toolkits effectiveness to support the nurses' orientation process will be appraised and need for revision.
Step four focuses on the collection of the trustworthy evidence that supports the development of an APV program and CBO toolkit. Evaluation of toolkit utilization rate produced a tangible output of its usefulness. The application of clinical judgement, analysis and translation of evidence summarizes step 5 of this system. Identification of themes on best practices provided a basis for the development of a CBO toolkit. The last step, number six, culminates with sharing the findings and recommendations of the evidence gathered to help sustain the program or need for revisions during scheduled quarterly meetings with use of PowerPoint presentations that include graphs and stakeholder feedback. A systematic approach will support the use of decision making when evaluating a programs sustainability and feasibility.

Evidence Search Strategy, Results, and Evaluation
A diligent literature review search was conducted in the following databases: CINAHL, OVID Medline, PubMed, and ProQuest. Articles published between January 1 sts 2005 and May 31 st , 2021, were included. Key terms and subject headings searched were Pneumococcal disease, pneumococcus pneumonia, vaccine compliance, adult vaccine programs, nurse-driven protocols, inoculation practices, best practice alerts, and clinical protocols. Inclusion criteria: adults 65 years and older and geriatrics; quasi-experimental studies and systematic review; ambulatory or primary care; and publication date 2005 to present. Exclusion criteria applied: pediatric, and young adults. During the search process of literature in several databases, a total of 100 results were generated by the search. The database Cumulative Index of Nursing and Allied Health (CINAHL) produced 53 citations using the mentioned keyword; the Ovid Medline yield 3 articles; PubMed database generated 23 articles and ProQuest resulted in only 1, while Google Scholar outputted 20 citations.

Evidence Search Results
After screening the titles and abstracts, a sum of ten studies remained for this project and included in a PRIMSA diagram (see Figure 1). One of the ten articles was a systematic review with a level II of evidence and B grade of evidence based on the Johns Hopkins EBP model. Seven of ten articles yield a level II of evidence with four B and three a level A grade of evidence. Three of the articles revealed a III level of evidence. Overall, the grade and level of evidence support the effectiveness of the use of vaccine standing order protocols within a multimodal approach.
The Johns Hopkins EBP model was utilized to determine each article's level and strength of evidence (Dang & Dearholt, 2017). Table 1 provides the articles found and is organized by levels and grades according to the John's Hopkins EBP tool. Appendix A and B provide details for each article that summarizes each finding.

Evidence Evaluation
The Advisory Committee of Immunization Practices (ACIP) offers the CDC advice and guidance on the most current pneumococcal vaccine recommendations which are published in the Morbidity and Mortality Weekly Report (MMWR). The ACIPs role is to develop vaccine recommendations that include age-appropriate timelines for vaccine administration, interval between recommended doses, frequency of doses, and precautions and contraindications to guide professionals practice (CDC, 2019). As of October 2021, ACIP recommends new doses, a 15valent PCV and a 20-valent PCV for adults 65 years-old or older and for those between 19-64 years with high-risk conditions with no prior history of a PCV (Kobayashi, 2022). The implication for public health practice is based on the simplification of the vaccine guidelines.
Studies with effective APV programs recommended computerized reminders or best practice alerts based on CDC guidelines to support vaccine administrators in addition to nurse-driven protocols (Capitano et al., 2018;Tan et al., 2020;Trick, 2009).
The CDC along with the National Vaccine Advisory Committee (NVAC) have created a set of standards to improve adult immunization practices and increase vaccine uptake rates by all healthcare professionals and mitigate missed opportunities (NVAC, 2014). A four-step approach to assess, recommend, administer or refer, and document, offers a model to all healthcare professionals along the care spectrum (CDC [NVAC], 2016). The CDC PPEO framework highlights the importance of stakeholder engagement to the success of any program. This recommendation provides a pathway to stakeholders that administer vaccines and to those that do not immunize but can refer patients to professionals that do so. Assessment of the patients' vaccine status by informed professionals along with integrated protocols and up-to-date policies ensure inoculation at every visit. Once the patient is agreeable, health professionals will proceed to administer the dose or refer to a site where vaccines are offered. The last practice of standard is the effective documentation of the vaccine received by the patient whether in an electronic health system or an immunization registry. Ensure that the patient leaves with an appointment reminder if an additional dose is required to complete a series. These standards offer effective practices that support the effectiveness and guidance to improve adult immunization programs.
Ensuring the health of communities through preventive services is at the core of the U.S.
Community Preventive Services Task Force [CPSTF]. This is an independent and nonfederal agency established by the U.S. Department of Health and Human Services in 1966. The CPSTF offers evidence-based interventions across many health topics applicable to an array of settings that contain approaches that improve health, disease preventive strategies, healthcare policies and system changes, to improve the delivery of services (Guide to Community Preventive Services Task Force [CPSTF], 2016). Recommendation of strong evidence interventions found in systematic reviews by the CPSTF to implement adult vaccination programs include the use of provider reminders, provider assessment and feedback, client reminder and recall systems, home visit vaccine programs, reducing client out-of-pocket costs, standing orders, and a health care system-based interventions implemented in combination (CPSTF, 2014).
Evaluation of these programs point out effecting interventions found in scholarly articles reviewed by this author. Articles that mention adult vaccine programs and the use strategies that improve access to health services with the integration of vaccine protocols and proven effective on improving vaccination rates in health care settings among adults 65 years-old and high-risk populations.

Critical Appraisal of the Evidence with Themes
After a close review and synthesis from the collected literature, the use of an adult pneumococcal vaccine program in ambulatory settings improved patient outcomes and staff compliance to inoculate. Higher immunization rates among high-risk adults were evident especially when paired a second intervention like clinical support tools embedded in electronic health systems, best practice alerts. The literature revealed effective correlation of increased vaccines rates with the use of effective adult vaccine programs integrated to daily workflows in primary and tertiary settings. Previsit planning to identify individuals in need of a vaccine was included in workflows recommended. Many of the scholarly articles reviewed were performed in tertiary settings and few in primary care and long-term care sites. Five of ten articles were carried out in primary care settings within the United States. One of the ten reviewed the use of pharmacist-driven pneumococcal immunization protocols in 56 inpatient acute settings and 38 outpatient areas. The following themes were identified during the literature appraisal.

Single Interventions
In single intervention studies, written or electronic vaccine standing order versus the use of immunization flow sheet post-immunization policy implementation did not yield a significant change in vaccine rates (Gamble, 2008;Goebel, 2005). Vaccine champions, yearly staff education, and effective computerized monitoring systems were recommended postimplementation of vaccine programs with protocols to maintain the success of vaccine programs in any setting (Capitano, 2018;Gamble, 2008;Goebel, 2005; Tan 2020). Gamble's (2008) single intervention study revealed a statistical increase in vaccine rate but not significant when evaluating the use of a standing order policy in three primary care sites pre-and postimplementation (38% versus 13%). Although vaccine rate was improved, a factor influencing rates was the clinicians' hesitancy to recommend inoculation leading to missed opportunities.
Capitano (2018) emphasized higher patient compliance with vaccine uptake when physicians educate on the importance of health promotion and disease prevention with strong vaccine recommendations. In this study, 80% (45 of 56) of inpatient settings used pneumococcal immunization protocols in comparison to 50% (19 of 38) of outpatient settings. Computerized standing orders for pneumococcal inoculation among adults were 51% effective compared to 31% when using electronic physician reminders as referenced in Table 2 (Capitano, 2018).

Multi-Modal Interventions
Most articles reported that multimodal interventions versus single interventions provided statistically significant results as evidenced by Loskutova (2020) immunization flow sheets, checklists, and patient outreach. Loskutova (2020) focused on adults that met the criteria for inoculation in a large primary care setting implementing a multimodal intervention in comparison to the use of a clinician reminder system. Post-intervention, vaccine rates increased by 18% in contrast to the comparison group, 16% as noted in Table 2. Although vaccine rate was improved, a factor influencing rates was the clinicians' hesitancy to recommend inoculation leading to missed opportunities. Capitano (2018) emphasized higher patient compliance with vaccine uptake when physicians educate on the importance of health promotion and disease prevention. In this study, 80% (45 of 56) of inpatient settings used pneumococcal immunization protocols in comparison to 50% (19 of 38) of outpatient settings. Computerized standing orders for pneumococcal inoculation among adults were 51% effective compared to 31% when using electronic physician reminders as referenced in Table 2 (Capitano, 2018).

Pharmacy-Driven Protocols
Pharmacy-driven vaccine programs that integrated protocols were identified during the literature review as effective interventions to improve vaccination rates. Articles revealed that the use of pneumococcal immunization protocols (PIPs) supported recommendations from the Advisory Committee on Immunization Practices (ACIP). A cross-sectional study of 94 surveys for inpatient and outpatient pharmacy settings within the United States, concluded that 56% (45 inpatient settings) followed PIP and only half of the outpatient sites had pneumococcal protocols in place. Common barriers identified to the implementation of PIPs were outdated protocols, lack of knowledge to ACIP vaccine recommendations and lack of staff accountability (Capitano, 2018;Hurley et al., 2020). Additionally, the Immunization Action Coalition (IAC) exhorts all licensed health care professionals who see adults to appraise, recommend, and inoculate or refer for needed vaccines to mitigate missed opportunities and increase vaccination rates (Hurley et al., 2020). Expansion of adult vaccine programs integrate settings like pharmacies to facilitate access and accommodate payor requirements.

Clinical Practice Recommendation
These studies yielded higher vaccine rate uptake by adults. Based on the scientific evidence from the synthesized literature, adult vaccine programs that include nurse-driven vaccine protocols in combination with other modalities proved effective towards increase pneumococcal and influenza vaccination rates among elderly and adults 18 years of age and older. Consistent use of adult vaccine programs and patient reminders correlated with higher vaccine rates in ambulatory settings in comparison to hospital settings (Capitano, 2018).
Evaluated vaccine programs and practice standards strongly recommended the integration of adult vaccine programs that included standing orders or protocols since they help expand access to vaccines by including ancillary staff like nurses and pharmacist to mitigate missed opportunities.

Program Review Recommendation Statement
Integration adult pneumococcal vaccine programs in ambulatory settings allows expansion of preventive health services through nurse clinic services without the need of a physician order. Changing the current culture of immunization practices within any organization promotes accountability, increase ease to recommend vaccines, and improves vaccines administrators' skills and ineffective practices, "cultural norms define what is encouraged, discouraged, accepted, or rejected within the group" (Groysberg et al., 2018, p.4). Evidencebased projects or quality improvement assignments enhance vaccine compliance culture in any organization and contributes to implementation of cost-effective vaccine programs. This APV program will reduce the time needed to seek a signed order from a clinician, close the gap for delivery of services, and improve the patients' experience with the delivery of care. Synthesized evidence supports the integration of adult vaccine programs to free up clinicians from minor assignments, shift tasks to trained licensed personnel to aid with inoculation, improve clinical workflow, and mitigate missed opportunities. Additionally, activating best practice alerts in electronic systems to guide patient care, will support the clinical decision making when recommending missing vaccines. Instituting workflows that support clinical staff, will reinforce vaccine practice guidelines, vaccine practice standards, and pneumococcal vaccine recommendations to identified patients. Implementing a CBO toolkit will enhance the nursing team's knowledge with vaccine indications and guidelines, administration practices and vaccine adverse effects management and reporting, effective documentation practices, and use of best practice alerts in electronic health systems. As improvements in vaccination rates are established, the project can be disseminated among other primary care settings that would benefit from this intervention like specialty clinics that service high-risk adults in need of pneumonia prevention vaccines. The use of a nurse-driven protocol provided clinical significance that answered the PICOT question since its use contributes to the reduction of the public health concern related to morbidity and mortality of VPD such as pneumococcal infections among high-risk and under-immunized individuals.

Program Analysis and Evaluation Plan
After careful evaluation of the literature evidence, nurse-driven protocols are considered an evidence-based intervention with effective program outcomes to increase vaccination rates (Capitano, 2018). Nurse-driven protocols utilized in adult vaccine programs expands access to vaccines (NVAC, 2014). Patients can easily schedule an appointment with a nurse for an immunization assessment without a required primary care physician order. Adding nurse schedules to clinical practices expands access to vaccines outside of the doctor visit norm.
Applicability of vaccine protocols extends to other health settings like pharmacies, urgent cares, specialty, and retail clinics. The utilization of nurse-driven protocols or vaccine standing orders is highly advised by governmental and independent health agencies as an effective tactic to improve vaccine administration practices and improve workflows. Inclusion of best practices in the development of an adult vaccine program (AVP) along with a competency-based orientation (CBO) toolkit, will guide nurses in ambulatory settings to improve vaccination uptake and increase their awareness on best vaccine practices (Guide to Community Preventive Services Task Force, 2016).

Applicability and Implementation Strategy
The CBO toolkit will be utilized as a training tool in an internal medicine ambulatory setting by nursing staff. A registered nurse will be assigned as vaccine champion and program facilitator to support existing staff and new hires when working in the nurse clinic. Review of the toolkit will be part of the orientation phase for onboarding nurses and annual review is recommended to assess competency. Formative assessment can be attained through monthly chart audits on staff performance and impact of vaccine program, see (Appendix D). This strategy will provide practice feedback, reinforcement on best practice, and corrective actions as needed. Summative assessment can be evaluated with an annual competency review of the nurse workflow and quiz of the AVP, (Appendix E). The AVP will include the National Vaccine Advisory Committee tool that integrates Standards for Adult Immunization Practice. A recommendation to create a template for nurse visits will be proposed to include these four standards to improve the assessment of immunization status for the nurse workflow, (see Appendix F). Review of the vaccine program is located in (Appendix G).
The program will increase demand for vaccination by sending reminders when vaccines are due or recall of missed vaccine appointments with support of electronic health system support tools. The adult vaccine program will decrease the number of missed opportunities due to missing provider orders and the utilization of a nurse-driven protocol to identify eligible adults. Workflow improvements can enhance immunization programs and decrease the probability of VPD in high-risk groups.

Selection of Best Practices
The inclusion of best practices for this adult vaccine program was synthesized by the review of three health organizations that recommended vaccine guidelines and offered evidencebased interventions focused on improving vaccine practices and optimizing preventive health measures that mitigate vaccine preventable diseases like pneumonia, see summary in Table 3.
The CDC program evaluation model offered a guide to appraise each entity and toolkits described in this project, highlighting key stakeholders (nurse, manager, patients, medical director, CNO) and their association with program goals and objectives, and linking metrics that justify recommendations. Agencies such as the CDC, NVAC, and CPSTF, are aligned with their recommendations to improve vaccination rates, improve the health of at-risk populations, mitigate missed opportunities to inoculate, integrate vaccination information systems, recommendation of new vaccine policies, and enhance performance feedback (CDC, 2019;Kobayashi, 2022;NVAC, 2014). These organizations support best immunization practices for all healthcare professionals in different clinical settings when considering vaccine initiatives. The toolkits identified that follow the CDC program evaluation model are the Kaiser Permanente Covid19 vaccine equity toolkit and the Adults Immunization Toolkit for Clinicians.
For this vaccine program evaluation, the use of nurse-driven vaccine protocols is strongly recommended by two of the reviewed organizations, ACIP and NVAC. Tools to support nursedriven protocols are found in these websites which provide a guide for implementation and resources that highlight the impact of protocols to change policy (see Table 4). The CDC represents a reputable and recognized entity for vaccine guidelines and recommendations.
Additionally, the National Vaccine Advisory Committee guides the recommendations published by the CDC when following best practices for the adult pneumococcal vaccine. Both agencies set the standards for vaccine administration and guidelines to follow when health professionals advise patients. All three organizations follow most of the CDC program evaluation model's standards and steps as depicted in Table 3. They clearly identify similar key stakeholders, provide a succinct program description with defined goals, an outline of focused design to increase vaccine rate and practices, summary of evidence-based interventions that support best vaccine practices, and strong recommendations to stakeholders. The toolkits reveal relevant metrics to improve vaccine practices and offer strong interventions underutilized during nurse visits. For the proposed AVP and CBO toolkit, this framework provides a broader understanding of key elements vital to its formulation. The logic model presents a flow of multiple actions required to implement the program and improve current workflows (see Table 5). Anticipated budgetary expense for vaccine expansion over an eight-week timeline is outlined in Table 6.
Expenses entail purchase of vaccines by the health institution with an approximate amount of $2,000 per month. Staffing will not create an additional expense since an RN and LVN are already part of the clinical site team.

Program Evaluation Discussion and Recommendations
Adult vaccine programs with integrated nurse-driven vaccine protocols, support clinical workflows in ambulatory settings by expanding access to preventive services and reducing the rate of vaccine preventable disease among adults (Harris, 2021). After the appraisal of the toolkits and adult immunization programs, findings revealed that the use of more than one evidence-based intervention are more effective at increasing immunization rates than single-led interventions. Practices with nurse-driven vaccine protocols offer vaccinators autonomy to assess, recommend, and inoculate under-immunized adults in a variety of settings. Additionally, ensuring services are delivered in an equitable form by including interpreting services for non-English speakers, extended hours of nurse schedule to accommodate working families, and access to schedule appointments by phone, patient portal, and on a walk-in basis.
To evaluate staff performance on the utilization of nurse-driven vaccine protocol policy, the nurse champion will perform monthly chart audits with the use of an audit tool (see Appendix D). This tool will support data collection on nurse performing assessment, whether the immunization history was reviewed, and if the patient was immunized or not. To assess knowledge retention, every vaccine administrator will complete an annual assessment by completing a four-question quiz. Opportunity to remediate will be available after nurse meets with nurse champion to review topic. To sustain the adult vaccine program with best practices identified, the nurse champion and/or nurse preceptor will use the Immunization Action Coalition (IAC) checklist to evaluate the workflow in place. This tool will help identify areas for improvement and a plan of action to support the vaccinator. With the support of the analytics or informatics team, weekly vaccine rates will be gathered by running reports specific to the clinical site and shared during staff meetings and/or huddles meets. Vaccine rates can be posted on the daily engagement board to provide a visual aid and feedback on success or challenges of the program.
Recommendation to survey new hires two months post mentorship program is ideal to identify any barriers or suggestions about the program. This will allow them the opportunity to voice any concerns regarding the orientation process and experience with preceptor. Potential limitations to this adult vaccine program are the impact of the Covid19 pandemic on the nurse clinic accessibility due to staffing shortages and stocking of vaccines due to shipment delays. Acceptance of new practice by current staff nurses is a concern since it poses risk of scrutiny of their current practice. Colmegna et al. (2021) identified personal beliefs and vaccine experiences as barriers to increase immunization rates. Implementation of an adult pneumococcal vaccine program that integrates best practices (nurse-driven protocol policy, patient reminder system, and standardized workflow) recommended by this project, will improve vaccine rates among adults 65 years and older. It will also provide a guide to help orient new hires in ambulatory settings coordinating preventive health services like vaccines. The tools gathered will support nurse champions and mentors with the orientation process and performance evaluation.
The developed CBO toolkit (Appendix H) is designed to help orient new nurse hires execute proper vaccine needs assessment of an electronic health record, decrease missed opportunities, advise adults on the pneumococcal vaccine, and increase pneumococcal vaccine uptake. This toolkit is intended for an audience of healthcare professionals, nurses, and quality improvement nurses that seek to enhance immunization practices in their clinical setting. It outlines glossary of terms, an implementation strategy, key stakeholders involved with the implementation process, and tools that support nurse instruction and patient engagement handouts.
This program evaluation has gathered evidenced-based interventions (see Appendix H) that support the success of an adult pneumococcal vaccine program and development of an extensive competency-based orientation toolkit. Based on the CDCs program evaluation framework, this CBO toolkit outlines the benefits of investing resources to engage key stakeholders (patients, nurses, manager, physicians) with the improvement of immunization processes to mitigate vaccine preventable diseases, especially among high-risk adults. Clinical sites can obtain baseline data with the support of the electronic health system to run reports and by the utilization of an audit tool and measure the process outcome and vaccine rate post intervention. Data will reveal current adult pneumococcal vaccine status to serve as a base to compare numbers post implementation of this CBO toolkit. This toolkit meets criteria described in the CDCs framework for a successful vaccine program. Collected data can support informed decision-making for stakeholders to determine the need to stock vaccines that reduce the likelihood of pneumococcal disease among high-risk adults, increase their quality of life, and decrease hospital admissions related to community acquired pneumonia. Additionally, the CBO toolkit is designed to ease new nurse hires orientation process when working immunization clinics, empower them with resources that support their skill set, and sustain evidence-based practice in nursing. Limitations of this project is the utilization of nurse-driven protocol policy to licensed nurses only, excluding medical assistants. Another limitation is the vaccine expense by smaller private practices with a small adult population aged 65 years and older.

Dissemination Plan
Upon conclusion of this adult vaccine program evaluation and CBO toolkit, findings will be shared with clinical site members during a general staff meeting with the use of a PowerPoint presentation as a visual aid with graphs. The use of a PPT can be posted on the ambulatory services intranet site for reference and easy access. Updating the clinical site team will provide a deeper insight of the application and utilization of the CBO to improve a staff orientation and safe preventive health services. A quicker response to the toolkit use can be obtained post presentation by allowing time for question-and-answer session. At an organizational level, this health entity holds professional development committee meetings every other month and allows the opportunity to share findings during one of these scheduled sessions. These sessions can be arranged with the support of the assistant nursing director and committee members.
Dissemination at this level, reaches other ambulatory nurse leaders and potential for them to adopt the findings. Additionally, staff members of the population health services and education department working directly with meeting immunization metrics and new employee orientation, can appreciate the application and utilization of the toolkit and advantage of an adult vaccine protocol. At a national level, an abstract of the results will be submitted for a poster presentation at the annual nursing conference hosted by the National Association of Hispanic Nurses. This manuscript will be published on the University of Saint Augustine for Health Sciences institutional scholarship and open access repository (SOAR). A written manuscript will be submitted to the Journal of American Academy of Ambulatory Care Nursing and Hispanic Health Care International Journal for publication consideration.

Conclusion
The purpose of this adult vaccine program evaluation and development of a competencybased orientation toolkit was to improve the quality of health among high-risk adults from preventable vaccine diseases by increasing vaccine uptake and support the delivery of safe injection practices in ambulatory settings by nurses. Implementation of an adult vaccine protocol in ambulatory settings, as proposed by the Immunization Action Coalition, enhanced access to preventive services and adds nurse autonomy to inoculate adults. Deaths correlated to pneumonia disease among the geriatric population remain a public health concern related to underimmunized adults. Integrating a nurse-driven protocol to adult vaccine programs has a direct correlation with increase pneumococcal vaccines rates among high-risk populations. An https://www.who.int/mediacentre/commentaries/vaccine-preventable-diseases/en/         Kaiser Permanente Covid-19 Vaccine Equity Toolkit

Visit KP Covod-19 Vaccine Equity Toolkit to access full document.
Agreement: By accessing, using or implementing this document (the "Content") you understand and agree to the following: 1) the Content is provided for general informational purposes only, 2) you must exercise independent professional judgment and make decisions based upon your particular situation, 3) due to rapidly evolving information related to the subject matter of the Content, the provider of this content takes no responsibility or assume any legal liability for the accuracy of the information or for the manner in which any person who references them may apply them to any particular person, 4) the Content is not intended as medical advice, or as a substitute for the medical advice of a physician, 5) you assume all liability and responsibility for the access, use and implementation of the Content, and 6) the Content may be, in whole or in part, copyrighted material and the copyright holder retains all rights, title and interest (including intellectual property rights) in and to the Content. When data from all 4 fiscal years were combined, physicians who used standing orders had a significantly higher rate of influenza vaccination (63%) than physicians who did not (38%).
The results of this study provide evidence that standing orders for the administration of influenza vaccine are associated with higher immunization rates in an ambulatory setting.
Several previous studies have shown that standing orders increase vaccine usage in the hospital.
In a study in six community hospitals, standing orders more effectively increased vaccination (40.3%) than did chart reminders (17%) or physician education (7%). In general, end-ofintervention-year vaccination rates were higher than baseline rates with a 4%-8% increase for most vaccines at most sites.
Sites that integrated a program clinical decision support into its EMR for risk-based recommendations for pneumococcal vaccinations (PCV13 and PPSV23) impacted their rates. The baseline immunization rate for PPSV23 highrisk patients was 24%; it increased to 60%.
Sustaining higher adult immunization rates needs intervention beyond standing order.
Prioritization of adult immunization is challenging without incentives.
Better integration of clinic and state data may increase adult immunization rates.
Challenges to increasing coverage rates included prioritization of acute and chronic conditions over adult vaccination, Medicare Part D reimbursement policies, electronic medical record issues related to data reporting and programming.
The implementation of SOPs provided critical infrastructure and successfully integrated adult immunization into office routines. This study group found that automatically generated physician orders were much more effective in increasing pneumococcal vaccination rates than EHR reminders to order a pneumococcal vaccination.1 EHR-generated order set for the pneumococcal vaccination might have been more successful to prompt provider assessment of patients' pneumococcal vaccination status ordering protocol, this vaccination assessment rate was again measured from another random sample. Week 3

Summary of Systematic Reviews (SR)
Week 5 Week 7 Week 9 Week 11 Week 13 Week 15 Week 1 Week 3 Week 5 Week 7 Week 9 Week 11 Week 13 Week 15 Week 1 Week 3 Week 5 Week 7 Week 9 Week 11 Week 13 Week 15 Meet with preceptor x Project identification x Prepare project proposal Week 3 Week 5 Week 7 Week 9 Week 11 Week 13 Week 15 Week 1 Week 3 Week 5 Week 7 Week 9 Week 11 Week 13 Week 15 Week 1 Week 3 Week 5 Week 7 Week 9 Week 11 Week 13 Week 15 Program Analysis and evaluation plan  c) Order the vaccine using the nurse-driven vaccine order policy then administer proceed to inoculate.

Appendix G Proposed Policy: Nurse-Driven Pneumococcal Vaccine Protocol POLICY STATEMENT/SCOPE:
To reduce morbidity and mortality from pneumococcal disease by vaccinating all eligible adults using a nurse-driven vaccine order to patients who meet the criteria established by the Centers for Disease Control and Prevention (CDC) and the Advisory Committee on Immunization Practices (ACIP). I. DEFINITIONS: a) Shared clinical decision-making (SCDM)-recommendation made based on the patient's risk for exposure to PCV13 serotypes and the risk for pneumococcal disease for that person because of underlying medical conditions. II.
POLICY: All eligible nurses (i.e., RN and LVNs) may vaccinate eligible adults in ambulatory settings using this standing order who meet the criteria below.

III.
PROCEDURES AND RESPONSIBILITIES: a) Identify adults in need of inoculation with pneumococcal conjugate vaccine (PCV13)

Purpose Statement
This CBO toolkit is designed to help orient new nurse hires execute proper vaccine needs assessment of an electronic health record, decrease missed opportunities, advise adults on the pneumococcal vaccine, and increase pneumococcal vaccine uptake.

Audience
This CBO toolkit is intended for the following audiences in mind: • New nurse hires orienting in vaccination clinics.
• Nurses interested in quality improvement projects that enhance immunization services.
• Ambulatory health care professionals interested in improving vaccine administration services.
• Nurse vaccine champions

Definitions/Glossary
• AVP: Adult Vaccine Program • Nurse-driven vaccine protocol: approved protocol policy that allows qualified nurses to order and administer vaccines based on CDC guidelines.
• VAERS: Vaccine Adverse Effects Reporting System • SCDM: Shared clinical decision-making: recommendation made based on the patient's risk for exposure to PCV13 serotypes and the risk for pneumococcal disease for that person because of underlying medical conditions.

Implementation Strategy
The following is an overview on how adult vaccine programs using nurse-driven protocols can increase pneumococcal vaccine uptake among adults 65 years and older. Integration of nursedriven protocols and following a standard immunization practice reduces the rate of missed opportunities to vaccinate. The Centers for Disease Control and Prevention (CDC) support the use of evidence-based interventions that increase the rate of vaccine rates during clinical visits.
• Integrate a nurse-driven pneumococcal vaccine protocol to the AVP.
• Recommend a nurse template to guide nurse visit. Template to include CDCs Standards for Adult Immunization Practice.
• Train and educate staff on CDCs pneumococcal vaccine guideline.
• Review nurse visit workflow (Appendix F) • Review health policy on nurse-driven vaccine (Appendix G) • Partner new hire nurse with a preceptor for 2 weeks.
• Review best practices on injection safety and administration.
• Review appropriate use of hand hygiene.
• Review how to report a VAERS report

Staff
The following key stakeholders are vital for the implementation and sustainability of the AVP.
See role and responsibility: • Nurse preceptors: responsible of new hires orientation.
• Clinical practice manager: to support the implementation of the program and facilitate available resources to sustain program.
• Nurse vaccine champion: to coordinate adult vaccine program and update vaccine guidelines as needed. Support preceptors and new nurse hires during the orientation process and life of the adult vaccine program.
• Medical Director: to revise, approve, and update vaccine policy as needed.
• Informatics/Analytics: will support with data reports to show vaccine rates.

Evaluation strategy and tools
• Nurse vaccine champion to perform monthly chart audits (Appendix D) • Nurse champion to perform annual competency assessment (Appendix E) • Collect baseline data and report to team monthly rates during staff meetings.
• Update huddle board weekly with data (vaccine rates).
• Nurse preceptor to utilize Immunization Action Coalition skills checklist to support new hire with performance evaluation Skills checklist

Stakeholder engagement and analysis tools
• Patients Handouts: provide the vaccine information sheet and vaccine fact sheet to support decision making • Nursing staff: provide the CDCs Standards for Adult Immunization Practice Standards for Adult Immunization Practice • Increase awareness on adult immunization programs Overview on Adult Immunization Practice • Report vaccine rates and percentage of nurse-driven protocol utilization during staff meetings.
• Provide baseline data on pneumonia infection rates among adults 65 years and older currently hospitalized for selected health system.
• Display organization goal for pneumococcal vaccine rate.

Communication planning tools
Vaccine champion will inform all staff about the adult pneumococcal vaccine program and CBO toolkit. Discuss the purpose of the program, impact of workflow, and resources available to support nurses. Possible formats of communication: • Email • Staff meetings • Huddles

• Microsoft Teams
• Informal communication during rounds.

Policy/purpose statement:
To reduce morbidity and mortality from pneumococcal disease by vaccinating all eligible adults using a nurse-driven vaccine order who meet the criteria established by the Centers for Disease Control and Prevention (CDC) and the Advisory Committee on Immunization Practices (ACIP).
See Appendix G for detailed policy.
• This order authorizes registered nurses (RNs) and licensed vocational nurses (LVNs)with active licenses in California to order and administer pneumococcal vaccines.

Education Materials
These forms are periodically updated and you should verify if you have the most current. Below is a list of the forms and links to the most current:

Staff/clinicians:
• Review the CDCs injection safety checklist Injection Safety Checklist • Record vaccination in the pharmacy or medical record within 24 hours and to the Virginia Immunization Information System. • Provide vaccine recipient with a personal vaccine record. • Schedule next dose before individual leaves the site.