National Immunization Technical Advisory Groups (NITAGs): A schema for evaluating and comparing foundation instruments and NITAG operations

The individual and community health benefits of vaccination have received significant attention and are now well understood. However, much less is known about immunization as a regulated space, its principles and standards and its institutions and instruments. In 2011, the World Health Organization (WHO) recommended that National Immunization Technical Advisory Groups (NITAGs) be established in each member country. NITAGSs are envisioned as independent, multidisciplinary expert groups within the national immunization framework, tasked with providing evidence-based evaluations and recommendations to governmental decision-makers about specific vaccines, vaccine-dosing, vaccine program development and immunization policy and practice more generally. As of 2020, 171 WHO countries have formed NITAGs. The widespread formation of NITAGs has highlighted an absence of sustained scholarship around immunization as a policy area subject to law, and it has given rise to many governance and operational questions. In 2017, for example, representatives of the Global NITAG Network (GNN) agreed that there is insufficient understanding of the impact of law on the functioning of NITAGs. Similarly, the Strategic Advisory Group of Experts on Immunization called for research into the variety of ways in which legislation and regulation have been used to promote immunization at a national level and to achieve different ends in relation to immunization and NITAG functioning. In answer to this call, the NITAG Environmental Scan (Project) was initiated. Drawing on scholarship around good governance, this article offers a comprehensive common assessment schema for critically and systematically approaching questions about NITAG governance and operation, applying that schema to the foundation instrument of the Côte d’Ivoire’s NITAG. It also reports on how well the schema is engaged by the NITAG foundation instruments in other GNN countries.


Introduction
Although the individual and community health benefits of vaccination are now well understood 1 and have been acknowledged in formal legal processes in multiple countries, 2 vaccine uptake rates are not where they need to be for adequate control of vaccine preventable diseases. 3These rates can be further eroded by unanticipated or unpreparedfor disruptive events such as the outbreak of an infectious disease for which there is yet no vaccine (e.g.COVID-19). 4Throughout the prevailing pandemic, the World Health Organization (WHO) has emphasized the human and social costs being visited upon peoples, regions, families and individuals. 5Of course, vaccines will be necessary to ultimately control the virus and permit the global community to emerge from the pandemic.However, the assessment and recommendation of COVID-19 vaccines, and of vaccines associated with routine immunization (which must continue and be revitalized despite the conditions imposed by the pandemic), must be undertaken through a reasoned and evidence-based process.
In 2011, the WHO recommended that National Immunization Technical Advisory Groups (NITAGs) be established in each member country to provide such service. 6hereafter, the Global Vaccine Action Plan (GVAP) called upon all countries to establish, or have access to, a NITAG by 2020. 7NITAGS are envisioned as independent, multidisciplinary expert groups within the national immunization framework, tasked with providing evidence-based evaluations and recommendations to governmental decision-makers about specific vaccines, vaccine-dosing, vaccine program development and immunization policy and practice more generally.It is hoped that NITAGs will increase access to vaccination worldwide, contributing to a significant decrease in vaccinepreventable illnesses.Since this call to action, many WHO states have formed NITAGs; some 171 currently report the existence of a NITAG within their jurisdiction. 8Given the unfolding COVID-19 pandemic, the creation and operation of these bodies must be viewed as a global health priority, and their performance is critically important to public health outcomes.
Despite the widespread formation of NITAGs, however, there has been a general absence of sustained scholarship around immunization as a policy area subject to law and policy, and scant scholarship on NITAGs themselves (though note some of the studies referred to below).In other words, insufficient attention has been paid to immunization as a 'regulated space', to its principles and standards and to its institutions and instruments.As such, many governance and operational questions remain underexplored or unanswered.In 2017, for example, representatives of the fledgling Global NITAG Network (GNN) agreed that there is insufficient understanding of the impact of law on the functioning of NITAGs. 9Similarly, the Strategic Advisory Group of Experts on 2020.Available at https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports (accessed 13 November 2020). 5. WHO Immunization (SAGE) called for research into the variety of ways in which legislation and regulation have been used to promote immunization at a national level and to achieve different ends in relation to immunization and NITAG functioning. 10In answer to this call, the NITAG Environmental Scan (Project) was initiated. 11n this article, after outlining the Project, we briefly outline the original process indicators that have been used to help steer contemporary NITAG design.Then, drawing on scholarship around good governance, we articulate a more comprehensive common assessment schema for critically and systematically approaching questions about NITAG governance and operation.As each of the seven elements of the assessment is explored, it is applied to the foundation instrument of the Côte d'Ivoire's NITAG, being Decree 226 Regarding the Creation, Organization, Properties and Functioning of the National Committee of Independent Experts for Vaccination of Côte d'Ivoire.This application is valuable for uncovering the approach (and the sufficiency of the approach) adopted in a typically understudied low-income African country.In the penultimate section, we report on how well the schema is engaged by the NITAG foundation instruments in other countries that were studied in the NITAG Environmental Scan, all of them GNN members.

The NITAG environmental scan
One of the primary objectives of the Project was to provide a framework for analysing NITAG-supporting law and policy, a hypothesis being that there may be a correlation between the legal status of a NITAG (i.e. the nature and content of its foundational instrument) and the effectiveness of that NITAG. 12While a detailed empirical analysis of effectiveness was beyond the scope of the Project, constructing a framework for systematically examining and comparing the instruments that found, ground and empower NITAGs was within its purview.
All GNN country members (40 as of June 2018) were invited to participate in the Project through a national representative drawn from the GNN Secretariat list; they entered an online survey (in English and French) via a password-protected portal.The survey contained three primary components, each aimed at gathering and corroborating evidence.They were (1) tick-box questions tabulated quantitatively using simple descriptive statistics, (2) free-text comments analysed qualitatively for specific jurisdictional insights and themes and (3) requests for legal and/or policy instruments (provided via URL or email).
Of the 53 legal and policy instruments collected in relation to the 28 responding GNN member countries, 18 instruments, identified in Table 1, founded or specified the organization and/or functioning of the country's NITAG (i.e. were 'foundational instruments').Identifying them was not as straightforward as one might imagine.One challenge was that not all NITAGs had a single foundational instrument; structural and operational guidance was often spread across multiple instruments.An opposite challenge was that some NITAGs had no known or locatable foundational instrument.In both cases, some had Terms of Reference which addressed function but were not technically foundational (i.e. they did not 'create' the NITAG). 13hese foundational instruments were reviewed in parallel by the legally trained authors, who then jointly categorized the instruments and the provisions within them according to evolving themes informed by elements of good governance.To understand the meaning or consequence of provisions, we applied standard approaches to statutory interpretation (i.e.we read the instrument or chapter in its entirety, interpreting words according to their grammatical and ordinary sense while remaining sensitive to the objective of the instrument). 14We also subjected individual provisions to a similar interpretative approach (i.e.we read the provision in light of the ordinary meaning of the words used and their overall reasonable implication).Where questions arose as a result of language or translation, we reached out to the national participant to seek clarity or verify understanding.
Despite the relative care that was taken in approaching the instruments, it should be made clear that little turns on the finer points of interpretation, or the idiosyncratic meaning of provisions or phrases within their originating political and legal culture.We required a relatively rough and ready interpretation of the language as expressed in, or translated into, English or French and as understood by national actors familiar with their operation.Ultimately, the instruments were examined for their general content and thrust, not the nuances of their application in fact scenarios.We were exploring what, broadly, they addressed with a view to designing a schema that could capture relevant practice matters from the perspective of good governance.

Existing NITAG self-assessment schema
Of course, the existence of a body labelled 'NITAG' does not guarantee the existence or sustained presence in that body of the capacities necessary to effectively and efficiently perform the functions envisioned for NITAGs.To help on this front, the GVAP developed six 'process indicators' pertaining to the basic functioning of a NITAG, compliance with which, it was felt, would signal that the NITAG is set up and operating in a manner 13.Note that we did consider both the Ethiopian and Swedish NITAG Terms of Reference to be foundational as they were both directly incorporated into a Ministerial Resolution.enabling reliable operation. 15Those indicators suggest that states should pay attention to, or undertake, the following in relation to their NITAG: 16 1.create a legislative or administrative basis, 2. have formal written Terms of Reference, 3. ensure the representation of diverse expertise among core members, 4. stipulate a number of meetings per year, 5. circulate an agenda and background documents at least 1 week prior to meetings and 6. require members to disclose any conflict of interest (COI).
Currently, some 123 countries self-report that their NITAGs meet all of these process indicators. 17Since publication of these original process indicators, they have been expanded, and several output and outcome indicators have been added. 18Stated aims of this expansion have been to reflect a range of stakeholder interests and to facilitate NITAGs in undertaking meaningful self-assessments of their processes and activities, enabling them to better identify challenges, shortfalls and matters needing special attention. 19The expanded process indicators are as follows: 1. Legislative/Administrative Basis (Is there a legislative or administrative basis for the NITAG?) 2. Terms of Reference (Are there formal terms of reference for the NITAG?) 3. Advisory Role (Is the NITAG's role technical advisory only?) 4. Membership (Is there a clearly defined selection process to become a core member and Chair?Are the WHO-recommended areas of expertise represented in the core membership?Are there non-core members?Are there rules for the rotation process for core members?) 5. Functioning standard operating procedures (SOPs) (Are they clearly defined and functioning?) 6.Independent Chair (Is the Chair independent from the Minister of Health and the immunization program?) 7. Meetings (How many meetings were held in each of the past 3 years?)8. Document Distribution (Were the agenda and background documents distributed and received at least 1 week in advance for each of the past three meetings?)9. COIs (Is there a COI policy?Were all core members asked to declare their interests at the beginning of each of the past 3 years and three meetings?)10.Official Requests (How many official requests for recommendations has the NITAG received from the Minister of Health and/or the immunization program?How many of them has the NITAG addressed?) While the expanded list was piloted in 14 countries in 2011, the only published selfassessment to date has been in relation to Chile, and it relied on the original GVAP process indicators. 20bviously, these process indicators are meant to provide points of inquiry and comparison across jurisdictions, with the aim of self-assessment.However, they offer minimal instruction as to how each criterion should be approached.Without some baseline expectation, the assessor will be unable to conclude with any confidence that the NITAG will successfully perform the functions of a NITAG in a consistent manner (i.e.effectively and efficiently perform its expert functions in a responsible and impactful manner).For example, the original six indicators represent a mix of high-level criteria, with two indicators inquiring after the nature of the NITAG's foundation (legislation, or administrative with written terms of reference), one addressing composition, two specifying processes and one concerned with legitimacy.The expanded indicators formulate at least one question for each indicator and add several new descriptive elements (e.g.around their advisory nature, the existence of SOPs and official requests for information from the NITAG).
However, if these are meant to assist in NITAG formation, particularly in capacitychallenged contexts, they provide little in the way of normative guidance.Indeed, with the exception of the call for COI declarations, there is little normative substance to the indicators.Thus, addressing each criterion in some way is no true indication that the NITAG will perform effectively, or even well, or that the NITAG will conform to others in substantive ways such that their structure and operation become harmonized across borders.Outstanding questions that belie any serious instructive power include the following: What does it mean to have an 'administrative base', and what is the value of having same?What, at a minimum, should the terms of reference contain?What is a sufficient diversity of experts, and how is that diversity ensured, particularly in small countries with a limited range of experts/expertise?Who determines the number of meetings, and what happens if that number is not met?What things must, as a matter of course, go on the agenda?What happens once a member discloses a COI, and what happens if it is discovered that the member failed to do so?
It is not at all our intention to dismiss the indicators as points of comparison for the emerging international NITAG community, but given the calls to better understand how law can be used to shape and facilitate this setting and to improve the operation of NITAGs, it is important to go beyond these indicators.Our intention, therefore, is to focus on what can and should be contained in a (legal) foundation instrument as envisioned but left entirely unpacked by process indicators one and two.Our findings and recommendations can then help further inform and strengthen existing NITAG evaluation tools.

A NITAG assessment schema
Our ambition is to offer criteria for critically assessing NITAG foundation instruments and NITAG operation, having regard to notions of 'good governance', which has been described as a concept, a policy and a principle, 21 but which we consider to be a morally sensitive sociopolitical imperative with obvious normative implications.Our target is therefore a framework of objectives, standards and powers that, if used to inform NITAG-founding/supporting instruments, will render them capable of delivering good governance.And good governance is absolutely essential in this setting given the strength and tactics of the anti-vaccination movement, which draws on pseudo-science and claims of poorly evidenced vaccines and collusive relations between manufacturers and immunization policy makers and service providers to encourage vaccine hesitancy and refusal in publics.The first challenge is to unpack 'good governance'.Despite its many conceptualizations, its meaning, very broadly, is the process by which state and non-state actors, through sometimes complex relationships and networks, exercise political and administrative authority to manage public affairs, adapt public activities and resolve social problems. 22So understood, it makes sense that it should reflect some of the pillars of administrative law, which arose as a result of the use of executive agencies (EAs) and non-departmental public bodies (NDPBs) to achieve policy ends. 23EAs operating under delegated powers tend to be closer and more beholden to government, which is not advisable given the claims of those in opposition to immunization.NDPBs are more arms-length, and often have a statutory foundation, though they too can be created through the executive prerogative.Empowering instruments typically address composition, staffing and hiring, services provided and powers to aid in their delivery, funding, reporting and auditing and the role of, or interactions with, the Minister. 24Concerns revolve around the process by which people are appointed, the relationship the NDPB has with its parent department, the answerability of the NDPB to the government and to the public and the means of achieving and measuring efficiency and effectiveness. 25iven the above definition of good governance and the concerns and responses of administrative law to delegated authority, and given the general desire to enhance competent policymaking, the principles or elements of good governance must include the following: 26

Accountability
This refers to holding decision-makers charged with a public mandate to account for their decisions and actions.It involves informing publics about the organization's remit and rules and demonstrating to publics through specified mechanisms that they have been met.Where they have not been met, it demands processes for remediation and the imposition of consequences.

Transparency
Closely allied to accountability, this is about ensuring that publics have knowledge about policies and objectives.It demands that processes exist for making accurate and verifiable records available for public scrutiny (and possibly feedback).Obviously, record production, management and dissemination are important to these first elements. 27nesty This is about holding actors to a high standard of conduct and ensuring that mechanisms exist to combat corruption, which is the misuse of public resources and the abuse of public trust for personal or nefarious gains (e.g.diversion of resources, bribery, etc.).

Accessibility
This necessitates opening up processes and empowering publics to engage with organizations or to more directly participate in decision-making or oversight so they can help identify needs and shape solutions.At base, the organization must be in some way accessible to the different communities that it serves.

Proportionality
This suggests the rule of law (i.e.supremacy of law, absence of arbitrariness and general or equal and equitable application of the law), 28 the availability of discretion (i.e. to overcome the limited capacity of rules to handle all situations) 29 and the balancing of regulatory costs and benefits (i.e. an opening of space for the exercise of common sense). 30hese elements are realized through lower-level rules and practices, and the law clearly has a role to play in defining and pursuing both the elements and their associated rules and practices in any given policy field, as well as erecting processes to ensure some level of compliance with them. 31n light of these elements, we suggest a seven-criteria schema for designing and/or assessing a legislative foundation for NITAGs as summarized in Table 2 and explained and justified in the following section.The goal of the schema is not to privilege a particular method of NITAG foundation (other than to suggest that a clear foundation instrument is favourable).It permits a wide variety of instrument types to be used and can be deployed to compare and critique any of those instruments.However, although rules and practices may vary over time and according to political culture, the schema offers normatively influenced 'ideal' content for each criterion.In this way, it offers a value-informed framework for evaluating and comparing foundation instruments, and we hope it encourages legislators and policymakers to approach their NITAG design and reform in a considered and systematic way.

The schema unpacked and applied to the Ivory Coast
In this section, we provide the 'ideal' content of each criterion and consider a NITAG from our study in relation to them, namely the NITAG of the Côte d'Ivoire, the Comité National d'Experts Indépendants pour la Vaccination et les vaccins de la Côte d'Ivoire (CNEIV-CI).The CNEIV-CI was founded in 2009 in response to the call to strengthen national capacity for evidence-based immunization policy decisions. 32Indeed, it was the first Gavi Alliance-eligible country in sub-Saharan Africa to set up a NITAG, doing so with support from the Supporting Independent Immunization and Vaccine Advisory Committees (SIVAC) Initiative. 33The CNEIV-CI is a reasonable case study because its foundation is typical of our surveyed NITAGs insofar as it is relatively new and was created by an executive instrument (i.e. a non-statutory but distinct and bespoke policy instrument, being Decree 226/2009). 34It is also justified by the general dearth of legal attention paid to low-income country regulatory practices and initiatives generally and to African efforts specifically.

Criterion 1: Broad remit and values
Articulation of a remit in general terms is useful not only for identifying the field of activity, policy objectives and primary role of the organization but also for setting the character of the organization and aiding in assessments of legitimacy and performance.For similar reasons, statements of values are salutary.Values are deeply held ideas or morally informed concepts which contribute to social culture, personal identity and institutional character.As has been observed, 35 values are often hidden both in policy debates, and in political institutions and legal instruments, either because some values are so deeply ingrained that they appear to require no articulation or because society or institutions lack frameworks to meaningfully discuss them. 36Clarity of value stance in value-laden fields like health and immunization can be incredibly useful; they provide platforms through which policy aims can be discussed, and they provide measures against which policy actions can be justified. 37AGE advises that the primary roles of NITAGs should be to collect, review, assess and organize scientific evidence on specific vaccine-related topics and make recommendations to national health authorities that take into account the local epidemiologic and social contexts. 38It also states that individual countries should adjust the specific roles and responsibilities of their NITAG to suit their needs and resources.This falls short of calling for value clarity, but it does allude to certain values.For example, evidence generation, reliance on scientific evidence in policymaking and objective and informed assessments are all advocated, which suggests substantive and procedural values such as transparency, honesty and accessibility.Researchers have suggested that the role of NITAGs could be broadened to include developing guidance on vaccine introduction and cold chain operation, addressing 'bottlenecks' in program delivery and conducting research. 39It has also been suggested that the power to revise national immunization schedules and mobilize funding might also be warranted. 40Of course, any powers enumerated (more on which below) should be in keeping with the character of the organization, and it is here, again, that clarity around remit and values is important.
Decree 226/2009 addresses remit by specifying that Côte d'Ivoire's CNEIV-CI will make scientific and technical recommendations to the Minister to help him or her make strategies and decisions with respect to vaccination and vaccines. 41It states that the CNEIV-CI will work in collaboration with relevant actors within the Ministry 42 and that the CNEIV-Cl is to advise the Minister on strategies for monitoring vaccine-preventable diseases and on data and information to be collected for decision-making in the field of immunization, inform the Minister about the latest scientific developments that have taken place in the field of immunization, and establish partnerships with other national or international committees of experts on vaccination. 43Combined, these make clear that the CNEIV-CI is a body under the Ministry of Health, with duties to collaborate and report, and that it is responsible for issuing opinions and recommendations only; final decisions rest with the Minister of Health.Although not explicitly stated, one clear operating value is collaboration, as evidenced by the instruction to work across the Ministry and establish beneficial partnerships.

Criterion 2: Composition
The second element of the schema relates to NITAG composition (i.e. the make-up and expertise of members).The number and diversity of members, their training, knowledge and experience, clearly play a critical role in the functioning of organizations. 44Vaccine quality, safety, utility and cost-effectiveness assessments require complex and technical analyses drawing on data from multiple fields.While some countries may struggle to provide the ideal breadth of experts, 45 individuals with training and experience in diverse fields of basic science and clinical science, as well as in epidemiology, healthcare systems administration/governance, economics and statistics should be sought.This range would undoubtedly have an advantage in processing and interpreting evidence and making recommendations over those more narrowly composed. 46Where the ideal or a sufficient range of experts is simply not within a country's capability, solutions include forming a collaborative, cross-jurisdictional or shared NITAG, as has been done in the Caribbean, 47 and by utilizing evidence-based advice from SAGE and other established NITAGS that can then be adjusted for local context.And, of course, the disciplinary experts will benefit immensely from administrative and operational support from a bespoke and responsible secretariat.
Given the above, we contend that foundation instruments should, at a minimum, address the following: With respect to disciplinary fields and experience, jurisdictions should seek to ensure that the composition of their NITAG reflects knowledge relevant to their particular national disease burdens, healthcare strengths and social priorities.
In Côte d'Ivoire, the CNEIV-CI's foundation instrument identifies three categories of members (Core, Ex Officio and Liaison) 48 and establishes a supporting Secretariat, 49 which is located in the Vaccinology Department of the National Institute of Public Hygiene (INHP).Core Members -who are appointed by the Minister on the recommendation of the Director General of Health -are responsible for forming opinions and making recommendations to the Minister, and they serve 4-year terms that are renewable indefinitely. 50Ex Officio Members, also appointed by the Minister, provide additional expertise and support to Core Members. 51Liaison Members represent other relevant bodies, including the Minister of Finance, Minister of the Interior, Director of Occupational Health and Director of the Armed Forces Health Branch, and local branches of international bodies such as the WHO, UNICEF, the Preventive Medicines Agency. 52NEIV-CI has 17 Core Members, 12 Ex Officio Members and 3 Liaison Members, with 11 specialties in the field of immunization and vaccines represented, 53 though none of these numbers or specialties are stipulated by the foundation instrument.

Criterion 3: Member integrity and independence
To provide objective advice about information needs, disease burdens, vaccine quality and value and appropriate delivery, members need to be free to form independent judgements based on multiple sources of evidence, and they need to be free from COIs that would cloud those judgements.Even the mere perception of subservience to a master with an agenda, or a body with a vested interest in the outcome of a decision relating to a particular vaccine, may be damaging to the reputation of the NITAG and its ability to perform its functions.In the vaccine setting, there are two fundamental sources of integrity compromise: ties to industry and membership in government.NITAG members who have direct ties to vaccine-manufacturing and -marketing pharmaceutical companies (e.g.members who are employed by such companies) or who have other indirect or financial interests vested in pharmaceutical companies (e.g.members who own shares in such companies) may not be able to neutrally or objectively analyse the evidence before them.Such members may -consciously or unconsciously -interpret the evidence before the NITAG in a manner beneficial to the company (and themselves). 54imilarly, NITAG members who have direct existing ties to the government (e.g.members who are employed by the Ministry of Health) may have compromised objectivity and may, again unconsciously, interpret the evidence before the NITAG in a manner favourable to the broader aims and perspective of the government.Even if members with ties to industry or government were able to analyse the evidence objectively, the existence of a close or subservient relationship with a party that may benefit from NITAG decisions might erode the public perception of the NITAG's objectivity, and so the public's confidence in the NITAG.This would almost certainly undermine NITAG standing and authority and offer ammunition to the unscrupulous anti-vaccination movement.
Given the critical nature of this issue, foundation instruments should address the following with respect to core members: manner of applying to become a core technical member, broad criteria for assessing the suitability or merit of applicants, manner of appointment (e.g. who appoints, on whose recommendation, etc.), nature of compensation or cost-reimbursement, term of membership, reappointment and maximum tenure, oversight (e.g. who has responsibility for monitoring participation/contribution, providing reviews and making recommendations for removal), process for removal, which should include the possibility of reply and review, COI (e.g.definition of conflicts, process for raising conflicts, process for handling conflict allegations, process for transparently reporting findings).
In addition, as expressed in the original process indicators, it should be standard for core members to sign COI statements at the time of appointment, to commit to declaring any potential COI on an ongoing basis as they may arise, to reaffirm the absence of any COIs at each NITAG meeting and, should a conflict arise, to refrain from participating in (or even being present for) discussions and votes involving the conflict.This close 54.The key is that the interest must be declared and then reviewed transparently and independently, not by the individual him or herself.In this regard, SAGE adopts a nuanced approach that takes into account the individual's relationship, the pharma company's market emphasis and the product under review: SAGE, Assessments of Declarations of Interest for the SAGE (2015).Available at: https://www.who.int/immunization/policy/sage/SAGE_DoI_SOPs.pdf?ua¼1 (accessed 21 May 2020).
attention to COIs is particularly important as an aid to combating vaccine hesitancy, which is in part driven by perceptions of proximity between industry and the public health and immunization community and undue influence of the former over the latter with the effect that the evidence base for many vaccines is questioned. 55ôte d'Ivoire's Decree 226/2009 offers no specific instruction on the process of Member identification and appointment, except that they are recommended by the Director General of Health.It is also silent as to performance measures, although a member may be removed by the Minister for an absence from three meetings, the existence of a major COI or repeated failure to disclose a COI. 56On the matter of conflicts, members are required to: 57 sign a Declaration of Confidentiality and absence of Conflicts of Interest, declare to the Chair of the CNEIV-CI any COI that may arise and refrain from participating in discussions on matters for which the member may have a COI.
Members are considered volunteers, and they receive no remuneration from the government other than travel and subsistence expenses related to attending meetings. 58ttendance has been identified as an issue, with the result that an increase in travel allowances has been recommended. 59It has also been recommended that member capacities should be strengthened. 60

Criterion 4: Organization independence
Complete and formal independence of administrative bodies is not always necessary or possible, nor is it always sufficient for achieving good outcomes.Nonetheless, it is generally agreed that regulatory actors and advisors, particularly in settings characterized by large infrastructure investments, are most effective when they are independent of government political and policy cycles. 61Similarly, they need to be independent from the industry on which they exert their influence.Independence, then, implies independence to acquire specialized skills, to manage without interference and to be accountable for results according to specified (and justified) performance criteria. 62Measuring independence is difficult, so actors often use proxies, some of which are contained in criteria 2 and 3 (e.g.membership terms, removal processes, conflicts rules and practices).Further factors include the NITAG being able to: accelerate its meeting schedule if circumstances are deemed appropriate by the members, set its own agenda (i.e.act on its own initiative absent a request/order from government), identify and seek out its own sources of information and evidence, identify outcomes considered critical, report publicly on its deliberations, findings and recommendations and reach out to other NITAGs and relevant international bodies.
In addition, and to avoid capture by non-governmental interests, a multi-member structure should be adopted whereby different communities of interest are represented in some way in the organization. 63n Côte d'Ivoire, the Chair and Vice-Chair are Core Members nominated by the Minister on the advice of the Director General of Health. 64While this could provide the basis for some subservience to the government insofar as the NITAG is not responsible for identifying and choosing its own leaders, this is not an uncommon administrative practice.Nothing else is said about member appointment.On fiscal considerations, the SIVAC Initiative supported most of the CNEIV-CI's costs during its first years, after which there was a gradual handover of budget items to the INHP. 65Now, the resources necessary for the functioning of the CNEIV-CI are drawn from that body's budget. 66

Criterion 5: Practice procedures
The fifth element of the schema relates to SOPs that shape organization conduct and decision-making and that encourage organizations to function in predictable and organized ways. 67The WHO has identified a lack of NITAG SOPs, particularly in the African, Western Pacific and Southeast Asian WHO regions, as a challenge to their effective functioning. 68bviously, care must be taken with respect to the articulation of SOPs in foundation instruments; unlike overarching values and operational principles, SOPs might be expected to change over time (and quite conceivably to change quickly).However, clarity around certain operational practices can improve functioning without undermining independence or flexibility.While there will not be many SOPs that ought to be addressed in a durable foundation instrument, that instrument can and should articulate the matters that are demanding of SOPs and may contain some minimal standards in that regard.For example, the original process indicators require NITAGs to meet at least once yearly and to circulate the agenda and background documents 1 week prior to meetings.Such instruction is useful but insufficient for effecting good governance.Foundation instruments might additionally address the minimum notice for calling an ordinary meeting, the time-period within which reports and recommendations are to be tendered/published and, importantly, who has the authority to trigger NITAG action.
Given the demands of good governance, attention to SOPs relating to public engagement, including lay input into the management or governance of the NITAG, is also advisable.It has been observed that accountability is enhanced when a body is open to public scrutiny and can be called on to answer the public's questions (particularly in relation to operational failure, assuming measures of success and failure are articulated). 69The direct inclusion of the public in some evidence-providing and/or oversight role is also in keeping with notions of 'responsible research and innovation' (RRI), which has gained considerable policy traction in Europe, 70 though careful consideration of such involvement is warranted. 71A key aim of RRI is to deepen the relationship between science and society and reinforce public confidence in science by fostering informed engagement between regulators, researchers and civil society. 72In addition to advancing accountability, transparency, and accessibility, this bestows greater anticipatory power on regulatory and advisory bodies. 73 further (and related) area of importance to the effective functioning of organizations is that of knowledge transmission and reception.Strategies and SOPs around knowledge exchange (KE), particularly in organizations occupying dynamic, knowledge-based settings, are important for both improving performance and for sustaining high-performance. 74There must be clear processes for moving knowledge into organizations, sharing knowledge within organizations and disseminating knowledge outwards across organizations.This also aids in organizational memory so that newer members quickly benefit from accumulated knowledge and experience.However, knowledge-transfer does not automatically occur within organizations, and there are numerous examples of KE failure. 75It is useful for foundation instruments to address processes for, or expectations around, both intra-organizational KE (e.g.effective mentoring, meaningful internal communications) and inter-organizational KE (i.e.meaningful connections and interactions with external actors and networks).In this regard, it has been reported that NITAG effectiveness is measurably improved when there exist individual capacity building and training materials and opportunities to visit other NITAGs. 76nder Decree 226/2009, normal meetings of the CNEIV-CI are convened by the Chair every 3 months, though extraordinary meetings can be called in response to unexpected or emerging issues. 77In its early years of operation, meetings were missed due to political crises, and ensuring regular Member attendance has been flagged as an ongoing challenge. 78The Chair (in consultation with Core Members) sets the agenda and manages debates, ensuring that all agenda matters are covered. 79Quorum is half-plusone of the Core Members, and an absence of quorum requires the CNEIV-CI to reconvene within 15 days. 80The Chair can form special working groups on specific technical questions. 81While the CNEIV-CI has written SOPs that set out its operating processes, 82 it has been recommended that it should also elaborate clear terms of reference for its working groups and subordinate bodies. 83Advice from the CNEIV-CI (and its working groups) can be requested by Ministry of Health or the Director of the National Immunization Program.Alternatively, the CNEIV-CI can offer advice at its own instigation.CNEIV-CI opinions and recommendations are sent to the Minister within 15 days, with recommendations adopted by simple majority of the Core Members. 84inutes, reports and other documents, including debates of the CNEIV-CI, are confidential, with the duty of confidentiality extending to any person who attends CNEIV-CI meetings. 85Although it is fairly standard procedure to require confidentiality in the event that protected or otherwise confidential material is presented to an NITAG, 86 a blanket policy of confidentiality such as this is contrary to the approach of other NITAGs in the study, which publish their minutes for purposes of transparency. 87Indeed, it is well-known that publication of documentation and accessibility of meetings improves transparency and, potentially, public trust.As such, openness of meetings has rightly been advocated. 88ecree 226/2009 also makes no allowance for broader public engagement with the CNEIV-CI.In addition to representing a potential shortcoming with respect to governance, this issue is relevant to the challenge of generating country-specific data (i.e. it has been recommended that greater funds be made available to conduct studies that generate local data). 89The foundation instrument also makes little reference to processes for internal or external knowledge exchange other than the above-noted references to transmitting reports.
Criterion 6: Powers re: evidence and policy NITAGs are (meant to be) creatures of evidence.They source evidence, collate and assess evidence and apply that evidence in the context of existing (and possible) immunization strategies and country conditions.They can be effective only if they are empowered to access and use the best available scientific evidence.On this point, an empirical study of 11 jurisdictions found the following: Though crucial for the quality of NITAG processes, the capacity to collect, synthesize, and interpret scientific evidence reportedly varied considerably.Interviewees noted that NITAG members often did not have sufficient time to engage thoroughly with evidence collection, review, and interpretation.Additionally, some NITAG members reported difficulties in accessing literature, with paywalls and publication language reported as barriers -a finding also noted in country evaluation literature.
. . .Little documented evidence was found of NITAG literature searching and use, despite this process being essential to NITAGs' role.Of nine NITAGs for which some data were available, two appeared to conduct structured literature searches, three appeared to have detailed standardized processes to analyze evidence and develop recommendations, and four appeared to miss one or more crucial step.The limited evidence available suggests a predictable preference for use of local data, but local data might not exist, might rely on personal relationships that members have with universities and hospitals and might be in formats that hinder utility.
Given the above, it is advisable for foundation instruments to address in some way evidence generation and use (the finer details of which one would expect to be articulated in SOPs).Aspects addressed might include the power to: direct certain bodies (hospitals, pharmacists, etc.) to comply with requests for data, direct public authorities to collect and furnish local or nationwide data, access data from national vaccine surveillance systems (and contribute to their development or improvement), identify and access sources of data when relevant evidence is scant, 91 access any and all relevant (proprietary) research data from vaccine developers. 92th respect to the final point, it is noted that most post-market regulatory action relating to drugs stems from passive or spontaneous reporting systems, which have limitations (e.g.uneven reporting of events compared to frequency, quality of data reported, etc.). 93Ultimately, enumerated powers to access information and generate data may be immensely valuable to an NITAG.
In addition, this criterion addresses formal decision-making authority in relation to immunization policy more generally, a matter which will also arise in relation to remit (above).Although the majority of NITAGs in our sample -and globally -are instructed to furnish technical opinions and policy recommendations which governments and health authorities may choose (or decline) to implement, some countries have empowered their NITAG to make final decisions in relation to some aspect(s) of immunization policy. 94It is important for foundation instruments to address any formal authority granted to the NITAG; examples of potential powers include the power to: require decisions from others within specified times (e.g.physicians or Chief Medical Officers of Health), make recommendations re amendments to the national immunization schedule generally or in limited ways (e.g.make binding recommendations to government), 91.While the first three bullets relate to country data, such data may not be available, in which case evidence will need to be sought from relevant neighbouring countries or international sources.92.This occasionally happens but is expensive.Data  make recommendations re declarations that certain aspects of the immunization schedule temporally or geographically mandatory.
Clear enumeration in this respect is much preferred to ambiguity or to allowing practices to evolve or to change in relation to the individual personalities of the Minister or NITAG Chair.
Decree 226/2009 says nothing specific about evidence or evidence generation, offering no guidance on reputable sources of evidence, or means of obtaining evidence, or funds for accessing pay-walled sources.However, the SIVAC Initiative did put on several capacity-building events for CNEIV-CI Members, which included workshops on methods of developing evidence-based recommendations (2015) and the use of economic data to support decision-making related to immunization (2016). 95With respect to authority over immunization policy, the CNEIV-CI is to provide information and advice only.Its recommendations have no binding effect, which places the CNEIV-CI squarely in the advisory (as opposed to directive) category of NITAGs. 96

Criterion 7: Review of performance
To remain effective, organizations must regularly re-examine their structure and practices. 97Periodic self-and group-reflection allows bodies and members to recognize strengths and weaknesses, identify opportunities for improvement and better enable efficiency and effectiveness. 98While approaches to performance review vary from country to country and sector to sector, some regularized review is necessary to ensure that organizations are delivering high-quality service, efficiently and in compliance with the public's and/or the associated host department's needs.Of course, objective and reliable measures of performance are challenging and often contingent, 99 and NITAG performance assessment will encounter this common phenomenon.Factors affecting NITAG impact include but are not limited to member makeup, funding and administrative support, availability of evidence, the formal authority of the NITAG as well as the economic, political, health and social climate of the NITAG's jurisdiction. 100Nevertheless, remit and objectives, actions, relationships and outputs and outcomes can be assessed on a regular basis, whether through 'landscape reviews' or 'business reviews' 101 or through other self-assessment tools, 102 modified as necessary for NITAG purposes. 103iven the above, we suggest that, at a minimum, foundation instruments should specify that regular performance reviews be undertaken in relation to NITAGs.Further, whether those reviews are expected to be rigorous or light-touch, it is also not improper that the foundation instrument instruct that such reviews reflect on the following end points: independence of the NITAG from government and industry, manner and circumstances of interacting with government, sufficiency of levers of influence re immunization policymaking, compliance of the NITAG with its own SOPs, success in (and barriers to) delivering timely evidence and advice to actors, sufficiency of expertise on the NITAG, sufficiency of resources (financial and human) to achieve the NITAG's goals, adequacy of evidence-generation in relation to achieving its technical remit, necessity of examining declined recommendations (and the reasons for same), signals of public trust in the NITAG, possible steps to increase public trust, necessity of identifying new and further NITAG objectives or end points.
With respect to the CNEIV-CI, Decree 226/2009 says nothing specific about performance review.

Summary
While Decree 226/2009 by no means addresses all criteria of the schema, or all the components of each criterion, exhibiting some important gaps that are particularly relevant to good governance, it is reasonably responsive compared to many of the other foundation instruments we received.Nonetheless, with a focus on member appointment, number of meetings and confidentiality and transmission of deliberative outputs, all in general terms, it can be characterized as wanting in relation to the elements of good governance, being accountability, transparency, honesty, accessibility and proportionality.While the CNEIV-CI has obviously been granted a level of flexibility in designing its practices, greater attention to some of these elements would be beneficial.
101.Landscape reviews focus on strategic alignment and are meant to ensure appropriate governance arrangements, institutional relationships and services targeting.Business reviews focus on target assessment and are meant to ensure that outputs are linked to objectives and measures.See Craig, note 23, at 106. 102.See Gill, Flynn and Reissing, 'The Governance Self-Assessment Checklist'.103.And the GNN has noted the importance of review, offering some tools for external and internal reviews through its NITAG Resource Centre.

The schema and other project respondents
As noted, we received some 53 instruments, 18 of which appeared to have a foundational nature and which were analysed in accordance with the above schema.We found significant variance in the extent to which and how the instruments addressed the schema criteria.Individual instruments addressed anywhere from one to all seven of the criteria, with an average of 3.4 criteria addressed per instrument, though none of them addressed every element of each criterion as articulated above (and in Table 2).Table 3 highlights the criteria that were addressed, at least to some extent, in the foundation instruments received.Overall, we would characterize this as low engagement with what we contend amounts to good governance criteria.(We acknowledge that this low rate of reflection does not necessarily speak to the actual quality of the individual NITAG's performance, a true assessment of which would demand more detailed empirical case studies.)One explanation for this low level of compliance is a possible belief by policymakers that some of the criteria are either unimportant or at least not worthy of formalization.Such is not supported by our evidence insofar as the criteria that were met and unmet were diverse, and no single criterion was consistently ignored, apart from the Performance Review criterion, which no instrument addressed.The most commonly addressed criterion is that of general immunization policy power.The least commonly addressed criterion, apart from Performance Review, is that of evidentiary power.This might be viewed as curious given the centrality of evidence to the NITAG's remit, but it might also be explained by the fact that evidence-based decision-making is the NITAG's raison d'etre, and so scope to gather and consider evidence is simply assumed.
Another explanation for the low level of compliance with the schema may be the evolutionary nature of the NITAG's history and the complexity of the legal landscape.Where NITAGs are the result of both statutory and executive instruments, or where they have both formal and informal elements (such as in the United Kingdom and United States), or simply -as alluded to above -have legal foundations that span multiple documents, it may be that different criteria are addressed in different source instruments or are met through practice or convention rather than through formal direction in one instrument.
Other possible explanations for low compliance are (1) policymakers may have wished to furnish their NITAG with greater freedom to act and respond, although compliance with these criteria does not undermine that capacity.(2) Governmental capacity or legal culture of the country at the time the NITAG was founded may have been such that a comprehensive framework was impossible.(3) Some NITAGs may have been established in haste or with support from time-limited schemes provided by organizations like SIVAC, making compliance with a comprehensive framework unlikely.In any event, and interestingly, the number of criteria addressed within a given jurisdiction does not appear to correlate to the legal authority of the NITAG.In other words, it is not the case that the greater the formal authority of the NITAG, the more criteria its foundational instrument addresses.Thus, for example, the founding instruments of advisory NITAGs address at least to some extent anywhere between one and all seven criteria.The founding instruments of directive or hybrid NITAGs address only one or two criteria, but in each case, they do address formal policy power.Although it stands to reason that such NITAGs' authority would need to be stated explicitly, it is not clear why other elements relating to good governance were not also explicitly stipulated for the more empowered NITAGs.
There was also no correlation between the type of instrument used and the number of criteria addressed.NITAGs with a statutory foundation complied with anywhere from one to seven of the criteria.For example, Germany's NITAG foundation instrument engaged with one criterion, while Nepal's engaged with five criteria.Similarly, those NITAGs with an informal executive instrument foundation (i.e.decree or policy statement) also ranged from one to seven criteria addressed.Only two countries (Argentina and Chile) had what we would consider under the schema as a comprehensive foundational instrument, in that they addressed all seven of our good governance criteria in some fashion or another.Obviously, there is scope within the criteria for quite diverse mechanisms of engagement/reflection.
Although there was no single criterion that every instrument addressed, most instruments (more than 50%) did address criterion 2 (composition), criterion 3 (member integrity) and criterion 6 (powers, most commonly in respect of policy advice).This suggests that policymakers may consider these three criteria, broadly considered, to be the most important to clarify so that a responsible body can be founded.However, they are obviously not universally viewed as essential to the foundation of an NITAG.Conversely, just six instruments addressed criterion 4 (organization independence), and five instruments addressed criterion 6 in relation to gathering and treatment of evidence.This may reflect perceptions that specifying these aspects of a NITAG are comparatively less important, and, with respect to the latter element, that such should be left to the discretion of the expert body itself.
No instrument addressed criterion 7 (performance review).This surprising omission may be explained by an assumption by governments that organizations will find ways to improve, or that other or general regulations already exist which require governmental bodies to undergo such self-assessment.Regardless of where such a requirement is located, governments should ensure that NITAGs have a mandate to assess their performance on a regular basis, not least because of the politically charged nature of this policy field.Under half of the foundational instruments (7 of 18) addressed guiding principles, meaning that specific values or principles were usually not articulated.Given that value identification can be extremely useful in signalling core objectives, shaping organizational culture and behaviour and offering measures for organizational decision-making, this is also somewhat surprising.Some recurring core principles applicable to NITAGs that were noted by respondents in free-text responses were conscientiousness and honesty, independence, transparency, expertise and professionalism, collaboration and an openness to good evidence.Beyond the principles that are meant to be embodied by the NITAGs themselves, the actions of the NITAGs are most commonly meant to achieve or work toward realizing the following core principles: social well-being, equality and fairness (i.e.non-discrimination in program delivery) and cost-effectiveness. 105

Conclusions
Given the importance of immunization to country and global public health objectives, and given the challenges of achieving sufficient vaccination rates in the face of countervailing preventive pressures, healthcare demands and anti-vaccination machinations, it is important to commit attention to, and exercise care in, the development of the policy and operational architecture relating to immunization.In other words, due attention must be paid to how NITAGs, as one important component of that architecture, are founded, structured and empowered.The schema developed here offers a framework informed by principles of good governance which should better enable foundation instrument drafting and NITAG operation, encouraging the harmonization of these instruments.In addition, it should facilitate the assessment and comparison of NITAGs.In short, though conceived as a policy tool to facilitate the shaping of effective NITAG foundation instruments, it is equally useful for assessing and comparing those instruments and for critiquing NITAG performance.As such, it could prove useful to jurisdictions developing or revisiting their NITAG and/or their NITAG's foundational instrument(s).The practical effect of full and robust compliance with our assessment schema remains to be seen, but it provides a means for investigating whether NITAGs with more comprehensive legal foundations differ significantly in their functioning, influence or perceptions of authority and offers a route for developing 'best practice' examples to countries that are developing instruments or processes in support of their NITAG.

Table 1 .
Respondent countries by type of foundation instrument.
14. R. Sullivan, Sullivan on the Construction of Statutes.6th ed.(New York: LexisNexis, 2014); Re Rizzo & Rizzo Shoes Ltd., [1998] 1 SCR 27.Chile Executive instrument Ministry of Health Decree 2028 Creating Advisory Committee on Vaccines and Strategies of Immunization (Spanish) China Executive instrument Opinions of the General Office of the State council on Further Strengthening Vaccine Circulation and Vaccination Management (Chinese) C ôte d'Ivoire Executive instrument Decree 226 Regarding the Creation, Organisation, Properties and Functioning of the National Committee of Independent Experts for Vaccination of C ôte d'Ivoire (French) Ethiopia Executive instrument Ethiopian NITAG Terms of Reference (English) Germany Statute Act on the Reform of the Communicable Diseases Law (German) (Reviewed semi-official English translation) Indonesia Executive instrument Decrees of ITAGI: 2006, 2010, 2013, 2015 (Indonesian) Jordan No foundational instrument received/located Kazakhstan Executive instrument 2012 Decree 116, On the Establishment of the National Advisory committee on Immunization (Unable to obtain entire Decree) Latvia Executive instrument Statute of the State Council of Immunization, Regulation 07-10-2015 No. J & A/M9 (Latvian)

Table 2 .
Foundation instrument assessment schema.
32.For more on the CNEIV-CI's formation, see Agence de Medecine Preventive, 'AMP Supports the Creation of a National Immunization Technical Advisory Group in Cote Vaccine 30 (2012), pp.2588-2593.33.The SIVAC Initiative (2008-2017), or Supporting Independent Immunization and Vaccine Advisory Committees Initiative, provides technical support for establishing and strengthening NITAGs in low and middle-income countries.It was funded primarily by the Bill & Melinda Gates Foundation, through the Agence de Médecine Préventive in collaboration with the International Vaccine Institute.Major SIVAC activities include advocacy, direct technical and financial support to countries and development of training materials.See K. Senouci, J. Blau, B. Nyambat, et al., 'The Supporting Independent Immunization and Vaccine Advisory Committees (SIVAC) Initiative: A Country-driven, Multi-partner Program to Support Evidence-based Decision Making', Vaccine 28 (2010), pp.A26-A30.34.Ministry of Health Decree No. 226 on the Organization, Attributes and Functioning of the National Committee of Independent Experts for Vaccination and Vaccines of the Ivory Coast, 16 December 2009, issued by then-Minister of Public Health, Dr Rémi Allah-Kouadio (Decree 226/2009).35.S. Harmon, 'Solidarity: A (New) Ethic for Global Health Policy', Health Care Analysis 14 (2006), pp.215-236.
Alves, A. Macedo and F. Marques, 'Sources of Information used by Regulatory Agencies on the Generation of Drug Safety Alerts', European Journal of Clinical Pharmacology 69 (2013), pp.2083-2094.94.In this respect, Germany and Sri Lanka are worthy of note.