Piloting an STI POC test involves steps to setup the pilot, processes to manage the pilot and inbuilt measures to monitor and evaluate the pilot. The support of the Ministry of Health is critical to the success of piloting a POC test.

2.1 Establish or revitalise a technical working group

The process of introducing STI POC testing is likely to need the oversight of an interdisciplinary technical working group that includes members with a diverse range of experience in STI control and prevention, health communications and behaviour change, laboratory and essential drugs management, health worker training, monitoring and evaluation, finance and accounting, and procurement and logistics. The members of the technical working group may need to come from a range of different organisations.

Key questions include:

Key Questions	Related Questions	Why ask the Qs?
Is there an existing working group (e.g. an STI working group at national level) that could be responsible for leading and coordinating the pilot of the POC test?	Do you have the requisite skills and experience available to form an interdisciplinary working group? What is missing? Who should be involved as core members? As associate members? How will responsibilities be defined and assigned? Can regional experts be drawn on to provide advice to the national technical working group?	Ensure that there the working group has sufficient breadth and depth of experience and technical know-how

Key activities in establishing a system of oversight for piloting an STI POC test:

• Determine the mix of skills and experience required for a functional technical working group
• Define leadership, advisory and decision making processes - develop terms of reference, elect a chair, agree on who is responsible for what, set timelines and schedules for meetings
• Develop links with national and regional individuals and organisations to draw on additional technical assistance as required – for example, professional societies, STI Regional Working Group for the Pacific, SPC, WHO, UNAIDS, UNICEF, UNFPA, The Sexually Transmitted Diseases Diagnostics Initiative (SDI) etc.

Key sources of information include:

Local

• Ministry of health
• Professional societies
• Laboratory specialists
• Relevant non-governmental organizations

Regional

• Dr Sophaganine Ty Ali, Chair of Regional STI Technical Working Group, STI Advisor & Clinical Support Cluster Coordinator, HIV and STI Section, Public Health Programme, Secretariat of the Pacific Community, sophaganinea@spc.int

Documents and materials

• Team Building Tool (2007). World Health Organization. Available at: who.int/entity/cancer/modules/Team building.pdf

2.2 Align the design of the pilot with the policies & legislation

STI testing is just one part of a comprehensive STI control strategy. Consequently, it is important that POC tests are integrated into existing PICT responses to STIs. This means that introduction of POC testing should happen with recognition of existing strategy documents, policies, protocols, legislation, guidelines and programs.

During the initial assessment, there should have been a review of the national policy and legislative context to understand the extent to which it will enable - or serve as a barrier to - the introduction of POC testing. For example, if nurses cannot legally prescribe drugs this could prevent the use of POC tests in remote services staffed only by nurses. The technical working group should lead efforts to amend policies or legislation as required. This will need to occur prior to the introduction of the POC test. Similarly, the working group should know about any local requirements for validation and accreditation of a new test.

Key questions include:

Key Questions	Related Questions	Why ask the Qs?
Has an initial assessment been conducted?	Can you describe why the POC test is needed, where it should be employed, and what difference it will make?	Ensure that there is a sound case for introduction of the POC test
Are there aspects of policy, strategy or legislation that impact on POC testing for STIs?	Is a broad range of practitioners able to offer STI testing and treatment? Can this be introduced through existing services such as antenatal clinics or other reproductive health services? Do policies and guidelines support nurses and health workers in a range of health service settings to offer STI (or HIV) testing and treatment? Are there specific national policies regarding procurement of test kits, related consumables and medications to treat STIs? Are there any policies or legislation that will need to be amended to enable the introduction of POC testing?	Ensure that POC test piloting does not proceed without first addressing
Is there a need for advocacy before proceeding?	Is there political and institutional support? Do you know how much funding is needed to pilot the test? Are resources available to pilot the test and will resources be available to introduce it on a wider scale? Whose support do you need? What messages will they listen to? How are these messages best delivered? Have you gathered credible evidence and made a case for the introduction of a POC test? Is it cost-effective? Are clients informed and supportive? Has this been done in a sensitive, acceptable way? Does the technical working group have the skills, experience and relationships to undertake advocacy?	Ensure that there is high level support

Key activities to ensure that the introduction of an STI POC test is aligned with national and where appropriate regional and international policies, strategies and legislation include:

• Review relevant findings from the initial assessment – the impact of local policies and legislation on the piloting of an STI POC test (and the wider application of the test if the pilot is successful); existing national strategies around STI testing, treatment and prevention that the pilot will need to be aligned with; the medical justification for the test should be clear; an appropriate test should have been selected for the intended population based on ASSURED criteria criteria and on the basis of any local requirements for validation and accreditation; and the benefits and risks of introducing the test should have been balanced.
• Determine if there are legislative or policy barriers that will need to be overcome and make plans to address these - there may be a need for the technical working group to identify the persons responsible for the policy/legislation as well as the people impacted by the policy/ legislation, and to then develop plans to advocate for the piloting of the STI POC test. Such planning may include the development of key messages and policy briefs, forging alliances and relationships, lobbying and negotiating.
• Ensure that there is high level support for a policy of introducing a new POC test and that the pilot will provide the right kind of information to influence policy – an advocacy package may need to be developed and could include information such as the local epidemiology of STIs; the limits of current domestic approaches to STI control; the added benefits of POC testing over existing STI control measures; expected outcomes of POC tests such as the impact on the burden of disease, the number of cases, complications, or deaths prevented, or costs saved; potential risks; the human and monetary costs; and the feasibility of implementation.

Key sources of information include:

Local

• Ministry of Health staff including the legal officer
• Professional societies
• Relevant non-governmental organizations
• Civil society and community representatives

Regional

• STI Regional Working Group for the Pacific
• Secretariat of the Pacific Community, HIV & STI section (spc.int/hiv/)
• Pacific Regional Rights Resource Team (RRRT) (http://www.spc.int/en/our-work/education-training-and-human-development/pacific-regional-rights-resources-team/241-pacific-regional-rights-resource-team-rrrt-.html)
• World Health Organization Western Pacific Regional Office, HIV/AIDS & STI programme (wpro.who.int/sites/hsi/)

Documents and materials

• Examples of relevant STI/HIV strategies
  • WHO, Global strategy for the prevention and control of sexually transmitted infections: 2006-2015
  • The Pacific Regional Strategy on HIV and other STIs: 2009-2013
  • National HIV /STI Strategic Plans
• Examples of relevant Reproductive health and related policies and protocols
  • WHO, et al., Asia-Pacific Operational Framework for Linking HIV/STI Services with Reproductive, Adolescent, Maternal, Newborn and Child Health Services. 2008.
  • WHO and CDC guidelines/frameworks on PMTCT
  • VCCT protocols
  • National reproductive health policies
• Examples of relevant STI/HIV management protocols and guidelines
  • WHO, Guidelines for the management of sexually transmitted infections. 2003: Geneva. WHO, who.int/hiv/pub/sti/pub6/en/
  • Sexually Transmitted Diseases treatment guidelines 2006, Centres for Disease Control and Prevention, MMWR 2006. cdc.gov/std/treatment/
  • LSHTM (2011). The Rapid Syphilis Test Toolkit: A Guide to Planning, Management and Implementation. http://www.lshtm.ac.uk/itd/crd/research/rapidsyphilistoolkit/ – The Planning section provides advice on how to develop advocacy and communications strategies including lobbying strategies, letters and petitions, logos and slogans, posters and leaflets, and use of the media
  • Oceania Society for Sexual Health and HIV Medicine: Recommendations for HIV Medicine and Sexual Health Care in Pacific Small Island Countries and Territories Second Edition September 2008. (osshhm.org/)
  • National STI/HIV management protocols

2.3 Identify stakeholders & establish coordination & governance mechanisms

The technical working group will need to consider the different groups who should be aware of the coming introduction of the STI POC test. The Ministry of Health will need to be informed of the reasons for introducing POC testing and the many implications. The support of political leaders, religious leaders, traditional leaders, managers of health facilities, professional and academic societies, and other stakeholders is important, and may be a necessary step to gaining the support of the community. Once the various stakeholders have been identified, it is important that the working group establishes ways to communicate progress and other information to stakeholders on a regular basis.

Key questions to ask:

• Who are the potential stakeholders in your country? Consider whether you need to do a formal stakeholder analysis listing how they can influence the pilot project, and how important their influence is. Sometimes documenting a formal plan of how you will manage stakeholders can be useful.
• What is the simplest, most effective way to reach the identified groups? For example, a joint meeting or a series of smaller consultations.
• Is there a need to engage an advisory group to provide specific advice, technical expertise or relay information to and from stakeholders?
• How might you communicate throughout the piloting of the POC test? For example, through periodic newsletters or stakeholder forums.
• Should there be some written information to support your discussions? Should other measures be used to engage the audience?
• What organizational measures need to be implemented to facilitate coordination and oversight?

Key activities

• Identify stakeholders who will need to be involved in decision making, implementation of the pilot, or who will need to be informed – potential stakeholders include representatives from relevant government departments including senior policy makers (Ministry of Education, Health, Social Welfare), traditional leaders, vulnerable and affected groups, Faith-Based Organizations, NGOs, national research institutes, medical colleges and teaching institutes, professional societies, the media, the private sector, and relevant external actors such as development partners and donors.
• Identify a representative for each stakeholder group and make contact with them – designate who will make contact with the stakeholder from the technical working group or decide if members of an advisory group can fulfil this function.
• Develop communication plans to engage stakeholders and share information with them on an ongoing basis – define the objectives of your communication plan and develop a covering email or letter and a package of key messages to share with the identified representatives. Bear in mind that different stakeholders may need different information and may best be reached through different channels. Sources of specialist advice may need to be drawn on if this is not available within the technical working group. Discuss options for ongoing communication and agree on one or two mechanisms. Consider the need for communications beyond immediate stakeholders. (See 2.7 Develop tools for implementation – Communications strategy for further information)
• Establish coordination and governance mechanisms to ensure that the pilot of the POC test is well implemented, monitored, and evaluated - implement appropriate and authoritative policy and regulatory frameworks for your pilot and ensure that roles, accountabilities and responsibilities are clear. For instance, coordination and governance mechanisms will need to ensure that testing algorithms based on scientific principles are defined and communicated, training tools and standard operating procedures are developed and delivered, assistance is provided to end users of POC tests, and quality management systems are instituted. Sometimes contracts will need to be drawn up between the team managing the project and other groups implementing parts of the pilot so that everyone knows what they are doing and what standard they need to meet. A plan to manage all the contracts can be useful in this case.

  Assigning small specialist working groups to work on particular tasks is one way of organising the implementation of the pilot test. Decide if an advisory group(s) would be beneficial to the pilot test to provide specific technical advice.

Key sources of information include:

Local

• Ministry of Health staff - program managers and STI/HIV coordinators
• Hospital medical staff - head of the medical department, clinicians, paediatricians, obstetricians, reproductive health personnel, laboratory and pharmacy personnel
• Professional societies
• Relevant non-governmental organizations
• Civil society and community representatives

Regional

• STI Regional Working Group for the Pacific
• Secretariat of the Pacific Community, HIV & STI section (spc.int/hiv/) – NB: SPC have a health communications advisor
• World Health Organization Western Pacific Regional Office, HIV/AIDS & STI programme (wpro.who.int/sites/hsi/)

2.4 Define the population to be tested & the purpose for testing

The initial assessment will provide the technical working group with an idea of where the introduction of the STI POC test could improve outcomes for patients and for the health system. Based on this information, the technical working group will need to define who will be tested and why.

Key questions to ask:

• Who are the group to be tested?

  The group will need to be clearly defined so that the performance of the test can be assessed for this specific group. The review of local epidemiology will provide information to guide who should be tested. Demographic features may define the group (e.g. age, sex, occupation) or behaviours (e.g. men having sex with men) or attendance at types of health services (e.g. antenatal clinics, sexual health clinics etc.). Targeting testing to selected groups allows for the detection of the majority of infections in the population without the costs of testing everyone. Geographic location may also be important to consider e.g. you may want to test the same group of people in different sites. In general, the following groups are considered to be at higher risk of STIs throughout the Pacific, although local assessments may reveal other groups or different age ranges:

  • Young men and women aged between 15 and 30 years
  • People who exchange sex for money or goods
  • Men who have sex with men; people who inject drugs
  • People who travel away from home to work (e.g. seafarers, loggers) In some situations, tests are offered to all individuals within a particular group, for example all pregnant women because the consequences of missing an infection are severe particularly for the child. Tests may even be restricted to just those individuals who present with symptoms of infection (although treatment should be provided even if a person with symptoms tests negative on a POC test).
• What will be the purpose of the POC test?

  Will the test be used to diagnose symptomatic patients or asymptomatic patients? Will the result of the test influence the management of the person tested? If the diagnosis and treatment is not changed (or leads to under treatment), then testing may not be justified.

  Will a single POC test be used or will there be a series of POC tests? Will there be a follow up confirmatory test conducted in a laboratory?

  Will you use a POC test that tests for a single infection or for multiple STIs? If the test is used to diagnose more than one infection, have you considered the performance of the test for each infection?

  Will people conducting the tests consider the result with the patient's history and examination? POC tests should also not be used in isolation.

• How many sites and how many people will need to be studied to evaluate the performance of the pilot test in this population?

  Statisticians, epidemiologists and laboratory technicians may need to be consulted to ensure that the pilot has been suitably designed, will include enough people, and will be implemented with sufficient rigor to provide health managers and policy-makers with the information required to make a decision about wider application of the POC test.

Key activities

• With key stakeholders, clearly define the population group for the pilot and the purpose for testing this group with a POC test
• Discuss the plan with technical and monitoring and evaluation experts to determine how many people and sites will need to be included in the pilot

2.5 Decide on how the test will be implemented

Having decided on the population to be tested and the purpose for testing them (screening or diagnostic), decisions will need to be made about how the pilot test will be practically implemented.

Key questions:

• How can this population be accessed?
  • Will people be offered the test through an existing health service in a clinical or non-clinical setting? Will they be tested through an existing screening program? Are there obvious service delivery points for integrating POC testing?
  • Will a new service be established? (This may be impractical in many settings)
  • Or will testing be done in the community through specialised outreach services or mass screening?

  Many people at risk of STIs already access health services but not necessarily for sexual or reproductive health issues - this means that when people do access health services, it is important for them to be offered appropriate information, testing and treatment. Women, in particular, access a range of primary health services often for reproductive health issues. Similarly, some risk groups such as seafarers or cannery workers may regularly access services through primary health clinics at their workplace. The easiest, most cost effective and sustainable way to access the most people at risk of STIs for POC testing is usually to integrate testing into primary health care delivery through existing services.

  Examples of services suited to integration of STI POC testing include:

  • Reproductive health services:

    Most antenatal services are provided through outpatient clinics within or linked to regional hospitals. Routine antenatal screening for a range of health issues is an established practice; adding testing for STIs to antenatal visits should be relatively straightforward. POC testing could be integrated into one or a few of the most frequently accessed antenatal clinics or a clinic where most women in the region will be referred to at some stage of their antenatal care.

    Most countries and territories in the Pacific have established reproductive health services for women such as government and NGO clinics, clinics affiliated with the International Planned Parenthood Federation (IPPF), and adolescent health services. Women access these services for a range of reproductive health issues including contraception, cervical cancer screening (Pap smear test), and termination of pregnancy. Testing can be offered to women without a significant increase in resources when they present for reproductive issues and staff are usually experienced in discussing sensitive health issues with patients. Consequently, integrating testing for STIs into consultations for young women should be reasonably straightforward.

    Young men also use reproductive health service in some settings. Testing can be offered to them when they present, however, it must be recognized that this strategy alone is unlikely to reach the majority of men at risk of STIs.

  • Sexual health services:

    Some countries and territories in the Pacific may have stand-alone sexual health services (perhaps catering for groups at high risk of STIs such as sex workers or men who have sex with men). These services offer an ideal entry point for STI/POC testing.

  • Youth services (15-30 year olds):

    A range of clinical and non-clinical services specifically targeting young people can be found throughout Pacific Island countries and territories. Young women commonly access such services for reproductive health concerns though young men are unlikely to attend youth clinics for these reasons. Testing can be integrated into reproductive health consultations for young women but may need to be offered opportunistically to ensure testing of young men. In some places, youth utilise youth services for non-clinical purposes (e.g. sporting facilities and peer education). Collaboration between clinical and non-clinical staff and services can be a valuable means of increasing youth access to clinical services and STI POC testing.

  • Primary health care services:

    Primary health care services may offer a way of testing men who, in general, present to primary health care services less frequently than women and rarely for reproductive health issues. Because of these patterns of service utilisation, STI testing has been offered proactively to all men attending primary health care clinics regardless of the reason they are attending the service (opportunistic testing). This improves the coverage of testing of men and enables detection of STIs in men who do not have any symptoms.

    Implementation will be easier in clinics that offer health screening for other heath issues. For example if cannery workers or seafarers are offered annual screening for diabetes, hypertension or hepatitis, testing for STIs could be integrated into screening in a routine way that normalises testing and improves acceptance. (When integrating testing for STIs in private clinics provided by companies to employees, care needs to be taken that STI testing and treatment is confidential and that results are not used to discriminate against employees.)

  The table below lists some options for integrating STI POC testing into existing services in order to improve STI testing for a range of subpopulations.

  Subpopulation		Services accessed	Options for integrating POC testing
  Women	Pregnant women	Antenatal services: Primary health care clinics ANC clinics attached to public hospitals	Integrate into existing antenatal screening 1st visit 3rd trimester
  	Aged 15-30 years (+/- Men accessing these services)	Reproductive health services Government IPPF affiliated Adolescent reproductive health clinics Other NGO	Integrate into routine reproductive health visits e.g. Cervical (Pap smear) screening Contraception Provision of HPV vaccine Referral for TOP
  Young people	Aged 15-30 years	Youth clinics (Government, NGO) School clinics	Proactively offer testing when youth access services for any reason (opportunistic screening)
  Specific risk groups	Men who have sex with men, sex workers, people with symptoms of or at risk of STIs	Sexual health clinics (specialized or general clinics accessed by people at risk of STIs due to factors such as staff or location)	Integrate into routine visits Testing proactively offered when people present for any reason (opportunistic screening)
  	Other risk groups (e.g. specific occupations such as loggers, cannery workers, seafarers)	Clinics may specifically target groups such as itinerant workers or students for primary health care rather than for sexual health e.g. private clinics at maritime training centres, canneries etc.	Proactively offer testing when people present for any reason (opportunistic screening)
  All important subpopulations		Primary health care services Public clinics Private GPs	Testing could be implemented at key clinics accessed by risk groups with proactive testing offered (e.g. to 15-30 year olds)
  		Hospital Emergency Departments	Test when people present with symptoms of an STI

  For some risk groups, access to services may be more limited and variable across and within countries. The best way to access those groups may be to proactively provide services to them (outreach). Providing outreach services is resource intensive, more difficult to sustain, and should not be done at the expense of integrating testing into existing services. However, well-planned, targeted outreach may be the most effective way of reaching people with no or limited access to services. Outreach can be made more cost effective by developing or strengthening partnerships between networks with existing access to risk groups. These networks may be clinical or non-clinical and could include adolescent and reproductive health services, youth centres, sporting clubs, secondary or tertiary education facilities, and peer educator networks. Partnerships between clinical and non-clinical services ensure that the skills and expertise of different personnel and organisations are used to maximum impact.

• Will specific strategies be used to increase the uptake of STI POC testing?

  A thorough assessment of the options for increasing the proportion of the target group who will be tested needs to be undertaken. Taking measures to make services more acceptable and accessible for specific groups – making them user-friendly – can be an important element in increasing testing. Strategies could include:

  • Providing outreach clinics to specific groups
  • Varying the opening hours of clinics to meet client preferences
  • Providing transport to and from clinics
  • Ensuring staff are non-judgemental and maintain confidentiality
  • Having separate, private entrances and waiting areas for men and women or entirely separate services for men and women
  • Making waiting areas friendly to the client groups – for example, through the use of appropriate posters, information materials

  Other strategies to increase uptake of screening could include linking testing to existing interventions and screening programs (e.g. Pap smears), educating doctors and patients, providing incentives to service providers or patients, and developing prompts to alert service providers or patients (computer alerts based on age groups & sex, text messaging etc.). Quality improvement initiatives can in themselves alert healthcare providers to differences in testing practices.

• How often will people in the target group need to be tested and how will they be recalled for subsequent tests? Will there be a need for a confirmatory test or a need for further medical review?

  A review of the literature to determine the optimal frequency for testing the target group may be required as ongoing debate about this is specific for the condition, the test and the population tested (see section 1.4 - the optimal interval for screening people for STIs).

  Ensuring regular testing of a high proportion of the target group can be challenging so all options for accessing the group should be discussed. Registers may be useful for recalling some groups for testing. Other groups may need other ways of engaging them and encouraging them to have a test such as use of social networking or mobile phones. Organised screening programs have to be carefully planned and evaluated, and they must be adequately resourced to maintain quality (see section 3.0 - Scale up).

  Regardless of which services are identified for the pilot of the POC test, links between services and referral pathways will need to be considered. For example, if a woman presents to a reproductive health clinic with lower abdominal pain that is suspected to be an ectopic pregnancy and not pelvic inflammatory disease as a result of an STI, what will be the referral process to a tertiary level service? If there is need for a follow-up test to confirm the diagnosis of an STI, who will do the test and how will the patient be referred for this test?

• What will be the impact of the introduction of the POC test on the health system?

  It is important to consider the potential negative effects of POC tests on service provision, staff workload and other aspects of the health system when deciding how tests will be implemented. For instance, who will perform the test, how long will it take to perform and what are the consequences for the health system of having this person now using part of their time to do the POC testing? How will the procurement, supply and distribution of the POC tests be achieved within the existing system?

  Different ways of introducing the POC test will have different resource implications. Costs will greatly influence how tests are introduced and how frequently people are tested. Overall, the economic impact on patient care and the total healthcare costs will need to be considered by health managers. The comparative cost of the test per patient compared with the cost for the standard laboratory procedure (if there is one) could also be assessed, as well as any benefits from preventing infection and the adverse consequences of infection (see section 1.4 - Decide which POC test to use & where it will be used).

• Will testing data be linked to the national STI surveillance system and so inform national health planning of STI prevention and control?

Key activities

• Define how the population can be accessed and list these options to pilot the introduction of the STI POC test – consider that it may be simpler to integrate the POC test with an existing service but discuss if this is feasible for the population to be tested. Also consider how the STI POC test will be integrated into procurement and supply management systems.

  List / map all the possible entry points for STI POC testing in your country. The information gathered as part of section 1.2 Review your local STI response will be useful to assess opportunities for integration (and gaps in existing services). Note that it may also be useful to map existing clinical beyond those related to STIs (and HIV) as well as non-clinical services provided by both the government and non-government sectors in order to assess other opportunities for integration.

• Detail the current utilization patterns and 'user-friendliness' of the services that are identified and decide if there is a need to employ strategies to improve access to and uptake of the STI POC test – this may be best discussed with stakeholders with in-depth knowledge of the health system, the population and communications.
• Consider how frequently this group needs to be tested and possible strategies to promote (or ensure) regular testing – The introduction of the STI POC test should lead to sufficient coverage of testing to impact on the prevalence of the infection.
• Review the likely positive and negative impacts of the introduction of the POC test on the specific health service(s) where it will be employed and the wider implications for the health system and the community – consider the information gathered in sections 1.3 (Decide if a POC test will be beneficial) and 1.4 (Decide which POC test to use & where it will be used) to make sure that the intended mode of introducing the POC test is feasible and likely to be beneficial. The pilot should be able to elucidate whether the POC test can be introduced at a reasonable cost and sustained, and whether the strategies of introducing and promoting the test are effective.
• Decide on the site(s) /service(s) where the pilot of the POC test can best be conducted – Do not be tempted to implement in too many sites or services simultaneously as it may become unmanageable and may not allow for a proper evaluation of the pilot. On the other hand, it may be necessary to pilot the POC test in a range of different settings where it is intended to be used.

2.6 Design a plan for implementing the pilot

After selecting the most appropriate way to introduce the POC test into the health system, a formalised plan will need to be designed to ensure that the pilot can be successfully implemented. A project management framework can assist in developing a clear outline or summary that explains what you hope to achieve, how you are going to do it, who will be responsible for each component, and when each component needs to be completed.

Key questions to ask:

• What are the expected health benefits that the new POC test hopes to achieve or contribute to (goal)?
• What changes will result from this pilot project (objectives)? (These changes will help you to achieve the overall goal or impact of introducing the new POC test)
• What will need to be in place (outputs) in order to accomplish these changes, and what work (activities) must occur to enable this?/li>
• What resources (inputs) will be necessary to carry out the activities of the pilot and contribute to the achievement of objectives?
• Are there any risks in implementing the pilot and can these be risks be managed in any way?

Key activities

• The technical working group should discuss the pilot project and detail the following to set up a plan for sound implementation –
  • The health benefits that you hope to achieve as a result of the project (goal) - for example, a reduced prevalence of Chlamydia among women attending antenatal clinics
  • How you plan to achieve these changes in health (objectives) - for example, through raising awareness of STI/POC testing among pregnant women and providing a quality service that is accessibly by pregnant women and expectant fathers
  • What things will need to be in place in order to accomplish this (outputs) - for example, trained staff, operational STI/POC services, and an effective communication strategy that promotes uptake of testing
  • The work (activities) that will need to be undertaken to train staff, operationalize STI/POC services and deliver an effective communication strategy; and
  • The various resources (inputs) that will be necessary to support all the work.
  • Identify risks to successful implementation of the pilot and ways of controlling these risks
• Summarize these discussions in a format that make sense to you – a project log-frame format or something more visual can provide an accessible, comprehensive overview of the elements of the pilot and the ultimate goal that the POC test is trying to contribute to. See the example below.

Key sources of information include:

Local

• Public health experts
• Monitoring and evaluation experts

Regional

• STI Regional Working Group for the Pacific
• Secretariat of the Pacific Community, HIV & STI section (spc.int/hiv/)
• The Oceania Society for Sexual Health and HIV Medicine (osshhm.org/)
• World Health Organization Western Pacific Regional Office, HIV/AIDS & STI programme(wpro.who.int/sites/hsi/)
• UNICEF
• UNAIDS
• UNFPA

Documents and materials

• AusGuideline. Activity design, Chapter 3.3: The Logical Framework Approach. Commonwealth of Australia, AusAID. 2005. http://unpan1.un.org/intradoc/groups/public/documents/un/unpan032502.pdf
• Tasmanian Government Project Management Guidelines. Version 7.0 (July 2011). Office of eGovernment, Department of Premier and Cabinet, Tasmania. 2011. http://www.egovernment.tas.gov.au/assets_for_review/tasmanian_government_project_management_guidelines

2.7 Develop tools for implementation

Specific plans will need to be developed to guide staff in the implementation of the pilot of the STI POC test. Reference has already been made to writing a plan to guide the whole pilot project (Section 2.6), but multiple plans for smaller elements of the project can be written and referred to in the overarching plan. For example, specific plans for implementation should include a procedure manual documenting standard operating procedures and quality management strategies to make sure the test is introduced appropriately and the impact of the pilot can be properly assessed. Plans should detail all aspects of the relevant project element, they should specify milestones and the outcomes expected, as well as the timeframe in which they should be achieved and who is responsible for their completion. Specific project management tools can help ensure the pilot is well governed and runs according to schedule and budget.

Some of the things to consider when implementing the pilot are:

• Creating a procedure manual
• Developing a communications strategy
• Planning for health worker training and supervision
• Managing procurement and logistics
• Employing other standard project management tools

Procedure manual

To further guide staff in the implementation of the pilot of the STI POC test, a procedure manual should be developed. The procedure manual should consist of a number of standard operating procedures (SOPs) covering all essential aspects of the pilot. To ensure that the test is introduced in such a way as to produce consistent and reliable results, one or more of the SOPs should deal with quality assurance of the POC testing.

Standard Operating Procedures (SOPs)

As a general guide, SOPs should:

• Have a standard format
• Be written in simple language and/or use simple intuitive diagrams
• Include enough detail to allow a health worker to perform the task without supervision – diagrams of the correct sequence are helpful
• Be followed exactly by all staff performing the test - they should be placed in a convenient site so that relevant staff will use the SOP
• Be uniquely identified by title and should include the version number of the SOP and the date of issue
• Be developed by an experienced professional at central level
• Be pilot tested with a group representative of the staff who will use the SOP to ensure that they understand the SOP as intended
• Be reviewed at regular intervals and updated to a new version if required (most of the SOPs used during the pilot phase will also be used during wider implementation of the POC test (3.0 Scale up))

The list of SOPs that are required for the STI POC pilot will differ from one situation to another. However, in general the SOPs should describe the following:

Who should be offered testing and how often How to explain the test to clients and how to obtain informed consent for testing How to take sample(s) for testing How to label specimens How to perform the test How to interpret the test result How to manage positive & negative test results Follow up and recall of people with positive test results Quality assurance of testing Infection control measures

How to safely dispose of waste Algorithms for providing treatment How to manage sexual partners of people with a positive test result Documentation of clinical management and test results for STI surveillance as per national quidelines Case management protocols for various possible scenarios such as ectopic pregnancy Testing for other STIs in addition to the STI/s covered by the POC test Appropriate referral practices Procurement of test kits and stock management

For many of the things on this list there will be existing resources that can be adapted e.g. the Oceania Society for Sexual Health and HIV Medicine has published clinical management guidelines for STIs (OSSHM. Recommendations for HIV Medicine and Sexual Health Care in Pacific Small Island Countries and Territories Second Edition. September 2008. osshhm.org/).

Example of an SOP for handwashing procedures

Quality assurance of POC testing

The successful delivery of the POC testing itself depends on a number of factors including the quality of the tests selected for use, the development of the procedure manual and the SOPs that it contains, and effective training of staff. It is also essential to ensure that testing meets and maintains the required standard. This is achieved using quality assurance. The details of a program of quality assurance will vary from situation to situation. However, the main available elements of quality assurance are:

• The use of external quality controls (QC)
• Participation in proficiency testing (QAP or EQAS)
• Sample referral
• Batch release of tests

Quality control (QC)

QC is designed to ensure that the tests used perform correctly and consistently from day to day, and to detect any errors in test performance. Tests may not have been manufactured correctly, may have deteriorated in storage or transport, or the testing may not have been performed as specified by the manufacturer. The use of QC helps detect and minimise these problems.

Some POC tests come with built-in internal controls that can alert the person doing the test that the test has not performed as expected. These internal controls are valuable, but their use will not detect all testing problems.

It is also very important to regularly test external QC samples that are of a similar sample type to the patient samples that are usually tested. In countries with lots of resources, such external QC samples are tested daily or even more frequently. This is likely to be difficult in Pacific Island countries and territories but periodic testing of external QC samples is important. All results obtained with external QC samples should be recorded along with the date, the batch or lot number of the test, and the name of the person doing the test. The relevant SOP should define the range of external QC results that are acceptable and the steps to take if the results obtained fall outside this range. The summary of QC results should be periodically reviewed to identify any trends that might warn of developing testing problems.

External quality assessment

External quality assessment schemes (EQAS) for medical testing are often known as quality assurance programs (QAP) or Proficiency Testing. The provider of an EQAS periodically sends identical panels of positive and negative samples to a number of participating testing services for testing. Staff at the testing services do not know the true results of each sample. Participants return their results to the coordinator of the scheme who reports the results. In this way, EQAS monitors the complete testing process from sample preparation to result reporting at one point in time, and the EQAS report allows participants to assess their own test performance and make improvements as required.

EQAS provides information that is complementary to the day-to-day use of external QC, and participation in an appropriate scheme is highly desirable. In cases where a relevant EQAS may not be available, different testing organisations or testing sites should periodically exchange samples so they can compare their results. In cases where sending actual samples for EQAS is too difficult or expensive, it may be possible for photos representing different levels of POC test reactivity to be sent to health facilities for review - this approach assesses the health worker’s ability to read and interpret POC test results but does not assess how they perform the test.

Sample referral

Another way to assure the quality of testing is to send a random subset of samples to a central testing facility for confirmation of the POC test results. The central testing facility can use the same or different tests to confirm the results. This approach, however, may not always be practical e.g. if the sample is capillary blood obtained from a finger prick at the point-of-care.

Batch release of tests

In some situations, each new batch of POC tests is tested at a central location against a panel of samples having known reactivity and the results are compared to pre-defined acceptance criteria. This process is known as batch release and is performed prior to the new batch being used to test patient samples. If the new batch fails to meet the pre-defined criteria, the manufacturer should be informed and the tests may be replaced. Batch release has the benefit of ensuring that the lot-to-lot variability of the POC test is minimised to an acceptable level. However, batch release testing does not detect problems with testing as a result of transport, poor storage or incorrect use of the tests.

Key questions to ask:

• What will be the screening age range (the upper and lower limits)?
• When/where should people be offered testing and how often?
• What will be the process for ensuring informed consent for testing?
• How should samples be collected?
• How is the test performed? What do you need to perform the test?
• How do you interpret the test result?
• What will be the process for managing positive and negative test results?
• What will be the algorithms or treatment protocols? Will these be the same in all regions of the country?
• What will be the process for managing partner(s) of people with a positive test result?
• What will be the process for follow up and recall of people with positive test results?
• What will be the expected requirements for documentation of clinical management and test results?
• What case definitions will be required?
• What case management protocols will be necessary e.g. management of ectopic pregnancy?
• What will be the expected approach to testing for other STIs in addition to the STIs covered by the POC test?
• When is referral appropriate and who will provide the necessary referral services?
• How will data feed into ongoing disease surveillance?
• How will quality assurance measures be implemented? Who will provide external quality assurance?
• Is the STI POC test being implemented as a new national screening program? If so, refer to guidelines

Key activities

There is a lot of work involved in preparing a procedure manual and establishing quality assurance processes. It might be helpful for the local Technical Working Group to commit time to working on this together and to consider adapting tools that have already been developed in the Pacific rather than starting from scratch. The following will need to be detailed and explained in the procedure manual:

• Identify the target population for screening and testing and the recommended interval for testing
• Explain how testing should be integrated into current services
• Describe the process and detail of ensuring that informed consent is gained from clients
• Explain in a step by step process (using photographs or images to assist), how samples should be collected, labelled and tested, and how waste should be safely disposed of
• Explain in detail (using photographs or images to assist) how the test should be conducted and how results should be interpreted
• Describe the required actions to manage positive and negative test results
• Develop treatment algorithms (or protocols) - consider using the existing guidelines by OSSHHM
• Describe the process for managing sexual partners of people with a positive test result
• Describe the process for follow up and recall of people with positive test results
• Detail requirements for documentation of clinical management (including history taking and examination) and test results
• Develop necessary case definitions (SPC & the STI Technical Working Group can provide assistance)
• Develop necessary case management protocols
• Detail expectations in relation to testing for other STIs not covered by the POC test
• Explain when referral is appropriate, the procedure to refer and where to refer to
• Describe any reporting requirements and whether data should periodically be provided for surveillance purposes
• Describe all quality assurance processes and develop necessary SOPs

Key sources of information include:

Regional

• SPC & the STI Technical Working Group - have been working on case definitions
• Oceania Society for Sexual Health and HIV Medicine (osshhm.org/) – for treatment guidelines

Documents and materials

• Oceania Society for Sexual Health and HIV Medicine: Recommendations for HIV Medicine and Sexual Health Care in Pacific Small Island Countries and Territories Second Edition September 2008. (osshhm.org/)
• National Association of Testing Authorities, Australia Policy Circular 2: Proficiency Testing Policy http://nata.asn.au/component/content/article/24-technical--document-updates/334-policy-circular-2-updated
• NRL. Quality Control in Infectious Disease Serology. Victoria, Australia, 2011. http://www.nrl.gov.au/CA25782200833499/Lookup/Resources%20%26%20Guidelines/$file/QualityControlinInfectiousDiseaseSerology.pdf
• National Pathology Accreditation Advisory Council. Requirements For Quality Management In Medical Laboratories (2007 Edition). Commonwealth of Australia, 2007. http://www.health.gov.au/internet/main/publishing.nsf/Content/6282D1966161A977CA25728B000FAB1B/$File/dhaquality.pdf

Communication strategy

Earlier identification of stakeholders and advocacy with them will help pave the way for POC test introduction. The next step is to take this communication down to the community level- to the people that you ultimately want to access with POC testing.

There are many specific resources that can guide you in developing a communication strategy. There is one manual that has been specifically developed for the Pacific region. This is highly recommended and listed under the reference tab in this section.

The following steps are similar to the communication strategy development process described in detail in this recommended manual. Please refer to the manual for further information.

Key questions to ask:

• What will it cost? What will your overall plan entail from start to finish? Who will be responsible for what?

  Develop a workplan detailing timeframes, activities and persons responsible for each step of developing, implementing and evaluating your communication strategy. This whole process might take 6-12 months if the communication strategy is complicated. A detailed budget should support the workplan. The budget might include:

  • Preliminary information gathering/research
  • Consultations and pre-testing costs (e.g. refreshments, venue hire, reimbursement of travel for participants)
  • Artist/ graphic design fees (or other technical support)
  • Translation costs
  • Costs of producing actual communication materials (e.g. pamphlets, posters, billboards, radio spots)
  • Costs of dissemination (e.g. postage or transport)
  • A contingency to cover unplanned costs
• What background information needs to be gathered to inform your communication strategy? How do you plan to do this?

  An effective communication strategy will depend on a good understanding of the issue(s) that need to be talked about and the target group or audience. Gathering this information can help you answer the following questions:

  • What is the extent of the STI problem and the common complications, and what are the likely benefits of diagnosis and treatment?
  • What do we know about the POC test itself, its ease of use and where it will be available?
  • Who is the target audience(s)? What do they know and what will they need to know?
  • What information or messages will have the most impact? E.g. you may find that it is better to frame your messages around healthy families rather than sexual health since the ability to have children is important in all the Pacific Island countries and territories; lessons from chlamydia screening in the Cook Islands suggest that it is essential to explore barriers to STI screening to develop appropriate messages and strategies.
  • What will makes your messages appealing to the audience(s)?
  • What are the best tools to reach your audience(s)? What is their preferred method of receiving health information? E.g. UNICEF has used text messaging as a strategy to communicate with young people in some Pacific Island countries and territories.
  • What skills will people need to change their behaviour? How can social and physical environments be made more supportive to enable the behaviour change?
  • Are there lessons from communication strategies for other health issues that are relevant to communications about a new STI POC test?
• Who is your target audience?

  To a large extent, the people you aim to test will already be defined. These are the people whose attitudes and behaviours we most want to change. Within the broad target group (15-30 year olds, pregnant women, specific risk groups) there may be a number of subgroups. The MOH will need to decide if they will introduce POC testing for all the target group or whether they will start by trialling POC testing with one or two subgroups only.

• What is the desired behaviour change? What are you 'asking' of the target audience?

  Overall, your aim should be to provide people with information so that they can make informed decisions. Think about what it is that you are asking of people. The "ask" is what compels people to take action. State this action as clearly as you can. For example, 'use a condom every time you have sex'. For a new POC test, we may want people to think about whether they could be at risk of STIs, and to go to a facility and ask for a POC test. Information on the safety, simplicity, and benefits of POC testing may be helpful in allaying fears. Also, ask whether this is an opportunity to strengthen existing messages about STI prevention and health seeking behaviour.

• What are your key messages? How will the messages be delivered? What communication tools will be used? Will you need an artist or other technical support?

  Messages will be most effective if you can consult with and involve your target audience in the development process. It is likely that you will need to use slightly different messages and methods (or tools) to communicate with each of your target groups.

  It can be tempting to give people too much information and think that they need to know everything all at once! Try to stay 'on the message'. This means you should stick to the key points and don't get lost in detail. This is an important thing to remember regardless of the communication tool used. The number of messages to include depends on the amount of time your audience will spend on it, though in most cases the audience will never spend more than a few short moments reading or listening to the information. A billboard is an example of a single message medium - passing drivers have only a split second to get the message. If the message is too complicated or if there are too many messages, the driver will miss the point. On the other hand, if you held a small group education session you may be able to present several messages, offer supporting information, and allow time to answer questions and discuss issues in detail. However you go about it, try to present the fewest possible messages to get the point across.

  When developing messages and materials it may help you to use a materials development worksheet. This can help organise your thoughts so you only include the most important bits of information.

  Example of a materials development worksheet

• How and where will you pre-test your materials?

  It is essential that you pre-test your draft materials with your target audience to help you develop a better end product. Communications materials often need to be pre-tested and revised several times before being finalized. Pre-testing might explore the following things:

  • Is the content understandable and clearly presented?
  • Does it convey the message we intend?
  • Is the material attractive? Will it get people's attention?
  • Is the material acceptable to the audience?
  • Does the audience feel involved? Does it speak to them?
  • Is there a 'call to action'?

  Once you have decided on a list of questions to explore through pre-testing, consider putting them into a question guide to assist you in the improving your messages.

  Example of a pre-test question guide

• How will you evaluate effectiveness of your communications strategy? Does your workplan allow for revisions to improve the strategy?

  After implementation of your communication strategy you need to figure out if it was a success or not (and the reasons why). This information can be used to improve future communications with the target group(s). It will be helpful to find out:

  • How the messages are being received and used by the audience and other relevant stakeholders such as health workers
  • Whether messages and the medium to transmit the message resulted in reaching the audience in the way(s) that you planned
  • Whether or not the messages and methods are accepted and understood by the audience and had the impact on attitude and behaviour that you aimed for

Key activities

• Develop workplan and budget
• Clarify target audience(s) - be specific!
• Gather background information / research to inform your communication strategy
• Specify the desired behaviour change or what are you 'asking' of the target audience
• Draft key messages and strategies for reaching the target audience
• Draft communication tool(s)
• Pre-test your materials
• Consider what support you will need from various groups to reach your target audience and implement the communication strategy
• Evaluate the communication strategy

Key sources of information include:

Documents and materials

• Secretariat of the Pacific Community (2008). Creating an effective communication project in the Pacific region for HIV/STI and other sexual and reproductive health projects. http://www.spc.int/hiv/index.php?option=com_docman&task=cat_view&gid=109&Itemid=148

Health worker training and supervision

All staff expected to provide POC testing will need to be trained to use the test.

Key questions to ask:

• Who will be responsible for the training? Are they experienced and motivated trainers with a strong background in sexual health? Do they use participatory training methods? Are they familiar with the principles of adult learning?

  Many trainers wrongly assume that health workers will automatically change their work practices following training. Training methods can influence how much of the training gets transformed into learning and action. To make training as effective as possible consider some of the following:

  • Who delivers the training: for this type of training a small highly skilled team of trainers (2-3 people) is preferable to a 'training of trainers' approach to avoid the reduction in the quality of training that can occur with 'training of trainers'. All health staff using the POC test should be competent in performing it.
  • How training is delivered (principles of adult learning): Adult learners have special needs and requirements. Answering 'yes' to the following questions means that you will be doing a good job to support adult learning:
    • Is the training relevant - does it fulfil an immediate need? Motivation to learn will be high if the training meets the immediate needs of the learner.
    • Is the training approach participatory and 'hands-on'? Does the approach include a variety of teaching methods to suit the subject matter and different learning styles? (E.g. people who learn by seeing like diagrams, pictures, videos, use of white board/ butcher's paper/ note taking, charts, handouts, demonstrations of technical skills or equipment; people who learn by hearing like interactive lectures, group discussions, reading aloud, brainstorming, debates; people who learn by doing will like handling equipment, demonstrations, moving around while learning, using training models that reflect real situations.)
    • Will the training encourage learners to draw on experiences and learn from each other?
    • Will participants be given the time to reflect on the training, draw conclusions and think about how the training can be applied in their workplace?
    • Will trainers provide feedback to participants in a constructive, supportive way? Discussion and time for questions is important if participants are to make sense of new information and link it to their own knowledge and experiences
    • Will trainers show respect for participants? Mutual respect and trust between trainer and learner help the learning process.
    • Will the training occur in a safe and comfortable environment? A happy and relaxed participant will learn much better than someone who is fearful or embarrassed. Similarly, someone who is physically uncomfortable (e.g. hungry, cold, sick) will not learn with maximum effectiveness.
• What will the training program look like? Can it piggyback onto other planned training? Can it be fully integrated into routine training?

  A training module on POC testing should be developed - you may be able to adapt existing tools or you may have to develop new training tools. This module could be delivered:

  • As a stand alone in-service training for staff who have recently participated in Comprehensive Management of Sexually Transmitted Infections for Pacific Island Countries training: some sections of the Comprehensive Management of STIs training would need to be refreshed to support POC testing. For example, the informed consent/counselling session would need to be revised to include discussion of outcomes of POC testing (such as false positive tests). (Note that it can be helpful to add this training onto in-service training that has already been planned for another reason.)
  • As an additional module integrated into Comprehensive Management of STIs training: This would be the ideal situation but may take time to include and the timeline of the pilot may no fit with the schedule for training.
• What will be the content of training? What will health workers need to know? What materials or resources will need to be developed to support the training?

  POC testing for STIs (other than syphilis and HIV) is new in the Pacific. At a minimum, the training module should address the following questions:

  • What are point-of-care tests? What added benefits do they offer?
  • What is the role of POC tests in the overall national STI control program?
  • Simple scientific principles of POC-tests (how they work)
  • What items are necessary to do a test?
  • What quality control measures are required?
  • How should tests be stored, tracked and procured?
  • Who should be offered a test?
  • What specific clinical or counselling skills are also required?

  Learners should be required to meet specific standards or competencies before using POC tests in the workplace. In addition to skills taught during the Comprehensive Management of Sexually Transmitted Infections for Pacific Island Countries Training (e.g. take a sexual history, conduct a physical examination, prescribe the correct treatment), health workers performing POC testing should be able to:

  • Use protocols to decide who should be offered a test and know when POC testing is not appropriate
  • Counsel the client about the POC test
  • Instruct the client to collect a sample or collect a sample from the client and ensure proper labelling of specimens
  • Perform a POC test and interpret the result
  • Appreciate quality control mechanisms (internal and external) including knowing when tests have not performed as expected and how to troubleshoot problems
  • Use flowcharts or treatment algorithms to provide treatment Document and report test results and management
  • Implement infection control procedures including safe disposal of test kits, reagents, and biological specimens
  • Correctly store test kits, reagents and specimens and manage stock

  An example of suggested training content specific for a new STI POC test according to different health workers

  A transfer of learning matrix can help support supervisors, trainers, learners and co-workers to implement the new training on the POC test.

  Example of a transfer of learning matrix

• What will be the overall plan for roll out of training? How much will it cost?

  Develop a course timetable with a brief description of each training activity.

  Example of a training course timetable

  For each topic area, describe learning objectives - describe what the participants will know or be able to do after the training e.g. a session on performing POC testing might have a learning objective like this: 'At the end of this session learners will be able perform a POC test and read the result according to the competency checklist'. Learning objectives should focus on the important knowledge, skills or attitudes that will be required by all staff performing the POC test.

  Each session should also have a lesson plan to guide the trainer – it should detail the time allocated to the session, the learning objective, the activities used to train staff (e.g. brainstorming, demonstration, group work, role-plays etc.), and the resources needed for the session (e.g. butcher's paper, markers, laptop, projector etc.).

  Example of a lesson plan

  Develop a training plan to guide the roll out of training. This should include: identification and selection of participants; numbers to be trained; pre-requisite (knowledge, skills, education, position) for training; identification of person(s) responsible for coordination and delivery of training; training dates and length of course; training sites; resources required (human, material); detailed budget (e.g. trainers, venue hire, materials, transport, accommodation, per diems etc.).

  The plan to roll out training should also take into consideration factors that may influence whether learning is put into practice: (1) is there a good understanding of the workplaces you aim to change and how can the training package be made relevant to the environment that staff operate in, and the anxieties, stresses and challenges they face; (2) training should be conducted as close as possible to the workplace where the POC test will be implemented; (3) personnel who will work together should be trained together; (4) training should be timed and linked to other health actions (e.g. introduction of new policies, new equipment, new staff); (5) training should be conducted when the system is ready to change (e.g. when POC tests are available, when staff have the authority to act on the training etc.).

• How will participants be assessed? Will there be some kind of accreditation process?

  Assessment will help trainers understand whether the training has achieved its objectives and might help identify sections of training that need improvement. Assessment is also useful for learners providing review and reinforcement of key points. A range of assessment tools may be appropriate (games, short quizzes, written assessment, practical demonstrations etc.). A competency based assessment checklist would be suited to assess clinical, laboratory and counselling skills. A checklist breaks down each skill into small parts. It can ensure that procedures are conducted in a standard way and to a minimum performance level, and can function as a self or peer assessment tool. Remember it is always advisable to repeat competency assessments some time after the training when learners have returned to their worksites.

  Example of a competency based assessment checklist

  Having the training accredited in some way can motivate health workers. For example, you may suggest that staff can become Ministry of Health accredited STI service providers after they have completed training on Comprehensive Management of STIs, Voluntary Counselling and Testing for HIV and STI POC test training.

• How will training be evaluated?

  Evaluation can help you understand whether the training met the expectations and needs of participants and how it could be improved. It is a good idea to leave a break between the first lot of training and the next- to allow time to reflect on how the training went and modify the approach if needed. Areas that might be evaluated include:

  • CONTENT: depth, breadth, relevance, appropriateness, whether objectives/expected outcomes and expectations were met, omissions, quality of handouts, learning aids, activities, whether training built on existing knowledge, attitudes, abilities, link between training and field – can learning be applied, aspects found most/least helpful
  • PROCESS: length/pace of workshop, organization, variety of tools used, suitability to content, whether information was clear and understandable, did students have a chance to practice skills and receive feedback?
  • FACILITATION: clarity of explanations and instructions, interpersonal skills, questioning and feedback skills, responsiveness to individual/group needs, whether learners were encouraged to participate and ask questions, organization.
  • FACILITIES: lighting, space, food, accommodation, ventilation, heating/cooling, seating, equipment etc.
  • SELF-ASSESSMENT OF TRAINER: teaching methods and tools, students were engaged and participated, training pitched at right level, syllabus was followed, revisions required during the conduct of training, how to improve preparation etc.
• How will participants be supported and supervised post training? What will be the plans for refresher training?

  Some kind of follow-up is essential to reinforce training. Multiple reminders can be useful (e.g. verbal and written prompts in the workplace, regular formal meetings to review and assess performance, requirements to report on progress at monthly meetings, help participants to see the results of change through feedback on disease trends etc.). Strategies may need to be specific to local circumstances. Line managers/ supervisors can play an important role in supporting the implementation of the POC test after training: (1) they can conduct a post-training debrief to make an action plan with the staff member to assist them to implement their new skills in the workplace; (2) they could have a role in evaluating the learner's on the job performance if they have been trained themselves – this could be done using a competency checklist; (3) they can ensure supervision and quality assurance processes are factored into annual planning so they are funded and supported; (4) they can assist with adding POC test checklists into existing checklists used to monitor STI service delivery; (5) they can provide feedback to trainers about the training and seek advice about how to assist with the transfer of knowledge and skills in workplace.

Key activities

• Identify training needs and trainers
• Decide on the design of the training program
• Adapt existing training materials or design new materials
• Develop a training plan and budget
• Discuss and decide on accreditation processes and means of assessment
• Conduct training
• Evaluate training
• Follow up with supervision and support as part of routine quality control
• Build refresher training into future training plans

Key sources of information include:

Documents and materials

• Comprehensive Management of Sexually Transmitted Infections for Pacific Island Countries training modules
• Transfer of Learning. www.intrah.org
• JPHIEGO, USAID, WHO, 2005. Effective Teaching: A guide for educating health professionals. www.jhpiego.org/resources/pubs/effteach/EffTeach_facgd.pdf
• The Centre for Development and Population Activities. The CEDPA Training Manual Series, Volume I. http://www.cedpa.org/content/publication/detail/757
• JHPIEGO, The Competency Based Approach to Training
• PATHFINDER INTERNATIONAL Trainer's Guide Advanced Training of Trainers

Procurement & logisitics

Procurement, supply and stock management can be challenging for many countries but are essential components of a functioning health system. It is important to develop standard operating procedures that are harmonised with national guidelines and to integrate the supply chain into the existing system.

Key questions include:

• What are the existing national guidelines for procurement and stock management?

  It is important that the pilot test of the POC test be run using the current set up for managing medicines and other consumables in your country. (Remember that WHO have a bulk procurement service that may be able to secure supplies of the POC test at a low price if the test meets the WHO ASSURED criteria.)

• How many POC tests will be needed for the pilot test?

  It is important to order the right number of tests to avoid shortages or wastage. The right quantity to order is often based on previous records of medications used or dispensed at each facility. In the case of a new POC test, there will be no such records. Calculations of the number of POC tests that will be needed must therefore be based on estimates of the number of people who will be tested at each testing site (as well as referring to the pilot design which will outline how many tests will need to be performed to document the test performance and feasibility of implementation). Ideally, these should be based on health records of the number of people who attend that testing site who met the criteria for testing with the new POC test. The testing sites will need to give this information to the central level personnel responsible for procuring and distributing the tests. The information will need to be provided early enough so that stocks are delivered when they are needed. Some distribution systems can take a long time to distribute stock to outlying facilities.

• Who will be responsible for collecting consumption data in each facility? Will they use existing tools such as stock cards and tally sheets? What is the procedure for testing sites to order the STI POC tests from the procurement department?

  A meticulous record of the numbers of tests used each month is needed for accurate orders and to improve forecasting of additional supplies. (These figures will be important to estimate the total number of tests needed if it is decided to scale-up.)

• Will there be a need to change the orders for related stock such as treatment for the STIs?

  There may be a need to change orders for STI treatment if the POC test influences the number of people attending a facility who are treated.

• Do facilities have procedures for checking that stock is managed appropriately including correct storage and disposal?
• How does your central medical store procure medical supplies?

  The pilot of the POC test will have to ensure that the ordering of tests coincides with national schedules. For example, if tests are only ordered every month then this timetable must be factored into the planning of the pilot test.

  Examples of standard tools for calculating usage and ordering supplies

  Example of a clinic order form

  Example of a clinic tally sheet

Key activities

• Identify persons responsible for national and local procurement and distribution of medical supplies
• Explore whether the POC tests can be bulk procured via the World Health Organization or UNICEF
• Work with key personnel to integrate procurement and supply management of the STI POC test into existing national protocols
• Use existing national tools that aid in the management of stock to include management of the new STI POC tests. For example, develop stock cards, inventory control checks, documentation on use, ordering and receiving supplies, tools to estimate monthly consumption of supplies

Key sources of information include:

Local

• Chief Pharmacist
• National Therapeutics Committee

Regional

• STI Regional Working Group for the Pacific
• Secretariat of the Pacific Community, HIV & STI section spc.int/hiv/
• World Health Organization Western Pacific Regional Office, HIV/AIDS & STI programme wpro.who.int/sites/hsi/
• The World Health Organization (WHO) Bulk Procurement Scheme. WHO Contracting and Procurement Services (WHO/CPS). Email: procurement@who.int http://www.who.int/diagnostics_laboratory/procurement/en/

Documents and materials

• Andersson S, Snell B (2010), Where there are no pharmacists: a guide to managing medicines for all health workers. Health Action International Asia Pacific. Sri Lanka.
• USAID DELIVER Project (2009) The Logistics Handbook: A Practical Guide for Supply Chain Managers in Family Planning and Health Programs.

Other standard project management tools

Experience project managers will consider the following aspects of managing the pilot of the STI POC test:

• Tools to ensure the project runs to schedule
  • A work or implementation plan so that inputs can be delivered at the right times in order to run activities and deliver on outputs, outcomes and impacts. This provides a roadmap for the pilot.
  • A list of the milestones and major activities of the pilot test so that progress can be monitored – a Gantt chart can provide a snapshot of these milestones; the Critical Path Method can be used to estimate timeframes, describe the relationship between various inputs and outputs, and track progress; software such as Microsoft Projects can help develop these project management tools.
• Tools to manage resources
  • Personnel – need to consider the mix of skills needed to implement the pilot of the POC test; the time and resources required to recruit staff; additional training or team building exercises for project team members; employment conditions and leave requirements; the need for external advisors and consultants etc.
  • Financial resources – a project budget should detail all the resources required to complete the activities of the pilot project including costs related to salaries, training, accommodation, per diems, project management, procuring and distributing POC tests, quality management etc. Individual costs should be linked to project activities or milestones to help with monitoring and reporting. Actual expenditures should be compared with planned expenditures.
  • Physical and material resources – any changes to the sites where testing will be offered and any materials required (e.g. computers, phones, vehicles etc.) will need to be budgeted for. Local means of procuring these things will need to be followed.
  • Information resources – it's a good idea to know who will manage information collected by the pilot project, how information will flow between different sites and people, where it will be kept and backed-up, how long it will be kept for and how it will be disposed of.
• Tools to manage stakeholders
  • It can be useful to list all the stakeholders, decide how they can influence the pilot test and how important this influence is in order to develop a plan for managing your engagement with stakeholders throughout the pilot test
• Tools to mange risks
  • Any factor that can negatively affect the success of the pilot test in terms of the timelines, budget and outcomes should be identified and analysed. Measures to deal with the risk and a person responsible for doing this should be documented part of the project management plan.
• Tools to manage issues
  • Issues that affect the progress of the pilot project could come up at any time during implementation. Describing in advance a process to document, analyse and manage any issue that arises and assigning responsibility to specific team members can be a good idea.

Key sources of information include:

Documents and materials

• Tasmanian Government Project Management Guidelines. Version 7.0 (July 2011). Office of eGovernment, Department of Premier and Cabinet, Tasmania. 2011. http://www.egovernment.tas.gov.au/assets_for_review/tasmanian_government_project_management_guidelines

2.8 Implement, monitor & evaluate pilot

There are several reasons why monitoring and evaluation (M&E) is important for an STI POC test pilot. As well as documenting progress and enabling reporting on activities, M&E can help us to understand:

• Whether the project is being implemented as planned
• Whether we are reaching the target population
• The problems and successes so activities can be revised and continually improved and gaps can be addressed
• Whether the project is cost-effective
• Whether the project is contributing to improved public health (for example, reducing the prevalence of STIs, preventing severe consequences of infection such as death and disability etc.)

There are many resources that deal with M&E in more depth than this toolkit. However, the principles of implementing an M&E plan for an STI POC test pilot are the same as for any project:

• Good monitoring and evaluation starts at the design phase
• Keep it simple - Information gathering methods and tools should be clear and simple and the information gathered must be relevant, of high quality and useful. Always ask: 'how will this information be used?' and 'what purpose will it serve?' If you cannot answer these questions then the information or the way in which it the information is gathered may not be appropriate.
• Choose valid, feasible indicators, measured at appropriate time intervals - Indicators of progress should only be measured at intervals when measurable change can be expected to have occurred. For example, access to POC tests may take many months to years to have an impact on the prevalence of STIs. Indicators that rely on expensive surveys should be avoided unless surveys are routinely conducted (e.g. for surveillance purposes) or are deemed important enough to support.
• Wherever possible integrate information gathering into existing processes - For example, the national health information system will gather information on STIs. This existing system may provide useful information for your own M&E plan and information collected by your M&E plan may add to the national health information system. If you do need additional information, think about whether it is possible to add to current data collection forms rather than creating new forms or additional processes.
• Involve stakeholders and feedback information to them – Often staff and community members are not involved in M&E. Think about how you can involve them and ensure that feedback is provided to staff and affected communities after the information has been collected, analysed and interpreted.
• Consider ‘gender' and 'equity' considerations - Monitoring should seek to understand any factors that lead to inequality in access to or utilization of STI POC testing (e.g. disaggregate data by sex, age, economic status, ethnic group, religion etc. to illustrate the impact of these factors). This may require additional follow-up such as focus group discussions or key informant interviews.
• Aim for continuous quality improvement - M&E should not be perceived as a judgment on staff performance. Rather, it is a chance to reflect on progress and identify strengths, weaknesses and lessons to improve future activities.
• It is important that an appropriate budget has been allocated to evaluate the pilot test – M&E takes time and resources and may require personnel with specialist skills. Budgets may be high if an external review is needed.

Key questions

• How would you describe the pilot project in terms of goal, objectives, key outputs, activities and inputs? What parts of the project will need to be monitored? What will need to be evaluated?

  Having a clear description of the project as documented in section 2.6 Designing a plan for implementing the pilot can help you to develop a Monitoring and Evaluation Pathway (below). This pathway is an excellent aid to thinking about what things need to be monitored and evaluated at different levels of the project to pilot the STI POC test.

  

• What indicators will be used to measure progress?

  Indicators are like signposts. They help us to know if we are doing and achieving what we set out to do at all levels of the project. There are a number of aspects about indicators that are important to understand: (1) the feature of the pilot test that the indicator is assessing; (2) the type of indicator; (3) the stage along the M&E pathway that the indicator relates to; and (4) factors to consider when choosing indicators.

  The aspect of the pilot test that the indicator is assessing - examples of indicators that could be monitored include:

  Feature of the pilot test	Examples of potential indicators
  Logistics and planning	Is the physical space adequate to perform testing and confidential informed consent and counselling? Do scheduling and the volume of client visits allow enough time to provide counselling, testing and treatment in a single visit? Is the storage of POC tests appropriate? Are staff able to track usage of tests and other essential items (such as condoms, drugs and consumables such as urine jars) to maintain adequate supplies?
  Test usage	Are health workers performing POC testing according to protocols and competency standards? Is testing being restricted to the target population? How much wastage is occurring? What is the rate of indeterminate or inconclusive testing?
  Reaching the target population	Is the target population clearly defined? What are the age, sex, and risk characteristics of the people being tested? Does this match up with our target population? Are the facility location, opening hours, and waiting space suited to clients? Of those tested, what proportion tests positive?
  Impact on client management	Are all clients getting appropriate treatment at time of testing? If not, why not?
  Impact on facilities offering testing	What kind of impact is POC testing having on staff workload? To what extent is POC testing being integrated into normal practice? What do health care providers think about POC testing?
  Acceptability / satisfaction of testing for clients	What do clients think about POC testing?

  Types of indicators - Indicators can be quantitative or qualitative. It can be a good idea to measure progress around a certain key aspect of the pilot project in several different ways often using both quantitative and qualitative indicators - this helps us to improve the quality of our information gathering. For example, if we are looking for change in the quality of an STI service we might explore this through exit interviews or focus group discussions with clients, as well as observation of the facility and staff (e.g. document whether all necessary supplies are in stock, whether staff manage clients according to established competencies, whether clients are seen in private and client information is treated confidentially).

  Type of Indicator	Examples of indicators	Examples of how to gather this information
  Quantitative – Commonly used Tell us about the amount (or size) of change that has occurred Not well suited to capture unexpected results Don't tell us why activities worked or did not work	The number of ANC staff trained in using POC test kits The percentage of trained staff using POC tests according to competency standards 6 months after training The number of pregnant women and partners receiving POC chlamydia test Prevalence of chlamydia among pregnant women at ANC	Review of Health Information Systems data Review of project reports Population surveys (e.g. knowledge/attitude/ practice surveys)
  Qualitative – Describe what has happened and explore the reasons why Good for the unexpected Can provide rapid feedback	Quality of the STI service provided Client satisfaction (with the service provided) Acceptability of POC testing for service providers Awareness of POC testing services	Observation Focus group discussions Community interviews Interviews on significant change Analysis of reports, plans and policies

  Change in 'impact level' indicators like STI prevalence is not always attributable to the project. Why?

  In any community, there will be many influences that contribute to changes in the prevalence of STIs. Change may occur because of the planned activities of your project or change may have been happened because of another factor occurring at the same time as your project. For example:

  • A national condom promotion program delivered by the Ministry of Health might be an effective contributor to a reduction in STI prevalence and it may have been this initiative rather than your project that was primarily responsible for the reduction.
  • An increase in STIs might have been precipitated by a large-scale natural disaster that disrupted people's access to STI services, so the worsened situation was due to this and not the failure of your project
  • An increase in reported STIs (or diagnoses) might be due to improved awareness of and access to POC testing, so what may look like a worsening health situation may be due to better diagnosis and greater use of services as a result of your project.

  Care should be taken when attributing change in higher-level indicators such as impact indicators to your project as it can sometimes be a big challenge to work out the exact contribution of your project to these changes.

  Stage along the M&E pathway that the indicator relates to - Indicators can be used to assess the pilot projects progress against all levels or stages of the project. In general, it is more difficult to measure indicators the further along the M&E Pathway that you go. For example, a measure of progress at the output level such as counting the number of ANC staff trained in using POC tests will be much easier than measuring change in the prevalence of STIs a measure of change at the impact level. Such high-level changes usually occur slowly and may require difficult and expensive surveys to measure. Indicators such as this are more likely to be measured at a provincial or national level rather than in a small project. Also, even if change does occur at the impact level it will not always be because of the project.

  An example of a monitoring and evaluation pathway for an STI POC test pilot with indicators at each stage for monitoring and questions for an end of project evaluation

  An evaluation can be focused on the: (1) processes (inputs and outputs) to see if these can be improved during the implementation; (2) outcomes to examine short term changes as a result of the pilot test; or (3) impacts to assess whether the project has achieved higher-level changes over the longer term.

  Factors to consider when choosing indicators: Do not choose too many indicators. It is tempting to list every indicator that you can think of but it could be too difficult for personnel to collect all this information in practice. Think about what best represents what you are trying to achieve, what will be the best measure or description of that, and what information will actually be reported on and used. Where possible, it is advisable to choose the same indicators that are used for routine reporting or surveillance. These indicators may be used for reporting at the international or national level, or they may be used by local health systems. For each indicator ask the questions below. If the answer to any of these questions is "no," or "don't know," then the indicator is probably not the best measure – cross it off the list. You may need to brainstorm to consider an alternative indicator.

  Question	Yes	No	Don't know
  Can we measure or describe this indicator before and after the activity or intervention? Will there be sufficient time and resources to do this?
  Does the indicator measure what we are most interested in? Does it relate to change somewhere along the M & E Pathway?
  Is the indicator valid? Does the indicator measure what it is supposed to measure?
  Is the indicator reliable? Can different people measure it consistently over time? (e.g. if two supervisors use a checklist to judge whether a staff member adheres to competency standards for doing a POC test, both supervisors should come up with the same rating.)
  Is it possible to measure a change in the indicator within the time period? (Remember that it may take a long time to see change in high-level impact indicators.)

  Indicators should be developed according to the SMART guidelines:

  Specific	precise, well defined and relevant to the objective
  Measurable	possible to measure and document changes either quantitatively or qualitatively
  Achievable	can be attained within the life of the project
  Realistic	given the resources, time period of the project and context
  Timebound	have a set timeframe within which they should be achieve

• How will information be gathered about these indicators, how often will this need to happen and who will be responsible?

  The timing of M&E activities will be influenced by the implementation timetable. Always consider practical factors such as when people can give time to gathering and reporting data. A monitoring and evaluation plan can summarise this information and is a useful tool for implementation:

  Example of a monitoring and evaluation plan

• How will the information gathered be analysed and how will it be used?

  Information should be collated and analysed periodically as determined by the technical working group. Setting milestones for the reporting of indicators is one way of monitoring progress. Remember that much of the information gathered for the purposes of your project M&E might contribute to the annual planning process for the national HIV/STI program.

  Consider bringing together key stakeholders to assist with reviewing information - to discuss, reflect and think about what it means for future project activities. This might happen in a 'reflection workshop' and should ideally feed into planning for the year ahead. For example, Project Reflection Questions could be asked in relation to each outcome area (or overall project) on a regular basis such as every 6 months:

  Example of project reflection questions

• How will information be fed back to stakeholders?

  Ensure that after information is gathered, analysed and interpreted appropriate feedback is provided to all stakeholders. (E.g. facility staff, clients, donors etc.). Remember that you will need to provide different information in different ways for each of these groups.

Key activities

There are many resources that deal with the important topic of monitoring and evaluation in more depth than we are able to in this toolkit. Please see Key References in this section for some examples. The idea of thinking about how to monitor and evaluate POC testing might seem a bit daunting. But, it doesn't need to be too complicated. Try and think about the process of developing an M&E plan by working through the following steps:

• Write down a good summary of the project – Before developing an M&E plan, it is important to have a good summary of the project written down (Refer to section 2.6 - Design a plan for implementing the pilot).
• Apply the 'monitoring and evaluation pathway' to your project to identify aspects of the project that might need monitoring or evaluation – Once you have a clear description of the project, you can start thinking about what things will need to be monitored and evaluated and when they will need to happen. Understanding the 'Monitoring and Evaluation Pathway' as it relates to your project can help you to do this.
• Decide on the measures or 'indicators' that will be used to help track progress – There will be indicators to signpost progress and achievements at different stages of the pilot project and for different aspects of the project. Both quantitative and qualitative indicators will usually be needed. Understanding the national and regional surveillance systems as well as the local health management system will assist in aligning the indicators used by these systems and the pilot project.

  (SPC, UNICEF, UNAIDS, UNFPA and WHO should be able to advise on up to date surveillance tools (forms, databases, other reporting mechanisms). As STI POC testing is new in Pacific Island countries and territories, it is not yet known if POC-diagnosed cases will be counted as equivalent to 'laboratory-diagnosed' for regional case definitions. Try to get an update on this from SPC and WHO prior to project start up- so you are clear on what data can be fed into the surveillance system.)

  For an evaluation(s) of the pilot, the technical working group will need to decide on the focus of the evaluation to make sure that it assesses the aspects that they deem to be most important.

• For each indicator, think about how you will get the information, how often you will collect this information, and who will collect it – Documenting this will enable you to draft an M&E plan and assign responsibility to particular staff.
• Think about how you will analyse the information, how you will use it for reporting and how you will use it for continual quality improvement of the pilot – Consider the frequency of reporting, who will do the preliminary analysis of gathered information, and who will review the findings. If stakeholders need to be involved with interpreting the information, how will they be engaged to do this? Ensure that once information is gathered and analysed, that feedback is provided to all stakeholders. For example, facility staff, clients, donors. Remember that you will need to provide different information in different ways for each of these groups.
• Consider the timing, type and focus of evaluations required and the resources required to do these.

Key sources of information include:

Local

• Local monitoring and evaluation experts
• National and local health indicators

Regional

• STI Regional Working Group for the Pacific
• Secretariat of the Pacific Community, HIV & STI section (spc.int/hiv/)
• World Health Organization Western Pacific Regional Office, HIV/AIDS & STI programme (wpro.who.int/sites/hsi/)
• UNICEF
• UNAIDS
• UNFPA

Documents and materials

• UNFPA, Programme Manager's Planning Monitoring & Evaluation Toolkit: Tool Number 6: Programme Indicators, August 2004. http://www.unfpa.org/monitoring/toolkit/tool6.pdf
• Handbook on Planning, Monitoring and Evaluating for Development Results. http://www.undp.org/evaluation/handbook/
• LSHTM (2011). The Rapid Syphilis Test Toolkit: A Guide to Planning, Management and Implementation. http://www.lshtm.ac.uk/itd/crd/research/rapidsyphilistoolkit/
• Tasmanian Government Project Management Guidelines. Version 7.0 (July 2011). Office of eGovernment, Department of Premier and Cabinet, Tasmania. 2011. http://www.egovernment.tas.gov.au/assets_for_review/tasmanian_government_project_management_guidelines

Next: 3.0 Scale Up