A Tool for Monitoring the Performance of Family Planning Programs in the Public and Private Sectors: An Application in Nigeria
A one-page family planning questionnaire developed by Nigeria's Federal Office of Statistics can be attached to nationally representative surveys that are routinely conducted to monitor development activities. The module collects data on household composition, source of family planning information, current use of family planning methods and the source of services and supplies. A comparison of data from two of these family planning modules, fielded in December 1992 and December 1993, with results from two earlier national surveys reveals that modern method use among women aged 15-49 went from 4% in 1990 to 9% in 1992, and then rose to 11% in 1993. Such a tool allows program managers to document the expansion of family planning practice between infrequently conducted national fertility surveys.
(International Family Planning Perspectives, 23:162-167, 1997)
In the past decade, the numbers and types of places where individuals and couples can receive family planning in developing countries have increased tremendously. Services and supplies can now be obtained from public facilities and community outreach programs, from private-sector channels (including family planning associations, women's organizations, pharmacies, employer-based distribution programs and private clinics) and from informal-sector outlets in traditional market settings.
This diverse environment of family planning services requires a wealth of information to monitor the expansion of population programs. However, few countries have systems in place that routinely assess the types of services and supplies that are rendered by alternative providers in the public and private sectors. Service statistics systems, which are traditionally used by most countries to monitor performance, collect information on the types of services and commodities provided and the volume and type of clients served in government and private nonprofit programs. Few of these systems attempt to monitor the expansion of services and supplies among commercial-sector providers.1
Although national contraceptive prevalence surveys, such as the Demographic and Health Surveys, provide information on the total service environment (including clinical and nonclinical services and supplies in the public and private sectors), these surveys are conducted infrequently because of both data collection costs and the slow processing of results. Thus, there is a real need for a low-cost monitoring tool that can provide national data to family planning managers between sporadic, large-scale surveys.
In this article, we present an example of a monitoring instrument that can be administered on a quarterly, biannual or annual basis—a one-page questionnaire to be attached to national household development surveys, which are routinely conducted in the developing world by a census bureau or other government statistical offices. The questionnaire collects information on household composition, sources of family planning information, types of methods currently used and sources of services and supplies.
Such an instrument can allow program planners to monitor the provision of services and supplies in the public and private sectors, to assess the performance of alternative information, education and communication efforts, and to evaluate progress in reaching different segments of the population. It is easy to administer and process, and produces quick results. As such, it is ideal for countries that have limited resources for collecting national information on family planning performance.
The tool described in this article was developed and tested throughout Nigeria by the country's Federal Office of Statistics. We discuss and evaluate its use in Nigeria in December 1992 and December 1993, and compare results from those modules with findings from two earlier national fertility surveys conducted in Nigeria.
Nigeria has the largest population in Sub-Saharan Africa, with 88.5 million people, according to the latest national census (1991). As in most African countries, contraceptive prevalence rates are low. According to the 1990 Nigeria Demographic and Health Survey (NDHS), only 6.0% of married women of reproductive age are currently practicing contraception, with 3.5% using a modern method and 2.5% a traditional one.2
Organized family planning activities began in Nigeria in 1958, when the Lagos medical officer of health offered contraceptives to postpartum women at his regular maternal welfare clinic.3 The national government's involvement in family planning services began with its Fourth National Development Plan of 1981-1985, and that involvement expanded with the release of the 1988 population policy.4
Government primary health care centers currently dispense one-third of family planning services in Nigeria. Primary health care, including family planning, is administered by 589 local government authorities, reflecting efforts to decentralize social services and development programs. According to the 1990 NDHS, the private commercial sector (private hospitals and clinics, pharmacies and medical stores) is the source for 47% of all modern method use in Nigeria.
Nigeria relies on public-sector service statistics, such as those provided by the Ministry of Health, and on surveys such as the 1990 NDHS to monitor and evaluate the performance of its national population program. Although the government coordinates data collection efforts with the Planned Parenthood Federation of Nigeria, limited information is collected on the private commercial sector. This lack of the information needed to monitor the expansion of family planning services among private providers led the Government of Nigeria, with financial and technical support from the U.S. Agency for International Development (USAID), to develop a low-cost approach for assessing program coverage in both the public and private sectors.
The Assessment Tool
The federal family planning module was designed by Nigeria's Federal Office of Statistics. The one-page questionnaire is based on the 1990 NDHS. Because most of the questions have precoded responses, the module is easy to administer, and the data are simple to code and enter.
The module was first used in Nigeria in 1992 and 1993, when it was attached to selected rounds of the National Integrated Surveys of Households. (These surveys are administered monthly to monitor household economic activity, housing, health status, electricity supply and education.) In 1992, it was attached each quarter to the household questionnaires, which survey more than 6,000 households, at administrative costs of about $5,000 per quarter. Costs were low because the tool was added to routine surveys that have their own operating budget. The frequency with which the module was administered was reduced in 1993 because planners felt that information needs could be met if data were collected just twice each year.
The 1992 modules collected information on the background characteristics of all household members, including their age, sex, relationship to the head of household, marital status, educational attainment and labor force status. Questions gauging knowledge of family planning were asked of all individuals aged 15 and older. Moreover, women aged 15 and older, regardless of marital status, and nonpregnant married women younger than 15* were asked if they were currently using a method of family planning. Current users were asked about their method and where they obtained it.
In 1993, the questionnaire was expanded to allow closer monitoring of the information, education and communication programs in family planning.Ý Men and women were asked whether they had heard a family planning message in the last six months, and the source of that message. In addition, a section was added to assess knowledge of AIDS and of condom use.
Full questions are not included on the one-page monitoring tool. Instead, they are listed in the Interviewer's Manual, along with instructions on the appropriate way to ask each question. Codes that do not appear on the questionnaire are also included in the manual. To keep the questionnaire to one page, detailed information on the number of interview visits, the interview length and data on the completion of the questionnaire are located on the first page of the larger survey.
About 120 female interviewers (roughly four interviewers per state) and 30 field supervisors were trained to administer the instrument. Field staff received five days of initial training and one day of retraining between rounds of the survey. In those states where most of the existing resident interviewers were male, additional female interviewers were recruited and trained to conduct the survey. (While women interviewers could interview both male and female respondents, male interviewers asked questions of male respondents only.)
A questionnaire manual was used in training as well as in the field. Interviews were conducted in private. While the actual questionnaire and manual were printed in English, the interviews were conducted in local languages, if needed. Repeat visits were made to locate and interview each eligible adult respondent.
Three months were allocated for collecting and processing the data and completing the reports, although the actual process took about six months for each survey, because insufficient time had been allocated for the numerous repeat visits required to locate respondents. In addition, data processing was slowed considerably because of electrical outages on a daily basis. There was also some unexpected turnover among the computer staff. Once they had gained skills in data processing and analysis, many obtained higher paying jobs in the private sector and within the donor community.
The sample selection process for the National Integrated Surveys of Households, including the one-page family planning survey, was part of the United Nations National Household Survey Capability Program. Samples for most surveys in Nigeria are drawn from a list of enumeration areas created in preparation for the 1973 census.5 Sample selection for the household surveys was implemented in phases, however, because of possible changes in the size of those areas since 1973.
The first phase involved selecting a master sample of enumeration areas and determining the size of each. Throughout its history, Nigeria has increased the number of states to meet the demands of its diverse population; thus, the number of enumeration areas, which is tied to the number of states, has changed over time. For the December 1992 round of the survey containing the family planning module, a random sample of 645 enumeration areas was selected (30 enumeration areas for each of the 21 states in that year and 15 for the Federal Capital Territory).
In the second phase, about 10 households were systematically chosen for each enumeration area—overall, 6,450 households. In both phases, the probability of selection was proportional to population size. However, because of political problems in four states—Anambra, Akwa Ibom, Benue and Kwara—the actual December 1992 survey included fewer enumeration areas (516) and households (5,726) than originally planned.
In the short period between the 1992 and 1993 modules, Nigeria increased its number of states from 21 to 30. Thus, the number of enumeration areas selected for the 1993 sample increased to 915 (30 enumeration areas for each of the 30 states and 15 for the Federal Capital Territory), and the total number of households likewise increased to 9,150. However, the actual number of enumeration areas (793) and households (8,073) in the December 1993 survey was lower than expected, because of shortfalls in another four states—Edo, Ondo, Rivers and Yobe. Political problems resulting from the failed elections of 1993 may have influenced interviewers' ability to visit all of the selected enumeration areas.
REPRESENTATIVENESS AND RELIABILITY
A key concern in administering the tool is whether it can collect representative data that can be used to assess program performance. Another concern is the instrument's ability to provide reliable family planning information during the 4-5 years between comprehensive national contraceptive prevalence surveys.
Unfortunately, because the last published census reports for Nigeria are based on the 1963 census (the 1973 census was never published), we could not use census data to determine whether the module respondents were representative of the population of the entire country. Although the most recent census was carried out in 1991, published reports on it are not yet available. Thus, to assess the representativeness of the data from the family planning modules, we relied on comparable data from the 1981-1982 Nigeria Fertility Survey (NFS),6 which was administered as part of the World Fertility Survey program, and from the 1990 NDHS.
Table 1 summarizes selected background characteristics for all women aged 15-49 who were interviewed in the NFS, the NDHS and the 1992 and 1993 family planning modules. Overall, only modest differences emerged among the key background characteristics. Age distributions among respondents were roughly similar across all four surveys, although a slightly higher proportion of women in the 1992 and 1993 surveys than in the earlier surveys were older than age 35. This slight difference could influence the contraceptive use data, since older women who have completed their families may have higher rates of contraceptive use for limiting childbearing.
Comparisons of marital status also show differences among the surveys. A higher percentage of women were currently married in the 1981-1982 and 1990 surveys (80% and 78%, respectively) than in the two more recent surveys (73%). However, both earlier surveys included women living with partners as "currently married," while the later surveys used the National Integrated Survey of Households categorization, whereby such women would be classified as "never married."
We expected educational differences to emerge among the surveys, given the strides made in education over the decade. In 1981-1982, for example, 66% of the women had received no formal education and only 11% had completed at least a secondary education; by 1993, however, the percentage with a secondary education had more than doubled, to 25%. The differences over time probably reflect efforts by government and parents to educate females. (In most Nigerian states, the government has introduced fines and other penalties for parents who take their daughters out of school.)
To approximate the urban-rural distribution of the 1990 NDHS sample, we had applied weights to the Federal Office of Statistics surveys; the proportions living in urban or rural areas were similar in these three surveys, as expected. The higher proportion of women living in urban areas in the 1981-1982 NFS, however, is difficult to explain. We could not compare regions of the country because of the increase over time in the number of Nigerian states.
Reliability refers to the likelihood that a given measurement procedure will yield the same description of a given phenomenon if that measurement is repeated. To ensure that the tool measured the same family planning indicators as the prior surveys, the 1992 and 1993 surveys used wording identical to the 1990 NDHS for questions on current use of family planning and on the source of the last method: "Are you currently using any method to avoid getting pregnant? Which method are you using? Where did you obtain [method stated as response] last time?" (Similar questions were also asked in the 1981-1982 NFS, but only of women currently in union.)
There are clear differences between the 1990 NDHS and the Federal Office of Statistics surveys in the questions that collected information on knowledge of family planning, however. These differences reflect the different aims of the surveys. The 1990 survey was designed to obtain information on the level of knowledge of specific family planning methods, while the latter surveys were designed to monitor ongoing strategies for reaching different segments of the population with family planning messages.
To keep the tool as short as possible, the respondents initially were asked one question on family planning knowledge: "Have you ever heard about family planning, the various ways or methods that a couple can use to delay or avoid a pregnancy?" This knowledge question was revised in 1993 to read: "Have you ever heard of family planning methods? If yes, which methods? Have you seen or heard a family planning message in the last six months? If yes, where?"
WHAT CAN THE TOOL MONITOR?
Trends in Contraceptive Use<
In Nigeria, the new survey tool played a critical role in documenting the rapid expansion of family planning use, while providing insights into the market for family planning in the public and private sectors. The data shown in Table 2 demonstrate the instrument's ability to track trends in contraceptive use. Use of any method among all women in their reproductive years appears to have doubled in the two years separating the NDHS and the administration of the family planning module in December 1992, going from nearly 8% to 16%. The proportion rose in 1993, to reach an overall contraceptive prevalence rate of 21%.
The proportion of all women currently using a modern method also doubled in the same short period, increasing from 4% to 9%. Further, by 1993, almost 11% of 15family planning. Such increases would not have been revealed until a new national fertility survey was conducted in Nigeria.
IDENTIFYING THE MARKET
The family planning module also provides insights into the market for contraceptive services and supplies, by generating data on preferences for different methods. Table 2 clearly shows that among currently married women, the pill is the most commonly used modern method, followed by the IUD and injectables.
From 1990 to 1993, however, the use of traditional methods increased substantially. Reliance on periodic abstinence, which is used in Sub-Saharan African societies both to space and to limit births,7 increased from nearly 6% of married women in 1992 to 8% in 1993. The political disturbances during 1993, which led to economic uncertainty for many Nigerians, probably influenced the decision to use contraceptives (both traditional and modern methods).
The recent family planning surveys also help document trends in contraceptive use and method choice among single women. Current use of a modern method among unmarried women almost tripled between 1990 and 1992, increasing from 5% in 1990 to 14% in 1992, and nearly reaching 16% by 1993. A rise in condom use, most likely to protect against AIDS, contributed to the increase in modern method use.
The family planning module also provides a comprehensive picture of individuals' sources of modern contraceptives. Such data indicate that the private commercial sector plays a major role in the provision of pills and condoms in Nigeria (Table 3): In 1993, the private sector was the source of supply for 58-59% of all pill and condom users. On the other hand, government facilities play a key role in the provision of long-term methods, since they are the source for 46-47% of IUDs and injectables. While the Planned Parenthood Federation of Nigeria provides a broad mix of methods, it primarily offers long-term methods. In 1993, it supplied nearly one-third of IUDs and injectables.
The tool also demonstrates shifts in the source of providers over time. For example, from 1992 to 1993, some of the provision of services and supplies shifted from government sources to the Planned Parenthood Federation of Nigeria, whose overall contribution nearly doubled (from 7% to 13%). There were also substantial increases in the organization's share of the total provision of injectables, and smaller ones for the pill and the condom.
This transformation is probably related in part to the failure of democratic elections in Nigeria in 1993, after which the donor community began transferring financial support from the government to the private sector, including the Planned Parenthood Federation of Nigeria. In that year, the federation initiated efforts to establish 57 new family planning clinics, beginning with nine new clinics in 1993. It also started to expand the method mix it offered, paying particular attention to injectables, a highly effective method that confers a high degree of privacy. Combined with the closing of government hospitals during political strikes, this expansion of services may account for the change in market share over such a short period.
Table 4 demonstrates how the tool assesses ways to reach men and women with family planning messages, showing the proportions of respondents to the 1993 survey who had heard or seen a message in the past six months, by gender and residence, and by the source of that information. Equal proportions of men and women had been the recipients of a family planning message in the past six months (about 61% each). Among urban men, radio was the most frequently cited source of messages (42%), followed by television (21%). In rural areas, radio programs appear to have reached more than half of the male population.
The sources for messages differed among women, however. Radio was the source of fewer than one-third of messages (32%), followed by government facilities (28%). These findings suggest different strategies are needed to reach women, especially rural women. Program planners and managers can coordinate efforts with women's peer groups, religious institutions and schools to disseminate family planning information in rural areas. Future rounds of the Federal Office of Statistics surveys will help program managers monitor the performance of alternative dissemination strategies for all segments of the population.
The substantial increases in contraceptive prevalence seen in Nigeria between 1990 and 1993 are consistent with experiences in other Sub-Saharan African countries. In Tanzania, for example, use of any method also doubled in two years, from nearly 10% in 1991-1992 to 18% in 1994, and use of a modern method increased from 6% to 12%.8 Current use of a modern method among married women in Ghana also increased, from 5% in 1988 to 10% in 1993.9 Kenya too experienced a rapid rise in the use of modern methods in a short period of time, with modern method use rising from 18% in 1989 to 27% by 1993.10
John Caldwell provides some observations and possible explanations for the increases in contraceptive use in Nigeria. According to his study of fertility among women aged 18 years and older in the Nigerian city of Ado-Ekiti, contraceptive use was widespread among all age-groups, and it was more common among unmarried women (75%) than among married women (43%).11 Among married women using a modern method, a high proportion relied on the pill, followed by injectables and the IUD. For single women, use of the condom was more likely than use of the pill. While married women obtained services and supplies from both the public (41%) and private sectors (59%), about 85% of unmarried users obtained services and supplies from the commercial sector, because they felt it offered them more privacy than the public sector.
Caldwell concluded that Nigeria's 1988 population policy played a key role in raising demand and supply for family planning, by increasing government efforts to provide services and educational programs on family planning, and assuring Nigerians (women in particular) that family planning was acceptable behavior. Further, the policy changed the environment regulating the importation of family planning commodities, making it easier and more profitable for private wholesalers to obtain inexpensive contraceptives to meet increased consumer demand.
Much of the increased support for commodities came from the donor community. The yields from social marketing efforts supported by USAID increased from a little over 100,000 couple-years of protection in 1990 to more than 500,000 couple-years of protection in 1993.12 The donor community also played a major role in financing and providing technical support of mass media programs. Music videos, short dramas on radio and television, newspaper articles and other forms of print media were used to gain acceptance of the policy and of family planning programs.13
LIMITATIONS OF THE TOOL
The tool that we describe here is not a substitute for Demographic and Health Surveys, comprehensive contraceptive prevalence surveys, management information systems, service statistics or other methods of assessing family planning programs. Rather, it is a low-cost way of extracting reliable information to guide routine decision-making.
Because the monitoring tool collects information on a few selected questions only, it is limited in its ability to examine key factors that influence contraceptive decisions. For example, as currently constructed, the instrument does not examine the attitudes of men and women toward family planning or the reasons why couples are not currently using a method, nor does it ask about current family size. It also cannot tell us why people choose not to practice family planning. The module can easily be adjusted to gain insights into nonuse of family planning, simply by inserting the word "why" after the question on current use of family planning. This minor adjustment would provide program managers with women's reasons for using a method as well as for not using one; precoded responses can be developed for this addition.
The decision to practice contraception can also be influenced by parity. Unfortunately, the tool does not provide information on this key attribute. Adding a simple question on the number of living children would help managers determine if women are using appropriate methods during given stages of their reproductive lives.
The monitoring tool, like most contraceptive surveys, is cross-sectional, and thus does not yield insights on method and source switching. At present, we have no information on the proportion of current users who are switching from public clinics to private-sector outlets, or from traditional to modern methods. Introducing a longitudinal component to the design would allow observation of method and source switching, since the behavior of the same women could be tracked over time. Unfortunately, a longitudinal study would increase the costs of the survey, create new problems in data collection and analysis, and potentially discourage collaboration with offices that conduct development surveys. However, for those governments that are implementing programs that encourage couples to switch from government services to the private sector, a longitudinal approach would help them monitor the effectiveness of their efforts.
The kind of short survey module described in this article can easily be adapted and applied in other countries that regularly conduct national surveys to monitor development activities. The one-page questionnaire can be attached to these surveys annually or biannually, with minimal costs.
However, to gain the interest and cooperation of statistical offices that conduct development surveys, the survey instrument should be kept as brief as possible. Such agencies are more likely to accept a simple one- or two-page questionnaire than an instrument that places heavy administrative demands on staff. We also suggest that it include questions that are of high priority to units responsible for implementing family planning programs.
In Nigeria, the selection of questions involved a team effort between management units and staff involved in monitoring and evaluation efforts. Senior information, education and communication staff and coordinators of commercial-sector activities, as well as those involved in implementing services through government clinics, worked closely together to select the types of questions that could provide critical information to guide management decisions.
To obtain the maximum benefit from the tool, it probably should first be conducted soon after (or just before) a large-scale contraceptive prevalence survey, so that similarities and differences in the two data collection procedures can be identified; understanding the differences that might emerge will be useful in interpreting the results of subsequent rounds of the survey.
Lastly, we suggest selecting an agency with experience in conducting surveys in both rural and urban areas and with a reputation for providing high-quality information. A key feature of the Federal Office of Statistics is that it has resident staff in both rural and urban areas, and its rural-based interviewers have years of experience working in isolated communities. Other advantages in working with the agency include its ability to process and provide quick results to family planning managers.
In most nations, family planning services are dispensed through a variety of service delivery channels in the public and private sectors. While management information and service statistics help monitor government services and the activities of private volunteer family planning associations, strategies do not exist to provide annual or biannual information on the commercial sector. National contraceptive prevalence surveys provide data to monitor and evaluate the total service and supply environment, but the costs of conducting these surveys annually are prohibitive.
The one-page questionnaire described in this article can be attached to routine development surveys. In Nigeria, this tool proved valuable in documenting substantial increases in contraceptive prevalence. It also provided information on the total market for family planning, including government outlets, alternative commercial providers and services provided by the volunteer family planning association.
The tool also proved useful in Nigeria in identifying gaps in coverage, in monitoring the dissemination of population messages, and in strategically guiding decisions about resource allocation. Thus, in countries that conduct national development surveys on a regular basis, such a tool is a viable and inexpensive way to monitor the expansion of family planning services and supplies between years in which large-scale contraceptive prevalence surveys are conducted.
*In Nigeria, and especially in the Islamic regions of the country, marriages among girls younger than 15 are common, although the practice is slowly declining. In recent years, the government has intervened by penalizing parents who take a daughter out of school for an arranged marriage.
1. J. Cleland, "The Adequacy of Service Statistics Systems for Programme Monitoring, Evaluation and Research," in Monitoring and Evaluating Family Planning Programmes in the 1990s, United Nations Economic and Social Commission for Asia and the Pacific, Asian Population Studies Series, No. 104, 1990, pp. 75-97; and A. Keller, "Management Information Systems in Maternal and Child Health/Family Planning Programs: A Multi-Country Analysis," Studies in Family Planning, 22:19-30, 1991.