Are Primary Care Residents Adequately Prepared To Care for Women of Reproductive Age?
A 1991 study of 115 internal medicine and 28 family practice residents at a large inner-city public hospital finds that both groups would perform poorly in providing preconception counseling to women of reproductive age. More than 40% of residents failed to indicate that they would provide a healthy woman with information on rubella immunization and family planning or counseling on sexually transmitted diseases and safer sex. When counseling a diabetic woman seeking pregnancy, 74% would not have discussed congenital anomalies with her and 45% would not have considered discontinuing oral hypoglycemics if she became pregnant. Furthermore, 58% would have neglected to review or change hypertension medications in a newly diagnosed pregnant woman. Although both internal medicine and family practice residents had positive attitudes toward offering preconception care, family practice residents had significantly higher attitude scores. No clear improvement was found in patient management, attitude or knowledge scores as residents progressed from their first to their third year of training.
(Family Planning Perspectives, 27:66-70, 1995)
Preconception care is defined as "health care before conception which may optimize the outcome of a given pregnancy."1 Expert medical opinion agrees that achieving good health before conception helps women to have healthier pregnancies with fewer complications. Appropriate counseling and medical management prior to conception have been shown to decrease congenital anomalies, neonatal complications, spontaneous abortions and cesarean section rates.2
With regard to infant mortality, the United States currently ranks 24th in the world, with a rate higher than those of many other developed countries.3 This situation and the disparity between black and white infant mortality rates (17.7 deaths per 1,000 live births and 8.2 deaths per 1,000 live births, respectively) remain among the nation's most urgent public health problems.4 One of the national health objectives for the year 2000 is to reduce the infant mortality rate for the total population to less than seven infant deaths per 1,000 live births—and, for the black population, to less than 11 per 1,000.5
Preconception care is a recommended part of primary care as a routine preventive intervention for women of reproductive age.6 The continuity of care and the close physician-patient rel ationship in primary care offer an opportunity for the physician to assess risk factors and to intervene in order to modify behaviors that increase perinatal mortality and morbidity. All women of reproductive age presenting for continuing care in primary care settings are candidates for preconception care.
Data from the National Ambulatory Medical Care Survey show that 417,496,000 office visits for medical care services were made by women in 1989, and that 173,175,000 of these were made by women aged 15-44.7 Primary care physicians such as family practitioners and internists, along with obstetrician-gynecologists and women's health centers, are the usual health care providers for women of reproductive age, but little evidence exists that they routinely take advantage of the opportunity to provide preconception care. Obstetricians' lack of training in preconception care has previously been noted.8 However, a MEDLINE computer search of the primary care literature from 1966 to the present failed to reveal any specific descriptions of preconception care or care of the newly diagnosed pregnant woman in primary care training.
Residency programs in the primary care specialties should be expected to include and to emphasize preconception care in the training of house staff. This expectation should be greater in hospitals that serve large numbers of poor and minority women, since the women most likely to benefit from preconception care, such as black women,9 are often those least likely to have access to it.
Cook County Hospital is one of the largest inner-city public hospitals in the nation. The infant mortality rates of the communities in which its patients live rank among the worst in the nation.10 Residents at Cook County Hospital are trained in the primary care of adults as part of two major programs—one in family practice and one in internal medicine. Family practice residents receive most of their obstetric training in obstetric and gynecologic clinical situations, where opportunities for preconception care or new diagnoses of pregnancy are infrequent. The training of internal medicine residents involves minimal contact with pregnant patients and is limited primarily to medical consultations. Neither program has developed curriculum topics or clinical experiences that specifically address preconception care within a primary care context.
The purpose of this study was to assess and compare the attitudes, knowledge and management skills of internal medicine and family practice residents in caring for women of reproductive age. We sought to accomplish this by evaluating the abilities of the resident trainees to reduce possible risks of adverse pregnancy outcomes for such women. We hypothesized that family practice residents would perform better than internal medicine residents with regard to preconception care because the former receive more obstetric training, and that as training progressed, senior residents would perform better than junior residents.
Our study emphasizes behavior modification and health promotion through education, and it focuses particularly on hypertension and diabetic care for women of reproductive age. Social and environmental factors also greatly affect pregnancy outcomes, but because of the difficulty of measuring them they are not included here. This study is intended as a preliminary examination of the field and as a reference to help revise residency training curricula in our hospital.
SAMPLE AND DESIGN
Residents from internal medicine and family practice programs at Cook County Hospital were asked in October 1991 to answer a specially designed, anonymous, self-administered questionnaire. Attached to the questionnaire was a description of the study explaining its purpose, assuring the confidentiality of responses and stating that participation was voluntary. To enhance the data collection and avoid bias in the results, questionnaires were delivered to residents at their work sites (clinics, wards and conference rooms, among others), where completion was monitored. Over a two-week period, 115 out of 140 (82%) residents in internal medicine and 28 out of 42 (67%) family practice residents completed the questionnaire. Demographic characteristics of nonrespo ndents were not collected.
INSTRUMENTS AND MEASUREMENTS
The questionnaire was developed from current literature,11 including recommendations for preconception care from the Guide to Clinical Preventive Services of the U.S. Preventive Services Task Force, 12 from the American College of Obstetricians and Gynecologists13 and from the U.S. Public Health Service Expert Panel on the Content of Prenatal Care.14 It was separated into three scaled components that assessed residents' management decisions and their knowledge of and attitudes toward preconception care. Questions focused on the primary care of women of reproductive age prior to conception, but they also included as a subtheme the care of women who have a newly diagnosed pregnancy.
The content of the three scales was evaluated by a panel of nine attending physicians—three internists, three family practitioners and three obstetricians—on the hospital staff, who reviewed and made recommendations on the questionnaires. Content validi ty was evaluated based on Nunnally's psychometric theory, and was quantified after a revision that incorporated the attending physicians' recommendations.15 The content validity index on the revised instruments was calculated to be 1.00 for the knowledge scale, 0.97 for the management scale and 0.89 for the attitude scale. Because of the small sample size, we were not able to assess the construct validity, and criteria validity also was not assessed. No similar instrument had been developed before this study.
A management decision score was constructed based on residents' open-ended responses to questions concerning the management of three hypothetical patients, and was scored against a list of specific items developed from published recommendations (see Table 1).16 The case examples consisted of a nonimmunized teenage girl who has a new boyfriend but is not currently sexually active (Case 1), a diabetic woman on oral hypoglycemic agents who is contemplating pregnancy (Case 2) and a chronic hypertensive woman who smokes, is on medication and has received a positive pregnancy test (Case 3). These cases were chosen as typical of patients likely to be encountered by residents during their training and also representative of patients who might be seen in the normal office practice of a primary care physician.
|Table 1. Three hypothetical patients and recommended management decisions for care of women of reproductive age, from questionnaire administered to primary care residents at Cook County Hospital, 1991|
|A 17-year-old Honduran female presents for her first primary care visit. She has never seen a physician nor been medically treated in her life. Her medical and social history is completely negative. She has never been sexually active but has a new boyfriend. You are assigned to be her primary physician. Would you perform any diagnostic/laboratory tests, perform any counselling, or provide immunizations if she was one of your regular patients in the last year? If so, please list.||STD/safe-sex counseling||Rubella screening/immunization||Family planning|
|The patient is a 30-year-old obese female diabetic who has been treated with glipizide 10 mg by mouth every morning for two years. She states that she is considering pregnancy. She asks you to tell her the risks of pregnancy and what will help assure the best pregnancy outcome for her. What would you tell her?||Inform her that her risk of congenital anomalies is increased||Discontinue oral hypoglycemics||Increase glucose control (more visits, chemstrip, hospitalization if needed)|
|The patient is a 33-year-old African American female whose medical history is notable for hypertension of two years' duration, treated with Nifedipine and HCTZ. Her medical history includes two obstetric hospitalizations for normal deliveries. She smokes 1/2 pack per day, uses neither alcohol nor drugs, and works as a supermarket cashier. She presents to you as a new patient and states that she thinks that she is pregnant. Her last menstrual period was 10 weeks ago and a urine pregnancy test is positive. You have a 1/2-hour office visit scheduled: Describe the most important things you would do after you have completed taking her history.||Check blood pressure||Measure fundal height||Advise to stop smoking||Consider changing blood pressure medications||Prescribe prenatal iron & vitamins||Check urine protein & glucose||Refer to prenatal clinic|
The knowledge score consisted of a total of 46 items in a true-false format (see Table 2, page 68). The questions were predominantly focused on clinical knowledge of what might affect the pregnancy outcome of female patients with diabetes or hypertension. Since some items called for negative points to be scored if the respondent answered incorrectly, total knowledge scores could be positive or negative.
|Table 2. True-false questions from knowledge scale and statements from attitude scale administered to residents|
|Question or statement|
|The following drugs are safe for all trimesters of pregnancy except:|
|The following drugs are safe in the third trimester of pregnancy:|
|Indicate whether the following items concerning pregnancy and blood pressure are true:|
Pregnancy lowers blood pressure in the first trimester.|
Pregnancy raises blood pressure in the second trimester.
Hypertensive pregnant women have a higher incidence of preeclampsia.
Hypertensive pregnant women have an increased incidence of early spontaneous abortion.
Hypertensive women who become pregnant rarely have spontaneous normalization of their blood pressure during pregnancy.
|You discover that your patient with noninsulin-dependent diabetes mellitus is pregnant. Select from the list below the most important things you should do:|
Refer to dietitian for review and recalculation of America Diabetes
Begin insulin therapy immediately.
Check hemoglobin A1C.
Attempt to normalize body weight (if patient is obese).
Review urine glucose-monitoring techniques.
Discontinue oral hypoglycemics if patient has long-standing diabetes.
Obtain baseline fluorescin angiogram, if patient has long-standing diabetes.
Instruct patient to obtain a prenatal appointment within six weeks.
Obtain stat serum bicarbonate to rule out diabetic ketoacidosis in pregnancy.
Offer genetics counseling to the patient.
|The diabetic patient asks you for advice and information regarding pregnancy and diabetes. You should tell her the following things:|
She should see a geneticist for specialized counseling.|
She is at high risk of developing new or worsening retinal disease. The likelihood of fetal anomaly is related to glucose control at conception.
Her diabetic nephropathy is likely to worsen permanently.
She should continue home urine-monitoring.
Her chance of having a baby without anomalies exceeds 75%.
|Congenital anomalies in offspring born to diabetic women...|
affect more than 25% born to diabetics.|
are the primary cause of perinatal mortality in this group.
commonly affect the gastrointestinal system.
may be screened for with maternal hexosaminidase B.
include an increased incidence of trisomy 25.
|Intensive home glucose-monitoring...|
should be initiated prior to conception in diabetic women.|
is advisable in a known diabetic but usually not necessary in women with gestational diabetes.
is less important in the third trimester of pregnancy than earlier.
may be avoided in pregnancy if frequent urine glucose determinations are made.
is only used if patients are on insulin therapy.
|Rubella infection in a nonimmune pregnant woman...|
can cause microcephaly.|
is a common cause of deafness in infants exposed in utero.
should be prevented by administration of the measles-mumps-rubella vaccine.
is associated with patent ductus arteriosis in exposed infants.
only causes fetal problems, as the exposure occurs in the first trimester.
Improving pregnancy outcomes is an important part of internal medicine
I feel I have sufficient knowledge to minimize poor pregnancy outcomes in patients I treat.
Internists' actions do NOT have an important effect on the health of newborn babies.
I especially like treating female patients ages 18-45.
I feel comfortable suggesting birth control to patients.
I feel comfortable counseling patients that delaying or avoiding pregnancy is the right clinical decision for them.
When I am in practice following my training, I believe that I will be in a position to help prevent poor pregnancy outcomes.
I feel that I adequately understand the effects of medications and treatments on a pregnant female.
The attitude statements were ranked on a four-point Likert scale (ranging from 1=strongly disagree to 4=strongly agree) that measured the residents' perceptions of the importance of preconception care and its role in improving pregnancy outcomes (Table 2). Through these statements, we also sought to assess the residents' comfort in providing care to women of reproductive age and their confidence in their own knowledge and management ability to adequately provide preconception care. Higher scores reflect amore positive attitude.
Kruskal-Wallis one-way analysis of variance was used for comparison of three or more groups. The Wilcoxon Rank Sum Test was performed for two-group comparison. A descriptive summary analysis was also performed. Statistical analysis was based on a nonparametric method, because of the relatively small sample size of family practice residents.
SOCIAL AND DEMOGRAPHIC CHARACTERISTICS
Table 3 (page 69) lists the social and demographic characteristics of the study subjects. Of the 115 internal medicine residents, 75% were male, and the same proportion had graduated from medical school more than five years previously. Thirty-one percent were in postgraduate year one, 45% were in postgraduate year two and 24% were in postgraduate year three. In contrast to internal medicine residents, family practice residents were more likely to be female (58%) and were more likely to have graduated from medical school fewer than five years before (64%). The median age for both groups was 30; internal medicine residents ranged in age from 23 to 47, and family practice residents ranged from 26 to 48.
|Table 3. Percentage distribution of medical residents, by social and demographic characteristics, according to area of specialization|
|Characteristic||Internal medicine||Family practice|
|No. of years since medical school graduation|
|*Sample sizes vary because of missing values.|
In general, both groups had low scores on questions of knowledge. Internal medicine residents' median knowledge score was 5.0 (out of a possible 18.0), while the family practice residents' median knowledge score was 8.5. Management scores for both groups were also low; out of a possible score of 14.0, median scores were 6.0 for internal medicine residents and 7.0 for family practice residents. In contrast, attitude scores were high for both groups; against a perfect score of 28.0, medians were 22.0 for internal medicine residents and 25.0 for family practice residents.
Table 4 shows the responses of internal medicine and family practice residents to selected questions concerning elements of preconception care and the care of the woman immediately after diagnosis of pregnancy. Overall, both groups showed a relatively low proportion of correct responses to management questions regarding risk reduction, health promotion and medication use during the preconception period or early in pregnancy.
|Table 4. Percentage of medical residents who would properly advise patient on selected elements of preconception care, by specialty|
|Element||Internal medicine||Family practice|
|Rubella screening and immunization||58||50.4||12||42.9|
|STD and safe-sex counseling||73||63.5||9||32.1|
|Discontinue oral hypoglycemic||64||55.7||15||53.6|
|Change hypertension medications||41||35.7||18||64.3|
For the nonimmunized young woman with a new boyfriend (Case 1), members of each group recommended rubella immunization and family planning only about half of the time. Fewer than one-third of residents from either group would specifically have included a discussion of congenital anomalies when counseling a diabetic woman who was seeking advice about the risks of a contemplated pregnancy. Furthermore, only about half of all residents mentioned the need to discontinue oral hypoglycemic agents during pregnancy. Similarly, only 36% of internal medicine residents reviewed and considered changing medications for the hypertensive woman who had been diagnosed as pregnant, compared with 64% of family practice residents. In both groups, residents were likely to advise a pregnant patient to quit smoking.
Table 5 stratifies the management, attitude and knowledge scores by the residents' number of postgraduate years. No significant differences were found for management scores between family practice and internal medicine residents at any postgraduate level. However, attitude scores were significantly higher for family practice residents than for internal medicine residents in postgraduate years one (24.0 vs. 22.0, p=0.0076), two (25.0 vs. 20.0, p=.0038) and three (25.0 vs. 21.0, p=0.0003). Family practice residents also scored statistically higher in knowledge at postgraduate year two (8.0 vs. 4.0, p=.0379), but not in postgraduate years one or three.
|Table 5. Median (and range) of scores on management, attitude and knowledge scales, by postgraduate year of medical residents and whether they had been rotated through the high-risk prenatal clinic, according to specialty|
|Scale, year of training and type||Internal medicine||Family practice||Wilcoxon rank sum test*|
|Management||36||7.0 (3-10)||8||5.5 (3-12)||-1.44||.1510|
|Attitude||35||22.0 (17-27)||8||24.0 (22-27)||-2.67||.0076|
|Knowledge||36||5.0 (0-13)||8||6.5 (3-14)||-1.18||.2393|
|Management||52||6.0 (1-13)||9||8.0 (3-11)||-1.50||.1338|
|Attitude||48||20.0 (13-28)||7||25.0 (22-28)||-2.89||.0038|
|Knowledge||52||4.0 (-4-13)||9||8.0 (2-12)||-2.08||.0379|
|Management||27||7.0 (2-11)||11||8.0 (2-12)||-1.32||.1863|
|Attitude||26||21.0 (14-27)||10||25.0 (22-28)||-3.58||.0003|
|Knowledge||27||7.5 (3-12)||11||9.0 (2-13)||-1.54||.1241|
|Management||8||8.0 (6-13)||14||8.5 (2-12)||-0.48||.6290|
|Attitude||8||24.0 (19-27)||11||25.0 (22-28)||-0.59||.5582|
|Knowledge||8||8.0 (4-12)||14||9.5 (2-14)||-0.86||.3916|
|Management||107||6.0 (1-11)||14||6.0 (3-9)||-0.54||.5889|
|Knowledge||107||5.0 (-4-13)||14||7.5 (2-12)||-1.89||.0590|
|*Z statistics calculated for this test were "corrected for ties" when the raw data gave the same value for two or more study participants. Two-tailed test.|
Score analysis stratified by whether a resident had been rotated through the high-risk prenatal clinic reveals that all scores were higher among the internal medicine and family practice residents who had such training than among those who did not, except for attitude scores among family practice residents (Table 5). No statistically significant differences were found between the scores of internal medicine and family practice residents who had rotated through the high-risk clinic.
Table 6 (page 70) shows no clear trend of improvement for residents as they advanced from postgraduate year one to postgraduate year three. The statistically significant differences among internal medicine residents for the knowledge score (p=.0045) and the management score (p=.0073) were caused by the extremely low scores among those in postgraduate year two. Attitude scores for residents in internal medicine did not differ significantly in postgraduate years one, two or three. Scores for family practice residents showed a trend of apparent improvement as they progressed from postgraduate year one to postgraduate year two and to postgraduate year three, but these differences did not achieve statistical significance.
|Table 6. Mean rank scores for management, attitude and knowledge scales among medical residents, by specialty, according to postgraduate year|
|Scale and specialty||Postgraduate year||Kruskal-Wallis one-way analysis of variance*|
|N||Mean rank||N||Mean rank||N||Mean rank||x2||p|
|*Corrected for ties.|
Our results suggest that primary care residents may be inadequately prepared by their training to take care of women of reproductive age. The internal medicine and family practice residents in our 1991 survey at Cook County Hospital often neglected to mention family planning, safer sex or sexually transmitted diseases in the information they would include when counseling such women. In addition, rubella immunization was not regularly addressed.
Preventing infant mortality and low birth weight, as well as congenital anomalies, is clearly of great benefit to individual patients and to society as a whole.17 Several chronic medical conditions and personal risk behaviors that adversely affect pregnancy outcomes can be identified and modified prior to conception, and primary care providers are in the best position to do this.18 Preconception care and early pregnancy care are excellent opportunities to modify the medical, social and behavioral risks on pregnancy outcomes and should be an integral part of primary care practice.
Recent epidemiologic studies show strong evidence that certain primary care interventions—most notably folic acid supplementation—undertaken prior to conception and continued into early pregnancy have a profound effect on the outcomes of pregnancy.19 These benefits to perinatal mortality and morbidity are not likely to be fully realized unless primary care physicians include preconception care as a routine intervention for all women of reproductive age in their practices.
Prepregnancy counseling shows promising results in improving pregnancy outcomes among women with chronic diseases.20 Intensive prepregnancy management of diabetes, for example, has been shown to improve outcomes in mothers and infants, and is clearly cost-effective.21 However, the knowledge and management choices in the preconception care of diabetics or hypertensive women by residents of both primary training programs that we surveyed were inadequate.
Pregnancies are often diagnosed by primary care physicians and afford an opportunity to counsel and advise a woman to modify risk behaviors that increase adverse pregnancy outcomes. In addition, medications and treatments may be changed, if necessary, to those safe for use in pregnancy. However, we have found that residents are not well-prepared to take advantage of this clinical opportunity; their management of medications that are unsafe in pregnancy was inadequate. Additionally, although the majority of residents said they would advise a smoker whom they diagnosed as pregnant to stop smoking, the case presented to them identified the woman as a smoker and specifically asked what counseling they might provide. In practice, a substantial number of residents might not have asked the smoking status of a woman with symptoms that lead to a diagnosis of pregnancy.
At the inception of this project, we hypothesized that family practice residents would perform significantly better than internal medicine residents, since at Cook County Hospital family practice residents receive standard obstetrics training. This hypothesis proved incorrect. For all three postgraduate years, family practice residents scored better than internal medicine residents only in their attitude toward preconception care, and no difference was found in the management score between these two groups of residents. Knowledge differed significantly between residents of the two training programs only at postgraduate year two.
Also, a clear improvement in scores was expected as residents' postgraduate levels progressed from year one to year three. The scores of neither group of residents showed significant improvement with increasing number of postgraduate years, however, and year-two internal medicine residents actually scored lower on knowledge and management than either year-one or year-three residents. In spite of scores that apparently reflected inadequate knowledge and management ability, both family practice and internal medicine residents perceived that this was an important and meaningful topic.
There are four components necessary for the successful practice of preventive health care, including preconception care—attitudes, organization, appropriate knowledge and management skills.22 The residents in this study seem to have had the necessary attitudes, but their residency training does not appear to have adequately provided the other three components. It is encouraging that a subgroup of residents (eight in internal medicine and 14 in family practice) who had rotated through the high-risk prenatal clinic scored higher; this group was relatively small and heterogeneous, however, and included both residents who had rotated through the high-risk clinic for at least a full month and those who may have had only passing contact there. Further prospective study of this experience might clarify the effect of such rotation.
This article represents an initial attempt to assess primary care residents' knowledge, attitudes and management decisions in caring for women of reproductive age in a large urban hospital. It is an exploratory study that needs independent replications from other institutions to validate the findings. The research also has certain limitations. The questionnaire was constructed solely for the purpose of this research; revision of this questionnaire may be warranted if it is to be used in other institutions. Also, the study was performed in two residency programs at the same hospital, and results may not be generalizable to other settings. Furthermore, because of the relatively small sample sizes, the statistical power to detect the significance of the results found is limited (ß: 0.37-0.80).
Infant mortality and poor pregnancy outcomes are among the leading health problems facing the United States today. The preconception and early pregnancy care skills of all relevant providers, including internists and family practitioners, should be adequate to prevent adverse pregnancy outcomes. Our research describes an attempt to evaluate whether primary care residency programs are succeeding in helping providers develop such skills. The results call for a revision of curricula and residency experiences in our hospital if residents are to be provided with the skills and knowledge necessary to meet existing recommendations.
We suspect that similar deficiencies could be discovered in other residency programs, since these topics are not widely discussed in the internal medicine and family practice literature. This topic is of great importance to primary care practitioners, their patients, and to society as a whole. Further research into the teaching of and delivery of preconception care is warranted.
2. J.A. Heslin and B. Natow, "Nutrition Needs for the Preconception Period," Occupational Health Nursing, 32:469- 473, 1984; D.G. Marrero et al., "Care of Diabetic Pregnant Women by Primary-Care Physicians: Reported Strategies for Managing Progestational and Gestational Diabetes," Diabetes Care, 15:101-107, 1992; D. Dicker et al., "Pregnancy Outcome in Gestational Diabetes with Preconceptional Diabetes Counselling," Australian and New Zealand Journal of Obstetrics and Gynecology, 27:184-187, 1987; J.L. Kitzmiller et al., "Preconception Care of Diabetes: Glycemic Control Prevents Congenital Anomalies," Journal of the American Medical Association, 265:731-736, 1991; D. Dicker et al., "Spontaneous Abortion in Patients with Insulin-Dependent Diabetes Mellitus: The Effect of Preconceptional Diabetes Control," American Journal of Obstetrics and Gynecology, 158:1161-1164, 1988; and B. Rosenn et al., "Preconception Management of Insulin-Dependent Diabetes: Improvement of Pregnancy Outcome," Obstetrics & Gynecology, 77:846-849, 1991.
3. Centers for Disease Control, Health, United States, 1990, Department of Health and Human Services (DHHS), Hyattsville, Md., 1991; and "Infant Mortality—United States, 1989," Mortality and Morbidity Weekly Report, 41:81-85, 1992.
4. National Center for Health Statistics, "Advance Report of Final Mortality Statistics, 1989," Monthly Vital Statistics Report, Vol. 40, No. 8, Supplement 2, 1992; and J.O. Mason, "Today's Challenges to the Public Health Service and to the Nation," Public Health Reports, 106:473-477, 1991.
10. Report of the Chicago and Cook County Health Care Summit: II. System Analysis and Design, Chicago and Cook County Health Care Action Plan, April 1990.
15.J.C. Nunnally, Psychometric Theory, 2nd edition, McGraw-Hill, New York, 1978, pp. 86-113; M.R. Lynn, "Determination and Quantification of Content Validity," Nursing Research, 35:382-385, 1986; and C.W. Waltz and R.B. Bausell, Nursing Research: Design, Statistics and Computer Analysis, F.A. Davis, Philadelphia, 1981.
16. B. Jack and L. Culpepper, 1990, op. cit. (see reference 1); DHHS, 1989, op. cit. (see reference 6); Report of the Chicago and Cook County..., 1990, op. cit. (see reference 10); and M. Fisher and C. Ecklhart, 1989, op. cit. (see reference 12).
19. M.A. Crawford, "The Role of Essential Fatty Acids in Neural Development: Implications for Perinatal Nutrition," American Journal of Clinical Nutrition, Vol. 57, Supplement, p. 703s, and discussion, p. 709s, 1993; M.M. Werler et al., "Periconceptional Folic Acid Exposure and Risk of Occurrent Neural Tube Defects," Journal of the American Medical Association, 269:1257-1261, 1993; and A.E. Czeizel and I. Dudas, "Prevention of the First Occurrence of Neural-Tube Defects by Periconceptional Vitamin Supplementation," New England Journal of Medicine, 327:1832-1835, 1992.
20. M.B. Willhiote et al., "The Impact of Preconception Counseling on Pregnancy Outcomes: The Experience of the Maine Diabetes in Pregnancy Program," Diabetes Care, 16:450-455, 1993; and M. Cox, "Prepregnancy Counselling: Experience from 1,075 Cases," British Journal of Obstetrics and Gynaecology, 99:873-876, 1992.
22. Preventive Health Care Committee, Society for Research and Education in Primary Care Internal Medicine, "Preventive Medicine in General Internal Medicine Residency Training," Annals of Internal Medici ne, 102:859-861, 1988.
Work for this paper was funded by the UK Department for International Development under the Knowledge Programme in Reproductive Health Policy and Practice at the London School of Hygiene and Tropical Medicine.