Despite recent declines, the United States still has one of the highest teenage pregnancy rates among industrialized nations.¹ While a growing number of programs have improved contraceptive prevalence or affected sexuality-related behaviors, few high-quality evaluations have documented programs' success in reducing teenage pregnancies and births, and even fewer have been able to delay the age of sexual debut. This article reports on the results achieved by a three-year, random-assignment evaluation of a Carrera-model teenage pregnancy prevention program.

The past 20 years have been filled with acrimony over how to best approach the problem of teenage pregnancy, but few successful strategies have emerged from this debate. Program evaluation has been sorely neglected and is frequently limited to measuring knowledge change or assessing intentions to remain abstinent. Moreover, many evaluations have lacked comparison groups, which has made it impossible to be sure that the programs themselves produced the observed outcomes.

What programs have been successful in reducing rates of teenage pregnancy? Two are early childhood or elementary school interventions. The Seattle Social Development Project used teacher training and parenting classes in elementary schools to increase children's sense of attachment to their school and family, while also increasing their social skills.² Some 18 schools were nonrandomly assigned to receive the intervention or not. By age 18, young people from the program schools were less likely to have had intercourse and also had lower pregnancy rates than those from control schools.

A second program, the Abecedarian project, randomly assigned children to receive interventions during preschool, elementary school, both or neither.³ Children in preschool received year-round, full-day child care from infancy through kindergarten, while those in the elementary school intervention worked with a home-school resource teacher whose objective was to increase parental involvement in the child's learning. The children in intervention classrooms had lower birthrates than children in the control group at age 21 and had delayed childbearing by more than one year.

Two communitywide projects have lowered teenage pregnancy rates. In one, pregnancy rates in a rural South Carolina county were tracked from 1977 to 1988 to detect changes created by an intervention that featured sexuality education training for school staff, classroom training in decision-making skills for students and the school nurse's providing transportation to a family planning clinic and dispensing condoms.⁴ Compared with another part of the county and with three similar counties, the intervention area had lower rates of teenage pregnancy; furthermore, these rates returned to previously high levels after the program ended. A second evaluation of this program described in more detail the interventions used to achieve these results.⁵

The other community-based intervention, Plain Talk, attempted to increase adults' communication skills in teenage sexuality issues and to motivate adults to encourage adolescent contraceptive use.⁶ Pregnancy rates among participants dropped over a three- or four-year period, but the research design did not include comparison communities or random assignment.

Clinic-based interventions have also proved useful. The Self Center, a Baltimore clinic, sent staff into a nearby high school to recruit teenage clients and to offer education and counseling. A 28-month follow-up showed that black female clients from the high school had lower pregnancy rates than black females attending two matched comparison schools.⁷

Educational and job-related interventions can be successful strategies to reduce teenage pregnancy rates and birthrates. The Youth Incentive Entitlement Pilot Projects offered part-time jobs during the school year and full-time summer jobs if participants stayed in school.⁸ A four-year study documented reduced birthrates among black female participants. The Conservation and Youth Service Corps offered work experience through community service, as well as remedial education.⁹ In this intervention, which used random assignment, 18-25-year-old-blacks experienced fewer nonmarital pregnancies than nonprogram blacks in the same age-group. Community service is also the main intervention for Teen Outreach, a yearlong curriculum and volunteer service program.¹⁰ That program, implemented in several cities, also used random assignment and achieved lower pregnancy rates among participating teenagers.

In addition to programs that have resulted in reductions in rates of teenage pregnancy, several interventions have effected changes in age at sexual debut and contraceptive use.¹¹ While evaluations of these projects did not document changes in pregnancy rates, progress in these two related outcomes should, at some point, affect pregnancies and births.

These evaluation results suggest that it is possible to reduce the incidence of teenage pregnancies and births, as well as delay sexual debut and increase contraceptive use. Several of these evaluations, however, did not track pregnancy rates and had relatively short follow-up intervals; moreover, only three that assessed the impact of programs on early pregnancy used true experimental designs (i.e., Teen Outreach, Abecedarian and Conservation and Youth Service Corps). Further, some evaluations had positive findings for only selected subgroups of participants, or tested the intervention on young people from one ethnic group only.

This article adds to the field by reporting the first findings from a longitudinal, random-assignment evaluation of teenage pregnancy prevention programs based on the Children's Aid Society (CAS)-Carrera model. The model focuses on reducing pregnancy, but uses a comprehensive youth development approach, coupled with sexuality education and contraceptive provision to those who become sexually active.

The CAS-Carrera Program

In 1984, the Children's Aid Society implemented a sexuality education and pregnancy prevention program for high-risk adolescents in Harlem. Michael A. Carrera (director of adolescent sexuality and pregnancy prevention programs at the agency) and colleagues designed and implemented the intervention, which is guided by the following principles: Staff treat children as if they were their own (parallel family system); each young person is viewed as pure potential; a holistic approach is used (incorporating multiple services to meet comprehensive interests and needs); contact with participants is continuous and long-term (i.e., through high school); services aim to involve parents and other adults; and services are offered under one roof in the community in a nonpunitive, gentle, generous and forgiving environment.

These principles infuse each of the program's seven critical parts—five activity components and two service components. The five major program activities are a work-related intervention called Job Club (with stipends, help with bank accounts, graduated employment experiences and career awareness); an academic component (featuring individual assessment, tutoring and homework help, PSAT and SAT preparation, and assistance with the college admissions process); comprehensive family life and sexuality education (weekly sessions emphasizing sexual knowledge given at age-appropriate and developmentally appropriate levels by an educator-reproductive health counselor); an arts component (designed to help young people discover and develop talent and confidence through weekly music, dance, writing or drama workshops led by theater and arts professionals); and an individual sports (as opposed to team sports) component that emphasizes activities requiring impulse control that can be practiced at all ages, such as squash, golf, snowboarding and swimming.

These five major activities are supplemented by two service components—mental health care (which includes counseling and crisis intervention, as needed, and weekly discussion groups led by a social worker) and medical care (which includes an annual comprehensive medical exam). Medical care is provided by the Mt. Sinai Hospital Adolescent Health Center; program staff schedule adolescents' appointments and accompany them on their visits. Reproductive health care offered through the center includes physical exams, testing for sexually transmitted infections, a wide range of contraceptive options (with condoms always being available) and counseling, as needed. If the health center refers a young person for specialty care, program staff follow up and help with accessing these services. The intervention also provides full dental care through the CAS dental clinic.

Throughout the school year, program activities run all five weekdays, generally for about three hours per day. Most program sites divide participants into 2-3 groups and rotate them among the five major activities offered. One group might receive sexuality education on Tuesday and Thursday, for example, while another group attends Job Club; on alternate days, the groups involved would be reversed. Most students participate in individual sports and creative expression activities at least once a week, and receive academic assistance daily.

Over the summer, program activities include maintenance meetings to reinforce young people's sexuality education and academic skills; during the summer cycle, participants also receive job assistance and participate in social events, recreational activities and cultural trips.

Each site is staffed by part-time employees, who run the various components, and by a full-time coordinator. In addition, a full-time community organizer handles day-to-day logistics at each site and maintains continuous contact with young people and their parents. The community organizer is a community member selected because of good rapport with residents; this staff person follows up promptly if a young person fails to attend the program.

In this article, we report the results of an evaluation of the CAS-Carrera program that gathered data from six sites in New York City. These sites were chosen (from 42 in the New York City area that applied to participate) for the experimental evaluation design because they were judged most likely to faithfully implement the program, given their reputation and history, site capacity and staff. All six are youth agencies that serve disadvantaged, inner-city populations. Staff for the academic assistance, sexuality education and Job Club components were hired by the Children's Aid Society to work at all six sites. Dr. Carrera and his staff trained all program staff and visited each site regularly to ensure high-quality program implementation.

DATA AND METHODS

Recruitment and Random Assignment

Adolescents were eligible to participate if they were not enrolled in an ongoing, structured after-school program; if they would be aged 13-15 on July 1, 1997; and, because the Carrera model is a primary prevention program, if they were not currently pregnant and were not parents. The agencies used a variety of recruitment strategies, including conducting outreach in schools, distributing flyers, contacting families on their mailing lists and recruiting adolescents who were already involved in their recreational activities.

Each site recruited 100 students. Baseline data were collected from February through April 1997. After the evaluation team conducted the baseline interview, students were asked to draw envelopes to determine whether they would be assigned to the Carrera program or to an alternative (control) program. At most sites, the alternative was the agency's regular youth program (which might include recreational activities, homework help, arts and crafts, or only drop-in privileges at the agency); none of the agencies had health care services on-site.

Prior to the baseline data collection, parents were given an extensive orientation about the evaluation design and the individual programs. Both parents and adolescents signed consent forms for participation in the program and its evaluation, including the random-assignment procedure. Complaints about program assignments were minimal, and some young people preferred the shorter time commitment required by the control programs.

Data Collection

The three-year program evaluation drew on linked data from three sources: annual surveys of teenagers' characteristics and program outcomes; annual tests of knowledge of sexual topics administered by the evaluation team at the same time as the annual surveys; and monthly attendance records provided by program staff.

To facilitate tracking, both program and control students were contacted several times a year, staff sent them birthday cards, and participants received cash and other incentives whenever data were collected. Home visits, telephone calls and visits to the program sites were used to locate and survey young people who did not participate in scheduled data collection efforts. In the program group, the adolescents' self-reported data on their sexual activity, pregnancies and births were comparable to information on those events provided by the program staff and obtained from the adolescents' medical records.

The Sample

Our analysis is based on the 81% of the original sample—484 program and control adolescents—who supplied data at the three-year follow-up (Table 1). The sample included adolescents of both sexes and was fairly evenly divided into each of the three targeted ages (13, 14 and 15).

Sixty percent of youth assigned to the program were non-Hispanic black (of African American or Caribbean descent), and most of the remainder were Hispanic. Many came from economically stressed families: Twenty-one percent lived in a household with no working adult and received entitlements; another 40% lived with an unemployed adult or received benefits. The majority of program participants lived in single-parent homes. Moreover, 28% reported that their parents or another adult family member had ever participated in or experienced one of the following social risk factors—abuse of substances, domestic violence, illness, incarceration or unemployment. Nineteen percent reported having parents with two or more of these factors.

Two-fifths of the program adolescents had taken part in an activity, but not in a structured program, at the site before being recruited for the evaluation. Three in 10 had paid employment at baseline, and almost nine in 10 had had a medical checkup in the previous year. One-quarter of the program participants had had sexual intercourse by the time they enrolled in the evaluation.

We created a six-point scale measuring how many of the following barriers to healthy social development each young person reported: having parents who had experienced two or more of the selected problems listed in the table; having a poor relationship with one's mother;* living in an unsafe neighborhood; having no relationship with a church or faith center; living in a household of low socioeconomic status; and having friends who engaged in three or more delinquent behaviors (specifically, participating in physical fights, carrying a weapon, using a weapon, stealing, being arrested and damaging school property.) Adolescents who participated in the program scored a mean of 1.5 on this barriers scale (not shown). They also reported a mean of 1.1 delinquent behaviors.

The experimental and control groups did not differ significantly by demographic and socioeconomic characteristics. The groups also did not differ, even within gender groups, in their relationships with their mother, school grade or previous participation in an after-school program (not shown).

Analytic Techniques

Our primary outcomes of interest were pregnancy and childbirth. Participants were asked about pregnancy and birth histories at each annual survey. Males were asked if they knew for sure whether they had caused a pregnancy or birth, if they did not know but thought they had, or if they did not know but thought they had not.

Our analysis also includes intermediate outcomes related to pregnancy. First, we assessed sexuality-related knowledge at three points in time, using a 72-item comprehensive instrument that included questions on physiology, contraception, gender differences, sexuality and pregnancy (alpha=.90). Adolescents completed this questionnaire prior to random assignment and again at the end of the first and second program years. We calculated changes in the percentage of correct responses to evaluate gains in knowledge over time.

All evaluation participants were also asked whether they had initiated sexual intercourse. For females only, we assessed whether they had been asked to have sex when they did not want to, and how they had responded in such situations. To gauge the extent of effective contraceptive use, we asked all sexually experienced adolescents whether they had used a condom or any other contraceptive at last intercourse.

We also questioned adolescents about comprehensive health care, because those who have better access to consistent, high-quality primary care are also likely to have better access to reproductive health care when they need it; in addition, overall health status affects other documented precursors of early pregnancy. For example, undiagnosed vision problems or ineffective asthma management can affect school performance, and success in school is related to the risk of early pregnancy and childbearing.¹²

We asked about five desirable health care outcomes: having received medical care in a setting other than an emergency room; having had a medical checkup in the last year; having been given a social assessment (i.e., answering questions about broader family and environmental factors) at that checkup; having had a hepatitis B vaccination; and having had a dental checkup in the last year. We converted these items into a dichotomous variable, coded one if young people reported four or five of these outcomes and zero otherwise.

We used chi-square analyses and analyses of variance to test for significant differences between the program and control groups in the sexuality, reproductive and primary health care outcomes. Logistic regression analyses were performed to assess whether participation in the control program had an independent impact on the outcomes, once baseline characteristics, age, ethnicity and number of barriers to social development were controlled for.

In each regression, all adolescents who had originally been assigned to either group and who were contacted after three years were included, regardless of the actual attendance records of program students. This means that the evaluation is likely to underestimate the effects of the CAS-Carrera model, especially when the model is compared with no intervention; thus, our analysis may more accurately estimate the effects of program exposure (which could have been minimal) compared with what young people can find on their own. (Although all control adolescents were offered an alternative program, some stayed with that alternative program, while others decided to try another or to drop out entirely.)

RESULTS

Bivariate Analyses

Three years after enrollment, 79% of participants were still involved at some level in their CAS-Carrera program: Forty-eight percent were actively involved in all program components, and 31% had contact with program staff outside of the weekday, after-school schedule. Those who were no longer involved had moved (8%); had never participated (5%); or had family issues that precluded participation, had scheduling conflicts or were incarcerated (8%). In contrast, only 36% of the control students were regularly participating in a program after three years, a retention rate that represents a decline from 42% at the end of the first two years.

Over three full years of programming (i.e., combining fall semester, spring semester and summer cycles), adolescents assigned to the CAS-Carrera program attended about 16 hours per month, on average; among the 48% who were most actively involved, the average was 22 hours. Participants spent the greatest number of hours receiving academic support, because most program sites offered tutoring, homework help and similar activities daily. (Job Club, family life and sexuality education, artistic self-expression and sports were generally offered on alternating days.)

The community organizers made about two contacts per month with adolescents or their families outside of program hours. Their logs suggest that absenteeism was caused by teenagers' family responsibilities (such as having to baby-sit younger siblings), family mobility, employment, educational activities and participation in extracurricular activities at school. Parents sometimes punished their children by making them miss program days, a practice that the program discouraged.

Among the program participants only, we examined the total number of hours spent in program activities during the fall and spring cycles over the three years by participants' characteristics. The oldest females attended significantly more hours than the oldest males (225 vs. 182—Table 2), and sexually experienced females attended significantly more hours than sexually experienced males (203 vs. 167). However, a multivariate analysis based on the total sample showed that only prior sexual experience was independently and negatively related to attendance, net of the other variables in the table (not shown).

When we compared the sexual, reproductive and health care outcomes among program and control students, we found gains in knowledge over time to be significantly greater among program participants than among controls: The number of correct responses on the knowledge questionnaire rose by 22% and 11%, respectively (Table 3). Females in the program were significantly more likely than those in the control group to say they had chosen not to have sex when pressured (75% vs. 36%). Program women were significantly less likely than controls to have ever had intercourse.

Moreover, sexually experienced program females were significantly more likely than controls to have used a condom with a highly effective method (i.e., the pill, the injectable or the implant) at last intercourse (36% vs. 20%). There were no significant differences by group assignment, however, in the proportions of young women who reported having used a condom at last coitus. Perhaps most important, at the third-year follow-up, females in the CAS- Carrera program had significantly lower rates of pregnancies and births than control females.

While male participants in the program also had significantly higher gains in knowledge than controls, the other positive sexual and reproductive outcomes found among women were not evident among men. In fact, program males were significantly less likely than control males to have used a condom along with a highly effective method at last intercourse (9% vs. 20%).

Young people in the CAS-Carrera program were more likely than controls to receive health care at a place other than the emergency room (94% vs. 83%). Further, the proportion of males who had received a social assessment at their last doctor visit was twice as high among program males as among control males (65% vs. 32%). Program participants of both sexes were significantly more likely than control students to have had a hepatitis B vaccination, an often neglected immunization. There were no significant differences by group assignment, however, in receipt of dental care or a medical checkup in the last year. Program participants of both genders were significantly more likely than controls to report five, or four of the five, desirable health care outcomes.

Sexually experienced adolescents also were asked about their reproductive health care-seeking behavior. Among males, the proportions who had made such a visit were significantly higher among program participants than among controls (74% vs. 46%). While the proportion having made such a visit was also higher among program females than among controls, the difference was not significant.

Multivariate Analyses

Because the significant differences at the bivariate level could have been caused by factors other than the program's effects, we present results of logistic regression analyses that controlled for age, ethnicity, baseline measures of the outcome variables and social development barriers at intake. These regressions were conducted for the four most important outcomes only—that is, having become pregnant or caused a pregnancy, having used a condom and hormonal method at last intercourse, being sexually active and having four or five of the positive health care outcomes. We did not assess the program's effects on the likelihood of a live birth because so few occurred over the period.

Although we conducted regressions that combined program males and females—and found that program participation was a significant, independent contributor in several regressions—because the significant findings were created for the most part by one gender group or the other, we present only gender-specific findings. Each regression was first performed using a dummy variable for the individual program site. However, since neither the significance nor the magnitude of the odds ratios changed when site variables were included, we excluded site variables from the analysis.

The odds ratios in the first panel of Table 4 (page 250) estimate the relative likelihood of each outcome among the program adolescents compared with that among control teenagers, net of the control variables. The odds of becoming pregnant were significantly reduced among young women in the CAS-Carrera program, compared with controls (odds ratio, 0.3). Further, female program participants had significantly reduced odds of currently being sexually active after three years of program exposure (0.5) and significantly increased odds of having used a condom and a hormonal method at last intercourse (2.4).

As in the bivariate analysis, we found no significant program effect on these outcomes among males. One outcome was significant for both males and females: The odds of having received good health care were twice as high among program participants as among controls (2.0-2.1).

As might be expected, having had intercourse before enrollment independently increased the odds that students would currently be sexually active and that they would have become pregnant or caused a pregnancy; age also had the expected positive effects on these outcomes. The number of social development barriers significantly affected only females' odds of being sexually active (1.5), while being black (as opposed to Hispanic) increased the odds of currently having sex among males only (2.4) and increased the odds of desirable health care outcomes among females only (2.0).

DISCUSSION AND CONCLUSIONS

This study has several potential limitations. Because program and control teenagers sometimes attended different programs located at the same site, some exchange of information, or "contamination" of the control group might have occurred. This would, however, likely diminish differences in outcome between program and control students. Further, our analysis followed these young people for three years, but the observed advantages among program students might dissipate over time.

These data are from New York City sites only, and they were collected from a sample that was overwhelmingly black and Hispanic. Thus, the data reported here do not reflect suburban and rural teenagers or those from other racial or ethnic groups. The sites in our study also benefited from the intensive training and support provided by the CAS staff. Sites that lack such support may find implementing the program to be challenging and make changes as they see fit. Indeed, we observed variations in program implementation, and quality, across sites. To date, not enough time has elapsed to assess which program components are most important; as more sites adopt the program, sufficient variations in implementation might allow such an analysis.

Our study, however, clearly documents the effectiveness among females of a comprehensive program to prevent adolescent pregnancy. Although our analyses cannot determine the relative importance of the model's components, the philosophy, structure and specific staff roles may each contribute to the successful long-term relationships that a large proportion of the young people formed with the program and its staff.

The CAS-Carrera philosophy emphasizes that working with young people is "a marathon, not a sprint"; the program design calls for "adopting" a group of young people and then sticking with them for several years. Adolescents who attend infrequently or only sporadically and those who have ongoing behavioral problems nonetheless remain part of the group. The program operates year-round, and staff are available even during nonprogram hours. All staff receive training and support in the program's overall philosophy, as well as in their specific responsibilities and tasks.

Within the overall structure of the five activities and two services, the CAS-Carrera model looked for creative solutions when participants had trouble staying connected with the program. For example, although the Job Club component provides internships and summer jobs (along with a classroom component involving discussions of job options and training in job readiness skills), some participants need to work even more hours than can be arranged through the Job Club. To solve the problem of attrition caused by participants' need to work, CAS-Carrera program staff looked for jobs for them in the immediate program vicinity and developed some jobs within the agency housing the program. In this way, staff could maintain an ongoing relationship with adolescents who were unable to attend during scheduled hours.

Although community organizers are rare in youth programming, their role appears to be an important one. These staff maintain regular and frequent contact with program youth and their families. Perhaps most important, community organizers give youth and families a continuous message that young people are noticed, valued and missed when they do not attend. Many youth programs take no action when adolescents do not attend.

The data show that the program maintains long-term connections with young people and that this affects young women's risk of pregnancy directly by improving their sexual literacy, delaying initiation of intercourse and increasing their use of effective contraceptives. These outcomes reflect the dual role of the sexuality educator and reproductive health counselor. The group family life and sexuality education sessions provide information on abstinence, contraception, pregnancy, physiology and gender roles. Through these sessions, staff also develop a close relationship with young people, so they are well positioned to provide support and follow-up as young people make decisions about sex. Again, the model emphasizes a flexible approach; conversations between the sexuality educator and an adolescent are more likely to happen over a slice of pizza in the neighborhood than in a counseling room.

While too few births occurred overall for meaningful analysis, ongoing follow-up data suggest that a difference between program and control women in the proportions who decide to carry pregnancies to term is emerging. Given program women's delays in initiating intercourse, their greater use of effective contraception and their lower pregnancy rates, there may soon be significantly fewer births among program than control females.

The program effects were weaker among young men, perhaps in part because young men who had had intercourse before enrolling (i.e., very early in their teenage years) were the least likely to attend regularly. Strong social norms among these inner-city young men might also stress the benefits (or lack of negative consequences) of early sexual behavior and parenthood. Finally, the female partners of male CAS-Carrera participants did not receive direct program support and services unless they were also enrolled. Perhaps the male participants could not, or did not, repeat the program messages to their partners. The data suggest that reaching young men sooner may strengthen outcomes at earlier ages; indeed, to achieve this goal, CAS has now implemented programs with 11- and 12-year-olds.¹³

Although participation in a CAS-Carrera program did not significantly affect males' reproductive outcomes, important benefits emerged in their overall access to primary health care. At the three-year follow-up, program males (as well as females) had much better access to health care than control males. Finally, although program males' significantly higher rates of reproductive health care visits did not result in their causing fewer pregnancies, such improved access might have influenced young men's health status in other ways. For example, increased use of reproductive health care services might have improved sexually transmitted infection prevention or resulted in earlier diagnosis and treatment, although the surveys did not ask directly about such infections.

How do the pregnancy results from the CAS-Carrera model compare with those from other successful and well-evaluated pregnancy prevention programs? Although direct comparisons with all such programs are not possible, we compared our results with those from an evaluation of the community service-based program Teen Outreach. According to those results, after one year, the odds of pregnancy were 41% as high among program females as among controls;¹⁴ our evaluation, in contrast, found that after three years, the odds of pregnancy were only 31% as high among CAS-Carrera females as among controls.

How much does such a comprehensive program cost? At the New York City sites, costs averaged $4,000 per year for each teenager enrolled, or about $16 a day (an amount that is less than what after-school child care would cost). These costs cover—for a program that operates 50 weeks a year, five and often six days a week—comprehensive medical and dental services; stipends for the hours spent in Job Club; and wages for work on entrepreneurial and community service projects or internships (i.e., three dollars per hour for younger teenagers and minimum wage once adolescents qualify for working papers).

Costs for some line items are likely to be higher in New York City than in other parts of the country; for example, teachers who worked in the academic component as tutors were paid the union wage of $34 per hour. While some may find these costs alarming, deciding how much should be invested in young people is clearly a policy issue. Such program costs seem less daunting, however, when they are viewed in juxtaposition with the costs that are avoided by preventing early pregnancies and promoting more positive behaviors.

Our evaluation results allow the CAS-Carrera program to join the fewer than 10 others that have shown an impact on teenage pregnancy rates or birthrates. The program is one of only four for which evaluations based on random assignment have demonstrated an impact. While the CAS-Carrera model appears to have achieved success by building long-term relationships with participants, by delaying sexual intercourse and by encouraging effective method use, further analysis by participants' ethnicity and attendance levels should increase our understanding of what other factors might contribute to program success. Reducing teenage pregnancy is an important goal for the nation and for disadvantaged communities in particular. At least for the young women studied here, the CAS-Carrera program is a strategy that works.