SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. BACKGROUND AND PRIOR STUDIES
  4. FINANCIAL INSTRUCTION AND STUDY DESIGN
  5. SAMPLE AND MEASURES
  6. RESULTS
  7. CONCLUSIONS
  8. REFERENCES

This study investigated the effects of a financial education program on high school students' knowledge of personal finance. A comparison of pretest and posttest scores achieved on a reliable and valid thirty-item instrument suggested that the Financing Your Future curriculum increased financial knowledge across many concepts. The scores increased regardless of the course in which the curriculum was used and across student characteristics. The assessment contributes to the growing literature showing that a well-specified and properly implemented program in financial education can positively and significantly influence the financial knowledge of high school students.

Interest in personal finance education in US schools has increased significantly since the 1990s. From 1998 to 2009, the following changes occurred: states with content standards for personal finance education in the schools rose from twenty-one to forty-four, states requiring implementation of those standards increased from fourteen to thirty-four and states requiring that a personal finance course or economics course with personal finance content be taken before graduation from high school grew from one to thirteen (CEE 2009). The Jump$tart Coalition for Personal Financial Literacy, formed in 1995, published the first national content standards for personal financial education in 1998 (Jump$tart Coalition for Personal Financial Literacy 2007). The organization also sponsored national testing of high school students and has used the results to call national attention to deficiencies in youth financial understanding (e.g., Mandell 1998). In addition, changes in economic conditions since the 1990s have led to more studies of adults' problems with personal finance (Braunstein and Welch 2002). These developments together with reports of poor student test scores have reinforced the perceived need for more financial education among youth (Bernanke 2006).

Published evidence of the effectiveness of precollege financial education has been somewhat mixed. Some studies have questioned the value of personal financial education in secondary schools. For example, Mandell (2008), using the Jump$tart data that he collected from 2000 to 2006, found no evidence that students taking a money management or personal finance course knew more about the subject than students who had not taken such a course. Other studies have reported positive effects on student knowledge or understanding of personal finance topics and concepts based on education with specific financial curriculum (e.g., Danes, Huddleston-Casas, and Boyce 1999; Harter and Harter 2009; Varcoe et al. 2005). If states, school districts, or teachers want to allocate more instructional time and resources to financial education in high schools, more research is needed on its potential value to justify those allocations. This study offers additional evidence that well-defined and properly implemented financial education programs in high school can increase students' financial knowledge.

The study also seeks to address concerns, such as content specification, instruction, measurement, research design and analysis, about research protocols in financial education program evaluation (Fox, Bartholomae, and Lee 2005; Lyons et al. 2006). The content for the DVD-based curriculum used in this study, Financing Your Future (FYF) (CEE 2006), is clearly described and specified so that different teachers provide the same instruction to each student. After training, teachers are familiar with the content and know how to teach it. The financial knowledge test employed to assess student achievement was developed to be a valid measure of knowledge of the content taught with the instructional materials, and the scores were found to be reliable.

BACKGROUND AND PRIOR STUDIES

  1. Top of page
  2. Abstract
  3. BACKGROUND AND PRIOR STUDIES
  4. FINANCIAL INSTRUCTION AND STUDY DESIGN
  5. SAMPLE AND MEASURES
  6. RESULTS
  7. CONCLUSIONS
  8. REFERENCES

In response to the perceived need to improve financial literacy, a wide range of private businesses, nonprofit organizations and government agencies have developed financial education materials and programs (see Fox, Bartholomae, and Lee 2005). Most have focused on adults and targeted specific financial behaviors (Hilgert, Hogarth, and Beverly 2003). However, some have focused on youth, as have some educational systems that have created state standards or curriculum mandates in personal finance. For example, nonprofit organizations such as the National Foundation for Financial Education, the Council for Economic Education and Junior Achievement all have published curriculum materials and taken other steps to improve personal finance education for precollege students (see Lopez-Fernandini and Murrell 2008 for a description). As programs and initiatives have expanded over the past decade, so has research on the effectiveness of financial education. Given the extensive number of educational initiatives for adults, most research has focused on this segment of the population (Martin 2007), but there has been evolving research in recent years on financial education for youth (McCormick 2009).

An interesting line of national research has investigated the effects of state mandates on financial literacy, which has generally been positive and important. For example, a longitudinal study reported that adults who had taken a personal finance course in high school saved a greater proportion of their income when they were older (Bernheim, Garrett, and Maki 2001). Another study showed that the characteristics of the state mandate for schools mattered when investigating its effect on improving financial knowledge scores (Tennyson and Nguyen 2001). Researchers have found more specific state mandates for personal finance education to have a more positive effect on student understanding than those more broadly defined.

By contrast, another line of national research focused on the effects of financial education on financial knowledge has drawn negative conclusions. One such study used Jump$tart test scores from 2000 through 2006 and reported that high school seniors who took a full-semester high school course in personal finance had average scores that were no higher than all seniors who took the test (Mandell 2008). These national test data, however, are limited in their usefulness to evaluate the effectiveness of financial education because of insufficient controls related to course content, test measurement, teacher preparation and amount of instruction. Unlike most high school courses in mathematics or science, there can be widespread national differences in the content of personal finance courses, and these courses can give different emphases to each topic even if they cover the same topics. The test also includes only thirty-one knowledge items that may not closely match the content of personal finance courses nationwide or give proper emphasis to the topics actually taught in a course. Furthermore, the quality and amount of instruction in a national sample can vary because teachers may not be well-trained to teach the material or because of differences in the amount of instruction provided over a semester. A final problem is that the test data do not measure the initial level of financial knowledge to assess changes resulting from instruction in personal finance.

One means for overcoming these limitations is to focus the evaluation on a specific financial education curriculum so that the content is well-defined and standardized. One of the earlier and more comprehensive evaluation studies of this type focused on the High School Financial Planning Program (HSFPP) (Danes, Huddleston-Casas, and Boyce 1999). The 1998–1999 study found positive and significant effects from the program on financial knowledge, financial behaviors and financial self-efficacy among 4,107 students at the conclusion of classroom instruction and three months later. The study was discussed in an extensive review of program evaluations in financial education and the authors of that review stated that, “Unfortunately, rigorous evaluations and reporting of this sort is not part of many programs currently offered in a school setting” (Fox, Bartholomae, and Lee 2005, p. 200).

The measure of financial knowledge used in the HSFPP study, and a subsequent study of HSFPP (Danes and Haberman 2007) using data collected in 2003–2004, was a subjective one and contained few items because the emphasis in these studies was on behavior change. The evaluation of student knowledge from the 1998–1999 sample asked students to provide a self-assessment of their knowledge of three topics (the cost of buying on credit; buying auto insurance; and investments) on a posttest basis and then a retrospective pretest basis. The same three survey items and testing procedures were used to collect data from the 2003–2004 sample, but a fourth knowledge item was added on “wants and needs difference.” These survey items offered useful insights about changes in students' perceptions of what they think they know about the financial topics before and after exposure to the curriculum. They were not designed, however, to provide test data on whether students' knowledge or understanding of the financial topics was correct or incorrect, as would be shown by item responses on an achievement test such as the Jump$tart survey or other similar achievement tests.

Another study evaluated the effectiveness of the MoneyTalks: Should I Be Listening? curriculum for high school teens (Varcoe et al. 2005). Pretest and posttest data were collected over a six-month period in 2002 from 114 students in four counties in California. In addition to a self-perception measure of knowledge, this study included a financial knowledge test composed of nineteen true–false questions that showed a statistically significant increase from 52% correct on the pretest to 72% correct on the posttest. The study, however, provided little information on this test's reliability or validity. In addition, the use of true–false items rather than four-option multiple-choice items increased the level of guessing. There also was no control group of students to use for comparison purposes.

Several evaluation studies have investigated the effectiveness of Financial Fitness for Life (FFL), a personal finance and economics curriculum published by the Council for Economic Education for high school students (grades 9–12) and lower grade levels (e.g., Morton and Schug 2001). One of the advantages of this curriculum from an evaluation perspective is that there are reliable and valid tests to measure student achievement. For the high school level, Walstad and Rebeck (2005a) developed a fifty-item multiple-choice test to assess high school student financial knowledge based on the overall content and the five financial education themes presented in the FFL curriculum. The reliability of the tests was .86, which showed there was a high degree of internal consistency among test items (Cronbach 1951). The content validity for the test was based on the national standards in personal finance published by the Jump$tart Coalition. The construct validity for the test was evaluated by having high school teachers in three states—who had been trained to use the FFL curriculum—administer the test to 524 students who were taught with the FFL curriculum, and to a control group of 335 similar students in the same schools who did not receive this instruction. The results showed that high school students receiving FFL instruction with FFL-trained teachers achieved higher test scores than the students in the control group. The positive results for FFL instruction also held even after accounting for other factors such as gender, race and ethnicity, income and type of community (Walstad and Rebeck 2005a).

Another extensive study assessed the effectiveness of the FFL curriculum in high school, middle school and elementary school grades in eastern Kentucky (Harter and Harter 2009). For the high school analysis, the investigators used test items from the high school FFL test for three FFL themes (saving, spending and credit, and money management) (Walstad and Rebeck 2005a). They also recruited teachers to participate in a workshop on use of FFL materials. In the semester prior to attending the workshop, the recruited teachers posttested their students after instruction using financial education materials that did not include FFL curriculum. After attending the workshop and teaching the FFL curriculum, the teachers pretested and posttested their students. The increases in the FFL test means from pretest to posttest were significant and positive. An ordinary least-squares (OLS) regression that controlled for general student ability, grade level, gender and race showed a positive and significant improvement in test scores for FFL students compared with students taught with other financial education curricula.

One additional FFL study used 2004–2006 data from 84,582 students who took a state-mandated end-of-course test in economics in Georgia (Swinton et al. 2007). This study evaluated the contribution made to student test scores from teacher participation in an FFL workshop. The study specified a fixed-effects regression to control for differences among schools, and it also controlled for student characteristics such as gender, race and ethnicity, disability and year of testing. The results showed that high school students whose teachers had participated in an FFL workshop (5.2% of the student sample) scored significantly higher on average on the mandated economics test than did students of teachers who did not receive such training. Although these positive findings provide insights for assessing FFL teacher training, the study is not an evaluation of the FFL curriculum or its effects on student learning. The knowledge test also only focused on economics content, with only minimal personal finance content included; thus, the test data do not permit an analysis of what students know about personal finance overall or about different topics in personal finance.

As is apparent from this focused review of prior studies, evaluating the effectiveness of a financial education program for high school youth requires careful attention to content, delivery, measurement, design and analysis. First, there should be a clear definition of the personal finance content so that students are taught the same content when instruction is given. Second, teachers need to be trained to use the program materials so they are familiar with the content and know best how to teach it. Third, the knowledge outcomes (overall and for subscales) have to be specified and measured with reliable and valid test instruments. Fourth, data need to be collected on a pretest basis to control for starting levels of financial knowledge and posttested at the end to assess changes in financial understanding. Fifth, data must be rigorously analyzed using an appropriate form of statistical analysis to determine whether there are meaningful changes in outcomes while controlling for other factors. These evaluation issues frame the context for this study. The current study contributes to the broader literature by describing a controlled and systematic assessment that targets its evaluation work at the fourth tier (progress-toward-objectives) and fifth tier (program impact) of Jacobs' (1988) five-tier model of evaluation.

FINANCIAL INSTRUCTION AND STUDY DESIGN

  1. Top of page
  2. Abstract
  3. BACKGROUND AND PRIOR STUDIES
  4. FINANCIAL INSTRUCTION AND STUDY DESIGN
  5. SAMPLE AND MEASURES
  6. RESULTS
  7. CONCLUSIONS
  8. REFERENCES

FYF is a personal finance program on DVD for high school teachers to use with their students (CEE 2006). The program contains five video segments on the DVD. The first video introduces students to basic vocabulary and key concepts in personal finance such as saving, wealth building and money management. The second video covers trade-offs and opportunity costs related to financial decision making, and earning an income. The third explains the importance of banking relationships and discusses banking and checking practices. The fourth explains topics such as credit, debt, annual percentage rate (APR), and credit scores. The fifth video focuses on financial budgeting, saving, investing and risk tolerance. With each video there are three printed lessons that teachers can use to reinforce the financial concepts in the video. These lessons contain written outlines for teachers to present ideas and concepts, and instructional activities and assignments for students to complete.

A primary purpose for this study was to assess whether FYF materials increased student knowledge of personal finance. To achieve that objective, the study used a quasi-experimental design with a treatment group and a control group (Shadish, Cook, and Campbell 2002). The design was intended to address concerns about the quality of evaluation of financial education programs (Fox, Bartholomae, and Lee 2005). The treatment group was senior high school students who viewed the FYF videos and used the FYF lesson materials for approximately six hours of total instruction. This instruction was spread over a two- to four-week period that varied across teachers, depending on the length of class sessions and daily teaching schedules. The control group was similar high school students who received no FYF instruction during the period of the FYF assessment. Test and survey data were collected from students prior to FYF instruction and at the completion of FYF instruction. The control students also were pretested and posttested at the same time as the FYF students.1

FYF was developed with the expectation that teachers would be trained to use both the DVDs and accompanying classroom lessons. In the delivery system created by the Council for Economic Education, this teacher training was supplied in workshops offered by its national network of affiliated state councils or centers for economic education. The FYF training was conducted by council or center staff members, who typically serve on the faculty at a college or university and have special expertise in educating teachers about the subject content and effective pedagogy.

The FYF teachers who tested their students received about three to four hours of training during Fall 2006. The fifteen teachers who volunteered and were selected for the study came from four states—New York (three), Minnesota (five), Texas (three) and Maryland (four). After their training, they provided FYF instruction in forty classes containing 913 students. This instruction was given in late fall of 2006 or winter of 2007. In addition, three of these teachers, one each in New York, Minnesota and Texas, arranged to have the assessment instruments administered on a pretest and posttest basis to six classes of similar students. The 153 students in these six classes were the control group. These classes were either taught by these three teachers or they were taught by one of their colleagues at the same school.

As with most pretest and posttest studies, there was incomplete data from some students. The likely reasons included illnesses or other reasons for missing school on a posttest day, or students moving out of the class or school during a semester. In addition, many students did not provide the complete or correct information to match their pretest data with their posttest data. The attrition from the pretest sample to a matched sample (pretest and posttest) did not appear to affect the results in any systematic way. The results for this study are based on a sample of eight hundred students (673 FYF and 127 controls) who took both the pretest and posttest instruments and provided background data.

SAMPLE AND MEASURES

  1. Top of page
  2. Abstract
  3. BACKGROUND AND PRIOR STUDIES
  4. FINANCIAL INSTRUCTION AND STUDY DESIGN
  5. SAMPLE AND MEASURES
  6. RESULTS
  7. CONCLUSIONS
  8. REFERENCES

Table 1 presents the distribution of the matched sample of 673 FYF students and 127 control students across course and personal characteristics. About 67% of the FYF students were taking economics, compared with 51% of the control students. The other 49% of the control students were in a US government or world history course. About half of the sample was female (52% of the FYF group and 48% of the control group). The FYF sample included a slightly greater percentage of seniors (82%) than did the control sample (79%). A majority (67%) of the control students planned to attend a four-year university after graduation, compared with 58% of the FYF sample. One likely reason the control group showed more interest in future education was that one of the control classes was actually a “college in the high schools” economics course. Each group was distributed similarly across employment history. Between 40% and 50% of each sample were from Minnesota. Maryland provided the smallest FYF group (8%) and had no control students. The Maryland students were the only students taking a personal finance course.

Table 1.  FYF Sample Characteristics (n = 800)a
 FYF (n = 673)Control (n = 127)
  1. aPercentage may not sum to hundred due to missing responses.

  2. bOne economics class in the control group was a “College in the High Schools” course.

Course
 Personal finance8.2
 Economics66.751.2b
 Accounting; business/personal law11.7
 US government or world history11.648.8
 English as second language for employment1.8
Gender
 Male47.752.0
 Female52.348.0
Class level
 Sophomore2.221.3
 Junior15.0
 Senior82.878.7
Plans after high school
 Work, no education2.83.9
 Attend two-year college25.317.3
 Attend four-year university57.766.9
 Other education/training7.06.3
 Undecided7.35.5
Employment history
 Work summers and during school56.544.1
 Work during summers, not school14.613.4
 Work during school, not summers4.54.7
 Worked in the past, not currently8.911.8
 Never formally employed15.626.0
Whose credit card do you use?
 My own10.015.7
 My parents'13.416.5
 Both my own and my parents'4.03.1
 None, I do not use a credit card72.764.6
State
 Maryland8.2
 Minnesota48.144.9
 New York17.426.8
 Texas26.328.3

A financial knowledge test was constructed to measure student understanding of personal finance as presented in the FYF videos and lesson materials. The thirty multiple-choice items on the test came from two sources, one independent of FYF and one dependent on FYF. The independent items came from a personal finance test that accompanies the FFL curriculum and which Walstad and Rebeck (2005a, 2005b) have shown to be a reliable and valid measure of personal financial understanding. Seventeen items were selected from this external test based on their close match to the personal finance content of the FYF materials. An additional thirteen items were written to cover other FYF content for which there were no published test items.2 The alpha reliability of the test was .85, which shows that there is a high degree of internal consistency among items (Cronbach 1951).

The content validity of the FYF knowledge test is evident in Table 2, which shows the distribution of the thirty multiple-choice items across the fifteen FYF lessons. Copyright restrictions do not permit publishing the test items, but they are briefly described in the lower half of Table 2. In addition, the test contained items written at different cognitive levels. Twenty-two items focused on testing for knowledge or comprehension of ideas or concepts in personal finance and eight items required application of knowledge to problems or situations. Each of the thirty items discriminated between students who knew more and those who knew less about the content of the test.3

Table 2.  FYF Knowledge Test: Content and Cognitive Coverage
 Cognitive Level
FYF LessonsKnowledge or ComprehensionApplication
Video 1
 1.1 What is a financial life?1, 20, 26 
 1.2 Setting financial goals630
 1.3 Spend less than you earn7 
Video 2
 2.1 Why can't I have everything I want?9, 18 
 2.2 Decisions about my human capital23, 2813
 2.3 Learning is a lifetime investment 10
Video 3
 3.1 Why keep your money in a bank?11, 2712
 3.2 The costs and benefits of being banked4, 8 
 3.3 Check, charge, debit card?29 
Video 4
 4.1 Make credit work for you1514
 4.2 Credit savvy2417
 4.3 The most important grade you will ever earn3, 19 
Video 5
 5.1 Why should I pay myself first?2, 2221
 5.2 How to create a financial plan 25
 5.3 Risks and rewards5, 16 
FYF Test Item Content
1. Net worth16. Risk and reward
2. Saving importance17. Credit card charges
3. Interest rate and risk18. Decision making
4. Financial institutions19. FICO score
5. Liquidity risk20. Education and wealth
6. Financial decisions21. Rule of 72
7. Saving and age22. Compound interest
8. Debit card23. Wages and markets
9. Human capital24. Revolving credit
10. Education and income25. Net income
11. Interest26. Saving and finances
12. FDIC insurance27. Banks and lending
13. Wages and labor demand28. Occupation and income
14. Income and debt level29. Credit card function
15. Annual percentage rate30. Financial goals

RESULTS

  1. Top of page
  2. Abstract
  3. BACKGROUND AND PRIOR STUDIES
  4. FINANCIAL INSTRUCTION AND STUDY DESIGN
  5. SAMPLE AND MEASURES
  6. RESULTS
  7. CONCLUSIONS
  8. REFERENCES

Table 3 reports the percentage correct scores achieved by the 673 students in the FYF group before and after implementation of the FYF program. Scores achieved by the 127 control students taking the exam also are reported. The students in the FYF group had a mean score of 49.2% correct on the pretest (SD = 15.1). The mean score for the control students was 49.5% correct (SD = 16.5). The difference was not statistically significant (t-value = 0.195) and indicated that the two groups had relatively the same level of financial knowledge before instruction began for the FYF students.

Table 3.  FYF Knowledge Test Pretest and Posttest Scores: FYF and Control
 FYF Students (n = 673)Control Students (n = 127)
 PretestPosttestDifferencePretestPosttestDifference
  1. aSee Table 2 for the contents of the FYF videos. Sample standard deviations in parentheses.

  2. **p < .01, two-tailed paired-samples t-test.

Total Score (α = .85)49.2 (15.1)68.9 (18.1)19.7**49.5 (16.5)50.5 (17.6)1.0
Videoa
 1 (α = .62)52.6 (24.9)73.1 (25.2)20.5**52.6 (26.2)54.5 (27.9)1.9
 2 (α = .53)45.8 (21.3)66.7 (24.2)20.9**48.8 (21.74)49.6 (23.5)0.8
 3 (α = .63)64.4 (25.3)80.6 (22.2)16.2**60.5 (28.47)63.0 (26.8)2.5
 4 (α = .43)37.9 (20.3)61.9 (22.9)24.0**38.6 (22.0)40.4 (22.0)1.8
 5 (α = .45)45.3 (21.5)62.1 (23.6)16.8**47.0 (21.9)45.1 (22.2)−1.9

After instruction, FYF students achieved a mean score of 68.9% correct (SD = 18.1), an average gain of 19.7 percentage points. A pairwise statistical test of the equality of the pretest and posttest scores for the FYF group rejected the null hypothesis that the true parameters were equal (t-value = 30.6). Control students achieved a mean score of 50.5% (SD = 17.6), a gain of 1 percentage point that was not statistically significant (t-value = 0.94). The nearly identical mean scores achieved at the time of the pretest and posttest for the control group suggest that learning due to taking the pretest was unlikely to affect achievement in the study.

Comparison of the total scores achieved by the FYF group at the time of the pretest and posttest provided strong evidence of the curriculum's ability to influence students' understanding of the material. It is possible, however, that this estimated increase in knowledge might merely reflect the curriculum's effectiveness in increasing understanding of some concepts covered, while having no effect on the understanding of others. To further explore the influence of the FYF curriculum, subscores were calculated for the five videos in the FYF curriculum. These subscores were calculated by summing the number of correct responses to the six items that covered the content of each of the five videos. The reliability estimates for these subscores were acceptable but lower than those for the entire test primarily because there were fewer items (six compared to thirty) for each subscore.4

Data for the subscores in Table 3 show the improvement in FYF students' financial knowledge resulted from increases across the concepts in all five FYF videos. Each difference was statistically significant at the .01 level. By comparison, control students showed no statistically significant average differences in pretest and posttest scores for any of the five videos. The average increase in FYF scores for the entire thirty-item test achieved by the treatment group was 19.7 percentage points. Three of the FYF five subscores showed an average increase that was somewhat greater than for the overall tests (20.5, 20.9 and 24.0) and two subscores showed an increase that was somewhat less than the overall test (16.2 and 16.8). Video 4 had the highest gains and it covered concepts involving the use of credit. The lowest gains were found in Video 3, which covered concepts related to banking.

To further explore the effect of the FYF curriculum on students' knowledge across concepts, we compared the pretest and posttest item percent correct for each of the thirty items. For the sake of parsimony, we only summarize the results instead of reporting them in a table. For the FYF group, there was a statistically significant increase in the percent correct from pretest to posttest for twenty-nine of the thirty items, with twenty-eight significantly different from zero at the .01 level and one at the .05 level.5 By contrast, the control group had a statistically significant change on only two items, one of which was positive and the other negative. The FYF-item results reinforced the findings from the total and subscore analysis and showed that FYF instruction increased students' knowledge of almost all the concepts covered on the thirty-item test.

Although the score and item analysis indicated that FYF instruction contributed meaningfully to understanding personal finance, an argument could be made that the significant improvement in scores from pretest to posttest might be only because of increases within some courses but not others, or only for students with certain characteristics. To investigate this issue, pretest and posttest average percentages were calculated and compared using a pairwise t-test across course and student characteristics. Table 4 presents the average pretest and posttest percentages and t-values for the FYF-matched sample of students.

Table 4.  Percentage Correct on FYF Knowledge Test by FYF Student Characteristics (n = 673)
 Pretest (%)Posttest (%)Changent-Value
Course
 Personal finance52.3661.038.67553.90
 Economics51.3573.9022.5544932.71
 Accounting; business/personal law48.4467.7619.32797.56
 US government or world history36.7148.9312.22787.18
 English as second language for employment41.1154.4413.33122.30
Gender
 Male49.4168.8419.4332119.75
 Female49.0368.9319.9035223.57
Class level
 Sophomore40.8963.3322.44155.40
 Junior51.0669.3118.2510110.95
 Senior49.1068.9619.8655728.02
Plans after high school
 Work, no education44.0464.9120.87194.16
 Attend two-year college45.2066.2021.0017017.56
 Attend four-year university51.7771.3119.5438824.44
 Other education/training45.8264.8919.07475.56
 Undecided48.1664.4216.26496.32
Employment history
 Work summers and during school49.8971.1421.2538024.76
 Work during summers, not school48.9165.1716.26989.63
 Work during school, not summers47.1165.8918.78305.42
 Worked in the past, not currently48.9464.1715.23606.87
 Never formally employed47.7867.7519.9710513.68
Whose credit card do you use?
 My own45.2268.0122.79679.13
 My parents'47.8963.1115.22908.84
 Both my own and my parents'47.6563.5815.93275.01
 None, I do not use a credit card50.0970.3620.2748927.89
State
 Maryland52.3661.038.67553.90
 Minnesota52.7473.5620.8232430.21
 New York40.6050.9110.311176.15
 Texas47.4868.8921.4117719.97

The increases in the average percentages from pretest to posttest were statistically significant across each course type using the FYF materials. The largest average gain of 22.55 percentage points was achieved in the economics course, which was not unexpected because economists and economics educators developed the FYF materials primarily for use in economics courses. The results, however, also showed that FYF instruction still can be beneficial when used in courses with a wide variety of subject content.

The achievement gains were statistically significant for both males and females (19.4 and 19.9, respectively). The starting pretest percentages were almost identical for both groups as were the posttest values. The great similarities in pretest and posttest scores indicate that FYF test and instruction is probably not subject to conditions that produce gender differences in financial knowledge outcomes.6

There was a statistically significant improvement in financial knowledge scores across sophomores, juniors and seniors. The differences in gains within this category were likely due to differences in sample size. The gains for sophomores were the largest (22.4), but the sample was small and probably a more select group of students than the other groups. However, the gains for juniors and seniors, which were the two largest groups, were more similar, with seniors showing slightly larger gains (19.9) than juniors (18.3)—as might be expected because of their greater maturity.

Students also were asked to state their plans for education and work after high school. The statistically significant gains (19.1 to 21.0) in financial knowledge were similar for those groups of students with some type of stated educational or work plans after high school. The gains (16.3) were lower, however, for students who were undecided about their future plans, presumably reflecting some lack of interest in academics or work preparation in high school.

Students were asked about their employment during high school. Substantial increases in test scores were found across all employment categories. The gains varied from 15.2 for students who worked in the past, but were not currently working, to 21.3 for students who worked summers and during the school year. Employment was not a critical factor for gains in test scores because even students who had never been employed formally had relatively high gains (20.0).

Both those students who had used either their own or a parent's credit card and those who had not used a credit card exhibited large gains in understanding. The largest increase in scores was among students who had their own credit cards (22.8), perhaps because they had a reason to be more interested in personal finance content. This gain, however, was not much larger than for students who reported that they did not use credit cards (20.3). Students with the smallest gains were those who used their parents' credit cards (15.2) or some combination of their own and parents' credit cards (15.9).

Finally, FYF students in all four states showed statistically significant gains in financial knowledge scores. The state gains were largest and most similar in Texas (21.4) and Minnesota (20.8) and smaller in New York (10.3) and Maryland (8.7). It should be noted that the Maryland sample contained only students in a personal finance course, and hence the test scores for this state and the personal finance course category were the same.

The analysis so far has focused on comparison of pretest and posttest scores for financial understanding to detect significant differences in mean percentages. The results from the percentage score analysis indicated there were positive and significant effects of financial education on students' financial knowledge. Although Table 1 presents similarities in the two groups, differences also exist and could potentially explain differences in the outcomes. Accordingly, a regression model was specified and estimated to investigate the effect of the FYF curriculum while controlling for course type, student characteristics and teacher effects that were not equally distributed across the FYF and control groups.

For this analysis, the following linear model was estimated.

  • image(1)

The dependent variable, Yit, is the posttest financial knowledge score for student i of teacher t. Bosshardt and Watts (1990) described the importance of controlling for differences across teachers when estimating the influence of student and course factors on precollege student achievement in economics, and included a teacher fixed effect in their regression analysis. We also included a fixed effect for each teacher (αt) to control for unobserved differences across teachers.

The independent variables included two course-level factors (C(1) and C(2)) and six student-level factors (S(1) through S(6)). The variable C(1), which represents the focus of our study, is a dummy variable for financial education. A “1” for this variable indicated that the student was in the treatment group—the student participated in a course that utilized the FYF curriculum. If financial education, specifically the FYF curriculum, effectively improves financial knowledge gains relative to the control group (C(1) = 0), the estimated coefficient will be positive. The variable C(2) controlled for course type, with a “1” indicating that the student's course was either economics or personal finance. It was expected that students in these courses were more likely to cover the concepts included in the test instrument, regardless of whether the course utilized the FYF curriculum.

The student's pretest knowledge score (S(1)) captured the initial level of student knowledge, and thus the coefficients on the remaining independent variables represented the effects of these variables on the change in knowledge from pretest to posttest. Student characteristics (S(2) through S(6)) were included to control for student-level factors that might influence changes in financial knowledge. Several studies show that a student's gender influences financial knowledge (Danes and Haberman 2007; Varcoe et al. 2005), so gender (S(2); 1 = female) was included in the equation. Students with greater academic ability and who devote more effort to studying are likely to learn more, so a student's educational plans (S(4); 1 = four-year college, 0 otherwise) was included as a proxy for academic ability and effort. Students expected to have more experience with financial concepts outside of the classroom and thus more opportunities to assimilate financial literacy also might possess a stronger basis upon which to learn new financial concepts. A student's class level (S(3); 1 = senior, 0 otherwise) was a proxy for age and thus extracurricular learning opportunities (as well as a proxy for maturity, also expected to have a positive effect on learning). A student's work experience (S(5); 1 = never worked) and credit card use (S(6); 1 = never used a credit card) also were included to capture extracurricular experiences that might be associated with financial knowledge.

The main purpose of the study was to estimate the effect of financial education on financial knowledge, so the discussion of the results will primarily focus on the estimated coefficients for financial education in the three regressions described below. The first column of values in Table 5 reports the regression results for the unrestricted sample (eight hundred students and fifteen teachers). It shows that the explanatory variables explained 43% of the variation in the dependent variable. After accounting for the effect of differences among teachers, pretest scores and other student characteristics, FYF students gained 15.93 percentage points from pretest to posttest relative to control students, an effect that was statistically significant at the .01 level.7

Table 5.  Fixed-Effects Regression Results
Variable(1) Unrestricted (n = 800)(2) Restricted (n = 308)(3) Economics (n = 514)
  1. Note: Standard errors in parentheses.

  2. *p < .05;

  3. **p < .01, two-tailed test. The constant reflects the average value of the fixed effects.

Dependent variable = financial knowledge posttest score
Financial education (1 = FYF student)15.932** (1.653)15.513** (1.952)21.297** (1.943)
Course (1 = economics or personal finance)8.300* (3.573)3.945 (6.517)
Financial knowledge pretest score0.467** (0.034)0.406** (0.072)0.492** (0.042)
Gender (1 = female)−1.722 (0.925)−1.970 (1.687)−2.155* (1.094)
Educational plans (1 = four-year college)3.355** (0.995)3.761* (1.754)3.347** (1.198)
Class level (1 = senior)−0.857 (1.894)4.072 (5.629)−4.552 (3.092)
Work (1 = never worked)0.373 (1.242)−0.773 (2.060)1.182 (1.448)
Credit card use (1 = no)1.530 (1.052)2.111 (1.925)1.408 (1.285)
Constant38.03938.15342.982
Adjusted R2.430.425.351
F43.34**16.89**36.04**

Although the research design called for treatment and control students to be drawn from the same general population of students, one limitation of the data set was that not all teachers taught both a treatment and control group. This factor raised an issue about whether there was a systematic difference in treatment and control teachers who accounted for the posttest differences in the financial knowledge scores of FYF and control students. To investigate this issue, we restricted the sample to students with teachers who taught both a treatment and control group. This restriction reduced the student sample to 308 students who were taught by three teachers. The values in the second column of Table 5 present these results. Again, the estimated effect from instruction with the FYF curriculum (β2 = 15.513) remained statistically significant at the .01 level. The size of the coefficient also was essentially the same as the coefficient for FYF when estimated with the full sample of eight hundred students who were taught by fifteen teachers. These results indicated that differential participation of teachers in treatment and control groups was unlikely to significantly influence posttest achievement scores.

Another finding is worth noting. The regression results from the full sample showed that students in economics courses had larger increases in measured gains in financial knowledge relative to students in other courses after controlling for other factors.8 This premium may arise because students who take such courses may be of higher general ability and thus better prepared to understand personal finance content or it may be that the teachers of such courses are better able to provide instruction because they already teach similar content.

Finally, it could be argued that the composition of the FYF and control students in different courses have produced the positive gains for FYF students over control students because FYF materials were used in some courses that had no control counterparts. This course issue was addressed with the results presented in column 3 of Table 5. In this case, the regression analysis was based on a restricted sample of 514 students taking only an economics course to test the robustness of the estimated value of the FYF education when isolating its effect within one course type. Six teachers taught this restricted sample of students. This same course analysis was possible because the largest number of students tested had taken an economics course, which was the main course target for the FYF curriculum. For the economics students, the estimated contribution to financial knowledge was 21.3 percentage points, which was highly significant at the .01 level.

CONCLUSIONS

  1. Top of page
  2. Abstract
  3. BACKGROUND AND PRIOR STUDIES
  4. FINANCIAL INSTRUCTION AND STUDY DESIGN
  5. SAMPLE AND MEASURES
  6. RESULTS
  7. CONCLUSIONS
  8. REFERENCES

Instruction in personal finance in high school has the potential to improve the financial knowledge of high school students, but questions have been raised in some studies about whether financial education does have a positive effect on high school students' financial knowledge. This study investigated this relationship under research conditions that controlled for consistency in the content coverage for financial education, assured that the personal finance instruction was provided by well-trained teachers, used a financial education test aligned with what was being taught and measured the initial level of understanding of personal finance to capture the improvement in financial knowledge. The regression analysis also accounted for the effects of a wide assortment of student characteristics that may influence financial knowledge. These practices are consistent with recommendations for a more systematic and comprehensive approach to evaluation (Fox, Bartholomae, and Lee 2005; Lyons et al. 2006).

The study, for example, devoted a substantial effort to the issue of measurement because the quality of the test measure has an important influence on the results and how they are interpreted. The researchers constructed an achievement test that was a reliable and valid instrument to assess student knowledge of personal finance taught with the FYF curriculum. In addition, the instrument was valuable for assessing student responses on subcategories of personal finance content. This measurement work would be considered an important part of the fourth tier (progress-toward-objectives) in Jacobs' model of evaluation described in Fox, Bartholomae, and Lee (2005, p. 213).

Another feature of the evaluation is its use of a quasi-experimental design with a treatment and control group and a pretest and posttest for each group (Shadish, Cook, and Campbell 2002). A comparison of the two groups showed that the control group of students was quite similar to the FYF treatment group of students because both groups were drawn from the same or very similar locations and conditions. The major advantage of a control group of students who did not receive FYF instruction was that it served as a baseline to assess whether FYF instruction improves student knowledge of personal finance. The inclusion of a control group is important because it enables a study to reach the fifth tier (program impact) in Jacobs' model of evaluation, but is rarely used in program evaluations in financial education according to Fox, Bartholomae, and Lee (2005, pp. 207–208).

The fixed-effect regression results showed that financial education does make a positive and important contribution to a high school student's knowledge of personal finance after controlling for other explanatory factors. These factors included ones such as type of high school course, gender, educational level, educational or work plans after high school, work history during high school, credit card use and teacher effects. The analysis also revealed that financial education can improve financial knowledge across a range of different courses taught in the school curriculum. Positive and significant effects from FYF instruction from pretest to posttest also were found across student characteristics. These general conclusions, however, about the effectiveness of FYF instruction in improving financial knowledge only apply to students who had just received such instruction. It is not known if students retain this increase in knowledge after the program ends and over time.

One final point should be kept in mind. These results apply to high school instruction in financial education, which is designed to develop general knowledge and understanding of various topics in personal finance. Although some of this financial education may have immediate application to issues or concerns of youth, the major purpose of most of this financial education is to build a foundation for greater understanding or an orientation to the financial world that will be useful at a later point in life after high school graduation. This financial education for youth, therefore, often differs from the financial education for adults, which is primarily designed to address an immediate financial problem or decision related to a particular financial matter. For these reasons, caution should be exercised when extrapolating from these findings for the effects of financial education of youth to the financial education of adults.

REFERENCES

  1. Top of page
  2. Abstract
  3. BACKGROUND AND PRIOR STUDIES
  4. FINANCIAL INSTRUCTION AND STUDY DESIGN
  5. SAMPLE AND MEASURES
  6. RESULTS
  7. CONCLUSIONS
  8. REFERENCES
  • Bernanke, Ben S. 2006. Financial Literacy. Testimony before the Committee on Banking, Housing, and Urban Affairs of the United States Senate. http://www.federalreserve.gov/newsevents/testimony/Bernanke20060523a.htm. (Accessed on May 23, 2006).
  • Bernheim, B. Douglas, Daniel M. Garrett, and Dean M. Maki. 2001. Education and Saving: The Long-Term Effects of High School Financial Curriculum Mandates. Journal of Public Economics, 85(June): 435465.
  • Bosshardt, William, and Michael Watts. 1990. Instructor Effects and Their Determinants in Precollege Economic Education. Journal of Economic Education, 21(Summer): 265276.
  • Braunstein, Sandra, and Carolyn Welch. 2002. Financial Literacy: An Overview of Practice, Research, and Policy. Federal Reserve Bulletin (November): 446457.
  • Council for Economic Education (CEE). 2006. Financing Your Future (DVD). New York: CEE. http://financingyourfuture.councilforeconed.org/.
  • Council for Economic Education (CEE). 2009. Survey of the States: Economics, Personal Finance and Entrepreneurship Education in Our Nation's Schools in 2009. New York: CEE. http://www.councilforeconed.org/about/survey2009/CEE_Survey_2009.pdf.
  • Cronbach, Lee J. 1951. Coefficient Alpha and the Internal Structure of Tests. Psychometrica, 16: 297334.
  • Danes, Sharon M., and Heather R. Haberman. 2007. Teen Financial Knowledge, Self-Efficacy, and Behavior: A Gendered View. Journal of Financial Counseling and Planning, 18 (2): 4860.
  • Danes, Sharon M., Catherine Huddleston-Casas, and Laurie Boyce. 1999. Financial Planning Curriculum for Teens: Impact Evaluation. Journal of Financial Counseling and Planning, 10 (1): 2637.
  • Fox, Jonathan, Suzanne Bartholomae, and Jinkook Lee. 2005. Building the Case for Financial Education. Journal of Consumer Affairs, 39(Summer): 195214.
  • Harter, Cynthia L., and John F.R. Harter. 2009. Assessing the Effectiveness of Financial Fitness for Life in Eastern Kentucky. Journal of Applied Economics and Policy, 28 (1): 2033.
  • Hilgert, Marianne S., Jeanne M. Hogarth, and Sondra G. Beverly. 2003. Household Financial Management: The Connection between Knowledge and Behavior. Federal Reserve Bulletin (July): 310322.
  • Jacobs, Francine H. 1988. The Five-Tiered Approach to Evaluation: Context and Implementation. In Evaluating Family Programs, edited by Heather B.Weiss and Francine H.Jacobs (3768). New York, NY: Aldine de Gruyter.
  • Jump$tart Coalition for Personal Financial Literacy. 2007. National Standards in K–12 Personal Financial Education. 3rd edition. Washington, DC: Jump$tart Coalition. http://www.jumpstart.org/guide.html.
  • Lopez-Fernandini, Alejandra, and Karen Murrell. 2008. The Effectiveness of Youth Financial Education. Washington, DC: New America Foundation. http://www.newamerica.net/publications/policy/effectiveness_youth_financial_education_1.
  • Lucey, Thomas A. 2005. Assessing the Reliability and Validity of the Jump$tart Survey of Financial Literacy. Journal of Family and Economic Issues, 26 (2): 283294.
  • Lyons, Angela C., Lance Palmer, Koralalage S.U. Jayaratne, and Erik Scherpf. 2006. Are We Making the Grade? A National Overview of Financial Education and Program Evaluation. Journal of Consumer Affairs, 40(Winter): 208235.
  • Mandell, Lewis. 1998. Our Vulnerable Youth: The Financial Literacy of American 12th Graders. Washington, DC: Jump$tart Coalition.
  • Mandell, Lewis. 2008. Financial Education in High School. In Overcoming the Saving Slump: How to Increase the Effectiveness of Financial Education and Saving Programs, edited by AnnamariaLusardi (257279). Chicago, IL: University of Chicago Press.
  • Martin, Matthew. 2007. A Literature Review on the Effectiveness of Financial Education. Working Paper 07-03. Richmond, VA: Federal Reserve Bank of Richmond.
  • McCormick, Martha H. 2009. The Effectiveness of Youth Financial Education: A Review of the Literature. Journal of Financial Counseling and Planning, 20 (1): 7083.
  • Morton, John S., and Mark C. Schug. 2001. Financial Fitness for Life: Teacher Guide, Grades 9–12. New York: Council for Economic Education.
  • Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inferences. Boston: Houghton-Mifflin.
  • Swinton, John R., Thomas W. De Berry, Benjamin Scafidi, and Howard C. Woodard. 2007. The Impact of Financial Education Workshops for Teachers on Students' Economic Achievement. The Journal of Consumer Education, 24: 6377.
  • Tennyson, Sharon, and Chau Nguyen. 2001. State Curriculum Mandates and Student Knowledge of Personal Finance. Journal of Consumer Affairs, 35(Winter): 241262.
  • Varcoe, Karen P., Allen Martin, Zana Devitto, and Charles Go. 2005. Using a Financial Education Curriculum for Teens. Journal of Financial Counseling and Planning, 16 (1): 6371.
  • Walstad, William B., and Ken Rebeck. 2005a. Financial Fitness for Life: High School Test Examiner's Manual (grades 9–12). New York: Council for Economic Education.
  • Walstad, William B., and Ken Rebeck. 2005b. Financial Fitness for Life: Middle School Test Examiner's Manual (grades 6–8). New York: Council for Economic Education.
Footnotes
  • 1

    Teachers for both the FYF and control groups asked for the full cooperation of students for the testing and emphasized the importance of the project. The test instrument was given in combination with other survey questions and was not designed to be used for grading, nor were answers to the test questions given to teachers so they could grade the test. No reports came from the teachers that the test counted for students' grades or indicated that student groups responded differently to testing. The pretest means for both groups also were essentially equivalent (see Table 3) indicating that there was no apparent difference in testing response by group.

  • 2

    The FYF test items that were taken from the FFL senior high test were 1, 2, 4, 6, 7, 8, 9, 10, 12, 16, 20, 23, 25, 26, 29 and 30 (Walstad and Rebeck 2005a). Item 13 came from the middle school FFL test (Walstad and Rebeck 2005b). Items 3, 5, 11, 14, 15, 17, 18, 19, 21, 22, 24, 27 and 28 were newly written for the FYF test.

  • 3

    Eleven items had point-biserial correlation greater than .4, and fourteen other items had point-biserial correlations ranging from .3 to .4. None of the items had negative point-biserial correlations.

  • 4

    For example, the reliability estimates for the FYF subscales (alphas: .43 to .62) are comparable with or higher than the reliability estimates for the subscales on the 2000 Jump$tart knowledge test (alphas: .23 to .59) (see Lucey 2005).

  • 5

    The one exception was a multiple-choice item (#10) asking students to correctly identify lower future earnings as a likely consequence of dropping out of high school. This item was the easiest one on the test (81.1% correct on the pretest), which limited its potential to measure the change in knowledge.

  • 6

    Other studies of high school curricula in financial education have reported gender differences in financial knowledge (Danes and Haberman 2007; Varcoe et al. 2005).

  • 7

    Alternative specifications of this equation that used the posttest minus the pretest as the dependent variable suggest that the drop in the estimated gain found in test score means (Table 3) from 19.7 percentage points for FYF students to 15.9 percentage points in the regression was from taking into account the pretest level of knowledge in the regression equations. In fact, when the pretest was removed from that equation, the estimated coefficient on the FYF variable was 20.2, with or without the inclusion of the other explanatory variables.

  • 8

    The dummy variable included both students in economics and personal finance courses, but few students in the entire sample (only fifty-five) took a personal finance course, so the dummy effect largely reflected taking an economics course. Further analysis was conducted with only FYF and control students taking an economics course (n = 514), as shown in column 3 of Table 5.