Original document at http://epaa.asu.edu/epaa/v7n8/

A critique of this study

 

This article has been retrieved  times since March 23, 1999

Education Policy Analysis Archives
Volume 7 Number 8	March 23, 1999	ISSN 1068-2341

A peer-reviewed scholarly electronic journal Editor: Gene V Glass, College of Education Arizona State University Copyright 1999, the EDUCATION POLICY ANALYSIS ARCHIVES. Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education.

Scholastic Achievement and Demographic Characteristics of Home School Students in 1998 Lawrence M. Rudner ERIC Clearinghouse on Assessment and Evaluation College of Library and Information Services University of Maryland, College Park This article has Commentary Abstract           This report presents the results of the largest survey and testing program for students in home schools to date. In Spring 1998, 20,760 K-12 home school students in 11,930 families were administered either the Iowa Tests of Basic Skills (ITBS) or the Tests of Achievement and Proficiency (TAP), depending on their current grade. The parents responded to a questionnaire requesting background and demographic information. Major findings include: the achievement test scores of this group of home school students are exceptionally high--the median scores were typically in the 70th to 80th percentile; 25% of home school students are enrolled one or more grades above their age-level public and private school peers; this group of home school parents has more formal education than parents in the general population; the median income for home school families is significantly higher than that of all families with children in the United States; and almost all home school students are in married couple families. Because this was not a controlled experiment, the study does not demonstrate that home schooling is superior to public or private schools and the results must be interpreted with caution. The report clearly suggests, however, that home school students do quite well in that educational environment.

By current estimates, there are between 700,000 and 1,200,000 students enrolled in home schools in the United States. Further, by all accounts, the movement has been growing steadily over the past few years (Lines, 1998). Yet, there is very little scientific literature concerning the population of home school students or even large samples of home school students.           This study describes the academic achievement levels and some basic demographic characteristics of a large sample of students and their families. While the academic levels of home school students are described in terms of public and private school norms, this study is not a comparison of home schools with public or private schools. Such comparisons would be fraught with problems. Home schooling is typically one-on-one. Public schools typically have classes with 25 to 30 students and an extremely wide range of abilities and backgrounds. Home school parents are, by definition, heavily involved in their children's education; the same, unfortunately, is not true of all public or private school parents. Home schools can easily pace and adapt their curriculum; public and private schools typically have a mandated scope and sequence. The list of differences could continue.           This study seeks to answer a much more modest set of questions: Does home schooling tend to work for those who chose to make such a commitment? That is, are the achievement levels of home school students comparable to those of public school students? Who is engaged in home schooling? That is, how does the home school population differ from the general United States population?

Methods           Bob Jones University Press Testing and Evaluation Service provides assessment services to home school students and private schools on a fee-for-service basis. In Spring 1998, 39,607 home school students were contracted to take the Iowa Tests of Basic Skills (ITBS; grades K-8) or the Tests of Achievement and Proficiency (TAP; grades 9-12). Students were given an achievement test and their parents were asked to complete a questionnaire entitled "Voluntary Home School Demographic Survey." A total of 20,760 students in 11,930 families provided useable questionnaires with corresponding achievement tests. The achievement test and questionnaire results were combined to form the dataset used in this analysis.           This section provides descriptions of the achievement measures, the questionnaires, the Bob Jones University Press Testing and Evaluation Service, and the procedures used to develop the dataset. Iowa Tests of Basic Skills (ITBS)           Home schooled students in Grades K-8 took the Iowa Tests of Basic Skills (ITBS) Form L, published by Riverside Publishing Company, a subsidiary of Houghton Mifflin. Developed by University of Iowa professors, the tests were designed and developed to measure skills and standards important to growth across the curriculum in the nation's public and private schools.           The ITBS reflects more than 50 years of test development experience and research on measuring achievement and critical thinking skills in Reading, Language Arts, Mathematics, Social Studies, Science, and Information Sources. The scope and sequence of the content measured by the ITBS were developed after careful review of national and state curricula and standards, current textbook series and instructional materials, and research (Riverside, 1993).           All items were tried out and tested for ethnic, cultural, and gender bias and fairness prior to the development of the final form of the tests. Data on a nationally representative sample of public and private schools were collected in 1992 and used to form the initial national norms. The norms were updated in 1995 by Riverside. This study used these 1995 spring norms. Tests of Achievement and Proficiency (TAP)           Home schooled students in Grades 9-12 took the Tests of Achievement and Proficiency (TAP), Form L, also published by Riverside Publishing Company. The TAP was designed and developed to measure skills and standards important to growth across the high school curriculum. Like the ITBS, the TAP scope and sequence were developed after careful review of national and state curricula and standards, and current textbook series and instructional materials. Developed as an upward extension of the ITBS, the specifications, format, and design of the TAP tests are similar to that of the ITBS. TAP is fully articulated with the Iowa Tests of Basic Skills (ITBS) Form L (Riverside, 1993). Background Questionnaires           Background questionnaires were designed by the staff of the Home School Legal Defense Association (HSLDA). Questions were determined by reviewing the questions in previous surveys, prioritizing them, and selecting only those that were most germane to the objectives of the study. Where possible, questions and responses were made to match those used by the U.S. Census, U.S. Department of Labor and the National Assessment of Educational Progress to facilitate comparisons of home school students with students nationwide.           HSLDA designed the survey to be much shorter than previous survey instruments. They also sought to pose all questions in an objective format, rather than a constructed response format. In keeping with this approach, HSLDA worked with National Computer Systems to design forms to be computer scanable, thereby removing the need for manual data processing. Bob Jones University Press Testing and Evaluation Service           The Bob Jones University (BJU) Press Testing and Evaluation Service is the largest and oldest of four organizations providing home school families access to standardized achievement tests. The Testing Service began offering the Iowa Tests of Basic Skills and Tests of Achievement and Proficiency in 1984. In subsequent years they added other helpful tools including practice materials, a personality inventory, and diagnostic tests. In 1993, the Stanford Achievement Test series was added as BJU Press assumed the testing that the Home School Legal Defense Association had been providing for its members. Since that time, a full range of writing evaluations (grades 3-12) and a career assessment have been added to the growing number of evaluation tools offered by the Testing Service.           Just as home school families were the impetus behind the start of the Testing Service, home school families continue to be the largest sector utilizing the service. However, there are also a number of private schools that have chosen to use the services provided. Testing is provided for students throughout the United States and Canada, as well as many foreign countries.           The BJU Press Testing and Evaluation Service sends testing materials to qualified testers who administer the tests and return them to the Testing Service for scoring. The results are then returned to the parent. Many parents test primarily for their own information to verify that their home schooled students are progressing academically at a normal pace. Other parents use the results to meet a state testing requirement or to provide documentation when they choose to return their students to a public or private school setting. Data Generation Procedures           The following steps were followed to produce the data set: Parents contracted with Bob Jones University to be administered the Iowa Tests of Basic Skills or the Tests of Achievement Proficiency (39,607 students in probably 22,000 families). Bob Jones certified test administrators, many of whom were the students' parents. BJU sent questionnaires and answer forms to the test administrators. Tests and questionnaires were returned to BJU. BJU bundled the tests and sent them to Riverside Publishers for machine scoring. BJU bundled the questionnaires and sent them to National Computer Systems for scanning. Unlike in previous studies, the parents did not know their scores ahead of time. Electronic copy of the 23,415 test results and 23,311 questionnaire results were sent to the author of this report. These sets were merged to provide 20,900 cases with matching identification numbers. In order to weight by state public school enrollment, 140 cases with missing state data were dropped. A total of 20,760 students formed the initial dataset used in the study. After we formed the dataset with 20,760 students, we asked for the remainder of the 39,607 achievement test scores. We were informed that it would not be possible to disaggregate the remaining home school students from students in private schools also contracting testing services.

Characteristics of Home School Students and Families           This section provides a description of home school students and their families based on the 20,790 respondents to our questionnaire. The distribution of students by state, gender, age, race, parent marital status, family size, mother's religion, parent education, family income, television viewing, money spent on educational materials, and other demographic characteristics are identified and, where possible, compared to national figures. State           As shown in Table 2.1, respondents came from each of the fifty states. Several states, including Ohio, Georgia, and Virginia, have exceptionally high representation given their size. This is probably due to the fact that these states require testing of home school students. To reduce the effects of these and other overrepresented states, the data were weighted in all subsequent analyses by the number of public school students in each state. While we would have preferred to weight by the number of home schooled students in each state, such data are not available for all 50 states (Lines, 1998). Table 2.1 Participating Home School Students Classified by State State Freq.   Percent of sample AK 61 .3% AL 181 .9 AR 42 .2 AZ 201 1.0 CA 815 3.9 CO 810 3.9 CT 54 .3 DC 17 .1 DE 28 .1 FL 860 4.1 GA 1547 7.4 GU 10 .0 HI 112 .5 IA 234 1.1 ID 28 .1 IL 451 2.2 IN 533 2.6 KS 319 1.5 KY 163 .8 LA 551 2.7 MA 343 1.6 MD 196 .9 ME 109 .5 MI 523 2.5 MN 794 3.8 MO 361 1.7 MS 25 .1 State Freq.   Percent of sample MT 112 .5 NC 972 4.7 ND 100 .5 NE 126 .6 NH 176 .8 NJ 324 1.6 NM 189 .9 NV 53 .3 NY 942 4.5 OH 2484 11.9 OK 382 1.8 OR 67 .3 PA 532 2.6 PR 8 .0 RI 32 .2 SC 579 2.8 SD 27 .1 TN 322 1.5 TX 1126 5.4 UT 35 .2 VA 1608 7.7 VI 2 .0 VT 59 .3 WA 787 3.8 WI 246 1.2 WV 92 .4 WY 40 .2 Student Age and Gender           Table 2.2 shows the distribution of the respondents by gender and age. About 50.4% or 10,471 of the respondents were females; 49.6% (10,319) were males. These figures are comparable to that of the population of 3 to 34 years old enrolled in school (see U.S. Bureau of the Census, 1998, Table A-2). Some 51.4% of school enrollees nationally are male. The percentages are comparable at all age levels. Table 2.2 Participating Home School Students Classified by Gender and Age Age at time of testing (in years) 6 7 8 9 10 11 12 13 14 15 16 17 Females 507 879 1148 1318 1301 1248 1049 936 774 516 264 119 56.1% 51.7% 50.2% 49.2% 52.4% 50.6% 47.2% 50.5% 50.7% 51.0% 49.3% 57.5% Males 397 820 1141 1360 1181 1216 1174 918 754 495 271 88 43.9% 48.3% 49.8% 50.8% 47.6% 49.4% 52.8% 49.5% 49.3% 49.0% 50.7% 42.5% Total 904 1699 2289 2678 2482 2464 2223 1854 1528 1011 535 207 Student Grade           Home school student grade placement was identified by their parents, presumably based on the grade level of the instructional materials. That grade was used by BJU to determine the test levels and used in this report as a grouping variable. Tables 2.3 shows the distribution of respondents and the nation by grade. There is a large difference in the proportions of high school (grades 9-12) home school students and the nation. Compared to the national data, a relatively small percentage of home school students are enrolled in high school. Possible reasons for this lower participation for high school students may be the relative newness of the home school movement, early graduation from high school, and possibly a desire on the part of some home school parents to enroll their children in a traditional high school. The distributional differences for students in grades 1 through 8 are minor. Table 2.3 Home School Students Classified by Grade with Percents and National School Percents Grade   1 2 3 4 5 6 7 8 9 10 11 12 Home school 7.4% (1504) 10.6% (2153) 14.1% (2876) 12.9% (2625) 12.6% (2564) 11.9% (2420) 10.3% (2087) 8.8% (1801) 5.7% (1164) 3.8% (775) 1.6% (317) 0.3% (66) Nation 9.1% 8.8% 8.9% 8.7% 8.6% 8.7% 8.7% 8.4% 9.0% 7.9% 7.1% 6.3% National data: US Census, 1997b, Table 254. Student Race           Table 2.4 shows the racial distribution of home school students in 1998 and for the students enrolled in elementary and secondary public and private schools nationally in 1994. The distributions are quite different. The vast majority of home schooled children are non-Hispanic White. The largest minority groups for home school students (not shown in the table) are American Indians and Asian students who comprise some 2.4% and 1.2% of the home school students, respectively. Table 2.4 Racial Distribution of Home School Students And the Nation, in Percents White (not Hispanic) Black (not Hispanic) Hispanic Other Home school 94.0% 0.8% 0.2% 5.0% Nationwide 67.2% 16.0% 13.0% 3.8% (National data: USDE, 1996; Indicator 27)   Marital Status           The great majority of home school students are in married couple families. In contrast, only 72% of the families with at least one child enrolled in school nationwide are in married couple families (Bruno and Curry, 1997, Table 19). Table 2.5 Home School Students Classified by Parents' Marital Status Marital Status Frequency Percent Divorced 80 0.7% Single (never married) 44 0.4 Married 11,335 97.2 Separated 131 1.1 Widowed 55 0.5 Missing data 16 0.1     11,661 100.0% Children at Home           Table 2.6 shows the distribution of children in home school families and families with children under 18 nationwide. On average, home school students are in larger families. Nationwide, most families with school-age children (79.6%) have only 1 or 2 children with a mean of about 1.9 children per family. Most home school families (62.1%) have 3 or more children with a mean of about 3.1 children per family. Table 2.6 Home School Families Classified by Family Size with National Comparison Home School Families Nationwide Number of Children Percent Number of Children Percent 1 8.3% 1 40.8% 2 29.6 2 38.8 3 28.6 3 14.3 4 18.6 4 or more 6.1 5 8.4 6 3.9   7 or more 2.6   National Data: US Census, 1997a, Table 77 Mother's Religion           We asked the home school families to identify the religious preference of each student's mother by selecting from a list of 27 religions. As shown in Table 2.7, the largest percentage of mothers identified themselves as Independent Fundamental, Baptist, Independent Charismatic, Roman Catholic, Assembly of God, or Presbyterian. The religious preference of the father was the same as that of the mother 93.1% of the time. Table 2.7 Home School Students Classified by Mother's Religion   Frequency Percent Independent Fundamental 5,119 25.1% Baptist 5,072 24.4 Independent Charismatic 1,681 8.2 Roman Catholic 1,106 5.4 Assembly of God 838 4.1 Presbyterian 772 3.8 Reformed 685 3.4 Other Protestant 500 2.5 Pentecostal 459 2.2 Methodist 420 2.1 Lutheran 353 1.7 Other Christian 2,213 10.9 Other 1,572 6.2   Total 20,790 100.0% Parent Academic Attainment           As shown in Table 2.8, home school parents have more formal education than the general population. While slightly less than half of the general population attended or graduated from college, almost 88% of home school students have parents who continued their education after high school. Table 2.8 Distribution of Home School Students and Students Nationally Classified by Parent Academic Attainment Percent Did not finish high school High school graduate Some college, no degree Associate degree Bachelors degree Masters degree Doctorate   Home school fathers 1.2%     9.3%     16.4%    6.9%     37.6%    19.8%    8.8%     Nation males   18.1      32.0      19.5      6.4       15.6      5.4       3.1         Home school mothers 0.5       11.3      21.8      9.7       47.2      8.8       0.7       Nation females   17.2      34.2      20.2      7.7       14.8      4.5       1.3                   National data: U.S. Census (1996; Table 8) Family Income           National data on family income are available for 1995. As shown in Table 2.9, home school families span all income levels. On average, home school families have a higher income level than do families with children nationwide and all families nationwide. The median family income level for home school families in 1997 is about $52,000. The median income for families with children in 1995, nationwide, was about $36,000. Table 2.9 Distribution of Family Income for Home School Families, Families with Children Nationwide, and All Families Nationwide by Income Levels, in Percents.     Home school Families with children All families Less than $10,000 0.8% 12.6% 10.5% $10,000 to $14,999 1.5     8.0     8.5     $15,000 to $19,999 2.2     6.1     6.8     $20,000 to $24,999 3.9     7.6     8.4     $25,000 to $29,999 4.9     7.5     7.8     $30,000 to $34,999 8.5     7.5     7.6     $35,000 to $39,999 8.1     7.1     7.0     $40,000 to $49,999 16.0     11.3     11.0     $50,000 to $74,999 32.5     18.4     18.1     $75,000 and over 21.6     13.8     14.3     National data: Bruno and Curry (1997, Table 19) Television Viewing           The National Assessment of Educational Progress collects information on the television viewing habits of fourth-graders. Home school fourth-graders and fourth-graders nationally differ markedly in terms of television viewing. Home school students rarely watch more than 3 hours of television per day; nearly 40% of the students nationwide watch that much television. Table 2.10 Fourth-grade students Classified by Hours of Television Viewing   Percent of students 6 or more hours per day 4 to 5 hours per day 2 to 3 hours per day 1 hour or less per day Home school 0.1%   1.6      33.1      65.3      Nationwide 19.0%   19.5       36.4       25.1       National data: NAEP Math 1997 Computer Use           The Condition of Education provides a tabulation of the percent of students nationwide who report using a computer by frequency of use for 4th, 8th, and 11th graders in 1996. At each grade level, the distribution of computer use in 1998 by home school students is different from that of the nation in 1996. At each of these three grade levels, much larger percentages of home school students never use a computer. At the fourth-grade level, a much larger percent of home school students use a computer every day. Table 2.11 Computer Use among Home School Students and Students Nationwide in Grades 4, 8, and 11, in Percent Grade 4 Grade 8 Grade 11 Home school Nationwide Home school Nationwide Home school Nationwide Never 28.2%   11.4%   37.1%   23.3%   40.5 %  16.0%   Less than once a week 29.4     16.3     28.9     29.2     28.9     34.2     Several times a week 21.6     62.5     18.0     30.7     17.5     31.8     Every day 20.8    9.9     16.0    16.7    13.1    18.1    National Data: Snyder and Wirt, 1998, Indicator 3. Money Spent on Educational Materials           The amount of money spent in 1997 on home school education for textbooks, lesson materials, tutoring and enrichment services, and testing ranged from less than $200 to more than $2000. As shown in Table 2.12, the median amount of money spent was about $400. Table 2.12 Home School Students Classified by Money Spent On Home School Education in 1997 Amount Frequency Percent <$200 3,718 17.9% 200-399 7,035 33.8     400-599 4,467 21.5     600-799 1,962 9.4     800-999 985 4.7     1,000-1,599 1,630 7.8     1,600-1,999 247 1.2     >2,000 411 2.0     Missing 336 1.6     Total 20,790 100.0%   Other Demographic Characteristics           Compared to the nation, a much larger percentage of home school mothers are stay-at-home mothers not participating in the labor force. Some 76.9% of home school mothers do not work for pay. About 86.3% that do work do so part time. Nationwide, in 1996, only 30% of married women with children under 18 did not participate in the labor force (US Dept of Census, 1997a, Table 632).           A very large percentage of home school parents are certified to teach. Some 19.7% of the home school mothers are certified teachers; 7.1% of fathers. Almost one out of every four home school students (23.6%) has at least one parent who is a certified teacher.           Only 7.7% of the respondents were enrolled in a full-service curriculum program, i.e., a program that serves students and their parents as a "one-stop" primary source for textbooks, materials, lesson plans, tests, counseling, evaluations, record keeping, and the like for the year's core required subjects such as language, social studies, mathematics, and science.

Academic Achievement           The complete batteries of The Iowa Tests of Basic Skills (ITBS) and the Tests of Achievement and Proficiency (TAP) were used to assess student achievement in basic skills. The ITBS was used for home school students in Grades K-8; the TAP for students in grades 9-12. Almost all students took Form L; a handful took parallel Form K.           Achievement test batteries like the ITBS and TAP are a collection of tests in several subject areas that have been standardized and normed. Norms for all tests within these test batteries are based on the same grou