by Shona Carter
Student educational proficiency rates vary widely across the United States. In an effort to learn what may be causing these variations, I will test whether the length of the school day influences student proficiency. I suppose that longer school hours result in more learning and better rates of proficiency. Some educators argue that quality (of curriculum) over quantity (hours) is the best way to increase student proficiency. I believe that time spent studying in school has major effect on the percentage of students that meet proficient standards for their grade level. The National Assessment of Educational Progress (NAEP) publishes a well-respected index for math and reading proficiency for grades 4, 8 and 12 for all 50 states. The question as to whether longer school hours increase test performance will be answered using both the average math and reading proficiency rates of 4th and 8th graders in 50 states for the 2007-2008 school year. I have chosen to omit 12th grade data because high school students have additional college/test preparations that may contribute their proficiency. I will consider the impact of the average number of hours spent in elementary school on the proficiency rates for each state. The proficiency data shows the percentage of students in a particular state and grade who are math and reading proficient. My measure of school hours takes the average amount of time spent in school by state for elementary school students. This analysis will take on a longitudinal approach in that the primary school hours will be observed for impact on both the current fourth grade and eight grade students. The average number of school hours is published by the National Center for Education statistics.
These regressions are important for making educational improvements in the U.S. If it is true that longer school hours lead to an increase in proficiency rates, then the states with lower proficiency rates and shorter school hours may benefit from increasing the amount of school time in order to increase student proficiency.
Since it is possible that there are other factors that may interfere with this relationship, I have chosen to consider the average income in each state, the amount that each state spends per pupil, and the rural-ness of each state. There is enough evidence to suggest that "Children who perform better on NAEP tests also tend to come from states with lower levels of student poverty”. Wealth has been traditionally associated with high achievement, often due to additional resources that wealthier families have to devote to education. The amount of spending per pupil is also related to relative wealth, but it is not true that all the states with the highest average incomes spend more money per pupil because some lower levels of spending can be better effective than more spending. I have also chosen to include the effect that rural states have on the relationship. It’s possible that states that are more rural have less school hours due to the need to have more students working in farming and agriculture. The National Center for Education Statistics publishes the amount that each state spends per pupil. The United States Census Bureau publishes an index that ranks urban and rural states, as well as the average income per state. All of these additional factors, which are known as confound variables, will be controlled for.
I do suspect that quality or rigor of school programs can also contribute to the variation in proficiency. The idea is that wealthier states may have stronger school performance based on a more rigorous academic approach. Since rigor can be difficult to test for empirically, I would assume that there is a strong correlation between rigor and wealth, and will therefore be largely accounted for in my regressions.
Results/Causality
4th and 8th grade math and reading proficiency, school time, and confounds are included in 4 regressions.
Table 1. Regression Table
Student educational proficiency rates vary widely across the United States. In an effort to learn what may be causing these variations, I will test whether the length of the school day influences student proficiency. I suppose that longer school hours result in more learning and better rates of proficiency. Some educators argue that quality (of curriculum) over quantity (hours) is the best way to increase student proficiency. I believe that time spent studying in school has major effect on the percentage of students that meet proficient standards for their grade level. The National Assessment of Educational Progress (NAEP) publishes a well-respected index for math and reading proficiency for grades 4, 8 and 12 for all 50 states. The question as to whether longer school hours increase test performance will be answered using both the average math and reading proficiency rates of 4th and 8th graders in 50 states for the 2007-2008 school year. I have chosen to omit 12th grade data because high school students have additional college/test preparations that may contribute their proficiency. I will consider the impact of the average number of hours spent in elementary school on the proficiency rates for each state. The proficiency data shows the percentage of students in a particular state and grade who are math and reading proficient. My measure of school hours takes the average amount of time spent in school by state for elementary school students. This analysis will take on a longitudinal approach in that the primary school hours will be observed for impact on both the current fourth grade and eight grade students. The average number of school hours is published by the National Center for Education statistics.
These regressions are important for making educational improvements in the U.S. If it is true that longer school hours lead to an increase in proficiency rates, then the states with lower proficiency rates and shorter school hours may benefit from increasing the amount of school time in order to increase student proficiency.
Since it is possible that there are other factors that may interfere with this relationship, I have chosen to consider the average income in each state, the amount that each state spends per pupil, and the rural-ness of each state. There is enough evidence to suggest that "Children who perform better on NAEP tests also tend to come from states with lower levels of student poverty”. Wealth has been traditionally associated with high achievement, often due to additional resources that wealthier families have to devote to education. The amount of spending per pupil is also related to relative wealth, but it is not true that all the states with the highest average incomes spend more money per pupil because some lower levels of spending can be better effective than more spending. I have also chosen to include the effect that rural states have on the relationship. It’s possible that states that are more rural have less school hours due to the need to have more students working in farming and agriculture. The National Center for Education Statistics publishes the amount that each state spends per pupil. The United States Census Bureau publishes an index that ranks urban and rural states, as well as the average income per state. All of these additional factors, which are known as confound variables, will be controlled for.
I do suspect that quality or rigor of school programs can also contribute to the variation in proficiency. The idea is that wealthier states may have stronger school performance based on a more rigorous academic approach. Since rigor can be difficult to test for empirically, I would assume that there is a strong correlation between rigor and wealth, and will therefore be largely accounted for in my regressions.
Results/Causality
4th and 8th grade math and reading proficiency, school time, and confounds are included in 4 regressions.
Table 1. Regression Table
4th
Grade Reading
|
8th
Grade Reading
|
4th
Grade Math
|
8th
Grade Math
|
|
Elementary Hours
|
17.177*
|
23.947*
|
26.120**
|
18.034
|
(2.249)
|
(2.673)
|
(3.150)
|
(1.723)
|
|
Spending Per
Pupil
|
-0.001
|
-0.001
|
0.000
|
-0.001
|
(-0.697)
|
(-0.602)
|
(0.185)
|
(-0.598)
|
|
Rural Percentage
|
0.249
|
0.073
|
-0.056
|
-0.028
|
(1.973)
|
(0.494)
|
(-0.406)
|
(-0.165)
|
|
Income By
State
|
0.000
|
0.001*
|
0.000
|
0.000
|
(1.669)
|
(2.144)
|
(1.273)
|
(0.991)
|
|
Constant
|
-65.164
|
-122.210
|
-122.032
|
-69.816
|
(-1.135)
|
(-1.814)
|
(-1.957)
|
(-0.884)
|
|
R-Squared
|
0.16
|
0.10
|
0.11
|
-0.01
|
48
|
48
|
48
|
47
|
|
The results show that when taken into account the three control
variables, school hours has an effect on the proficiency rates for eight grade
reading, fourth grade reading, and fourth grade math. It does not, however, show a significant effect on 8th grade math, although it comes very close to suggest that there may be a relationship.The R squared values
are not particularly high and therefore time does not explain a large portion of the variation in proficiency rates in each state. The graphs all show a positive relationship between
school hours and proficiency.
Scatter Plots
The proficiency rates show
evidence of association to school hours, which is part of the evidence for establishing causality. I believe that a proper time
order sequence has been established being that the test shows no evidence that
test scores cause variation in school hours (the school hours are similar by
state outside of the 2007-2008 school year). While holding constant the other
variables, the effect of school hours on proficiency rates is clearly shown. I would argue that the
answer to the question of whether longer school days contribute to better test
scores is yes. The hypothesis is true and shows the strongest relationship for
4th and 8th grade reading and math scores with the
proficiency rates for fourth grade having a particularly strong relationship.
Table 2. Summary Statistics
Mean
|
Std.
Dev.
|
Min.
|
Max.
|
||
4th
Grade Reading
|
74.96
|
11.06
|
46.0
|
90.0
|
|
8 Grade Reading
|
70.55
|
12.43
|
35.0
|
92.0
|
|
4th Grade Math
|
72.34
|
12.02
|
45.0
|
91.0
|
|
8th Grade Math
|
62.32
|
13.89
|
26.0
|
88.0
|
|
Elem. Hours
|
6.67
|
0.24
|
6.2
|
7.2
|
|
Income By State
|
50238.09
|
7568.33
|
37279.0
|
67576.0
|
|
Spending Per
Pupil
Rural
|
10984.62
27.22
|
2456.30
14.45
|
7336.0
5.32
|
17818.0
61.34
|
|
Observations
|
48
|
These regressions give good insight into the importance of classroom time and student learning. This information can help strengthen the idea that school days should be longer in lower performing states. Proponents of this position would have strong evidence to make the argument that students who spend the most time in school help to increase the proficiency rates in their state. It may not be an easy task to increase school hours, and there are likely many obstacles faced by a school district or state when trying to increase school time. For these schools, an alternative could be to consider other ways to increase time spent on learning in order to increase the amount of students who meet the educational standards for their grade level.
Sources
Digest of Education Statistics
http://nces.ed.gov/programs/digest/d09/tables/dt09_186.asp
NAEP
http://nces.ed.gov/nationsreportcard/studies/statemapping/2007_naep_state_table.aspx
NCES
http://nces.ed.gov/surveys/sass/tables/sass0708_035_s1s.asp
U.S. Census Bureau
http://www.census.gov/geo/reference/ua/urban-rural-2010.html
U.S. Census Bureau
http://www.census.gov/hhes/www/income/data/statemedian/
The research question and the importance of the question are both clear to the audience. The author answered the question but the effect of school hours on test scores can only be seen on 4th graders. The author explained the result clearly and the tables were readable and pretty. However, due to technical problem, I couldn't see the graphs and thus have no comment on that.
ReplyDeleteThere are some suggestion I would make to the author. First, the control variables may have multicollinearity problem as the average income of a state is associated with income tax as well as the money a state government can spend on education. As the author already thought of other potential confounds, she could include some of them in the next model.
Besides, it would be better if the author could explain why she chose the test scores of 4th graders and 8th graders as dependent variables but not the other grades and if she had expected to see the differences of effect on 4th graders and 8th graders before the research.
The research question of this blogpost is clear and concise. However, the variables are not clear: are the test scores average test scores across the state? If so, that makes sense and should be pointed out to the reader. No information on the data sources were provided, or the years from which they were taken. There are also several references to statistical terms without provision of context for the reader: X & Y, p-value and R-squared. On a small formatting note, the regression table is labelled incorrectly as a summary statistics table. The discussion on confounding factors was a bit confusing as it delved into school district level data and should not be compared with statewide data used for the IV and DV. In terms of the regression, because the sample is so small, maybe it would be better to do a dummy variable for 4th/8th grade rather than running them separately. Although I could not see your graphs, more references should have been made to them in the text. The text should explain to readers what they are seeing.
ReplyDeleteDear Shona, The research question is clear. However, it was confusing to me to understand why you chose to do the test results in four different groups instead of having only one Y value and more explanatory variables . If you were going to use other grades results maybe you could have used a bigger and random sample that could have included kids for other grades. Is there any particular reason you combined these two grades?. There are two graphs. The first one is incorrectly labeled, but it conveys important information about the regression of the results. I think there is a multicollinearity problem and maybe an ecological issue since you are using state and district level data. Regarding transformation, I am not sure if you had to transform any of your variables. I believe you could have included more explanatory variables to have a better model. In sum, i believe if you use only one Y or DV value this will make it much easier for the reader to understand the test and its results.
ReplyDelete