The Department of Education released recommended cut scores on the Florida Standards Assessments this week. Once approved, these scores will ultimately draw the lines between achievement levels and group students based on test performance.
The results highlight significant disparities in proficiency among subgroups, mostly between white and black students, and weaknesses in subject areas, specifically high school math. A comparison of FSA to former FCAT cut scores also raise questions about the process behind demarcating achievement levels and, once again, the test itself.
Historically, Leon County students have performed above the state average by a handful of percentage points, but trends — like demographic performance — mirror what is seen statewide. District-level data has not been released yet.
Four key findings from education panelists and a Q&A examining the nuts and bolts behind the process follow below.
1. The achievement gap is very, very real.
On every single FSA, a disproportionately high percentage of black students scored at the lowest achievement levels. DOE labels achievement levels as: 1 being inadequate; 2, below satisfactory; 3, satisfactory, or passing; 4, above satisfactory; and 5, mastery.
"The results tell us that work needs to be done," said Stu Greenberg, chief academic officer for Leon County Schools. "We are going to review these discrepancies and use other types of formative assessments because we want to take a look at the individual student."
There wasn't one exam where more than 43 percent of black students passed. However, at least 60 percent of white students passed all tests, with the exception of Algebra 2.
When looking at the top scores on every test, no more than five percent of black students hit mastery level. At least three times as many white students, however, earned a 5.
The widest disparity at the top is seen in the geometry end-of-course cut scores. Two percent of black students, compared to 14 percent of white students receive a 5. On the same test, 49 percent of black students and 17 percent of white students receive a 1.
2. We need help with math – desperately.
On the Algebra 1 EOC, 49 percent of all students were below satisfactory; on Algebra 2, 69 percent; and on Geometry, 51 percent. Algebra 1 and Geometry credits are required by state statutes to receive a standard high school diploma.
The added literacy component to math assessments — a majority of questions are word problems — presents new challenges, Greenberg said. Students who were able to answer straight-forward math problems in the past are now grappling with text where there used to be numbers.
3. Battle of the sexes? There really isn't one.
Academic achievement levels did not vary much between boys and girls taking the exams.
On every ELA test, more girls than boys earned a 5. However, fewer boys received a 1 or 2. The difference in percentages between boys and girls at the bottom or top of the achievement scale was negligible.
4. Is the FSA a more difficult test than the FCAT? The jury is out.
Educators stacked FSA cut scores against FCAT and FCAT 2.0 scores. In almost all instances, more students were placed in passing achievement levels on past FCATs.
Greenberg is waiting on further reporting from DOE, like analysis on the complexity of each test item, before saying whether one exam is more challenging than another — "especially considering the concerns raised in the validity test," he said.
"There are more questions to be answered. Like: 'What is the causal factor for the drop from the FCAT to the FSA?' or 'Were passages of interest enough for students to want to work through them?'"
Aside from technical glitches reported during the administration of the computer-based FSA, the report contracted to Alpine Testing Solutions by legislators this spring found that many questions were not aligned to Florida Standards or were not rigorous enough.
Thirty-three percent of third-grade ELA test questions and more than 20 percent of questions in the seventh-grade math exam did not match Florida Standards. Only one of 65 questions on the 10th-grade ELA assessment measured whether students possessed an "advanced depth of knowledge."
Questions & Answers
How did officials come up with these scores?
After DOE controversially ruled the test was valid earlier this month, panelists began poring over FSA results.
Two separate standard-setting workshops — one conducted by an educator panel consisting of teachers and academic experts; the other held by a reactor panel of superintendents, school board politicians, community officials and business leaders — determined what students across the state should know and be able to do. Judges themselves took the exams and reviewed content standards while keeping the achievement level scale in mind.
The results also show that the educator and reactor panelists do not see eye to eye.
The educator panel, for instance, usually set the bar higher when determining proficiency. Under the reactor panel, more students met the satisfactory requirement. On some exams, the panels differed by eight or nine percentage points.
Do I get a say in how cut scores are determined?
This is only the first phase in approving FSA achievement levels. The step-by-step process to set cut scores has been done before — recently with the FCAT 2.0 in 2011.
It will be the public's turn to comment on cut score recommendations Thursday, at the Turlington Building, located 325 W. Gaines St., or online, from 2 to 4 p.m.
Results from the panels and public input will go to DOE Commissioner Pam Stewart, who is supposed to evaluate the feedback before submitting recommendations of her own to the Legislature for a 90-day review. The State Board of Education has final approval, which LCS officials say will probably not happen until spring.
So... what now? What can school districts do with these results?
While the results from preliminary panelists offer educators a few clues about what student-level results to expect, there are many unknowns that could affect the district's next steps.
"We are waiting on information about specific questions and analysis about the complexity of those questions," Greenberg said. "We need to look at what type of skills are giving kids the most level of difficulty.
"When we know more, we can determine the instructional implications of results, and use that to continue our professional development," he said.
Alright, alright. When will students know if they passed?
DOE has not officiated a plan to release "partial student data," like raw scores and percentiles, for students yet, said Gillian Gregory, LCS director of testing, research and evaluation. The district expects to receive some information before the end of the second quarter and will mail reports home.
"These won't be the test scores parents are used to," Gregory said. "We're not really sure how we are going to get them from the state or what the reports will look like."
Student will most likely have to wait until the spring to find out FSA achievement levels.Scores for end-of-course exams aligned to Next Generation Sunshine State Standards, however, were released on time.
