Understanding the 2021-22 ESSA Academic Progress (Growth) Measure: Relative School Improvement Measure (RSIM)
Based on extensive stakeholder engagement, New Jersey’s Every Student Succeeds Act (ESSA) State Plan weighs academic progress, or “growth”, measures more heavily than other metrics, such as proficiency. The New Jersey Department of Education (NJDOE) selected median student growth percentiles (mSGPs) to measure academic progress in English Language Arts (ELA) and mathematics for grades 4–8 (non-high schools).
Because data is not available to calculate mSGPs for 2021-22, the NJDOE received approval in the COVID-19 State Plan Addendum (PDF) to use a modified approach to calculate academic progress for the 2021-22 school year. The NJDOE will use the Relative School Improvement Measure for 2021-22 ESSA School Accountability and resume measuring academic progress using mSGPs for the 2022-23 school year.
Overview of the Relative School Improvement Measure (RSIM)
The NJDOE developed RSIM as one of several metrics to use to identify schools that may need additional support as compared to schools that had similar performance prior to the COVID-19 pandemic. The RSIM can also show the NJDOE what schools may have outperformed schools with similar prior year performance.
It is a relative improvement measure, like student growth percentiles, so a school’s performance is based on a comparison to other schools, not strictly on the actual change in scale score. This means that it is possible to have a high or low RSIM score, regardless of prior year performance. It is also possible to have an RSIM that meets the annual target, even if a school’s proficiency level has gone down. It all depends on how other schools with similar prior year aggregate scores performed in 2021-22.
RSIM is based on a school’s average scale score improvement on the NJSLA when comparing prior year performance to 2021-22 results. Schools are assigned a percentile rank based on their relative improvement as compared to schools with similar prior year performance. Schools receive a separate RSIM for ELA and mathematics. The same calculation is used to calculate RSIM at the district, school, and student group level. More details about the calculation can be found in the 2022 ESSA Accountability Profiles Companion Guide.
Calculating Average Scale Scores
To calculate the RSIM measure, average scale scores for ELA and mathematics are calculated for 2017-18, 2018-19, and 2021-22. When calculating the average scale score for each year, the NJDOE uses the following rules to determine which students will be included:
- Only include students in grades 3 through 8.
- This includes students in middle school taking Algebra I, Geometry, or Algebra II assessments.
- Only include students who were enrolled in the school before December 1 of the given school year.
- This is based on the school entry date that districts report in NJ SMART.
- Only calculate an average scale score if data is available for 20 or more students.
Calculating Prior Year Performance
First, the average scale scores for 2017-18 and 2018-19 are each calculated based on the average score all students in grades 3–8 taking the NJSLA for the given year.
Next, the prior year performance is calculated using the 2017-18 and 2018-19 Average Scale scores and the number of valid scores in 2017-18 (N17-18) and 2018-19 (N18-19):
$\text{Prior year performance} = \frac{({\text{2017-18 Average Scale Score} \times N_{17-18})} + (\text{2018-19 Average Scale Score} \times N_{18-19})}{N_{17-18} + N_{18-19}}$
If an average scale score is only available for one prior year, the prior year performance score will be based only on that one year. If a school does not have an average scale score for both 2017-18 and 2018-19, the school will not receive an RSIM score.
Grouping Schools Based on Similar Prior Year Performance
Each school will be grouped with other schools that had similar prior year performance. These groups are assigned so that the range of scores within each group is similar.
Calculating Change in Average Scale Score
The 2021-22 Average Scale score is calculated based on the average scale score of all students in grades 3–8. The average scale score change is the difference between the 2021-22 average scale score and the prior year performance (defined above).
Ranking Schools within Groups
Within each group of schools with similar prior year performance, the schools are ranked based on their average scale score change and assigned a percentile rank based on their ranking. The percentile rank is that school’s RSIM score for 2021-22.
School A is a K–8 school with the following average scale scores for 2017-18, 2018-19, and 2021-22:
Area | 2017-18 Average Scale Score | 2017-18 Valid Scores | 2018-19 Average Scale Score | 2018-29 Valid Scores | 2021-22 Average Scale Score |
---|---|---|---|---|---|
ELA | 750 | 50 | 735 | 48 | 740 |
Math | 710 | 47 | 715 | 47 | 705 |
$\text{ELA Prior Year Performance}=\frac{(750 \times 50)+(735 \times 48)}{50 + 48}=742.7$
$\text{Mathematics Prior Year Performance}=\frac{(710 \times 47)+(715 \times 47)}{47 + 47}=712.5$
Example Ranges
Note that these are example ranges, the actual ranges used to group schools for RSIM may differ.
- ELA Prior Year Comparison Group: Schools with prior year performance between 735 and 745
- Mathematics Prior Year Comparison Group: Schools with prior year performance between 698 and 718
$\text{ELA Average Scale Score Change}=740 -742.7 =-2.7$
$\text{Mathematics Average Scale Score Change}=705 - 712.5 =-7.5$
For ELA, many schools within the comparison group had decreases in their performance, so even though School A had a decrease in average scale score, School A’s average scale score change of $-2.7$ was better than 84% of other schools within that group. School A’s ELA RSIM will be 84.
For mathematics, School A’s average scale score change of $-7.5$ was better than 73% of other schools within that group. School A’s Mathematics RSIM will be 73.
Annually, all schools are identified as “below target”, “meets target”, or “exceeds target” for academic progress.
Academic Progress Measure | Below Target | Meets target | Exceeds Target |
---|---|---|---|
mSGP | mSGP below 40 | mSGP between 40 and 60 | mSGP of 60 or higher |
RSIM | RSIM below 15 | RSIM between 15 and 85 | RSIM of 85 or higher |
District and school RSIMs, for all students and by student group, will be reported in the 2022 ESSA Accountability Profiles, which can be found on the NJDOE Accountability page. There will also be a link in the 2021-22 School Performance Reports to spreadsheet containing the RSIM results for all schools, districts, and student groups.
For more information, contact reportcard@doe.nj.gov.