HONOR ROLL METHODOLOGY
California methodology
Texas methodology

California Honor Roll Methodology

Top

Educational Results Partnership uses a data-driven approach called multilevel latent class modeling to identify California Honor Roll schools. This statistical technique assumes that:

  • (a) Hidden groups (latent classes) of schools exist among a larger set of schools,
  • (b) A set of observable criteria that represents various dimensions of school performance can distinguish these hidden groups from one another,
  • (c) Schools within the same district may be more similar to one another than typically assumed because they are affected by district-wide factors, and
  • (d) No single criteria is presumably better than another in distinguishing high performing schools among a larger set of schools; the pattern of results will determine which are the more important distinguishing criteria.

To identify Honor Roll schools, these models first estimate the characteristics (i.e., the average and spread of scores on each criterion) for each hidden group. Then, the model assesses the degree to which each school were similar to the characteristics of each hidden group. Finally, the model classifies schools into a specific group according to their highest probability of membership. We interpret the group with the highest scores on most of the criteria as the Honor Roll group. Schools that are members of that group are Honor Roll candidates.

Seven different models identified candidates among schools that served students at various grade segments and levels of poverty. Data from the Education Data Partnership (EdData) was used to identify elementary, middle, high, and “other” schools. Elementary, middle, and high schools were further split into two categories:

  • STAR Schools: schools with greater than or equal to 33% of students designated as socioeconomically disadvantaged (high poverty)
  • Scholar Schools: schools with fewer than 33% of students designated as socioeconomically disadvantaged (low poverty)

There was insufficient data to analyze high and low poverty schools in the "other" segment separately.

The number of Honor Roll schools in each category was not predetermined; the results of the model determined the number. Twelve measures were derived for all schools using public data at the California Department of Education (CDE) website:

  • Achievement in English, Math, and Science
    • The percentage of tested students that met or exceeded standards for their respective grade levels in the 2015-2016 California Smarter Balanced Tests (English and Math) and California Standards Tests (Science).
  • Improvement in English, Math, and Science
    • In each school, the difference between the percentage of tested students that met or exceeded standards in the respective grade levels and subject areas in each school for the last two years. Positive scores indicate higher achievement rates in 2015-2016 than 2014-2015.
  • Equity among Ethnic Groups in English, Math, and Science
    • We defined equity among ethnic groups as the difference between achievement rates of ethnic minority students in comparison to their White American peers in the same school.
    • For each school, we examined the achievement rates from six ethnic minority subgroups (African Americans, American Indian/Alaskan Natives, Asians, Filipino/as, Hispanics, and Pacific Islanders). Schools did not report achievement rates for ethnic minority groups with 10 or less tested students in a subject area.
    • Each school's equity score in the respective subjects was calculated in three steps:
      • For each ethnic minority group, derive the number of students who met or exceeded standards from the number of students tested and the achievement rate.
      • Sum up the number of students in each ethnic minority groups that had more than 10 tested students in the subject.
      • Divide the sum of ethnic minority students who met or exceeded standards by the sum of tested ethnic minority students.
    • The equity score provides a general indicator about the degree to which ethnic minority students were outperforming (more positive score), underperforming (more negative score), or similarly (score closer to zero) their White American peers.
    • Schools that did not report achievement rate for any of the significant ethnic minority or White American student groups did not receive an equity score for the subject.
  • Equity in Socioeconomic Status in English, Math, and Science
    • We defined equity in socioeconomic status as the difference in achievement rates between students who were socioeconomically disadvantaged and those who were not within the same school.
    • Schools did not report achievement rates for socioeconomic groups with 10 or less tested students in a subject area.
    • This equity score provides a general indicator about the degree to which socioeconomically disadvantaged students were outperforming (more positive score), underperforming (more negative score), or similarly (score closer to zero) peers in the same school who were not socioeconomically disadvantaged.
    • Schools that did not report achievement rate for either group did not receive an equity score for the subject.

The two models for high schools included seven additional measures from EdData on their 2014-2015 cohort:

  • Percentage of graduates who met the UC/CSU course requirements
  • Average Math, Reading, and Writing exam scores from the SAT
  • Percentage of Advanced Placement (AP) exams taken that were given a score of 3 or higher
  • Graduation rate
  • Dropout rate

STEM Schools

  • The top 10 percent among high poverty schools in math and science achievement within each grade span that were also recognized as Honor Roll schools. Both subjects were equally weighted in the determination.

Notes

  • These models accounted for the relationships in performance between schools in the same district as well as the relationship between achievement and improvement within each school.
  • Missing data did not automatically disqualify a school from Honor Roll consideration.  Based on the pattern of relationships among available data from all schools in the same grade segment and poverty level, the models estimated each school’s likely group membership using their available data.  However, schools must participate in CAASPP in order to receive recognition. 

The modeled data was current as of January 5, 2017.

  1. California Assessment of Student Performance and Progress. (2016). Research Files for Online Assessments [Data file]. Retrieved from http://caaspp.cde.ca.gov/sb2015/ResearchFileList
  2. California Assessment of Student Performance and Progress. (2017). Research Files for Online Assessments [Data file]. Retrieved from http://caaspp.cde.ca.gov/sb2016/ResearchFileList
  3. California Department of Education. (2016). 2015 CAASPP Paper-based Test Results [Data file]. Retrieved from http://caaspp.cde.ca.gov/caaspp2015/ResearchFileList.aspx
  4. California Department of Education. (2017). 2016 CAASPP Paper-based Test Results [Data file]. Retrieved from http://caaspp.cde.ca.gov/caaspp2016/ResearchFileList.aspx
  5. California Department of Education. (2017). Comparisons [Data file]. Retrieved from http://www.ed-data.org/Comparisons

Texas Honor Roll Methodology

Top

Educational Results Partnership uses a data-driven approach called multilevel latent class modeling to identify Texas Honor Roll schools. This statistical technique assumes that:

  • (a) Hidden groups (latent classes) of schools exist among a larger set of schools,
  • (b) A set of observable criteria that represents various dimensions of school performance can distinguish these hidden groups from one another,
  • (c) Schools within the same district may be more similar to one another than typically assumed because they are affected by district-wide factors, and
  • (d) No single criteria is presumably better than another in distinguishing high performing schools among a larger set of schools; the pattern of results will determine which are the more important distinguishing criteria.

To identify Honor Roll schools, these models first estimate the characteristics (i.e., the average and spread of scores on each criterion) for each hidden group. Then, the model assesses the degree to which each school were similar to the characteristics of each hidden group. Finally, the model classifies schools into a specific group according to their highest probability of membership. We interpret the group with the highest scores on most of the criteria as the Honor Roll group. Schools that are members of that group are Honor Roll candidates.

Seven different models identified candidates among schools that served students at various grade segments and levels of poverty. Data from the Texas Education Agency's Texas Academic Performance Reports (TAPR) was used to identify elementary, junior high, senior high, and "other" schools which serve students in more than one segment. Elementary, junior high, and senior high schools were further split into two categories:

  • STAR Schools: schools with greater than or equal to 33% of students designated as socioeconomically disadvantaged (high poverty)
  • Scholar Schools: schools with fewer than 33% of students designated as socioeconomically disadvantaged (low poverty)

There was insufficient data to analyze high and low poverty schools in the “other” category separately. Sixteen measures were derived from TAPR's downloadable data:

  • Achievement in Math, Reading, Writing, and Science
    • Change in proficiency scores across subjects over three years was placed into percentile rankings for all schools in the state of similar grade spans (i.e., elementary schools were compared only to elementary schools).
    • Senior high schools students did not assess students in the subject of writing.
  • Improvement in Math, Reading, Writing, and Science
    • In each school, the difference between the percentage of tested students that met the advanced standards in the respective grade levels and subject areas in each school for the last two years. Positive scores indicate higher achievement rates in 2015-2016 than 2014-2015.
    • Improvement in math was not evaluated for elementary, middle, and "other" schools because of major changes in STAAR math proficiency standards.
  • Equity in Socioeconomic Status in Math, Reading, Writing, and Science
    • We defined equity in socioeconomic status as the difference in achievement rates between students who were socioeconomically disadvantaged and those who were not within the same school.
    • Schools did not report achievement rates for socioeconomic groups with less than five tested students in a subject area.
    • This equity score provides a general indicator about the degree to which socioeconomically disadvantaged students were outperforming (more positive score), underperforming (more negative score), or similarly (score closer to zero) peers in the same school who were not socioeconomically disadvantaged.
    • Schools that did not report achievement rate for either group did not receive an equity score for the subject.
    • Schools that did not report achievement rate for any of the significant ethnic minority or White American student groups did not receive an equity score for the subject.
  • Equity among Ethnic Groups in Math, Reading, Writing, and Science
    • We defined equity among ethnic groups as the difference between achievement rates of ethnic minority students in comparison to their White American peers in the same school.
    • For each school, we examined the achievement rates from five ethnic minority subgroups (African Americans, American Indian/Alaskan Natives, Asians, Hispanics, and Two or More Races). Schools did not report achievement rates for ethnic minority groups with less than five tested students in a subject area.
    • Each school's equity score in the respective subjects was calculated in three steps:
      • For each ethnic minority group, derive the number of students who met the advanced standard from the number of students tested and the achievement rate.
      • Sum up the number of students in each ethnic minority groups that had more than five tested students in the subject.
      • Divide the sum of ethnic minority students who met the advanced standard by the sum of tested ethnic minority students.
    • The equity score provides a general indicator about the degree to which ethnic minority students were outperforming (more positive score), underperforming (more negative score), or similarly (score closer to zero) their White American peers.
    • Schools that did not report achievement rate for any of the significant ethnic minority or White American student groups did not receive an equity score for the subject.

The two models for senior high schools included three additional measures on postsecondary readiness:

  • Percentage of 2014 graduates who enrolled in public higher education institution in Texas in 2015 and met the Texas Success Initiative (TSI) requirements in all subjects
  • Percentage of 2015 graduates who were college and career-ready
  • Percentage of 2015 graduates who completed advanced/dual credit coursework

STEM Schools

  • The top 10 percent among high poverty schools in math and science achievement within each grade span that were also recognized as Honor Roll schools. Both subjects were equally weighted in the determination.

Notes

  • These models accounted for the relationships in performance between schools in the same district as well as the relationship between achievement and improvement within each school.
  • Missing data did not automatically disqualify a school from Honor Roll consideration.  Based on the pattern of relationships among available data from all schools in the same grade segment and poverty level, the models estimated each school’s likely group membership using their available data.   However, schools must participate in STAAR in order to receive recognition.

The modeled data was current as of February 20, 2017.

  1. Texas Education Agency. (2016). 2014-2015 APR Advanced Data Download [Data file]. Retrieved from https://rptsvr1.tea.texas.gov/perfreport/tapr/2015/download/DownloadData.html
  2. Texas Education Agency. (2017). 2015-2016 APR Advanced Data Download [Data file]. Retrieved from https://rptsvr1.tea.texas.gov/perfreport/tapr/2016/download/DownloadData.html