18_07 Public Comments on 19 TAC §97.1001

ATTACHMENT III

Summary of Public Comments and Agency Responses Related to Proposed Amendment to 19 TAC Chapter 97, Planning and Accountability, Subchapter AA, Accountability and Performance Monitoring, §97.1001, Accountability Rating System

Comment. Lewisville Independent School District (ISD) commented that it is not legislatively mandated to include the Masters Grade Level standard in accountability and proposed accountability calculations only include the Approaches and Meets Grade Level standards as required by the legislature.

Agency Response. The agency disagrees. The average of the three performance levels is very close to the percentage of students who achieve the Meets Grade Level standard. The Meets Grade Level standard equates to a 60% chance of completing one year of college without remediation, which aligns with the Texas Higher Education Coordinating Board's (THECB's) 60x30TX plan.

Comment. An individual proposed adding the following statement to the description on page 3: "Student Achievement evaluates all performance levels across all subjects for all students, on both general and alternate assessments, College, Career, and Military Readiness (CCMR) indicators, and graduation rates."

Agency Response. The agency disagrees. Districts and campuses earn points in the Student Achievement domain for the percentages of assessment results at the Approaches Grade Level or above, Meets Grade Level or above, and Masters Grade Level standards. No points are awarded for assessments at the Did Not Meet Grade Level standard.

Comment. New Frontiers Public Schools proposed that the Student Achievement domain include only assessments at the Approaches Grade Level standard in the denominator when calculating the percentage of results at the Meets Grade Level standard and only the number of assessment at the Meets Grade Level standard in the denominator for calculating the percentage of results at the Masters Grade Level standard.

Agency Response. The agency disagrees. The accountability system must ensure that districts and campuses are held accountable for all students, including those who have yet to meet satisfactory standards.

Comment. Premont ISD proposed allowing end-of-course (EOC) testers who met the Approaches Grade Level standard to retest until they achieve the Meets Grade Level or Masters Grade Level standard. Premont ISD proposed the use of the higher outcome in accountability calculations.

Agency Response. The agency disagrees with including the commenter's recommendation in the 2018 Accountability Manual. This allowance needs to be considered more thoroughly before inclusion in the accountability system. However, the accountability manual is adopted annually and TEA may consider including the recommendation in future manuals.

Comment. Petersburg ISD commented that the CCMR component should be evaluated even if the district or campus has fewer than ten annual graduates. The district also questioned how requiring a minimum size of annual graduates meets TEC, §39.053(c).

Agency Response. The agency agrees. A two-year-average CCMR rate will be calculated for the all students group. The calculation will be based on an aggregated two-year uniform average using the district's or campus's 2018 CCMR data and the 2017 modeled CCMR data. The all students group will be evaluated if the two-year average has at least ten students. At adoption, Figure: 19 TAC §97.1001(b) was updated to include small number analysis methodologies on page 14 of Chapter 2 and page 34 of Chapter 4.

Comment. One district staff member asked how CCMR can be heavily weighted in each domain as they see the measure as unfair for the first two years. The commenter proposed that districts have a few years to adjust to the way that CCMR will be evaluated.

Agency Response. The agency disagrees. The CCMR component is legislatively mandated under House Bill (HB) 22, 85th Texas Legislature, Regular Session, 2017, and CCMR weighting was developed in response to stakeholder feedback.

Comment. One district staff member asked why CCMR is included in calculations for high school campuses and districts, noting that there are many differences and variables among districts.

Agency Response. The agency disagrees. HB 22 requires the inclusion of the CCMR component in high school, K-12, and district accountability calculations.

Comment. One district staff member asked why October is the cut off for TEA to receive Texas Success Initiative Assessment (TSIA) results while June is the cut off for SAT and ACT.

Agency Response. The agency provides the following clarification. The TSIA may be taken by students after enrolling in higher education and is intended to determine whether the student should receive academic remediation. The SAT and ACT are higher education entrance examinations and, therefore, are typically taken before enrolling in higher education.

Comment. One district staff member asked about accountability implications for students who choose to not report SAT, ACT, or TSIA scores to their district.

Agency Response. The agency provides the following clarification. TEA does not receive SAT, ACT, or TSIA results from districts. The College Board provides SAT results to TEA and the TSIA results to the Texas Higher Education Coordinating Board, who then provides the results to TEA. ACT provides ACT results to TEA.

Comment. The College Board encouraged TEA to add additional weight or a bonus point for scores of 3 or better on Advanced Placement (AP) examinations to incentivize campuses to increase access to and participation in AP.

Agency Response. The agency disagrees. Each annual graduate is included once in the denominator of the CCMR component. If an annual graduate earns a 3 or better on an AP examination, they earn one full point for the numerator. Adding additional weighting would not be mathematically sound as it could make the numerator greater than the denominator.

Comment. Lewisville ISD recommended that TEA consider the American Association of School Administrators research on career and college readiness and the inclusion of the metrics in the accountability system.

Agency Response. The comment is outside the scope of the proposed rulemaking. The CCMR indicators evaluated in the 2018 accountability system were established by HB 22.

Comment. One district staff member commented that the 2018 implementation of the Career and Technical Education (CTE) Coherent Sequence with Coursework Aligned to an Industry-Based Certification indicator does not allow adequate time for districts to transition from the previous indicator (solely CTE Coherent Sequence) to offering the coursework aligned to the industry-based certifications. A commenter also noted that the industry-based certification list is too narrow.

Agency Response. The agency disagrees. Phasing out CTE Coherent Sequence by awarding one-half point for CTE coherent graduates who complete and receive credit for at least one industry-based certification-aligned CTE course allows districts and campuses to receive credit for efforts already in progress.

Comment. The Texas Urban Council requested that students who complete a coherent sequence of CTE courses receive credit for a full point instead of one-half point in the Student Achievement domain and that this measure be maintained in the accountability system beyond 2022.

Agency Response. The agency disagrees. Phasing out CTE Coherent Sequence by awarding one-half point for CTE coherent graduates who complete and receive credit for at least one industry-based certification-aligned CTE course allows districts and campuses to receive credit for efforts already in progress.

Comment. An individual proposed implementing the CTE Coherent Sequence Coursework indicator with an incoming class in the same manner as new graduation plans.

Agency Response. The agency disagrees. Phasing out CTE Coherent Sequence by awarding one-half point for CTE coherent graduates who complete and receive credit for at least one industry-based certification-aligned CTE course allows districts and campuses to receive credit for efforts already in progress.

Comment. One district staff member asked whether there will be a public comment period for the CTE pathway of courses indicator and how the list is being developed. The commenter noted that counselors are advising students now for four-year graduation plans and need information to counsel students.

Agency Response. The agency provides the following clarification. TEA will develop the CTE pathway of courses indicator, which is expected to be implemented in 2020, collaboratively with stakeholders throughout the 2018-2019 school year.

Comment. One district staff member asked if the CTE Coherent Sequence indicator includes TSDS PEIMS code 2 students only.

Agency Response. The agency provides the following clarification. In order to receive credit for this indicator, an annual graduate must be coded as a 2 in TSDS PEIMS Element ID E0031 during his or her final year of enrollment and be coded as having completed coursework from the list of 85 industry-based certification-aligned courses.

Comment. One district staff member asked whether TEA will consider awarding credit in the CCMR component of the Student Achievement domain for annual graduates who earned an industry-based certification and CTE dual enrollment.

Agency Response. The agency provides the following clarification. The denominator for the CCMR component is the number of annual graduates; the numerator is the number of annual graduates who accomplishes at least one of the CCMR indicators. The CCMR component currently awards credit for students who earn an industry-based certification or earn at least three hours of dual credit in English language arts (ELA)/reading or mathematics or nine hours of dual-credit in any subject area, including CTE hours.

Comment. A district staff member commented that TEA should consider adding Culinary Arts and Food Preparation/Handling as a CTE course aligned with industry-based certifications.

Agency Response. The agency disagrees. Culinary Arts and Food Preparation/Handling is not aligned with the list of 73 industry-based certifications in 19 TAC §74.1003.

Comment. The Texas School Alliance (TSA), the Texas Association of School Administrators (TASA), and six district staff members commented that the scaling methodology for the graduation rate component does not comply with HB 22 provisions, which require that the rating system provide the mathematical possibility for all districts and campuses to receive an A rating. TSA stated that the scaling appears to show a forced distribution and goes against the general public's understanding of letter grades.

Agency Response. The agency disagrees. The scaling methodology used in the graduation rate component reflects the historical high graduation rates across the state of Texas. The scaling methodology distinguishes among districts and campuses with high graduation rates and those with comparably low graduation rates. Additionally, the scaling does not prohibit the possibility for any district or campus to achieve an A rating.

Comment. TSA and six district staff members requested clarity on which graduation rate is used in the Student Achievement domain for high schools and districts.

Agency Response. The agency provides the following clarification. Chapter 2 indicates that TEA evaluates the four-year, five-year, and six-year longitudinal high school graduation rates. The graduation rate that results in the highest score is used for the Graduation Rate component of the Student Achievement domain.

Comment. One district staff member commented that the annual dropout rate conversion of 10 times the annual dropout rate is unnecessarily punitive. The commenter proposed using a conversion with five times the annual dropout rate.

Agency Response. The agency disagrees. The annual dropout rate conversion multiplier of 10 allows a district or campus to accumulate points toward the Student Achievement domain score only if its annual dropout rate is less than 10%. Alternative education accountability (AEA) districts and campuses use a conversion multiplier of five, which allows an AEA district or campus to accumulate points toward the Student Achievement domain score if its annual dropout rate is less than 20%.

Comment. One district staff member commented that using the dropout rate for new high schools often penalizes these campuses because the dropout rate produces a lower score than the campus would have received if the graduation rate was available. The staff member proposed that TEA either compute a graduation rate for students in Grade 9 in both year one and year five of the cohort or use an annual dropout rate proxy, whichever would earn more points.

Agency Response. The comment about changing the computation of graduation rates is outside the scope of the proposed rulemaking.

Comment. TSA, TASA, the Texas Urban Council, and six district staff members proposed that the three components of the Student Achievement domain should be weighted equally.

Agency Response. The agency disagrees. The weighting for the Student Achievement domain was developed in response to stakeholder feedback.

Comment. One district staff member requested that the weighting for the CCMR component in the Student Achievement domain be reduced to 20% and the graduation weight be increased to 40%.

Agency Response. The agency disagrees. The weighting for the Student Achievement domain was developed in response to stakeholder feedback.

Comment. One district staff member proposed excluding high schools from evaluation in the School Progress Domain, Part A: Academic Growth domain.

Agency Response. The agency disagrees. Stakeholder feedback indicated that although growth data for high schools is limited, it is still valuable to recognize the growth for these campuses. Therefore, the better of Part A: Academic Growth or Part B: Relative Performance will be used for the School Progress domain score.

Comment. Regarding School Progress, Part A: Academic Growth, one district staff member commented that it is unfair to penalize districts or campuses for students who were at "commended" or Masters Grade Level in 2017 but fell just short of "commended" or Masters Grade Level in 2018. A commenter proposed that if the student is on the "cusp" of Masters Grade Level, the district or campus receive credit for the result.

Agency Response. The agency disagrees. Based on recommendations from stakeholders, districts and campuses receive credit for STAAR® results that either meet the student-level criteria for the STAAR® progress measure or maintain proficiency.

Comment. One district staff member commented that if students achieve the Meets Grade Level standard this year and in the prior year, they should receive one full point for Academic Growth. The commenter stated that Meets Grade Level satisfies the THECB's 60x30TX plan.

Agency Response. The agency disagrees. The School Progress, Part A: Academic Growth methodology was developed in response to stakeholder feedback. STAAR® results that maintain proficiency at the Approaches Grade Level or Meets Grade Level standard are awarded one-half point.

Comment. An individual proposed updating the list of assessments evaluated in School Progress, Part A: Academic Growth on page 21 to Grades 4-8, English II, and Algebra I.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated to correct Grade 3 to Grade 4 on page 21 of Chapter 3.

Comment. TSA, the Texas Urban Council, and six district staff members commented that the School Progress, Part A: Academic Growth domain should include the Spanish to English proxy calculation. TSA recommended that if the inclusion lowers the rating of the campus, then a hold harmless provision should be applied.

Agency Response. The agency disagrees. The School Progress, Part A: Academic Growth domain evaluates eligible STAAR® progress measures. Spanish to English transition assessments are not evaluated for progress. It is not possible to have a viable accountability system based on analyses of best-case scenarios for districts and campuses. The decision to exclude the Spanish to English proxy was made in response to accountability advisory committee and stakeholder feedback.

Comment. TSA and six district staff members requested that the state award a full point in School Progress, Part A: Academic Growth for students who skip a STAAR® grade level and score at either Meets Grade Level or Masters Grade Level.

Agency Response. The agency disagrees. A student's grade level is not accounted for in the methodology for the STAAR® progress measure calculations. Likewise, a student's grade level is not considered in the School Progress domain.

Comment. TSA and five district staff members requested that the state evaluate growth for all English II assessments, regardless of their English I administration date.

Agency Response. The comment is outside the scope of the proposed rulemaking.

Comment. TSA and TASA suggested that the cut points and scaled scores in the School Progress domain be revised because they are inconsistent with the general public's understanding of letter grades.

Agency Response. The agency disagrees. The cut points used in the 2018 accountability system reflect high expectations for student achievement, school progress, and reducing achievement gaps among students of different racial and ethnic backgrounds and different socioeconomic statuses.

Comment. One district staff member asked why the percentage of economically disadvantaged students is gathered from TSDS PEIMS snapshot instead of STAAR® answer documents and whether the percentage of economically disadvantaged students is for the entire campus or just the testing grades.

Agency Response. The agency provides the following clarification. The percentage of economically disadvantaged students used in the School Progress, Part B: Relative Performance domain is based on the TSDS PEIMS snapshot data for all students at the campus or district. In the Closing the Gaps domain, the economically disadvantaged student group is identified based on information on STAAR® answer documents. Commonly, these answer documents are prepopulated with information provided from the TSDS PEIMS snapshot.

Comment. The Texas Urban Council requested that TEA consider weighting the graduation rate component at least one-third of the overall score for the School Progress domain.

Agency Response. The agency disagrees. The School Progress domain methodology was developed in response to stakeholder feedback.

Comment. A district staff member commented that TEA should consider phasing in the Meets Grade Level and Masters Grade Level requirements for student groups in each component of the Closing the Gaps domain.

Agency Response. The agency disagrees. The indicators and construction of the Closing the Gaps domain is aligned with the federal Every Student Succeeds Act (ESSA), which does not allow for the phasing in of components.

Comment. An individual proposed adding language to page 27 specifying that the Academic Achievement component evaluates assessment results at the Meets Grade Level standard in ELA/reading and mathematics.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated to add "at the Meets Grade Level or above standard" on page 27 of Chapter 4.

Comment. A district staff member commented that the Academic Achievement component in the Closing the Gaps domain is redundant since proficiency is already included in the Student Achievement domain.

Agency Response. The agency disagrees. The Academic Achievement component measures STAAR® performance in ELA/reading and mathematics at the Meets Grade Level or above standard. Academic Achievement is evaluated in Closing the Gaps to align with the federal requirement of ESSA to evaluate the proficiency of individual student groups.

Comment. An individual proposed updating the list of assessments evaluated in the Academic Growth Status component on page 30 to Grades 4-8, English II, and Algebra I.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated to correct Grade 3 to Grade 4 on page 30 of Chapter 4.

Comment. TSA and six district staff members commented that TEA should apply the same methodology for CCMR in all components of the accountability system. The commenters proposed not including Grade 12 students who did not graduate in the CCMR component of the Closing the Gaps domain.

Agency Response. The agency disagrees. The indicators and construction of the Closing the Gaps domain are aligned with ESSA, which does not allow for the evaluation to be limited to annual graduates.

Comment. One district staff member requested that AEA campuses, which serve as dropout recovery campuses, be excluded from the four-year Federal Graduation Status component in the Closing the Gaps domain. The commenter proposed that AEA campuses use School Progress, Part A: Academic Growth in the Closing the Gaps domain in place of the four-year federal graduation rate.

Agency Response. The agency disagrees. The indicators and construction of the Closing the Gaps domain aligns with ESSA, which does not allow for AEA provisions.

Comment. TASA, TSA, and three district staff members commented that student groups who have met the state's final long-term graduation rate target of 94% should be given credit and that a cap should be applied to the federal graduation rate so that any minuscule changes do not affect a campus or district performance target.

Agency Response. The agency disagrees. The four-year graduation rate requirement under ESSA requires that campuses demonstrate improvement annually.

Comment. One district staff member asked whether students who graduate early remain in their original cohort year of graduates for graduation rates and if this will affect the Closing the Gaps domain if they remain in their cohort.
Agency Response. The agency provides the following clarification. Early graduates remain in their original cohort and are included as graduates in the four-year graduation rate, just as other graduates in their cohort who graduate in four years.

Comment. One district staff member questioned the minimum size criteria for student groups in the Student Success component of the Closing the Gaps domain and stated that the criteria in Chapter 4, page 32, incorrectly states that it is by subject area.

Agency Response. The agency agrees. The Student Success component of the Closing the Gaps domain does not disaggregate by subject area. At adoption, Figure: 19 TAC §97.1001(b) was updated to correct the minimum size criteria and small numbers analysis details on page 33 of Chapter 4.

Comment. Four district staff members requested clarification on the minimum size criteria for student groups in the Closing the Gaps domain and recommended the following statement: "Student groups are evaluated if there are 25 assessments in mathematics and 25 assessments in reading."

Agency Response. The agency agrees that clarification is necessary. A student group is evaluated in an Academic Achievement ELA/reading indicator if there are 25 assessments in ELA/reading. A student group is evaluated in an Academic Achievement mathematics indicator if there are 25 assessments in mathematics. A student group is evaluated in an Academic Growth Status ELA/reading indicator if there are 25 or more STAAR® progress measures in ELA/reading. A student group is evaluated in an Academic Growth Status mathematics indicator if there are 25 or more STAAR® progress measures in mathematics. Subject areas are considered separately. At adoption, Figure: 19 TAC §97.1001(b) was updated to clarify the minimum size criteria and small numbers analysis details on page 29 of Chapter 4.

Comment. Arrow Academy Charter School and five district staff members commented that minimum size criteria in the Closing the Gaps domain adversely affects smaller schools as small student group results are not included or reported in the component score calculation. Two commenters recommended that the growth component be used even if minimum size is not met or that the minimum size for student groups be modified to 10. A commenter recommended using a weighted matrix to measure for growth achievements of student groups.

Agency Response. The agency disagrees. The all students group will be evaluated if there are at least 10 assessments in the subject area or at least 10 students in the CCMR or graduation components. Individual student group results are evaluated if there are at least 25 or more assessments in the subject area or at least 25 students in the CCMR or graduation components. If a student group does not meet minimum size criteria, the results are reported on data tables but not evaluated.

Comment. An individual commented that the chart on page 36 was confusing as the example has 7 columns instead of 14.

Agency Response. The agency provides the following clarification. While 14 student groups are evaluated in the Closing the Gaps domain, the examples provided on page 36 have 7 student groups with component data available for evaluation.

Comment. An individual proposed listing the remaining components that require only one evaluated indicator on page 36 of Chapter 4.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated to list the two components that require only one evaluated indicator on page 36 of Chapter 4.

Comment. An individual proposed revising the targets on page 40 to consistently present elementary and middle school targets before high school, K-12, and district targets.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated to list the targets for elementary and middle schools before the targets for high schools, K-12s, and districts on page 40 of Chapter 4.

Comment. One district staff member expressed concern that the Academic Achievement performance target for some ethnicities is lower than that of other ethnicities in the Closing the Gaps domain. The commenter encouraged the state to identify different parameters for the Closing the Gaps domain targets.

Agency Response. The agency disagrees. The indicators and construction of the Closing the Gaps domain are aligned with ESSA. In the original submission of the ESSA state plan, TEA set equal targets for all ethnicities with a hold harmless provision that recognized incremental achievements toward the target. Ultimately, TEA revised the targets to align with baseline data based on feedback from the U.S. Department of Education.

Comment. TSA requested clarification on the calculation of the Closing the Gaps domain, including the weights if components are not available or if the TELPAS waiver is approved.

Agency Response. The agency provides the following clarification. If a district or campus has a component that is not available for evaluation in Closing the Gaps, its weight is distributed proportionally among the remaining evaluated indicators.

Comment. An individual questioned how campuses will be identified for Additional Targeted Support in 2018-2019 if campuses are not identified for Targeted Support and Improvement until August 2019.

Agency Response. The agency provides the following clarification. Comprehensive Support and Improvement, Targeted Support and Improvement, and Additional Targeted Support are required identifications under ESSA. The identification of campuses for Additional Targeted Support is independent of identification of campuses for Targeted Support and Improvement.

Comment. One district staff member commented on the calculation adjustment of the Academic Achievement component of Closing the Gaps for a student group whose participation rate falls below 95%. The commenter stated that if a group falls below 95 percent, then the denominator will be increased by the number of assessments necessary to achieve a 95 percent participation rate. If this procedure is applied without regard to student group size, the commenter noted that it is possible for a student group below the minimum size before the participation adjustment to meet the minimum size after the adjustment. The commenter proposed that TEA not apply the participation adjustment to any student group that does not meet minimum size.

Agency Response. The agency disagrees. The participation rate methodology was developed to align with the federal accountability requirements under ESSA, which do not allow for the exclusion of student groups from the recalculation.

Comment. One district staff member commented that AEAs with a majority of students enrolled as recovered dropouts may not meet minimum size criteria for School Progress, Part A: Academic Growth nor would these campuses have Part B: Relative Performance data.

Agency Response. The agency provides the following clarification. If an AEA or alternative education campus (AEC) does not have data for the School Progress domain, the Student Achievement domain will be weighted at 70% of the overall rating.

Comment. One district staff member commented that AEAs with a majority of students enrolled as recovered dropouts potentially have little opportunity to obtain points in the Academic Growth Status and Federal Graduation Status components in the Closing the Gaps domain.

Agency Response. The agency provides the following clarification. If an AEA or AEC does not have data for the Federal Graduation Status or Academic Growth Status components of the Closing the Gaps domain, the weight from those components is distributed proportionally among the remaining evaluated indicators.

Comment. One district staff member commented that AEAs are penalized in the accountability system for dropout recovery and that the system does not fairly represent the success of the AEA campus.

Agency Response. The comment is outside the scope of the proposed rulemaking.

Comment. One district staff member commented that the scaling formulas shown in Chapter 5, page 49, for letter grade F and scaled scores 30-59 appear to be incorrect based on the other scaling formulas in the table.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated to correct the F and scaled scores 30-59 scaling formulas shown on page 49 of Chapter 5.

Comment. One district staff member requested clarification regarding the methodology for combining the STAAR® and CCMR rates in the School Progress, Part B: Relative Performance domain.

Agency Response. The agency agrees that clarification is needed. School Progress, Part B: Relative Performance uses either the STAAR® raw component score or the average of the Student Achievement STAAR® and CCMR raw components from the Student Achievement domain, depending upon campus type. At adoption, Figure: 19 TAC §97.1001(b) was updated on page 26 of Chapter 3 to clarify that the raw component scores are used.

Comment. One district staff member commented that 89 should not be the highest available scaled score for School Progress if either Part A: Academic Growth or Part B: Relative Performance has a scaled score equal to or below 59.

Agency Response. The agency disagrees. The better of methodology for Part A: Academic Growth or Part B: Relative Performance aligns with the better of methodology established under HB 22. Under HB 22, when a domain outcome is determined by using the better of outcome, if one domain has an F performance rating, the district or campus may not be assigned a performance rating higher than a B for the composite for the two domains.

Comment. TSA requested clarification on scaling tables, scaled scores, cut scores, and lookup tables that indicate the values will remain constant for the next five years.

Agency Response. The comment is outside the scope of the proposed rulemaking. While TEA intends on holding cut points constant for five years, the rule adoption process must occur annually.

Comment. TSA requested that the word "score" always be preceded with either "raw" or "scaled."
Agency Response. The agency provides the following clarification. The terms "component score" or "domain score" reference raw scores. If the score is a scaled score, the manual uses the term "scaled score."
Comment. One district staff member commented that for the Postsecondary Readiness Distinction Designation, some students meet TSI criteria and meet the criteria to be considered a CCMR graduate. The commenter asked whether districts and campuses get double points for these graduates. The commenter also asked whether the TSI Criteria Graduates indicator in the Postsecondary Readiness Distinction Designation is the same as the Student Achievement TSI Criteria indicator and why only participation in SAT, ACT, and AP is included in the Postsecondary Readiness Distinction Designation.

Agency Response. The agency provides the following clarification. TSI Criteria Graduates and CCMR Graduates are two separate indicators in the Postsecondary Readiness Distinction Designation; therefore, graduates who satisfy the TSI criteria are included in the numerator for both indicators. The TSI Criteria Graduates indicator used in the Postsecondary Readiness Distinction Designation is identical to the TSI Criteria used in the Student Achievement domain. Only participation in SAT, ACT, AP, and IB assessments is included in the Postsecondary Readiness Distinction Designation because the subject-specific distinction designations evaluate performance.

Comment. Seventy-five district staff members commented on the inclusion of percentage of students enrolled in an Early College High School (ECHS) as a variable used to construct campus comparison groups. The commenters noted that some ECHSs select only a small percentage of students to participate. The commenters requested that consideration be given to campuses that service all students in a school-wide early college model. The commenters proposed considering the other demographic variables higher than the ECHS variable when determining campus comparison groups.

Agency Response. The agency provides the following clarification. The methodology for assigning campuses to unique comparison groups was updated in 2017 based on accountability advisory group and stakeholder feedback to include the percentage of students enrolled in an ECHS program. Using the percentage of ECHS students enrolled at the campus in conjunction with other demographic variables refines the calculation that determines campus comparison groups; campuses will be grouped with more comparable peers than they would be otherwise. Accountability advisory committees review and refine the methodology for campus comparison groups and distinction designations annually. TEA will forward the comment to the committees for consideration for 2019 accountability recommendations.

Comment. TASA, TSA, the Texas Urban Council, and six district staff members proposed that substitute assessments be included at the Masters Grade Level standard instead of at the Meets Grade Level standard.

Agency Response. The agency disagrees. The standard-setting and equating processes for aligning substitute assessments with STAAR® takes more time than what was available in the 2017-2018 school year. For the 2018 accountability ratings, substitute assessments will be included at the Meets Grade Level standard. TEA will explore identifying cut points for Approaches Grade Level and Masters Grade Level on all substitute assessments. TEA will also explore using the differentiated performance level descriptors to calculate simple academic growth for high school students on substitute assessments. The goal is for this to be in place for the 2020 accountability ratings.

Comment. The Texas Chapter of the American Federation of Teachers (Texas AFT) commented that while substitute assessments are available to students, the standard setting process would not conclude until 2020, at which time substitute assessments could be included at the Masters Grade Level standard. Texas AFT stated that this does not help districts currently and, therefore, the accountability system is not ready for implementation.

Agency Response. The agency disagrees. The 2018 accountability system credits districts and campuses at the Meets Grade Level standard for the results of students who have successfully completed a substitute assessment while an equating process is conducted.

Comment. TASA, TSA, and six district staff members commented that the appeal process should allow for districts to appeal the identification of Comprehensive, Targeted, and Additional Targeted Support. The commenters stated that several factors outside of a campus's control, such as online testing issues and Hurricane Harvey, could have negative impacts on a campus's performance, and TEA should not identify schools for circumstances outside of their control.

Agency Response. The agency disagrees. The identification of Comprehensive, Targeted, and Additional Targeted Support and Improvement is federally required under ESSA and is not appealable. TEA has developed state accountability provisions for districts, open-enrollment charter schools, and campuses affected by Hurricane Harvey. Additionally, identification for Comprehensive, Targeted, and Additional Targeted Support and Improvement only includes assessment results affected by testing interruptions for districts, open-enrollment charter schools, and campuses if the overall rating benefits from such inclusion.

Comment. The Texas Charter Schools Association proposed revising three sentences in Chapter 8 related to appeals. The proposed language would state: "Districts and charter schools may appeal for any reason. The most successful basis for appeals are generally related to data or calculation error attributable to TEA, an ESC, or the testing contractor(s). The appeals process may not be an appropriate method to correct data that were inaccurately reported by the district. A district that submits inaccurate data must follow the procedures and timelines for resubmitting data (e.g., the Texas Education Data Standards). Appeals based on poor data quality may not receive favorable consideration."

Agency Response. The agency disagrees. The appeals process is not an appropriate method to correct data that were inaccurately reported by the district. A district that submits inaccurate data must follow the procedures and timelines for resubmitting data (e.g., the Texas Education Data Standards). Appeals based on poor data quality will not receive favorable consideration.

Comment. An individual questioned why Chapter 9 does not specifically mention the interventions found in TEC, §39A.051, for any campus rated Improvement Required.

Agency Response. The agency provides the following clarification. The TEA Division of School Improvement provides interventions for campuses that receive an Improvement Required rating. Intervention requirements are available through documents provided by this division.

Comment. One district staff member recommended the inclusion of STAAR® results for year two English learners (ELs) if the inclusion helps a campus with an Improvement Required rating achieve a Met Standard rating. The commenter noted that in 2017 some year two ELs were included and that many year two ELs take Spanish STAAR® and do well.

Agency Response. The agency disagrees. The state accountability system is intended to accurately reflect and distinguish among district and campus performance. It is not possible to have a viable accountability system based on analyses of best-case scenarios for districts and campuses.

Comment. The Texas Public Policy Foundation, Texans for Education Opportunity, and the Texas Business Leadership Council commented that the accountability system is transparent, fair, and holds schools accountable for performance of students. The commenters stated that specifically, the system utilizes an easy to understand A-F letter grade designation, allows for a school district to design its own local campus accountability rating, relies on the STAAR®, provides that all districts and schools can possibly earn an A, and will maintain the same targets for at least five years.

Agency Response. The agency agrees.

Comment. Texas AFT and Lewisville ISD commented that the accountability system is difficult to understand and calculating an overall rating takes too long. Texas AFT recommended that each domain be rated, similar to the School Report Card.

Agency Response. The agency disagrees. Districts and campuses receive a rating for each domain and overall as statutorily required by HB 22.

Comment. Texas AFT commented that campuses should be rated on indicators such as school funding, class sizes, and teacher quality, none of which are in the current accountability system.

Agency Response. The comment is outside the scope of the proposed rulemaking. The 2018 accountability system indicators are statutorily required.

Comment. Texas AFT commented that elementary and middle school campuses are rated based solely on performance on STAAR® assessments, which have not been determined to be reliable and valued for the purposes described in HB 743 (84th Texas Legislature).

Agency Response. The agency disagrees. The 2018 accountability system indicators used for elementary and middle schools are statutorily required.

Comment. Two district staff members asked why EOC retest results are included in accountability each year until the student graduates.

Agency Response. The agency provides the following clarification. Each accountability cycle, one result per student, per subject is included in accountability calculations. This ensures that districts and campuses are held accountable for students who have yet to meet satisfactory standards. This also credits districts and campuses for successful EOC retest results.

Comment. One district staff member asked how retests are factored into accountability.

Agency Response. The agency provides the following clarification. Each student who tests or retests during the 2018 accountability cycle (summer 2017, fall 2017, or spring 2018) contributes one assessment result, either the best or most recent, per subject tested if he or she meets the accountability subset. To meet the subset, a student must be reported as enrolled in the same district or campus on the previous TSDS PEIMS October snapshot as where he or she tested. The June administration for Grades 5 and 8 is not considered for accountability calculations.

Comment. An individual proposed adding a phrase to pages 7 and 8 explaining the process for selecting the single best result for progress and performance separately for retesters.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated on pages 7 and 8 of Chapter 1 to clarify that the best result is found for performance and progress, considered separately.

Comment. One district staff member asked whether the consolidated accountability file (CAF) will be available in a friendlier format and stated that with the new accountability system, districts are going to want to know exactly where every number is coming from.

Agency Response. The agency provides the following clarification. The CAF is in the same format as years past. Districts need to use software to read the file. Each year TEA provides student lists to districts explaining how each student was included in accountability calculations.

Comment. An individual commented that the rigor of a child's STAAR® assessment should be based on the child's current abilities rather than the grade level.

Agency Response. The comment is outside the scope of the proposed rulemaking.

Comment. A district staff member commented that the phrase "Tests as of" on page 9 is ambiguous and requested that TEA use more explicit language.

Agency Response. The agency provides the following clarification. "Tests as of" refers to any tests taken prior to or during the month referenced in the chart on page 9.

Comment. Two individuals commented that open-enrollment charter schools practice selective enrollment and should be held to a higher standard in the academic accountability system. A commenter recommended an accountability system in which these campus types are identified and rated with different cut scores.

Agency Response. The agency disagrees. Districts, open-enrollment charter schools, and campuses are held accountable for the students enrolled. With the implementation of local accountability systems, districts and open-enrollment charter schools have the opportunity to develop accountability indicators to highlight successes within their campuses.

Comment. The Texas Urban Council commented that the accountability manual does not explain why a school may receive a low letter grade and fails to provide context for A-F ratings such as school funding or high poverty rates.

Agency Response. The agency disagrees. The indicators, methodology, and cut points for ratings are clearly defined within the manual.

Comment. An individual commented that the colors used for elementary and middle schools in the school types chart on page 6 are hard to differentiate.

Agency Response. The agency disagrees. The colors that were chosen vary considerably. As an alternative, the user may print the school types chart in greyscale or request a greyscale version of the chart from the TEA Department of Performance Reporting.

Comment. A district staff member proposed changes to the school types and cut points for campuses that serve early elementary through Grade 8. The commenter proposed holding these types of campuses to either the elementary or middle school target, whichever is lower; holding these campuses to targets based on which grades the majority of test results come from; or allowing districts to appeal ratings if a change in school type classification would result in a higher rating.

Agency Response. The agency disagrees. The cut points by campus type were developed in response to stakeholder feedback. TEA has determined that the assignment of cut points should be based on grade configurations known at the beginning of the year rather than on a variable that is not known until after all assessments have been administered. It is not possible to have a viable accountability system based on analyses of best-case scenarios for districts and campuses.

Comment. Austin Chamber of Commerce commented on the possibility for a district or campus to receive an F in the Student Achievement; School Progress, Part A: Academic Growth; and Closing the Gaps domains and a B in the School Progress, Part B: Relative Performance domain and have the B used as the overall rating due to the "best of" methodology. The commenter noted that it does not seem fitting to implement a "best of" approach among Student Achievement and School Progress, Part A and Part B. Such a scenario would allow a district or campus to fail in three domains, get a B in a fourth, and receive an overall rating of B.

Agency Response. The agency agrees. At adoption, Figure: 19 TAC §97.1001(b) was updated to address the possibility of a district or campus overall rating inaccurately reflecting performance across all four areas. A step was added to the overall rating methodologies on pages 42 and 43 of Chapter 5. If a district receives an F in three of the four areas (Student Achievement; School Progress, Part A: Academic Growth; School Progress, Part B: Relative Performance; or Closing the Gaps), the highest scaled score a district or open-enrollment charter school can receive for the overall rating is a 59. Likewise, if a campus receives an Improvement Required rating in three of the four areas (Student Achievement; School Progress, Part A: Academic Growth; School Progress, Part B: Relative Performance; or Closing the Gaps), the highest scaled score a campus can receive for the overall rating is a 59. In order for this provision to be applied, the district, open-enrollment charter school, or campus must be evaluated on all four areas.