February 26, 2024
Fostering Equity Through a Shared Understanding of School Performance: American Samoa’s School Accountability Journey
In 2017, the American Samoa Department of Education (ASDOE) requested technical assistance from the Pacific Regional Comprehensive Center (PRCC) to develop a school accountability system to encourage student performance and to promote access to equitable education throughout the territory. Through partnership with ASDOE, the Region 19 Comprehensive Center (R19CC) has continued this work into 2024.
As a U.S. territory located in the South Pacific, American Samoa consists of five main islands with a total population of 49,710 (2020 U.S. Census). The school system comprises 22 elementary and 6 high schools with school populations ranging from 22 students to over a thousand students. As a result of its geography and varied school size, ASDOE and R19CC have worked to ensure that each school’s unique situation would be accommodated in the accountability system. ASDOE’s overarching goal has been to provide a shared understanding to parents, educators, and community members of how their schools were performing based on what stakeholders identified as the most important outcomes. ASDOE wanted to create its own system that would reflect its unique context, even if that meant departing from features that are typical in the accountability systems of the states.
Since 2017, Christine Le’iato, Director of ASDOE’s Office of Accountability and School Improvement Systems (OASIS) and OASIS team members have collaborated with the Comprehensive Center to develop, pilot, and implement the new School Accountability Improvement System, or SAIS. The R19CC has continued the support initiated by PRCC, and ASDOE is now preparing for a full public release of results for the 2023-2024 school year. As ASDOE approaches the milestone of the full SAIS launch, reviewing how equity and transparency have driven the system’s development highlights how the agency is ushering in unprecedented parent and community engagement.
Two consistent themes of Region 19’s accountability work with ASDOE—dating back to its predecessor center, PRCC—have been an emphasis on defining and implementing ASDOE’s vision and making full and sound use of education data. In meetings during 2018 and 2019, Comprehensive Center staff guided Director Le’iato and her ASDOE colleagues through discussions about the vision for the accountability system. These discussions were supported by an accountability reporting questionnaire. The discussions and questionnaire explored values and preferences regarding accountability reporting, including: the information that would best support school improvement activities, the timing of reports, and the relative importance of school growth-over-time versus school-to-school comparisons. At the start of the work, Comprehensive Center staff reviewed data from different schools; this review highlighted some concerns about data quality. One example was whether schools were tracking attendance in a consistent way. This early review suggested that the territory needed to focus on accurate data collection and reporting. Together with a focus on vision for the system, data quality represented an important goal for the early years of accountability development.
In working with American Samoa’s accountability committee, Comprehensive Center staff recognized that these officials would be the first ambassadors of the new accountability system. Their roles at the ASDOE include school visits and interactions with principals and teachers. Helping committee members to understand key accountability terms and begin using those terms consistently was one strand of technical assistance during the 2018-19 school year.
It was especially important to establish consistent usage of three terms: indicators, measures, and weights. In an accountability system, the indicators are the valued outcomes that indicate the degree to which students or schools are performing well. Reading achievement, math growth, and graduation are examples of indicators. Indicators are evaluated by measures, the specific tools and calculations that quantify performance. Examples of measures include: the percentage of students scoring proficient or advanced on the American Samoa Standards Based Assessment in English language arts, the change in the percentage of students scoring below basic on the American Samoa Standards Based Assessment in mathematics, and the four-year adjusted cohort graduation rate. Overall performance is measured by combining the various components through applying weights. Examples of weights include 25% for reading achievement for elementary (K-8) schools and 20% for reading achievement at high (9-12) schools.
The Comprehensive Center team brought examples of different states’ systems for the committee to consider. The committee pushed back on a practice common in the states of emphasizing a summary school rating or categorization. There was a consensus in the committee to avoid a summary A-through-F or multiple-stars rating. Instead, the committee wished to draw attention to the support that schools would receive. In addition, the committee liked state approaches that combined different components, with a clear weighting, for a summative score on a 0-100 scale.
The system design has been an iterative process. When the accountability committee first wanted to look at the technical properties of different options, not all the data needed was available. Region 19 staff conducted simulations using historical data and plausible data; the simulation results allowed the committee to see the effects of different approaches to determining indicator points and weighting. Over the years, the committee has adjusted the proposed weighting, based on simulation results and the availability of data.
Even as the Covid-19 pandemic disrupted data collection and set back the timing for a release of accountability results, the American Samoa accountability committee continued discussions about its vision for the system. Seeing the variability of growth in the simulation results led the committee to propose adjusting downward the weights given to growth. Discussions about engaging interest holders became more specific, as different committee members took responsibility for the various parts of a communication plan.
As the accountability committee has grown more familiar and comfortable with the technical aspects of the system, the committee has embraced shared responsibility. In the early years of the work, committee members deferred to Director Le’iato, who had a much deeper background in accountability than the members. In the last two years, however, Director Le’iato has empowered the team with more responsibility for decision making.
On issues related to communicating about the system and engaging a wider audience, the accountability committee has especially grown in its agency. Supported by R19CC, committee members have driven the development of a communications and outreach plan. One highlight from an April 2023 committee meeting was the insight that hand fans would be an effective vehicle for communicating about the system. Hand fans represent an item that is familiar in the Samoan context, inexpensive to produce in large quantities, and with a surface that can accommodate a brief message about the coming accountability system.
School accountability is new in American Samoa. As the 2023-24 school year winds down, the accountability committee is expanding its efforts to communicate about the system. In January 2024, committee members recorded a segment for local television. In the spring and summer, ASDOE is scheduling informational workshops with school staff. Additional outreach to parents will take place in the summer of 2024. Through all these efforts, ASDOE is seeking feedback from the community. The agency sees accountability as a dynamic, evolving process that gets better with input and improvements in data.
Apart from integrating feedback into system revisions, next steps for Director Le’iato and the accountability committee include collaborating with agency staff who support schools and specifying the support actions for schools in each of the broad categories of performance, called model, focus, and priority. For the first time in its history, American Samoa will have in place a school accountability system—a blueprint for actions driven by school performance data.
Type: Blog
Newsletter
