My greatest concern for AI in education: the invisible handshake

I had a mentor teacher who was fairly unethical, and once said to me,
“Horne, if you start getting complaints, give ’em all A’s. Those complaints will dry up.”

It was meant as a joke (I think), but like most bad jokes, it carried too much truth. The idea was simple: avoid scrutiny by keeping everyone happy even if it meant compromising the core purpose of education: teaching and learning.

That memory has been on my mind lately as I think about the role of AI in schools.

What happens when a teacher uses AI to design the lesson… AI to generate the assessment… students use AI to complete it… and the teacher uses AI to grade it… and everyone makes an A?

Nobody complains.
But nobody learns.
And everyone quietly agrees not to say the obvious part out loud: This isn’t real.

That’s my greatest concern—not that AI will destroy education, but that it will dull it. That we’ll settle for the illusion of learning because it’s easier, faster, quieter. That we’ll stop asking, “Did they grow?” and start asking, “Did it look good?”

We could end up with a system where everyone is satisfied—teachers, students, parents, even administrators—and yet nothing of substance is happening. It looks like learning, but it’s just going through the motions.

That’s the invisible handshake.

Tennessee School Letter Grades: A Machine Learning Update

In early 2024, I published a blog post exploring how machine learning could predict Tennessee school letter grades based on demographic data. That analysis provided insight into the structural factors influencing school performance metrics, particularly the significant role of economic disadvantage. Since then, I’ve expanded the dataset to include both the 2023 and 2024 school letter grades, refining the model to capture trends over time and further assess the predictive power of demographic variables. This post serves as an update on that work and a precursor to a more detailed research paper.

Revisiting the Predictive Model

Tennessee assigns letter grades (A–F) to schools based on multiple criteria, including student achievement, academic growth, and college/career readiness. While these grades are intended to reflect school quality, they often correlate strongly with socioeconomic and demographic factors.

For this updated analysis, I merged data from both years to improve model robustness. The methodology remained consistent:

  • Data Collection: Letter grades and demographic data were sourced from the Tennessee Department of Education.

  • Data Processing: Letter grades were converted to a numeric scale, and missing values were cleaned.

  • Machine Learning Models:

    • Random Forest Regression for feature importance analysis

    • Linear Regression to estimate the impact of individual factors

    • Correlation Analysis to identify relationships between variables

  • Letter Grade Scale

    A 4.5-5.0

    B 3.5-4.4

    C 2.5-3.4

    D 1.5-2.4

    F 1.0-1.4

Key Findings

Economic Disadvantage Remains the Strongest Predictor
Schools with higher percentages of economically disadvantaged students continue to receive lower letter grades. The model estimates that for every 10% increase in economically disadvantaged students, the expected letter grade drops by 0.35 on a 1.0–5.0 scale.

Feature Importance Score

Special Education Enrollment Also Impacts Letter Grades
The presence of students with disabilities (SWDs) is another contributing factor, though to a lesser extent. A 10% increase in SWD enrollment is associated with a 0.11-point decrease in letter grade.

Race and Economics Are Closely Linked
Schools with higher percentages of Black, Hispanic, and Native American students tend to have lower grades. However, when controlling for economic disadvantage, racial composition becomes a less significant predictor. This suggests that economic factors, rather than race itself, drive these disparities.

See the correlation matrix below:

Correlation Matrix

Demographics Explain About One-Third of Grade Variation
The updated linear regression model explains 29% of the variance in school letter grades, meaning that while demographics play a measurable role, other factors—such as instructional quality, funding, and school leadership—are also crucial.

What If Demographics Were Removed?
A simulated model scenario where a school had no economically disadvantaged students and no SWDs predicted an average letter grade of 4.5 (A range). This reinforces the idea that many low-performing schools face structural challenges beyond their control.

Public Schools: A Testament to Resilience

In the context of Tennessee’s voucher era, public schools operate under significantly different—and often more challenging—conditions than private schools, making their achievements all the more remarkable. The assumption that private schools inherently provide a superior education doesn’t hold up under scrutiny.

Unlike private institutions, which can be selective in their admissions, public schools serve every student—regardless of economic status, disability, or other challenges. Private schools often have the luxury of limiting enrollment to students who fit their preferred criteria, while public schools are tasked with educating all children who walk through their doors—often with fewer resources and greater accountability.

Despite these challenges, many public schools deliver exceptional outcomes. Educators work tirelessly to support students from diverse backgrounds, implementing innovative instructional strategies and targeted interventions to ensure student success. Their ability to foster academic achievement and social growth, even in the face of structural obstacles, speaks to the public education system's resilience, dedication, and effectiveness.

Public schools should be celebrated rather than maligned for their role in serving all students, strengthening communities, and proving that educational excellence is not exclusive to selective institutions.

This blog post was proofread with suggested corrections by Grammarly.

Vouchers in Northeast TN

Only 12 private schools qualify for vouchers in Northeast TN. Part of the voucher law said that only private schools that were category I, II, or III can receive voucher funds. You can see the list of Non-Public Schools here: https://www.tn.gov/education/families/school-options/non-public-schools.html

In Tennessee, non-public schools are classified into five categories, each with distinct approval processes and operational requirements:

  1. Category I: These schools are directly approved by the Tennessee Department of Education. They must adhere to state regulations similar to public schools, including employing licensed teachers and conducting annual standardized testing.

  2. Category II: Schools in this category are accredited by private agencies recognized by the State Board of Education. These accrediting agencies oversee the schools to ensure they meet specific educational standards.

  3. Category III: These are schools accredited by regional accrediting bodies approved by the State Board of Education. They operate under the guidelines of their respective accrediting organizations.

  4. Category IV: Known as church-related schools, these institutions are affiliated with religious organizations and are exempt from certain state regulations. They are recognized by associations listed in Tennessee Code Annotated § 49-50-801.

  5. Category V: These schools are acknowledged for operation with minimal state requirements. Teachers are required to hold at least a bachelor's degree but are not mandated to have teaching certificates.

Private Schools Eligible for Vouchers in Northeast Tennessee

  • Lakeway Christian Schools: Tri-Cities Christian Academy Campus – Blountville

  • Living Springs Christian Academy – Gray

  • Greeneville Adventist Academy – Greeneville

  • Lakeway Christian Schools: Boones Creek Christian Academy – Johnson City

  • Munsey Kindergarten – Johnson City

  • Providence Academy – Johnson City

  • St. Mary School - Johnson City – Johnson City

  • Central Baptist Kindergarten – Johnson City

  • St. Dominic Catholic School – Kingsport

  • All Saints' Episcopal School – Morristown

  • Lakeway Christian Schools: Cornerstone Christian Academy Campus – Morristown

  • Lighthouse Christian Academy (formerly Morristown SDA School) – Morristown

2023-2024 TN Letter Grades: Standout Schools

In my previous blog, I provided a detailed breakdown of the TN Letter Grades. In this post, I want to shift focus and highlight the schools that stand out as outliers, defying the norms. We’ll not only look at the schools that achieved perfect scores on their TN Letter Grades—excelling in Achievement, Growth, Growth 25, and College and Career Readiness (for high schools)—but also examine schools with high percentages of Economically Disadvantaged (ED) students, Students with Disabilities (SWD), and Black, Hispanic, and Native American (BHN) students that still managed to earn an “A.”

Elementary Schools with Perfect Scores

  • Oak Ridge: Willow Brook Elementary

  • Blount County: Carpenters Elementary School

  • Maryville: John Sevier Elementary

  • Carter County: Little Milligan

  • Elizabethton: West Side Elementary

  • Cheatham County: Kingston Springs Elementary

  • Cocke County: Grassy Fork Elementary

  • Metro Nashville Public Schools: Meigs Middle

  • Metro Nashville Public Schools: Valor Flagship Academy

  • Metro Nashville Public Schools: Valor Voyager Academy

  • Decatur County: Decaturville Elementary

  • Dickson County: Stuart Burns Elementary

  • Dickson County: Centennial Elementary

  • Gibson Co Sp Dist: South Gibson County Elementary School

  • Greeneville: Tusculum View Elementary

  • Hamilton County: Daisy Elementary School

  • Hamilton County: McConnell Elementary School

  • Hamilton County: Thrasher Elementary School

  • Hawkins County: Mt Carmel Elementary

  • Humphreys County: Waverly Elementary

  • Knox County: Blue Grass Elementary

  • Knox County: A L Lotts Elementary

  • Knox County: Farragut Intermediate

  • Knox County: Shannondale Elementary

  • Loudon County: Highland Park Elementary

  • McNairy County: Selmer Elementary

  • Madison County: Community Montessori School

  • Madison County: South Elementary

  • Marion County: Whitwell Middle School

  • Montgomery County: Sango Elementary

  • Rhea County: Frazier Elementary

  • Rutherford County: Buchanan Elementary

  • Rutherford County: Christiana Elementary

  • Rutherford County: Eagleville School

  • Rutherford County: Lascassas Elementary

  • Rutherford County: Thurman Francis Arts Academy/Magnet School for the Arts

  • Rutherford County: Walter Hill Elementary

  • Rutherford County: Wilson Elementary School

  • Rutherford County: Stewartsboro Elementary

  • Rutherford County: Stewarts Creek Elementary School

  • Collierville: Tara Oaks Elementary School

  • Germantown: Dogwood Elementary School

  • Lakeland: Lakeland Elementary School

  • Bristol: Holston View Elementary

  • Kingsport: Thomas Jefferson Elementary School

  • Kingsport: Andrew Johnson Elementary School

  • Sumner County: Bethpage Elementary

  • Sumner County: Howard Elementary

  • Sumner County: Liberty Creek Elementary

  • Sumner County: Oakmont Elementary

  • Sumner County: Clyde Riggs Elementary

  • Sumner County: Union Elementary School

  • Johnson City: Lake Ridge Elementary

  • Johnson City: South Side Elementary

  • Williamson County: Creekside Elementary School

  • Williamson County: Trinity Elementary

  • Wilson County: Lakeview Elementary School

  • Wilson County: Rutland Elementary

High Schools with Perfect Scores

  • Oak Ridge: Oak Ridge High School

  • Metro Nashville Public Schools: Hume - Fogg High

  • Metro Nashville Public Schools: Martin Luther King Jr School

  • Metro Nashville Public Schools: Valor Flagship Academy

  • Greeneville: Greeneville High School

  • Hamilton County: Chattanooga High School Center for Creative Arts

  • Hamilton County: Chattanooga School for the Arts and Sciences Upper

  • Hamilton County: East Hamilton High School

  • Hamilton County: Hamilton County Collegiate High at Chattanooga State

  • Hamilton County: STEM School Chattanooga

  • Knox County: L N STEM Academy

  • Madison County: Jackson Central-Merry Early College High

  • Madison County: Madison Academic Magnet High School

  • Rutherford County: Central Magnet School

  • Rutherford County: Eagleville School

  • Memphis-Shelby County Schools: Medical District High School

  • Memphis-Shelby County Schools: East High

  • Memphis-Shelby County Schools: Middle College High

  • Memphis-Shelby County Schools: University High School

  • Memphis-Shelby County Schools: White Station High

  • Memphis-Shelby County Schools: Whitehaven High

  • Arlington: Arlington High

  • Bartlett: Bartlett High School

  • Collierville: Collierville High School

  • Williamson County: Franklin High School

  • Williamson County: Independence High School

  • Williamson County: Nolensville High School

  • Williamson County: Ravenwood High School

  • Wilson County: Green Hill High School

  • Tennessee Public Charter School Commission: KIPP Antioch Global High School

A Schools with High Economically Disadvantaged Percentages (50% or Higher)

For 2023-2024 students, the average percentage of ED students in F schools was 49.64%. Using that as a benchmark, here are the schools that earned an A with ED percentage of 50% or higher.

  • Putnam County: Burks Elementary (66%)

  • Memphis-Shelby County Schools: Delano Elementary (63%)

  • Putnam County: Northeast Elementary (58%)

  • Memphis-Shelby County Schools: Leadership Preparatory Charter School (58%)

  • Memphis-Shelby County Schools: Newberry Elementary (55%)

A Schools with High Black, Hispanic, and Native American (BHN) Percentages (83% or Higher)

For 2023-2024 students, the average percentage of BHN students in F schools was 82.2%. Using that as a benchmark, here are the schools that earned an A with a BHN percentage of 83% or higher:

  • Memphis-Shelby County Schools: Medical District High School (97.5%)

  • Memphis-Shelby County Schools: Southwind Elementary (94.0%)

  • Memphis-Shelby County Schools: Delano Elementary (97.5%)

  • Memphis-Shelby County Schools: East High (85.0%)

  • Memphis-Shelby County Schools: Middle College High (97.5%)

A Schools with High Students with Disabilities (SWD) Percentages (16% or Higher)

For 2023-2024 students, the average percentage of SWD students in F schools was 15.96%. Using this as a benchmark, here are the schools that earned an A with an SWD percentage of 16% or higher:

  • Anderson County: Grand Oaks Elementary (25%)

  • Clinton: North Clinton Elementary (31%)

  • Oak Ridge: Willow Brook Elementary (22%)

  • Bedford County: Cascade Elementary (16%)

  • Blount County: Carpenters Elementary School (18%)

  • Putnam County: Burks Elementary (66%)

  • Memphis-Shelby County Schools: Delano Elementary (63%)

  • Putnam County: Northeast Elementary (58%)

  • Memphis-Shelby County Schools: Leadership Preparatory Charter School (58%)

  • Memphis-Shelby County Schools: Newberry Elementary (55%)

Comparing 2022-2023 and 2023-2024 School Letter Grades in Tennessee

The impact of school letter grades on schools, school leaders, teachers, and students has largely unfolded as anticipated. It's not just a coincidence that letter grade legislation often serves as a catalyst for voucher programs across numerous states. By simplifying school accountability into easily digestible grades, we risk oversimplifying a complex system that even experts grapple with. We must urge a more comprehensive understanding of educational quality that goes beyond mere letters, ensuring that real progress is made for our students and educators alike.

To understand how the computation of these letter grades affects schools, I took a preliminary look last year at the grade distribution. The improvement of achievement percentages is fundamental to federal accountability, but it has been entirely removed from these state letter grades.

In the second year, I was confident we would witness significant improvements. Given the wealth of experience that many educators possess, having navigated through the complexities of school accountability, I believe we would have effectively deciphered the strategies needed to enhance our performance and elevate those letter grades.

Let’s compare the two years.

Data Sources

2024 School-Level Profile Data

2023-24 A-F Letter Grade File

Out of the 1,905 schools listed for letter grades in Tennessee, 215 schools (11.29%) were ineligible to receive a grade. These schools were excluded from the analysis to focus on the distribution of grades among eligible schools. This is similar to 2022-2023 where 210 out of 1900 schools were ineligible. To see how this is determined, you can visit the TN DOE Webpage on letter grades here.

Distribution

In 2023-2024, the letter grades were distributed as follows:

  • A: 320 (17%)

  • B: 450 (24%)

  • C: 500 (27%)

  • D: 330 (18%)

  • F: 80 (4%)

Compared to 2022-2023, the distribution has shifted, highlighting some key trends:

  1. Increase in A Grades:

    • The percentage of schools receiving an A increased slightly from 17% to 17% (294 to 320 schools).

  2. Slight Decline in C Grades:

    • C grades decreased from 30% (513 schools) to 27% (500 schools), with schools likely moving into higher or lower grade categories.

  3. Reduction in D and F Grades:

    • D grades fell from 21% (350 schools) to 18% (330 schools), and F grades dropped from 5% (92 schools) to 4% (80 schools). This is a positive trend, suggesting fewer schools are struggling at the lowest levels.

This year's distribution continues to resemble a normal curve, with a slight skew toward A grades. While last year, 77% of schools fell into the B, C, or D categories, this year that number has dropped slightly to 75%, reflecting improvements at both the top and bottom of the grading scale.

Generated by ChatGPT

Performance Trends Across Metrics (2022-2023 vs. 2023-2024)

The comparison of average scores across metrics provides insight into how schools in Tennessee have performed over the past two years. Here’s a breakdown of the trends by letter grade:

Average Achievement Scores by Letter Grade

  • 2022-2023:

    • A: 4.85 | B: 3.95 | C: 2.96 | D: 2.00 | F: 1.02

  • 2023-2024:

    • A: 4.84 | B: 3.99 | C: 3.04 | D: 2.06 | F: 1.05

Key Changes:

  • Grades B and C saw slight improvements, with C grades showing a noticeable increase from 2.96 to 3.04.

  • Grades A, D, and F remained relatively stable, with minor differences that do not indicate significant shifts.

Average Growth Scores by Letter Grade

  • 2022-2023:

    • A: 4.96 | B: 3.92 | C: 2.80 | D: 1.62 | F: 1.00

  • 2023-2024:

    • A: 4.92 | B: 3.82 | C: 2.73 | D: 1.60 | F: 1.03

Key Changes:

  • A Grades dropped slightly from 4.96 to 4.92, although they remain well above the threshold for full growth points.

  • Grades B and C also saw modest declines, while F Grades slightly improved from 1.00 to 1.03.

Average Growth25 Scores by Letter Grade

  • 2022-2023:

    • A: 4.45 | B: 3.58 | C: 3.10 | D: 2.64 | F: 3.21

  • 2023-2024:

    • A: 4.40 | B: 3.54 | C: 2.99 | D: 2.64 | F: 2.69

Key Changes:

  • F Grades dropped from 3.21 to 2.69, reflecting a notable decrease in the lowest-performing schools' progress for students in the bottom quartile.

  • Grades A, B, and C showed small decreases, while D Grades remained unchanged at 2.64.

Grade-by-Grade Observations

  • Grade A:

    • Continues to represent the highest levels of achievement and growth. However, small declines in both growth and growth25 scores may warrant further investigation into sustaining top-tier performance.

  • Grade B:

    • Slight improvements in achievement but minor decreases in growth and growth25 suggest consistent performance with room for growth.

  • Grade C:

    • The improvement in achievement (from 2.96 to 3.04) is encouraging, though declines in growth and growth25 could indicate challenges in maintaining momentum.

  • Grade D:

    • Minimal changes across metrics show stability but limited progress in improving scores.

  • Grade F:

    • While achievement scores improved slightly (1.02 to 1.05), the sharp decline in growth25 (3.21 to 2.69) highlights ongoing difficulties in addressing the needs of the lowest-performing students.

Subgroup Performance

One of the most revealing pieces of data from 2022-2023 was the high percentage of economically disadvantaged (ED) and Black/Hispanic/Native American (BHN) students in F schools. I wondered if that trend would stay the same or change, and for ED students, it has improved.

Generated by ChatGPT

Key Insights

  1. Increase in Economic Disadvantage for Higher Grades:

    • Schools with an A grade saw an increase from 18.34% (2022-2023) to 22.27% (2023-2024) in economically disadvantaged students. This suggests a positive trend toward equity in top-performing schools.

  2. Slight Increase for B and C Grades:

    • B grades rose from 28.92% to 31.10%, and C grades increased from 36.49% to 38.90%. These shifts indicate a growing representation of disadvantaged students in mid-tier schools.

  3. Reduction in D and F Grades:

    • The percentage of disadvantaged students in D grades dropped from 41.97% to 45.93%. Similarly, F grades saw a reduction from 54.52% to 49.64%.

    • This may reflect targeted interventions or progress in struggling schools.

This trend is further reflected in the correlation between the percentage of economically disadvantaged students and letter grade scores. In 2022-2023, the correlation was -0.50, while in 2023-2024, it decreased slightly to -0.44. Although still significantly negative, the weaker correlation suggests a modest reduction in the impact of economic disadvantage on school performance.

BHN Improvement

One of the most disturbing and glaring trends from 2022-2023 was the high percentage of BHN students in F schools. This has improved dramatically in 2023-2024. Here is a comparison graph.

Generated by ChatGPT

Key Insights

  1. Increases in A and B Grades:

    • BHN representation in A-grade schools increased significantly from 21.82% to 35.79%.

    • B-grade schools also saw a large jump from 27.74% to 48.72%, reflecting positive strides toward equity in higher-performing schools.

  2. Consistent Growth Across Grades:

    • BHN percentages rose steadily across all letter grades, with C and D schools showing increases of ~20 percentage points each.

  3. Marginal Change in F Grades:

    • BHN representation in F-grade schools increased slightly from 80.05% to 82.20%, indicating persistent challenges for the most disadvantaged schools.

The Pearson Correlation Coefficient between BHN student percentages and letter grade scores remained virtually unchanged, increasing slightly from -0.37 in 2022-2023 to -0.39 in 2023-2024.

Students with Disabilities Populations

The impact of Students with Disabilities (SWD) on letter grades felt counter-intuitive in 2022-2023. Schools with A and F grades had nearly identical percentages of SWD students, raising questions about how disability representation aligns with school performance metrics. In 2023-2024, this pattern shifted slightly, with SWD representation increasing across all letter grades.

Notably, the largest growth occurred in F-grade schools, where SWD percentages rose from 12.24% in 2022-2023 to 15.96% in 2023-2024. This sharp rise highlights a growing concentration of SWD students in the lowest-performing schools, emphasizing systemic challenges that disproportionately affect these populations.

Meanwhile, higher-performing schools also experienced modest increases in SWD percentages, with A-grade schools rising from 12.87% to 13.53%. Despite these gains, the gap between high- and low-performing schools widened, underscoring the need for equity-driven interventions to support SWD students more effectively.

This evolving dynamic invites further exploration into the role of SWD representation in shaping school letter grades and how policies can better address the unique challenges faced by these students.

Generated by ChatGPT

The Pearson Correlation Coefficient between SWD percentages and letter grade scores remained relatively weak, with a value of -0.13 in 2023-2024, compared to -0.09 in 2022-2023, indicating no significant relationship between the two.

Interestingly, the SWD distribution graph for 2023-2024 more closely resembles the patterns observed in the ED and BHN graphs, aligning with expectations and suggesting potential connections between these demographics and school performance metrics.

Conclusions

Schools in the state performed better in 2023-2024 on their school letter grades than they did in 2022-2023. The percentage of ED, BHN, and SWD students increased, reflecting a positive shift toward greater inclusion and representation across various demographics. However, disparities persist, particularly in the correlation between economic disadvantage and letter grades, as well as the overrepresentation of BHN and SWD students in lower-performing schools.

While the overall improvement in letter grades is encouraging, these results underscore the need for targeted interventions and support for schools serving disadvantaged populations. Policies should focus on addressing systemic inequities to ensure all students, regardless of background, have access to the resources and opportunities needed to succeed.

Future analyses should continue monitoring these trends to assess the long-term impact of accountability measures and demographic shifts on educational equity and performance.

AI Disclaimer

I used ChatGPT to generate comparative bar charts since I had them in two different Jupyter Notebooks. It was easier to send screenshots of them and have it create new ones. I also used ChatGPT to clean up my writing and formatting.

Six years of Improvements in Campbell County

I only have a few weeks left in Campbell County, and I wanted to reflect on it. I want to write about the improvements that we have made since I’ve been here. In district supervisor positions, you’re always on a continuum of change and improvement. The truth is the job is never finished.

The other truth is that you never do this job alone. I have been part of one of the strongest teams I can even imagine. So please don’t read this blog that these are my accomplishments. I’ve only been fortunate to have been part of them.

It is a team effort that is led by the Director of Schools. If you don’t have a strong Director, then it is tough to get anything accomplished. In this situation, I had a very strong Director in Ms. Fields who pushed all of us to balance improvement while considering the already strenuous workload on teachers, principals, and students. Ms. Fields also reminded us regularly not to forget what it’s like to be in a classroom and to lead a school. She encouraged us to be empathetic to our people but to have high standards for our outcomes. This balance is what has allowed all of us to be successful.

In no order of importance, I thought I would reflect on some of the accomplishments that come to mind.

Collaborative Conferencing

Being involved in PECCA was a great experience. It really put me in touch with the past, present, and future of Campbell County. When I initially came on the Administration Team, there were a lot of employee organization leaders who had been involved in Collective Bargaining prior to 2011. I remember these folks and their work, and I noticed the MOU felt more like the Collective Bargaining Agreement than an MOU.

Something else that became obvious to me is that most teachers had no idea that they had an MOU and that they had all of these people working on it. It is hours of work after hours for which no one is getting paid anything extra. It is truly a labor of love, and for employee organizations, it is the most direct form of advocacy that they have. 

While I agreed with most of what was there, there were some glaring loopholes that needed to be closed. For example, the MOU originally read that a teacher could take up to 20 consecutive days of unpaid leave before they were put on unpaid leave by the Director of Schools. This meant that a teacher could take 19.5 days off, come back for half a day, and then take 19.5 days off again. This means a teacher could keep their position while only showing up for 8 days a school year. It wasn’t until this was being discussed that I saw an employee test these waters, but we finally changed it.

The section on unpaid leave is now more in line with policy and procedures. It states, “To prevent being placed on automatic leave, in cases where the teacher cannot apply for leave under the Family Medical Leave Act, the teacher must provide documentation to the Director of Schools to justify their unpaid leave. “This was a lot of work and discussion by the PECCA Members from both the Admin Team and the Teacher Team. I consider it a huge accomplishment that we got this done, and even added a definition of sick leave which was completely missing.

Let me be even more emphatic about it: there was no sick leave policy for certified employees at all. This is why it was being abused and people had an entitled attitude about “their days.” These days technically belong to the state which is why you get to transfer them from system to system, and the state is very clear about how they are to be used. Scanning other systems’ policies for sick leave showed just how restrictive they were elsewhere, and I felt like I was spinning my wheels trying to solve Chronic Student Absenteeism when I couldn’t solve Chronic Teacher Absenteeism.

It is very satisfying to look back on that work to see that everyone in the group saw that this wasn’t a tenable situation for a school system that wanted to be whole and healthy. At our best, we had everyone working together. There were some long evenings and not everyone saw eye-to-eye, but in the end, we got things done, and I will always be proud of that.

Test Scores

To be clear, the test scores always belong to the students. After all, they’re the ones who take the tests. Even though this is true, test scores are the backbone of school accountability, and even though that changed dramatically with the school letter grades, we’ve been getting school report cards for a long time.

For those who aren’t in education, there are two ways to look at your progress in test scores. One way is called “Achievement” and the other way is called “Growth.” Achievement is the percentage of students who score “proficient” on the TCAP or EOC. Growth is a complex measure that predicts what a student’s test score should be, and then whether it is above that or below it, it is considered positive or negative growth. When put all of that together, you get a teacher’s growth, a school’s growth, and so on.

In Campbell County, our Achievement has improved every single year. As you can see from this School Board Data Presentation, it has improved every year.  

District Achievement Overview

The provided charts illustrate the trends in district achievement over time, specifically focusing on the growth in English Language Arts (ELA) and Math across different grade levels: elementary (grades 3-5), middle (grades 6-8), and high school (grades 9-12).

District Achievement Over Time

  • Grades 3-5: Achievement increased from 23 in 2020-2021 to 34 in 2022-2023, indicating an 11-point growth.

  • Grades 6-8: Achievement rose from 17.5 in 2020-2021 to 25 in 2022-2023, marking a 7.5-point growth.

  • Grades 9-12: Achievement saw a significant rise from 14.4 in 2020-2021 to 26.8 in 2022-2023, showing a 12.4-point growth.

District-wide Performance in ELA and Math

  • Overall Trends: Both ELA and Math have shown considerable growth over the years, with notable increases from 2021 to 2023.

    • ELA: Increased from 21.6 in 2017 to 28.4 in 2023.

    • Math: Grew from 17.7 in 2017 to 24.6 in 2023.

Elementary School Performance

  • ELA: The performance has steadily increased, reaching 34 in 2023 from 22 in 2017.

  • Math: Similarly, Math performance rose to 33 in 2023 from 22 in 2017.

Middle School Performance

  • ELA: After a drop in 2021, ELA performance rebounded to 22 in 2023, slightly below the 2019 peak of 25.

  • Math: Following a dip in 2021, Math performance recovered, reaching 27 in 2023, just shy of the 2019 peak of 31.

High School Performance

  • ELA: High school ELA performance has shown significant improvement, reaching 28 in 2023, up from 21 in 2017.

  • Math: Despite some fluctuations, Math performance improved to 12 in 2023 from a low of 5 in 2018.

Comparative Insights

  1. Elementary School Improvement:

    • Both ELA and Math have shown consistent improvement, suggesting effective early education strategies. The continuous rise in performance indicates a solid foundation being built at this level.

  2. Middle School Improvement:

    • ELA and Math faced a dip in 2021, likely due to disruptions caused by external factors (e.g., the pandemic). However, the rebound in 2022 and 2023 suggests recovery efforts are taking effect, though there's room for improvement to reach or exceed past peaks.

  3. High School Improvement:

    • ELA shows a marked improvement, indicating effective teaching strategies and student engagement at the high school level. Math, while improving, shows more variability, highlighting the need for targeted interventions to sustain and further this growth

Growth

Growth tells a different story than achievement. While many of our students are performing better than they were in 2017, the overall district growth decreased from a level 3 to a level 1 in 2023. The reason for this is because of so many large negative growth scores. This suggests that students are not performing their best on the test. Given their benchmarking scores suggest they are capable of performing better than they did, it suggests that students are just not trying on the TCAP and EOC tests. We are putting in a lot of programs to make that happen.

Here are high school growth levels over the past three years:

The analysis of high school growth indices for the subjects Algebra I, Algebra II, English I, English II, and Geometry over the years 2021 to 2023 reveals several trends and insights.

  1. Algebra I:

    • The Growth Index shows a positive trend initially, peaking in 2022. However, there is a noticeable decline in 2023, bringing the index closer to its 2021 level. This suggests that while there was initial improvement, the gains were not sustained.

  2. Algebra II:

    • Similar to Algebra I, Algebra II also experienced an upward trend, peaking in 2022 before a significant decline in 2023. This indicates a potential issue in maintaining growth in Algebra II over the long term.

  3. English I:

    • The Growth Index for English I shows an initial increase, peaking in 2022, followed by a decline in 2023. Despite the fluctuation, the overall trend suggests a need for focused strategies to sustain and improve growth.

  4. English II:

    • English II demonstrates a consistent decline in the Growth Index from 2021 to 2023. This steady decrease highlights the need for interventions and targeted support to reverse the downward trend.

  5. Geometry:

    • Geometry shows a positive and continuous increase in the Growth Index from 2021 to 2023, indicating successful strategies and improvements in teaching and learning practices in this subject.

Overall, the high school data reflects a mixed performance with notable areas for improvement, especially in maintaining growth in Algebra I, Algebra II, and English I, and reversing the decline in English II. The positive trend in Geometry is encouraging and may serve as a model for other subjects.

Here are growth rates for Math and ELA in Grades 3-8:

Subjects: English Language Arts (ELA) and Math

The analysis of growth indices for English Language Arts (ELA) and Math for grades 4-8 over the years 2021 to 2023 provides insights into student performance and areas needing attention.

  1. English Language Arts (ELA):

    • Grade 4: The Growth Index for ELA in Grade 4 remained relatively stable, indicating consistent performance without significant fluctuations.

    • Grade 5: The Growth Index shows minor fluctuations but remains generally stable, suggesting a need for sustained support to drive further improvement.

    • Grade 6: There is a steady decline in the Growth Index from 2021 to 2023, indicating challenges in maintaining growth and highlighting the need for targeted interventions.

    • Grade 7: The trend shows a slight decline, calling for strategies to enhance growth and support students in this grade.

    • Grade 8: The Growth Index shows a consistent decline, indicating significant challenges and a need for robust interventions to reverse the trend.

  2. Math:

    • Grade 4: Math shows a stable but slightly declining trend, emphasizing the need for continuous support.

    • Grade 5: The Growth Index for Math displays a declining trend, suggesting areas needing improvement.

    • Grade 6: Similar to ELA, the Math Growth Index shows a decline, pointing to a need for targeted efforts to boost performance.

    • Grade 7: The trend shows a decrease, indicating challenges in maintaining growth.

    • Grade 8: The Growth Index shows fluctuations, with a decline from 2022 to 2023, indicating areas needing attention and support.

Overall, the data for grades 4-8 highlights the need for sustained efforts to improve and maintain growth in both ELA and Math. While some grades show stability, others indicate significant areas for improvement, particularly in grades 6-8. Focused interventions and continuous support are essential to reverse declining trends and enhance overall student performance in these critical subjects.

One of the biggest changes we’ve made is the transparency with data. When I came on board, I was told that AMOs (Annual Measurable Objectives) were handed out on a sticky note. I was also told that they would present whatever they wanted about data before the board. While testing accountability wasn’t as hot and heavy as it now, they still had the same numbers to give the board.

Ultimately, more students are performing better on their state tests, and that is good news for the students in Campbell County. This is a group effort from students, teachers, principals, and central office administrators.

ACT

Campbell County improved its ACT average from 16.6 in 2020-2021 to 17.3 in 2022-2023. Also, the county the most students with an ACT over 21 in its history in 2022-2023. Also the participation rate for ACT is 99%. It had fallen below 95% in 2017-2018.

Ready Graduate

The percentage of Ready Graduates has increased from 26.5% in 2017-2018 to 40.1% in 2023-2024.

CTE Concentrators

The percentage of CTE Concentrators improved from 36.5 in 2017-2018 to 54.3% in 2022-2023.

PostSecondary Going Rates

  1. Pre-Pandemic Growth:

    • The district saw a steady increase in postsecondary enrollment rates from 2016-2017 to 2018-2019, peaking at 57.3%. This suggests effective strategies and strong support systems were in place to encourage students to pursue higher education.

  2. Impact of COVID-19:

    • The 2019-2020 and 2020-2021 academic years saw a decline in enrollment rates, dropping to 47.3% and 46.9%, respectively. The disruptions caused by the pandemic likely played a significant role in this decline. These years highlight the challenges faced by students and the need for adaptive strategies during crises.

  3. Post-Pandemic Recovery:

    • The increase to 51.8% in 2021-2022 indicates a positive recovery trend. This suggests that the district's efforts to support students in the aftermath of the pandemic are beginning to take effect.

Processes

There have been so many processes that have improved since 2018. It is impossible to list them all, but here are the ones that come to mind. Again, these are all group efforts.

·      A comprehensive VOIP telephone system for the entire county.

·      A modern website with an app.

·      A robocall system for absences and emergencies.

·      Upgraded technology for every school.

·      A Facebook page for the district and every school.

·      Teacher of the Year recognition

·      An online application and HR system

·      Compliance with the new counselor standards

·      Updated 504 procedures

·      Enrollment Dashboarding

·      Conduct Data Collection

·      A Tiered System for Truancy

·      Data and Accountability updates for principals

·      Weekly TEAM reports

·      Ayers training for Academic Coaches

·      Regular meetings for Academic Coaches

·      A leadership academy for teachers.

·      A process for awarding and tracking tenure.

·      A solid chain of command structure.

·      McREL Training

·      SREB Training

·      Interventionist Positions

·      Increased AP and Dual Enrollment participation

·      Vans for CTE and for Homeless student transportation

·      Ayers Scholars Program

·      FAFSA Frenzy

·      Turf fields at both high schools (JHS is currently being planned.).

·      We trained over 250 substitute teachers while I was there.

·      We built an online platform during COVID.

·      We had some of the fewest COVID closures in the state.

TN Letter Grades: An Unsupervised Learning Clustering Approach

One of the more interesting ways to look at data is to use machine learning to sort your data into clusters. The particular tool I used for this is called K-Means Clustering, a popular algorithm in the field of data science for its simplicity and efficiency. But what exactly is K-Means Clustering? At its core, K-Means is a method that aims to partition a dataset into distinct groups (clusters) such that the data points in each group are as similar to each other as possible, while also being as different as possible from the points in other groups.

Why is this approach useful? We often deal with large amounts of data that can seem impenetrable at first glance. By organizing this data into clusters, we can identify patterns and characteristics that are not obvious to us at first. For instance, when we analyze schools across various districts, K-Means Clustering can reveal groupings of schools with similar challenges or successes, helping us to tailor support and resources more effectively.

For this analysis, I wanted to look closer at Overall Success Rate, Economically Disadvantaged Percentage, and Black/Hispanic/Native American percentage using the same dataset I used for my initial Letter Grades report. I reduced the dataset to those three features, and I had to convert the values in the Success Rate column from <5% to 2.5 and >95% to 97.5 and the convert those values to floats.

Here are the basic descriptive statistics.

Descriptive Statistics

Here is how the data is distributed.

Histograms of each feature

The Success Rate histogram shows a unimodal distribution centered around 30-40%. The distribution is slightly skewed to the right, indicating that while most schools have a success rate in the middle range, there are fewer schools with very high success rates.

The distribution of the percentage of economically disadvantaged students is also unimodal and seems to be slightly skewed to the right. Most schools have between 20% to 40% economically disadvantaged students, with fewer schools having very high or very low percentages.

The BHN histogram is different from the other two, showing a bimodal distribution. One peak is around the 0-10% range, and another, more pronounced peak, is at the 90-100% range. This suggests that schools tend to have either a very low or very high percentage of Black, Hispanic, or Native American students, with fewer schools having a moderate percentage. This histogram supports Kozol’s research that American schools are still segregated.

Next, I wanted to see how each of these values correlated. I did scatterplots and ran a Pearson’s r to see the relationships between the data.

Scatterplots and Pearson’s r correlation coefficients

No surprise, but the data shows the following:

Success Rate vs. Economically Disadvantaged: The correlation coefficient is -0.72, indicating a strong negative correlation. This means that as the percentage of economically disadvantaged students increases, the overall success rate tends to decrease.

Success Rate vs. BHN: The correlation coefficient is -0.56, suggesting a moderate negative correlation. So, higher percentages of BHN students are associated with lower overall success rates.

Economically Disadvantaged vs. BHN: The correlation coefficient is 0.61, showing a strong positive correlation. This implies that higher percentages of economically disadvantaged students are often found in schools with higher percentages of BHN students.

The Clustering Model

Before running the model, I scaled the data using the standard scaler. This is crucial for K-Means Clustering. Here is an article about that if you want to read it. And of course, I ran an elbow plot to find the optimal number of clusters.

The elbow plot

The elbow plot settled on five clusters. After fitting the model and running it for 5 clusters, I generated a 3D Scatterplot of the 5 clusters just to have a visual of the differences. The red star represents the centroid of the cluster.

3D Scatterplot

The clusters that it generated can be described as follows:

  • Cluster 0 (286 schools) has a relatively low overall success rate of about 25%, a moderate percentage of economically disadvantaged students (around 38%), and a very high percentage of Black, Hispanic, or Native American students (approximately 70%).

  • Cluster 1 (187 schools) is characterized by a high overall success rate of around 71%, a low percentage of economically disadvantaged students (about 8%), and a lower percentage of Black, Hispanic, or Native American students (roughly 18%).

  • Cluster 2 (490 schools) features a low-to-moderate overall success rate of about 33%, a moderate percentage of economically disadvantaged students (also around 38%), but a lower percentage of Black, Hispanic, or Native American students (about 12.5%).

  • Cluster 3 (231 schools) has the lowest overall success rate of approximately 15%, the highest percentage of economically disadvantaged students (around 66%), and a very high percentage of Black, Hispanic, or Native American students (nearly 95%).

  • Cluster 4 (476 schools) shows a moderate overall success rate of around 45%, with a lower percentage of economically disadvantaged students (about 21%) and a percentage of Black, Hispanic, or Native American students similar to the previous value (around 21%).

Here is a bar chart showing the Cluster Profiles.

A bar chart of each cluster

And to illustrate how many schools are represented in each cluster, here is a humble pie-chart.

Conclusions

Diversity in School Profiles: The clusters represent a wide range of school profiles, from those with high success rates and low percentages of economically disadvantaged and minority students (Cluster 1) to those facing significant challenges with high percentages of disadvantaged and minority students and low success rates (Cluster 3).

Economic Disadvantage and Success Rates: There appears to be a correlation between economic disadvantage and overall success rates, as seen in the negative correlation coefficients and the cluster characteristics. Schools with a higher percentage of economically disadvantaged students tend to have lower overall success rates (Cluster 0 and Cluster 3).

Racial and Economic Segregation: The bimodal distribution of the percentage of Black, Hispanic, and Native American students indicates potential racial and economic segregation within the school system. Some schools have very high percentages of minority students, while others have very low percentages, with fewer schools in between.

Most Schools Do Not Have a High Success Rates: Cluster 4 and Cluster 1 schools have high success rates. Typically, 45% is the bar schools want to reach because that represents maximum points in the federal accountability model for success rate. These only represent 39.6% of all schools. This means that 60.4% of all schools are falling below that mark.

Cluster 4 stands out

Cluster 4 stands out as a cluster with some diversity and high success rate. The means for Economically Disadvantaged (20.99) and BHN (21.24) are still much lower than the overall means for those categories.

What do you see in this data?

What do you tell teachers about AI?

I have spent the past year and change exploring the possibilities, limitations, and risks of Large-Language-Model AI, especially ChatGPT. In my role at work, I haven’t done a lot on it because we don’t have an adopted Board Policy on it, and TSBA hasn’t written a model policy for boards. This leaves us in a weird space where we know that this is out there and people are using it, but we don’t have any guidance or governance for it. I don’t think ignoring it for now is the answer, and I wanted to share what I have communicated so far so that it might be helpful to other districts.

Handbook Policy

You don’t need a board policy to have a handbook policy, so we put this in our high school handbooks at the beginning of the year:

AI Handbook Entry for Academic Integrity and Honesty in the Use of Large Language Models

·      Purpose

This handbook entry aims to ensure the upholding of academic integrity and honesty in the context of the use of large language models (LLMs) such as ChatGPT in our school environment.

·      Scope

This handbook entry covers all students, staff, and any other individuals who interact with our school's academic programs and services, and who use LLMs for academic purposes.

·     Handbook entry Guidelines

    • Proper Citation: Students must properly acknowledge and cite the use of LLMs in their work. Any idea, phrase, or output generated by AI must be cited just as any other source would be.

    • Original Work: While LLMs can be used for assistance and guidance, the work submitted by students must fundamentally be their own. The use of AI should be to facilitate and enhance the learning process, not to replace individual effort and creativity.

    • Collaboration: While working collaboratively, students must clearly state the contributions made by AI. Collective work should reflect a clear understanding of the contributions made by each student and the AI model used.

    • Access: All students should have equitable access to AI tools to ensure fairness. The school will strive to provide the necessary resources and training for all students.

    • Educator Guidelines: Teachers should educate students about the ethical use of AI and its potential impacts on academic integrity. They should also receive regular training to stay updated on the capabilities and limitations of AI.

·      Implementation and Compliance

This handbook entry should be communicated effectively to all relevant parties. The school will conduct regular checks to ensure compliance. Any violation of this handbook entry will be considered a breach of academic integrity, and the school's standard disciplinary measures will be applied.

Simply enough, we have let students know ahead of time that students can’t use LLMs to produce final products. We’ve also let teachers know that they need to be teaching students how to use LLMs to their advantage.

Where are we now?

We haven’t, to my knowledge, had any issues with students getting caught cheating with LLMs, but that doesn’t mean it hasn’t happened. In fact, the whole inspiration for me writing this is that a student told me that she wouldn’t use AI to help her study French because another student had submitted an essay in her English class using AI and she got the same grade as him and it made her angry.

Because of that conversation, I put together a document for teachers, and I thought I’d share that content here.

So why not just avoid AI for as long as we can?  

·      You can tell when AI has written something, and I’m surprised when anyone can’t. Have you used AI enough to pick up its tone and patterns? It uses too many adverbs. In emails, it always says some affectation like “I hope this email finds you well.”

·      AI isn’t going anywhere. As a matter of fact, it’s the worst quality and the least integrated today than it will ever be in our students’ lives. We have to learn to live with it, and students are going to need to know how to interact with AI now. It can really give them a huge advantage in life if used ethically and responsibly.

·      Withholding the power of any technology from our students only withholds it from certain students. Typically, only the students who are disadvantaged will not learn to use technology when it is withheld from them in school.

·      We can’t have students using this technology to cheat, and avoiding teaching them how to use it responsibly will not prevent them from cheating. In fact, letting students know that we are very knowledgeable about it will make them think twice about using it to cheat.

So how should students be learning to use AI?

·      Helping them get organized.

·   Asking it simple questions and interacting with it. For example, this student is having trouble with conversational French. It can have a conversation with her, and she can practice her French with it. You can’t get that anywhere else without a pen pal or French friend.

·      Asking it to make a study guide.

·      Asking it to quiz you on something.

·      Asking it to help you with the pre-writing phase of writing.

·      Asking it to proofread your paper (that you wrote) and give you feedback on it. You could even ask it to evaluate the paper with a rubric that the teacher gave them.

·      Asking it to explain difficult concepts in simple ways.

·      And many other ways…

Here are some samples:

Example: Helping them get organized.

Sample Prompt: We’re learning about cellular energy in my high school biology class in Tennessee. Can you help me get organized with an outline? I will keep you posted on what we’re studying in class so you can help me make a study guide.

Example: Asking it to quiz them on something.

Sample Prompt: We’re studying slope in Algebra I in Tennessee, can you give me some quiz questions and tell me how I did?

Example: Asking it to help with the pre-writing phase of writing.

Sample Prompt: I am writing a research paper on Romeo and Juliet and comparing it to other famous family feuds in more recent history. We’re going to the library to do research next week, and I need to get organized. Can you give me a checklist of what I should be searching? Do you know of any feuds I can research?

Example: Asking it to help with brainstorming

Sample Prompt: In US History, our teacher has asked us to explore the causes of war leading up to World War I. We are supposed to represent a country and their point of view. Help us brainstorm some ideas for this. We can’t choose Germany, Britain, France, or the US. We don’t know these other countries as well. What information do you need to help us with this?

Example: Ask it to proofread your paper and give you feedback.

Sample Prompt: I’m writing a paper for my World History class on the Ming Dynasty, but I need someone to proofread it for me. Can you proofread this and give me a list of suggestions for improving it. Please do not rewrite the paper for me; I do not want to get accused of cheating.

How do I stop cheating?

·      Consider whether your assignments are easy for students to cheat on using AI.

·      Get experienced enough with AI that you can spot how it writes.

·      Take a writing sample at the beginning of the year for a comparison.  

·      Let students know that you won’t tolerate them using AI for final products, but you’d love for them to use it for brainstorming, outlining, and pre-writing.

I’d love to have a deeper conversation about this, but I want to be clear that we must tackle this issue head-on, and at some point, we’re all going to have to accept that AI is a technology tool that our students need to know how to use. Just like we teach students to use TI-85 calculators, nail guns, MIG and TIG welders, and 3D printers, we have to expose students to all technologies that will help them be successful in life.

There are many AI tools other than ChatGPT that are meant specifically for the classroom. I’m trying to keep a list of them: https://www.jasonhorne.org/ai-tools

TISA Dashboard

I decided to build a TISA Dashboard to keep track of funding coming in the next year. I wanted to also keep track of numbers that inform the dashboard in order to spot potential data-entry errors.

Click here to access the dashboard.

TN Letter Grades: A Machine Learning Approach

This blog continues the series on TN School Letter Grade data. You can find the first post here. The same dataset applies.

Having looked at some of the preliminary results of statewide school letter-grade data, I wanted to do a machine-learning approach to see the importance of the different features in the data. Typically, this type of approach is used for predictive modeling of a given outcome, but it also gives a lot of insights into the data.

What is Machine Learning?

Machine Learning is an AI approach to data that allows the computer to learn insights about the data to become more accurate at predicting outcomes without being programmed to do so. It spots patterns in data, and the more data it’s exposed to, the better it does. This is why I chose this approach: this type of analysis works best on a large set of data, but when applied to an individual school or school system, it will help with goal setting and interventions. If certain challenges are identified as barriers to higher performance, you can work on targeted interventions to address these issues.

The beauty of machine learning in this context is its ability to handle complex, multifaceted data and reveal insights that might not be immediately apparent through traditional analysis. This can lead to more informed decision-making and, ultimately, better educational outcomes for the students in your school system.

Exploring Machine-Learning Methods

In Machine Learning, the process typically involves considering various algorithms and conducting testing to identify the best approach for the data. In this analysis, I began with a logistic regression model, which initially showed promising results with ROC AUC scores for individual letter grades: A = 0.97, B = 0.82, C = 0.84, D = 0.95, F = 0.99 (see the graph below). However, it's important to note that logistic regression is primarily a binary classifier, and it provided different feature scores for each letter grade individually. While these insights were valuable, I sought a more comprehensive model capable of collectively predicting all letter grades, rather than focusing on each one individually.

ROC AUC Scores for Logistic Regression

Even though I ultimately settled on a different algorithm to look comprehensively at the data, given that the letter grades A and F had such high ROC AUC scores, I thought it would be interesting to look at the coefficients scores for each of those letter grades.

Keep in mind that a positive coefficient for a feature means that as the value of the feature increases, the likelihood or the probability of the predicted outcome also increases. The opposite is true of negative coefficients. As the value of the feature decreases, the likelihood or probability of the predicted outcome decreases. Also, the magnitude of the coefficients matter. Larger coefficients, whether positive or negative, imply a stronger influence of the corresponding feature on the outcome. 

Class 0 (Letter grade of A):
overall_success_rate_all_students: 4.337198187954319
growth_numeracy_score: 2.2796755844815197
growth_literacy_score: 1.685213811073859
growth_social_studies_score: 1.6173875332744483
growth_science_score: 1.5986916515324763
economically_disadvantaged_pct: -1.3345206477116285
limited_english_proficient_pct: -1.1511514311464113
overall_success_rate_ed: 1.0341733483583349
growth_ela_math_score_bhn: 0.8022573064964893
growth_ela_math_score_ed: 0.7036926608187116
growth_ela_math_score_swd: 0.42283743131419366
black_hispanic_native_american_pct: -0.40066877214648894
homeless_pct: -0.4000903731301108
overall_success_rate_el: 0.34087624502095853
military_pct: -0.33136098524287755
overall_success_rate_swd: -0.2607763057899938
african_american_pct: -0.20352688080879153
asian_pct: -0.15963467577880144
multirace_pct: 0.09164345297812754
native_american_pct: 0.08977420050427559
growth_ela_math_score_el: -0.07558107837346213
white_pct: -0.06735814037053171
students_with_disabilities_pct: -0.022687826552016045
male_pct: 0.00627481315433192
migrant_pct: -0.0007504063512127447
Class 4 (Letter grade of F):
overall_success_rate_all_students: -4.321832443606035
overall_success_rate_ed: -2.1215531229812146
growth_science_score: -1.7027841781095818
growth_literacy_score: -1.6491294242431007
growth_numeracy_score: -1.6129511803548933
growth_ela_math_score_bhn: -1.22590960402722
growth_social_studies_score: -1.178614571621578
economically_disadvantaged_pct: 1.1473369934362392
growth_ela_math_score_ed: -0.7225782040444474
limited_english_proficient_pct: 0.4933486617717649
students_with_disabilities_pct: -0.46909851837241345
asian_pct: 0.34961287513000733
homeless_pct: 0.3360394684863425
growth_ela_math_score_swd: -0.3165323640351417
overall_success_rate_swd: 0.3134225518072883
black_hispanic_native_american_pct: 0.28520884553047776
male_pct: -0.13205589211946425
white_pct: -0.12722495610537785
military_pct: -0.09357110355603143
multirace_pct: 0.06537517896094194
growth_ela_math_score_el: 0.060166568067282857
overall_success_rate_el: 0.04748262863384597
african_american_pct: 0.004185611416037893
native_american_pct: 0.0030283931838529874
migrant_pct: -0.0029266625919092625

As you can see, the best predictors for a letter grade of A are overall_success_rate_all_students (4.3372), growth_numeracy_score, (2.2797), growth_literacy_score (1.6852), growth_social_studies_score (1.6174), and growth_science_score (1.5987). Of course, success rate for all students is 50% of the letter grade score and the overall growth score is 40% of the letter grade score. It isn’t surprising to see these in the top 5.

Looking at negative scores can be telling as well. The negative scores with the greatest magnitude for the letter grade of A were economically_disadvantaged_pct (-1.3345) and limited_english_proficient_pct: (-1.1511). This insinuates that having a lower percentage of economically disadvantaged students and students with limited english proficiency were important to scoring a letter grade of A.

For a letter grade of F, we examined the coefficients to identify the most influential factors. The results shed light on the key determinants of a low letter grade. Just as with the letter grade A analysis, we discovered both positive and negative contributors.

The most prominent positive predictor for a letter grade of F was overall_success_rate_all_students, with a coefficient of -4.3218. This indicates that a low overall success rate for all students strongly correlates with a letter grade of F. Additionally, several growth scores had negative coefficients, including growth_science_score (-1.7028), growth_literacy_score (-1.6491), and growth_numeracy_score (-1.6130). These findings imply that poor performance in these growth areas negatively affects the letter grade.

On the other hand, certain factors with positive coefficients slightly mitigated the impact of negative predictors. For instance, economically_disadvantaged_pct had a positive coefficient of 1.1473, suggesting that a lower percentage of economically disadvantaged students was associated with a slightly better letter grade. Limited_english_proficient_pct had a positive coefficient of 0.4933, indicating that a lower percentage of students with limited English proficiency had a positive influence on the letter grade.

I was also curious about false negatives and false positives with logistic regression. The confusion matrix for the logistic regression showed that the model most accurately predicted schools that scored a D at a very high rate (97.5%) compared to the other letter grades. Here's the percentage of correct predictions for each class:

  • Class A: Approximately 81% correct

  • Class B: Roughly 66% correct

  • Class C: About 80% correct

  • Class D: Approximately 97.5% correct, which indicates a high accuracy for this class

  • Class F: Around 61.5% correct

While these are promising results for predicting an A, C, and a D, it does not predict the others with accuracy, which throws off the entire model as a reliable method for looking at this data. This is why other machine learning approaches needed to be explored.

Logistic Regression Confusion Matrix

Finding a better algorithm

To find an algorithm that would predict the letter grade rather than look at each of them individually, I decided to look at the following algorithms: Decision Tree, Random Forest, Gradient Boosting, and Support Vector Machine. I used the same 70/30 training/testing set that was used for Logistic Regression, and I had it compute the Accuracy, ROC AUC, Precision, Recall, and F1 Score.

Here are the results:

Decision Tree showed an accuracy of 66.5%. The model had a Precision of 66.6%, closely matching its accuracy. Recall and F1 Score were both approximately 66.5%, indicating a balanced performance in terms of precision and recall. The ROC AUC scores were strong across the classes, with the highest being for class 4 (98.9%).

Random Forest performed better in terms of accuracy with 76.6%. Precision was notably higher at 77.3%, with a Recall of 76.6% and an F1 Score close behind at 76.3%. The ROC AUC values mirrored those of the Decision Tree, which suggests consistent performance across different thresholds.

Gradient Boosting edged out with an accuracy of 77.4%, the highest among the tested models. It also had the highest Precision at 77.5% and F1 Score at 77.3%. Recall was in line with accuracy at 77.4%. ROC AUC scores were consistent with the other models.

Support Vector Machine had an accuracy of 74.4%. It achieved a Precision of 75.1% and an F1 Score of 74.1%, with a Recall of 74.4%. ROC AUC scores for this model were also similar to the others.

Overall, Gradient Boosting stood out as the most accurate model for this task. Despite the similarity in ROC AUC scores across the models, I considered the balance between all metrics. Gradient Boosting showed the best balance, with the highest scores in Precision, Recall, and F1 Score, indicating its strength in both classifying correctly and maintaining a balance between false positives and false negatives. This balance is crucial for models where both types of errors carry significant weight, such as predicting school grades.

  • Accuracy: Approximately 77.4%, which signifies the proportion of total correct predictions made out of all predictions.

  • ROC AUC: About 93.9%, reflecting the model's ability to distinguish between the classes across different thresholds.

  • Precision: Roughly 77.5%, indicating the model's accuracy when predicting a positive class.

  • Recall: Also about 77.4%, showing the model's capability to identify all actual positives.

  • F1 Score: Approximately 77.3%, which is a harmonic mean of Precision and Recall, providing a single score to measure the model's accuracy.

Gradient Boosting Results

Using the Gradient Boosting algorithm, I did a chart of the top 10 features.

Gradient Boosting top 10 features

Top 10 Features

  1. Overall Success Rate for All Students (Feature importance: 42.91%): This is the most significant predictor, indicating that the overall success rate is strongly associated with the school's letter grade.

  2. Growth in Numeracy Score (14.49%): The second most important feature, which suggests that improvements in numeracy significantly influence the grade.

  3. Growth in ELA and Math Score for ED (6.98%): The progress in English Language Arts and Mathematics for economically disadvantaged students is also a key indicator.

  4. Growth in Science Score (6.40%): Science score growth is another substantial factor.

  5. Growth in Literacy Score (5.68%): Literacy improvements are crucial, although less so than numeracy.

  6. Growth in ELA and Math Score for BHN (5.43%): The growth in ELA and Math for Black, Hispanic, and Native American students is also a notable predictor.

  7. Growth in Social Studies Score (4.39%): This shows a moderate influence on the grade.

  8. Percentage of Economically Disadvantaged Students (2.76%): While this has a smaller weight, it's still a relevant feature.

  9. Percentage of Students with Disabilities (1.93%): This has a lesser impact but is part of the top 10 features.

  10. Overall Success Rate for ED (1.52%): The overall success rate for economically disadvantaged students rounds out the top 10 features.

Conclusions

It turned out to be a happy accident that I ran the logistic regression first because it allows me to look at the coefficients for features for individual grades before I looked at them as coefficients to predict any grade. Doing this really shows that the beyond common sense that overall success rate is going to determine the letter grade, how ED students score on their ELA and math growth score and how BHN students score on their ELA and math growth scores influences the scores. Additionally, how students scored in science and social studies, two subjects no longer included in federal accountability, were also important to the overall letter-grade score. It will be interesting to look at how these coefficients differ when looking at federal accountability that takes improvement and subgroup performance into account when assignment a school score.

Disclaimer

Keep in mind that these scores are literally from one test for each subject and both achievement and growth scores were derived from the one test. This is not the most accurate measure of a student’s knowledge even if that is what is used for accountability for political expediency and not for actually data reliability.

Finally, this or any other analysis isn’t a substitution for doing the right things for students. Building relationships with students, teaching the things that matter in every subject, and helping students develop a love and desire for learning are always going to produce the best results for students no matter what scoring apparatus is used. As The Fox said in the Little Prince: “On ne voit bien qu’avec le coeur; l’essentiel est invisible pour les yeux.”

I had ChatGPT check this for errors and make some writing suggestions.

A preliminary look at TN School Letter Grade Data

Summary

I analyzed the statewide school letter grade data and found that poverty and race significantly impact the letter grades schools receive. The disregard for improvement and the lack of weight given to subgroup performance are evident in the letter grades.

Purpose of this study

Following the release of school letter grades, which I previously discussed here, I delved into the data. This blog is the first in a series where I will cover basics such as distributions, means, and some demographic data. Due to the urgency of disseminating this information, I will rely on basic Python graphs rather than more sophisticated visualizations.

Data Sources

The data files used are available here. I used demographic data from the 2021-2022 academic year as the 2022-2023 data is not yet accessible. The slight variation should not significantly affect the outcomes, but I intend to reanalyze with the new data upon release.

In the state demographic data, categories with less than 5% or more than 95% of students do not disclose actual numbers. For consistency, I substituted these with 2.5% and 97.5%, respectively, even though it may minimally affect correlation calculations.

Out of 1900 schools listed for letter grades, 210 (11.05%) were ineligible. These schools were excluded from this analysis.

Distribution

The letter grades were distributed as follows:

A: 294 (17%), B: 441 (26%), C: 513 (30%), D: 350 (21%), F: 92 (5%)

This distribution resembles a normal curve, skewed slightly with more A's than expected. While a normal curve would have 68% of data in the middle, here we see 77% of schools as B, C, or D grades.

distribution of grades bar chart

The distribution of letter grades across all schools in TN.

What Influences a Grade

I examined the means for each category (Achievement, Growth, and Growth25) for each letter grade to provide insights similar to our understanding of federal accountability.

Grade A:

  • The overall success rate for all students is at an average of 57.34%.

  • The average growth numeracy score stands at 4.45.

  • The average growth literacy score is 4.11.

  • The average letter grade score  is 4.85.

Grade B:

  • The overall success rate for all students averages 43.87%.

  • The average growth numeracy score is 3.72.

  • The average growth literacy score comes in at 3.36.

  • The average letter grade score is 3.90.

Grade C:

  • The overall success rate for all students is around 32.72%.

  • The growth numeracy score averages at 2.94.

  • The growth literacy score averages at 2.92.

  • The average letter grade score is 2.92.

Grade D:

  • The overall success rate for all students averages 22.76%.

  • The average growth numeracy score is 1.88.

  • The average growth literacy score is 2.31.

  • The average letter grade score is 1.95.

Grade F:

  • The overall success rate for all students is the lowest, at an average of 11.67%.

  • The average growth numeracy score is 1.55.

  • The average growth literacy score is 1.91.

  • The average letter grade score is 1.23.

For schools receiving an 'A' on the state report card, the average success rate is 57.34%, indicating that over half of their students achieved proficiency on TN Ready tests. This surpasses the full points threshold for achievement, which is set at 45% for elementary, 42.5% for middle, and 40% for high schools. In stark contrast, schools with an 'F' have only 11.67% of students reaching proficiency. These discrepancies suggest that factors beyond instruction quality, teacher performance, or student effort are at play.

To understand the breadth of these disparities, I analyzed how schools within each Letter Grade Group performed across different categories, all measured on a 5-point scale.

Average Achievement Scores by Letter Grade:

  • A: 4.85

  • B: 3.95

  • C: 2.96

  • D: 2.00

  • F: 1.02

Average Growth Scores by Letter Grade:

  • A: 4.96

  • B: 3.92

  • C: 2.80

  • D: 1.62

  • F: 1.00

Average Growth25 Scores by Letter Grade:

  • A: 4.45

  • B: 3.58

  • C: 3.10

  • D: 2.64

  • F: 3.21

Average Scores by Letter Grade by Category

It is immediately apparent that schools with an 'F' were disproportionately affected by the inability to demonstrate improvement. The Growth25 metric assesses the advancement of the lowest-performing 25% of test-takers, which often reflects the effectiveness of interventions, tutoring, and other targeted efforts aimed at fostering student progress. Although it's encouraging to see this segment of students making gains, such improvements were not factored into the overall grade assessment by the state.

Subgroups

What impact did subgroups have on this data?

Percent of economically disadvantaged students by letter grade.

Economically disadvantaged students' percentage and letter grades are closely related. For instance, "A schools" average 18.34% of economically disadvantaged students, compared to 54.52% in "F schools."

The relationship between these demographics and school grades is pronounced and merits further investigation.

Scatterplot of Economically Disadvantage Numbers and Letter Grade Score

The correlation between the percentage of economically disadvantaged students and letter grade scores (r = -0.50, p < 0.05) indicates a moderate negative relationship, suggesting that schools with a higher percentage of economically disadvantaged students tend to have lower letter grades. This is statistically significant, with a p-value near zero.

Other subgroup data needs to be explored as well. This is what the same reports look like for the BHN (Black, Hispanic, Native American) subgroup.

BHN by letter grade group.

This is even more extreme than the Economically Disadvantaged data in terms of the difference in means. Let’s look at the correlation data.

Scatterplot of BHN versus Letter Grade Score

For the BHN (Black, Hispanic, Native American) subgroup, the correlation (r = -0.37, p < 0.05) also indicates a moderate negative relationship, with race data showing a significant correlation with letter grades.

Finally, what impact did the percentage of Students with Disabilities have on the data.

Students with Disabilities Percentage by Letter Grade Group

It's intriguing to note that schools with 'A' and 'F' grades have almost identical percentages of students with disabilities. This raises questions that I intend to delve into in an upcoming blog post.

Students with Disabilities percentage scatterplot with letter grade score

The percentage of Students with Disabilities (SWD) shows a different trend, with a non-significant correlation (r = -0.09, p > 0.05). This may be due to the fact that this subgroup's score constitutes only 10% of the letter grade score in the Growth25 category.

Conclusions

More detailed analyses will follow, but this preliminary look suggests significant disparities that need to be addressed. The intent is to bring this to light swiftly.

This blog post was edited by ChatGPT.

TN School Letter Grades Explained

Introduction

Whenever you write something, you’re always supposed to have audience in mind. I do not have one in mind for this blog. It can be for educators or laypeople. I just want people to understand what is going on in education in the state of TN, what the implications are for their school, and how to interpret the results that they are seeing for letter grades for their school.

I will not debate the merits of this letter-grade system here. It’s already done, and I already spoke publicly at the public forum offered. In addition, I have signed more than one letter regarding these letter grades, and everything we warned about them has already borne fruit in less than 24 hours. The marks are already out here marking. Having seen that happen is what has inspired me to explain how these work and what they mean.

Disclaimer

I am not going to dumb this down. It isn’t that complex, but it’s more complex than, say, basic sports stats. Laypeople shouldn’t have any problems understanding this, but I just want to ward off any criticism that I’m writing some sort of arcane nonsense. If that’s what it is to you, then that’s just what it is.

Also, to be clear, I am writing this in my capacity as a private citizen, and any opinions here are my own and are not representative of the Campbell County School System or of East Tennessee State University.

Something that needs to be said

  • We weren’t informed that we would be given these grades until this Fall. This is like the teacher telling students that the way their grade is going to be calculated is totally different than what it was all semester.

  • The state has issued accountability protocols with the accountability metrics. We’re used to having these at the beginning of the school year for that year’s data, not for the previous year’s.

How were we “graded” before?

Accountability, agreed to in our ESSA plan, looked at several different factors, including Achievement, Growth, Chronic Absenteeism, Graduation Rate, Ready Graduate Rate, and English Language Proficiency Assessment. You can see how this accountability is laid out below (source).

Those are the weights and measures. The final score is calculated by weighting all students with 60% and students in subgroups (Black/Hispanic/Native American, Students with Disabilities, Economically Disadvantaged, English Language Learner) 40%. This is what a typical federal heat map looks like.

As you can see, this school scored 3.5. This is high enough to be a reward school in years past (anything school with a score greater than 3.2 was designated as a reward school).

You can see they had an “Absolute” and “Target” score. And the state takes the higher of the two. The TDOE completely did away with this. This is why many schools who are used to having high accountability scores are receiving low grades this year. The goalposts have moved.

How we are “graded” now:

This is a much simpler formula, and that might seem like a positive thing at first glance, but once you dig into how all of these metrics are determined, then you really see how much different of a system this is.

Let’s look first at how achievement scores are determined by federal accountability.

This means that schools with 45% or better of their students scoring proficient on the state test or meeting their double AMO (twice the amount set for their Annual Measurable Objective) will receive a score of 4 out of 4 for the achievement category. Let’s look how the state changed this for their accountability.

They spilt it into three grade bands, and they made the elementary and middle achievement higher at 49.5% and 45.4% respectively. The high school achievement score is lower at 40.1%. This is to earn maximum points. This begs the question why didn’t they include the pathway for improvement?

Growth

Growth is measured the same in both accountability systems, however, growth has been interesting in the state of TN since COVID. Because of Shelby County and Davidson County having multiple virtual years, this has really changed growth for smaller counties. Since growth is a metric that’s based on comparing students to their peers, and specifically, it’s comparing how students perform from one year to the next (have fun reading about it here) for grades 3-8 and comparing how students perform based on a predicted score, it’s hard to show growth when over a hundred thousand students in the pool would have had artificially low scores during and right-after COVID. Can you still show growth despite this? Yes, but you would have to really outscore the means from before. This is why despite showing tremendous gains in Achievement, Campbell County schools are still having trouble showing growth. Look at the improvement Campbell County has had in Achievement over the past three years. It is counterintuitive that this did not also turn into growth like it would have in pre-COVID years.

Subgroups

The following groups of students are recognized as historically underserved student groups:

  • BHN (Black/Hispanic/Native American)

  • ED (Economically Disadvantaged)

  • SWD (Students with Disabilities)

  • ELL (English Language Learners)

For federal accountability, we were held accountable for how these groups of students performed. It is a huge part of our planning. As you can see below, they’re weighted for 40% of federal accountability.

Schools that have a high percentage of these students have depended on their performance to have a high score for federal accountability. Repeating myself here, but it’s 40% of the school’s accountability score.

For state accountability, this completely changed. Now, these students only count for 10% of the accountability score, and it’s the growth score for only the students who score in the bottom 25%. If that sentence is confusing, you’re not alone. Having groups of students go from 40% of your accountability to 10% is shocking enough, but using a metric like a bottom quartile also makes this group of students fluid and not easily identified for intervention.

CCR (not the band)

College and Career Ready is a new metric that the state is using to see the percentage of students who are taking advantage of Early Postsecondary Opportunities (EPSOs). This is like the federal metric, Ready Graduate, but it differs slightly, and it’s an easier metric to attain. Here is a chart of how they differ. Also included is a metric for TISA graduates, which is something that triggers additional outcome funding, but it isn’t part of the accountability model.

Conclusion

Hopefully this will help you understand how Letter Grade accountability differs from what schools are used to. Let’s keep in mind that all of this is based on some pretty flawed logic. How a student performs on one test on one day is interesting information for a teacher, but it shouldn’t be used to evaluate a teacher or school or district.

If you want to read more about accountability, I suggest this blog post.

If you want to read a Director of School’s thoughts on all that’s happening here, please read this.

Creating GPTs and the future of EdTech

ChatGPT plus users now have the ability to make their own GPTs. Think of these like personalized ChatGPT bots that have specific parameters. For example, the first one I created is a writing tutor for students in TN. It doesn’t take any special coding to do one; you just chat your way through it with ChatGPT. You can upload documents, and that’s what I did. I found the latest writing rubrics for TN and the anchor papers that are provided. Because ChatGPT doesn’t care about formatting, I wrote a python script to write all of those dense PDFs into one RTF file, and I uploaded it.

I know a lot of teachers are going to fear that students are going to use these programs to cheat, so what I did was tell ChatGPT that I didn’t want it to create writing for students, only to give feedback on it. So instantly, I built a fence around my GPT that I wanted.

People like me are going to be building a ton of these, and eventually, we will probably have the opportunity to monetize these like YouTube Videos or TikToks. And honestly, that’s my plan. I want to built great resources that are specific to TN, and if I make some extra dough on that, good for me. Honestly, building it is its own reward for me.

You can keep track of my GPTs here: http://jasonhorne.org/gpt

Josh Heupel's Penalty Rates

Despite Josh Heupel’s success as a football coach at both the University of Central Florida (UCF) and the University of Tennessee (UT), and also despite perennial gaudy offensive numbers, Josh Heupel’s teams have been among the most penalized in college football over his six years of coaching. I will look at the extent of this, but I won’t get into why this is true; someone else can do that.

I took data from https://www.teamrankings.com/ to look at how Josh Heupel’s team did with penalties at UCF and UT. By comparison, I wanted to see how Gus Malzahn’s UCF teams have done the past three years, and I also wanted to see how Jeremy Pruitt’s UT teams did the previous three years before Heupel became the coach.

Because penalties aren’t the complete picture, I also wanted to look at winning percentage and the number of plays per game. Just taking a look at 6 years of UCF and UT, it’s obvious that penalties are not having any causal effect on winning percentage for a season.

So what is the reason Josh Heupel’s teams commit so many penalties? His teams’ penalty rankings seem to be inversely proportional to number of play rankings. Heupel shows an average ranking of 116.2 (out of 133 teams most years) in terms of penalties per game (a lower ranking meaning more penalties per game). Conversely, he shows an average ranking of 20.7 for plays per game, even having the number 1 ranking in 2020-2021. Because of this, I wanted to see if there was a relationship between plays per game and penalties per game.

I scraped all the penalty per game and plays per game data from https://www.teamrankings.com/ to get the data for all college football teams for both categories. I combined all the data into one data frame that included the average penalties per game, average plays per game, and the rankings for both. To look the relationship, I wanted to see a scatterplot of average penalties per game vs average plays per game, and I also wanted to see the Pearson r correlation coefficient.

scatterplot of penalties v plas

The scatterplot shows that there is a weak relationship between the average penalties and the average plays per game (r = 0.16, p < 0.05). Even though the relationship is weak, it is a statistically significant relationship. Ultimately, the number of penalties are a trend with Heupel-coached teams, and it doesn’t appear that he can blame it on the number of plays his teams has per game.

A deeper analysis of Tennessee’s penalties might show where they are happening. Are they happening more frequently on offense or defense? Are certain players or position groups committing these penalties? Against which teams are the most penalties committed: teams like Alabama and Georgia, or is it teams that aren’t as stiff competition?

The evidence suggests that while Heupel's offensive strategy correlates with a high volume of plays, it does not inherently lead to increased penalties, hinting at other factors at play. The nature of the penalties, their timing, and their distribution among players and game situations are dimensions still to be explored. In-depth examination could offer insights into whether the penalties are a byproduct of aggressive play-calling, lack of discipline, or strategic trade-offs deemed acceptable by the coaching staff.

Understanding the subtleties behind these penalties can be crucial for refining practice strategies and in-game decision-making. It can help in developing targeted coaching interventions to mitigate unnecessary losses while maintaining the aggressive edge that characterizes Heupel's approach. For the University of Tennessee, such insights are not just academic; they could be the key to fine-tuning a powerful offensive engine into a more efficient, disciplined unit that capitalizes on its strengths without succumbing to self-inflicted setbacks.

For now, the data provides a starting point for a more nuanced discussion on the interplay between plays and penalties under Heupel's tenure. Further research may illuminate the path to optimizing performance where it matters most — on the field where every play, and every penalty, can alter the course of the game.

Tennessee TN Ready/EOC Rankings and Average Proficiency

I wanted to see where my county ranked annually with other school systems in the state in terms of TN Ready/EOC proficiency. I also wanted to compare our county with other benchmark districts. Click here to access the Looker Studio.

I built this Looker Studio (formerly Google Data Studio) to illustrate this data. While I was at it, I also built some charts to look at proficiency over time.

I pulled the raw data from the state data downloads page here.

I used Python to concatenate the files from 2018 through 2023. I also used Python to clean the data and separate it into two different files (EOC and TN Ready). I then uploaded those files to Google Sheets for my Looker Studio.

Here is the raw TN Ready Data.
Here is the raw EOC Data.

Update: I added a spot for school ranks and average proficiency and included data up to 2022. We still do not have 2023 data for the entire state, and my local data is currently embargoed.

I will update that when it is public.