This thread of posts was triggered by the investigation of (as we discovered, quite opportunistic and useless) “equity audits” providers that Urbana School District 116 (USD116) was planning to hire. As is common, the research forced us to consider the broader context, in particular to study comparative performance of three school districts in metro areas in Central Illinois. The outline of the results are below. Besides USD116, we will look at Champaign Unit 4 School District 4 (CUSD4), and McLean County Unit 5, the locus of our previous installment.
Urbana and Champaign are, of course, the twin towns that are separated by the UIUC campus, but share the county government, bus system and what not. Unit 5 is a school district covering most of the metro Bloomington-Normal area, with the exception of a small carveout in the center of it.
We will start with a general view of the towns hosting the school districts. They have a somewhat similar feel: richer, more liberal than the counties surrounding them, homes to noticeable public universities. So it is all the more interesting to look at where they differ. Here’s what the census tells us.
| census year | Bloomington- Normal | Champaign | Urbana |
| 2020 | 131,416 | 88,302 | 38,300 |
| 2010 | 129,107 | 81,055 | 41,250 |
Population changes in three cities over the
past decade. (The data from the US Census Bureau.)
As we can see, over the past decade, the populations in both Bloomington-Normal, and in Urbana-Champaign as a whole stayed about flat. However, there was a strong disparity within: while Champaign grew, by a lot, Urbana, noticeably, lost people. It does not necessarily mean that Urbana dwellers picked their sticks and moved over to Champaign, – rather that the newcomers, somehow, statistically preferred to settle in Champaign.
There might be several conjectures, as to why, – zoning (perhaps), taxes (not really), culture (just kidding)… Here’s our candidate for the explanatory variable.
In Urbana, school age kids are 12.9% of the population, and in Champaign there are 16.5%: a difference of 28% (the data still from the Census). In other words, the observable difference between Urbana and Champaign, – recall, there is no real cultural or economic distinction between the towns, – is that the former seems to be the place where far fewer people want to raise school age kids.
So let’s take a brief look at the school districts’ performances. Helpfully, the State of Illinois (nudged by the US government) implemented IAR, the Illinois Assessment of Readiness: a uniform assessment program. Here’s what the state says:
From the Illinois Report Card:
The Illinois Assessment of Readiness is a federally required measure of student mastery of the Illinois Learning Standards in English language arts and mathematics in grades 3 through 8 – and their readiness for what’s next.
Same Standards, Same Content. Students, families, and schools will experience essentially no difference in the assessment this year. The Illinois Assessment of Readiness measures the same standards and includes the same high-quality test questions used the last four years. Using the same content and measuring the same standards ensures comparability from year to year – an essential commitment to including growth in our support and accountability system. IAR results are also be used to measure student growth for school and district accountability.
Let’s bracket out the general question of the utility or fairness of any particular system of measurements and tests; Goodhart’s law etc. For the purposes of this story, it is enough to assume (plausibly) that IAR is correlated with the educational outcomes.
So, here’s what the data say (all taken from the Illinois State Board’s report card):
Each block shows the (aggregated – the site has the data split by the grades; they show essentially the same picture throughout) readiness of the students. Right of the zero line, good: students do as expected or better. Left, bad. Each district has two sets of assessments, for Language and Math, and two years, 2021 (top), 2019 (bottom). At the ’19 part of the plot, the statewide average bar is also shown, for calibration (the statewide averages for ’21 are not on the site as of now).
The picture is quite clear here: Unit 5 is doing the best of the three districts, being slightly above the state averages in ’19. Champaign’s District 4 is slightly below the state average. Urbana, way below.
All three districts took a hit from ’19 to ’21 (pandemic, – also the reason no data for ’20 were reported). But if in Unit 5 and District 4 one still has quarter to third of the kids performing adequately, in Urbana the performance collapsed. However one gauges the results, it is obvious that USD116’s handling of the pandemic is catastrophically bad.
Let’s look at another key issue, the performance gap: how well the traditionally underserved (coming from poor, or minority families), or otherwise disadvantaged (kids with disabilities, or kids from immigrant families, where English is not spoken at home) populations fare in our districts.
When one looks at comparisons, there is always a hard and subtle problem of the data representation, often driven by that zero-sum-game perception. The folks from the majority, well-served populations often view any equity work as a redistribution of sorts, where someone else being better off means you are doing worse.
I tried to visualize the gap as a scatter plot: the performance by the core (base) populations versus the performance of the under-served or under-privileged population. We put on the abscissa, the x-axis, the performance of the base population, – i.e. non-poor, non-special ed, white… The y-axis shows how the underserved students fared. As before, the exact methodology of the assessments is not especially relevant here. The uniformity however, is: better educational results will be reflected by better score, whichever the interpretation of the actual numerical value is.
Such scatter plots are well-suited to analyze the inequities. A point on the diagonal means that the performances are the same: no gap at all. Getting further from the diagonal indicates an increased gap; getting closer to it means the gap is closing. Increasing both coordinates is obviously good; decreasing both is unquestionably bad.
One can argue about about the comparative importance of getting to the diagonal, – closing the gap, – versus moving the dot up and right.
From practical experience, very rarely one has to choose between moving towards or away from the diagonal along the yellow lines (the dreaded zero-sum situation when one population is not getting better without making the other worse). Most of the motions happen along the pink lines, and the rate of convergence to (or divergence from) the diagonal is the implicit cusp of contention.
OK, let’s move to the actual data: the plots are shown below. I focused on three populations: students with disabilities (IEP, Individualized Education Program, is the bureaucratese for the kids in special education); low income families (traditionally defined as those qualified for free lunch), and students identified as Black vs. White. For each of the populations there are two plots, one for English Language Arts, and Math: the data, as in the previous batch, come from the IAR.
Each plot has six dots: two red for Urbana, two blue for Champaign and two yellow for Bloomington-Normal. One dot represents the situation in 2019, the other that in 2021 (always, moving left and down: getting worse, thanks to the pandemic). I’ve put arrows connecting the corresponding dots for visual help.
The picture these plots show is grim. All districts are far, far, far from the diagonal, and if they move closer to it from ’19 to ’21, it’s only because the underserved kids’ situations just cannot get any worse. But even in this bleak landscape, USD116 stands out. In terms of Black/White plot, USD116’s performance before the pandemic is worse than Unit 5’s performance during the pandemic for both Black and White students. It is not, again, that Unit 5 is a spectacular success; the district is middling even within the so-so schools of Illinois. It is that USD116 is a spectacular failure, by these metrics.
So, among the three districts we look at, USD116 is clearly the worst performer. That are the underlying reasons for its failures? This is, as they say, an overdetermined problem, so it is useful to take a step or two back, and to look at the issue at larger scale. At the national level, it is impossible not to conclude that the US dramatically underinvests in its teachers. Comparing the ratios of incomes of teachers and, say, doctors, or lawyers, or any other mass profession requiring education in the US and just about any other liberal economically successful country is painful.
And underpaying teachers starts a vicious cycle: uneducated population sees no need in shared knowledge and pressures legislators to cut school funding; colleges cannot rely on solid knowledge among the K-12 graduates set up remedial programs that make easier to ignore school failures; businesses seeing a drying supply of educated workforce look for it abroad, rendering national educational systems less and less important, and so on, and so forth…
Let’s take a look then, where our three districts stand on that front: how they are paying their teachers, – and, while we are at it, their administrators?
Here’s a plot. The data, again, taken from the ISBE site, although, there is a caveat: see the postscript below.
As expected, the better the district educates, the better it pays their teachers. Not dramatically, but very clearly more. (Although, I think, adding $8,500 to Urbana’s current average salary of $54,300 would make a lot of difference to them.)
The plot, however reveals another unexpected (or not), yet clear pattern: the districts that pay their teachers less, and have worse educational outcomes for their students, also pay their administrators more, on average.
I guess that’s enough of material to digest for now.
PostScript
I published a TL;DR version of this post in a Letter to News-Gazette on Jan. 9. During the Board meeting the next day the district administration pushed back on the idea they earn too much.
It seems they were pointing to the fact that in USD116, the 9% retirement contributions are rolled into the staff salaries, effectively making them look 9% higher by comparison with other districts, where this contribution is made separately, and is not showing in the salary data.
If this is the case, the red dot in the plot above would move 9% left and down: still keeping the average administration salaries the largest among the three district we compared ($108,000 in USD116 vs. $99,000 in CUSD4 and $89,000 in Unit 5), but also making the effective teachers salaries in USD116 even smaller.
As I had no solid numbers, I asked the district for clarifications. At that point the things started to turn a bit bizarre. The CFO told me that this is a FIOA case, and bounced it to the superintendent’s assistant who stated they opened a FOIA inquiry on my behalf, and looped in the district’s legal counsel.
I checked the rules. Turned out that the data I was asking should be open by default, and posted on the district’s web site (before they are are sent to the state board, whose site I was using).
I asked why that law is not followed. The district then told me, amazingly, that they just now (or last Fall, – I have two conflicting statements to this effect) discovered that they were reporting the wrong data to the State Board of education for years. (It’s worth noting that prior to filing the compensation data with the State Board, the district is obligated to present it to our Board, and then to the public. If their data were wrong, there should be a public notice telling us about it, and so on and so forth…)
I hope the district administration will come to senses and stop this quite inept attempt at stonewalling. But meanwhile, please consider the salary data as preliminary.