📄 Gender Inequality in English Textbooks Around the World: an NLP Approach

Summary

Earlier work on gender bias in English-language school textbooks usually studied one country, or a few countries from the same region. This paper compares textbooks across cultures. The author collected English textbooks for grades 7–9 from 22–23 countries and regions and grouped them into seven cultural spheres, following Huntington's civilizational map. The PDFs were converted to text, using OCR where needed, and cleaned. The text was then cut into short windows. Each window was assigned to a "male" or "female" document according to a paired list of gendered keywords.

Four kinds of evidence were gathered:

• how many windows each gender gets;
• which gender is named first in mixed pairs (firstness);
• TF IDF keyword lists for each gender, checked by counting gendered names, asking LLMs to guess the gender of each list, and measuring distances in GloVe.

Males dominate in count, firstness and names in every sphere. The Latin American sphere shows the smallest gap. The bias in textbooks also matches the gender associations found in general web-scale text. This matters because these books shape how young learners see gender roles.

Key points

• Corpus: middle-school (grades 7–9) English textbooks from 22–23 countries and regions in 7 cultural spheres. Examples include New Horizon (Japan), Honey Comb (India), Super Goal (Saudi Arabia), Streamline English Departure (Ukraine) and Fly High (Uzbekistan).
• Pipeline:
  • text extraction with PyMuPDF (fitz), with Adobe OCR for scanned books;
  • lowercasing, punctuation and stopword removal with NLTK;
  • filtering out non-English text with LangDetect;
  • 100-character segments, sorted using 21 male/female keyword pairs (father/mother, sir/madam, hero/heroine, he/she…).
• Count and firstness tests: both are tested against a fair-coin (p = 0.5) null, with p < 0.01 taken as evidence of inequality. Firstness is only scored when the two words come from the same row of the pair list and are adjacent or separated by a single stopword.
• GloVe check: in GloVe space, "pretty", "baby", "love" and "nurse" sit closer to the female side, and "death" to the male side, by at least 0.03.
• Limitations: only about three countries per sphere, which may hide differences in firstness. Countries that publish their textbook lists are easier to sample, so they are overrepresented.

Interesting ideas

• I1. Firstness only counts pairs that are true counterparts (father/mother, but not mother/son), which removes confounds from asymmetric pairs. This works for any measure of ordering bias.
• I2. Every imbalance is tested against an explicit 50/50 binomial null with a strict threshold, so "more X than Y" becomes a falsifiable claim.
• I3. A blind LLM is asked to tell which of two unlabeled keyword lists belongs to which group, as an outside check that the lists differ meaningfully. The author also notes that memory carried across trials could inflate the results.
• I4. The paper assigns gender by whether a keyword appears somewhere in a text window, which is crude. A window can land in both documents, and pronouns are never tied to specific characters.
• I5. Textbook keyword lists are compared with a general-corpus embedding to see whether the books repeat the wider bias or depart from it.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): Detect such pairs through the UD conj relation instead of a surface window. Long or interrupted coordinations would then be counted correctly, and each decision could be traced to an edge in the tree.
• I2 (learn): Apply the same null-model framing to parser or frame-extractor outputs. For example, test whether the agent role is assigned to male and female entities at chance rates on a balanced control set. This would serve as a bias-oriented negative control.
• I3 (learn): Use LLM teacher tiers as blind discriminators of silver-data slices. Run each trial in a fully isolated session so that leakage does not inflate agreement.
• I4 (frames): Resolve coreference and fill VerbNet roles instead. Gender then attaches to entities, and one can ask what women versus men actually do (agent or patient, which verb classes) rather than which words happen to occur nearby.
• I5 (world): When populating entities and occupations from school texts, compare the attribute distributions against the WordNet/Wikidata priors. This would flag stereotyped defaults before they harden into compiled knowledge.