📄 Improving Information Extraction from Wikipedia Texts using Basic English

Summary

The paper asks whether the language style of a source text affects unsupervised extraction of definitional knowledge. The authors pull (concept, be, property) triplets that express IS A or IS relations from three crowd-edited sources: the standard English Wikipedia, its Simple English counterpart, and the Simple English Wiktionary. The two Simple resources follow Basic English guidelines, which mean a restricted vocabulary, short sentences and active voice. Each page is analysed with Freeling 2.2, which does morphology and dependency parsing. The system keeps only sentences whose root is a form of "to be" and that mention the target concept, then reads definitions and properties off the object subtree. Human judges score each triplet as 1, 0.5 or 0 for how well it fits the relation and the concept. The simplified sources give fewer triplets, but those triplets are clearly better: the Simple Wiktionary output is roughly twice as good as the Wikipedia output. Long, detailed encyclopedia articles produce many wrong triplets. The work matters because it shows that choosing a cleaner source can matter as much as improving the extractor itself.

Key points

• The sources are English Wikipedia (about 4.98M articles), Simple Wikipedia (about 115K) and Simple Wiktionary (about 24K). The pipeline is Freeling 2.2 morphological analysis followed by dependency parsing.
• There are three extraction rules on the object of a copular sentence:
  • A noun head gives an IS A triplet.
  • Adjective modifiers of that head give extra IS triplets, e.g. chocolate–be–dark, chocolate–be–brown.
  • A coordinating "and"/"or" head is searched through its conjunct children.
• Evaluation follows Krawczyk et al. (2015), with manual scores of 1, 0.5 or 0 per triplet.
• Simple Wiktionary triplets are about twice as good as Wikipedia ones. Of 15 proper-noun concepts, 7 produced no triplets from any source.
• The authors analyse several error types:
  • Past-tense copulas yield outdated facts ("Germany was pagan").
  • Example constructions are misread as definitions (toy–be–teddy bear).
  • Name variants such as "Willis" vs "Bruce Willis" make the system miss sentences.

Interesting ideas

• I1. When building definitional knowledge, a source written in simplified English trades quantity for precision, so source choice works as a quality filter in its own right.
• I2. Splitting "X is a dark brown colour" into one IS A triplet for the head noun and separate IS triplets for each adjective yields simple, machine-joinable facts. The cost is that some meaning carried by the noun–adjective combination is lost.
• I3. Copular extraction fails in predictable ways: past tense turns history into present fact, and "an example of X is Y" reverses the direction of the relation.
• I4. A coarse three-level human score (good / mediocre / wrong) is a cheap way to evaluate extracted relations. Checking against an existing resource such as ConceptNet could auto-approve the overlapping triplets.
• I5. Matching a concept only by its exact string misses aliases, which hurts most for people and named entities.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (world): Seed the WordNet/Wikidata-backed ontology with IS A facts from Simple Wiktionary/Simple Wikipedia first, and treat full Wikipedia only as a lower-trust supplement.
• I2 (frames): Copular frames could emit both the head-type link and property links, while keeping a pointer to the full noun phrase so the lost combination can be recovered when needed.
• I3 (world): Tag copular facts with tense and time anchors so that past states do not overwrite current ones, and add a repair rule that detects example/type constructions before asserting class membership.
• I4 (learn): Use agreement with ConceptNet/WordNet as an automatic witness for extracted facts, and reserve human or LLM-teacher grading for the remaining unmatched ones.
• I5 (world): Resolve entity mentions through Wikidata alias lists or coreference before fact extraction, so that "Willis" and "Walter Bruce Willis" contribute to the same entity.