Elamur Research⌕
☰
home / papers / 2016 / Predicting judicial decisions of the European Court of Human Rights: a Natural Language Processing perspective

Predicting judicial decisions of the European Court of Human Rights: a Natural Language Processing perspective

Paper · 2016 · Nikolaos Aletras, Dimitrios Tsarapatsanis, Daniel Preoţiuc-Pietro et al.

Summary

The paper asks whether the outcome of a case before the European Court of Human Rights (ECtHR) can be forecast from text alone. The authors treat this as binary classification: given text from a judgment, decide whether the Court found a violation of a particular Article of the Convention. The cases come from the public HUDOC database. Judgments are split into their standard parts, such as facts, applicable law and the parties' arguments. Each part is represented with N-grams and with topics, and linear-kernel Support Vector Machine (SVM) classifiers are trained on these features. The models reach about 79% accuracy on average. The factual part of a judgment turns out to be the strongest single predictor. The authors read this as support for legal realism, the view that judges respond mainly to the facts rather than to abstract legal rules. Topics also carry real signal, and the authors study them qualitatively. The work matters as an early systematic attempt at text-based outcome prediction for a major international court, and as a possible aid for triaging the Court's large backlog.

Key points

  • Task: binary prediction of "violation" vs. "no violation" of a Convention Article, using textual content only.
  • Features and model: N-grams and topic features, with linear-kernel SVM classifiers.
  • Result: average accuracy of about 79%. The facts section is the most informative part of the case.
  • Data: HUDOC case-law database, with the authors' ECHR dataset released on figshare.
    • Only cases that passed the admissibility stage are covered, because rejected applications are not published.
    • For scale, in 2015 roughly 78,700 applications were dismissed by single judges, against 891 judgments on the merits.
  • Topic analysis for Article 8: topics were manually labelled and ranked by SVM weight.
    • Themes such as death and military action, or judicial procedure, lean towards violation.
    • Themes such as migration/residence-permit cases and social-care disputes lean towards no violation.
    • Errors mostly come from cases whose features look like those of the opposite class.

Interesting ideas

  • I1. Section-wise ablation of structured documents: train separate models on each part of a document (facts, law, arguments) to see which part drives predictions. This works for any genre with a fixed layout, such as patents, contracts or parliamentary records.
  • I2. Linear models for interpretability: a linear SVM's weights can be read directly, so the most predictive N-grams and topics can be listed for each class. This is a cheap way to turn a classifier into an exploratory analysis tool for domain experts.
  • I3. Manually labelled topics ranked by weight: clustering words into topics, naming them by hand and sorting them by classifier weight gives a readable picture of what pushes decisions each way. This is useful when results must be explained to non-NLP colleagues.
  • I4. Proxy data with stated assumptions: published judgments stand in for unavailable inputs (applications and briefs), and the authors explicitly frame their conclusions as conditional on the two being similar. This is a good model for honest reporting when only downstream data is accessible.
  • I5. Linking NLP results to a domain theory: the authors connect which feature group works best to a substantive debate (legal model vs. legal realism). Prediction experiments can test domain hypotheses in the same way, not only chase accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PennsylvaniaPhiladelphia · United States233 works · 65,583 citations🎓University College LondonLondon · United Kingdom107 works · 22,924 citations🎓University of SheffieldSheffield · United Kingdom117 works · 22,645 citations🎓University of LondonLondon · United Kingdom13 works · 1,896 citations

Also on the byline: Amazon (United Kingdom).

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksGeneral Social Sciences · 254 worksPolitical Science and International Relations · 206 worksLaw · 84 works

🧭 Topics 1Research subjects and languages the work is about.Legal AI · 150 works

Cite

Cite as Nikolaos Aletras, Dimitrios Tsarapatsanis, Daniel Preoţiuc-Pietro and Vasileios Lampos. 2016. Predicting judicial decisions of the European Court of Human Rights: a Natural Language Processing perspective. PeerJ Computer Science.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.7717/peerj-cs.93

@inproceedings{aletras2016predicting,
  title = {{Predicting judicial decisions of the European Court of Human Rights: a Natural Language Processing perspective}},
  author = {Nikolaos Aletras, Dimitrios Tsarapatsanis, Daniel Preoţiuc-Pietro and Vasileios Lampos},
  booktitle = {PeerJ Computer Science},
  year = {2016},
  doi = {10.7717/peerj-cs.93},
  url = {https://doi.org/10.7717/peerj-cs.93}
}

References

40 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bamman D, Eisenstein J, Schnoebelen T. 2014. Gender identity and lexical variation in social media. Journal of Sociolinguistics 18(2):135–160 DOI 10.1111/josl.12080. ↗
  2. Baum L. 2009. The puzzle of judicial behavior. University of Michigan Press. find
  3. Chang Y-W, Lin C-J. 2008. Feature ranking using linear SVM. In: WCCI causation and prediction challenge, 53–64. European Court of Human Rights. 2015. Factsheet on life imprisonment. Strasbourg: European Court of Human Rights. Available at http://www.echr.coe.int/ Documents/ FS_Life_sentences_ENG.pdf . Greer SC. 2000. The margin of appreciation: interpretation and discretion under the European Convention on Human Rights, vol. 17. Council of Europe. ↗
  4. Grey TC. 1983. Langdell’s orthodoxy. University of Pittsburgh Law Review 45:1–949. find
  5. Helfer LR. 2008. Redesigning the European Court of Human Rights: embeddedness as a deep structural principle of the European human rights regime. European Journal of International Law 19(1):125–159 DOI 10.1093/ejil/chn004. ↗
  6. Joachims T. 2002. Learning to classify text using support vector machines: methods, theory and algorithms. Kluwer Academic Publishers. find
  7. Kennedy D. 1973. Legal formality. The Journal of Legal Studies 2(2):351–398 DOI 10.1086/467502. ↗
  8. Keown R. 1980. Mathematical models for legal prediction. Computer/LJ 2:829. find
  9. Kiestra LR. 2014. The impact of the European Convention on Human Rights on private international law. Springer. ↗
  10. Kort F. 1957. Predicting Supreme Court decisions mathematically: a quantitative analysis of the ‘‘right to counsel’’ cases. American Political Science Review 51(01):1–12 DOI 10.2307/1951767. ↗
  11. Lampos V, Aletras N, Preoţiuc-Pietro D, Cohn T. 2014. Predicting and characterising user impact on Twitter. In: Proceedings of the 14th conference of the European Chapter of the Association for Computational Linguistics, 405–413. find
  12. Lampos V, Cristianini N. 2012. Nowcasting events from the social web with statistical learning. ACM Transactions on Intelligent Systems and Technology 3(4):72:1–72:22. ↗
  13. Lauderdale BE, Clark TS. 2012. The Supreme Court’s many median justices. American Political Science Review 106(04):847–866 DOI 10.1017/S0003055412000469. ↗
  14. Lauderdale BE, Clark TS. 2014. Scaling politically meaningful dimensions using texts and votes. American Journal of Political Science 58(3):754–771 DOI 10.1111/ajps.12085. ↗
  15. Lawlor RC. 1963. What computers can do: analysis and prediction of judicial decisions. American Bar Association Journal 49:337–344. find
  16. Leach P. 2013. Taking a case to the European Court of Human Rights. Oxford: Oxford University Press. find
  17. Leach P, Paraskeva C, Uelac G. 2010. Human rights fact-finding. The European Court of Human Rights at a crossroads. Netherlands Quarterly of Human Rights 28(1):41–77. find
  18. Leiter B. 2007. Naturalizing Jurisprudence: essays on American legal realism and naturalism in legal philosophy. Oxford: Oxford University Press. find
  19. Leiter B. 2010. Legal formalism and legal realism: what is the issue? Legal Theory 16(2):111–133 DOI 10.1017/S1352325210000121. ↗
  20. Llewellyn KN. 1996. The common law tradition: deciding appeals. William S. Hein & Co., Inc.. find
  21. Miles TJ, Sunstein CR. 2008. The new legal realism. The University of Chicago Law Review 75(2):831–851. find
  22. Nagel SS. 1963. Applying correlation analysis to case prediction. Texas Law Review 42:1006. find
  23. Pildes RH. 1999. Forms of formalism. The University of Chicago Law Review 66(3):607–621 DOI 10.2307/1600419. ↗
  24. Popple J. 1996. A pragmatic legal expert system. Applied Legal Philosophy Series, Dartmouth (Ashgate), Aldershot. find
  25. Posner RA. 1986. Legal formalism, legal realism, and the interpretation of statutes and the constitution. Case Western Reserve Law Review 37:179–217. find
  26. Pound R. 1908. Mechanical jurisprudence. Columbia Law Review 8(8):605–623. find
  27. Preoţiuc-Pietro D, Lampos V, Aletras N. 2015. An analysis of the user occupational class through Twitter content. In: Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th international joint conference on natural language processing (Volume 1: Long Papers). 1754–1764. find
  28. Preoţiuc-Pietro D, Volkova S, Lampos V, Bachrach Y, Aletras N. 2015. Studying user income through language, behaviour and affect in social media. PLoS ONE 10(9):1–17 DOI 10.1371/journal.pone.0138717. ↗
  29. Priest GL, Klein B. 1984. The selection of disputes for litigation. The Journal of Legal Studies 13(1):1–55 DOI 10.1086/467732. ↗
  30. Salton G, McGill MJ. 1986. Introduction to modern information retrieval. New York: McGraw-Hill, Inc. find
  31. Salton G, Wong A, Yang C-S. 1975. A vector space model for automatic indexing. Communications of the ACM 18(11):613–620 DOI 10.1145/361219.361220. ↗
  32. Schauer F. 1998. Prediction and particularity. Boston University Law Review 78:773. find
  33. Segal JA. 1984. Predicting Supreme Court cases probabilistically: the search and seizure cases, 1962–1981. American Political Science Review 78(04):891–900 DOI 10.2307/1955796. ↗
  34. Segal JA, Spaeth HJ. 2002. The Supreme Court and the attitudinal model revisited. Cambridge: Cambridge University Press. ↗
  35. Sim Y, Routledge BR, Smith NA. 2015. The utility of text: the case of Amicus briefs and the Supreme Court. In: Twenty-Ninth AAAI conference on artificial intelligence. find
  36. Tamanaha BZ. 2009. Beyond the formalist-realist divide: the role of politics in judging. Princeton: Princeton University Press. ↗
  37. Tsarapatsanis D. 2015. The margin of appreciation doctrine: a low-level institutional view. Legal Studies 35(4):675–697 DOI 10.1111/lest.12089. ↗
  38. Turney PD, Pantel P. 2010. From frequency to meaning: vector space models of semantics. Journal of Artificial Intelligence Research 37:141–188. ↗
  39. Vapnik VN. 1998. Statistical learning theory. New York: Wiley. von Luxburg U. 2007. A tutorial on spectral clustering. Statistics and Computing 17(4):395–416. find
  40. Wang S, Manning CD. 2012. Baselines and bigrams: simple, good sentiment and topic classification. In: Proceedings of the 50th annual meeting of the Association for Computational Linguistics: short papers-Volume 2. 90–94.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aletras-2016-predicting-judicial-decisions-european/