Elamur Research⌕
☰
home / papers / 2018 / Few-Shot Charge Prediction with Discriminative Legal Attributes

Few-Shot Charge Prediction with Discriminative Legal Attributes

Summary

The paper addresses charge prediction, which means deciding which criminal charge applies from the fact description of a case. Neural models handle common charges well but fail on two kinds of charges: rare ones with only a handful of training cases, and pairs such as theft and robbery whose facts look nearly identical. The authors manually label 149 charges from Chinese criminal law with 10 legal attributes, such as violence, profit purpose and death. Each case then inherits the attribute labels of its charge. A multi-task model learns to predict attributes and charges together. For each attribute, an attention module picks out the relevant parts of the fact text. These attribute-aware representations are combined with a general fact encoding before the final charge classifier. On three Chinese criminal datasets (Criminal-S, -M, -L), the model beats SVM, CNN, LSTM and Fact-Law Att. baselines. The gains are clearest on macro-F1 and are largest for rare charges, where it improves by over 50% relative. This shows that cheap structured domain knowledge can compensate for missing data.

Key points

  • The charge distribution is heavily skewed. The 10 most frequent charges cover 78.1% of cases, while the 50 rarest together cover under 0.5%, often with about ten cases each.
  • There are 10 discriminative attributes. Each charge–attribute pair is labeled Yes, No or NA, and a case can take only Yes or No. Annotation was done once per charge for 149 charges, not per case.
  • Macro-F1 of the proposed model is 64.9 on Criminal-S, 67.1 on Criminal-M and 73.1 on Criminal-L. The authors report absolute macro-F1 gains of 7.9, 4.4 and 5.2 points over baselines.
  • On Criminal-S, macro-F1 for low-frequency charges (≤10 cases) rises from 32.6 (LSTM-200) to 49.7. Medium-frequency charges gain 5.0 points and high-frequency charges gain 1.9.
  • In the ablation, removing either the attribute attention or the concatenation of attribute-aware representations lowers macro-F1 by at least 4 points. Code and data are available at github.com/thunlp/attribute_charge.

Interesting ideas

  • I1. Label-level rather than instance-level annotation. Experts label a small table of class properties once, and every training example inherits them automatically. This supervision is very cheap and could suit any large label set with a known taxonomy, such as product categories, tariff codes or administrative classes.
  • I2. Shared attributes as a bridge for rare classes. Attributes are common to all classes, so what the model learns about them from frequent classes carries over to rare ones. This borrows from attribute-based zero-shot learning in vision and could help long-tail text classification in general.
  • I3. Attributes that target confusable pairs. Choosing attributes that separate near-duplicate classes (for example, violence for theft versus robbery) puts the key distinguishing factor directly into the model. A similar approach could be used wherever error analysis reveals systematic class confusions.
  • I4. A per-attribute attention head plus an attribute-free encoding. Combining both views keeps general context while adding focused evidence. The attention weights also give a degree of interpretability, since they show which words support each attribute.
  • I5. Evaluation stratified by class frequency. Reporting macro-F1 separately for low-, medium- and high-frequency classes shows gains that accuracy hides. This is a useful habit for any imbalanced benchmark.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing · Law

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 72 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{hu2018few,
  title = {{Few-Shot Charge Prediction with Discriminative Legal Attributes}},
  author = {Hu, Zikun and Li, Xiang and Tu, Cunchao and Liu, Zhiyuan and Sun, Maosong},
  booktitle = {Proceedings of the 27th International Conference on Computational Linguistics},
  year = {2018},
  url = {https://aclanthology.org/C18-1041/}
}

References

39 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. 2013. Label-embedding for attribute-based classification. In Proceedings of CVPR, pages 819–826. find
  2. Baharum Baharudin, Lam Hong Lee, and Khairullah Khan. 2010. A review of machine learning algorithms for text-documents classification. JAIT, 1(1):4–20. find
  3. Antoine Bordes, Sumit Chopra, and Jason Weston. 2014. Question answering with subgraph embeddings. In Proceedings of EMNLP, pages 615–620. find
  4. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. JMLR, 12:2493–2537. find our card
  5. George E Dahl, Dong Yu, Li Deng, and Alex Acero. 2012. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE TASLP, 20(1):30–42. find
  6. Mohamed Elhoseiny, Babak Saleh, and Ahmed Elgammal. 2014. Write a classifier: Zero-shot learning using purely textual descriptions. In Proceedings of ICCV, pages 2584–2591. find
  7. Clement Farabet, Camille Couprie, Laurent Najman, and Yann LeCun. 2013. Learning hierarchical features for scene labeling. IEEE TPAMI, 35(8):1915–1929. find
  8. Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82– 97. find
  9. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  10. Dinesh Jayaraman and Kristen Grauman. 2014. Zero-shot recognition with unreliable attributes. In Proceedings of NIPS, pages 3464–3472. find
  11. Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On using very large target vocabulary for neural machine translation. In Proceedings of ACL, volume 1, pages 1–10. find our card
  12. Daniel Martin Katz, Michael J Bommarito II, and Josh Blackman. 2017. A general approach for predicting the behavior of the supreme court of the united states. PloS one, 12(4):e0174698. find
  13. R Keown. 1980. Mathematical models for legal prediction. Computer/Law Journal, 2:829. find
  14. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of EMNLP, pages 1746–1751. find our card
  15. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. find
  16. Fred Kort. 1957. Predicting supreme court decisions mathematically: A quantitative analysis of the ”right to counsel” cases. American Political Science Review, 51(1):1–12. find
  17. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of NIPS, pages 1097–1105. find
  18. Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. 2014. Attribute-based classification for zero-shot visual object categorization. IEEE TPAMI, 36(3):453–465. find
  19. Wan-Chen Lin, Tsung-Ting Kuo, Tung-Jia Chang, Chueh-An Yen, Chao-Ju Chen, and Shou-de Lin. 2012. Exploiting machine learning models for chinese legal documents labeling, case classification, and sentencing prediction. In Processdings of ROCLING, pages 140–141. find
  20. Chao-Lin Liu and Chwen-Dar Hsieh. 2006. Exploring phrase-based classification of judicial documents for criminal charges in chinese. In Proceedings of ISMIS, pages 681–690. find
  21. Chao-Lin Liu, Cheng-Tsung Chang, and Jim-How Ho. 2004. Case instance generation and refinement for case-based criminal summary judgments in chinese. JISE, pages 783–800. find
  22. Bingfeng Luo, Yansong Feng, Jianbo Xu, Xiang Zhang, and Dongyan Zhao. 2017. Learning to predict charges for criminal cases with legal basis. In Proceedings of EMNLP, pages 2727–2736. find
  23. Ejan Mackaay and Pierre Robillard. 1974. Predicting judicial decisions: The nearest neighbour rule and visual representation of case patterns. Datenverarbeitung im Recht, 3(3/4):302–331. find
  24. Tomáš Mikolov, Anoop Deoras, Daniel Povey, Lukáš Burget, and Jan Černockỳ. 2011. Strategies for training large scale neural network language models. In Proceedings of ASRU workshop, pages 196–201. find
  25. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, pages 3111–3119. find
  26. Stuart S Nagel. 1963. Applying correlation analysis to case prediction. Texas Law Review, 42:1006. find
  27. Tara N Sainath, Abdel-rahman Mohamed, Brian Kingsbury, and Bhuvana Ramabhadran. 2013. Deep convolutional neural networks for lvcsr. In Proceedings of ICASSP, pages 8614–8618. find
  28. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information processing & management, 24(5):513–523. find
  29. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. JMLR, 15(1):1929–1958. find
  30. Octavia Maria Sulea, Marcos Zampieri, Mihaela Vela, and Josef Van Genabith. 2017. Exploring the use of text classi cation in the legal domain. In Proceedings of ASAIL workshop. find
  31. Maosong Sun, Xinxiong Chen, Kaixu Zhang, Zhipeng Guo, and Zhiyuan Liu. 2016. THULAC: An efficient lexical analyzer for chinese. find
  32. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS, pages 3104–3112. find
  33. Johan AK Suykens and Joos Vandewalle. 1999. Least squares support vector machine classifiers. Neural processing letters, 9(3):293–300. find
  34. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, Jen-Hao Rick Chang, et al. 2015. Going deeper with convolutions. In Proceedings of CVPR, pages 1–9. find
  35. Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of EMNLP, pages 1422–1432. find our card
  36. Jonathan J Tompson, Arjun Jain, Yann LeCun, and Christoph Bregler. 2014. Joint training of a convolutional network and a graphical model for human pose estimation. In Proceedings of NIPS, pages 1799–1807. find
  37. Shuang Wu, Sravanthi Bondugula, Florian Luisier, Xiaodan Zhuang, and Pradeep Natarajan. 2014. Zero-shot event detection using multi-modal fusion of weakly supervised concepts. In Proceedings of CVPR, pages 2665– 2672. find
  38. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J Smola, and Eduard H Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of NAACL, pages 1480–1489. find our card
  39. Rowan Zellers and Yejin Choi. 2017. Zero-shot activity recognition with verb attribute induction. In Proceedings of EMNLP, pages 946–958. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hu-2018-few-shot-charge-prediction-discriminative/