Elamur Research
☰
home / papers / 2002 / SMOTE: Synthetic Minority Over-sampling Technique

SMOTE: Synthetic Minority Over-sampling Technique

Written for the Research library (claude-opus-5-5) from the paper text. Original — jair.org/index.php/jair/article/download/10302/24590

Summary

Many real-world classification tasks have heavily skewed class distributions. Examples include fraud detection, oil-spill detection in satellite images and mammogram pixel classification, where the rare class is usually the one that matters. Plain accuracy hides poor minority-class detection, so the authors evaluate with ROC analysis, AUC and the ROC convex hull. They propose SMOTE, which expands the minority class with artificial examples rather than duplicates. Each new example is interpolated between a minority sample and one of its nearest minority-class neighbours in feature space. SMOTE is combined with random under-sampling of the majority class. It is tested with C4.5, Ripper and Naive Bayes on datasets that differ in size and degree of imbalance. Across nearly all settings, the combination dominates under-sampling alone, Ripper's loss-ratio tuning and Naive Bayes prior adjustment in ROC space. The authors argue this works because synthetic points push learners toward broader minority decision regions. Replication tends to make those regions narrower and overfitted. The work is a foundational resampling baseline for imbalanced learning.

Key points

  • SMOTE creates new minority examples along line segments joining existing minority points and their nearest minority neighbours. It is paired with majority-class under-sampling.
  • The baselines are under-sampling only (e.g., Under-C4.5, Under-Ripper), varied loss ratios in Ripper and varied class priors in Naive Bayes. All comparisons use AUC and the ROC convex hull.
  • SMOTE-based classifiers were best in all but 4 of 48 experiments:
    • Naive Bayes won on Pima, the least imbalanced dataset.
    • Under-Ripper beat SMOTE-Ripper on Oil.
    • The curves overlapped on Can.
  • Other datasets mentioned include Satimage and a mammography set (about 98% normal vs. 2% abnormal pixels).
  • Proposed extensions include SMOTE-N, which builds synthetic nominal feature values by majority vote among neighbours. The authors also suggest applying SMOTE to information retrieval with bag-of-words features and feature selection.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

224 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 221 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1613/jair.953

@article{chawla2002smote,
  title = {{SMOTE: Synthetic Minority Over-sampling Technique}},
  author = {Chawla, Nitesh V. and Bowyer, Kevin W. and Hall, Lawrence and Kegelmeyer, W. Philip},
  journal = {Journal of Artificial Intelligence Research},
  year = {2002},
  doi = {10.1613/jair.953},
  url = {https://doi.org/10.1613/jair.953}
}

References

36 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Blake, C., & Merz, C. (1998). UCI Repository of Machine Learning Databases http://www.ics.uci.edu/∼mlearn/∼MLRepository.html. Department of Information and Computer Sciences, University of California, Irvine. ↗
  2. Bradley, A. P. (1997). The Use of the Area Under the ROC Curve in the Evaluation of Machine Learning Algorithms. Pattern Recognition, 30(6), 1145–1159. find
  3. Chawla, N., Bowyer, K., Hall, L., & Kegelmeyer, P. (2000). SMOTE: Synthetic Minority Over-sampling TEchnique. In International Conference of Knowledge Based Computer Systems, pp. 46–57. National Center for Software Technology, Mumbai, India, Allied Press. find
  4. Chawla, N., & Hall, L. (1999). Modifying MUSTAFA to capture salient data. Tech. rep. ISL-99-01, University of South Florida, Computer Science and Eng. Dept. find
  5. Cohen, W. (1995a). Learning to Classify English Text with ILP Methods. In Proceedings of the 5th International Workshop on Inductive Logic Programming, pp. 3–24. Department of Computer Science, Katholieke Universiteit Leuven. find
  6. Cohen, W. W. (1995b). Fast Effective Rule Induction. In Proc. 12th International Conference on Machine Learning, pp. 115–123 Lake Tahoe, CA. Morgan Kaufmann. find
  7. Cohen, W. W., & Singer, Y. (1996). Context-sensitive Learning Methods for Text Categorization. In Frei, H.-P., Harman, D., Schäuble, P., & Wilkinson, R. (Eds.), Proceedings of SIGIR-96, 19th ACM International Conference on Research and Development in Information Retrieval, pp. 307–315 Zürich, CH.… find
  8. Cost, S., & Salzberg, S. (1993). A Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features. Machine Learning, 10 (1), 57–78. find
  9. DeRouin, E., Brown, J., Fausett, L., & Schneider, M. (1991). Neural Network Training on Unequally Represented Classes. In Intellligent Engineering Systems Through Artificial Neural Networks, pp. 135–141 New York. ASME Press. find
  10. Domingos, P. (1999). Metacost: A General Method for Making Classifiers Cost-sensitive. In Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 155–164 San Diego, CA. ACM Press. find
  11. Drummond, C., & Holte, R. (2000). Explicitly Representing Expected Cost: An Alternative to ROC Representation. In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 198–207 Boston. ACM. find
  12. Duda, R., Hart, P., & Stork, D. (2001). Pattern Classification. Wiley-Interscience. find
  13. Dumais, S., Platt, J., Heckerman, D., & Sahami, M. (1998). Inductive Learning Algorithms and Representations for Text Categorization. In Proceedings of the Seventh International Conference on Information and Knowledge Management., pp. 148–155. find
  14. Ezawa, K., J., Singh, M., & Norton, S., W. (1996). Learning Goal Oriented Bayesian Networks for Telecommunications Risk Management. In Proceedings of the International Conference on Machine Learning, ICML-96, pp. 139–147 Bari, Italy. Morgan Kauffman. find
  15. Fawcett, T., & Provost, F. (1996). Combining Data Mining and Machine Learning for Effective User Profile. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, pp. 8–13 Portland, OR. AAAI. find
  16. Ha, T. M., & Bunke, H. (1997). Off-line, Handwritten Numeral Recognition by Perturbation Method. Pattern Analysis and Machine Intelligence, 19/5, 535–539. find
  17. Hall, L., Mohney, B., & Kier, L. (1991). The Electrotopological State: Structure Information at the Atomic Level for Molecular Graphs. Journal of Chemical Information and Computer Science, 31 (76). find
  18. Japkowicz, N. (2000). The Class Imbalance Problem: Significance and Strategies. In Proceedings of the 2000 International Conference on Artificial Intelligence (IC-AI’2000): Special Track on Inductive Learning Las Vegas, Nevada. find
  19. Kubat, M., Holte, R., & Matwin, S. (1998). Machine Learning for the Detection of Oil Spills in Satellite Radar Images. Machine Learning, 30, 195–215. find
  20. Kubat, M., & Matwin, S. (1997). Addressing the Curse of Imbalanced Training Sets: One Sided Selection. In Proceedings of the Fourteenth International Conference on Machine Learning, pp. 179–186 Nashville, Tennesse. Morgan Kaufmann. find
  21. Lee, S. (2000). Noisy Replication in Skewed Binary Classification. Computational Statistics and Data Analysis, 34. find
  22. Lewis, D., & Catlett, J. (1994). Heterogeneous Uncertainity Sampling for Supervised Learning. In Proceedings of the Eleventh International Conference of Machine Learning, pp. 148–156 San Francisco, CA. Morgan Kaufmann. find
  23. Lewis, D., & Ringuette, M. (1994). A Comparison of Two Learning Algorithms for Text Categorization. In Proceedings of SDAIR-94, 3rd Annual Symposium on Document Analysis and Information Retrieval, pp. 81–93. find
  24. Ling, C., & Li, C. (1998). Data Mining for Direct Marketing Problems and Solutions. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining (KDD-98) New York, NY. AAAI Press. find
  25. Mladenić, D., & Grobelnik, M. (1999). Feature Selection for Unbalanced Class Distribution and Naive Bayes. In Proceedings of the 16th International Conference on Machine Learning., pp. 258–267. Morgan Kaufmann. find
  26. O’Rourke, J. (1998). Computational Geometry in C. Cambridge University Press, UK. find
  27. Pazzani, M., Merz, C., Murphy, P., Ali, K., Hume, T., & Brunk, C. (1994). Reducing Misclassification Costs. In Proceedings of the Eleventh International Conference on Machine Learning San Francisco, CA. Morgan Kauffmann. find
  28. Provost, F., & Fawcett, T. (2001). Robust Classification for Imprecise Environments. Machine Learning, 42/3, 203–231. find
  29. Provost, F., Fawcett, T., & Kohavi, R. (1998). The Case Against Accuracy Estimation for Comparing Induction Algorithms. In Proceedings of the Fifteenth International Conference on Machine Learning, pp. 445–453 Madison, WI. Morgan Kauffmann. find
  30. Quinlan, J. (1992). C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA. find our card
  31. Solberg, A., & Solberg, R. (1996). A Large-Scale Evaluation of Features for Automatic Detection of Oil Spills in ERS SAR Images. In International Geoscience and Remote Sensing Symposium, pp. 1484–1486 Lincoln, NE. find
  32. Stanfill, C., & Waltz, D. (1986). Toward Memory-based Reasoning. Communications of the ACM, 29 (12), 1213–1228. find
  33. Swets, J. (1988). Measuring the Accuracy of Diagnostic Systems. Science, 240, 1285–1293. find
  34. Tomek, I. (1976). Two Modifications of CNN. IEEE Transactions on Systems, Man and Cybernetics, 6, 769–772. find
  35. Turney, P. (1996). Cost Sensitive Bibliography. http://ai.iit.nrc.ca/bibiliographies/cost-sensitive.html. van Rijsbergen, C., Harper, D., & Porter, M. (1981). The Selection of Good Search Terms. Information Processing and Management, 17, 77–91. ↗
  36. Woods, K., Doss, C., Bowyer, K., Solka, J., Priebe, C., & Kegelmeyer, P. (1993). Comparative Evaluation of Pattern Recognition Techniques for Detection of Microcalcifications in Mammography. International Journal of Pattern Recognition and Artificial Intelligence, 7(6), 1417–1436. find