Elamur Research⌕
☰
home / papers / 2021 / I-BERT: Integer-only BERT Quantization

I-BERT: Integer-only BERT Quantization

Summary

Large Transformer encoders such as BERT and RoBERTa are costly to run, especially on edge hardware. Earlier quantization work for these models was only partly integer: it still converted values back to floating point for GELU, Softmax and LayerNorm. That prevents deployment on integer-only processors (e.g., ARM Cortex-M cores) and wastes the fast integer units on GPUs. I-BERT removes floating point from inference entirely. GELU and Softmax are replaced by low-cost second-order polynomial approximations that can be evaluated with integers. LayerNorm uses a classic integer square-root algorithm. Embeddings and MatMuls run as INT8 multiplications with INT32 accumulation, the nonlinearities operate on the INT32 results, and outputs are re-quantized to INT8. The authors applied this to RoBERTa-Base/Large and evaluated on GLUE. Accuracy matched or slightly exceeded the FP32 baselines. On a T4 GPU with TensorRT, inference was 2.4–4.0× faster. The work matters because it shows that full Transformer inference can run on hardware with no floating-point support. The PyTorch code is open-sourced.

Key points

  • Approximation error: the integer-only kernels reach a maximum error of 1.8 × 10⁻² for GELU (i-GELU) and 1.9 × 10⁻³ for Softmax.
  • GLUE dev accuracy:
    • RoBERTa-Base averages rose from 86.0 (FP32) to 86.3 (INT8, integer-only).
    • RoBERTa-Large averages rose from 89.0 to 89.5.
    • The largest per-task gains were on CoLA and RTE.
  • Latency: with TensorRT on a T4 GPU, INT8 was on average 3.08× faster than FP32 for BERT-Base and 3.56× for BERT-Large, peaking at 4.00×. This was measured at sequence lengths 128/256 and batch sizes 1–8.
  • Ablation on RoBERTa-Large (QNLI, SST-2, MRPC, RTE):
    • i-GELU averaged 92.7.
    • Exact floating-point GELU averaged 92.3.
    • The h-GELU integer approximation averaged 92.0, dropping up to 1.1 on RTE.

Interesting ideas

  • I1. Polynomial stand-ins for nonlinearities. Fitting a low-degree polynomial to a nonlinear function over its relevant input range lets it run with plain integer arithmetic. This applies to any model that must target accelerators or microcontrollers without an FPU.
  • I2. Integer pipeline end to end. Values are multiplied in INT8, accumulated in INT32, processed by nonlinearities at that higher precision, and only then squeezed back to INT8. Precision is kept where it matters without ever dequantizing, which is a reusable pattern for designing quantized pipelines.
  • I3. Reuse number-theory algorithms for awkward operations. LayerNorm's dynamic square root is computed with a known integer square-root method rather than being kept in floating point. Classical integer algorithms can often unblock operations that seem to require floats.
  • I4. Judge approximation tightness by downstream accuracy. Comparing i-GELU, h-GELU and exact GELU on actual tasks shows that a looser approximation costs measurable accuracy. Kernel error alone does not reveal this, so it is a good evaluation habit for any approximate component.
  • I5. Report speedups over a grid of settings. Speedups are reported across sequence lengths and batch sizes rather than at a single point. This gives a more honest picture of deployment gains and is worth copying when benchmarking compressed models.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing · Computing & GPUs

Cited by

105 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 99 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2101.01321 · arXiv 2101.01321

@misc{kim2021i,
  title = {{I-BERT: Integer-only BERT Quantization}},
  author = {Kim, Sehoon and Gholami, Amir and Yao, Zhewei and Mahoney, Michael W. and Keutzer, Kurt},
  year = {2021},
  doi = {10.48550/arxiv.2101.01321},
  eprint = {2101.01321},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2101.01321}
}

References

96 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. ARM. Cortex-M, https://developer.arm.com/ip-products/processors/cortex-m, 2020. ↗
  2. Ba, J. L., Kiros, J. R., and Hinton, G. E. Layer normalization. arXiv preprint arXiv:1607.06450, 2016. find our card
  3. Bai, H., Zhang, W., Hou, L., Shang, L., Jin, J., Jiang, X., Liu, Q., Lyu, M., and King, I. Binarybert: Pushing the limit of bert quantization. arXiv preprint arXiv:2012.15701, 2020. find
  4. Bhandare, A., Sripathi, V., Karkada, D., Menon, V., Choi, S., Datta, K., and Saletore, V. Efficient 8-bit quantization of transformer neural machine language translation model. arXiv preprint arXiv:1906.00532, 2019. — [arXiv:1906.00532](https://arxiv.org/abs/1906.00532) find
  5. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. find
  6. Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., and Specia, L. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055, 2017. find
  7. Chen, T., Moreau, T., Jiang, Z., Zheng, L., Yan, E., Shen, H., Cowan, M., Wang, L., Hu, Y., Ceze, L., et al. TVM: An automated end-to-end optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pp. 578–594, 2018. find our card
  8. Choi, J., Wang, Z., Venkataramani, S., Chuang, P. I.-J., Srinivasan, V., and Gopalakrishnan, K. PACT: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085, 2018. — [arXiv:1805.06085](https://arxiv.org/abs/1805.06085) find
  9. Courbariaux, M., Bengio, Y., and David, J.-P. BinaryConnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems, pp. 3123–3131, 2015. find
  10. Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., and Bengio, Y. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830, 2016. find
  11. Crandall, R. and Pomerance, C. B. Prime numbers: a computational perspective, volume 182. Springer Science & Business Media, 2006. find
  12. Dagan, I., Glickman, O., and Magnini, B. The pascal recognising textual entailment challenge. In Machine Learning Challenges Workshop, pp. 177–190. Springer, 2005. find
  13. Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Kaiser, Ł. Universal transformers. arXiv preprint arXiv:1807.03819, 2018. find
  14. Detrey, J. and de Dinechin, F. A parameterized floating-point exponential function for fpgas. In Proceedings. 2005 IEEE International Conference on Field-Programmable Technology, 2005., pp. 27–34. find
  15. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. find
  16. Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., and Smith, N. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305, 2020. find
  17. Dolan, W. B. and Brockett, C. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005. find our card
  18. Dong, Z., Yao, Z., Gholami, A., Mahoney, M. W., and Keutzer, K. HAWQ: Hessian aware quantization of neural networks with mixed-precision. In Proceedings of the IEEE International Conference on Computer Vision, pp. 293–302, 2019. find
  19. Esser, S. K., McKinstry, J. L., Bablani, D., Appuswamy, R., and Modha, D. S. Learned step size quantization. arXiv preprint arXiv:1902.08153, 2019. — [arXiv:1902.08153](https://arxiv.org/abs/1902.08153) find
  20. Fan, A., Grave, E., and Joulin, A. Reducing transformer depth on demand with structured dropout. arXiv preprint arXiv:1909.11556, 2019. find our card
  21. Fan, A., Stock, P., Graham, B., Grave, E., Gribonval, R., Jegou, H., and Joulin, A. Training with quantization noise for extreme fixed-point compression. arXiv preprint arXiv:2004.07320, 2020. — [arXiv:2004.07320](https://arxiv.org/abs/2004.07320) find
  22. Gholami, A., Kwon, K., Wu, B., Tai, Z., Yue, X., Jin, P., Zhao, S., and Keutzer, K. SqueezeNext: Hardware-aware neural network design. Workshop paper in CVPR, 2018. find
  23. Gordon, M. A., Duh, K., and Andrews, N. Compressing bert: Studying the effects of weight pruning on transfer learning. arXiv preprint arXiv:2002.08307, 2020. find our card
  24. Han, S. and Dally, B. Efficient methods and hardware for deep learning. University Lecture, 2017. find
  25. Han, S., Pool, J., Tran, J., and Dally, W. Learning both weights and connections for efficient neural network. In Advances in neural information processing systems, pp. 1135–1143, 2015. find
  26. Hauser, J. W. and Purdy, C. N. Approximating functions for embedded and asic applications. In Proceedings of the 44th IEEE 2001 Midwest Symposium on Circuits and Systems. find
  27. MWSCAS 2001 (Cat. No. 01CH37257), volume 1, pp. 478–481. find
  28. Hendrycks, D. and Gimpel, K. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415, 2016. find
  29. Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. Workshop paper in NIPS, 2014. find our card
  30. Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., et al. Searching for MobilenetV3. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1314–1324, 2019. find
  31. Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., and Keutzer, K. SqueezeNet: Alexnet-level accuracy with 50x fewer parameters and\< 0.5 mb model size. arXiv preprint arXiv:1602.07360, 2016. — [arXiv:1602.07360](https://arxiv.org/abs/1602.07360) find
  32. Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015. find
  33. Iyer, S., Dandekar, N., and Csernai, K. First quora dataset release: Question pairs.(2017). URL https://data.quora. com/First-Quora-Dataset-Release-Question-Pairs, 2017. ↗
  34. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2704–2713, 2018. find
  35. Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351, 2019. find
  36. Jin, J., Liang, C., Wu, T., Zou, L., and Gan, Z. Kdlsq-bert: A quantized bert combining knowledge distillation with learned step size quantization. arXiv preprint arXiv:2101.05938, 2021. — [arXiv:2101.05938](https://arxiv.org/abs/2101.05938) find
  37. Kim, S. https://github.com/kssteven418/i-bert, 2021. ↗
  38. Krishnamoorthi, R. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018. — [arXiv:1806.08342](https://arxiv.org/abs/1806.08342) find
  39. Kwon, K., Amid, A., Gholami, A., Wu, B., Asanovic, K., and Keutzer, K. Co-design of deep neural nets and neural net accelerators for embedded vision applications. In 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC), pp. 1–6. find
  40. Lai, L., Suda, N., and Chandra, V. CMSIS-NN: Efficient neural network kernels for arm cortex-m cpus. arXiv preprint arXiv:1801.06601, 2018. — [arXiv:1801.06601](https://arxiv.org/abs/1801.06601) find
  41. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942, 2019. find
  42. LeCun, Y., Denker, J. S., and Solla, S. A. Optimal brain damage. In Advances in neural information processing systems, pp. 598–605, 1990. find
  43. Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., and Chen, Z. GShard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668, 2020. find
  44. Levesque, H., Davis, E., and Morgenstern, L. The winograd schema challenge. In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning. Citeseer, 2012. find
  45. Li, F., Zhang, B., and Liu, B. Ternary weight networks. arXiv preprint arXiv:1605.04711, 2016a. — [arXiv:1605.04711](https://arxiv.org/abs/1605.04711) find
  46. Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H. P. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710, 2016b. — [arXiv:1608.08710](https://arxiv.org/abs/1608.08710) find
  47. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. find
  48. Mao, H., Han, S., Pool, J., Li, W., Liu, X., Wang, Y., and Dally, W. J. Exploring the regularity of sparse structure in convolutional neural networks. Workshop paper in CVPR, 2017. find
  49. Mao, Y., Wang, Y., Wu, C., Zhang, C., Wang, Y., Yang, Y., Zhang, Q., Tong, Y., and Bai, J. Ladabert: Lightweight adaptation of bert through hybrid model compression. arXiv preprint arXiv:2004.04124, 2020. find
  50. Michel, P., Levy, O., and Neubig, G. Are sixteen heads really better than one? arXiv preprint arXiv:1905.10650, 2019. find our card
  51. Mishra, A. and Marr, D. Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy. arXiv preprint arXiv:1711.05852, 2017. — [arXiv:1711.05852](https://arxiv.org/abs/1711.05852) find
  52. Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440, 2016. — [arXiv:1611.06440](https://arxiv.org/abs/1611.06440) find
  53. Mukherjee, P., Weill, E., Taneja, R., Onofrio, D., Ko, Y.-J., and Sharma, S. Real-time natural language understanding with bert using tensorrt, hhttps://developer.nvidia.com/blog/nlu-with-tensorrt-bert/, 2019. find
  54. NVIDIA. TensorRT: https://developer.nvidia.com/tensorrt, 2018. ↗
  55. Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., and Auli, M. FairSeq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations, 2019. find our card
  56. Park, E., Yoo, S., and Vajda, P. Value-aware quantization for training and inference of neural networks. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 580–595, 2018. find
  57. Polino, A., Pascanu, R., and Alistarh, D. Model compression via distillation and quantization. arXiv preprint arXiv:1802.05668, 2018. — [arXiv:1802.05668](https://arxiv.org/abs/1802.05668) find
  58. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. Improving language understanding by generative pre-training, 2018. find
  59. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. find
  60. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019. find
  61. Raganato, A., Scherrer, Y., and Tiedemann, J. Fixed encoder self-attention patterns in transformer-based machine translation. arXiv preprint arXiv:2002.10260, 2020. find
  62. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. SQuAD: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250, 2016. find our card
  63. Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. XNOR-Net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision, pp. 525–542. Springer, 2016. find
  64. Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., and Bengio, Y. FitNets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014. find
  65. Rosset, C. Turing-NLG: A 17-billion-parameter language model by microsoft. Microsoft Blog, 2019. find
  66. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. MobilenetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510– 4520, 2018. find
  67. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019. find
  68. Sanh, V., Wolf, T., and Rush, A. M. Movement pruning: Adaptive sparsity by fine-tuning. arXiv preprint arXiv:2005.07683, 2020. — [arXiv:2005.07683](https://arxiv.org/abs/2005.07683) find
  69. Schraudolph, N. N. A fast, compact approximation of the exponential function. Neural Computation, 11(4):853– 862, 1999. find
  70. Shen, S., Dong, Z., Ye, J., Ma, L., Yao, Z., Gholami, A., Mahoney, M. W., and Keutzer, K. Q-BERT: Hessian based ultra low precision quantization of bert. In AAAI, pp. 8815–8821, 2020. find
  71. Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., and Catanzaro, B. Megatron-LM: Training multi-billion parameter language models using gpu model parallelism. arXiv preprint arXiv:1909.08053, 2019. find
  72. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp. 1631–1642, 2013. find
  73. Stewart, G. W. Afternotes on numerical analysis. SIAM, 1996. find
  74. Sun, S., Cheng, Y., Gan, Z., and Liu, J. Patient knowledge distillation for bert model compression. arXiv preprint arXiv:1908.09355, 2019. find our card
  75. Sun, Z., Yu, H., Song, X., Liu, R., Yang, Y., and Zhou, D. Mobilebert: a compact task-agnostic bert for resource-limited devices. arXiv preprint arXiv:2004.02984, 2020. find our card
  76. Tan, M. and Le, Q. V. EfficientNet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946, 2019. find our card
  77. Tang, R., Lu, Y., Liu, L., Mou, L., Vechtomova, O., and Lin, J. Distilling task-specific knowledge from bert into simple neural networks. arXiv preprint arXiv:1903.12136, 2019. — [arXiv:1903.12136](https://arxiv.org/abs/1903.12136) find
  78. Thomas, J. W., Okada, J. P., Markstein, P., and Li, R.-C. The libm library and floatingpoint arithmetic in hp-ux for itanium-based systems. Technical report, Technical report, Hewlett-Packard Company, Palo Alto, CA, USA, 2004. find
  79. Turc, I., Chang, M.-W., Lee, K., and Toutanova, K. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962, 2019. find our card
  80. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Attention is all you need. In Advances in neural information processing systems, pp. 5998–6008, 2017. find
  81. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. R. GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018. find our card
  82. Wang, K., Liu, Z., Lin, Y., Lin, J., and Han, S. HAQ: Hardware-aware automated quantization. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2019. find
  83. Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., and Zhou, M. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957, 2020. find
  84. Waring, E. Vii. problems concerning interpolations. Philosophical transactions of the royal society of London, 1779. Warstadt, A., Singh, A., and Bowman, S. R. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641, 2019. find
  85. Williams, A., Nangia, N., and Bowman, S. R. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426, 2017. find
  86. Wu, B., Wang, Y., Zhang, P., Tian, Y., Vajda, P., and Keutzer, K. Mixed precision quantization of convnets via differentiable neural architecture search. arXiv preprint arXiv:1812.00090, 2018. — [arXiv:1812.00090](https://arxiv.org/abs/1812.00090) find
  87. Wu, J., Leng, C., Wang, Y., Hu, Q., and Cheng, J. Quantized convolutional neural networks for mobile devices. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4820–4828, 2016. find
  88. Xu, C., Zhou, W., Ge, T., Wei, F., and Zhou, M. Bert-of-theseus: Compressing bert by progressive module replacing. arXiv preprint arXiv:2002.02925, 2020. find
  89. Yang, T.-J., Chen, Y.-H., and Sze, V. Designing energy-efficient convolutional neural networks using energy-aware pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5687– 5695, 2017. find
  90. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R., and Le, Q. V. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems, pp. 5753–5763, 2019. find our card
  91. Yao, Z., Dong, Z., Zheng, Z., Gholami, A., Yu, J., Tan, E., Wang, L., Huang, Q., Wang, Y., Mahoney, M. W., and Keutzer, K. HAWQV3: Dyadic neural network quantization. arXiv preprint arXiv:2011.10680, 2020. find
  92. Zadeh, A. H., Edo, I., Awad, O. M., and Moshovos, A. Gobo: Quantizing attention-based nlp models for low latency and energy efficient inference. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 811–824. find
  93. Zafrir, O., Boudoukh, G., Izsak, P., and Wasserblat, M. Q8BERT: Quantized 8bit bert. arXiv preprint arXiv:1910.06188, 2019. — [arXiv:1910.06188](https://arxiv.org/abs/1910.06188) find
  94. Zhang, D., Yang, J., Ye, D., and Hua, G. LQ-Nets: Learned quantization for highly accurate and compact deep neural networks. In Proceedings of the European conference on computer vision (ECCV), pp. 365–382, 2018. find
  95. Zhang, W., Hou, L., Yin, Y., Shang, L., Chen, X., Jiang, X., and Liu, Q. Ternarybert: Distillation-aware ultra-low bit bert. arXiv preprint arXiv:2009.12812, 2020. find
  96. Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., and Zou, Y. DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160, 2016. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kim-2021-i-bert-integer-only-bert/