Elamur Research⌕
☰
home / papers / 2020 / Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling

Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling

Paper · 2020 · Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt et al.

Summary

GPU vendors change their machine ISAs and microarchitectures with almost every generation and rarely document them. Academic simulators therefore fall behind, and research built on outdated baselines can reach wrong conclusions. Accel-Sim tackles this with three components. The first is a frontend that can run either trace-driven simulation of NVIDIA's native SASS, using traces captured with NVBit, or execution-driven simulation of the virtual ISA PTX. The second is a heavily revised, more configurable GPGPU-Sim performance model, released as GPGPU-Sim 4.0. The third is validation tooling: microbenchmarks, an automatic parameter tuner and correlation plots built on counters that match NVIDIA profiler outputs. Because SASS traces can be taken from any CUDA binary, the framework also handles closed-source, hand-tuned libraries such as cuDNN. Compared with GPGPU-Sim 3.x, it cuts cycle error by 79 percentage points across 80 workloads, and it is validated on Kepler through Turing GPUs. This matters because a better-calibrated baseline changes which architectural ideas look promising.

Key points

  • Cycle error drops by 79 percentage points versus the prior state of the art on a Volta V100. The evaluation covers 80 workloads with 1,945 kernel instances, and the reported mean error is about 15%.
  • Accel-Sim additionally simulates 60 Deepbench workloads (11,440 kernel instances). These rely on closed-source SASS kernels that have no PTX version, which no other open-source simulator could run.
  • The framework models four NVIDIA generations: Kepler, Pascal, Volta and Turing. Simulation speed is about 12.5 KIPS in trace mode and about 6 KIPS in execution mode.
  • Counter-by-counter correlation revealed several undocumented hardware features:
    • a streaming, adaptive L1 data cache;
    • sectored L1 and L2 caches;
    • sub-warp, sector-based memory coalescing;
    • an L2 write policy that saves bandwidth when reads touch only part of a sector.
  • Two case studies show how baseline choice changes conclusions:
    • Out-of-order memory access scheduling looks weak on the old model but gives clear gains with a detailed memory system.
    • The L1 throughput bottleneck has disappeared on modern GPUs, which weakens the motivation for L1-bypassing schemes.

Interesting ideas

  • I1. Separate the performance model from the functional ISA. Traces are converted into an ISA-neutral intermediate form, and a small definition file maps each opcode to an execution unit. Supporting a new ISA then means writing a mapping rather than a full emulator, which is useful whenever an upstream format keeps changing.
  • I2. Offer two frontends on one core model. Trace mode gives speed and coverage, while execution mode remains available for studies that depend on actual data values or synchronization. Researchers choose the mode based on their study instead of being locked into one.
  • I3. Make simulator outputs comparable to real measurements. Designing the simulator's counters to correspond one-to-one with what hardware profilers report allows automatic, fine-grained comparison. Any modeling project can apply this by aligning its metrics with real-world measurements from the start.
  • I4. Turn validation into a tuning loop. Targeted microbenchmarks, an automated parameter tuner and correlation plots together make calibration repeatable rather than a one-off manual effort. This suits any model that must track a moving target over time.
  • I5. Re-test published conclusions on an updated baseline. Re-running known optimizations on a more accurate baseline shows how stale assumptions can hide or exaggerate benefits. This is a cheap sanity check for evaluations in any field.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of British ColumbiaVancouver · Canada104 works · 37,402 citations🎓Purdue University West LafayetteWest Lafayette · United States26 works · 2,076 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksHardware and Architecture · 167 worksComputer Graphics and Computer-Aided Design · 16 works

🧭 Topics 1Research subjects and languages the work is about.GPU & parallel computing · 41 works

Cite

Cite as Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt and Timothy G. Rogers. 2020. Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling. ISCA 2020.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1109/isca45697.2020.00047

@inproceedings{khairy2020accel,
  title = {{Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling}},
  author = {Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt and Timothy G. Rogers},
  booktitle = {ISCA 2020},
  year = {2020},
  doi = {10.1109/isca45697.2020.00047},
  url = {https://doi.org/10.1109/isca45697.2020.00047}
}

References

76 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. T. M. Aamodt, W. W. Fung, I. Singh, A. El-Shafiey, J. Kwa, T. Hetherington, A. Gubran, A. Boktor, T. Rogers, and A. Bakhoda, “GPGPU-Sim 3. x manual,” http://gpgpu-sim.org/manual/index.php/Main Page, 2016. ↗
  2. A. Adileh, C. González-Alvarez, J. M. D. H. Ruiz, and L. Eeckhout, “Racing to Hardware-Validated Simulation,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2019, pp. 58–67. find
  3. A. M. Aji, M. Daga, and W.-c. Feng, “Bounding the Effect of Partition Camping in GPU Kernels,” in Proceedings of the 8th ACM International Conference on Computing Frontiers (CF), 2011, p. 27. find
  4. N. Ardalani, C. Lestourgeon, K. Sankaralingam, and X. Zhu, “Cross-Architecture Performance Prediction (XAPP) Using CPU Code to Predict GPU Performance,” in Proceedings of the 48th International Symposium on Microarchitecture (MICRO), 2015, pp. 725–737. find
  5. A. Bakhoda, G. L. Yuan, W. W. Fung, H. Wong, and T. M. Aamodt, “Analyzing CUDA Workloads Using a Detailed GPU Simulator,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2009, pp. 163–174.
  6. B. Beckmann and A. Gutierrez, “The AMD gem5 APU Simulator: Modeling Heterogeneous Systems in gem5,” in Tutorial at the International Symposium on Microarchitecture (MICRO), 2015. find
  7. T. E. Carlson, W. Heirman, and L. Eeckhout, “Sniper: Exploring the Level of Abstraction for Scalable and Accurate Parallel Multi-Core Simulation,” in Proceedings of International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2011, pp. 1–12. find
  8. N. Chatterjee, M. O’Connor, G. H. Loh, N. Jayasena, and R. Balasub-ramonian, “Managing DRAM Latency Divergence in Irregular GPGPU Applications,” in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2014, pp. 128–139. find
  9. S. Che, M. Boyer, J. Meng, D. Tarjan, J. W. Sheaffer, S.-H. Lee, and K. Skadron, “Rodinia: A Benchmark Suite for Heterogeneous Computing,” in Proceedings of IEEE international symposium on workload characterization (IISWC), 2009, pp. 44–54. find
  10. J. Choquette, O. Giroux, and D. Foley, “Volta: Performance and Programmability,” IEEE Micro 2018, vol. 38, no. 2, pp. 42–52. find
  11. H. Dai, C. Li, Z. Lin, and H. Zhou, “The Demand for a Sound Baseline in GPU Memory Architecture Research,” in Proceedings of the Workshop on Duplicating, Deconstructing and Debunking (WDDD), 2017. find
  12. V. M. Del Barrio, C. González, J. Roca, A. Fernández, and E. Espasa, “ATTILA: A Cycle-Level Execution-Driven Simulator for Modern GPU Architectures,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2006, pp. 231–241. find
  13. R. Desikan, D. Burger, and S. W. Keckler, “Measuring Experimental Error in Microprocessor Simulation,” in Proceedings of the 28th annual International Symposium on Computer Architecture (ISCA), 2001, pp. 266–277. find
  14. P. Gera, H. Kim, H. Kim, S. Hong, V. George, and C.-K. C. Luk, “Performance Characterisation and Simulation of Intel’s Integrated GPU Architecture,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2018, pp. 139–148. find
  15. X. Gong, R. Ubal, and D. Kaeli, “Multi2Sim Kepler: A Detailed Architectural GPU Simulator,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2017, pp. 269– 278. find
  16. S. Grauer-Gray, L. Xu, R. Searles, S. Ayalasomayajula, and J. Cavazos, “Auto-tuning a High-Level Language Targeted to GPU Codes,” in Innovative Parallel Computing (InPar), 2012, pp. 1–10. find
  17. A. A. Gubran and T. M. Aamodt, “Emerald: Graphics Modeling for SoC Systems,” in ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA), 2019, pp. 169–182. find
  18. A. Gutierrez, B. M. Beckmann, A. Dutu, J. Gross, M. LeBeane, J. Kalamatianos, O. Kayiran, M. Poremba, B. Potter, S. Puthoor, M. Sinclair, M. Wyse, J. Yin, X. Zhang, A. Jain, and T. G. Rogers, “Lost in Abstraction: Pitfalls of Analyzing GPUs at the Intermediate Language Level,” in IEEE International… find
  19. A. Gutierrez, J. Pusdesris, R. G. Dreslinski, T. Mudge, C. Sudanthi, C. D. Emmons, M. Hayenga, and N. Paver, “Sources of Error in Full-System Simulation,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2014, pp. 13–22. find
  20. S. Hong and H. Kim, “An Integrated GPU Power and Performance Model,” in Proceedings of the 37th annual international symposium on computer architecture (ISCA), 2010, pp. 280–289. find
  21. J.-C. Huang, J. H. Lee, H. Kim, and H.-H. S. Lee, “GPUMech: GPU Performance Modeling Technique based on Interval Analysis,” in Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2014, pp. 268–279. find
  22. A. Jain, M. Khairy, and T. G. Rogers, “A Quantitative Evaluation of Contemporary GPU Simulation Methodology,” Proceedings of the ACM on Measurement and Analysis of Computing Systems (SIGMETRICS), vol. 2, no. 2, p. 35, 2018. find
  23. JEDEC Standard, “GDDR5X,” JESD232A, 2016. find
  24. ——, “High bandwidth memory (HBM) DRAM,” JESD235, 2020. find
  25. W. Jia, K. A. Shaw, and M. Martonosi, “MRPB: Memory Request Prioritization for Massively Parallel Processors,” in IEEE 20th International Symposium on High Performance Computer Architecture (HPCA), 2014, pp. 272–283. find
  26. Z. Jia, M. Maggioni, B. Staiger, and D. P. Scarpazza, “Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking,” arXiv preprint arXiv:1804.06826, 2018.
  27. N. P. Jouppi, “Cache Write Policies and Performance,” in 20th Annual International Symposium on Computer Architecture (ISCA), 1993, pp. 191–201. find
  28. H. Kim, J. Lee, N. B. Lakshminarayana, J. Sim, J. Lim, and T. Pho, “MacSim: A CPU-GPU Heterogeneous Simulation Framework User Guide,” Technical Report, Georgia Institute of Technology, 2012. find
  29. G. Koo, Y. Oh, W. W. Ro, and M. Annavaram, “Access Pattern-Aware Cache Management for Improving Data Utilization in GPU,” in Proceedings of the 44th Annual International Symposium on Computer Architecture (ISCA), 2017, pp. 307–319. find
  30. C. J. Lee, V. Narasiman, E. Ebrahimi, O. Mutlu, and Y. N. Patt, “DRAM-Aware Last-Level Cache Writeback: Reducing Write-Caused Interference in Memory Systems,” Technical Report, The University of Texas at Austin, TR-HPS-2010-002, 2010. find
  31. J. Lew, D. A. Shah, S. Pati, S. Cattell, M. Zhang, A. Sandhupatla, C. Ng, N. Goli, M. D. Sinclair, T. G. Rogers, and T. M. Aamodt, “Analyzing Machine Learning Workloads Using a Detailed GPU Simulator,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2019, pp… find
  32. J. S. Liptay, “Structural Aspects of the System/360 Model 85, II: The Cache,” IBM Systems Journal, vol. 7, no. 1, pp. 15–21, 1968. find
  33. Y. Liu, X. Zhao, M. Jahre, Z. Wang, X. Wang, Y. Luo, and L. Eeckhout, “Get Out of the Valley: Power-Efficient Address Mapping for GPUs,” in ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), 2018, pp. 166–179. find
  34. X. Mei and X. Chu, “Dissecting GPU Memory Hierarchy through Microbenchmarking,” IEEE Transactions on Parallel and Distributed Systems (TPDS), vol. 28, no. 1, pp. 72–86, 2017. find
  35. S. Narang and G. Diamos, “Baidu Deepbench,” https://svail.github.io/DeepBench, 2016. ↗
  36. T. Nowatzki, J. Menon, C.-H. Ho, and K. Sankaralingam, “Architectural Simulators Considered Harmful,” IEEE Micro, vol. 35, no. 6, pp. 4–12, 2015. find
  37. C. Nugteren, G.-J. van den Braak, H. Corporaal, and H. Bal, “A Detailed GPU Cache Model Based on Reuse Distance Theory,” in IEEE 20th International Symposium on High Performance Computer Architecture (HPCA), 2014, pp. 37–48. find
  38. NVIDIA, “Kepler Architecture,” https://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf, 2012. ↗
  39. ——, “GeForce GTX Titan,” https://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan/specifications, 2013. ↗
  40. ——, “CUDA C/C++ SDK Code Samples,” http://developer.nvidia.com/cuda-cc-sdk-code-samples, 2015. ↗
  41. ——, “NVIDIA TITAN X,” https://www.nvidia.com/en-us/geforce/products/10series/titan-x-pascal/, 2016. ↗
  42. ——, “CUDA Binary Utilities,” https://docs.nvidia.com/cuda/cuda-binary-utilities/index.html, 2017. ↗
  43. ——, “CuDNN Devloper Guide,” https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html, 2017. ↗
  44. ——, “Nvidia Profiler 8.0,” http://docs.nvidia.com/cuda/profiler-users-guide/index.html, 2017. ↗
  45. ——, “Volta Cache,” http://on-demand.gputechconf.com/gtc/2017/presentation/s7798-luke-durant-inside-volta.pdf, 2017. ↗
  46. ——, “Volta V100 White paper,” http://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf, 2017. ↗
  47. ——, “CUTLASS: CUDA template library for dense linear algebra at all levels and scales,” https://github.com/NVIDIA/cutlass, 2018. ↗
  48. ——, “GeForce RTX 2060,” https://www.nvidia.com/en-us/geforce/graphics-cards/rtx-2060/, 2019. ↗
  49. ——, “NVIDIA Nsight CLI,” https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html, 2019. ↗
  50. ——, “Programming Guide,” https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html, 2019. ↗
  51. M. O’Connor, “Highlights of the High-Bandwidth Memory (HBM) Standard,” in Memory Forum Workshop, Co-located with the 41st International Symposium on Computer Architecture (ISCA), 2014. find
  52. K. Pagiamtzis and A. Sheikholeslami, “A Low-Power Content-Addressable Memory (CAM) Using Pipelined Hierarchical Search Scheme,” IEEE Journal of Solid-State Circuits, vol. 39, no. 9, pp. 1512– 1519, 2004. find
  53. Paulius Micikevicius, “Kepler Architecture,” http://on-demand.gputechconf.com/gtc/2013/presentations/ S3466-Programming-Guidelines-GPU-Architecture.pdf, 2013. ↗
  54. J. Power, J. Hestness, M. S. Orr, M. D. Hill, and D. A. Wood, “gem5-gpu: A heterogeneous CPU-GPU Simulator,” IEEE Computer Architecture Letters (CAL), vol. 14, no. 1, pp. 34–36, 2014. find
  55. M. A. Raihan, N. Goli, and T. M. Aamodt, “Modeling Deep Learning Accelerator Enabled GPUs,” in IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2019, pp. 79– 92. find
  56. B. R. Rau, “Pseudo-Randomly Interleaved Memory,” in Proceedings of the 18th Annual International Symposium on Computer Architecture (ISCA), 1991, pp. 74–83. find
  57. M. Rhu, M. Sullivan, J. Leng, and M. Erez, “A Locality-Aware Memory Hierarchy for Energy-Efcient GPU Architectures,” in Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2013, pp. 86–98. find
  58. S. Rixner, W. J. Dally, U. J. Kapasi, P. Mattson, and J. D. Owens, “Memory Access Scheduling,” in Proceedings of the 27th Annual International Symposium on Computer Architecture (ISCA), 2000, pp. 128–138. find
  59. T. G. Rogers, M. O’Connor, and T. M. Aamodt, “Cache-Conscious Wavefront Scheduling,” in Proceedings of the 45th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2012, pp. 72– 83. find
  60. R. H. Saavedra-Barrera, “CPU Performance Evaluation and Execution Time Prediction Using Narrow Spectrum Benchmarking,” Technical Report No. UCB/CSD-92-684, University of California, Berkeley, 1993. find
  61. A. Sethia, D. A. Jamshidi, and S. Mahlke, “Mascar: Speeding up GPU Warps by Reducing Memory Pitstops,” in IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), 2015, pp. 174–185. find
  62. A. Seznec, “Decoupled Sectored Caches: conciliating low tag implementation cost and low miss ratio,” in Proceedings of the 21st Annual International Symposium on Computer Architecture (ISCA), 1994, pp. 384–393. find
  63. M. D. Sinclair, J. Alsop, and S. V. Adve, “Chasing Away RAts: Semantics and Evaluation for Relaxed Atomics on Heterogeneous Systems,” in ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA), 2017, pp. 161–174. find
  64. M. Stephenson, S. K. S. Hari, Y. Lee, E. Ebrahimi, D. R. Johnson, D. Nellans, M. O’Connor, and S. W. Keckler, “Flexible Software Profiling of GPU Architectures,” in ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), 2015, pp. 185–197. find
  65. J. A. Stratton, C. Rodrigues, I.-J. Sung, N. Obeid, L.-W. Chang, N. Anssari, G. D. Liu, and W. Hwu, “Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing,” Center for Reliable and High-Performance Computing, vol. 127, 2012. find
  66. J. Stuecheli, D. Kaseridis, D. Daly, H. C. Hunter, and L. K. John, “The Virtual Write Queue: Coordinating DRAM and Last-Level Cache Policies,” in Proceedings of the 47th Annual International Symposium on Computer Architecture (ISCA), 2010, pp. 72–82. find
  67. Y. Sun, T. Baruah, S. A. Mojumder, S. Dong, X. Gong, S. Treadway, Y. Bao, S. Hance, C. McCardwell, V. Zhao, H. Barclay, A. K. Ziabari, Z. Chen, R. Ubal, J. L. Abelln, J. Kim, A. Joshi, and D. Kaeli, “MG-PUSim: Enabling Multi-GPU Performance Modeling and Optimization,” in Proceedings of the 46th Inte… find
  68. J. Tuck, L. Ceze, and J. Torrellas, “Scalable Cache Miss Handling for High Memory-Level Parallelism,” in Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2006, pp. 409–422. find
  69. R. Ubal, B. Jang, P. Mistry, D. Schaa, and D. Kaeli, “Multi2Sim: A Simulation Framework for CPU-GPU Computing,” in Proceedings of the 21st International Conference on Parallel Architectures and Compilation Techniques (PACT), 2012, pp. 335–344. find
  70. O. Villa, M. Stephenson, D. Nellans, and S. W. Keckler, “NVBit: A Dynamic Binary Instrumentation Framework for NVIDIA GPUs,” in Proceedings of the 52Nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2019, pp. 372–383. find
  71. X. Wang, K. Huang, A. Knoll, and X. Qian, “A Hybrid Framework for Fast and Accurate GPU Performance Estimation through Source-Level Analysis and Trace-Based Simulation,” in IEEE International Symposium on High Performance Computer Architecture (HPCA), 2019, pp. 506–518. find
  72. D. Wong, N. S. Kim, and M. Annavaram, “Approximating Warps with Intra-warp Operand Value Similarity,” in 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2016, pp. 176–187. find
  73. H. Wong, M.-M. Papadopoulou, M. Sadooghi-Alvandi, and A. Moshovos, “Demystifying GPU Microarchitecture through Microbenchmarking,” in IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS), 2010, pp. 235–246. find
  74. G. Wu, J. L. Greathouse, A. Lyashevsky, N. Jayasena, and D. Chiou, “GPGPU Performance and Power Estimation Using Machine Learning,” in IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), 2015, pp. 564–576. find
  75. Y. Zhang and J. D. Owens, “A Quantitative Performance Analysis Model for GPU Architectures,” in IEEE 17th International Symposium on High Performance Computer Architecture (HPCA), 2011, pp. 382–393. find
  76. H. Zhu, M. Akrout, B. Zheng, A. Pelegris, A. Jayarajan, A. Phanishayee, B. Schroeder, and G. Pekhimenko, “Benchmarking and Analyzing Deep Neural Network Training,” in IEEE International Symposium on Workload Characterization (IISWC), 2018, pp. 88–100. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khairy-2020-accel-sim-extensible-simulation-framework/