Elamur Research
☰
home / papers / 2020 / Array Programming with NumPy

Array Programming with NumPy

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2006.10256v1

Summary

This paper describes NumPy, the main library for array programming in Python. It argues that a few basic array concepts give scientists a compact and efficient way to organize, explore and analyze data. NumPy appeared in 2005 when two earlier competing packages, Numeric and Numarray, were merged, which ended a split in the user community. The authors explain the core data structure: a pointer to memory plus metadata describing its data type, shape and strides. They then cover the operations built on it: indexing that returns either views or copies, vectorized arithmetic, broadcasting and axis-wise reductions, with linear algebra handled by optimized backends. NumPy is presented as the base layer of the scientific Python ecosystem. The paper also covers its newer array protocols, which pass NumPy API calls through to other array libraries designed for GPUs or distributed systems. This matters because NumPy is now core research infrastructure and the common API for tensor computation, with formal governance and outside funding supporting it.

Key points

  • NumPy came out in 2005 and combined Numarray's features (structured arrays, memory mapping, flexible indexing) with Numeric's speed on small arrays and its C API.
  • Array metadata includes dtype, shape and strides. For example, a float64 array of shape (4, 3) has strides (24, 8). Arrays can use C or Fortran memory order.
  • Libraries such as SciPy, Matplotlib, pandas, scikit-learn and scikit-image are built on NumPy. It was part of the software used to detect gravitational waves and to make the first image of a black hole (eht-imaging).
  • Array protocols let NumPy functions run on Dask, CuPy and similar arrays without changes, including CuPy arrays inside Dask arrays on multi-GPU clusters. Framework arrays such as PyTorch and TensorFlow are part of the same trend.
  • The project is fiscally sponsored by NumFOCUS and has received funding from the Moore and Sloan Foundations and the Chan Zuckerberg Initiative. It still depends heavily on volunteer contributors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Mathematics & statistics

Cited by

383 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 378 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1038/s41586-020-2649-2 · arXiv 2006.10256

@misc{harris2020array,
  title = {{Array Programming with NumPy}},
  author = {Harris, Charles R. and Millman, K. Jarrod and Walt, Stéfan J. van der and Gommers, Ralf and Virtanen, Pauli and Cournapeau, David and Wieser, Eric and Taylor, Julian and Berg, Sebastian and Smith, Nathaniel J. and Kern, Robert and Picus, Matti and Hoyer, Stephan and Kerkwijk, Marten H. van and Brett, Matthew and Haldane, Allan and Río, Jaime Fernández del and Wiebe, Mark and Peterson, Pearu and Gérard-Marchant, Pierre and Sheppard, Kevin and Reddy, Tyler and Weckesser, Warren and Abbasi, Hameer and Gohlke, Christoph and Oliphant, Travis E.},
  year = {2020},
  doi = {10.1038/s41586-020-2649-2},
  eprint = {2006.10256},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1038/s41586-020-2649-2}
}

References

16 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. K. E. Iverson, “Notation as a tool of thought,” Communications of the ACM, vol. 23, p. 444465, Aug. 1980. find
  2. P. F. Dubois, “Python: Batteries included,” Computing in Science & Engineering, vol. 9, no. 3, pp. 7–9, 2007. find
  3. T. E. Oliphant, “Python for scientific computing,” Computing in Science & Engineering, vol. 9, pp. 10–20, May-June 2007. find
  4. K. J. Millman and M. Aivazis, “Python for scientists and engineers,” Computing in Science & Engineering, vol. 13, no. 2, pp. 9–12, 2011. find
  5. F. Pérez, B. E. Granger, and J. D. Hunter, “Python: an ecosystem for scientific computing,” Computing in Science & Engineering, vol. 13, no. 2, pp. 13–21, 2011. find
  6. B. P. Abbott, R. Abbott, T. Abbott, M. Abernathy, F. Acernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. Adhikari, et al., “Observation of gravitational waves from a binary black hole merger,” Physical Review Letters, vol. 116, no. 6, p. 061102, 2016. find
  7. A. A. Chael, M. D. Johnson, R. Narayan, S. S. Doeleman, J. F. Wardle, and K. L. Bouman, “High-resolution linear polarimetric imaging for the event horizon telescope,” The Astrophysical Journal, vol. 829, no. 1, p. 11, 2016. find
  8. P. F. Dubois, K. Hinsen, and J. Hugunin, “Numerical Python,” Computers in Physics, vol. 10, no. 3, pp. 262–267, 1996. find
  9. D. Ascher, P. F. Dubois, K. Hinsen, J. Hugunin, and T. E. Oliphant, “An open source project: Numerical Python,” 2001. find
  10. T.-Y. Yang, G. Furnish, and P. F. Dubois, E. Gouillart, T. Yu, and the scikit-image con-“Steering object-oriented scientific computa-tributors, “scikit-image: image processing in tions,” in Proceedings of TOOLS USA 97. In-Python,” PeerJ, vol. 2, p. e453, 2014. ternational Conference on Technology of… find
  11. P. Greenfield, J. T. Miller, J. Hsu, and R. L. Science & Engineering, vol. 13, no. 2, pp. 22– White, “numarray: A new scientific array pack-30, 2011. age for Python,” PyCon DC, 2003. [19] Q. Wang, X. Zhang, Y. Zhang, and Q. Yi, find
  12. T. E. Oliphant, Guide to NumPy. Trelgol Pub-“Augem: automatically generate high perfor-lishing USA, 1st ed., 2006. mance dense linear algebra kernels on x86 cpus,” in SC’13: Proceedings of the International Con-[13] P. Virtanen, R. Gommers, T. E. Oliphant, ference on High Performance Computing, Net-… find
  13. J. D. Hunter, “Matplotlib: A 2D graphics envi- [22] T. Kluyver, B. Ragan-Kelley, F. Pérez, ronment,” Computing in Science & Engineering, B. Granger, M. Bussonnier, J. Frederic, K. Kel-vol. 9, no. 3, pp. 90–95, 2007. ley, J. Hamrick, J. Grout, S. Corlay, P. Ivanov, D. Avila, S. Abdalla, and C. Willi… find
  14. W. McKinney, “Data structures for statistical Notebooks—a publishing format for repro-computing in Python,” in Proceedings of the 9th ducible computational workflows,” in Position-Python in Science Conference (S. van der Walt ing and Power in Academic Publishing: Play-and J. Millman, eds.), pp. 51 –… find
  15. F. Pedregosa, G. Varoquaux, A. Gramfort, B. Schmidt, eds.), pp. 87–90, IOS Press, 2016. V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Van- [23] A. A. Hagberg, D. A. Schult, and P. J. Swart, “Exploring network structure, dynamics, and derplas, A. Passos, D. C… find
  16. S. van der Walt, J. L. Schönberger, J. Nunez- [24] Astropy Collaboration, T. P. Robitaille, E. J. Iglesias, F. Boulogne, J. D. Warner, N. Yager, Tollerud, P. Greenfield, M. Droettboom, E. Bray, T. Aldcroft, M. Davis, A. Ginsburg, A. M. Price-Whelan, W. E. Kerzendorf, A. Con-ley, N. Crighton, K. Bar… find our card