Elamur Research⌕
☰
home / papers / 2020 / Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

Paper · 2020 · Miles Brundage, Shahar Avin, Jasmine Wang et al.

Summary

AI developers increasingly make public promises about the safety, fairness, security, and privacy of their systems. Outsiders such as users, regulators, civil society, and rival labs often have no way to check those promises. This policy report, written by authors from 30 organisations across industry, academia, and civil society, asks how such statements could become verifiable claims: assertions concrete enough to be supported or refuted with evidence. It grew out of an April 2019 workshop and later discussions. It does not offer an experiment or dataset. Instead, it reviews ten existing or emerging mechanisms in three groups: institutional, software, and hardware. For each, it describes the current state, the gaps, and a concrete recommendation. The authors argue that no single tool is enough and that the mechanisms should reinforce one another. The report matters because it moves debate on responsible AI away from broad ethics principles and toward practical ways of building checkable trust. That also helps reduce suspicion and corner-cutting among developers.

Key points

  • Institutional mechanisms cover how organisations behave:
    • independent outside reviews of developers' systems and practices;
    • adversarial testing, where dedicated teams deliberately probe for weaknesses;
    • rewards for people who uncover bias or safety problems, modelled on bug bounties;
    • sharing of AI incidents between developers.
  • Software mechanisms cover how claims about systems can be checked:
    • audit trails for documenting development, especially in safety-critical uses;
    • interpretability research;
    • privacy-preserving machine learning, such as federated learning, differential privacy, and encrypted computation, with a push for accessible open-source tooling.
  • Hardware mechanisms cover the computing layer:
    • secure hardware for ML, such as trusted execution environments for accelerators;
    • high-precision measurement and reporting of compute use;
    • greater compute support for academia, so independent researchers can evaluate claims about large industry models.
  • Each section ends with an actionable recommendation addressed to AI developers, funders, or governments, such as running pilot audits, pilot bounties, or shared incident reporting.
  • The core conceptual contribution is to define "verifiable claims" and to treat trust as something earned through evidence, not something simply asserted by developers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States533 works · 443,885 citations🎓Stanford UniversityStanford · United States567 works · 335,190 citations🎓University of California, BerkeleyBerkeley · United States243 works · 251,774 citations🎓University of CambridgeCambridge · United Kingdom287 works · 125,174 citations🎓University of TorontoToronto · Canada121 works · 179,943 citations🎓University of OxfordOxford · United Kingdom138 works · 161,448 citations🎓Université de MontréalMontreal · Canada96 works · 92,262 citations🎓McGill UniversityMontreal · Canada97 works · 11,755 citations🎓École Polytechnique Fédérale de LausanneLausanne · Switzerland56 works · 8,189 citations🎓Polytechnique MontréalMontreal · Canada12 works · 35,462 citations🔬The Alan Turing InstituteLondon · United Kingdom35 works · 5,111 citations🔬École Normale Supérieure - PSLParis · France25 works · 4,704 citations🎓Coventry UniversityCoventry · United Kingdom5 works · 731 citations🎓Eindhoven University of TechnologyEindhoven · Netherlands11 works · 956 citations💡RAND CorporationSanta Monica · United States4 works · 1,233 citations💡Center for Security and Emerging TechnologyWashington D.C. · United States2 works · 358 citations

Also on the byline: Center for a New American Security · Centre Universitaire de Mila · Adelard.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksSocial Sciences · 2,009 worksSafety Research · 665 worksEngineering · 206 worksSafety, Risk, Reliability and Quality · 18 works

🧭 Topics 1Research subjects and languages the work is about.Bias, fairness & safety · 590 works

Cite

Cite as Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O'Keefe, Mark Koren, Théo Ryffel, JB Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hÉigeartaigh, Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio and Markus Anderljung. 2020. Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. arXiv:2004.07213.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2004.07213 · arXiv 2004.07213

@misc{brundage2020toward,
  title = {{Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims}},
  author = {Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O'Keefe, Mark Koren, Théo Ryffel, JB Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah {de Haas}, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hÉigeartaigh, Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio and Markus Anderljung},
  year = {2020},
  doi = {10.48550/arxiv.2004.07213},
  eprint = {2004.07213},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2004.07213}
}

References

238 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amanda Askell, Miles Brundage, and Gillian Hadfield. “The Role of Cooperation in Responsible AI Development”. In: arXiv (July 2019). arXiv: 1907.04534 . URL : http://arxiv.org/abs/1907.04534 . — [arXiv:1907.04534](https://arxiv.org/abs/1907.04534) ↗
  2. Meredith Whittaker et al. AI Now Report 2018. Tech. rep. New York, NY, USA: AI Now Institute, 2018, p. 68. URL : https://ainowinstitute.org/AI%7B%5C_%7DNow%7B%5C_%7D2018%7B%5C_%7DReport.pdf . ↗
  3. Kate Crawford et al. AI Now 2019 Report. Tech. rep. New York, NY, USA: AI Now Institute, 2019, p. 100. URL : https://ainowinstitute.org/AI%7B%5C_%7DNow%7B%5C_%7D2 019%7B%5C_%7DReport.pdf . ↗
  4. Elizabeth Gibney. “The battle for ethical AI at the world’s biggest machine-learning conference”. In: Nature (2020). ISSN : 0028-0836. DOI : 10.1038/d41586-020-00160-y . URL : http ://www.nature.com/articles/d41586-020-00160-y . — [doi:10.1038/d41586-020-00160-y](https://doi.org/10.1038/d41586-020-00160-y) find
  5. Brent Mittelstadt. “AI Ethics - Too Principled to Fail?” In: Nature Machine Intelligence (June 2019). DOI : 10.2139/ssrn.3391293 . arXiv: 1906.06668 . URL : https://arxi v.org/ftp/arxiv/papers/1906/1906.06668.pdf . — [doi:10.2139/ssrn.3391293](https://doi.org/10.2139/ssrn.3391293) find
  6. Karen Hao. This is how AI bias really happens-and why it’s so hard to fix - MIT Technology Review. 2019. URL : https://www.technologyreview.com/s/612876/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/ (visited on 02/01/2020). ↗
  7. University of California - Berkeley. Artificial intelligence advances threaten privacy of health data: Study finds current laws and regulations do not safeguard individuals’ confidential health information – ScienceDaily. 2019. URL : https://www.sciencedaily.com/releases/201 9/01/190103152906.htm (v… ↗
  8. Adam Alter. Irresistible : the rise of addictive technology and the business of keeping us hooked. The Penguin Group, 2017, p. 354. ISBN : 9781594206641. URL : http://adamalterauthor.com/irresistible . ↗
  9. Julia Angwin et al. Machine Bias. 2016. URL : https://www.propublica.org/articl e/machine-bias-risk-assessments-in-criminal-sentencing (visited on 02/01/2020). ↗
  10. Irene Solaiman et al. “Release Strategies and the Social Impacts of Language Models”. In: arXiv (Aug. 2019). arXiv: 1908.09203 . URL : http://arxiv.org/abs/1908.09203 . ↗
  11. Mary L. Gray and Siddharth Suri. Ghost work : how to stop Silicon Valley from building a new global underclass. Eamon Dolan / Houghton Mifflin Harcourt, 2019, p. 254. ISBN : 1328566242. URL : https://www.hmhco.com/shop/books/Ghost-Work/9781328566249 . ↗
  12. Carl Benedikt Frey and Michael A Osborne. The Future of Employment: How Susceptible Are Jobs to Computerisation? Tech. rep. The Oxfort Marting Programme on Technology and Employment, 2013, p. 72. URL : https://www.oxfordmartin.ox.ac.uk/downloads/acade mic/The%7B%5C_%7DFuture%7B%5C_%7Dof%7B%5C_%7DEmp… ↗
  13. Jess Whittlestone et al. “The role and limits of principles in AI ethics: Towards a focus on tensions”. In: AIES 2019 - Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. Association for Computing Machinery, Inc, Jan. 2019, pp. 195–200. ISBN : 9781450363242. DOI : 10.1145/330661… — [doi:10.1145/3306618.3314289](https://doi.org/10.1145/3306618.3314289) find
  14. Anna Jobin, Marcello Ienca, and Effy Vayena. “The global landscape of AI ethics guidelines”. In: Nature Machine Intelligence 1.9 (Sept. 2019), pp. 389–399. DOI : 10.1038/s42256-019-0088-2 . URL : https://www.nature.com/articles/s42256-019-0088-2 . ↗
  15. Beijing AI Principles Blog. Beijing AI Principles. 2019. URL : https://www.baai.ac.cn/blog/beijing-ai-principles (visited on 02/01/2020). ↗
  16. Yi Zeng, Enmeng Lu, and Cunqing Huangfu. “Linking Artificial Intelligence Principles”. In: arXiv (Dec. 2018). arXiv: 1812.04814 . URL : http://arxiv.org/abs/1812.04814 . — [arXiv:1812.04814](https://arxiv.org/abs/1812.04814) ↗
  17. Edelman. 2019 Edelman Trust Barometer \| Edelman. 2019. URL : https://www.edelman.com/research/2019-edelman-trust-barometer (visited on 02/01/2020). ↗
  18. Baobao Zhang and Allan Dafoe. Artificial Intelligence: American Attitudes and Trends. Tech. rep. Future of Humanity Institute, 2019. URL : https://governanceai.github.io/US-Public-Opinion-Report-Jan-2019/ . ↗
  19. Gregory Travis. How the Boeing 737 Max Disaster Looks to a Software Developer. 2019. URL : https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-m ax-disaster-looks-to-a-software-developer (visited on 02/08/2020). ↗
  20. Jenny et al. Gesley. “Regulation of Artificial Intelligence in Selected Jurisdictions”. In: (Jan. 2019). URL : https://www.loc.gov/law/help/artificial-intelligence/regulation-artificial-intelligence.pdf . ↗
  21. E. L. Trist. The evolution of socio-technical systems : a conceptual framework and an action research program. Ontario Quality of Working Life Centre, 1981, p. 67. ISBN : 9780774362863. URL : https://www.semanticscholar.org/paper/The-Evolution-of-Socio-Tech nical-Systems%7B%5C%%7D3A-A-and-an-Trist/… ↗
  22. Sundar Pichai. AI at Google: our principles. 2018. URL : https://www.blog.google/te chnology/ai/ai-principles/ (visited on 02/01/2020). ↗
  23. OpenAI. OpenAI Charter. 2018. URL : https://openai.com/charter/ (visited on 02/01/2020). ↗
  24. Microsoft. AI Principles & Approach from Microsoft. URL : https://www.microsoft.com/en-us/ai/our-approach-to-ai (visited on 02/01/2020). ↗
  25. Government of Canada. Algorithmic Impact Assessment (AIA) - Canada.ca. 2019. URL : https://www.canada.ca/en/government/system/digital-government/modern-emerging-technologies/responsible-use-ai/algorithmic-impact-a ssessment.html . ↗
  26. Treasury Board of Canada Secretariat. Directive on Automated Decision-Making. Tech. rep. Treasury Board of Canada Secreteriat, 2019. URL : https://www.tbs-sct.gc.ca/pol/d oc-eng.aspx?id=32592 . ↗
  27. Inioluwa Deborah Raji and Jingying Yang. “ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles”. In: arXiv (Dec. 2019). arXiv: 1912.06166 . URL : http://arxiv.org/abs/1912.06166 . — [arXiv:1912.06166](https://arxiv.org/abs/1912.06166) ↗
  28. Margaret Mitchell et al. “Model Cards for Model Reporting”. In: FAT ’19: Proceedings of the Conference on Fairness, Accountability, and Transparency. Oct. 2019. DOI : 10.1145/3287560.3287596 . arXiv: 1810.03993 . URL : http://arxiv.org/abs/1810.03993%20http://dx .doi.org/10.1145/3287560.3287596 . ↗
  29. Timnit Gebru et al. “Datasheets for Datasets”. In: arXiv (Mar. 2018). arXiv: 1803.09010 . URL : http://arxiv.org/abs/1803.09010 . ↗
  30. Cullen O’Keefe et al. “The Windfall Clause: Distributing the Benefits of AI for the Common Good”. In: arXiv (Dec. 2019). arXiv: 1912.11595 . URL : http://arxiv.org/abs/1912.11 595 . — [arXiv:1912.11595](https://arxiv.org/abs/1912.11595) ↗
  31. James M. Buchanan. “An Economic Theory of Clubs”. In: Economica 32.125 (Feb. 1965), p. 1. ISSN : 00130427. DOI : 10.2307/2552442 . URL : https://www.jstor.org/stabl e/2552442?seq=1 . — [doi:10.2307/2552442](https://doi.org/10.2307/2552442) ↗
  32. ETSI. Intellectual Property Rights. URL : https://www.etsi.org/intellectual-pr operty-rights (visited on 02/01/2020). ↗
  33. International Trade Forum. “Voluntary standards in developing Countries: The potential of voluntary standards and their role in international trade”. In: (Mar. 2010). URL : http://www.t radeforum.org/Voluntary-Standards-in-Developing-Countries-The -Potential-of-Voluntary-Standards-and-their-Role-in-… ↗
  34. Future of Life Institute. AI Principles. 2017. URL : https://futureoflife.org/ai-pr inciples/?cn-reloaded=1 (visited on 02/01/2020). ↗
  35. Andy Pasztor and Alison Sider. Internal Boeing Documents Show Cavalier Attitude to Safety. 2020. URL : https://www.wsj.com/articles/internal-boeing-documents-sho w-cavalier-attitude-to-safety-11578627206 (visited on 02/06/2020). ↗
  36. Inioluwa Deborah Raji et al. “Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing”. In: arXiv (Jan. 2020). arXiv: 2001.00973 . URL : http://arxiv.org/abs/2001.00973 . — [arXiv:2001.00973](https://arxiv.org/abs/2001.00973) ↗
  37. David Engstrom and Daniel Ho. “Algorithmic Accountability in the Administrative State”. In: CSAS Working Papers (Nov. 2019). URL : https://administrativestate.gmu.edu/wp-content/uploads/sites/29/2019/11/Engstrom-Ho-Algorithmic-Accountability-in-the-Administrative-State.pdf . ↗
  38. Jan Barton. “Who Cares about Auditor Reputation?” In: Contemporary Accounting Research 22.3 (Oct. 2005), pp. 549–586. ISSN : 0823-9150. DOI : 10.1506/C27U-23K8-E1VL-20R0 . URL : http://doi.wiley.com/10.1506/C27U-23K8-E1VL-20R0 . — [doi:10.1506/c27u-23k8-e1vl-20r0](https://doi.org/10.1506/c27u-23k8-e1vl-20r0) ↗
  39. Ryan Carrier. “Governance and Oversight Coming to AI and Automation: Independent Audit of AI Systems”. In: ACM blog (Feb. 2019). URL : https://cacm.acm.org/blogs/blog-cac m/234723-governance-and-oversight-coming-to-ai-and-automation -independent-audit-of-ai-systems/fulltext . ↗
  40. Jesse Hempel. “Want to Prove Your Business is Fair? Audit Your Algorithm”. In: (May 2018). URL : https://www.wired.com/story/want-to-prove-your-business-is-fa ir-audit-your-algorithm . ↗
  41. Oren Etzioni and Michael Li. “Want to Prove Your Business is Fair? Audit Your Algorithm”. In: (July 2019). URL : https://www.wired.com/story/ai-needs-to-be-audite d . ↗
  42. Jack Clark and Gillian K. Hadfield. “Regulatory Markets for AI Safety”. In: arXiv (Dec. 2019). arXiv: 2001.00078 . URL : http://arxiv.org/abs/2001.00078 . — [arXiv:2001.00078](https://arxiv.org/abs/2001.00078) ↗
  43. Andrew Marshall et al. “Threat Modeling AI/ML Systems and Dependencies”. In: (Nov. 2019). URL : https://docs.microsoft.com/en-us/security/threat-modeling-aiml . ↗
  44. Ariel Herbert-Voss. Dont Red Team AI Like a Chump. 2019. URL : https://www.youtube.com/watch?v=ivlc1yPw76A (visited on 02/01/2020). ↗
  45. Hackerone. The Hacker-Powered Security Report 2017. Tech. rep. Hackerone, 2017. URL : https ://www.hackerone.com/sites/default/files/2017-06/The%20Hacker-Powered%20Security%20Report.pdf . find
  46. J. B. Rubinovitz. Bias Bounty Programs as a Method of Combatting Bias in AI. 2018. URL : https://rubinovitz.com/post/187669528132/bias-bounty-programs-as-a-method-of-combatting (visited on 02/01/2020). ↗
  47. Ziad Obermeyer et al. “Dissecting racial bias in an algorithm used to manage the health of populations”. In: Science 366.6464 (2019), pp. 447–453. ISSN : 0036-8075. URL : https://s cience.sciencemag.org/content/366/6464/447 . find
  48. Taylor Telford. Apple Card algorithm sparks gender bias allegations against Goldman Sachs. Nov. 2019. URL : https://www.washingtonpost.com/business/2019/11/11/app le-card-algorithm-sparks-gender-bias-allegations-against-gold man-sachs/ (visited on 02/06/2020). ↗
  49. Dario Amodei et al. “Concrete Problems in AI Safety”. In: arXiv (June 2016). arXiv: 1606.06565 . URL : http://arxiv.org/abs/1606.06565 . ↗
  50. Warwick Ashford. Bug bounties not a silver bullet, Katie Moussouris warns. 2018. URL : https://www.computerweekly.com/news/252450337/Bug-bounties-not-a-sil ver-bullet-Katie-Moussouris-warns (visited on 02/08/2020). ↗
  51. David Dao. Awful AI. URL : https://github.com/daviddao/awful-ai (visited on 02/01/2020). ↗
  52. Victoria Krakovna. “Specification gaming examples in AI”. In: (Apr. 2018). URL : https://vkr akovna.wordpress.com/2018/04/02/specification-gaming-examples -in-ai/ . find
  53. Peter Lee. “Learning from Tay’s introduction”. In: (Mar. 2016). URL : https://blogs.mic rosoft.com/blog/2016/03/25/learning-tays-introduction/ . ↗
  54. Thomas Dietterich. “Robust Artificial Intelligence and Robust Human Organizations”. In: arXiv (Mar. 2018). URL : https://arxiv.org/abs/1811.10840 . — [arXiv:1811.10840](https://arxiv.org/abs/1811.10840) ↗
  55. Aviv Ovadya and Jess Whittlestone. “Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning”. In: arXiv (July 2019). arXiv: 1907.11274 . URL : http://arxiv.org/abs/1907.11274 . — [arXiv:1907.11274](https://arxiv.org/abs/1907.11274) ↗
  56. Jan Leike et al. “AI Safety Gridworlds”. In: arXiv (Nov. 2017). arXiv: 1711.09883 . URL : http://arxiv.org/abs/1711.09883 . — [arXiv:1711.09883](https://arxiv.org/abs/1711.09883) ↗
  57. Christian Szegedy et al. “Intriguing properties of neural networks”. In: 2nd International Conference on Learning Representations, ICLR 2014 - Conference Track Proceedings. International Conference on Learning Representations, ICLR, 2014. arXiv: 1312.6199 . URL : https://arx iv.org/abs/1312.6199 .
  58. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. “Explaining and harnessing adversarial examples”. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations, ICLR, 2015. arXiv: 1412.6572 . UR…
  59. Alice Xiang and Inioluwa Deborah Raji. “On the Legal Compatibility of Fairness Definitions”. In: arXiv (Nov. 2019). arXiv: 1912.00761 . URL : http://arxiv.org/abs/1912.0076 1 . ↗
  60. Reuben Binns. “Fairness in Machine Learning: Lessons from Political Philosophy”. In: arXiv (Dec. 2017). arXiv: 1712.03586 . URL : http://arxiv.org/abs/1712.03586 . — [arXiv:1712.03586](https://arxiv.org/abs/1712.03586) ↗
  61. Mark Koren, Anthony Corso, and Mykel Kochenderfer. “The Adaptive Stress Testing Formulation”. In: RSS 2019: Workshop on Safe Autonomy. Freiburg, 2019. URL : https://openrev iew.net/pdf?id=rJgoNK-oaE . find
  62. Ritchie Lee et al. “Adaptive stress testing of airborne collision avoidance systems”. In: AIAA/IEEE Digital Avionics Systems Conference - Proceedings. Institute of Electrical and Electronics Engineers Inc., Oct. 2015. ISBN : 9781479989409. DOI : 10.1109/DASC.2015.7311613 . URL : https://ieeexplore.… — [doi:10.1109/dasc.2015.7311613](https://doi.org/10.1109/dasc.2015.7311613) ↗
  63. N. V. Chawla et al. “SMOTE: Synthetic Minority Over-sampling Technique”. In: Journal of Artificial Intelligence Research (June 2002). DOI : 10.1613/jair.953 . arXiv: 1106.1813 . URL : http://arxiv.org/abs/1106.1813%20http://dx.doi.org/10.1613/jai r.953 . ↗
  64. Guillaume Lemaître, Fernando Nogueira, and Christos K. Aridas. “Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning”. In: Journal of Machine Learning Research 7 (2016), pp. 1–5. arXiv: 1609.06570v1 . URL : http://www.jmlr.o rg/papers/volume18/16-365/16-3… — [arXiv:1609.06570](https://arxiv.org/abs/1609.06570) ↗
  65. Aleksander Madry et al. “Towards Deep Learning Models Resistant to Adversarial Attacks”. In: arXiv (June 2017). arXiv: 1706.06083 . URL : http://arxiv.org/abs/1706.0608 3 . ↗
  66. Matthias Hein and Maksym Andriushchenko. “Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation”. In: arXiv (May 2017). arXiv: 1705.08475 . URL : http://arxiv.org/abs/1705.08475 . — [arXiv:1705.08475](https://arxiv.org/abs/1705.08475) ↗
  67. Mathias Lecuyer et al. “Certified Robustness to Adversarial Examples with Differential Privacy”. In: arXiv (Feb. 2018). arXiv: 1802.03471 . URL : http://arxiv.org/abs/1802.03 471 . — [arXiv:1802.03471](https://arxiv.org/abs/1802.03471) ↗
  68. Jeremy M Cohen, Elan Rosenfeld, and J. Zico Kolter. “Certified Adversarial Robustness via Randomized Smoothing”. In: arXiv (Feb. 2019). arXiv: 1902.02918 . URL : http://arxiv.org/abs/1902.02918 . — [arXiv:1902.02918](https://arxiv.org/abs/1902.02918) ↗
  69. Guy Katz et al. “Reluplex: An efficient smt solver for verifying deep neural networks”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 10426 LNCS. Springer Verlag, 2017, pp. 97–117. ISBN : 9783319633862.… — [doi:10.1007/978-3-319-63387-9_5](https://doi.org/10.1007/978-3-319-63387-9_5) find
  70. Kevin Bonsor and Nathan Chandler. How Black Boxes Work. URL : https://science.h owstuffworks.com/transport/flight/modern/black-box.htm (visited on 02/02/2020). ↗
  71. Mustafa Suleyman and Ben Laurie. Trust, confidence and Verifiable Data Audit. 2017. URL : https ://deepmind.com/blog/article/trust-confidence-verifiable-data -audit (visited on 02/02/2020). find
  72. Joanna Bryson. “AI & Global Governance: No One Should Trust AI”. In: (Nov. 2018). URL : https://cpr.unu.edu/ai-global-governance-no-one-should-trust-ai.h tml . ↗
  73. Cynthia Rudin. “Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead”. In: arXiv (Nov. 2018). arXiv: 1811.10154 . URL : http://arxiv.org/abs/1811.10154 . ↗
  74. Zachary C. Lipton. “The Mythos of Model Interpretability”. In: arXiv (June 2016). arXiv: 1606.03490 . URL : http://arxiv.org/abs/1606.03490 . — [arXiv:1606.03490](https://arxiv.org/abs/1606.03490) ↗
  75. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. “Axiomatic Attribution for Deep Networks”. In: arXiv (Mar. 2017). arXiv: 1703.01365 . URL : http://arxiv.org/abs/1703.01 365 . ↗
  76. Mark Sendak et al. “"The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning”. In: arXiv (Nov. 2019). arXiv: 1911.08089 . URL : http://arxiv.org/abs/1911.08089 . — [arXiv:1911.08089](https://arxiv.org/abs/1911.08089) ↗
  77. Berk Ustun, Alexander Spangher, and Yang Liu. “Actionable Recourse in Linear Classification”. In: FAT ’19: Proceedings of the Conference on Fairness, Accountability, and Transparency. Sept. 2019. DOI : 10.1145/3287560.3287566 . arXiv: 1809.06514 . URL : http://arxiv .org/abs/1809.06514%20http://dx.d… — [doi:10.1145/3287560.3287566](https://doi.org/10.1145/3287560.3287566) find
  78. Forough Poursabzi-Sangdeh et al. “Manipulating and Measuring Model Interpretability”. In: arXiv (Feb. 2018). arXiv: 1802.07810 . URL : http://arxiv.org/abs/1802.0781 0 . — [arXiv:1802.07810](https://arxiv.org/abs/1802.07810) ↗
  79. Neil Vigdor. Apple Card Investigated After Gender Discrimination Complaints - The New York Times. 2019. URL : https://www.nytimes.com/2019/11/10/business/Apple-cred it-card-investigation.html (visited on 02/02/2020). ↗
  80. David Bau et al. “Semantic photo manipulation with a generative image prior”. In: ACM Transactions on Graphics 38.4 (2019). ISSN : 15577368. DOI : 10.1145/3306346.3323023 . URL : http://ganpaint.io/Bau%7B%5C_%7Det%7B%5C_%7Dal%7B%5C_%7DSemanti c%7B%5C_%7DPhoto%7B%5C_%7DManipulation%7B%5C_%7Dpre%20pr… — [doi:10.1145/3306346.3323023](https://doi.org/10.1145/3306346.3323023) ↗
  81. Fred Hohman et al. “Gamut: A design probe to understand how data scientists understand machine learning models”. In: Conference on Human Factors in Computing Systems - Proceedings. Association for Computing Machinery, May 2019. ISBN : 9781450359702. URL : https://www.m icrosoft.com/en-us/research/up… ↗
  82. Isaac Lage et al. “An Evaluation of the Human-Interpretability of Explanation”. In: arXiv (Jan. 2019). arXiv: 1902.00006 . URL : http://arxiv.org/abs/1902.00006 . — [arXiv:1902.00006](https://arxiv.org/abs/1902.00006) ↗
  83. Berkeley J. Dietvorst, Joseph P. Simmons, and Cade Massey. “Algorithm aversion: People erroneously avoid algorithms after seeing them err”. In: Journal of Experimental Psychology: General 144.1 (2015), pp. 114–126. ISSN : 00963445. DOI : 10.1037/xge0000033 . URL : https://repository.upenn.edu/cgi/vi… — [doi:10.1037/xge0000033](https://doi.org/10.1037/xge0000033) ↗
  84. Umang Bhatt et al. “Explainable Machine Learning in Deployment”. In: arXiv (Sept. 2019). URL : https://arxiv.org/abs/1909.06342 . — [arXiv:1909.06342](https://arxiv.org/abs/1909.06342) ↗
  85. Julius Adebayo et al. “Sanity Checks for Saliency Maps”. In: arXiv (Oct. 2018). arXiv: 1810.03292 . URL : http://arxiv.org/abs/1810.03292 . — [arXiv:1810.03292](https://arxiv.org/abs/1810.03292) ↗
  86. Aravindh Mahendran and Andrea Vedaldi. “Salient deconvolutional networks”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 9910 LNCS. Springer Verlag, 2016, pp. 120–135. ISBN : 9783319464657. DOI : 10.1007… — [doi:10.1007/978-3-319-46466-4_8](https://doi.org/10.1007/978-3-319-46466-4_8) find
  87. Pieter-Jan Kindermans et al. “The (Un)reliability of saliency methods”. In: arXiv (Nov. 2017). arXiv: 1711.00867 . URL : http://arxiv.org/abs/1711.00867 . — [arXiv:1711.00867](https://arxiv.org/abs/1711.00867) ↗
  88. Sara Hooker et al. “A Benchmark for Interpretability Methods in Deep Neural Networks”. In: arXiv (June 2018). arXiv: 1806.10758 . URL : http://arxiv.org/abs/1806.1075 8 . — [arXiv:1806.10758](https://arxiv.org/abs/1806.10758) ↗
  89. Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. “BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain”. In: arXiv (Aug. 2017). arXiv: 1708.06733 . URL : http://arxiv.org/abs/1708.06733 . ↗
  90. Juyeon Heo, Sunghwan Joo, and Taesup Moon. “Fooling Neural Network Interpretations via Adversarial Model Manipulation”. In: arXiv (Feb. 2019). arXiv: 1902.02041 . URL : http://arxiv.org/abs/1902.02041 . — [arXiv:1902.02041](https://arxiv.org/abs/1902.02041) ↗
  91. Dylan Slack et al. “How can we fool LIME and SHAP? Adversarial Attacks on Post hoc Explanation Methods”. In: arXiv (Nov. 2019). arXiv: 1911.02508 . URL : http://arxiv.org/abs/1911.02508 . — [arXiv:1911.02508](https://arxiv.org/abs/1911.02508) ↗
  92. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “"Why should i trust you?" Explaining the predictions of any classifier”. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Vol. 13-17-Augu. Association for Computing Machinery, Aug. 2016, pp. 11… — "Why Should I Trust You?": Explaining the Predictions of Any Classifier
  93. B Dimanov et al. “You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods”. In: European Conference on Artificial Intelligence. 2020. URL : http ://ecai2020.eu/accepted-papers/ . find
  94. Kamalika Chaudhuri and Claire Monteleoni. “Privacy-preserving logistic regression”. In: Neural Information Processing Systems Conference. 2018. URL : https://papers.nips.cc/pap er/3486-privacy-preserving-logistic-regression . ↗
  95. Peter Kairouz et al. “Advances and Open Problems in Federated Learning”. In: arXiv (Dec. 2019). arXiv: 1912.04977 . URL : http://arxiv.org/abs/1912.04977 . ↗
  96. Nicholas Carlini et al. “The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks”. In: arXiv (Feb. 2018). arXiv: 1802.08232 . URL : http://arxiv.org/abs/1802.08232 . ↗
  97. H. Brendan McMahan et al. “Learning Differentially Private Recurrent Language Models”. In: arXiv (Oct. 2017). arXiv: 1710.06963 . URL : http://arxiv.org/abs/1710.0696 3 . — [arXiv:1710.06963](https://arxiv.org/abs/1710.06963) ↗
  98. Cynthia Dwork et al. “Calibrating noise to sensitivity in private data analysis”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 3876 LNCS. 2006, pp. 265–284. ISBN : 3540327312. URL : https://p eople.csai… find
  99. Aaron Roth and Michael Kearns. The Ethical Algorithm: The Science of Socially Aware Algorithm Design. Oxford University Press, 2019. find
  100. Rachel Cummings and Deven Desai. “The Role of Differential Privacy in GDPR Compliance”. In: FAT ’18: Proceedings of the Conference on Fairness, Accountability, and Transparency. 2018. URL : https://piret.gitlab.io/fatrec2018/program/fatrec2018-cummings. pdf . ↗
  101. Mihailo Isakov et al. “Survey of Attacks and Defenses on Edge-Deployed Neural Networks”. In: arXiv (Nov. 2019). arXiv: 1911.11932 . URL : http://arxiv.org/abs/1911.1193 2 . — [arXiv:1911.11932](https://arxiv.org/abs/1911.11932) ↗
  102. Nathan Dowlin et al. “CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy”. In: Proceedings of the 33rd International Conference on Machine Learning. 2016. URL : http://sealcrypto.codeplex.com . ↗
  103. Tim Hwang. “Computational Power and the Social Impact of Artificial Intelligence”. In: SSRN Electronic Journal (Apr. 2018). DOI : 10.2139/ssrn.3147971 . arXiv: 1803.08971 . URL : https://arxiv.org/abs/1803.08971 . — [doi:10.2139/ssrn.3147971](https://doi.org/10.2139/ssrn.3147971) ↗
  104. Rich Sutton. The Bitter Lesson. 2019. URL : http://www.incompleteideas.net/Inc Ideas/BitterLesson.html (visited on 02/02/2020). ↗
  105. Rodney Brooks. A Better Lesson. 2019. URL : https://rodneybrooks.com/a-better -lesson/ (visited on 02/02/2020). ↗
  106. Huili Chen et al. “DeepAttest: An end-to-end attestation framework for deep neural networks”. In: Proceedings of the International Symposium on Computer Architecture. Institute of Electrical and Electronics Engineers Inc., June 2019, pp. 487–498. ISBN : 9781450366694. URL : https://www.microsoft.com… ↗
  107. Zahra Ghodsi, Tianyu Gu, and Siddharth Garg. “SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted Cloud”. In: Conference on Neural Information Processing Systems. 2017. URL : https://papers.nips.cc/paper/7053-safetynets-verifiabl e-execution-of-deep-neural-networks-on-an-untrust… ↗
  108. Seyyed Ahmad Javadi et al. “Monitoring Misuse for Accountable ’Artificial Intelligence as a Service’”. In: arXiv (Jan. 2020). arXiv: 2001.09723 . URL : http://arxiv.org/abs/200 1.09723 . — [arXiv:2001.09723](https://arxiv.org/abs/2001.09723) ↗
  109. Frank Mckeen et al. “Innovative Instructions and Software Model for Isolated Execution”. In: HASP ’13: Proceedings of the 2nd International Workshop on Hardware and Architectural Support for Security and Privacy. 2013. DOI : 10.1145/2487726.2488368 . URL : https://www. eit.lth.se/fileadmin/eit/cours… — [doi:10.1145/2487726.2488368](https://doi.org/10.1145/2487726.2488368) find
  110. Jo Van Bulck et al. “Foreshadow: Extracting the Keys to the Intel {SGX} Kingdom with Transient Out-of-Order Execution”. In: 27th {USENIX} Security Symposium ({USENIX} Security 18). Baltimore, MD: {USENIX} Association, Aug. 2018, 991{\\textendash}1008. ISBN : 978-1-939133-04-5. URL : https://www.useni… ↗
  111. Mark Seaborn and Thomas Dullien. Exploiting the DRAM rowhammer bug to gain kernel privileges How to cause and exploit single bit errors. 2015. URL : https://www.blackhat.com/doc s/us-15/materials/us-15-Seaborn-Exploiting-The-DRAM-Rowhammer -Bug-To-Gain-Kernel-Privileges.pdf (visited on 02/02/2020). ↗
  112. Nick Hynes, Raymond Cheng, and Dawn Song. “Efficient Deep Learning on Multi-Source Private Data”. In: arXiv (July 2018). arXiv: 1807.06689 . URL : http://arxiv.org/abs/180 7.06689 . ↗
  113. Christian Priebe et al. “SGX-LKL: Securing the Host OS Interface for Trusted Execution”. In: arXiv (Aug. 2019). arXiv: 1908.11143 . URL : http://arxiv.org/abs/1908.11143 . — [arXiv:1908.11143](https://arxiv.org/abs/1908.11143) ↗
  114. Ehsan Hesamifard et al. “Privacy-preserving Machine Learning as a Service”. In: Proceedings on Privacy Enhancing Technologies. 2018, pp. 123–142. arXiv: 1803.05961 . URL : https://pe tsymposium.org/2018/files/papers/issue3/popets-2018-0024.pdf . — [arXiv:1803.05961](https://arxiv.org/abs/1803.05961) find
  115. Stavros Volos, Kapil Vaswani, and Rodrigo Bruno. “Graviton: Trusted Execution Environments on GPUs”. In: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. OSDI’18. USA: USENIX Association, 2018, pp. 681–696. ISBN : 9781931971478. URL : https://www.microsoft.c… ↗
  116. Florian Tramèr and Dan Boneh. “Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware”. In: arXiv (June 2018). arXiv: 1806.03287 . URL : http://ar xiv.org/abs/1806.03287 . find
  117. Zhongshu Gu et al. “YerbaBuena: Securing Deep Learning Inference Data via Enclave-based Ternary Model Partitioning”. In: arXiv (July 2018). arXiv: 1807.00969 . URL : http://arx iv.org/abs/1807.00969 . — [arXiv:1807.00969](https://arxiv.org/abs/1807.00969) find
  118. Dario Amodei et al. AI and Compute. 2018. URL : https://openai.com/blog/ai-and -compute/ (visited on 02/02/2020). ↗
  119. Lily Hay Newman. Google Is Helping Design an Open Source, Ultra-Secure Chip \| WIRED. 2019. URL : https://www.wired.com/story/open-titan-open-source-secure-enclave/ (visited on 02/02/2020). ↗
  120. Cisco. Cisco Global Cloud Index: Forecast and Methodology, 2016-2021 White Paper. Tech. rep. Cisco, 2018. URL : https://www.cisco.com/c/en/us/solutions/collatera l/service-provider/global-cloud-index-gci/white-paper-c11-738 085.html . ↗
  121. Roy Schwartz et al. “Green AI”. In: arXiv (July 2019). arXiv: 1907.10597 . URL : http://a rxiv.org/abs/1907.10597 .
  122. Christopher Berner et al. “Dota 2 with Large Scale Deep Reinforcement Learning”. In: arXiv (Dec. 2019). arXiv: 1912.06680 . URL : http://arxiv.org/abs/1912.06680 . ↗
  123. J Vitek and T Kalibera. “Repeatability, reproducibility and rigor in systems research”. In: 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT). Oct. 2011, pp. 33–38. DOI : 10.1145/2038642.2038650 . URL : https://ieeexplore.ieee.org/document/6064509 . — [doi:10.1145/2038642.2038650](https://doi.org/10.1145/2038642.2038650) ↗
  124. T Hoefler and R Belli. “Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results”. In: SC ’15: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Nov. 2015, pp. 1– 12. DOI : 10.… — [doi:10.1145/2807591.2807644](https://doi.org/10.1145/2807591.2807644) find
  125. Steve Lohr. At Tech’s Leading Edge, Worry About a Concentration of Power. 2019. URL : https://www.nytimes.com/2019/09/26/technology/ai-computer-expense.htm l (visited on 02/02/2020). ↗
  126. Martijn Rasser et al. The American AI Century: A Blueprint for Action. Tech. rep. Center for a New American Security, Dec. 2019. URL : https://www.cnas.org/publications/repo rts/the-american-ai-century-a-blueprint-for-action . ↗
  127. John Etchemendy and Fei-Fei Li. “National Research Cloud: Ensuring the Continuation of American Innovation”. In: (Mar. 2020). URL : https://hai.stanford.edu/news/natio nal-research-cloud-ensuring-continuation-american-innovation . ↗
  128. Susanne Barth and Menno D.T. de Jong. The privacy paradox - Investigating discrepancies between expressed privacy concerns and actual online behavior - A systematic literature review. Nov. 2017. DOI : 10.1016/j.tele.2017.04.013 . URL : https://www.sciencedirect.c om/science/article/pii/S073658531730… — [doi:10.1016/j.tele.2017.04.013](https://doi.org/10.1016/j.tele.2017.04.013) ↗
  129. Ehsan Toreini et al. “The relationship between trust in AI and trustworthy machine learning technologies”. In: arXiv (Nov. 2019). arXiv: 1912.00782 . URL : http://arxiv.org/ab s/1912.00782 . — [arXiv:1912.00782](https://arxiv.org/abs/1912.00782) ↗
  130. Mike Ananny and Kate Crawford. “Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability”. In: New Media & Society 20.3 (Mar. 2018), pp. 973–989. ISSN : 1461-4448. DOI : 10.1177/1461444816676645 . URL : http://journals.sagepub.com/doi/10.1177/1… — [doi:10.1177/1461444816676645](https://doi.org/10.1177/1461444816676645) ↗
  131. Joshua A. Kroll et al. “Accountable Algorithms”. In: University of Pennsylvania Law Review 165 (2017). URL : https://papers.ssrn.com/sol3/papers.cfm?abstract%7B% 5C_%7Did=2765268%7B%5C\#%7D%7B%5C\#%7D . ↗
  132. Partnership on AI. “Human-AI Collaboration Trust Literature Review - Key Insights and Bibliography”. In: (Sept. 2019). URL : https://www.partnershiponai.org/human-ai-co llaboration-trust-literature-review-key-insights-and-bibliogr aphy/ . ↗
  133. High-Level Expert Group on AI. Ethics guidelines for trustworthy AI. Tech. rep. High-Level Expert Group on AI, 2019. URL : https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai . ↗
  134. Google PAIR. Explainability + Trust. URL : https://pair.withgoogle.com/chapte r/explainability-trust/ (visited on 02/05/2020). ↗
  135. Matthew Arnold et al. “FactSheets: Increasing Trust in AI Services through Supplier’s Declarations of Conformity”. In: arXiv (Aug. 2018). arXiv: 1808.07261 . URL : http://arxiv.org/abs/1808.07261 . ↗
  136. Roger C Mayer, James H Davis, and F David Schoorman. “An Integrative Model of Organizational Trust”. In: The Academy of Management Review 20.3 (1995), pp. 709–734. ISSN : 03637425. URL : http://www.jstor.org/stable/258792 . ↗
  137. D Gambetta. Trust: Making and Breaking Cooperative Relations. B. Blackwell, 1988. ISBN : 978063 1155065. URL : https://books.google.com.tr/books?id=97VmQgAACAAJ . ↗
  138. ANDREW J COE and JANE VAYNMAN. “Why Arms Control Is So Rare”. In: American Political Science Review (2019), pp. 1–14. DOI : 10.1017/S000305541900073X . URL : https://ww w.cambridge.org/core/journals/american-political-science-revi ew/article/why-arms-control-is-so-rare/BAC79354627F72CDDD%20B 102FE82… — [doi:10.1017/s000305541900073x](https://doi.org/10.1017/s000305541900073x) find
  139. Suzanne Massie and Nicholas (ed.) Daniloff. “Suzanne Massie, Trust but Verify: Reagan, Russia, and Me. Rockland, ME: Maine Authors Publishing, 2013. 380 pp.” In: Journal of Cold War Studies 18.4 (2016), pp. 225–228. DOI : 10.1162/JCWS_r_00693 . URL : https://doi.org/10.1162/JCWS%7B%5C_%7Dr%7B%5C_%7D… — [doi:10.1162/jcws_r_00693](https://doi.org/10.1162/jcws_r_00693) ↗
  140. Robin Bloomfield et al. “Disruptive Innovations and Disruptive Assurance: Assuring Machine Learning and Autonomy”. In: Computer 52.9 (Sept. 2019), pp. 82–89. ISSN : 15580814. DOI : 10.1109/MC.2019.2914775 . URL : https://openaccess.city.ac.uk/id/e print/23295/1/bloomfield%7B%5C_%7DpreIEEE%7B%5C_%7Da… — [doi:10.1109/mc.2019.2914775](https://doi.org/10.1109/mc.2019.2914775) ↗
  141. Office for Nuclear Regulation (ONR). ONR Guide. Tech. rep. Office for Nuclear Regulation, 2019. URL : http://www.onr.org.uk/operational/tech%7B%5C_%7Dasst%7B%5C _%7Dguides/ns-tast-gd-051.pdf . ↗
  142. Uber. Uber ATG Safety Case. 2019. URL : https://uberatg.com/safetycase/ (visited on 02/01/2020). ↗
  143. Xingyu Zhao et al. “A Safety Framework for Critical Systems Utilising Deep Neural Networks”. In: arXiv (Mar. 2020). URL : https://arxiv.org/abs/2003.05311 . — [arXiv:2003.05311](https://arxiv.org/abs/2003.05311) ↗
  144. Stephen Edelston Toulmin, joint author Rieke Richard D., and joint author Janik Allan. An introduction to reasoning. New York Macmillan, 1979. ISBN : 0024210307. URL : http://open library.org/books/OL4723061M . find
  145. Donald G. (ed) Brennan. Arms Control, Disarmament, and National Security. G. Braziller, 1961. URL : https://www.jstor.org/stable/pdf/20025353.pdf?seq=1 . ↗
  146. Bernard Brodie. “HEDLEY BULL. The Control of the Arms Race: Disarmament and Arms Control in the Missile Age. (Studies in Inter national Security, II.) Pp. 215. New York: Frederick A. Praeger for the Institute for Strategic Studies, 1961. $3.95”. In: The ANNALS of the American Academy of Political an… — [doi:10.1177/000271626234100116](https://doi.org/10.1177/000271626234100116) find
  147. Richard Dean Burns. Encyclopedia of arms control and disarmament. Encyclopedia of Arms Control and Disarmament v. 1-3. Scribner’s, 1993. ISBN : 9780684192819. URL : https://book s.google.com.tr/books?id=f42yAAAAIAAJ . find
  148. Richard Dean Burns. The Evolution of Arms Control: From Antiquity to the Nuclear Age. Rowman & Littlefield, 2013, p. 264. ISBN : 1442223790. URL : https://rowman.com/ISBN/97814 42223790/The-Evolution-of-Arms-Control-From-Antiquity-to-the-Nuclear-Age . ↗
  149. Jozef Goldblat. Agreements for Arms Control: A Critical Survey. Taylor & Francis, 1982, p. 387. ISBN : 0-85066-229-X. URL : https://www.sipri.org/publications/1982/agr eements-arms-control-critical-survey . ↗
  150. Tim Caughley. Nuclear Disarmament Verification: Survey of Verification Mechanisms. Tech. rep. United Nations Institute for Disarmament Research, 2016, p. 48. URL : https://unidir.org/publication/nuclear-disarmament-verification-survey-verif%2 0ication-mechanisms . ↗
  151. Nancy W. Gallagher. The politics of verification. Johns Hopkins University Press, 2003, p. 311. ISBN : 9780801877391. URL : https://www.tandfonline.com/doi/abs/10.108 0/13523269708404165 . ↗
  152. John H. Herz. “Political Realism and Political Idealism: A Study in Theories and Realities”. In: American Political Science Review 46.2 (June 1952), pp. 550–552. ISSN : 0003-0554. DOI : 10.2307/1950848 . URL : https://www.cambridge.org/core/journals/am erican-political-science-review/article/politic… — [doi:10.2307/1950848](https://doi.org/10.2307/1950848) ↗
  153. Robert Jervis. “Cooperation Under the Security Dilemma”. In: World Politics 30.2 (1978), pp. 167– 214. ISSN : 00438871, 10863338. URL : http://www.jstor.org/stable/2009958 . ↗
  154. Robert Jervis. “Arms Control, Stability, and Causes of War”. In: Political Science Quarterly 108.2 (1993), pp. 239–253. ISSN : 00323195. URL : http://www.jstor.org/stable/2152 010 . ↗
  155. Organisation for the Prohibition of Chemical Weapons. Chemical Weapons Convention. Tech. rep. Organisation for the Prohibition of Chemical Weapons, 1993. URL : https://www.opcw.org/chemical-weapons-convention . ↗
  156. Frederic Joseph Brown. Chemical Warfare: A Study in Restraints. Princeton University Press, 1968, p. 376. ISBN : 9780691069012. URL : http://www.jstor.org/stable/j.ctt 183ph9d . ↗
  157. Carol Barner-Barry. “The Diplomacy of Biological Disarmament: Vicissitudes of a Treaty in Force”. In: Politics and the Life Sciences 9.1 (1990), pp. 149–149. DOI : 10.1017/S0730938400010352 . URL : https://www.cambridge.org/core/journals/politics-and-the-l ife-sciences/article/diplomacy-of-biologica… — [doi:10.1017/s0730938400010352](https://doi.org/10.1017/s0730938400010352) ↗
  158. Kenneth R Rutherford. “The Evolving Arms Control Agenda: Implications of the Role of NGOs in Banning Antipersonnel Landmines”. In: World Politics 53.1 (2000), pp. 74–114. ISSN : 00438871, 10863338. URL : http://www.jstor.org/stable/25054137 . ↗
  159. United Nations. Convention on the Prohibition of the Use, Stockpiling, Production and Transfer of Anti-Personnel Mines and on their Destruction. Oslo, 1997. URL : https://treaties.un.org/Pages/ViewDetails.aspx?src=IND%7B%5C&%7Dmtdsg%7B%5C_%7Dno=X XVI-5%7B%5C&%7Dchapter=26%7B%5C&%7Dclang=%7B%5C_%7De… ↗
  160. United Nations. The Convention on Cluster Munitions. 2008. URL : https://www.cluster c%20onvention.org/ . ↗
  161. Heather Roff and Richard Moyes. “Meaningful Human Control, Artificial Intelligence and Autonomous Weapons”. In: Informal Meeting of Experts on Lethal Autonomous Weapons Systems, UN Convention on Certain Conventional Weapons. 2016. URL : http://www.article36.o rg/wp-content/uploads/2016/04/MHC-AI-and… ↗
  162. Atomic Heritage Foundation. Non-Proliferation, Limitation, and Reduction. 2017. URL : https://www.atomicheritage.org/history/non-proliferation-limitation-and-reduction (visited on 02/02/2020). ↗
  163. United States and USSR. Treaty On The Limitation of Anti-Ballistic Missile Systems. 1972. URL : https://www.nti.org/learn/treaties-and-regimes/treaty-limitat%2 0ion-anti-ballistic-missile-systems-abm-treaty/ . ↗
  164. Rebecca Crootof. “The Killer Robots Are Here: Legal and Policy Implications”. In: Cardozo law review 36 (2015), p. 1837. URL : https://papers.ssrn.com/sol3/papers.cfm?abstract%7B%5C_%7Did=2534567 . ↗
  165. Sean Watts. “Autonomous Weapons: Regulation Tolerant or Regulation Resistant?” In: SSRN Electronic Journal (Nov. 2015). DOI : 10.2139/ssrn.2681283 . URL : https://paper s.ssrn.com/sol3/papers.cfm?abstract%7B%5C_%7Did=2681283 . — [doi:10.2139/ssrn.2681283](https://doi.org/10.2139/ssrn.2681283) find
  166. Paul Scharre. Army of None. W. W. Norton & Company, 2019, p. 448. URL : https://wwnor ton.com/books/Army-of-None/ . find
  167. Kenneth Payne. “Artificial Intelligence: A Revolution in Strategic Affairs?” In: Survival 60.5 (2018), pp. 7–32. DOI : 10.1080/00396338.2018.1518374 . URL : https://doi.org/10.1080/00396338.2018.1518374 . — [doi:10.1080/00396338.2018.1518374](https://doi.org/10.1080/00396338.2018.1518374) ↗
  168. Miles Brundage et al. “The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation”. In: arXiv (Feb. 2018). arXiv: 1802.07228 . URL : http://arxiv.org/abs/1802.07228 . ↗
  169. International Committe of the Red Cross. Report of the ICRC Expert Meeting on ’Autonomous weapon systems: technical, military, legal and humanitarian aspects’. Tech. rep. The Red Cross, 2014. URL : https://www.icrc.org/en/doc/assets/files/2014/expert-m eeting-autonomous-weapons-icrc-report-2014-05-0… ↗
  170. Nehal Bhuta et al., eds. Autonomous weapons systems : law, ethics, policy. Cambridge University Press, 2016, p. 410. ISBN : 9781316607657. URL : https://www.cambridge.org/tr/a cademic/subjects/law/humanitarian-law/autonomous-weapons-syst ems-law-ethics-policy?format=PB . ↗
  171. Greg Allen and Taniel Chan. Artificial Intelligence and National Security. Tech. rep. Belfer Center for Science and International Affairs, Harvard Kennedy School, 2017. URL : https://www.be lfercenter.org/publication/artificial-intelligence-and-nati%2 0onal-security . ↗
  172. Andrew Imbrie and Elsa B. Kania. AI Safety, Security, and Stability Among Great Powers. Tech. rep. Center for Security and Emerging Technology, 2019, p. 25. URL : https://cset.georget own.edu/wp-content/uploads/AI-Safety-Security-and-Stability-A mong-the-Great-Powers.pdf . ↗
  173. Richard Moyes. “Key elements of meaningful human control”. In: Informal Meeting of Experts on Lethal Autonomous Weapons Systems, UN Convention on Certain Conventional Weapons. 2016. URL : http://www.article36.org/wp-content/uploads/2016/04/MHC-20 16-FINAL.pdf . ↗
  174. Heather Roff. “Autonomous weapon systems: Evaluating the capacity for ’meaningful human control’ in weapon review processes”. In: Convention on Certain Conventional Weapons Group of Governmental Experts meeting on Lethal Autonomous Weapons Systems. 2017. URL : http://ww w.article36.org/wp-content/up… find
  175. Haydn Belfield. “Activism by the AI Community: Analysing Recent Achievements and Future Prospects”. In: arXiv (Jan. 2020). arXiv: 2001.06528 . URL : http://arxiv.org/abs/2001.06528 . — [arXiv:2001.06528](https://arxiv.org/abs/2001.06528) ↗
  176. Charli Carpenter. "Lost" Causes: Agenda Vetting in Global Issue Networks and the Shaping of Human Security. Cornell University Press, 2014. ISBN : 9780801448850. URL : http://www.jstor.org/stable/10.7591/j.ctt5hh0r5 . — [doi:10.7591/j.ctt5hh0r5](https://doi.org/10.7591/j.ctt5hh0r5) ↗
  177. Serif Onur Bahcecik. “Civil Society Responds to the AWS: Growing Activist Networks and Shifting Frames”. In: Global Policy 10.3 (2019), pp. 365–369. DOI : 10.1111/1758-5899.12671 . URL : https://onlinelibrary.wiley.com/doi/abs/10.1111/1758-5899.12671 . — [doi:10.1111/1758-5899.12671](https://doi.org/10.1111/1758-5899.12671) ↗
  178. Maaike Verbruggen. “The Role of Civilian Innovation in the Development of Lethal Autonomous Weapon Systems”. In: Global Policy 10.3 (2019), pp. 338–342. URL : https://onlinelib rary.wiley.com/doi/abs/10.1111/1758-5899.12663 . — [doi:10.1111/1758-5899.12663](https://doi.org/10.1111/1758-5899.12663) find
  179. Kerstin Vignard. Manifestos and open letters: Back to the future? 2018. URL : https://thebul l%20etin.org/2018/04/manifestos-and-open-letters-back-to-the-future/ (visited on 02/03/2020). find
  180. Kara Frederick. The civilian private sector: part of a new arms control regime? 2019. URL : https://www.orfonline.org/expert-speak/the-civilian-private-sector-part-of-a-new-arms-control-regime-57345/ (visited on 02/03/2020). ↗
  181. Future of Life Institute. Open Letter on Autonomous Weapons. Tech. rep. Future of Life Institute, 2015. URL : https://futureoflife.org/open-letter-autonomous-we apons/ . ↗
  182. Emanuel Adler. “The emergence of cooperation: national epistemic communities and the international evolution of the idea of nuclear arms control”. In: International Organization 46.1 (1992), pp. 101–145. DOI : 10.1017/S0020818300001466 . URL : https://www.cam bridge.org/core/journals/international-o… — [doi:10.1017/s0020818300001466](https://doi.org/10.1017/s0020818300001466) ↗
  183. Peter M. Haas. Introduction: Epistemic Communities and International Policy Coordination. 1992. DOI : 10.2307/2706951 . URL : https://www.jstor.org/stable/2706951 . — [doi:10.2307/2706951](https://doi.org/10.2307/2706951) ↗
  184. Standard Oil Co. of New Jersey v. United States :: 221 U.S. 1. May 1911. URL : https://supr eme.justia.com/cases/federal/us/221/1/ . find
  185. William E. Kovacic and Carl Shapiro. “Antitrust Policy: A Century of Economic and Legal Thinking”. In: Journal of Economic Perspectives 14.1 (2000), pp. 43–60. URL : https://perma.c c/N6EF-NGD3 . ↗
  186. B. Y. Orbach. “THE ANTITRUST CONSUMER WELFARE PARADOX”. In: Journal of Competition Law and Economics 7.1 (Mar. 2011), pp. 133–164. ISSN : 1744-6414. URL : https://academ ic.oup.com/jcle/article-lookup/doi/10.1093/joclec/nhq019 . — [doi:10.1093/joclec/nhq019](https://doi.org/10.1093/joclec/nhq019) find
  187. Lina M Khan. “Amazon’s Antitrust Paradox”. In: The Yale Law Journal 126 (2017), pp. 710– 805. URL : http://www.internetretailer.com/2016/ol/28/amazon-sale s-climb-22 . ↗
  188. Astead W. Herndon. Elizabeth Warren Proposes Breaking Up Tech Giants Like Amazon and Facebook. 2019. URL : https://www.nytimes.com/2019/03/08/us/politics/eli zabeth-warren-amazon.html (visited on 02/03/2020). ↗
  189. Arizona v. Maricopa County Med. Soc’y :: 457 U.S. 332. June 1982. URL : https://supreme.justia.com/cases/federal/us/457/332/ . ↗
  190. Practical Law Antitrust. US Antitrust Laws: Overview. URL : https://uk.practicallaw.thomsonreuters.com/9-204-0472 . ↗
  191. State Oil Co. v. Khan :: 522 U.S. 3. Nov. 1997. URL : https://supreme.justia.com/c ases/federal/us/522/3/ . ↗
  192. FTC v. Indiana Fed’n of Dentists :: 476 U.S. 447. June 1986. URL : https://supreme.just ia.com/cases/federal/us/476/447/ . ↗
  193. National Soc’y of Prof. Engineers v. United States :: 435 U.S. 679. Apr. 1978. URL : https://s upreme.justia.com/cases/federal/us/435/679/ . find
  194. Federal Trade Commision and U.S. Department of Justice. Antitrust Guidelines for Collaborations Among Competitors. Tech. rep. Federal Trade Commision, Apr. 2000. URL : https://www.ft c.gov/sites/default/files/documents/public%7B%5C_%7Devents/joi nt-venture-hearings-antitrust-guidelines-collaboration… ↗
  195. Osbert Bastani, Carolyn Kim, and Hamsa Bastani. “Interpreting Blackbox Models via Model Extraction”. In: arXiv (May 2017). arXiv: 1705.08504 . URL : http://arxiv.org/abs/1 705.08504 . — [arXiv:1705.08504](https://arxiv.org/abs/1705.08504) ↗
  196. Luca Pulina and Armando Tacchella. “An Abstraction-Refinement Approach to Verification of Artificial Neural Networks”. In: CEUR Workshop Proceedings. 2010, pp. 243–257. URL : http://link.springer.com/10.1007/978-3-642-14295-6%7B%5C_%7D24 . — [doi:10.1007/978-3-642-14295-6%7b%5c_%7d24](https://doi.org/10.1007/978-3-642-14295-6%7b%5c_%7d24) ↗
  197. Changliu Liu et al. “Algorithms for Verifying Deep Neural Networks”. In: arXiv (Mar. 2019). arXiv: 1903.06758 . URL : http://arxiv.org/abs/1903.06758 . — [arXiv:1903.06758](https://arxiv.org/abs/1903.06758) ↗
  198. Daniel Selsam, Percy Liang, and David L. Dill. “Developing Bug-Free Machine Learning Systems With Formal Mathematics”. In: arXiv (June 2017). arXiv: 1706.08605 . URL : http://arx iv.org/abs/1706.08605 . find
  199. C Hutchison et al. “Robustness Testing of Autonomy Software”. In: 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP). May 2018, pp. 276–285. URL : https://users.ece.cmu.edu/%7B~%7Dkoopman/p ubs/hutchison18%7B%5C_%7Dicse%7B%5C_%7Dro… ↗
  200. Joe Gibbs Politz et al. “Python: The Full Monty”. In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications. OOP-SLA ’13. New York, NY, USA: Association for Computing Machinery, 2013, pp. 217–232. ISBN : 9781450323741. DOI : 10.1… — [doi:10.1145/2509136.2509536](https://doi.org/10.1145/2509136.2509536) find
  201. Philippa Anne Gardner, Sergio Maffeis, and Gareth David Smith. “Towards a Program Logic for JavaScript”. In: SIGPLAN Not. 47.1 (Jan. 2012), pp. 31–44. ISSN : 0362-1340. URL : https://doi.org/10.1145/2103621.2103663 . — [doi:10.1145/2103621.2103663](https://doi.org/10.1145/2103621.2103663) ↗
  202. Lindsey Kuper et al. “Toward Scalable Verification for Safety-Critical Deep Networks”. In: arXiv (Jan. 2018). arXiv: 1801.05950 . URL : http://arxiv.org/abs/1801.05950 . — [arXiv:1801.05950](https://arxiv.org/abs/1801.05950) ↗
  203. Pang Wei Koh and Percy Liang. “Understanding Black-box Predictions via Influence Functions”. In: arXiv (Mar. 2017). arXiv: 1703.04730 . URL : http://arxiv.org/abs/1703.04 730 . ↗
  204. Rajiv Khanna et al. “Interpreting Black Box Predictions using Fisher Kernels”. In: arXiv (Oct. 2018). arXiv: 1810.10118 . URL : http://arxiv.org/abs/1810.10118 . — [arXiv:1810.10118](https://arxiv.org/abs/1810.10118) ↗
  205. Boris Sharchilev et al. “Finding Influential Training Samples for Gradient Boosted Decision Trees”. In: arXiv (Feb. 2018). arXiv: 1802 . 06640 . URL : http://arxiv.org/abs/1 802.06640 . ↗
  206. Chih-Kuan Yeh et al. “Representer Point Selection for Explaining Deep Neural Networks”. In: Conference on Neural Information Processing Systems. 2018. URL : https://papers.nips.cc/paper/8141-representer-point-selection-for-explaining-deep-ne ural-networks.pdf . ↗
  207. Quanshi Zhang et al. “Interpreting CNNs via Decision Trees”. In: IEEE Conference on Computer Vision and Pattern Recognition. 2019. URL : https://arxiv.org/abs/1802.00121 . ↗
  208. Himabindu Lakkaraju, Stephen H. Bach, and Jure Leskovec. “Interpretable decision sets: A joint framework for description and prediction”. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Vol. 13-17-Augu. Association for Computing Machinery, Aug. 2016… find
  209. Sarah Tan et al. “Learning Global Additive Explanations for Neural Nets Using Model Distillation”. In: arXiv (Jan. 2018). arXiv: 1801.08640 . URL : http://arxiv.org/abs/180 1.08640 . — [arXiv:1801.08640](https://arxiv.org/abs/1801.08640) ↗
  210. Quanshi Zhang et al. “Interpreting CNN Knowledge via an Explanatory Graph”. In: arXiv (Aug. 2017). arXiv: 1708.01785 . URL : http://arxiv.org/abs/1708.01785 . — [arXiv:1708.01785](https://arxiv.org/abs/1708.01785) ↗
  211. Wieland Brendel and Matthias Bethge. “Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet”. In: arXiv (Mar. 2019). arXiv: 1904.00760 . URL : http://arxiv.org/abs/1904.00760 . — [arXiv:1904.00760](https://arxiv.org/abs/1904.00760) ↗
  212. Quanshi Zhang, Ying Nian Wu, and Song-Chun Zhu. “Interpretable Convolutional Neural Networks”. In: arXiv (Oct. 2017). arXiv: 1710.00935 . URL : http://arxiv.org/abs/17 10.00935 . — [arXiv:1710.00935](https://arxiv.org/abs/1710.00935) ↗
  213. Xi Chen et al. “InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets”. In: arXiv (June 2016). arXiv: 1606.03657 . URL : http://arxi v.org/abs/1606.03657 . find
  214. Irina Higgins et al. “β-VAE: LEARNING BASIC VISUAL CONCEPTS WITH A CONSTRAINED VARI-ATIONAL FRAMEWORK”. In: International Conference on Learning Representations. 2017. URL : https://openreview.net/references/pdf?id=Sy2fzU9gl . ↗
  215. Abhinav Verma et al. “Programmatically Interpretable Reinforcement Learning”. In: arXiv (Apr. 2018). arXiv: 1804.02477 . URL : http://arxiv.org/abs/1804.02477 . — [arXiv:1804.02477](https://arxiv.org/abs/1804.02477) ↗
  216. Andrew Slavin Ross and Finale Doshi-Velez. “Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients”. In: arXiv (Nov. 2017). arXiv: 1711.09404 . URL : http://arxiv.org/abs/1711.09404 . — [arXiv:1711.09404](https://arxiv.org/abs/1711.09404) ↗
  217. Saleema Amershi et al. “Modeltracker: Redesigning performance analysis tools for machine learning”. In: Conference on Human Factors in Computing Systems - Proceedings. Vol. 2015-April. Association for Computing Machinery, Apr. 2015, pp. 337–346. ISBN : 9781450331456. DOI : 10.1145/2702123.2702509 .… — [doi:10.1145/2702123.2702509](https://doi.org/10.1145/2702123.2702509) find
  218. Tongshuang Wu et al. “Errudite: Scalable, Reproducible, and Testable Error Analysis”. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019, pp. 747–763. URL : https://www .aclweb.org/antholo… find
  219. Jesse Vig. “A Multiscale Visualization of Attention in the Transformer Model”. In: arXiv (June 2019). arXiv: 1906.05714 . URL : http://arxiv.org/abs/1906.05714 . ↗
  220. David Bau et al. “GaN dissection: Visualizing and understanding generative adversarial networks”. In: 7th International Conference on Learning Representations, ICLR 2019. International Conference on Learning Representations, ICLR, 2019. arXiv: 1811.10597 . URL : https://arxiv.org/abs/1811.10597 . — [arXiv:1811.10597](https://arxiv.org/abs/1811.10597) ↗
  221. Shan Carter et al. “Activation Atlas”. In: Distill 4.3 (Mar. 2019). ISSN : 2476-0757. URL : https ://distill.pub/2019/activation-atlas . find
  222. Jost Tobias Springenberg et al. “Striving for Simplicity: The All Convolutional Net”. In: arXiv (Dec. 2014). arXiv: 1412.6806 . URL : http://arxiv.org/abs/1412.6806 . — [arXiv:1412.6806](https://arxiv.org/abs/1412.6806) ↗
  223. Matthew D. Zeiler and Rob Fergus. “Visualizing and understanding convolutional networks”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, 2014, pp. 818–833. ISBN : 9783319105895. DOI : 10.1007/…
  224. Ruth Fong and Andrea Vedaldi. “Interpretable Explanations of Black Boxes by Meaningful Perturbation”. In: IEEE International Conference on Computer Vision. Apr. 2017. arXiv: 1704.03296 . URL : http://arxiv.org/abs/1704.03296%20http://dx.doi.org/10.110 9/ICCV.2017.371 . — [arXiv:1704.03296](https://arxiv.org/abs/1704.03296) find
  225. Ruth Fong, Mandela Patrick, and Andrea Vedaldi. “Understanding Deep Networks via Extremal Perturbations and Smooth Masks”. In: IEEE International Conference on Computer Vision. 2019. URL : https://arxiv.org/abs/1910.08485 . — [arXiv:1910.08485](https://arxiv.org/abs/1910.08485) ↗
  226. Piotr Dabkowski and Yarin Gal. “Real Time Image Saliency for Black Box Classifiers”. In: Advances in Neural Information Processing Systems 30. Ed. by I Guyon et al. Curran Associates, Inc., 2017, pp. 6967–6976. URL : http://papers.nips.cc/paper/7272-real-time-imag e-saliency-for-black-box-classifier… ↗
  227. Vitali Petsiuk, Abir Das, and Kate Saenko. “RISE: Randomized Input Sampling for Explanation of Black-box Models”. In: arXiv (June 2018). arXiv: 1806.07421 . URL : http://arxiv.org/abs/1806.07421 . — [arXiv:1806.07421](https://arxiv.org/abs/1806.07421) ↗
  228. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. “Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps”. In: arXiv (Dec. 2013). arXiv: 1312.6034 . URL : http://arxiv.org/abs/1312.6034 . ↗
  229. Himabindu Lakkaraju et al. “Interpretable & Explorable Approximations of Black Box Models”. In: Fairness, Accountability, and Transparency in Machine Learning. 2017. arXiv: 1707.01154v1 . URL : https://arxiv.org/abs/1707.01154 . ↗
  230. Anh Nguyen et al. “Synthesizing the preferred inputs for neurons in neural networks via deep generator networks”. In: Advances in Neural Information Processing Systems 29. Ed. by D D Lee et al. Curran Associates, Inc., 2016, pp. 3387–3395. URL : http://papers.nips.cc/pape r/6519-synthesizing-the-pre… ↗
  231. David Bau et al. “Network Dissection: Quantifying Interpretability of Deep Visual Representations”. In: arXiv (Apr. 2017). arXiv: 1704.05796 . URL : http://arxiv.org/abs/170 4.05796 . ↗
  232. Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. “Feature Visualization”. In: Distill 2.11 (Nov. 2017). ISSN : 2476-0757. DOI : 10.23915/distill.00007 . URL : https://distill.pub/2017/feature-visualization . — [doi:10.23915/distill.00007](https://doi.org/10.23915/distill.00007) ↗
  233. Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. “Deep Image Prior”. In: The IEEE Conference on Computer Vision and Pattern Recognition. 2018. URL : https://arxiv.org/ab s/1711.10925 . ↗
  234. Chris Olah et al. “The Building Blocks of Interpretability”. In: Distill 3.3 (Mar. 2018). ISSN : 2476-0757. DOI : 10.23915/distill.00010 . URL : https://distill.pub/2018/bu ilding-blocks . — [doi:10.23915/distill.00010](https://doi.org/10.23915/distill.00010) ↗
  235. Ari S. Morcos et al. “On the importance of single directions for generalization”. In: arXiv (Mar. 2018). arXiv: 1803.06959 . URL : http://arxiv.org/abs/1803.06959 . — [arXiv:1803.06959](https://arxiv.org/abs/1803.06959) ↗
  236. Karel Lenc and Andrea Vedaldi. “Understanding image representations by measuring their equivariance and equivalence”. In: arXiv (Nov. 2014). arXiv: 1411.5908 . URL : http://arxiv.org/abs/1411.5908 . ↗
  237. Laurens Van Der Maaten and Geoffrey Hinton. “Visualizing Data using t-SNE”. In: Journal of Machine Learning Research 9 (2008), pp. 2579–2605. URL : http://www.jmlr.org/pap ers/volume9/vandermaaten08a/vandermaaten08a.pdf . ↗
  238. Carrie J. Cai et al. “Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making”. In: arXiv (Feb. 2019). arXiv: 1902.02960 . URL : http://arxiv.org/abs/1902.02960 . — [arXiv:1902.02960](https://arxiv.org/abs/1902.02960) ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/brundage-2020-trustworthy-ai-development-mechanisms/