Elamur Research⌕
☰
home / papers / 2026 / AttentionApp: An Interactive Tool for Analyzing Transformer Attention Patterns in Portuguese

AttentionApp: An Interactive Tool for Analyzing Transformer Attention Patterns in Portuguese

Paper · 2026 · Ricardo G. Oliveira and Daniela Barreiro Claro

Summary

Transformer models perform well on Portuguese, but it is still unclear how they represent linguistic structure. Most attention-visualization software also assumes English. This demo paper introduces AttentionApp, an open-source Python application with a web interface. Users can enter any Portuguese sentence and examine the self-attention weights of a Portuguese pre-trained model such as BERTimbau.

The system has three parts:

  • a preprocessing module that segments text, applies BERT-style subword tokenization and aligns tokens;
  • an extraction module that loads the model and collects attention from every layer and head;
  • an interactive viewer for token-to-token attention matrices, with layer and head selection.

The paper reports no quantitative evaluation; its contribution is the tool itself. The authors intend it for exploratory, qualitative work, such as checking whether specific heads reflect syntactic dependencies or word-order effects. They also suggest teaching uses. They argue that a Portuguese-focused tool helps close the interpretability gap between English and less-resourced languages.

Key points

  • The authors are from the FORMAS Research Center, UFBA (Brazil). The paper appeared in the PROPOR 2026 Demonstration Track (Vol. 2, pp. 18–20).
  • The reference model is BERTimbau. The architecture is modular, so other models or linguistic annotations can be added later.
  • The tool runs locally on an ordinary laptop and needs no specialized hardware.
  • The source code is on GitHub (github.com/rgoliveirati/attention app), with installation instructions, dependency specifications and example configurations.
  • The work draws on previous studies of attention heads by Voita et al. (2019) and Clark et al. (2019), as well as an earlier investigation of syntactic patterns in Portuguese attention (Oliveira et al., 2025).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Ricardo G. Oliveira and Daniela Barreiro Claro. 2026. AttentionApp: An Interactive Tool for Analyzing Transformer Attention Patterns in Portuguese. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 2.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{oliveira2026attentionapp,
  title = {{AttentionApp: An Interactive Tool for Analyzing Transformer Attention Patterns in Portuguese}},
  author = {Ricardo G. Oliveira and Daniela Barreiro Claro},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 2},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-2.6/}
}

References

7 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT’s attention. In Proceedings of the BlackboxNLP Workshop at EMNLP 2019, pages 276–286. Association for Computational Linguistics. find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  3. Ricardo G. Oliveira. 2025. Attention app: Interactive visualization and analysis of attention heads in transformer models. https://github.com/rgoliveirati/attention_app. Source code repository. ↗
  4. Ricardo G. Oliveira, Daniela B. Claro, and Rerisson Cavalcante. 2025. Syntactic analysis in transformers through attention heads. In Anais do XVI Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL 2025), pages 295–306, Fortaleza, CE, Brazil. Sociedade Brasileira de Computaçã… find
  5. Fabio Souza, Rodrigo F. Nogueira, and Roberto A. Lotufo. 2020. BERTimbau: Pretrained BERT models for brazilian portuguese. In Intelligent Systems: 9th Brazilian Conference on Intelligent Systems (BRACIS 2020), Part I, volume 12066 of Lecture Notes in Computer Science, pages 403–417, Rio Grande, Braz… find
  6. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, pages 5998–6008. Curran Associates, Inc.
  7. Elena Voita, Jean Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of ACL 2019, pages 5797–5808.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/oliveira-2026-attentionapp-interactive-tool-analyzing-transformer/