Elamur Research⌕
☰
home / papers / 2006 / Get out the vote: Determining support or opposition from Congressional floor-debate transcripts

Get out the vote: Determining support or opposition from Congressional floor-debate transcripts

Summary

The paper asks whether individual speeches in U.S. Congressional floor debates can be automatically labeled as supporting or opposing the bill under discussion. This is a document-level sentiment-polarity task. It is hard because the language is rich and digressive, and persuasive speakers often argue through evidence rather than stating a stance outright. The authors build a corpus, released as the ConVote data, from 2005 House of Representatives transcripts on GovTrack. Labels come for free from roll-call voting records, so no manual annotation is needed. Their method follows Blum and Chawla (2001). SVM scores for each speech are combined with weighted links between speeches, both same-speaker links and links found by a simple learned agreement detector. A minimum-cut-style optimization then assigns labels jointly across a debate. Adding agreement information clearly beats classifying speeches in isolation on both development and test data. The work matters because it shows that relations within a discussion can be a strong extra signal for opinion classification in political and other conversational texts.

Key points

  • Data source and scope: The corpus is built from GovTrack transcripts of 2005 House floor debates (3268 transcript pages) together with roll-call votes.
    • Only "controversial" bills were kept, meaning the losing side produced at least 20% of the speeches.
    • Each speech segment is labeled with its speaker's yea/nay vote.
  • Corpus size and split: There are 3857 speech segments across 53 debates, split roughly 70/20/10 by debate.
    • Train: 2740 segments in 38 debates.
    • Test: 860 segments in 10 debates.
    • Development: 257 segments in 5 debates.
  • Cleaning steps: Formulaic time-yielding utterances were removed as off-topic. Segments mentioning "amendment" were also dropped, because they may express a view on the amendment rather than on the bill itself.
  • Model: The individual-document component is an SVM. Its signed distance to the decision plane is mapped into a preference score, with clipping at ±2σ. These scores are combined with link strengths in a graph optimization that is provably efficient to solve.
  • Main findings:
    • Treating agreement as soft, finite-weight preferences gives substantial gains.
    • Forcing linked speeches to share a label, either through infinite-weight links or concatenation, made results worse on both dev and test.
    • Party affiliation was deliberately not used, to keep the method domain-general.

Interesting ideas

  • I1. Free labels from behavioral records: Gold labels are derived from an existing public record of real behavior (votes) instead of human annotation. Similar setups could use other public outcome records paired with texts, such as parliamentary votes or court decisions, to build large labeled datasets cheaply.
  • I2. Soft rather than hard relational constraints: Agreement signals from an imperfect detector are encoded as weighted preferences, not mandatory constraints. This lets strong individual evidence override a noisy link. The idea suits any pipeline where relation extraction is error-prone.
  • I3. Simple collective classification via graph cuts: Per-item classifier scores are combined with pairwise "same label" preferences in an easily solvable optimization. This is a lightweight alternative to complex collective models and can wrap around any existing classifier.
  • I4. Splitting data by debate: All segments of one debate stay in the same split. This prevents leakage and matches the goal of using discussion context, and the same care applies to threads, documents, or speakers in other corpora.
  • I5. Not tuning on the test set after the fact: The authors kept their original dev/test split even when the best dev method did not win on test. They report the robust conclusion rather than reshuffling the data, which is a useful evaluation discipline to copy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

144 papers of our library cite it (of the 140884 whose bibliographies we read); 20 of them have a page here, 124 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{thomas2006get,
  title = {{Get out the vote: Determining support or opposition from Congressional floor-debate transcripts}},
  author = {Thomas, Matt and Pang, Bo and Lee, Lillian},
  booktitle = {Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing},
  year = {2006},
  url = {https://aclanthology.org/W06-1639/}
}

References

47 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Agarwal, P. Bhattacharyya. 2005. Sentiment analysis: A new approach for effective use of linguistic knowledge and exploiting similarities in a set of documents to be classified. In Proceedings of the International Conference on Natural Language Processing (ICON). find
  2. R. Agrawal, S. Rajagopalan, R. Srikant, Y. Xu. 2003. Mining newsgroups using networks arising from social behavior. In Proceedings of WWW, 529–535. find
  3. N. Bansal, A. Blum, S. Chawla. 2002. Correlation clustering. In Proceedings of the Symposium on Foundations of Computer Science (FOCS), 238– 247. Journal version in Machine Learning Journal, special issue on theoretical advances in data clustering, 56(1-3):89–113 (2004). find
  4. R. Barzilay, M. Lapata. 2005. Collective content selection for concept-to-text generation. In Proceedings of HLT/EMNLP, 331–338. find
  5. A. Blum, S. Chawla. 2001. Learning from labeled and unlabeled data using graph mincuts. In Proceedings of ICML, 19–26. find
  6. C. Cardie, C. Farina, T. Bruce, E. Wagner. 2006. Using natural language processing to improve eRulemaking. In Proceedings of Digital Government Research (dg.o). find
  7. V. Carvalho, W. W. Cohen. 2005. On the collective classification of email “speech acts”. In Proceedings of SIGIR, 345–352. find
  8. W. Daelemans, V. Hoste. 2002. Evaluation of machine learning methods for natural language processing tasks. In Proceedings of the Third International Conference on Language Resources and Evaluation (LREC), 755–760. find
  9. S. Das, M. Chen. 2001. Yahoo! for Amazon: Extracting market sentiment from stock message boards. In Proceedings of the Asia Pacific Finance Association Annual Conference (APFA). find
  10. K. Dave, S. Lawrence, D. M. Pennock. 2003. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. In Proceedings of WWW, 519–528. find
  11. M. Efron. 2004. Cultural orientation: Classifying subjective documents by cociation [sic] analysis. In Proceedings of the AAAI Fall Symposium on Style and Meaning in Language, Art, Music, and Design, 41–48. find
  12. A. Esuli. 2006. Sentiment classification bibliography. liinwww.ira.uka.de/bibliography/Misc/Sentiment.html . find
  13. M. Galley, K. McKeown, J. Hirschberg, E. Shriberg. 2004. Identifying agreement and disagreement in conversational speech: Use of Bayesian networks to model pragmatic dependencies. In Proceedings of the 42nd ACL, 669–676. find our card
  14. L. Getoor, N. Friedman, D. Koller, B. Taskar. 2002. Learning probabilistic models of relational structure. Journal of Machine Learning Research, 3:679–707. Special issue on the Eighteenth ICML. find
  15. A. B. Goldberg, J. Zhu. 2006. Seeing stars when there aren’t many stars: Graph-based semi-supervised learning for sentiment categorization. In TextGraphs: HLT/NAACL Workshop on Graph-based Algorithms for Natural Language Processing. find
  16. G. Grefenstette, Y. Qu, J. G. Shanahan, D. A. Evans. 2004. Coupling niche browsers and affect analysis for an opinion mining application. In Proceedings of RIAO. find
  17. M. Hearst. 1992. Direction-based text interpretation as an information access refinement. In P. Jacobs, ed., Text-Based Intelligent Systems, 257–274. Lawrence Erlbaum Associates. find
  18. D. Hillard, M. Ostendorf, E. Shriberg. 2003. Detection of agreement vs. disagreement in meetings: Training with unlabeled data. In Proceedings of HLT-NAACL. find
  19. T. Joachims. 2003. Transductive learning via spectral graph partitioning. In Proceedings of ICML, 290– 297. find
  20. R. I. Kondor, J. D. Lafferty. 2002. Diffusion kernels on graphs and other discrete input spaces. In Proceedings of ICML, 315–322. find
  21. N. Kwon, S. Shulman, E. Hovy. 2006. Multidimensional text analysis for eRulemaking. In Proceedings of Digital Government Research (dg.o). find
  22. J. Lafferty, A. McCallum, F. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML, 282–289. find
  23. M. Laver, K. Benoit, J. Garry. 2003. Extracting policy positions from political texts using words as data. American Political Science Review. find
  24. W. Lehnert, C. Cardie, E. Riloff. 1990. Analyzing research papers using citation sentences. In Program of the Twelfth Annual Conference of the Cognitive Science Society, 511–18. find
  25. D. Marcu. 2000. The theory and practice of discourse parsing and summarization. MIT Press. find
  26. A. McCallum, B. Wellner. 2004. Conditional models of identity uncertainty with application to noun coreference. In Proceedings of NIPS. find
  27. T. Mullen, R. Malouf. 2006. A preliminary investigation into sentiment analysis of informal political discourse. In Proceedings of the AAAI Symposium on Computational Approaches to Analyzing Weblogs, 159–162. find
  28. A. Munson, C. Cardie, R. Caruana. 2005. Optimizing to arbitrary NLP metrics using ensemble selection. In Proceedings of HLT-EMNLP, 539–546. find
  29. J. Neville, D. Jensen. 2000. Iterative classification in relational data. In Proceedings of the AAAI Workshop on Learning Statistical Models from Relational Data, 13–20. find
  30. B. Pang, L. Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the ACL, 271–278. find our card
  31. B. Pang, L. Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the ACL. find our card
  32. B. Pang, L. Lee, S. Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of EMNLP, 79–86. find our card
  33. S. Purpura, D. Hillard. 2006. Automated classification of congressional legislation. In Proceedings of Digital Government Research (dg.o). find
  34. W. Sack. 1994. On the computation of point of view. In Proceedings of AAAI, pg. 1488. Student abstract. find
  35. S. Shulman, D. Schlosberg. 2002. Electronic rulemaking: New frontiers in public participation. Prepared for the Annual Meeting of the American Political Science Association. find
  36. S. Shulman, J. Callan, E. Hovy, S. Zavestoski. 2005. Language processing technologies for electronic rulemaking: A project highlight. In Proceedings of Digital Government Research (dg.o), 87–88. find
  37. S. S. Smith, J. M. Roberts, R. J. Vander Wielen. 2005. The American Congress. Cambridge University Press, fourth edition. find
  38. A. Stolcke, N. Coccaro, R. Bates, P. Taylor, C. Van Ess-Dykema, K. Ries, E. Shriberg, D. Jurafsky, R. Martin, M. Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. find our card
  39. B. Taskar, P. Abbeel, D. Koller. 2002. Discriminative probabilistic models for relational data. In Proceedings of UAI, Edmonton, Canada. find
  40. B. Taskar, C. Guestrin, D. Koller. 2003. Max-margin Markov networks. In Proceedings of NIPS. find
  41. B. Taskar, V. Chatalbashev, D. Koller. 2004. Learning associative Markov networks. In Proceedings of ICML. find
  42. S. Teufel, M. Moens. 2002. Summarizing scientific articles: Experiments with relevance and rhetorical status. Computational Linguistics, 28(4):409–445. find our card
  43. P. Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the ACL, 417–424. find our card
  44. J. M. Wiebe, W. J. Rapaport. 1988. A computational theory of perspective and reference in narrative. In Proceedings of the ACL, 131–138. find
  45. J. M. Wiebe. 1994. Tracking point of view in narrative. Computational Linguistics, 20(2):233–287. find
  46. H. Yang, J. Callan. 2005. Near-duplicate detection for eRulemaking. In Proceedings of Digital Government Research (dg.o). find
  47. J. Zhu. 2005. Semi-supervised learning literature survey. Computer Sciences Technical Report TR 1530, University of Wisconsin-Madison. Available at http://www.cs.wisc.edu/∼jerryzhu/pub/ssl survey.pdf ; has been updated since the initial 2005 version. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/thomas-2006-get-out-vote-determining-support/