Rudman, Joseph, Department of English, Carnegie Mellon University, USA, jr20@heps.phys.cmu.edu
Introduction
This paper discusses the controversy over the authorship of the Federalistpapers as seen and studied by traditional historians and by over 100 non-traditional authorship attribution practitioners.
The Federalist papers were written during the years 1787 and 1788 by Alexander Hamilton, John Jay, and James Madison. These 85 ‘propaganda’ tracts were intended to help get the U.S. Constitution ratified. They were all published anonymously under the pseudonym, ‘Publius.’ The authorship of certain of the Federalist essays was disputed from the beginning. Both Hamilton and Madison produced lists that claimed some of the same papers. There followed a series of lists, some claiming authorship for Madison and some for Hamilton.
The consensus of traditional scholarship, seconded by Mosteller and Wallace, allocates the papers: Hamilton 51 (1, 6-9, 11-13, 15-17, 21-36, 59-61, 65-85); Madison 29 (10, 14, 18-20, 37-58, 62, 63); Jay 5 (2-5, 64).
In 2005, I presented a paper at the ACH/ALLC conference, ‘The Non-Traditional Case for the Authorship of the Federalist Papers: A House Built on Sand?’’ After 7 more years of ‘traditional’’ research and the addition of over 70 non-traditional studies of the Federalist, I am able to remove the question mark and put forth a reasoned argument that many, if not all, of the twelve disputed papers are a collaboration and not written solely by Madison, as the consensus of traditional scholarship and non-traditional authorship studies claim.
In 1964, Mosteller and Wallace, building on the earlier unpublished work of Frederick Williams and Frederick Mosteller, published their non-traditional authorship attribution study, Inference and Disputed Authorship: The Federalist. It is arguably the most famous and well respected of all of the non-traditional attribution studies. Since then, well over a thousand papers have cited the Mosteller and Wallace work and over 100 non-traditional practitioners have analyzed and/or conducted variations of the original study.
Mosteller and Wallace set the boundary conditions for the subsequent non-traditional work – e.g., not using the Jay articles as a control. Most of these later practitioners do not select or prepare the input text as carefully as Mosteller and Wallace – and their selection and preparation was not as rigorous and complete as it should have been – as we will see.
Problems with the non-traditional case
There are many problems with the Mosteller and Wallace study and with the over 90 other non-traditional studies that cast strong doubts on their results:
(1.) The Federalist Papers
A crucial step of any non-traditional authorship study is to obtain a ‘starting text.’ As a rule, the closest text to the ‘final’ holograph should be found and used. Every step away from that holograph introduces systematic errors.
(2.) The Hamilton Texts
Mosteller and Wallace go outside of the Federalist papers to construct their block of Hamilton tracts.
(3.) The Madison Texts
Mosteller and Wallace also go outside of the Federalist papers to construct their block of Madison tracts. In the case of the Madison block, there is a 20 year difference in their production dates. And what is worse – some of these outside essays have been shown to be not by Madison.
(4.) The Control Texts
There is not one non-traditional study that uses any meaningful controls. The Mosteller and Wallace and Wachal use of a ‘training set’ is not a control. The closest test to a necessary control is found in the ‘validation’ of four Hamilton papers (79, 80, 82, 85) and other sets by Mosteller and Wallace.
More problems – Arriving at the analysis texts
(A) Background and Definitions
No attribution practitioner should question the fact that valid texts are needed if valid results are to be obtained. No matter how sophisticated the statistical analysis is, a bad text invalidates the results.
(B) Unediting
- Editorial Interpolation
- Printer Interpolation
(C) De-editing
- Quotes
- Plagiarism
- Collaboration
- Graphs and Numbers
- Guide Words
- Foreign Languages
- Translations
(D) Editing
- Encoding the Text
- Regularizing
- Lemmatizing
There are many significant weaknesses in the text preparation part of the Mosteller and Wallace study. For example, they state that:
- They ignore the extent of the editing done by the other man [i.e. Hamilton or Madison] and by all of the ‘outside’ editors.
- They did not disambiguate words – e.g. the personal pronoun ‘I’ is teated the same as the Roman numeral ‘I’ – the noun ‘abuse’ is treated exactly like the verb ‘abuse’.
- They do not publish complete details of their ‘little book of decisions’ or the rational behind any of these decisions. –
- They do not use the newspaper versions of the first 77 papers.
- They typed the text onto cards to be read by the computer, but for reasons of ‘economics’ they used hand counts for much of their study. They write about the many problems this introduces but do not assign any systematic errors – e.g. (1) they show the differences in the hand counts vs. the machine counts, (2) they tell us that their proofreaders missed missing words, repeated lines, and single word repetitions.
- Missing bibliographical sources – e.g. Smyth and Wachal.
One of the guiding principles of any scientific study is ‘reproducibility.’ Any other practitioner should be able to reproduce a given study and get identical results. None of the over 90 Federalist studies mentioned in this paper give anywhere near the information needed – a fatal flaw.
Time will not allow for a detailed explication of these studies. In essence, they are all fatally flawed – many do not indicate which edition they used, most either did no unediting, de-editing, or editing – or they fail to say if they did anything to the text.
The case for ‘collaboration’
There can be no doubt that the Federalist project was a collaboration – a collaboration on many levels – but the depth of that collaboration is what is in question.
We know that Publius’ Federalist series was the product of three men – Hamilton, Madison, and Jay. No one disputes this, as long as specific papers are not discussed.
Most scholars agree that Federalist 18, 19, and 20 were written jointly by Hamilton and Madison. Exactly what parts were by which man is not agreed upon.
There is strong evidence that Hamilton and Madison also had joint hands in many other numbers of the Federalist – including the twelve disputed papers.
The evidence comes from traditional and non-traditional authorship attribution methodologies.
Traditional
- 1840 – Renwick’s Lives of John Jay and Alexander Hamilton.
- 1894 – Whitaker’s, ‘A Problem in Authorship: Who Wrote “The Federalist?”’
- 1978 – Smyth’s, ‘The Federalist: The Authorship of the Disputed Papers.’
- 1984 – Cary’s, ‘Publius – A Split Personality.’
- 1984 – Furtwangler’s, The Authority of Publius: A Reading of the Federalist Papers.
- 1999 – Kesler’s ‘Introduction to the Federalist Papers.’
Non-Traditional
The Collins et al. study, ‘Detecting Collaborations in Text: Comparing the Authors’ Rhetorical Language Choices in the Federalist Papers’ confirms the premise of a deeper collaboration. Most of the non traditional authorship studies do not agree with each other. A list will be shown that reveals how many times each of the twelve disputed papers have been attributed to Hamilton in the over 90 other non-traditional studies.
Conclusion
This paper concludes with a discussion of the following:
- Acceptance of Results by Non-Traditional Practitioners
- Acceptance of Results by History Scholars
- Do the Multiple Flaws in the Non-traditional Studies Invalidate the Results
The bibliography for this paper contains well over 300 entries. The following is just a sample.
References
Adair, D. (1944). The Authorship of the Disputed Federalist Papers. The William and Mary Quarterly (Third Series) 1(2): 97-122.
Bosch, R. A., and J. A. Smith (1998). Separating Hyperplanes and the Authorship of the Disputed Federalist Papers. The American Mathematical Monthly 105(7): 601-607.
Carey, G. W. (1984). Publius – A Split Personality? The Review of Politics 46(1): 5-22.
Collins, J., et al. (2004). Detecting Collaborations in Text: Comparing the Authors’ Rhetorical Language Choices in The Federalist Papers. Computers and the Humanities 38(1):15-36.
Cooke, J. E., ed. (1956).The Federalist. Cleveland: Meridian Books, The World Publishing Company.
Davis, G. (2003). `RE: Gutenberg Edition of Federalist. Private E-mail, 20 November 2003, 18:46:51.
Farringdon, M. G., and A. Q. Morton (1990). Fielding and the Federalist. CS Report Series University of Glasgow CSC 90/R6.
Forsyth, R. S. (1995). Stylistic Structures: A Computational Approach to Text Classification. Diss. University of Nottingham.
Fung, G. (2003). The Disputed Federalist Papers: SVM Feature Selection via Concave Minimization. Proceedings of the 2003 Conference on Diversity in Computing. Atlanta, Georgia, pp. 42-46.
Gerritsen, C. M. (2003). Authorship Attribution Using Lexical Attraction. M.S. Dissertation, Massachusetts Institute of Technology.
Hilton, M. L., and D. I. Holmes (1993). An Assessment of Cumulative Sum Charts for Authorship Attribution. Literary and Linguistic Computing 8(2): 73-80.
Holmes, D. I., and R. S. Forsyth (1995). The Federalist Revisited: New Directions in Authorship Attribution.Literary and Linguistic Computing 10(2): 111-127.
Jockers, M. L., and D. M. Witten (2010). A Comparative Study of Machine Learning Methods for Authorship Attribution. Literary and Linguistic Computing 25(2): 215-224.
Khmelev, D. V., and F. J. Tweedie (2001). Using Markov Chains for Identification of Writers. Literary and Linguistic Computing 16(3): 299-307.
Kjell, B. (1994). Authorship Determination Using Letter Pair Frequency Features with Neural Network Classifiers. Literary and Linguistic Computing 9(2): 119-124.
Koppel, M., N. Akiva, and I. Dagan (2006). Feature Instability as a Criterion for Selecting Potential Style Markers. Journal of the American Society for Information Science and Technology 57(11): 1519-1525.
Levitan, S., and S. Argamon (2006). Fixing the Federalist: Correcting Results and Evaluating Attribution.’ Paper presented at Digital Humanities 2006. Paris, Sorbonne 2006 (Paper courtesy of authors.)
Martindale, C., and D. McKenzie (1995). On the Unity of Content Analysis in Authorship Attribution: The Federalist. Computers and the Humanities 29: 259-270.
Marcus, L. S. (2000). Afterword: Confessions of a Reformed Uneditor. In A. Murphy (ed.), The Renaissance Text: Theory, Editing, Textuality. Manchester: Manchester UP, pp. 211-216.
Marcus, L. S. (1996). Unediting the Renaissance: Shakespeare, Marlow, Milton. London: Routledge 1996.
McColly, W., and D. Weire (1983). Literary Attribution and Likelihood-Ratio Tests: The Case of the Middle English Pearle-Poems. Computers and the Humanities 17: 65-75.
Merriam, Th. (1989). An Experiment with the Federalist Papers. Computers and the Humanities 23(3): 251-254.
Mosteller, F., and D. L. Wallace (1964). Applied Bayesian and Classical Inference: The Case of The Federalist Papers. New York: Springer 1984. CSLI Publications published a reprint of the second edition in 2007 with a new introduction by John Nerbonne.
Mustafa, T. K., et al. (2010). Dropping Down the Maximum Item Set: Improving the Stylometric Authorship Algorithm in the Text Mining for Authorship Investigation. Journal of Computer Science 6(3): 230-238.
Oaks, M. (2004) Ant Colony Optimisation for Stylometry: The Federalist Papers. Proceedings of the 5th International Conference on Recent Advances in Soft Computing, pp. 86-91.
Pennebaker, J. W. The Federalist. Unpublished preliminary work – courtesy of the Author.
Piaia, J. [For Frederick Mosteller] Private E-mail, Tuesday, 22 July 2003, 10:57:38.
Project Gutenberg http://promo.net/pg/ [9-30-2003].
Rokeach, M., et al. (1970). A Value Analysis of the Disputed Federalist Papers. Journal of Personality and Social Psychology 16(2): 245-250.
Scigliano, R., ed. (2000). The Federalist. New York: The Modern Library.
Smyth, L. Qu. (1978). The Federalist: The Authorship of the Disputed Papers. Ph.D. University of Virginia.
Stamatatos, E., N. Fakotakis, and G. Kokkinakis (2001). Computer-Based Authorship Attribution Without Lexical Measures. Computers and the Humanities 35: 193-214.
Stillinger, J. (1991). Multiple Authorship and the Question of Authority. In. D. C. Greetham and W. Speed Hill (eds.), Text: Transactions of the Society for Textual Scholarship (5). New York: AMS Press, pp. 283-293.
Wachal, R. S. (1966). Linguistic Evidence, Statistical Inference, and Disputed Authorship. Dissertation, University of Wisconsin.
Waugh, S., A. Adams, and F. Tweedie (2000). Computational Stylistics Using Artificial Neural Networks. Literary and Linguistic Computing 15(2): 187-197.
Yang, A. C. C., et al. (2003). Information Categorization Approach to Literary Authorship Disputes. Physica A 329(3-4): 473-483.
