AI Reads the Classics: Machine Learning Extracts Themes from Political Theory

Alan Turing’s 1950 paper “Computing Machinery and Intelligence” contained far more than the famous imitation game now known as the Turing Test. In that paper, Turing imagined that artificial intelligence would develop through a process resembling the education of a child, an idea that anticipated modern machine learning decades before the field took shape. A new commentary published in the journal AI Perspectives argues that this educational vision raises an unsolved problem that remains urgent today: how humans might teach or shape artificial intelligences to behave in ways that align with moral standards. The author, Stephen J. DeCanio, suggests that part of any such teaching process would likely involve machines absorbing lessons from human writings, and he demonstrates one small step toward that goal using a natural language processing technique applied to works of classical political theory.

The technique at the center of the demonstration is called Latent Dirichlet Allocation, or LDA. It belongs to a family of methods known as “bag-of-words” approaches, which analyze texts by counting word frequencies while ignoring the syntactical structure of sentences. LDA is a well-defined but circumscribed statistical technique that enables the classification and systematization of certain kinds of information contained in a set of texts. Given its limits, it can identify major themes, the “topics” that form its output, without human intervention. In the study, LDA was applied to classical texts in political theory that had been downloaded in machine-readable form from Project Gutenberg, a scholarly resource that makes public-domain works freely available in the United States so long as the source is noted.

The results showed that even corpora made up of only a few authors and works could be successfully processed by LDA. The analysis also demonstrated that natural language processing techniques are applicable to texts that are distant in time from the modern day. Just as an unsupervised AI can distinguish between fiction and non-fiction books, an AI employing LDA can extract the main topics from a set of classical texts dealing with a particular field, in this case political theory. The author notes that deploying more computing power would enable results like these to encompass a wider range of texts and a greater number of topics in finer detail. All the calculations reported in the paper were performed on a desktop PC running Mathematica 12, underscoring that the demonstration did not require supercomputing resources.

Yet the author is emphatic about what these results do not show. The findings do not, and cannot, demonstrate human-level understanding of texts by an unsupervised AI. The LDA “topics” are not summaries of the contents of the works, and texts with differing or even opposing political implications might display highly similar LDA topic signatures. All of the authors whose writings were analyzed exhibit a strong orientation to “politics,” but their philosophies are quite different. Identification of topics in texts, the paper stresses, is not sufficient to define or develop political ethics. The gap between extracting statistical patterns of word co-occurrence and genuinely grasping an argument remains wide.

To mark that gap, the paper adopts a deliberate terminological convention. Words like “analysis,” “reading,” or “comprehension” are preceded by an “M” to denote that the cognitive operations are being performed by machines. This sidesteps the question of what constitutes “thinking,” a question Turing himself dismissed as too meaningless to deserve discussion. The M- terminology emphasizes that what AIs do, at least currently, is quite distinct from human understanding. While the extraction of topics from classical texts in political theory is remote from an AI actually being able to formulate plans and principles for action, political or otherwise, some kind of M-understanding of such works would, in the author’s view, be pertinent to any future machine reasoning about collective life.

The commentary situates this technical demonstration within a larger argument about moral development in machines. If the motivations of an emergent AI were formed through a learning process, it is plausible that part of that learning would involve the assimilation of human-generated works about ethics, morality, and politics. The long human history of the struggle to understand politics and formulate an appropriate moral foundation for collective action reflects experiences and insights that would be relevant to an AI confronting the same issues. At the same time, the paper concludes that Turing’s thought experiment about educating a child-AI highlights just how far humanity is from being able to know whether actual AIs, if and when they emerge, will be benign. As the author puts it, we have no idea what it is like to be an AI.

That uncertainty is compounded by the fact that human beings are embodied creatures, a condition the paper argues is relevant both to learning and to moral development. Educational theorists, one cited scholar observes, have often appeared uncomfortable with the brute fact of corporeality, discussing cognition and moral reasoning as if bodies were something of an embarrassment. The commentary also notes the sheer scale of the human educational enterprise: direct costs of bringing up a child in the United States have been estimated at roughly 233,610 dollars per child in 2015, excluding college education, and combined public and private spending on education plus direct childrearing expenses amount to roughly 10 percent of U.S. national output. Each human child must be brought up separately, whereas AI training is likely to be much more easily transferable, a difference with unclear consequences for what machines might learn.

The paper also draws distinctions among generations of machine intelligence. The machine learning of Alphabet and Google’s AlphaZero takes place according to programmed rules but produces a neural network whose internal functioning is not at all transparent. Learning systems like AlphaZero differ from the more conventional brute-force, search-intensive models that solved checkers in the 2000s and defeated the human world champion in chess during the 1990s. This transparency problem matters for the moral question at the heart of the commentary: even the best efforts of software developers have not been able to guarantee error-free behavior, and one cited estimate holds that even the most careful software engineering practices in industry rarely achieve an error rate better than one error per 1,000 lines of code.

Several further caveats frame the technical work. Computational irreducibility does not imply that complex systems are entirely random; such systems can exhibit statistical regularities as well as predictability at coarse levels of granularity. On the methodological side, convergence diagnostics for the kind of sampling used in LDA have no hard-and-fast rules, though one cited analysis suggests reasonable accuracy may often be achieved with 5,000 iterations or less. The three contributors to The Federalist Papers were treated as a single author in the analysis. Machiavelli’s insight that good intentions are not enough to produce good outcomes, the paper argues, should be the starting point for the development of truly moral political practice, even though his position is often reduced to a shorthand for amoral politics.

What remains is a modest but suggestive result. A statistical text-analysis method running on an ordinary desktop computer can pull recognizable themes out of texts written centuries ago, without human guidance about what those themes should be. That capability says nothing about whether a machine could ever weigh justice against expediency or understand why a political argument moves anyone. But if future artificial intelligences are to be shaped by the accumulated human conversation about ethics and politics, as the commentary contends is plausible, then knowing what patterns machines can and cannot extract from that conversation is a necessary first measurement. The distance between topic signatures and moral understanding, the paper makes clear, is still enormous, and no current technique closes it.

Subject of Research: Machine learning analysis of classical political theory texts using Latent Dirichlet Allocation

Article Title: Can an AI learn political theory?

Article References: DeCanio, S. J. (2020). Can an AI learn political theory?. AI Perspectives, 2(1), Article 3. https://doi.org/10.1186/s42467-020-00007-2

Image Credits: AI Generated

DOI: 10.1186/s42467-020-00007-2

Keywords: artificial intelligence, machine learning, natural language processing, Latent Dirichlet Allocation, political theory, Alan Turing, text analysis, AI ethics, Project Gutenberg, moral development, learn, political