When I wanted to dig deeper into J.R.R. Tolkien‘s creative imagination, my first purchase after The Silmarillion was his letter collection. Humphrey Carpenter was a colourful 20th-century public intellectual who took an interest in Tolkien. A well-known voice of “Great Lives” on the BBC, Carpenter published a literary biography of Tolkien in 1977 and then turned to editing the letters.

With help from Christopher Tolkien, Carpenter published The Letters of J.R.R. Tolkien in 1981. The initial collection included 354 letters, some of which were fragments or drafts of correspondence. Carpenter shares in a preface that the original collection was completed in 1979, but publishers Allen & Unwin determined that it was too long. Thus, we might call the 2023 publication of The Letters of J. R. R. Tolkien: Revised and Expanded Edition a restored edition. I don’t know if all 50,000 words that were cut for the 1981 edition are now included, but there are 165 new letters.

Fans and scholars were disappointed that the collection did not include much more from the letters between Tolkien and his wife-to-be, Edith, during the period of the war. I might be the only one who is disappointed that the boring, day-to-day, mundane letters are still in the vault, but I am appreciative of what we have.

Introducing Voyant

Recently, I have been playing with the text analysis tools at Voyant https://voyant-tools.org/. Voyant is a digital humanities tool developed by Canadian scholars Stéfan Sinclair & Geoffrey Rockwell. While Voyant is not intuitive to the nonspecialist, some time and creative risk are easily rewarded, making it one of the most accessible digital text analysis tools I know.

When I start a new study, I create an Excel sheet of data–some of which I’ve talked about before (e.g., see here). However, I am sometimes limited in my visual capabilities. For example, here is my Tolkien Timeline:

And that is getting a head start from something I found on reddit a while ago!

Anyway, so my limitations: Even though I am the least of digital humanities scholars, I wanted to test Voyant’s tools on a large text. So I turned to Voyant with the new expanded Letters text. There are some limitations. I cleaned the text but did not do much work text markup, indexing, or tokenization. I also did not select out the new letters for topic analysis or compare the Letters to other documents. Still, I hope readers find my wanderings here will help them get a sense of how the Letters shape our perception of Tolkien’s life and work.

This week is about some background information and textual trends; next week, I will post about Tolkien’s relationships.

Some Text Trends

Here are the details in Voyant’s summary of the text that has been stripped of all footnotes and editorial comments: This document with 231,865 total words and 16,273 unique word forms.

• Vocabulary Density:  0.070
• Readability Index: 7.947
• Average Words Per Sentence: 22.5

Voyant’s word cloud is satisfying because it selects out most of the common words that can clutter up our visuals. The visualization is set to capture the 50 main words of interest:

I find these word clouds so visually satisfying that I sometimes forget to use it to look for trends. While we can expect that “letter” is common (525 occurrences; I left the titles of the letters in place), it is significant that “time” (526) is the most common word. For much of his life, Tolkien was harried by time.

Here are some of the other words that pop out to me among the top 100 as defining the collection: long, course, world, hope, story and tale, work, book, lord, hobbit(s), Oxford, know, old, little and small, read, fact, English, ring(s), God, elves/elvish, road, life, written/writes/writing, years, words, power, children, mind, language, Frodo, war, earth, Sauron, Silmarillion, evil, middle.

The Collocates Trends visualization begins to cluster these words by significance. You should be able to hover over a term and see its word occurrence count, but I wasn’t able to figure out how to let you see the edges (or links) between the nodes (words).

It does not take much additional context before the interrelationships become too complex to map well in this visual:

Letter Statistics Over Time

Before launching into the full Voyant Voyage, though, some of the work is done by old-fashioned counting and spreadsheeting. As the document is nearly a quarter-million words, I wanted to create a relatively even spread of the letters rather than setting out a set of periods ahead of time (e.g., WWI, Leeds, early Oxford, etc.). I divided the doc into ten chapters, each about 1/10^th of the volume. Then I did a letter count:

The more complex version of this chart is hard to show on WordPress, but I was able to make a compressed chart that connects the letters to the major moments of Tolkien’s life:

I understand that my list of events is not going to be helpful for everyone, but the frame is there for you to play with.

When I did the math, I was suddenly struck by how few letters we have. Take a look at a graphic of the letters in each period:

In Period 1, 1914-39, we have an average of 1 letter for every 4 months of life! Even in the periods where we have the most letters, we are only getting 1 or 2 letters a month—during an age where letter-writing was part of everyday life. Flipping the phrase, I made a graph of monthly letter rate of what’s included in the volume:

Conclusion: We have so few of Tolkien’s letters! While the letters of figures like Dorothy L. Sayers, C.S. Lewis, and T.S. Eliot are carefully edited for public consumption, the vast bulk of Tolkien’s correspondence is either in archives or no longer extant.

Voyant Tools I Can’t Show on WordPress

Still, we have what we have, so back to Voyant. Before a deeper dive into relational connections in Part 2, I wanted to note some things I can’t show from Voyant’s platform (at least, with this version of WordPress).

This tool is called “Loom.” It shows thousands of significant words in their woven reality (occurrences graphed across the 10 sections). On the left, there is a rolling word alphabetical selection tool, which allows you to highlight the words you are looking for. I love this—though I don’t know if it shows me anything new, except that a fresh way of looking at things always helps me make new connections.

The Textual Arc tool is so dynamic I cannot even show it, but I think we can use our imaginations about the Word Bubble feature. The app plays the major unique words through the text, while the common word bubbles move and float according to linked significances. It is a dynamic tool you need a lot of time to read.  

A more linear tool is TermsBerry, showing the most popular words and revealing usage and ngram data (links between the words).

I didn’t understand the significance of the “Knots” tool, but if you want to watch slow hypnotic movements on your screen, try it out. “Mandala” and “MicroSearch” are better for comparing documents.

For the new user, though, most of the most essential tools are set up for you when you add your first text.

On this main page, we have the Cirrus Word Cloud creator (top left), a Reader that allows you to tap a word and show its trendline (top middle), the Trends of featured words (top right), the document Summary (bottom left), and a Contexts/Bubblelines/Collocates tab, which I have set to Bubblelines (bottom right).

Until Next Week

Next week, I will share Part 2 of “Voyaging with Voyant in Tolkien’s Expanded Letters,” focussed on Tolkien’s Relationships. Before stepping back, though, a couple of fun notes.

I fed the Bubbleline visualization tool some of Tolkien’s major projects. I added the words “lecture*” and “essay*” to compare the way Tolkien thinks about his Oxford University work over against his “secret vice” of language invention, poetry, and mythopoeic storytelling. The distinctions made me smile: Notice how intensely interested Tolkien is in his correspondence about the Middle-Earth legendarium—Silmarillion, Lord of the Rings, Hobbit—and other fictional bits of fancy. While the letters are far too limited to draw deeper conclusions, I feel like this chart is visually representative of reality.

The Topics tool is one that I’m still playing with, but I can see some intriguing possibilities. The Topics tool analyzes the text and selects related terms into various … well, various colours, as far as I can see. Calling them “topics” is a bit allusive and elusive to me … I can almost see the links, but not quite–and yet, they do feel connected in some ways to me. Very aethereal. I could increase the words and make fewer topics, but it goes some direction towards a user-guided topic or sentiment analysis.

Because of its ridiculous length, I’ll leave this strange list of repeating phrases at the bottom. “Count” means how many times that exact phrase appears. “Length” has to do with the length of the phrase. “Trend” multiplies the two to calculate a relative power, I guess. It is cool that these 17-word fragments occur twice: “he that breaks a thing to find out what it is has left the path of wisdom” and “own skill in healing it shall not be so i myself will go to war to fall.” But a more personal one spoke to me: The 7-word message, “all the love of your own father” occurs six times, not including variants. The “Trend” setting calls attention to some endearing features but needs cleaning up. I’ve omitted the most boring phrases (i.e., addresses).

I am open to sharing the basic spreadsheets of my letter counts for you to do your own work or correct mine. Contact me by email if that would help you: junkola[at]gmail[dot]com.