# What Makes a Best-Selling Novel? A Machine Learning Approach (2016)

> Source: <https://marcinciura.wordpress.com/2016/04/17/what-makes-a-best-selling-novel/>
> Published: 2026-08-11 10:56:04+00:00

# A Machine Learning Approach

In 2013, [Ashok et al.](http://aclweb.org/anthology/D/D13/D13-1181.pdf) answered this question basing on the writing style, with 61–84% accuracy. This post, on the other hand, examines plot themes in best sellers. Note that my model can hardly predict the commercial success of a novel from its plot. That would be quite a surprising feat, making reviewers obsolete. My goal was more modest: finding statistically profitable topics to write about.

Using [PetScan](https://petscan.wmflabs.org/) and Wikipedia’s [page export](https://en.wikipedia.org/wiki/Special:Export/), I downloaded 25,359 Wikipedia articles belonging to [Category:Novels by year](https://en.wikipedia.org/wiki/Category:Novels_by_year). From each article, I extracted the section named **Plot**, **Plot summary**, **Synopsis**, etc. if present and, stripped of MediaWiki markup, saved it into an SQLite database along with the title of the novel, its year of publication, and a Boolean that indicates if it [ever topped the New York Times Fiction Best Seller list](https://en.wikipedia.org/wiki/Lists_of_The_New_York_Times_Fiction_Best_Sellers):

```
SELECT title, year, was_bestseller, length(plot) FROM Novels
ORDER BY random() LIMIT 5;
Sharpe's Havoc            | 2003 | 0 | 2759
The Rescue (Sparks novel) | 2000 | 1 |
Slayers                   | 1989 | 0 |
The Warden                | 1855 | 0 | 2793
The Fourth Protocol       | 1984 | 1 | 5666

SELECT count(*) FROM Novels
WHERE plot IS NOT NULL;
17744

SELECT count(*) FROM Novels
WHERE plot IS NOT NULL AND was_bestseller;
398

SELECT min(year) FROM Novels  -- The year of publication.
WHERE was_bestseller;  -- The NYT list starts in 1942.
1941
```

To obtain easy to interpret results, I have built a logistic regression model on top of the [TF–IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) transformation of articles processed by the [Porter stemmer](http://tartarus.org/martin/PorterStemmer/). The parameters have default values. In particular, the logistic regression uses L2 regularization so all lowercase words that are not stopwords appear in the model.

``` python
import nltk
from nltk.corpus import stopwords
from nltk.stem import porter
from sklearn import cross_validation
from sklearn import linear_model
from sklearn import pipeline
from sklearn.feature_extraction import text

def Tokenize(
    text,
    stemmer=porter.PorterStemmer(),
    uppercase=set(string.uppercase),
    stop_set=set(stopwords.words('english')),
    punctuation_re = re.compile(
        ur'[’“”…–—!"#$%&\'()*+,\-./:;?@\[\\\]^_`{|}~]',
        re.UNICODE)):
  text = punctuation_re.sub(' ', text)
  tokens = nltk.word_tokenize(text)
  return [stemmer.stem(x) for x in tokens
          if x.lower() not in stop_set and x[0] not in uppercase]

X = []
y = []
connection = sqlite3.connect('novels.sqlite')
for row in connection.cursor().execute(
    """SELECT plot, was_bestseller FROM Novels
    WHERE year >= 1941 AND plot IS NOT NULL"""):
  X.append(row[0])
  y.append(row[1])
connection.close()
X_train, X_test, y_train, y_test = (
    cross_validation.train_test_split(X, y, test_size=0.3))
model = pipeline.Pipeline(
    [('tfidf', text.TfidfVectorizer(
          lowercase=False, tokenizer=Tokenize)),
     ('logistic', linear_model.LogisticRegression())])
model.fit(X_train, y_train)
```

The model can return the probability of being a best seller for any novel *b* with a plot summary:

logit(*b*) = −4.6 + 2.5 tfidf(lawyer, *b*) + 2.4 tfidf(kill, *b*) + ⋯ − 1.5 tfidf(planet, *b*)

Pr(was_bestseller(*b*)|plot(*b*)) = *e*logit( b) / (1 +

*e*

logit()

*b*)To put these coefficients in context, tfidf(lawyer, *The Firm*) ≈ 0.06. As it happens, the model returns logit(*b*) > 0, that is Pr(was_bestseller(*b*)|plot(*b*)) > 1/2 for no novel *b* from the train or test set. The highest probability, 0.39, is predicted for [ Cross Fire](https://en.wikipedia.org/wiki/Cross_Fire_%28novel%29), indeed a best seller in December 2010. Only if I disable the normalization in TF–IDF or weaken the regularization in the logistic regression, I can overfit the model to the train set while for the test set both its precision and recall would be at most 20%. But, like I wrote in the introduction, this is not the point of this exercise. Let us look at the words with high absolute value of coefficients.

- Apparently, it pays off to write legal thrillers: lawyer +2.5, case +2.4, law +1.5, client +1.3, jury +1.3, trial +1.3, attorney +1.0, suspect +1.0, judge +0.9, convict +0.8;
- kill +2.4, murder +1.8, terrorist +1.2, shoot +1.1, body +1.1, die +1.0, serial +0.9, attack +0.9, assassin +0.8, kidnap +0.8, killer +0.8.
- Political thrillers are not bad either: agent +1.4, politics +1.4, president +1.3, defector +1.2.
- Business may be involved: firm +1.3, company +1.3, career +1.1, million +1.0, success +1.0, business +0.9, money +0.9.
- Finally, the characters should have families: husband +1.4, family +1.3, house +1.2, couple +1.2, daughter +1.2, baby +1.1, wife +1.0, father +1.0, child +0.9, birth +0.8, pregnant +0.8, and use a car +1.5 and a phone +0.8.

The genres to avoid for prospective best-selling authors?

- Sci-fi: planet −1.5, human −1.0, space −0.7, star −0.4, robot −0.3, orbit −0.3.
- Children’s literature: boy −1.3, school −1.0, young −0.8, girl −0.8, youth −0.4, teacher −0.4, aunt −0.4, grow −0.4.
- Geography and travels: village −1.0, city −1.0, ship −0.8, way −0.7, go −0.7, land −0.6, adventure −0.6, colony −0.5, native −0.5, follow −0.5, mountain −0.5, crew −0.5, forest −0.5, travel −0.5, inhabit −0.4, sail −0.4, road −0.4, map −0.3, tribe −0.3.
- War: fight −1.0, warrior −0.6, war −0.6, weapon −0.5, soldier −0.5, army −0.5, ally −0.4, enemy −0.3, conquer −0.3.
- Fantasy: magic −0.9, creature −0.5, magician −0.4, zombie −0.3, treasure −0.3, dragon −0.3.
- History: princess −0.5, rule −0.5, kingdom −0.4, castle −0.4, century −0.4, ruler −0.3, palace −0.3 (for what it’s worth,
*A Game of Thrones*only made it[to the third place on the list](http://www.nytimes.com/best-sellers-books/2011-07-10/combined-print-fiction/list.html)so it does not count as a best seller).

Note that the code above ignores capitalized words. If it does not, the most significant words become the names of characters from best selling book series: [Scarpetta](https://en.wikipedia.org/wiki/Kay_Scarpetta) +3.0, [Stephanie](https://en.wikipedia.org/wiki/Stephanie_Plum) +2.9, [Ayla](https://en.wikipedia.org/wiki/Ayla_%28Earth's_Children%29) +2.0, etc., with additional insights like FBI +1.3, CIA +1.3, NATO +0.9, Soviet +0.9, or Earth −1.1.