{"slug": "what-makes-a-best-selling-novel-a-machine-learning-approach-2016", "title": "What Makes a Best-Selling Novel? A Machine Learning Approach (2016)", "summary": "A 2016 analysis of 17,744 Wikipedia plot summaries found that a logistic regression model using TF-IDF features could not predict best-seller status for any novel, with the highest probability at 0.39 for James Patterson's 'Cross Fire'. The study, based on data from Wikipedia and the New York Times Fiction Best Seller list, identified 'lawyer' and 'kill' as positive predictors and 'planet' as a negative predictor, but concluded that plot alone is insufficient to forecast commercial success.", "body_md": "# A Machine Learning Approach\n\nIn 2013, [Ashok et al.](http://aclweb.org/anthology/D/D13/D13-1181.pdf) answered this question basing on the writing style, with 61–84% accuracy. This post, on the other hand, examines plot themes in best sellers. Note that my model can hardly predict the commercial success of a novel from its plot. That would be quite a surprising feat, making reviewers obsolete. My goal was more modest: finding statistically profitable topics to write about.\n\nUsing [PetScan](https://petscan.wmflabs.org/) and Wikipedia’s [page export](https://en.wikipedia.org/wiki/Special:Export/), I downloaded 25,359 Wikipedia articles belonging to [Category:Novels by year](https://en.wikipedia.org/wiki/Category:Novels_by_year). From each article, I extracted the section named **Plot**, **Plot summary**, **Synopsis**, etc. if present and, stripped of MediaWiki markup, saved it into an SQLite database along with the title of the novel, its year of publication, and a Boolean that indicates if it [ever topped the New York Times Fiction Best Seller list](https://en.wikipedia.org/wiki/Lists_of_The_New_York_Times_Fiction_Best_Sellers):\n\n```\nSELECT title, year, was_bestseller, length(plot) FROM Novels\nORDER BY random() LIMIT 5;\nSharpe's Havoc            | 2003 | 0 | 2759\nThe Rescue (Sparks novel) | 2000 | 1 |\nSlayers                   | 1989 | 0 |\nThe Warden                | 1855 | 0 | 2793\nThe Fourth Protocol       | 1984 | 1 | 5666\n\nSELECT count(*) FROM Novels\nWHERE plot IS NOT NULL;\n17744\n\nSELECT count(*) FROM Novels\nWHERE plot IS NOT NULL AND was_bestseller;\n398\n\nSELECT min(year) FROM Novels  -- The year of publication.\nWHERE was_bestseller;  -- The NYT list starts in 1942.\n1941\n```\n\nTo obtain easy to interpret results, I have built a logistic regression model on top of the [TF–IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) transformation of articles processed by the [Porter stemmer](http://tartarus.org/martin/PorterStemmer/). The parameters have default values. In particular, the logistic regression uses L2 regularization so all lowercase words that are not stopwords appear in the model.\n\n``` python\nimport nltk\nfrom nltk.corpus import stopwords\nfrom nltk.stem import porter\nfrom sklearn import cross_validation\nfrom sklearn import linear_model\nfrom sklearn import pipeline\nfrom sklearn.feature_extraction import text\n\ndef Tokenize(\n    text,\n    stemmer=porter.PorterStemmer(),\n    uppercase=set(string.uppercase),\n    stop_set=set(stopwords.words('english')),\n    punctuation_re = re.compile(\n        ur'[’“”…–—!\"#$%&\\'()*+,\\-./:;?@\\[\\\\\\]^_`{|}~]',\n        re.UNICODE)):\n  text = punctuation_re.sub(' ', text)\n  tokens = nltk.word_tokenize(text)\n  return [stemmer.stem(x) for x in tokens\n          if x.lower() not in stop_set and x[0] not in uppercase]\n\nX = []\ny = []\nconnection = sqlite3.connect('novels.sqlite')\nfor row in connection.cursor().execute(\n    \"\"\"SELECT plot, was_bestseller FROM Novels\n    WHERE year >= 1941 AND plot IS NOT NULL\"\"\"):\n  X.append(row[0])\n  y.append(row[1])\nconnection.close()\nX_train, X_test, y_train, y_test = (\n    cross_validation.train_test_split(X, y, test_size=0.3))\nmodel = pipeline.Pipeline(\n    [('tfidf', text.TfidfVectorizer(\n          lowercase=False, tokenizer=Tokenize)),\n     ('logistic', linear_model.LogisticRegression())])\nmodel.fit(X_train, y_train)\n```\n\nThe model can return the probability of being a best seller for any novel *b* with a plot summary:\n\nlogit(*b*) = −4.6 + 2.5 tfidf(lawyer, *b*) + 2.4 tfidf(kill, *b*) + ⋯ − 1.5 tfidf(planet, *b*)\n\nPr(was_bestseller(*b*)|plot(*b*)) = *e*logit( b) / (1 +\n\n*e*\n\nlogit()\n\n*b*)To put these coefficients in context, tfidf(lawyer, *The Firm*) ≈ 0.06. As it happens, the model returns logit(*b*) > 0, that is Pr(was_bestseller(*b*)|plot(*b*)) > 1/2 for no novel *b* from the train or test set. The highest probability, 0.39, is predicted for [ Cross Fire](https://en.wikipedia.org/wiki/Cross_Fire_%28novel%29), indeed a best seller in December 2010. Only if I disable the normalization in TF–IDF or weaken the regularization in the logistic regression, I can overfit the model to the train set while for the test set both its precision and recall would be at most 20%. But, like I wrote in the introduction, this is not the point of this exercise. Let us look at the words with high absolute value of coefficients.\n\n- Apparently, it pays off to write legal thrillers: lawyer +2.5, case +2.4, law +1.5, client +1.3, jury +1.3, trial +1.3, attorney +1.0, suspect +1.0, judge +0.9, convict +0.8;\n- kill +2.4, murder +1.8, terrorist +1.2, shoot +1.1, body +1.1, die +1.0, serial +0.9, attack +0.9, assassin +0.8, kidnap +0.8, killer +0.8.\n- Political thrillers are not bad either: agent +1.4, politics +1.4, president +1.3, defector +1.2.\n- Business may be involved: firm +1.3, company +1.3, career +1.1, million +1.0, success +1.0, business +0.9, money +0.9.\n- Finally, the characters should have families: husband +1.4, family +1.3, house +1.2, couple +1.2, daughter +1.2, baby +1.1, wife +1.0, father +1.0, child +0.9, birth +0.8, pregnant +0.8, and use a car +1.5 and a phone +0.8.\n\nThe genres to avoid for prospective best-selling authors?\n\n- Sci-fi: planet −1.5, human −1.0, space −0.7, star −0.4, robot −0.3, orbit −0.3.\n- Children’s literature: boy −1.3, school −1.0, young −0.8, girl −0.8, youth −0.4, teacher −0.4, aunt −0.4, grow −0.4.\n- Geography and travels: village −1.0, city −1.0, ship −0.8, way −0.7, go −0.7, land −0.6, adventure −0.6, colony −0.5, native −0.5, follow −0.5, mountain −0.5, crew −0.5, forest −0.5, travel −0.5, inhabit −0.4, sail −0.4, road −0.4, map −0.3, tribe −0.3.\n- War: fight −1.0, warrior −0.6, war −0.6, weapon −0.5, soldier −0.5, army −0.5, ally −0.4, enemy −0.3, conquer −0.3.\n- Fantasy: magic −0.9, creature −0.5, magician −0.4, zombie −0.3, treasure −0.3, dragon −0.3.\n- History: princess −0.5, rule −0.5, kingdom −0.4, castle −0.4, century −0.4, ruler −0.3, palace −0.3 (for what it’s worth,\n*A Game of Thrones*only made it[to the third place on the list](http://www.nytimes.com/best-sellers-books/2011-07-10/combined-print-fiction/list.html)so it does not count as a best seller).\n\nNote that the code above ignores capitalized words. If it does not, the most significant words become the names of characters from best selling book series: [Scarpetta](https://en.wikipedia.org/wiki/Kay_Scarpetta) +3.0, [Stephanie](https://en.wikipedia.org/wiki/Stephanie_Plum) +2.9, [Ayla](https://en.wikipedia.org/wiki/Ayla_%28Earth's_Children%29) +2.0, etc., with additional insights like FBI +1.3, CIA +1.3, NATO +0.9, Soviet +0.9, or Earth −1.1.", "url": "https://wpnews.pro/news/what-makes-a-best-selling-novel-a-machine-learning-approach-2016", "canonical_source": "https://marcinciura.wordpress.com/2016/04/17/what-makes-a-best-selling-novel/", "published_at": "2026-08-11 10:56:04+00:00", "updated_at": "2026-08-11 11:12:41.386978+00:00", "lang": "en", "topics": ["machine-learning", "natural-language-processing"], "entities": ["Ashok et al.", "PetScan", "Wikipedia", "New York Times Fiction Best Seller list", "James Patterson", "Cross Fire"], "alternates": {"html": "https://wpnews.pro/news/what-makes-a-best-selling-novel-a-machine-learning-approach-2016", "markdown": "https://wpnews.pro/news/what-makes-a-best-selling-novel-a-machine-learning-approach-2016.md", "text": "https://wpnews.pro/news/what-makes-a-best-selling-novel-a-machine-learning-approach-2016.txt", "jsonld": "https://wpnews.pro/news/what-makes-a-best-selling-novel-a-machine-learning-approach-2016.jsonld"}}