{"slug": "kesselman-list-of-estimative-words-fig-5-2-gwern-finds-this-usefull-for-llm-s-so", "title": "Kesselman List of Estimative Words (fig 5.2) (gwern finds this usefull for LLM's and so do I)", "summary": "A developer highlights the Kesselman List of Estimative Words, a calibrated set of probability terms for intelligence analysis, as useful for large language models. The list assigns specific percentile ranges to words like 'Almost Certain' (86-99%) and 'Remote' (1-15%), aiming for consistent interpretation across audiences.", "body_md": "Kesselman List of Estimative Words (fig 5.2) (gwern finds this [usefull](https://gwern.net/system-prompts-2025) for LLMs and so do I). These are probability words calibrated to meaning where the general public understand them to be, with ambiouso words avboided.\n\nWords of estimative probability (WEP or WEPs) are terms used by intelligence analysts in the production of analytic reports to convey the likelihood of a future event occurring.\n\n| Word | Certainty |\n|---|---|\n| Almost Certain | 86-99% |\n| Highly Likely | 71-85% |\n| Likely | 56-70% |\n| Chances a little better [or less] than even | 46-55% |\n| Unlikely | 31-45% |\n| Highly Unlikely | 16-30% |\n| Remote | 1-15% |\n\n```\n  This issue prompts the researcher to suggest the Kesselman List of Estimative Words\n  for use within the intelligence community. It builds on Sherman Kent’s original WEP list in\n  the 1960s and the National Intelligence Council’s current list as well as draws from\n  Mercyhurst College’s WEP list. The new scale includes seven words of estimative\n  probability which is in line with what Kent and the NIC have proposed; however, it differs in\n  its phraseology and odds equivalents. The percentile ranges are broken down into groups of\n  15%, except for the middle range of chances a little better [or less] which was assigned only\n  10% and the upper and lower ranges which number 14%. Absolute certainty or impossibility\n  generally is not conveyed in intelligence assessments, but the two extremes are represented at\n  the top and bottom of the new scale.\n\n  Most importantly, the list uses words that large groups of people perceive in similar\n  manners. Subjects have never had problems with the extreme ends of a scale; therefore,\n  perceptions of almost certain and remote as well as highly likely and highly unlikely should\n  remain fairly constant. It is important to note that these terms are present in the NIC’s new\n  word list, although they vary slightly. Rather than using almost certainly, this researcher\n  believes that it is possible to use almost certain in far more grammatical structures, thus the\n  elimination of –ly. Second, the word highly conveys a much clearer picture with likely and\n  unlikely than does very, so those two phrases were also tweaked.\n  Where the problems appear, however, are with the words buried in the middle of the\n  scale. In weather forecasting, respondents of the Juneau survey indicated that the word likely\n  conveyed a 62.5% numerical equivalent and in the medical professions physicians have\n  indicated that the word’s value is approximately 70%. An odds equivalent in the scale above\n  of 56-70% mirrors that of researchers’ findings in several disciplines. Most notably, use of\n  only the word likely, rather than likely/probably as synonyms for one another (as in the NIC’s\n  new scale), should serve to eliminate confusion and standardize that particular percentile range.\n\n  The next question to tackle was how to convey odds that fell directly above or below\n  50%. The terms chances are even and fifty-fifty tells a decision maker nothing and\n  essentially asks them to toss a coin in the air. Therefore, a term was needed that would\n  convey odds slightly above or below the halfway benchmark and chances a little better [or\n  less] does exactly this. Only assigning the category 10% forces the analyst to make a call\n  depending on whether the chances are indeed better or less than a particular situation coming\n  to fruition. For example, if an analyst wants to convey that certain odds are better, they would have to equate the statement with at least 51%. It may sound as if 51% is not much different than 50%, but saying in essence that there is the slightest probability something may occur is a progress within the IC.\n\n  Finally, this list is not only extremely easy to use but also simple for analysts to\n  remember and produce on their own if they did not have a copy with them. The words are\n  arranged in such a way that the top of the list generally mirrors that of the bottom (with the\n  exceptions of almost certain and remote): highly likely realizes its counterpart in highly\n  unlikely and likely and unlikely mirror each other as well. Analysts simply need to remember\n  that each category is broken down into groups of 15% except for the middle category at 10%\n  and the two upper boundaries which will never reach complete certainty or impossibility.\n  The scale also eliminates the need for synonyms in estimative language. If these words can\n  be used consistently, analysts will always know exactly what they are attempting to convey\n  and decision makers will receive clarity, allowing them to enact policy that is in line with an\n  analyst’s thinking.\n\n  While there is an understanding of the value of consistent terminology in the IC, it\n  has yet to be operationalized. With the increased scrutiny that the IC is likely to receive in\n  the new information age, it is only to their benefit to adopt such a list. While the Kesselman\n  List of Estimative Words will likely be tweaked by others in the community, it is a step in the\n  right direction. Analysts have an obligation to communicate as effectively as they can the\n  results of their estimates. The best case scenario is that the NIC and the IC take into\n  consideration this new estimative scale above and produce several more iterations of their\n  own list until employees of the community can come to agreement on a set of clear-cut words\n  that all are both willing to accept and employ in daily practice.\n```\n\nFrom fig 5.2 in [Kesselman's 2008 List](https://gwern.net/doc/statistics/bayes/2008-kesselman.pdf#p71) of Estimative Words.\nSee also [Kent's words of estimative probability, 1964](https://en.wikipedia.org/wiki/Words_of_estimative_probability)", "url": "https://wpnews.pro/news/kesselman-list-of-estimative-words-fig-5-2-gwern-finds-this-usefull-for-llm-s-so", "canonical_source": "https://gist.github.com/wassname/780d0f01a3ee71b6430325202c56d992", "published_at": "2026-07-23 04:22:38+00:00", "updated_at": "2026-07-23 05:27:09.394978+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing"], "entities": ["Kesselman", "Sherman Kent", "National Intelligence Council", "Mercyhurst College"], "alternates": {"html": "https://wpnews.pro/news/kesselman-list-of-estimative-words-fig-5-2-gwern-finds-this-usefull-for-llm-s-so", "markdown": "https://wpnews.pro/news/kesselman-list-of-estimative-words-fig-5-2-gwern-finds-this-usefull-for-llm-s-so.md", "text": "https://wpnews.pro/news/kesselman-list-of-estimative-words-fig-5-2-gwern-finds-this-usefull-for-llm-s-so.txt", "jsonld": "https://wpnews.pro/news/kesselman-list-of-estimative-words-fig-5-2-gwern-finds-this-usefull-for-llm-s-so.jsonld"}}