Linguistics | Anxious Door

SELECT fearful;

sqlite> SELECT ratings_1.q3,articles.title FROM articles LEFT JOIN ratings_1 ON articles.id = ratings_1.id ORDER BY ratings_1.q3 DESC LIMIT 10;

0.622115874383|Spanish PM talks tough on terrorism after latest ETA blast
0.532897708158|Britain backs US sanctions against Iran
0.45492276759|World's most influential wine critic accused of becoming too 'cosy' with the growers
0.394539399442|Cuba and Castro pay tribute to 'El Che, the flower cut early from its stem'
0.371606310847|Citizen Crikey: Missives from the marginals
0.360371702545|New EU treaty 'same as Constitution' say MPs
0.359015961078|1915 Armenian massacre WAS a genocide, insists US foreign affairs committee
0.358686637582|PETER HITCHENS: North Korea, the last great Marxist bastion, is a real-life Truman show
0.353321728998|World Briefing | Asia: The Philippines: Arroyo Pardons Predecessor and Draws Fire
0.349470571259|Terror as 14-year-old shoots five at his school in Cleveland

Gallows pole.

[‘happy’, 0.2646606779829892]
[‘angry’, 0.20234941569505008]
[‘fearful’, 0.19969005601320605]
[‘sad’, 0.0]
[‘disgusted’, 0.0]
[‘surprised’, 0.0]

Hangman, hangman, hold it a little while,
Think I see my friends coming, Riding a many mile.
Friends, did you get some silver?
Did you get a little gold?
What did you bring me, my dear friends, To keep me from the Gallows Pole?
What did you bring me to keep me from the Gallows Pole?
Continue reading Gallows pole.

Affect rating

A new possibility came up from dorkbot last night.

conceptnet from MIT. Two versions, one that uses montylingua, which I am already using, and a more recent version that seems to rely on a massive sql database.

The project draws on the Open Mind Common Sense Project corpus, an open and distributed project, unlike the Balanced affective word list.

Software combination

Current verison

Linux waikato 2.6.32-5-686 #1 SMP Wed Aug 25 14:28:12 UTC 2010 i686 GNU/Linux
Python 2.6.6 (r266:84292, Aug 27 2010, 20:59:12)
feedparser
pytz
paramiko
pyserial
pytwitter
PyRSS2Gen
pysqlite2
AsciiDammit
MontyLingua
xmlrpclib

First version

Python 2.3.5 (#1, Aug 19 2006, 21:31:42)

feedburner 4.1

MontyLingua 2.1

upgraded php & sqlite with macports
$ php -v PHP 5.2.4 (cli) (built: Oct 13 2007 11:43:55) $ sqlite3 -version 3.5.1 $ uname -a Darwin whale.local 8.10.1 Darwin Kernel Version 8.10.1: Wed May 23 16:33:00 PDT 2007; root:xnu-792.22.5~1/RELEASE_I386 i386 i386

Robert Plutchik’s eight basic emotions

anger, fear, sadness, disgust, surprise, curiosity, acceptance, joy

American Scientist

Some uncommon ‘z’ words.

sqlite> select count,word,pos from words where count = 1 and word like “z%” order by word limit 100;
1|Zamora|NNP
1|Zandvoort|NNP
1|Zanzibar|JJ
1|Zapatero|NNP
1|Zarko|NNP
1|Zavier|NNP
1|Zawahri|NNP
1|Zealand’|NNP
1|Zealand-Pacific|NNP
1|Zealand-Samoan|NNP
1|Zealand-registered|NNP
1|Zebon|NNP
1|Zeev|NNP
1|Zehi|NNP
1|Zeppelin|NNP
1|Zero’|NNP
1|Zhanjiang|NNP
1|Zhao|NNP
1|Zhejiang|NNP
1|Zhengrong|NNP
1|Zhengzhou|NNP
1|Zhenli|NNP
1|Zhenxing|NNP
1|Zhisheng|NNP
1|Zhou|NNP
1|Zhu|NNP
1|Zigzag’|NNP
1|Zille|NNP
1|Zimbabwean-controlled|NNP
1|Zion|NNP
1|Zip|NNP
1|Zlin|NNP
1|Zobaie|NNP
1|Zodiac|NNP
1|Zookeepers|NNP
1|Zubiarre|NNP
1|Zugna|NNP
1|Zumino|NNP
1|zapped|VBD
1|zeal|NN
1|zealot|NN
1|zero-carbon|JJ
1|zero-tolerance|JJ
1|zeroed|VBD
1|zip|NN
1|zodiac|NN
1|zombies|NNS
1|zones’|NN
1|zookeepers|NN
1|zorse|NN
sqlite>

Current top twenty proper nouns.

Current source feeds.

cursor.execute(‘INSERT INTO “sources” VALUES(2, “ABC News: Breaking Stories”, “http://abc.net.au/news/syndicate/breakingrss.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(3, “ABC News: World”, “http://abc.net.au/news/syndicate/worldrss.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(4, “ABC News: New South Wales”, “http://www.abc.net.au/xmlcontent/indexes/nsw/NSW_rss_index.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(5, “ABC News: Top Stories”, “http://abc.net.au/news/syndicate/topstoriesrss.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(6, “Crikey RSS”, “http://www.crikey.com.au/rss.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(7, ” the Mail online | World news”, “http://feeds.feedburner.com/dailymail/worldnews”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(8, “NYT > Middle East”, “http://www.nytimes.com/services/xml/rss/nyt/MiddleEast.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(9, “NYT > International”, “http://www.nytimes.com/services/xml/rss/nyt/International.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(10, “New Zealand Herald – World”, “http://syndication.apn.co.nz/rss/nzhrsscid_000000002.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(11, “New Zealand Herald – National”, “http://syndication.apn.co.nz/rss/nzhrsscid_000000001.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(12, “The Sydney Morning Herald World Headlines”, “http://feeds.smh.com.au/rssheadlines/world.xml”, 1)’)
cursor.execute(‘INSERT INTO “sources” VALUES(13, “The Sydney Morning Herald National Headlines”, “http://feeds.smh.com.au/rssheadlines/national.xml”, 1)’)

Data structure

Table of words
Word/Phrase
Index ( smallint )
Usage/definition
Examples

Table of results
Word/Phrase index ( smallint )
Number of results ( mediumint )
Aggregate score per affect index ( DOUBLE(9,2) UNSIGNED ) up to 1 billion to 2 decimal places

Table of affects
Affect ( varchar )
Affect id ( tinyint )

Table of voters
IP address who have voted
array of Word/Phrase index voted for ( smallint )

Word List statement of intent

Build an affective word list employing the minds of blog readers to rate words
Provide the data and data structure freely to all
Build ‘feed-horns’ : AJAX plugins and widgets for blog administrators
Provide contributors with feedback on their submission