The simplest TF-IDF library imaginable.
Usage
Add your documents as two-element lists [doc_name, [list_of_words_in_the_document]] with add_document(doc_name, list_of_words).
table.add_document("foo", ["alpha", "bravo", "charlie", "delta", "echo", "foxtrot", "golf", "hotel"])Get a list of all the [doc_name, similarity_score] pairs relative to a list of
words by calling similarities([list_of_words]). Resulting similarities will
always be between 0.0 and 1.0, inclusive.
table.similarities(["alpha", "bravo", "charlie"])So, for example:
from tfidf import TfIdf
table = TfIdf()
table.add_document("foo", ["alpha", "bravo", "charlie", "delta", "echo", "foxtrot", "golf", "hotel"])
table.add_document("bar", ["alpha", "bravo", "charlie", "india", "juliet", "kilo"])
table.add_document("baz", ["kilo", "lima", "mike", "november"])
print table.similarities(["alpha", "bravo", "charlie"]) # => [['foo', 0.6875], ['bar', 0.75], ['baz', 0.0]]Run the tests
The tests use the standard library's unittest module, so there's no need to
install anything. Just run:
$ python test_tfidf.pyDisclaimer
This library is a pretty clean example of how TF-IDF operates. However, it's
totally unconcerned with efficiency (it's just an exercise to brush up my Python
skills), so you probably don't want to be using it in production. If you're
looking for a more heavy-duty Python library to do information retrieval and
topic modeling, I'd suggest taking a look at Gensim.