A. Document Frequiency (DF)Document

A. Document Frequiency (DF)
Document frequency is the number of documents in which a
term occurs in a dataset. It is the simplest criterion for term
selection and easily scales to a large dataset with linear
computation complexity. A basic assumption of this method is
that terms appear in minority documents are not important or
will not influence the clustering efficiency. It is a simple but
effective feature selection method for text categorization [9].
B. Term Contributtion (TC)
Because the simple method like DF assumes that each term
is of same importance in different documents, it is easily
biased by those common terms which have high document
frequency but uniform distribution over different classes. TC
is proposed to deal with this problem [10].
We will introduce TF.IDF (Term Frequency Inverse
Document Frequency) first [11]. TF.IDF synthetically
considers the frequency of a term in a document and the
document frequency of the term. It believes that if a term
appears in too many documents, it's too common and not
important for clustering. So Inverse Document Frequency is
considered. That is, if the frequency of a term in a document is
high and it does not appear in many documents, the term is
important. A common form of TF.IDF is



The result of text clustering is highly dependent on the
documents similarity. So the contribution of a term can be
viewed as its contribution to the documents' similarity. The
similarity between documents Di and D is computed by dot
product:




Term variance quality method is introduced by lnderjit
Dhillon, Jacob Kogan and Charles Nicholas [12]. It follows
the ideas of Salton and McGill [13]. The quality of the term t
is measured as follows:


Where n is the number of documents in which t occurs at
least once, and fij>=I,j=1,...,n.