Dynamic Document Search Engine – Part 1 Page 5

As a result we get a list of words stored in an array returned to the called function.

FilterCommonAndDuplicateWords() Function:

This function is called after ExtractWords() function. This parses filtered words removes common words like ??a??,??is??,
??was??,??and????. Other words are taken as valid words, remove duplicate among them and then stored in an associative array $wordMap and this array is returned to the called function.

<?php
function FilterCommonAndDuplicateWords( $wordList ) {
    global $COMMON_WORDS;
    global $MAX_WORD_LENGTH;
    $wordMap = array();
    foreach ( $wordList as $word ) {
        $len = strlen( $word );
        if ( ($len > 1) && ($len < $MAX_WORD_LENGTH) ) {
            if ( !$wordMap[$word] ) {
                if ( !$COMMON_WORDS[$word] ) {
                    $wordMap[$word] = 1;
                }
            }
        }
    }
?>