Austin Matzko's Blog

Highlighting search terms

An article at A List Apart from August 2004, “Enhance Usability by Highlighting Search Terms,” suggests that websites highlight the search terms found in the referrer information. For example, if someone Googles “widget” then clicks a resulting link to your site, you might want to highlight every instance of “widget” on your page. The script promoted by authors Brian Suda and Matt Riggott uses PHP; I’m more comfortable with Perl, so not finding a robust, easy-to-use Perl version I wrote my own.

Controversy

Some writing in the article’s comments questioned the purpose and method of the PHP highlighting script. The principle objections, in my opinion:

• Highlighting is an unnecessary distraction
• Highlighting should be done on the client side (with Javascript) to save server resources
• The script breaks, for example when the greater than symbol “>” appears in an image’s alt tag

The authors offer their own response. Here’s mine:

Highlighting is an unnecessary distraction

This is probably the strongest objection, and my response is pretty weak: I happen to like it. A better response is that you can add to the top of the page a brief explanation of the highlighting along with a link to the same page, so users can click the link and be rid of the highlighting.

Highlighting should be done on the client side (with Javascript) to save server resources

Another good objection. However, as the authors point out, this approach works only on Javascript-enabled browsers. More importantly, Javascript doesn’t have the thorough HTML parsing capability I’d like to see (although it might approach the regular expression ability offered by the article’s PHP script). This pertains to the next objection:

The script breaks, for example when the greater than symbol “>” appears in an image’s alt tag

The problem here has to do with what the authors have the PHP script doing. As they admit, “Implementing a full SGML/XML parser was well beyond the scope of our project.” I didn’t find that very satisfying. In fact, my early Perl version of their script did wacky things to the text within the alt and title tags. Here’s where Perl modules come in handy.

Perl HTML::Parser class

I’m not presuming to weigh in on the Perl vs. PHP dispute. My own opinion is that each has its value for different applications. The many Perl modules available for free download from CPAN make Perl a pleasure to use. The HTML module does all the heavy lifting for my script, and for that I’m thankful—I doubt my ability to come up with an efficient yet thorough regular expression to deal with HTML. I can let the minds behind the HTML::Parser module worry about that, and as circumstances change, I only have to update my module reference. Perl modules are easy to install. If your web host doesn’t have the HTML::Parser module (they probably do have it) and doesn’t install it when you ask, without a doubt you should switch hosts. This is a feature that the cheapest hosts offer.

The script—a subroutine

I’ll discuss it more thoroughly at the bottom, but here’s what the script does in general: It is a subroutine that takes the HTML to be parsed as its sole argument; then it returns that HTML with text highlighted (if it should be). So for example, if your Perl variable “$myhtml” contains the page you want to be parsed, you could include a line such as the following in your code. It would then replace $myhtml with the relevant search terms highlighted. I do something similar just before printing the HTML to the browser. Okay, the subroutine:

Explanations

$highlightstarttag is whatever tag you want to use to highlight your text. I chose to use the span tag with class “texthighlight” so I can do all the formatting in my CSS stylesheet. For example, here are the relevant lines from my stylesheet, setting the properties for class “texthighlight”: The next section puts a list of the tags to ignore into the @ignoretags array. I want to ignore <title> tags to keep <span . . .> from appearing in the browser title bar, and I ignore <script> tags to keep my Javascript from getting messed up. The next part is a list of search query keys taken directly from the PHP used for the A List Apart article. Google “widget” and you’ll see “q=widget” in the argument string that appears in the results’ URL. Yahoo uses “p” instead of “q.” Next: $content takes the first (and only) argument passed to this subroutine, in this case, the page HTML to be parsed. We’ll come back to the other two variables I’m just initializing here. We start looking for search terms to highlight only if a question mark (“?”) appears somewhere in the HTTP_REFERER value sent by the browser to the server. Let’s say your referrer looks like “http://www.google.com/search?hl=en&q=%22widget%22&btnG=Google+Search” shortens the string down to “hl=en&q=%22widget%22&btnG=Google+Search” The next little bit splits the string by “&,” translates stuff such as %22 into ", then stuffs the key-value pairs separated by “=” into the hash named %form. What we care about is that the key

q

now is associated with the value

"widget"

in the %form hash. Above we look through the @querykeys array to see if one of our %form hash keys is in that list. Sure enough, because q appears in the @querykeys array, $searchtext will now have "widget" as its value. We don’t want to highlight “and”’s and “or”’s just because someone used those boolean terms, so we replace them with spaces. Then we split up $searchtext by non-word characters (in this example, spaces or ") into the @words array. We create a package called HTMLStrip, invoking the HTML::Parser class. I got the idea for this section from Ken MacFarlane’s tutorial on HTML::Parser. “start” is a HTML::Parser subroutine called each time the parser encounters a start tag in the HTML. We’re modifying it for our purposes. “$html .= $origtext;” adds the original start tag back to $html variable, the variable which is going to contain our highlighted HTML. “$num_ignoretags = grep(/^$tag$/i, @ignoretags);” looks to see if the start tag is one of the tags we want to ignore, such as “title,” and returns a 1 in $num_ignoretags if it is. (More precisely, $num_ignoretags takes as its value the number of times $tag appears in the @ignoretags array.) This will come in handy next. The parser subroutine “text” deals with the text found between start and end tags. If $num_ignoretags is 0, i.e. we’re not inside a tag we’re supposed to ignore, then we replace all the search terms (if there are any) with the search terms surrounded by our highlighting start and end tags. So for example

Oh, how I love widgets!

would become

Oh, how I love <span class="texthighlight">widget<span>s!

Then we add the text to our $html variable. As you could guess, the “end” subroutine references end tags. Here we just add the end tag to the $html variable and reset $num_ignoretags to 0 in preparation for the next tag. We’ve reworked HTML::Parser’s subroutines, now it will call them when we call “$p->parse($content);” to parse the HTML. “$p->eof;” flushes HTML::Parser’s internal buffer, so we could re-invoke the parser if we wanted to. Finally, we assign the highlighted $html value to $content, and return $content. If you have any suggested improvements or criticisms, please leave a comment.