stressfree - search

Integrating Google Site Search into SilverStripe

David — Tue, 15 Sep 2009 08:32:53 +0000

SilverStripe is an excellent, user-friendly content management system but its internal search functionality is, to put it kindly, useless. Fortunately with Google Site Search you can embed a Google-powered custom search engine into your SilverStripe site. Doing so requires a paid Site Search account, pricing for which starts at $100/year.

This tutorial explains how to integrate this Google Site Search XML feed into your SilverStripe site. Doing so has a number of benefits over the standard means of integrating Site Search, namely:

No Javascript is required to display results within the SilverStripe site.
The user is not taken to a separate, Google operated website to view results.
The look and feel is consistent with the rest of the SilverStripe site.
Multiple Site Search engines can be integrated into a single SilverStripe site.
Site Search results pages are integrated into SilverStripe's management console.

Note: To integrate Site Search into SilverStripe using the described method a Site Search plan must be purchased as this provides results in XML. The free, advertising supported, Site Search engine does not provide search results in XML and cannot be used.

Loading XML data from an external source

Before the search page can be added to SilverStripe we need a reliable means of loading XML content. This is complicated by the fact many Web hosts disable PHP's built in URL fetcher (fopen) with the following php.ini directive:

allow_url_fopen = Off

Assuming it is installed, the cURL can get around this restriction, hence the XmlLoader helper library includes both methods (cURL is used by default in search.php).

Create a XmlLoader.php file in your SilverStripe's mysite/code directory with the following contents:

mysite/code/XmlLoader.php

<?php
class XmlLoader {

public function pullXml($url, $parameters, $useCurl) {
$urlString = $url."?".$this->buildParamString($parameters);

if ($useCurl) {
return simplexml_load_string($this->loadCurlData($urlString));
} else {
return simplexml_load_file($urlString);
}
}

private function loadCurlData($urlString) {

if ($urlString == -1) {
echo "No url supplied<br/>"."/n";
return(-1);
}

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $urlString);
curl_setopt($ch, CURLOPT_TIMEOUT, 180);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec($ch);
curl_close($ch);

return $data;
}

private function buildParamString($parameters) {
$urlString = "";

foreach ($parameters as $key => $value) {
$urlString .= urlencode($key)."=".urlencode($value)."&";
}

if (trim($urlString) != "") {
$urlString = preg_replace("/&$/", "", $urlString);
return $urlString;
} else {
return (-1);
}
}
}
?>

With the helper library in place to load the XML, it is now time to implement the SilverStripe "search" page type and logic. Create a search.php file in your SilverStripe's mysite/code directory with the following contents:

mysite/code/search.php

<?php

require_once 'XmlLoader.php';

class Search extends Page {
static $db = array(
'GoogleSearchId' => 'Text',
'NoResults' => 'HTMLText',
);
static $has_one = array(
);

function getCMSFields() {
$fields = parent::getCMSFields();

$fields->addFieldToTab('Root.Content.Main', new TextField(
'GoogleSearchId', 'Google Site Search ID'), 'Content');
$fields->addFieldToTab('Root.Content.Main', new HtmlEditorField(
'NoResults', 'No results message'), 'Content');

# Remove the content field
$fields->removeFieldFromTab("Root.Content.Main","Content");

return $fields;
}
}

class Search_Controller extends Page_Controller {

function SearchForm() {
$input = array_merge($_GET, $_POST);
$query = $input['q'];

$output = "<form class=\"search\" action=\"/search/results\"><fieldset>";
$output .= "<input type=\"text\" size=\"40\" name=\"q\" value=\"$query\"/>";
$output .= "<input type=\"hidden\" name=\"p\" value=\"1\"/>";
$output .= "<input type=\"submit\" value=\"Search\"/>";
$output .= "</fieldset></form>";

return $output;
}

function SearchResults() {

$output = "";

$input = array_merge($_GET, $_POST);
$page = isset($input['p']) ? $input['p'] : '1';
$query = $input['q'];

$perPage = 10;
if ($page < 1) { $page = 1; }

$xml = $this->getGoogleSearchResults($this->GoogleSearchId, $perPage, $page, $query);
$results = $this->parseGoogleSearchResults($xml);

$totalResults = $this->getResultCount($xml);

$output .= $this->getFormattedResults($results);

if (count($results) == 0) {
// Show no results message
$output .= $this->NoResults;;
} else {
// Append paging
$output .= $this->getPagingForResults($totalResults, $query, $perPage, $page);
}

return $output;
}

private function getGoogleSearchResults($googleId, $perPage, $page, $query) {

$startingRecord = ($page - 1) * $perPage;

$url = "http://www.google.com/search";
$parameters = array();
$parameters["client"] = "google-csbe";
$parameters["output"] = "xml_no_dtd";
$parameters["num"] = $perPage;
$parameters["cx"] = $googleId;
$parameters["start"] = $startingRecord;
$parameters["q"] = $query;

$XmlLoader = new XmlLoader();

return $XmlLoader->pullXml($url, $parameters, true);

}

private function parseGoogleSearchResults($xml) {

$results = array();

$attr["title"] = $xml->xpath("/GSP/RES/R/T");
$attr["url"] = $xml->xpath("/GSP/RES/R/U");
$attr["desc"] = $xml->xpath("/GSP/RES/R/S");

foreach($attr as $key => $attribute) {
$i = 0;
foreach($attribute as $element) {
$results[$i][$key] = (string)$element;
$i++;
}
}
return $results;
}

private function getFormattedResults($results) {

$output = "";

if (count($results) > 0) {
$output .= "<ul class=\"results\">";
foreach($results as $i => $result) {
$title = "";
$url = "";
$desc = "";
foreach($result as $key => $value) {
if ($key == "title") {
$title = $value;
}
if ($key == "url") {
$url = $value;
}
if ($key == "desc") {
$desc = $value;
}

$output .= "<li><a href=\"$url\">$title</a><p>";
$output .= str_replace("<br>", "<br/>", $desc);
$output .= "</p></li>\n";
}
$output .= "</ul>";
}
return $output;
}

private function getResultCount($xml) {

$totalResults = 0;
$count = $xml->xpath("/GSP/RES/M");
foreach($count as $value) {
$totalResults = $value;
}
return $totalResults;
}

private function getPagingForResults($totalResults, $query, $perPage, $page) {

$maxPage = ceil($totalResults/$perPage);

if ($totalResults > 1) {
$output = "<div class=\"searchPaging\"><p>";

for($pageNum = 1; $pageNum <= $maxPage; $pageNum++) {
if ($pageNum == $page) {
$output .= " <strong>$pageNum</strong> ";
} else {
$output .= " <a href=\"".$this->AbsoluteLink()."results?q=$query&p=$pageNum\">$pageNum</a> ";
}
}
$output .= "</p></div>";
}
return $output;
}
}

?>

This file defines a search page type with two fields, a Google Search Id and an HTML field that is displayed if no search results are found. As this page does not have any content of its own the default SilverStripe content field is also disabled to avoid confusion.

With the backend logic in place it is time to implement the templates. The templates themselves will vary from site to site, but the examples given are good starting points. There are two templates, one which simply displays the search box and a second that displays the results.

Create a Search.ss file in your SilverStripe's mysite/templates/Layout directory with the following contents:

mysite/templates/Layout/Search.ss

<% if Menu(2) %>
<div class="pageWithMenu">
<% end_if %>
<div class="page">
<% if Menu(2) %>
<div class="content contentStandard">
<% else %>
<div class="content contentFull">
<% end_if %>
<h1>$Title</h1>
<div class="contentWrapper">
$SearchForm
<div class="clear"></div>
</div>
</div>

<% if Menu(2) %>
<div id="sidepanel">
<% include SideBar %>
</div>
<div class="clear"></div>
</div>

<% end_if %>

Now create the results page named Search_results.ss in your SilverStripe's mysite/templates/Layout directory with the following contents:

mysite/templates/Layout/Search_results.ss

<% if Menu(2) %>
<div class="pageWithMenu">
<% end_if %>
<div class="page">
<% if Menu(2) %>
<div class="content contentStandard">
<% else %>
<div class="content contentFull">
<% end_if %>
<h1>$Title</h1>
<div class="contentWrapper">
$SearchForm
<div id="searchResults">
$SearchResults
</div>
</div>
<div class="clear"></div>
</div>
</div>

<% if Menu(2) %>
<div id="sidepanel">
<% include SideBar %>
</div>
<div class="clear"></div>
</div>

<% end_if %>

Note: The content of these two files will vary depending on your site. In the above example a SideBar include file is used to load the secondary menu.

With the backend logic and template files in place it is time to rebuild the SilverStripe database so that the new page type can be recognised. Enter the following (modified) URL into your browser: http://yourwebsite/dev/build?flush=all

All going well the rebuild command will execute correctly. If it does browse to the administration section of your site and create a 'search' page type.

The search page type in the create menu

On the search page enter your relevant Google Search Id and Results Not Found message. For the page URL use /search as this is hard coded into the search.php file. It is possible to change this URL (or use a dynamic one) but for the purposes of this tutorial it is not necessary.

Note: You can get your Google Search Id from the Google Search administration console, or it can be found within the embed URL used in the Javascript or external search forms.

The search page settings (click to enlarge)

Once published open the page and try a search. Assuming your code and settings are correct the Google search results should be displayed within your SilverStripe page. Now all there is left for you to do is style the results.

For an example of this technique at work, checkout the Parliamentary Counsel Office's search interface which is implemented using the method just described.

Autodesk Seek gets a new look and more content

David — Wed, 17 Dec 2008 10:21:50 +0000

Autodesk has not yet abandoned their web-based services endeavours in spite of a wilting construction industry and sinking global economy. Just prior to Autodesk University 2008 their Seek service received a significant makeover. Now this week it was announced BIMWorld has been acquired by Autodesk so that its BIMLibrary catalogue can be folded into Seek's. These events all sound good on paper, but how do they stack up, and more importantly is this a step forward for the Seek service?

The new user interface

The old white on black style of Seek has disappeared in favour of pastels on white. Overall this is a welcome change, but more importantly the overall appearance has been tidied up, with more attention paid to the rendering of onscreen elements. The result still feels very database-driven, but compared to the previous interface it does have a better flow and a less haphazard look. The Javascript-based Yahoo! User Interface library has been used to good effect and overall it feels very snappy. Unfortunately under this new coat of paint some things have not changed, for example the URIs for each product are shockingly bad. The option to email a link of the product has improved, but most people are used to simply copying and pasting URLs from the browser. If Autodesk expect others to link to content they need to resolve this problem. Until then it is very difficult for people to collaborate using Seek as a point of reference.

The revised look and feel of Autodesk Seek (click to enalrge)

It is good to see that on some 3D models the option to preview the model before downloading is available. Whether this option appears seems to depended on the supplier, but when it was present the result worked quite well. On the Mac with Safari the preview was limited to Project Freewheel's static 3D rendering of model, but apparently an interactive 3D viewer is available for Windows users. Even with this limitation the preview provided by Freewheel is a welcome addition, and for most tasks provides all the validation a user may require.

Seek's new 3D model preview option (click to enlarge)

Beyond these specific improvements the overall interface has not fundamentally changed. This is not bad, you cannot help but feel one or two UI iterations centered around navigation and filtering could lead to something really innovative. This could be helped if 2009 Autodesk exposed Seek's index as a series of web services. Third parties would then be free to develop more varied interfaces and "mash-up" the data with external services.

The BIMWorld purchase

Whenever one large company buys a smaller one it is always fun to watch. Autodesk's acquisition of BIMWorld appears to be centered around two assets; the content catalogue and its supporting community. Whilst the catalogue and its supporting content should be relatively easy to integrate with Seek, preserving the community may prove more of a challenge. The motivation behind "buying" a community is simple, on the Internet page views are the linga franca of advertising dollars. The more users Seek can claim, the more valuable it becomes to advertisers, suppliers and potential users.

The challenge facing this forthcoming community migration is that BIMWorld has significant functionality Seek currently lacks, for example membership, ratings and the ability to upload new content. Such deficiencies may prove too much for many BIMWorld users to stomach. However it may turn out this "missing" functionality is added to Seek sometime in 2009. The inability to submit or comment on content handicaps the fledgling service, so you would hope planning and implementation of such features was underway prior to the acquisition.

Conclusion: It is getting there slowly

Seek is demonstrating slow but steady improvement on both the interface and content fronts. The purchase of BIMWorld illustrates that Autodesk are serious about the idea of a universal product catalogue and are willing to spend money to ensure it establishes itself. Hopefully in 2009 we see the implementation of user and community-centric functionality that lifts the service from a simple database interface to a thriving AEC specifications centre. Such an endeavour will prove most successful if the functionality can be tightly integrated with Autodesk's 2010 range of AEC products. The majority of AEC professionals spend their working lives buried in these tools, so for Seek's sake lets hope Autodesk does not expend too much effort bringing people to the website, and instead focus on taking the content to the people.

Autodesk Seek steps towards ubiquitous AEC search

David — Fri, 06 Jun 2008 21:59:29 +0000

Note: Before reading this critque I would recommend checking out this Autodesk Seek presentation as it answers many of the questions raised here.

In May Autodesk released a beta of Autodesk Seek, a web-based Architecture, Engineering and Construction (AEC) specific, 3D model and specifications search tool. Rather than a free for all model index in a similar guise to Google's 3D Warehouse or CADoogle, the service is focused on exposing the model and specification catalogues of AEC suppliers. This is hardly going to interest the armchair designer, but for architects and engineers the ability to quickly locate, access and reference specifications and 3D data could potentially reduce design development time and costs significantly.

Gauging by the initial contents of Seek it would appear Autodesk have partnered with some large U.S. suppliers in order to kick-start their index. Whilst the index signals a clear sign of intent its current contents is hardly awe inspiring. That being said raw index size itself does not ensure success, to really make a mark and stand the test of time the Seek team need to execute on three things:

Quickly build out this index with up to date and relevant content so that it becomes the first place AEC professionals head to.
Create a compelling user experience which overcomes the idea that a specifications catalogue must be dull, unhelpful and always two months out of date.
Work to integrate Seek into as many aspects of Autodesk's existing modeling and drafting tools. By doing so the line between desktop and Web will be blurred and Seek will become a natural extension of their professional digital toolset.

What differentiates Seek from the crowd?

The idea of an online product catalogue for AEC specifications is certainly not new. However Seek is unique in that it is the first online product catalogue backed by a large company who's primary customer-base is not AEC suppliers. In the past online AEC catalogue initiatives have been spearheaded by suppliers or third-parties financially dependent on these suppliers. This close association has hindered growth and because for a Web-based, universal product catalogue to be successful it must stand independently from its data suppliers. This independence establishes trust which is important because users do not want the relevancy of their search influenced by who is paying the bills, nor do they want a 'walled garden' where only products from selected (paying) suppliers are on show. Consequently even though many supplier-backed catalogues exist, none can be considered the Google of the AEC world.

Seek has the potential of filling this 'Google' void because Autodesk's primary income is from people who make material purchasing decisions (architects, engineers and contractors, etc.) and not the suppliers themselves. This difference places Seek in the position of being able to design a catalogue that acts in the best interests of the search consumer. At the same time suppliers are practically forced to take part given Autodesk's vast global audience. The challenge facing Seek it is that Autodesk are not known for producing search indexes or successful Web products.

So given this background and the potential rewards on offer what works and what doesn't in this early beta release? Let's take a look...

Judging a book by its cover

The Seek team have yet to reveal anything about its inner workings so in this beta release all that can really be judged is the front-end usability and functionality. Unfortunately look and feel wise Seek is nothing to write home about. The interface is not confusing, it just has that late-20^th Century database feel that the cool 'Web 2.0' crowd left many years ago. This may sound immaterial but if the target audience is aesthetically motivated people like designers and architects the user interface has to look really good. Adobe understand this and make every one of their websites a work of art, for example just checkout their latest acrobat.com site. This is not to say that Seek should become one big Flash animation, they just need to break down the grid layout, pare back the interface elements and add a dedicated interface/graphic designer to the team.

Aspects that work

Aesthetics aside a good deal of Seek's exposed functionally works well; namely the classification, filtering and linked file support. Whilst these may not not seem that exciting they do illustrate the developers have a solid understanding of what AEC professionals are interested in getting out of Seek.

Classification

Seek applies multiple classification systems to the data stored within its index. As a result rather than coming across as a want-to-be Google for AEC search, the experience is more akin to the 90's-era manual categorisation by Yahoo. This may not sound sexy as we approach 2010 but it actually works pretty well. Experience has shown that performing 'dumb' searches within the rigid world of architecture generally is not very productive. When looking for AEC content we will have specific contexts in mind and classification systems help define their respective boundaries. By limiting search results to a specific subset of the building industry the potential for finding what you are looking for increases dramatically. Unfortunately the option does not seem to be available to perform a sub-search within a specific category. For example there did not seem to be a way to search for the term 'stainless steel' within the Transportion section of the CSI MasterFormat category. It is possible to filter the results from a category search but this is not as flexible as being able to define a custom search term within a category.

What would be interesting to learn about is how these classification systems are applied to the items in the index. Do the manufacturers have to manually define which categories their models/specifications fall under, or have Autodesk developed some intelligent algorithm that classifies incoming data similar to that described in this paper by Robert Amor? If it is the later then that would be very cool, but how accurate is it? Along this same line of thought is whether Seek can add other classifications systems to its index and if so how would this 'foreign' semantic system be mapped to the existing entities in the index? For example in New Zealand many practitioners use the CBI classification system, so for Seek to be accepted in this locale it would ideally need more than just the three categorisation systems currently supported. Personally I hope Autodesk do have plans for supporting a plethora of categorisation systems through some sort of mapping algorithm (i.e. system A, section 1 = system B, section 3 & 4). Such a mechanism could never be 100% accurate but it would make Seek more appealing to international users and future proof it for new categorisation systems which will surely emerge.

Filtering

Seek offers a wide range of attributes to filter on once a category or basic search term has been defined. This mechanism enables quick culling of large sets of results to identify a couple of the most relevant models or specifications. In this regard Seek behaves more like an e-commerce site rather than a search engine because the emphasis is not on providing you 50 relevant suggestions but one or two specific answers. At the moment it is difficult to test the usefulness or performance of the filter functionality given the limited size of the current search index. Still the interface is intuitive, so even when there are only ten search results the ability to quickly filter them still feels useful. This maybe because the filters provide a visual prompt as to how you can drill down to the product you really want, i.e. arctic white, gun grey or stainless steel window frames?

Seek's search results with its filtering system on the left (click to enlarge)

The question remains as to exactly how Seek facilitates filtering of its index. Are the filterable attributes automatically derived from indexed data or are they manually defined by the manufacturer or Autodesk themselves? The solution to this question is a double edged sword; an automatically derived filtering mechanism enables efficient scale (indexing bots) whilst a manual process provides a high degree of trust. Unfortunately automation generally comes at the expense of accuracy and manual data entry will stunt the index's growth. To be judged a success Seek needs to balance these demands to build a large yet accurate index. If it fails in this task the end result will be an index which is large and inaccurate, or accurate but too small to care about. Neither alternative will be suffered by users long before they go back to their conventional, paper-based specification libraries.

Linked file support

Of all the major AEC software vendors Autodesk is arguably one of the worst for equally supporting competing formats. Fortunately this attitude seems to have changed when it comes to Seek as it prominently supports a range of non-Autodesk formats such as Bentley's DGN and Adobe's PDF. The first time you download a linked file you must agree to Autodesk's terms of use which is interesting considering the files themselves are not hosted by Autodesk. Now I am no lawyer and there is probably good reason for this, but it still feels strange to have a search engine imparting terms of use on the content it links to.

Most products have numerous file attachments. Unfortunately the preview option is very limited at the best of times (click to enlarge)

In the future it would be great to see a more comprehensive file previewer be made available. There are currently thumbnail images of many referenced files but something that behaved in a similar way to Apple's Quick Look would be really useful. Whilst this may sound far fetched it is not an impossible considering Autodesk have Freewheel, a Web-based 3D model viewer. If used alongside a Web-based document viewer similar to Scribd it would allow users to quickly preview the contents of files without having to commit to a download. The inclusion of such functionality would not only enrich the user experience but open up further revenue opportunities. Attached files could be made available online through Seek for free and offered for download at a reasonable price from Autodesk or the third-party. This revenue model works well for many book publishers and stock photo distributors, so in theory the same could be achieved within the AEC industry if the content holds enough value.

Aspects that could do with development

Growing the index

To help build out the search index the Seek team will need to make public how interested third parties can get their own content into it. Given the intention of the project it is obvious that strangers off the street will not be granted publishing rights and for the integrity of the index it is important that some validation processes be put in place. That being said in the interests of long-term success the route to participation should be a public, devoid of NDAs, exchanges of money or unnecessary bureaucracy.

When it comes to creating an accurate and timely index blog search engines have demonstrated that the ability to push structured data to the search engine is far more efficient than using a conventional Web crawler approach. With this capability the very nature of the catalogue would shift from that of an online book to a living entity. If suppliers were able to push availability details and news about a particular product into the index it would mean that any consumer of Seek data would also be able to utilise this information. For example consider the following scenario:

An architect assigns a product specification from Seek to a component in their AutoCAD model. On making this assignment they select to be notified of important information on this product until the project is complete. In effect this configures AutoCAD a subscriber to the product-specific RSS feed on Seek. As any new information is announced by the supplier, for example it will be discontinued in December or a national safety test found it did not perform well under certain situations, then anyone opening the model would be alerted of this news. Note: The RSS product feeds themselves would not necessarily need to be contained within the AutoCAD file itself, they could exist in an externally referenced OPML file.

There are two proven means of achieving this push-like model, Seek could consume supplier generated Atom feeds, or Google Sitemap-like structured documents. In both cases the supplier should be able to 'ping' Seek to inform it that new content is available for indexing and be confident that in a timely manner Seek would be accurately displaying the data to users. The key behind such an approach is that the AEC-specific data (classifications, physical properties, etc.) within these feeds would need to be conveyed in a non-proprietary format. Whilst it would be easy for Autodesk to write their own format a wiser approach would be to take an existing Web format, for example GData and build upon that. By starting out with a ubiquitous format it makes the task of gaining third-party support much easier and it ensures effort expended by suppliers to create Seek feeds can be leveraged by other search engines and compatible software.

Exposing the Data

Crucial to the success of Seek is its web service component, i.e. the ability for other applications on the Web or desktop to use the data this service returns. Whilst Autodesk currently describe Seek as a 'web service' this is not the case in the contemporary sense. Seek's value will increase exponentially once it makes the leap from a visual catalogue to a service which forms the functional backbone of desktop and web-based applications. For example consider the following two scenarios and how a service-centric Seek could have:

An architect working on an ArchiCAD model is about to make a design decision regarding a set of shelves. The ArchiCAD Seek plug-in recognises that this is the case because the user has selected the appropriate modeling tool and layer set. The plug-in queries Seek and returns a list of appropriate 3D models based on the properties of the project (a residential dwelling in Auckland). The plug-in filters and orders this data to suit the architect's personal preferences - in this case supplies that satisfy green building standards. Without a single extra mouse click Seek in partnership with the desktop software is able to present a reasonably intelligent set of shelving options. This task, which would have taken hours of searching through conventional product catalogues and manual 3D modelling is completed in seconds.

A design team working on a medium sized industrial plant in Sydney is having a discussion within their Project Intranet on appropriate door fixtures to use. None of the available products seem to suit the requirements so the outstanding issue is recorded as something that needs attention later. The Intranet software constructs a Seek search query out of the issue's defined parameters and begins regularly checking Seek for potential matches. Weeks pass and the problem is forgotten about by the team. Then one afternoon the Intranet service issues an email informing the interested parties that a local supplier has just that morning started producing a new line of industrial strength fixtures which satisfy the design requirements.

In both these scenarios the use of web services transforms Seek from a user-initiated search tool to a context-aware information delivery service. With this relatively simple shift the value proposition of Seek from both the user and supplier's perspective changes immensely. The catalogue ceases to be a passive object and becomes a tool that can proactively solve decisions and sell products.

When it comes to exposing the Seek index via web services this should be implemented as simply as possible. This means following the example of many Internet-savvy companies and releasing lots of well documented, REST-based web services. These services should return simple XML (preferably Atom) or JSON data streams which any application developer can utilise. To support this API there also needs to be a set of Javascript components to enable non-developers (i.e. the CAD managers and "I.T. guys" of this world) to embed Seek functionality within company Intranets. Once Autodesk make this set of web services available it is a relatively easy task for third party developers to integrate Seek into their own applications to achieve the functionality described and more. Unfortunately the cloud hanging over this rosy picture is that Autodesk are not known for producing open and simple to use web services. Time will tell whether they can execute on such a strategy, but let's hope the thought has at least crossed their collected minds.

Sharing search results

Search results from Seek can be emailed to others for later reference but the mechanism for this is very primitive. This option feels like a nod in the direction of useful functionality rather than something that has been given serious attention. When it comes to externally sharing results the feature-set needs to be greatly expanded upon to be of real value. Currently it is not possible to email a subset of your search results or highlight specific results other than writing "here are the results and I prefer options 2 and 5". Also in need of development is the functionality surrounding the email format. Rather than emailing a dull URL there should be the option to send the selected results as a self contained message complete with thumbnails. Many (if not all) AEC companies use email as a quasi filing system, so a URL that displays different results over time is of little historical value. In effect what the email option needs to become is the equivalent of the Amazon or Dell 'wish list'. This way architects can have their clients look through Seek and build a list of 'things' they like the look of, i.e the digital equivalent of turning up to the practice with three Home & Garden magazines.

Seek does not use friendly URLs or have unique meta-data for each page. Consequently tools like Google Reader do not work well (click to enlarge)

Another way that specific items can be quickly shared or referenced is by hyperlinking. Unfortunately this is not made easy because Seek URLs are in no way 'friendly'. Take for example this URL that Seek generates for 'Window Guards':
http://seek.autodesk.com/#query=search%5Edetail%5EH4sIAAAAAAAAAEtMT7fKTU4vSizPyMzJsfILDrG0MDEGAOhs_2kWAAAA
Not only is this meaningless but the meta information within the Seek page does not change to suit the context being viewed. The title of the page is always 'Autodesk Seek' and the meta description a constant 'Autodesk Seek is the online source for building product information (...)'. As a result if you were to bookmark this page in a browser or online service the default bookmark attributes (the name and description) do not accurately reflect the subject matter of the page. In comparison if you take a look at a page from Amazon.com the URL, page title and description all clearly identify what the subject matter is:
http://www.amazon.com/Harry-Potter-Deathly-Hallows-Book/dp/0545010225/

This may sound a little pedantic but it is the details like this that make comprehension of hyperlinks possible for both people and computers alike. A failure of comprehension means that third-party services like Google Reader's ability to share interesting web pages with peers do not work so well, if at all when it comes to Seek.

Trusting search results

Whilst traditional web search engines like Google, Yahoo or Microsoft need to worry about 'search engine fraud' the factual correctness of the information is of little importance as long as it is considered relevant. In comparison for a service like Seek the user's ability to trust the information returned is exceptionally important. There is no point in using the service if it returns models or specifications that are incorrect or out of date. This issue becomes a real problem as the index grows, things get out of date and malicious suppliers attempt to game the system to boost sales by 0.1%. Even well intentioned suppliers and Seek itself are capable of making mistakes. Given these factors there needs to be a feedback mechanism in place for consumers to alert Autodesk and their fellow users of these problems. This feedback could be enabled passively or through active participation.

General purpose search engines establish 'correctness' through the concept of PageRank (i.e. if it is linked to it is probably right). Unfortunately for the closed and competitive world of architectural design this concept cannot be applied even if it was possible for Autodesk to go crawling the design plans of AEC professionals to identify which models and specifications are referenced the most. However it would be feasible to deploy an opt-in system within Seek where users could identify models and specifications they made use of regularly. For example during the drafting of construction details the CAD program could notify Seek whenever specifications stored in the index were referenced by the designer. In practice this would be similar to Google's Web History as the aggregate, anonymised data returned would help assist others to identify popular, and therefore by logical extension trusted, models and specifications.

Beyond passive observation is the ability for users to directly feed into Seek's index their own opinions and content. For example the real value of the Amazon web experience is not the search results but the user reviews. Basic online specifications is one thing, but knowing that someone in a very similar situation as yours found the actual product did not measure up to expectations is considerably valuable. However if this kind of conversation were to take place on Seek itself it may place Autodesk in an awkward position with suppliers who do not take kindly to poor feedback. Whether or not such functionality is possible will depend on the business model Seek eventually establishes. If the intention is to add value to Autodesk's other software offerings then such functionality is a possibility, but if Seek they are looking to generate an income stream through paid listings from suppliers then the chances of seeing such functionality is low. Personally I feel it would be great to see Autodesk take the open road and encourage user feedback on listings through a variety of mechanisms. Unlike dedicated online catalogue companies Autodesk can afford to take such a risk because their customer base is the decision makers in the building process, hence they stand to profit most when they give this group better decision making tools.

Working within an organisation or project team

Currently Seek is a standalone search service that lacks functionality specific to project work-groups. It would be useful to be able to register a practice or project account with Seek in order for selected index items or suppliers to be listed, shared and discussed within this social network. Many architecture practices work with selected suppliers and components that they are familiar with and trust. These trust networks would make the process of browsing, selecting and recalling past decisions easier for users because it establishes an internal hierarchy within the index unique to the individual or group (i.e. another layer of meta-categorisation). For example consider the following scenario and how this trust network would be useful:

An architecture graduate is trying to pick commercial kitchen fittings for the first project he has been issued at his new job. The graduate has no experience in this field so they elect to filter Seek search results to suppliers trusted by their architecture practice. On selecting a fitting Seek informs the graduate that a similar fitting was used in a project two years ago by a senior partner. This is reassuring in two ways; firstly it indicates that this choice is not completely off track (a senior partner made a similar decision), and secondly it identifies a pathway for future research ("hey John, how did that kitchen work out last year?").

Whether or not all the historical information referenced in this scenario would be stored in the Seek 'cloud' would be an operational preference of the organisation or project team. Even excluding the ability to track historical choices made by the collective the ability to create a Seek user account and limit search results to an identified subset of other users would go a long way to achieving this functionality.

Leveraging the social

Without a doubt the "Web phenomenon" of the past two years has been the move towards social-centric networks (e.g. Facebook, MySpace and Twitter). AEC professionals subscribe to magazines and catalogues, visit interesting buildings and attend lectures because they want to know what their peers are up to. Implemented correctly Seek would enable users to track what was 'in' and what was 'out' in the same way that a fashionista attends catwalk shows. Where this could get really interesting is in the field of paid recommendations like that of Amazon's Associate programme . The idea being that if my actions or recommendations influence the purchasing decision of another party then shouldn't I be rewarded by the supplier in some way? The supplier has sold a product, the third-party has solved their problem and Autodesk has promoted their brand - doesn't the person who initiated this transaction deserve some reward? Greed is a powerful motive for participation and a rewards mechanism would certainly encourage use. Such a system would not neccessary have to involve financial rewards, it could operate with the concept of 'Autodesk Points' which acrue in the same way as frequent flier miles and can be redeemed for discounts on Autodesk software. After all a similar concept to this made the AdSense programme Google arguably the largest advertising agency in the world.

In Conclusion

Seek is a promising step in a potentially very interesting direction for Autodesk. If they can continue to evolve its functionality and integrate it into their other product offerings it stands a higher than average chance of becoming a valuable resource within the AEC industry. Central to its success is a demonstration that the search index is capable of growing quickly and that the underlying data can be easily exposed to third-party developers. With these improvements and a lot of luck it is not unreasonable to suggest that Autodesk Seek may become the Google/Amazon (Goomazon?) of AEC specific search and specification.

Swoogle semantic search

David — Thu, 24 Aug 2006 22:30:17 +0000

Swoogle is a semantic search engine project by the Computer Science and Electrical Engineering Department at the University of Maryland. They have taken a useful approach by ripping off Google's interface so that new users understand how to use the tool from the very beginning. Unfortunately the major limiting factor of the experience seems to be the results page. Rather than creating human readable snippets of information below each of the links the results simply output a snippet of the RDF code from the returned file. RDF is difficult for computers to understand so asking people to make sense of a brief quote is expecting the possible. As a consequence it is difficult to meaningfully interrogate the results in order to find content that is most relevant to you, in fact in practice the value of the returned results seems almost zero.

I am not sure how semantic search engines (if they ever come into fruition) will look but I do not think they will resemble Google. It seems like the real value of semantic search lies in identifying the user's search context and linking this to their attention stream (something Steve Gillmor talks about a lot). It is way too time consuming to describe many of the semantic nuances you wish to include in your search, and in search engines where you must manually enter this data a 'dumb' Google search is of more immediate value because of its speed. In the time that you describe all your semantic search parameters you could perform a dozen standard searches and probably find twice as many relevant results. If on the other hand the semantic search engine tied into your working context, say for example it new were working on concrete detailing within your CAD file, and it had access to your attention meta-data so that it could determine relevance then the potential for semantic search could be very strong. However whilst these two essential building blocks do not exist semantic search will always be either too time consuming and difficult to use or so simple that it is not as effective as plain old Google approach of using PhD's in tanks to get through walls (from Adam Bosworth's MySQL conference speech).