stressfree - google

Integrating Google Site Search into SilverStripe

David — Tue, 15 Sep 2009 08:32:53 +0000

SilverStripe is an excellent, user-friendly content management system but its internal search functionality is, to put it kindly, useless. Fortunately with Google Site Search you can embed a Google-powered custom search engine into your SilverStripe site. Doing so requires a paid Site Search account, pricing for which starts at $100/year.

This tutorial explains how to integrate this Google Site Search XML feed into your SilverStripe site. Doing so has a number of benefits over the standard means of integrating Site Search, namely:

No Javascript is required to display results within the SilverStripe site.
The user is not taken to a separate, Google operated website to view results.
The look and feel is consistent with the rest of the SilverStripe site.
Multiple Site Search engines can be integrated into a single SilverStripe site.
Site Search results pages are integrated into SilverStripe's management console.

Note: To integrate Site Search into SilverStripe using the described method a Site Search plan must be purchased as this provides results in XML. The free, advertising supported, Site Search engine does not provide search results in XML and cannot be used.

Loading XML data from an external source

Before the search page can be added to SilverStripe we need a reliable means of loading XML content. This is complicated by the fact many Web hosts disable PHP's built in URL fetcher (fopen) with the following php.ini directive:

allow_url_fopen = Off

Assuming it is installed, the cURL can get around this restriction, hence the XmlLoader helper library includes both methods (cURL is used by default in search.php).

Create a XmlLoader.php file in your SilverStripe's mysite/code directory with the following contents:

mysite/code/XmlLoader.php

<?php
class XmlLoader {

public function pullXml($url, $parameters, $useCurl) {
$urlString = $url."?".$this->buildParamString($parameters);

if ($useCurl) {
return simplexml_load_string($this->loadCurlData($urlString));
} else {
return simplexml_load_file($urlString);
}
}

private function loadCurlData($urlString) {

if ($urlString == -1) {
echo "No url supplied<br/>"."/n";
return(-1);
}

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $urlString);
curl_setopt($ch, CURLOPT_TIMEOUT, 180);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec($ch);
curl_close($ch);

return $data;
}

private function buildParamString($parameters) {
$urlString = "";

foreach ($parameters as $key => $value) {
$urlString .= urlencode($key)."=".urlencode($value)."&";
}

if (trim($urlString) != "") {
$urlString = preg_replace("/&$/", "", $urlString);
return $urlString;
} else {
return (-1);
}
}
}
?>

With the helper library in place to load the XML, it is now time to implement the SilverStripe "search" page type and logic. Create a search.php file in your SilverStripe's mysite/code directory with the following contents:

mysite/code/search.php

<?php

require_once 'XmlLoader.php';

class Search extends Page {
static $db = array(
'GoogleSearchId' => 'Text',
'NoResults' => 'HTMLText',
);
static $has_one = array(
);

function getCMSFields() {
$fields = parent::getCMSFields();

$fields->addFieldToTab('Root.Content.Main', new TextField(
'GoogleSearchId', 'Google Site Search ID'), 'Content');
$fields->addFieldToTab('Root.Content.Main', new HtmlEditorField(
'NoResults', 'No results message'), 'Content');

# Remove the content field
$fields->removeFieldFromTab("Root.Content.Main","Content");

return $fields;
}
}

class Search_Controller extends Page_Controller {

function SearchForm() {
$input = array_merge($_GET, $_POST);
$query = $input['q'];

$output = "<form class=\"search\" action=\"/search/results\"><fieldset>";
$output .= "<input type=\"text\" size=\"40\" name=\"q\" value=\"$query\"/>";
$output .= "<input type=\"hidden\" name=\"p\" value=\"1\"/>";
$output .= "<input type=\"submit\" value=\"Search\"/>";
$output .= "</fieldset></form>";

return $output;
}

function SearchResults() {

$output = "";

$input = array_merge($_GET, $_POST);
$page = isset($input['p']) ? $input['p'] : '1';
$query = $input['q'];

$perPage = 10;
if ($page < 1) { $page = 1; }

$xml = $this->getGoogleSearchResults($this->GoogleSearchId, $perPage, $page, $query);
$results = $this->parseGoogleSearchResults($xml);

$totalResults = $this->getResultCount($xml);

$output .= $this->getFormattedResults($results);

if (count($results) == 0) {
// Show no results message
$output .= $this->NoResults;;
} else {
// Append paging
$output .= $this->getPagingForResults($totalResults, $query, $perPage, $page);
}

return $output;
}

private function getGoogleSearchResults($googleId, $perPage, $page, $query) {

$startingRecord = ($page - 1) * $perPage;

$url = "http://www.google.com/search";
$parameters = array();
$parameters["client"] = "google-csbe";
$parameters["output"] = "xml_no_dtd";
$parameters["num"] = $perPage;
$parameters["cx"] = $googleId;
$parameters["start"] = $startingRecord;
$parameters["q"] = $query;

$XmlLoader = new XmlLoader();

return $XmlLoader->pullXml($url, $parameters, true);

}

private function parseGoogleSearchResults($xml) {

$results = array();

$attr["title"] = $xml->xpath("/GSP/RES/R/T");
$attr["url"] = $xml->xpath("/GSP/RES/R/U");
$attr["desc"] = $xml->xpath("/GSP/RES/R/S");

foreach($attr as $key => $attribute) {
$i = 0;
foreach($attribute as $element) {
$results[$i][$key] = (string)$element;
$i++;
}
}
return $results;
}

private function getFormattedResults($results) {

$output = "";

if (count($results) > 0) {
$output .= "<ul class=\"results\">";
foreach($results as $i => $result) {
$title = "";
$url = "";
$desc = "";
foreach($result as $key => $value) {
if ($key == "title") {
$title = $value;
}
if ($key == "url") {
$url = $value;
}
if ($key == "desc") {
$desc = $value;
}

$output .= "<li><a href=\"$url\">$title</a><p>";
$output .= str_replace("<br>", "<br/>", $desc);
$output .= "</p></li>\n";
}
$output .= "</ul>";
}
return $output;
}

private function getResultCount($xml) {

$totalResults = 0;
$count = $xml->xpath("/GSP/RES/M");
foreach($count as $value) {
$totalResults = $value;
}
return $totalResults;
}

private function getPagingForResults($totalResults, $query, $perPage, $page) {

$maxPage = ceil($totalResults/$perPage);

if ($totalResults > 1) {
$output = "<div class=\"searchPaging\"><p>";

for($pageNum = 1; $pageNum <= $maxPage; $pageNum++) {
if ($pageNum == $page) {
$output .= " <strong>$pageNum</strong> ";
} else {
$output .= " <a href=\"".$this->AbsoluteLink()."results?q=$query&p=$pageNum\">$pageNum</a> ";
}
}
$output .= "</p></div>";
}
return $output;
}
}

?>

This file defines a search page type with two fields, a Google Search Id and an HTML field that is displayed if no search results are found. As this page does not have any content of its own the default SilverStripe content field is also disabled to avoid confusion.

With the backend logic in place it is time to implement the templates. The templates themselves will vary from site to site, but the examples given are good starting points. There are two templates, one which simply displays the search box and a second that displays the results.

Create a Search.ss file in your SilverStripe's mysite/templates/Layout directory with the following contents:

mysite/templates/Layout/Search.ss

<% if Menu(2) %>
<div class="pageWithMenu">
<% end_if %>
<div class="page">
<% if Menu(2) %>
<div class="content contentStandard">
<% else %>
<div class="content contentFull">
<% end_if %>
<h1>$Title</h1>
<div class="contentWrapper">
$SearchForm
<div class="clear"></div>
</div>
</div>

<% if Menu(2) %>
<div id="sidepanel">
<% include SideBar %>
</div>
<div class="clear"></div>
</div>

<% end_if %>

Now create the results page named Search_results.ss in your SilverStripe's mysite/templates/Layout directory with the following contents:

mysite/templates/Layout/Search_results.ss

<% if Menu(2) %>
<div class="pageWithMenu">
<% end_if %>
<div class="page">
<% if Menu(2) %>
<div class="content contentStandard">
<% else %>
<div class="content contentFull">
<% end_if %>
<h1>$Title</h1>
<div class="contentWrapper">
$SearchForm
<div id="searchResults">
$SearchResults
</div>
</div>
<div class="clear"></div>
</div>
</div>

<% if Menu(2) %>
<div id="sidepanel">
<% include SideBar %>
</div>
<div class="clear"></div>
</div>

<% end_if %>

Note: The content of these two files will vary depending on your site. In the above example a SideBar include file is used to load the secondary menu.

With the backend logic and template files in place it is time to rebuild the SilverStripe database so that the new page type can be recognised. Enter the following (modified) URL into your browser: http://yourwebsite/dev/build?flush=all

All going well the rebuild command will execute correctly. If it does browse to the administration section of your site and create a 'search' page type.

The search page type in the create menu

On the search page enter your relevant Google Search Id and Results Not Found message. For the page URL use /search as this is hard coded into the search.php file. It is possible to change this URL (or use a dynamic one) but for the purposes of this tutorial it is not necessary.

Note: You can get your Google Search Id from the Google Search administration console, or it can be found within the embed URL used in the Javascript or external search forms.

The search page settings (click to enlarge)

Once published open the page and try a search. Assuming your code and settings are correct the Google search results should be displayed within your SilverStripe page. Now all there is left for you to do is style the results.

For an example of this technique at work, checkout the Parliamentary Counsel Office's search interface which is implemented using the method just described.

Google O3D may finally bring 3D to the Web

David — Wed, 22 Apr 2009 11:23:22 +0000

Today Google released a very early preview of O3D, a cross-platform, open source plug-in that enables OpenGL accelerated graphics within Web browsers. Delivering 3D graphics within browsers is not a new thing, (remember VRML?) but what makes this initiative promising is that it works on all platforms and is backed by Google. Performance-wise O3D seems very snappy when compared to alternatives such as Flash 3D. As a result some of the initial demonstrations are very impressive, and it hints at a future where Google Earth and SketchUp leave their desktop roots behind to become pure web applications.

From an architectural collaboration perspective O3D is valuable for a number of reasons. Firstly in a review situation it would mean remote clients can experience 3D designs without having to download, install and learn a separate "viewer" application. Likewise within an intranet such functionality would be valuable when navigating a project or company knowledge base. Whereas at the moment textual (web) data is quite distinct from 3D models, in a O3D-enabled future the two could be seamlessly intertwined in a variety of powerful ways. Finally by freely distributing 3D capabilities to everyone with a browser O3D opens up the possibility for new types of 3D-centric web applications that allow all design team participants to more effectively communicate ideas with one and other. For examples of these potential markets checkout the section 'Where will Dragonfly land?' in my earlier 'Autodesk Dragonfly emerges from its larvae' post. The people behind O3D's demo applications seem to appreciate this fact too, because the "Interiors" demo showcases such a tool (i.e. it 'copies' Project DragonFly).

A 3D, Google Trends view of the Earth in O3D

I have not yet created or rendered content using O3D, but if I get the chance there is a significant amount of developer documentation online. Anybody with web development experience should feel at home because the rendering engine is initiated and controlled using Javascript. This is a great choice, not only does it make the technology accessible, but it means 3D can be integrated into "traditional" web applications using standard Javascript event handlers. For example if you were creating a web-based CAD application you could create the majority of the user-interface using standard HTML/Javascript and leave O3D to handle just the rendering of the model window(s). Such an approach also means developers who already have already created 3D web applications in Flash and HTML could leverage O3D without being completely rewritten.

Overall O3D comes across as a very powerful and surprisingly polished early preview. Google are obviously very serious about 3D in the browser, and this implementation seems to be the most promising yet.

Safari 4 on Windows looks good

David — Wed, 25 Feb 2009 23:37:47 +0000

It is really nice to see Apple have dropped the unsightly bushed metal look in their latest Safari 4 beta on Windows. Whilst that aesthetic worked fine in OSX, in a Windows world next to "traditional" applications it came across as being really unsightly. Let's hope this is a sign iTunes will take on more of a native Windows aesthetic in a future release.

Identical twins: Safari 4 and Google Chrome (click to enlarge)

However what is really crazy is how much Safari 4 looks and behaves like Google Chrome on Windows. Apple's developers have spent the last few months "borrowing" many of Chrome's features, such as the top-sites view and browser tabs on top, but on Windows they have gone a step further with the menu system itself. This is not a bad thing as the result looks nice, but beyond the little Google logo in the top right, your average Windows user is going to have a hard time telling the difference between the two browsers.

Now if only the Apple engineers would implement process isolation for each tab or window like in Chrome. It is hugely annoying to have all the Safari windows disappear just because somebody at Adobe cannot write a stable Flash plug-in.

StressFree Webmin theme version 2.0 released

David — Mon, 16 Feb 2009 09:30:05 +0000

Version 2.0 of the StressFree Webmin theme adds limited support for Google Gears. Whilst this addition does not provide true "offline" functionality it does help speed up the performance of the Webmin theme, especially over slow connections. In a nutshell Gears is used to cache a significant portion of Webmin's static content (images, javascript and CSS).

To enable Gears click the Gears link on the top right of the screen and follow the instructions. If your browser does not have Gears installed you will be provided a link to install the library. A demonstration of this functionality is given in the video below. There is not much to see, but the performance improvement is noticeable if you use Webmin a lot.

The concept and initial code for this functionality was kindly provided by Dwi Kristianto. I have integrated his proposal into the core StressFree theme and in the process tweaked it a fair bit to ensure it works in a variety of Webmin configurations.

This release also fixes a login screen rendering bug introduced since version 1.450 of Webmin. Unfortunately to resolve this bug backwards compatibility with Webmin versions > 1.450 is broken. If you are not using Webmin 1.450+ it is recommended that you do not install this theme, or better yet, upgrade Webmin prior to installing this theme.

The updated theme can be downloaded from here.

Google Chrome rethinks the browser

David — Wed, 03 Sep 2008 11:15:45 +0000

Yesterday (after a comic strip teaser) Google finally took the plunge and released their own web browser named Chrome. For years they have had a defacto relationship with Mozilla Firefox, but now they have decided to go it alone with their own, radically different offering. How this affects the Firefox/Google relationship is anyone's guess, but presumably for Mozilla having your number one revenue stream release a competing product is not a good sign.

So why should I care?

Rather than a simple re-branding of Firefox, Google Chrome is a completely new beast built on top of the WebKit rendering engine (the same engine that drives Apple's Safari). Innovation is a term used pretty lightly in the technology industry, but in this case Google has really tried to break conventions and create something that is genuinely a generation better than the competition.

The Chrome interface with its Task Manager (click to enlarge)

Process isolation comes to tabs

The biggest conceptual leap the developers have made is thinking of each tab as its own distinct process. Traditionally your browser has run as a single process, which means when one tab or window goes haywire the whole thing goes up in a puff of smoke. By running each tab as a distinct, protected process the browser gains a level of robustness never considered possible. In fact in some respects Google Chrome is a lightweight operating system unto itself, it even has its own Task Manager for monitoring and selectively killing errand tabs.

What it looks like when you kill a tab (click to enlarge)

A lightweight, tab-centric interface

Google have also taken the interface concept of tabs to a new level by making it the primary interface element. Whereas conventionally the browser tab has been subservient to the navigation bar, in Chrome this relationship is reversed. Tabs are the high-level interface element which everything else is related to. Also gone is the 90's concept of a 'home page'. When you create a new tab in Chrome you are presented with a very tidy activity overview which displays frequently visited sites, new bookmarks, closed tabs and of course a big search box.

The activity overview tab (click to enlarge)

Chrome is fast... really fast

The decision to use WebKit over Mozilla's Gecko rendering engine helps in the speed stakes. For a while now Apple has been trumpeting how fast their browser is, but now Google have trumped them with this effort. Given that the majority of Google's web properties rely heavily on Javascript it is unsurprising to see that they have used a new and very fast Javascript engine. Thanks to the recent developments in Javascript run-time technology the language can no longer be considered 'dead slow'. Whilst it may not be as fast as compiled C, for most day to day tasks it is now more than adequate.

What does it mean for me the poor old web developer?

With Chrome's WebKit rendering engine you can be fairly confident that if your website displays correctly in Safari 3 it will work fine. In all the tests I did there were no obvious rendering differences between the two browsers. As far a Javascript goes this will be a different story as Chrome uses a custom engine. My guess is as long as you stick to well trodden tracks and make use of popular Javascript libraries such as Prototype, JQuery and Dojo you will not experience too many issues. To help developers out a Firebug-like inspector is included which lets you drill down into your HTML and Javascript for easy troubleshooting. Chrome also ships with Google Gears built in, so if you have been considering whether adding offline support to your applications is worthwhile, now maybe a good time to look into it.

The HTML/Javascript inspector (click to enlarge)

The beta for Windows is available now, OSX and Linux to follow

Like most things, first out of the starting blocks is a Windows beta release. It sounds like OSX and Linux versions will not be too far behind, but releases will not be anytime this month. Installation is very straightforward, simply run the small (475kb) installer and it will go ahead and pull the latest release off the network. When the browser first starts you can import your settings from Firefox. Then to ward off any anti-trust lawsuits a prominent opportunity is provided to change the default search engine to something other than Google (does anyone actually use Live.com?).

The 'don't sue us' change the default search engine pop-up

Final thoughts

Google Chrome is a great beta release of a truly next generation web browser. For anyone like myself who writes web applications for a living the introduction of tab-level process isolation is a very welcome sight. Whether or not the average user accepts Chrome will be another story. Many businesses will hesitate due to the privacy cloud hanging over anything from Google (even though the majority of their desktops, servers and mobile phones are exclusively Microsoft). Even if Google Chrome does not succeed in the market lets hope that some of its innovations at least rub off on its competitors.

Web Forms in Google Docs

David — Thu, 07 Feb 2008 21:59:33 +0000

Yesterday Google Docs users where given the ability to define spreadsheet-backed Web Forms for gathering feedback from people. The interface for doing this is very clean and it is a piece of functionality that could prove to be a real time-saver for those wishing to conduct quick surveys or gather structured feedback on a topic. Even though Google Docs doesn't compare well in a straight-up comparison with Microsoft Excel it is Web-centric functionality like this that put it ahead, especially in environments where spreadsheets are used more for communication than data crunching.

Below is a screen-cast demonstrating the use of Web Forms within Google Docs. It runs for a couple of minutes and covers the creation of the form, filling it out and receiving the data.

Although this new functionality is very powerful there are many areas where Google Docs need to evolve in order to become a compelling application suite.

Copying Documents

It may seem trivial but there does not seem to be a way of making a copy of a document within Google Docs. Setting up a Web Form takes time and the ability to make a copy of an existing form without modifying the original would seem like a simple yet very useful piece of functionality to include.

Templates

Along the same line of thought the ability to define and share document templates would be very useful especially in a business environment. When it comes to data collection a lot of companies have standardised forms that need to be completed at various times. The ability for staff to create a new document based on one of these templates would save time and ensure that data/form consistency is maintained. Even better would be if these templates could be exposed as list of hyperlinks so that staff could browse and instantiate documents from websites outside of the Google Docs environment (i.e. the company Intranet).

Smart logging of respondents

Whilst a timestamp is recorded in the spreadsheet for each set of results there is no option to automatically record who responded to the Web Form, it is up to the Web Form designer to ask. It would seem relatively straight forward to embed an id key into the returned Web Form so that respondents could be more efficiently monitored. Also as currently each response is in effect anonymous there is no validation that someone has not submitted the form multiple times. Again including some form of identification within the returned form would mean that respondents could be limited to only one response. In a casual environment this is not such a big deal, but in formal studies or for business use having this level of validation is a necessity.

Embedding Web Forms in Text Documents

The current client-side presentation of Web Forms is pretty sparse even by Google standards and quite often a Web Form will need to relate to a larger document. This maybe because the feedback sought is about the referenced document or there maybe legal requirements to satisfy such as a statement of ethics and privacy terms. It would seem like the best way of solving this problem would be to enable Web Forms to be embedded within other Google Docs document types. This would provide a flexible tool-set for presenting and gathering feedback from people, especially if more than one Web Form could be embedded within a document or presentation.

Even though these functional shortcomings currently exist the Web Form functionality in Google Docs is a brilliant step towards creating a collaborative office suite that isn't just a striped down copy of Microsoft Office. Hopefully this represents just the tip of the functionality iceberg and over time we see its feature-set and integration with other Google offerings expand and evolve.

google

Off-line enabling existing applications with Gears

David — Thu, 31 May 2007 12:21:28 +0000

Today Google released their open source Google Gears extension for Firefox and Internet Explorer. Also announced at the same time was upcoming support for the Safari and Opera browsers and inclusion of the technology into Adobe's Apollo framework. All these things should help make the concepts underpinning the software fairly ubiquitous. You may at this point be asking yourself what is Gears? In a sentence it is a set of Javascript API's and supporting backend functionality to enable offline use of Web applications. For more information on the subject I would recommend checking out David Berlind's recent podcast interview with Linus Upson of Google on the topic.

Google Gears is big news because a Web-based application's biggest stumbling block has always been the fact that if there is no Internet access your application stops working. If you are like me and write Web-based database applications then this is really interesting because it opens up opportunities for your users to access their data in a variety of ways that are currently impossible. For example WhichDoctor manages physician information ranging from their contact information through to their training history. From my perspective it would be nice for a user of this application to be able to pull down a sub-set of this data so they can take it away to a secluded beach to work on for the weekend. Gears helps in realising this functional requirement on the browser but for existing and complex applications like WhichDoctor there is still a great many problems to overcome before such a possibility becomes reality.

Off-line enabling a legacy Web application

Gears is great if you are starting out with a brand new application because you have the opportunity to design the architecture and interface from scratch to work seamlessly with this new on-line/off-line paradigm. However in an existing application where there is already a well-entrenched user-interface and code base that cannot be simply thrown away taking advantage of Gears is a more complicated process. For example it is unrealistic to suggest that each client keep even partial copy of the entire database schema if it is complicated and has a lot of business logic associated with it. In this case it is my opinion (at the moment) that the wisest course of action is to utilise the page-caching capability of Gears alongside a greatly simplified browser-based SQL database for basic searching. Such a strategy would provide a limited yet functional off-line experience whilst working within the constraints of the existing codebase and database schema.

In the above diagram above the user of the application would identify a subset of the total database population they would like to have 'synced' to their browser. The server would return this subset in XML or JSON format so that it could be parsed by the browser and inserted into the off-line database. The resultset would include basic search fields, for example if related to people it would break the names down to their first and last components so that relatively basic, browser-side searches could be performed (e.g. everyone with a last name of Harrison). Once inserted the routine would then loop through the dataset and request the associated HTML 'view' of each piece of data from the server for storing in the file cache. During the server-side page rendering process any application functionality not available in the offline, cached version would be excluded to ensure a smooth user experience (there is no point confusing the user with options that do not work). With the dataset and cached pages all stored locally on the browser the user would then be able to go on holiday and use the application in partnership with their data-subset quite happily.

Off-line write operations

The example given above does not include write functionality which would need to be implemented at a later date. Whilst possible with Gears version control in the event of an off-line data change would raise a couple of issues that would need dealing with. Firstly any offline data change would require some notification or re-rendering of the cached version of the page view so to notify the user the underlying data had been modified. Secondly when returning to an on-line state and the change committed to the primary database a consistency check would need to be made to ensure newer data did not already exist in the database, the last thing we want to do is overwrite new data with old.

Conclusion

Google Gears is a great development that holds significant potential for Web applications designed with it in mind. Off-line enabling existing Web applications, especially complicated ones is a challenging task that will only be achieved through a number of small but logical first steps. Still it is great to see off-line capability enter the vocabulary of the Web application. In the long-term off-line innovations such as Gears will have as significant an effect on the Web application market as AJAX has had in the last few years.

Google Apps gets serious but where is the offline option?

David — Mon, 26 Feb 2007 22:01:00 +0000

On February 21st Google released their anticipated Premier and Education versions of its Google Apps service. In a rare move for Google the Premier option actually costs the business US$50 per user per year rather than simply relying on advertising revenue. For your money you get quite a few more features: 10gig of storage per user, some interesting looking API's and a guarantee of 99.9% uptime with support.

On the whole deal is pretty good but Google has gone out of their way to stress that this service is targeted at the significant portion of employees who do not actually have email accounts on the basis that it is too expensive for the business to maintain them internally. Aiming at this market is a good strategy considering Google Apps lacks one significant feature that will see it struggle to gain acceptance in the majority of businesses where staff email is an entrenched and essential service: offline access.

Offline access from a businesses perspective is not a question of whether the computer is connected to an internal network but whether this internal network can access to the Internet. This internal network is generally very reliable but Internet access will never be 100% reliable due to its inherit complexities. Consequently whilst network administrators can comfortably guarantee their bosses that the internal network will function during business hours they cannot provide the same assurance with the Internet, especially considering many factors are beyond their control. From a business perspective this is an important issue because even when the Internet is not accessible employees need to be digitally productive. To do so they need to reliably achieve three basic tasks:

Send messages
Receive messages
Review old messages

In a GMail-centric environment if the Internet fails none of these primary tasks can be undertaken. In an internally hosted email environment employees are still able to review and compose new messages for sending as soon as Internet access is restored. Anyone will tell you having two thirds of some functionality available during a disaster is a lot better than none at all:

"I want the people to know that they still have 2 out of 3 branches of the government working for them, and that ain't bad." _{Mars Attacks}

How can Google Apps provide a similar level of offline service even in the event of Internet downtime? For the most part people have answered this question by pointing to emerging, browser-based offline caching technologies like that planned in Firefox 3. The problem with this solution is that it dictates the client-side software requirements. Web-based services like GMail have become popular because they allow any standards complaint Internet browser to provide the same level of functionality to its user. Therefore to move away from such a generic and easily deployable strategy is detrimental to the overall value of the Google Apps experience.

An outline of how a Google Apps Local Proxy could operate (click to enlarge)

I think a far more practical approach to providing offline access to business users would be for Google to offer a 'Google Apps Local Data Proxy' application. This would be a Java application installed on an office server or integrated into a Google search appliance. The data proxy would be accessed via the businesses' Web-proxy server (e.g. Squid, ISA Server, BorderManager) which would redirect calls to the proxy in instances where the Google Apps online service was not available or the quality of service was poor (slow response times). Using this methodology to resolve the offline problem would ensure all Internet browsers would provide the same level of Google Apps service. From a deployment and maintenance perspective the solution would be more efficient because administrators would be focused on one central service instead of troubleshooting issues at the Internet browser level.

The Local Data Proxy would be a local cache of some or all user data. The exact size of this cache would depend on the rules established by the system administrator and resources available. For example it could be set to cache all managerial level data and only emails sent to staff in the last week. Along with data the service would also cache offline versions of the Google Apps family and act as an SMTP relay. This would allow employees to use the productivity applications in an offline state almost oblivious to the fact the Internet is not available. Once Internet access is restored or reaches acceptable levels Local Data Proxy would synchronise all data changes back to the Google cloud and relay sent mail. From the users perspective nothing much would change apart from the fact they would regain access to all of their data and not be limited to a recent subset.

A setup such as this would not provide offline access to those people working outside the office network. Such an approach would also require the maintenance of an internal proxy service and the Google Local Data Proxy. This is a small price to pay especially considering most businesses already maintain an internal server infrastructure. Even with these shortcomings in mind such a service would still go a long way to solving many of the offline access issues posed without resorting to client-side, browser specific solutions.

Google Reader is now my primary RSS experience

David — Mon, 19 Feb 2007 19:37:54 +0000

I have been using NewsGator's NetNewsWire for Mac RSS reader for years but yesterday I moved all my RSS feeds over to Google Reader and closed NetNewsWire for good. There are lots of things to like about NetNewsWire like its tabbed windows, responsiveness and polished look and feel, but even with these things in mind it struggles to compete with the best online RSS readers out there today.

I played around with Google Reader in its first carnation last year but its interface went over my head. It seemed to me they were trying to be clever with news rather than give users an interface they were used to and actually wanted. Recently Google re-released Reader in a far more conventional guise making the overall experience very similar to traditional RSS readers with a few added bonuses.

Google Reader with the Google Reader Firefox extension visible in the bottom right corner of the window (click to enlarge)

So how does Google Reader, a Web-based application overcome NetNewsWire's OSX savvy advantages?

Responsiveness

Thanks to a hefty dose of AJAX Google Reader is almost as responsive as NetNewsWire even though it is running in a browser. Also to keep an eye on unread news I have also installed the Google Reader Firefox extension which gives brief summary of the current unread news without having to open Reader.

Aesthetics

Google's aesthetics with Reader seem to be significantly more polished than that of its more mature products like search and GMail. Sure it does not have all of the OSX bling that NetNewsWire supports but it is clean and stylish enough to be used without getting sore eyes.

Always synchronised

Being Web-based Google Reader is always available and synchronised no matter which computer you maybe using. NetNewsWire supports synchronisation to NewsGator's online reader and between different NetNewsWire instances but it has never quite worked. This all stems back to the fact NewsGator does not appear to be focusing on sorting out these bugs and seems quite content to let their users struggle along or move to another service.

Web-centric functionality

Google Reader offers little pieces of Web-centric functionality like a one-click option to share a selected story with other people via RSS or embedded within a web page. This means you can go through your news highlighting and sharing things that maybe of interest to others with minimal effort. Easy to use functionality like this just is not available in NetNewsWire without signing up to a third party service like del.icio.us.

Resource Usage

NetNewsWire can also be a resource hog once it has been running for a while with multiple tabs open. Generally this equates to about 100meg of RAM and anywhere between 300-600meg of swap space consumed. Google Reader on the other hand works within an Internet browser so it only uses as much as what its host browser needs. With Firefox this can equate to a lot of resources but by not having NetNewsWire open the overall system resource hit is not nearly as bad.

Conclusion

Given these benefits and the functionality of Google Reader making the decision to switch from a program I have used for many years was not that difficult. So far I am enjoying the experience and my Mac is welcoming the extra system resources. Rather ironically the only thing Google Reader needs now is search functionality and the ability to tag news items based on these search results and I will be content.

google
rss

Forget about fetchmail thanks to Gmail

David — Wed, 14 Feb 2007 11:33:15 +0000

Gmail's highly anticipated mail fetcher service went fully operational over the weekend. It is a nice addition for existing Gmail users but an even better option for those currently using fetchmail, getmail or some other pop3 checking tool for multiple email accounts. Traditionally you would have to run one of these applications or pay for a service such as Runbox to consolidate your email around a single account, but with Gmail this process has become significantly easier and cheaper.

Simply sign up for a free Gmail account (or log onto your existing one) and set Gmail to forward all incoming email to your primary account (assuming it isn't Gmail). Once the forwarder is in place configure the mail fetcher to retrieve mail from the accounts you want to check. Before you know it you have in place a free, spam filtered and highly redundant fetchmail/getmail replacement all without the cost and stress of maintaining your own server or application on your desktop for this task.

Google to replace Microsoft Exchange & Office at Pixar?

David — Wed, 14 Feb 2007 10:00:21 +0000

This Business Week article 'Google Steps Into Microsoft's Office' is interesting because it is the first time I have seen a rather large corporate entity (Pixar/Disney) express an interest in moving to a non-Microsoft, Web-based platform like Google Apps for your Domain. The other interesting point to note is published user-base of Microsoft Office Live which is sitting at 250,000 businesses, quite a respectable number for a new, non-free service.

I have been a user of Google Apps for your Domain for a while now on a number of domains and have got a few other well entrenched in-house email fans to sign up as well. By far and away the best things about it has been the spam protection and the fact that within an evening you can have a fully functioning email and calendar service up and running with no fuss at all. It has been slightly disappointing that the applications list has not grown to cover the Docs and Spreadsheets products already and many of the new GMail features such as the very handy retrieve mail service or increased storage sizes have not become available to Apps users sooner.

Chris DiBona Wellington meetup on Tuesday

David — Wed, 31 Jan 2007 01:03:14 +0000

This Tuesday the 6th of February Chris DiBona from Google will be in Wellington to talk open source. I am organising a meet-up and dinner with him at a local Wellington restaurant. There will be people from a wide variety of backgrounds there (government, small business, academic) and if you would like to come along please contact me and I will send you more details. I only hope the weather is as good next Tuesday as it was yesterday...

Google Trends: A cool comparison tool

David — Fri, 26 Jan 2007 00:57:36 +0000

When you think of Google you think search but a consequence of all of those searches is some very useful trends. This is where Google Trends comes into play. With it you can enter a few different terms (separated by commas) and then the tool will return a graph of the number of search hits for each term over the last few years. Not only that you can break down the search to look at from where people are searching for these terms. I did a quick trend analysis of C#, Java, PHP and Ruby and the results are pretty interesting, especially the fact that six of the top ten cities searching for these terms are in India (although in hindsight it is hardly surprising).

google

The emerging future: Software as electricity

David — Wed, 11 Oct 2006 10:04:23 +0000

There has been rumblings about software as a service and Web 2.0 revolutionising the way people work with their computers and data but very little to really illustrate this point. Whilst the average person is comfortable with having their email and chat hosted by Google, Microsoft or Yahoo the idea of businesses trusting third parties with hosting their software and data is a foreign one.

Companies like Google, SalesForce.com and now Zoho are out to change this attitude with a direct push at the business marketplace. You know an I.T. concept has momentum when it has its own conference, and now with the Office 2.0 Conference there is a showcase for business orientated, software as a service applications. TechCrunch has been doing a very good job covering the new applications released during this conference. Zoho seems very interesting from a software capability perspective although successfully marketing this service within the industry as it stands today will no doubt be difficult. However it is probably more likely that Zoho is concentrating on their application feature-set with an end-game of being purchased by a larger company (Yahoo, AOL). This larger company will then rebrand Zoho and use it to compete with Google's emerging online office suite and Microsoft Office Live.

Whilst promising the major barrier to serious online office migration is psychological rather than physical. Arguably the Web applications and infrastructure exists to move many traditional office tasks and accompanying data to a remote, online environment. Sun's CEO Jonathan Schwartz has been talking and blogging a great deal about businesses ability to centralise and externalise their data-centers and in a completely practical world this makes sense. However the largest barrier to all this is people's assumption that the Internet will go down. Unfortunately in many places this belief is a reality, such as with low-end DSL/cable connections in New Zealand, but at the high-end Internet connections are just as reliable as electricity or other established services.

As Internet reliability improves there will come a time when network access is viewed in the same light as phone, gas and electricity services; as long as you pay your monthly fee it will work. When that time comes there will no doubt be a significant shift in offices from internally hosted identity, email, file services and applications to remote services maintained by third parties. The office space and infrastructure savings alone will justify the remote hosting costs and more importantly removing the headache of maintaining internal data-center infrastructure will, in theory at least, allow businesses to concentrate on the things they do best.

The company that stands to potentially loose the most from all this is Novell as their primary market is internal server infrastructure. In an environment where office server infrastructure declines Microsoft will still have the desktop whilst online companies such as Google and Yahoo will be at a greater advantage compared to Novell with their compartmentalised application suites. In this light it is especially unfortunate that iFolder, Novell's most promising product in this emerging environment, appears to have been put out to pasture without it ever gaining the emphasis or adoption it so greatly deserves. Whilst it will be a long time before software as a service and the whole Office 2.0 concept becomes a reality when it does I hope Novell is not caught napping like they were with every other technology change that has occurred in the last ten years.

Google surprises all with its purchase of YouTube

David — Wed, 11 Oct 2006 08:53:47 +0000

YouTube provides online video quickly when you want it and those related video links are just addictive. My favourites are World of Warcraft clips and comedian videos like Bill Hicks. If you have not had your head stuck in the sand over the last few days you would have heard Google has purchased YouTube, or more precisely they have given the owners a tonne of Google stock ($1.65 billion worth) and not a single real dollar changed hands. Hopefully this purchase by Google provides a search speed increase, at the moment searching for videos is often very slow when compared to just browsing which is always very snappy.

Probably the most interesting reaction to this came from Jason Calacanis who was all for the deal which is quite unlike his tone on the Gillmor Gang of late. The most unsurprising reaction was from Mark Cuban who stuck to his guns by saying YouTube is a copyright lawsuit waiting to happen and bad news for any future owner.

stressfree - google

Integrating Google Site Search into SilverStripe

Loading XML data from an external source

Google O3D may finally bring 3D to the Web

Safari 4 on Windows looks good

StressFree Webmin theme version 2.0 released

Google Chrome rethinks the browser

So why should I care?

Process isolation comes to tabs

A lightweight, tab-centric interface

Chrome is fast... really fast

What does it mean for me the poor old web developer?

The beta for Windows is available now, OSX and Linux to follow

Final thoughts

Web Forms in Google Docs

Copying Documents

Templates

Smart logging of respondents

Embedding Web Forms in Text Documents

Off-line enabling existing applications with Gears

Off-line enabling a legacy Web application

Off-line write operations

Conclusion

Google Apps gets serious but where is the offline option?

"I want the people to know that they still have 2 out of 3 branches of the government working for them, and that ain't bad." Mars Attacks

Google Reader is now my primary RSS experience

Responsiveness

Aesthetics

Always synchronised

Web-centric functionality

Resource Usage

Conclusion

Forget about fetchmail thanks to Gmail

Google to replace Microsoft Exchange & Office at Pixar?

Chris DiBona Wellington meetup on Tuesday

Google Trends: A cool comparison tool

The emerging future: Software as electricity

Google surprises all with its purchase of YouTube

"I want the people to know that they still have 2 out of 3 branches of the government working for them, and that ain't bad." _{Mars Attacks}