stressfree - nas

Large file support with an Unslung NSLU2

David — Mon, 14 Aug 2006 04:03:57 +0000

I have written previously on how cool the little NSLU2 is as a customisable NAS device. I have set mine up as a little backup device, it silently backs up my server files (using rsync), creates tar files from all the files and then presents these archives to Retrospect on my Mac for backing up to external media and taking off-site. Unfortunately the default 'ls' and 'tar' programs that come with the Unslung distribution do not support large file sizes or long filenames to fix this problem download the far more up-to-date versions using ipkg:

# Whilst logged in as root (or use sudo)
ipkg install coreutils
ipkg install tar
mv /bin/ls /bin/ls.old
mv /bin/tar /bin/tar.old
ln -s /opt/bin/ls /bin/ls
ln -s /opt/bin/tar /bin/tar

Running these commands will replace the outdated programs with their newer counterparts which results in far smoother backups.

Linsys NSLU2, one cool little NAS

David — Thu, 11 May 2006 10:47:43 +0000

Weighing in at a couple of hundred dollars the Linksys NSLU2 is a very tidy little NAS device. It's a small (three CD cases stacked) unit that holds a 266mhz PPC processor (underclocked to 133mhz), 40meg of RAM, two USB ports and one network interface. Linksys have fashioned together a Linux-based OS running Samba to provide a very tidy, home/home-office level NAS device that can be easily administered via a clean web-based interface.

The Linksys NAS connected to a 2.5" 80gig drive (click to enlarge)

Where the little box gets really interesting is its ability to be hacked in almost any direction. There is a large community of Linux hackers producing custom Flash images that allow everything from the addition of extra software packages to the installation of a full-blown Debian system on the tiny box. The hackers have cleverly got around the flash memory limits of the onboard hardware through a method known as 'unslinging', or more precisely the ability to boot and run the device off a connected hard drive. Coupled with this there is a raft of hardware hacks that range from the relatively simple (removing the underclocking on the CPU) through to the really difficult (boosting RAM to 256meg by soldering together RAM chips).

As soon as I read about the device I new I had to try one out to see if it was powerful enough to satisfy some PhD testing needs I have. On Tuesday the device turned up and after verifying the hardware was working I proceeded to void the warranty by installing the Unslung flash image and removing the hardware underclocking resistor. The Unslung image is a variation on the default Linksys OS, the major difference from the default OS being that it is 'unslung' to the external drive and the ability to install new software packages is provided. The available package list is surprisingly complete and very up to date. It includes such things as PHP 5.1, Apache 2, MySQL 4.1, CUPS 1.1 and Samba 3.0.22. The package management tool is called ipkg and I've found it very easy to use and surprisingly quick. New software gets installed into the /opt/ directory which makes experimenting and distinguishing between the default packages a lot easier.

With all these software packages its easy to put together a really powerful little NAS device that does everything from database driven PHP webpages to complex Active Directory integrated file and print serving. The only limiting factor is the 266mhz CPU and 40meg of RAM (which rules out really tricky stuff involving Java, Mono or Ruby). But ignoring this the device has a tonne of potential for the Linux software or hardware hacker. As it is only a few hundred dollars voiding the warranty is not really an issue, plus if everything turns to custard it is really easy to just re-flash the device with a pre-rolled image and start from square one again. The Linksys NAS is well worth the money if you are a tinkerer, in the market for a low-end file/print/web server or have a particular hardware/software project in mind.

Software as a Service - myths destroyed

David — Mon, 01 May 2006 11:48:03 +0000

Recently I have been giving a lot of thought to the Software as a Service concept. The basic idea behind this catch phrase is that rather than paying a lot of money up front for a piece of software and/or server combination you subscribe to the software online and have it hosted remotely. There's obviously a couple of concerns over such a model, firstly you have got to trust that these companies do not go out of business plus you have to be confident that your Internet connection (and theirs) will be working whenever you need the service. A lot of these myths have been put to rest as this Business Week article illustrates, but for hard-core (old school) network administrators the idea of out-sourcing essential software sends cold shivers down their spines.

The concept really got stuck in my head when a friend of mine sent a general email asking for people to help him convince a small business to purchase a general purpose (mail/calendar/file/print) server. After having my share of bad server experiences and over demanding clients I eventually came to the conclusion that installing a new server in a small/medium business is a really difficult thing to justify when pitted against the new Software as a Service model. For example if you were to sign-up for the GMail hosted email service (where they host your email domain) you get email, calendar and instant messaging services all for just a few only slightly annoying adverts. Alternatively you could go the Microsoft Office Live path which does not provide as much storage space and costs more but could get very interesting as these services are integrated into the traditional Office suite. For data there are some pretty interesting looking NAS (Network Attached Service) devices available that provide local network storage without the hassle of running a full blown server. Then as long as you purchase decent network-enabled printers you have a complete small/medium office solution for a lot less setup and ongoing support costs than a conventional Linux/Windows/Netware server.

From a network administrator's perspective this model is at first scary (on one likes being out of a job) until you realise that you can now spend your time working with users to get these services working for them rather than just simply working, which is their current (and only) ongoing concern.

S3 provides unlimited cheap online storage

David — Mon, 01 May 2006 11:15:45 +0000

It has been a while since I last posted, mainly because I have been very busy working on Reasonate, going to Japan and doing end of year taxes...

Anyway one thing that really blew my mind the other day was Amazon's new S3 service. On the surface it seems really simple, a basic web service that provides cheap online storage (US$0.15 per Gigabyte for storage per month). What is exciting is the ramifications, if somebody (like you or I) want to store a lot of information for ourselves or others online there is no need to invest in big servers and fat Internet connections to serve that data. It also means that by design your web applications will scale effortlessly at least in the sense of the data storage mechanism assuming Amazon's server farm is up to the task. I have signed up for an account and read through the documentation and some of the features are pretty nice (access control lists, time sensitive url's and a lot more). There has been very good things said about it on TechCrunch and other places, plus some criticisim for not supporting the very simple XML-RPC protocol (which would have been very nice to have in simple applications).

What is needed now from someone like Google/MySQL is a distributed database platform that translates a similar business model across to the relational database market, say US$0.15 per thousand queries or database service in exchange for adverts on the site. It would be great to be able to harness to a massively scalable disk and database array with guaranteed uptime and almost unlimited scalability without having to spend thousands (or even millions) of dollars on server hardware and support. This type of setup would be really exciting and would benefit almost everyone involved (except for out of work server support staff). Whether or not something like this happens is uncertain but it certainly is something that Adam Bosworth from Google has obliquely hinted at in various speeches and blog posts.