stressfree - linux

Tuning Ubuntu's software RAID

David — Fri, 23 Apr 2010 03:04:31 +0000

Recently I encountered an issue where the read/write performance of Ubuntu's software RAID configuration was relatively poor. Fortunately, others have encountered this problem and have documented a potential cause and solution here:

The short story is that Ubuntu uses some very conservative defaults for RAID caching. Whilst this may ensure reliable behavior across a range of hardware, it does mean that for many read/write performance will be lacklustre. The solution to this problem is to define a more aggressive caching options on any software RAID partitions that are in use.

Setting the stripe_cache_size and read ahead caches

The following example assumes that the Ubuntu server has two software-based RAID-5 partitions, /dev/md0 (the root partition) and /dev/md1 (the /var partition).

Set the stripe_cache_size and read ahead caches in the /etc/rc.local script. In the example below the stripe_cache_size is set to 8192, and the read ahead cache 4096:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution bits.
#
# By default this script does nothing.

# Tune the RAID5 configuration
echo 8192 > /sys/block/md0/md/stripe_cache_size
echo 8192 > /sys/block/md1/md/stripe_cache_size

blockdev --setra 4096 /dev/md0
blockdev --setra 4096 /dev/md1

exit 0

Restart Ubuntu to apply these settings.
Note: It is possible to apply these changes without a restart by executing each directive at the command line.

The pages linked to above explain how to test the influence of these cache changes. In general I have found that the parameters given in the example above have improved performance without influencing the reliability of the system, or the data stored on it.

USB devices with VMWare Server 2.0 on Ubuntu

David — Mon, 17 Aug 2009 05:17:18 +0000

One of the nice features of VMWare Server 2.0 is that it supports the forwarding of USB devices to virtual machines. Unfortunately when it comes to Linux the VMWare team have leveraged an old method (/proc/bus/usb) for scanning the USB bus which newer distributions, such as Ubuntu Server 8.04 no longer support.

To resolve this problem the "old" method for scanning for USB devices must be enabled in the underlying operating system. In the case of Ubuntu Server 8.04 this is a case of editing the file /etc/init.d/mountdevsubfs.sh and uncommenting the following section:

#
# Magic to make /proc/bus/usb work
#
mkdir -p /dev/bus/usb/.usbfs
domount usbfs "" /dev/bus/usb/.usbfs -obusmode=0700,devmode=0600,listmode=0644
ln -s .usbfs/devices /dev/bus/usb/devices
mount --rbind /dev/bus/usb /proc/bus/usb

Reboot the server and /proc/bus/usb should be functional once more.

Activating a USB device within a virtual machine

Once the underlying USB subsystem is configured the USB device needs to be associated with a virtual machine. For this to occur the virtual machine must have the USB Controller added to its virtual hardware configuration. If the controller is not already part of the virtual machine's configuration shutdown the VM, add the device and restart.

The VMWare web console with USB device selection (click to enlarge)

Assuming there are USB devices attached to the server, once the virtual machine boots a small USB icon will appear within the VMWare web management console. Click on the icon and select the relevant USB device to attach it to the running virtual machine.

All going well the USB device will appear within the virtual machine as an accessible device. VMWare Server remembers this selection, so the next time the virtual machine (or server itself) is restarted the USB device will automatically be attached to the running VM.

VMWare Server 2.0 optimisations

David — Sun, 19 Jul 2009 08:42:08 +0000

VMWare Server 2.0 is emerging as a capable, zero cost alternative to VMWare ESX when used in combination with Ubuntu Server 8.04LTS. Unfortunately "out of the box" performance can be a little disappointing, especially when running guest Windows virtual machines. What follows are a few system tweaks that can improve performance without hampering overall system stability. I have not come up with these myself, instead they are pruned from the following pages:

Kernel parameters

In addition to the default Ubuntu Server kernel parameters, the following should be appended to the end of /etc/sysctl.conf.

vm.swappiness=0
vm.overcommit_memory=1
vm.dirty_background_ratio=5
vm.dirty_ratio=10
vm.dirty_expire_centisecs=1000
dev.rtc.max-user-freq=1024

Once added reboot the server to ensure their application is successful and permanent.

Create an in-memory temp drive

In the host's /tmp directory create a new directory named vmware (e.g. /tmp/vmware). This will be used as the mount point for a tmpfs (in-memory) partition for storing VM related, temporary files.

Edit /etc/fstab and add the /tmp/vmware partition to your list of mount points:

tmpfs /tmp/vmware tmpfs defaults,size=100% 0 0

Now if you execute the following command the tmpfs filesystem will be mounted at /tmp/vmware:

sudo mount /tmp/vmware

If successful, reboot the Ubuntu server to ensure the tmpfs partition is mounted at boot time.

VMWare Server configuration

Edit the /etc/vmware/config file and ensure the following configuration declarations are set:

prefvmx.minVmMemPct = "100"
prefvmx.useRecommendedLockedMemSize = "TRUE"
mainMem.partialLazySave = "TRUE"
mainMem.partialLazyRestore = "TRUE"
tmpDirectory = "/tmp/vmware"
mainMem.useNamedFile = "FALSE"
sched.mem.pshare.enable = "FALSE"
MemTrimRate = "0"
MemAllowAutoScaleDown = "FALSE"

These configuration declarations instruct VMWare Server to keep all virtual machines in memory and not to write unused blocks to disk. It also sets the temporary directory to the newly created tmpfs partition at /tmp/vmware.
Restart the VMWare Server process (sudo /etc/init.d/vmware restart) or reboot the server for these changes to take effect. The net result should be notably smoother virtual machine performance, especially when it comes to Windows guests.

Virtual machine tips

Always use fully allocated disk images.
Do not use snapshots as they are approximately 20% slower.
Always install the VMWare Tools package.
If running Linux make sure the kernel is compiled for running within a VM, or is using the correct boot time parameters.

Fixing yum's "Metadata file does not match checksum" error

David — Mon, 13 Apr 2009 10:21:30 +0000

Centos is a "free" distribution of Red Hat Enterprise Linux which I enjoy using. Whilst it does not have Debian's apt-get for package management it does have yum, which is not as fast but still works pretty well in most circumstances. Unfortunately today I ran into a problem in a clean install of Centos 5.3 where yum was returning the following error:

Error Message: Metadata file does not match checksum

A quick look around "the Google" turned up the relatively simple solution; at the terminal execute as root:

yum clean all
yum makecache
yum update

This process will take a little time, but the end result should be a nice, clean yum repository cache, complete with no annoying meta-data errors.

Transparent Squid Authentication to eDirectory

David — Tue, 07 Oct 2008 10:36:45 +0000

This post explains how to setup a Squid HTTP proxy to transparently authenticate users against a Novell eDirectory. In the Novell eco-system Border Manager is the venerable choice for an internal firewall and proxy but it is showing its age. This guide is based on this Novell Cool Solution. Unlike Border Manager, which requires the CLNTRUST client-side tool, the setup described works without the need for any desktop client software.

How it works

Within a Novell managed network the eDirectory stores authenticated user's I.P. addresses. Squid performs an LDAP search against eDirectory using the incoming I.P. address of the client. If successful the authenticated username is returned and a proxy session established. If the search comes up empty Squid prompts the client to manually enter their credentials for authentication against the eDirectory. If this too fails the proxy request is denied.

eDirectory 8.8 incompatability

This solution currently only works with eDirectory < 8.8 because Novell has slightly changed the format they store network addresses in newer versions. At the time of writing I have not been able to test against eDirectory 8.8 so I cannot determine the required code changes or test results. Hopefully in the near future this situation will change.

Squid's external_acl_type option

Transparent authentication is made possible thanks to Squid's external_acl_type configuration option. This allows external identities and groups to be identified via any external script. Once Squid is installed setting up transparent eDirectory authentication is a two step process:

Create and tweak the squid_edir_iplookup.pl file.
Edit the squid.conf configuration file

The engine room: squid_edir_iplookup.pl

To begin create a file named squid_edir_iplookup.pl. This file can be anywhere on your system as long as the Squid process can access it. In this example I have created the file in /usr/lib/squid/ as this is where Red Hat stores all of Squid's authentication related scripts.

/usr/lib/squid/squid_edir_iplookup.pl

#!/usr/bin/perl
use Net::LDAP;
use Net::LDAP::LDIF;
use File::Path qw(rmtree);
use File::Basename qw(basename);

$HOST = 'your.edirectory.server';
$PORT = 389;
$ADMIN = "cn=squid,ou=tech,o=company";
$PASSWD = "squidpassword";
$BASEDN = "o=company";
@SITES = qw(ou=groups);

$|=1;

START: while (<>) {

($IP,$GROUP) = split(/ /,$_);
# $SITE =~ tr/\n//d;
$GROUP =~ tr/\n//d;
$group_filter_string="";
for $site (@SITES) {
$group_filter_string=$group_filter_string."(groupMembership=cn=$GROUP,${site},$BASEDN)";
}

$netaddress = "1\#";
@octets = split(/\./,$IP);
foreach $octet (@octets) {
# The IP address is stored in eDirectory as four unsigned chars. ASCII 40, 41, 42 and
# 92 are characters ( ) *\ which are known tokens in LDAP search filters If you dont
# escape these with a backslash they will cause LDAP errors and he script will fail.
if ((($octet >= 40) && ($octet <= 42)) || ($octet == 92)) {
$netaddress = $netaddress.sprintf("\\%c",$octet)
} else {
$netaddress= $netaddress.sprintf("%c",$octet);
}
}
$filter="(&(objectclass=user)(|$group_filter_string)(networkAddress=$netaddress))";
$attnames=['CN'];

#connect to the server
until($ldap = Net::LDAP->new($HOST, port => $PORT)) {
die "Can not connect to ldap://$HOST:$PORT/" if ++$count > 10;
sleep 1;
}

$r = $ldap->start_tls();

$r = $ldap->bind($ADMIN, password => $PASSWD, version=>2);
die $r->error if $r->code;

$r = $ldap->search(base => $BASEDN,
scope => 'sub',
filter => $filter,
attrs => $attnames);

$count = $r->count;
if ($count == 0) {
print "ERR\n";
} else {
foreach my $entry ($r->entries){
my @values = $entry->get_value(CN);
foreach $value (@values) {
# Many users in eDirectory have multiple CN values - usually from the user template
# used to create them - sometimes their maiden name is noted in the Other Name
# attribute in ConsoleOne we want to report the proper CN to squid not these bogus
# values.
if ($value =~ m/template|previously/i) {
next;
} else {
$value =~ tr/- //d;
print "OK user=$value\n";
next START;
}
}
}
}
$ldap->unbind;
}

At the beginning of this file you want to change the $HOST, $PORT, $ADMIN, $PASSWD and $BASEDN parameters to ones that are relevant for your internal network. The @SITES array is not important as this is used in the Cool Solution example to define different groups of users (something we are not concerned about here).

Once you have saved the file make it executable:

chmod a+x /usr/lib/squid/squid_edir_iplookup.pl

The trickiest step is making sure you have the correct Perl libraries installed on your system for the script to run. These libraries are defined at the top of the file (the USE statements). The easiest way of testing whether the script is working is to run it:

/usr/lib/squid/squid_edir_iplookup.pl

The Perl interpreter will soon tell you if something is not right. If a required library is missing you will need to install it using your system's package management tool or directly from source (CPAN is useful).

All going well when the script is run you will be presented with a new, blank line. Test the LDAP lookup by typing an I.P. address and pressing enter. After a brief delay a success/fail message will be returned:

/usr/lib/squid/squid_edir_iplookup.pl
192.168.1.10
ERR

In the above example no valid user was found in the eDirectory at this I.P. address. If the look-up was successful the relevant username will be returned. To quit the script press CTRL+C.

Editing squid.conf

With the squid_edir_iplookup.pl script in place and working it is now time to edit the Squid configuration. The exact location of this file will vary depending on your operating system, but in the case of Red Hat this is:

/etc/squid/squid.conf

http_port 3128
http_port 8080
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
access_log /var/log/squid/access.log squid

external_acl_type IPUser ttl=3600 %SRC /usr/lib/squid/squid_edir_iplookup.pl

auth_param basic program /usr/lib/squid/squid_ldap_auth -b "ou=users,o=company" -u cn your.edirectory.server
auth_param basic children 5
auth_param basic realm Squid Web Proxy
auth_param basic credentialsttl 1 hours

# Used to pull LDAP group membership based on a supplied username - not currently used
# external_acl_type ldap_group %LOGIN /usr/lib/squid/squid_ldap_group -D cn=squid,ou=tech,o=company -w squidpassword -b o=company -s sub -f "(&(objectclass=inetOrgPerson)(cn=%u)(groupMembership=%g))" -h your.edirectory.server

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl localnet src 192.168.1.0/255.255.255.0
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

acl edirectory_users external IPUser Everyone
acl authenticated_users proxy_auth REQUIRED localnet

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost

http_access allow edirectory_users
http_access allow authenticated_users
http_access allow localhost
# http_access allow localnet
http_access deny all

http_reply_access allow all

icp_access allow all

coredump_dir /var/spool/squid

debug_options ALL,1
# For debugging ACLs
# debug_options ALL,1 33,2 28,9

You will need to change the relevant parts in this file to suit your eDirectory and internal network settings (i.e. the localnet setting).

The Squid configuration file is a tricky beast, but what is listed above is fairly standard. The first highlighted section is where the external squid_edir_iplookup.pl script is referenced. If this check fails an LDAP-backed HTTP BASIC authentication request is made.

The second highlighted area identifies two Access Control Lists, the first based on the result of the eDirectory I.P. lookup and the second on the HTTP BASIC authentication response. If the incoming request is from someone authenticated within the eDirectory OR by the HTTP BASIC process then outbound access is granted.

It is possible to use LDAP group information to form ACLs that further limit Web access. The above example does not utilise this functionality, but the Novell Cool Solution goes into it in further detail. My advice is get the Squid/eDirectory authentication working at a very basic level and then make it more complicated. Starting at the most complicated scenario will only lead to failure and frustration.

If you are encountering authentication issues then enable the debugging options at the bottom of the configuration file. This is fairly verbose but can be very useful in identifying configuration mistakes.

Reviewing HTTP access logs with Webmin & SARG

SARG (Squid Analysis Report Generator) is a handy tool for reviewing Squid access files. There is a very good Webmin module for this application, so rather than struggling with the command line install Webmin on the proxy. By the way, while you are at it install my Webmin theme, your eyes will thank you for it.

The SARG module within Webmin (click to enlarge)

Building a webcam with an old laptop, Canon camera and Linux

David — Fri, 22 Aug 2008 09:51:03 +0000

Recently I put together a webcam for the Aorangi Ski Club's lodge on Ruapehu. The system consists of a second-hand laptop, an old Canon camera and Ubuntu Linux. Images are automatically captured every hour, timestamped and posted to Flickr. The end result is a pretty impressive looking Flickr slideshow:

To put a similar setup together follow these steps.

Step 1: Get the gear

Find a Linux compatible laptop and a Canon camera compatible with the Capture software.

Step 2: Install Linux

I installed Ubuntu 8.04 on the laptop because from my experience it has the best 'out of the box' support for laptop hardware. It didn't prove me wrong when installing onto the four year old Toshiba, it even detected the winmodem.

Step 3: Internet connectivity

The problem of getting Internet connectivity will vary depending on where your webcam is located. In my case it was in a relatively isolated position half-way up one of New Zealand's largest mountains. That meant the only Internet connection available was via the phone line.

Unfortunately in this day and age winmodems are not supported that well by the popular Linux distributions. Your best bet is to get a hardware modem rather than relying on a cheaper 'winmodem' alternative. For me however budget was a driving factor so the winmodem had to stay. Fortunately the Conexant winmodem in the laptop did have a binary kernel driver available.

Setting up a modem in Linux will vary depending on your distribution, but if all else fails read the PPP howto and edit the configuration files directly.

Step 4: Install the software requirements

At this point it is a good idea to plug your laptop into your LAN and perform a full software update (i.e. apt-get update or yum update). Once that is complete install Imagemagick, Capture and the Perl library flickr_upload.

Imagemagick is very popular and should be just a case of running apt-get install imagemagick (or yum install imagemagick). Unfortunately it is highly unlikely you will find prebuilt .deb or .rpm packages for Capture or flickr_upload. In both cases it is best to download the source code and follow the compile/installation instructions. Both these packages have dependencies which you will need to track down and install, but trust me, both do compile and install on Ubuntu 8.04.

Step 5: Test your camera

At this point you should have all the prerequisits in place to start writing your webcam script.

Note: For simplicity these scripts will run as root. This gets around all the permissions issues when accessing modems and USB devices. Also the fact that this will be a standalone device doing non-mission critical work means it isn't a huge problem.

Create a directory somewhere (say /usr/local/webcam) and create a file called capture.sh.

nano capture.sh

Give it the following contents:

#!/bin/bash
# Get the new image from the camera
capture 'start'
capture 'flash off'
capture 'zoom 1'
capture 'metering spot'
capture 'focuspoint center'
capture 'capture raw.jpg'
capture 'detach'

Set the execute properties on this script.

chmod a+x

Plug the camera in to the laptop via a USB cable, turn it on and execute this script.

./capture.sh

All going well the camera should come alive, focus itself, take a photo and download it to the current directory (i.e. /usr/local/webcam) with a filename of raw.jpg.

If things do not work Google the error message, check the cable connections and try again. I tried the Capture application with a couple of supported Canon cameras and they all worked fine.

Step 6: Setup Flickr API access

Assuming your camera is happily taking photos when directed by your laptop you can go ahead and setup Flickr.

Sign up for a Flickr account (they are free). This will be where your webcam photos are sent to. Next sign up for a Flickr API key here: http://www.flickr.com/services/api/keys/apply/

Record the Flickr API key and secret that you are provided at the end of this process.

In the /usr/local/webcam directory create a file named flickr-auth.pl.

nano flickr-auth.pl

Give the file the following contents (replace YOUR_FLICKR_KEY/SECRET with your actual key and secret):

#!/usr/bin/perl

use Flickr::API;
use Flickr::Upload;

my $flickr_key = 'YOUR_FLICKR_KEY';
my $flickr_secret = 'YOUR_FLICKR_SECRET';

my $ua = Flickr::Upload->new(
{
'key' => $flickr_key,
'secret' => $flickr_secret
});
$ua->agent( "perl upload" );

my $frob = getFrob( $ua );
print "FROB:$frob;\n";

my $url = $ua->request_auth_url('write', $frob);
print "1. Enter the following URL into your browser\n\n",
"$url\n\n",
"2. Follow the instructions on the web page\n",
"3. Hit when finished.\n\n";

<>;

my $auth_token = getToken( $ua, $frob );
die "Failed to get authentication token!" unless defined $auth_token;

print "Token is $auth_token\n";

sub getFrob {
my $ua = shift;

my $res = $ua->execute_method("flickr.auth.getFrob");
return undef unless defined $res and $res->{success};

return $res->{tree}->{children}->[1]->{children}->[0]->{content};
}

sub getToken {
my $ua = shift;
my $frob = shift;

my $res = $ua->execute_method("flickr.auth.getToken",
{ 'frob' => $frob ,
'perms' => 'write'} );
return undef unless defined $res and $res->{success};

return $res->{tree}->{children}->[1]->{children}->[1]->{children}->[0]->{content};
}

Set the execute permissions for the file and run the script.

chmod a+x flickr-auth.pl
./flickr-auth.pl

Follow the onscreen instructions. You will be directed to open a browser window to the location specified, and accept the request. Once accepted press enter on the laptop and the script with return your Flickr auth token. Write this token down somewhere as it will be needed in a moment.

Step 7: Setup the Flickr upload script

Create a file named flickr-upload.pl and give it the following contents (again replace YOUR_FLICKR_KEY/SECRET/TOKEN with your actual key, secret and token):

#!/usr/bin/perl
use strict;

use Flickr::Upload;

my $flickr_key = "YOUR_FLICKR_KEY";
my $flickr_secret = "YOUR_FLICKR_SECRET";
my $auth_token = "YOUR_FLICKR_TOKEN";

my $photograph = "";
foreach my $arg (@ARGV) {
$photograph = $arg;
}

my $ua = Flickr::Upload->new(
{
'key' => $flickr_key,
'secret' => $flickr_secret
});

$ua.agent => ( "perl upload" );

$ua->upload(
'photo' => $photograph,
'auth_token' => $auth_token,
'tags' => 'webcam',
'is_public' => 1,
'is_friend' => 1,
'is_family' => 1
) or die "Failed to upload image";

You can also edit the last couple of lines to define the tags applied to the uploaded photo and its privacy settings.

Step 8: The webcam script

Create a file named process.sh and give it the following contents:

#!/bin/bash
TEMP="/usr/local/webcam"
DATESTRING=`date +%l:%M%P\ on\ %A,\ %d\ %B\ %Y`
RAWFILE="raw.jpg"
RESIZEDFILE="resized.jpg"
STAMPEDFILE="stamped.jpg"
UPLOADFILE="Webcam $DATESTRING.jpg"

logger "Webcam process initiated"

# Change to the temp directory
cd $TEMP

# Get the new image from the camera
echo "Taking photograph..."
./capture.sh

# Connect to the Internet
pon

# Convert and stamp the photograph
echo "...converting image for the web"
convert -resize 1024x768 $RAWFILE $RESIZEDFILE
convert -fill "#333333" -draw "rectangle 0,722 1024,768" $RESIZEDFILE $STAMPEDFILE
convert -fill "#eeeeee" -draw "rectangle 0,723 1024,768" $STAMPEDFILE $STAMPEDFILE
convert -font "Helvetica" -pointsize 28 -fill "#111111" -draw "text 25, 755 '$DATESTRING'" $STAMPEDFILE $STAMPEDFILE

# Wait for connection
echo "Connecting to the Internet..."
sleep 45

# Update the system time
ntpdate nz.pool.ntp.org

# Upload the photograph
echo "Uploading photograph"
# echo $UPLOADFILE
cp $STAMPEDFILE "$UPLOADFILE"
./flickr-upload.pl "$UPLOADFILE"

# Disconnect
echo "Disconnecting..."
sleep 5
poff
echo "...complete"

# Clean up - delete working files (if they exist)
rm *.jpg

logger "Webcam process complete"

This script ties everything together:

it executes the capture.sh script and takes a photo
connects the modem to the Internet
syncs the system time with a nearby NTP server
resizes and stamps the photograph with the current date and time
uploads the resized image to your Flickr account
disconnects from the Internet
removes the temporary image files

Note: You will probably want to edit parts of this script. For example you could use different image names or temp directories or a time server pool for your country.

Set the execute permissions of the file, set the camera up and run the script.

chmod a+x process.sh
./process.sh

All going well your laptop and camera should run through the routine outlined above. If you experience any errors it is just a case of debugging and trying again.

Step 9: Setup the cron job

Once the script is working as expected setup a cron job to run on a regular basis. How often is semi-dependent on your daylight hours and available Flickr bandwidth. A free Flickr account has limited upload bandwidth per month so you want to be careful your webcam does not take too many photographs. In the case of the Aorangi webcam it takes a photo once per hour between 7:30am and 4:30pm.

Create a file in /etc/cron.d named webcam and give it the following contents:

30 7-15 * * * root /usr/local/webcam/process.sh

Restart cron and you should find everything runs like clockwork from that point forward.

linux
ubuntu

Mounting CIFS shares at login with SELinux enabled

David — Sat, 17 May 2008 02:36:57 +0000

SELinux is as painful to use sometimes as it is powerful when it comes to locking down server permissions. Unfortunately even with distributions such as Red Hat which supports SELinux out of the box, you will still experience problems.

One such issue I came across recently was automounting CIFS shares on boot using netfs. At startup the netfs service was returning an "error 13 - error opening credentials file" when attempting to mount the CIFS shares. The problem was the SELinux was not allowing the netfs script to access the file that contained the CIFS authorisation details.

For example, my /etc/fstab had the following entry:

//WINDOWSSERVER/SHARE /mnt/windowsshare cifs credentials=/etc/samba/auth.cifs 0 0

And in the /etc/samba/auth.cifs file were the following details:

username=windowsuser
password=windowspassword

The solution to the problem was to change a SELinux boolean parameter with the following command (found here):

setsebool -P allow_mount_anyfile 1

This lets the mount command open any referenced file, effectively side-stepping the netfs error. Sure it is not 100% secure but it works without having to completely disable SELinux which seems to be most people's answer to any problems.

Installing VMWare Server 1.0 on Ubuntu 6.06LTS

David — Wed, 23 Jan 2008 09:04:58 +0000

Ubuntu 6.06LTS is a useful platform for VMWare because it has a small footprint by todays standards and is supported by Canonical until 2011. Unfortunately installing VMWare Server 1.0 can be a little painful given there are no binary kernel modules for Ubuntu in this release.

Fortunately there are some excellent guides for installing VMWare on this platform such as this one from HowtoForge. What follows is an installation script based on the HowtoForge guide that saves the administrator a lot of time and solves a bug along the way.

Build your Ubuntu server

A basic install of 6.06LTS is pretty light weight and by default doesn't have anything running that isn't needed. A few things that you may want to install is the openssh-server package for remote access and ntp-server for keeping your system clock accurate.

sudo apt-get install openssh-server ntp-server

By default Ubuntu 6.06 configures its network via DHCP which is fine for a desktop system but probably not ideal for a server. To set a static IP address edit the /etc/network/interfaces file and set the relevant static network interface and address parameters. For example:

iface eth0 inet static
address 192.168.1.10
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

Also check the /etc/resolv.conf file contains the correct DNS server details before restarting the network interfaces with:

sudo /etc/init.d/network restart

The VMWare Installation Script

1. Create a vmware directory somewhere handy to store the installation files, your home directory is a good place. Once the installation is complete you can remove this directory although it is worth making a backup should you wish to install VMWare again.

2.Download the Linux tar archives of VMWare Server 1.0 and the Management User Interface (MUI) from the VMWare website. The current stable release of VMWare Server at the time of writing was version 1.0.4-56528. Also as you are downloading remember to record your serial code as this will be needed during the installation process.

3. In the same directory where you have downloaded these install archives create a new file named rundir.httpd.vmware with the following contents:

#! /bin/sh
# /var/run gets purged at every reboot!

RUNDIR="/var/run/vmware/httpd"
OWNER="www-data"
GROUP="www-data"

/usr/bin/test -d "$RUNDIR" || \
/bin/mkdir -p "$RUNDIR" && /bin/chown "$OWNER:$GROUP" "$RUNDIR"

This file is copied to the /etc/init.d/ directory by the installation script and overcomes an permissions issue which stops the VMWare MUI from loading.

4. Now create a file called install-vmare.sh with the following contents:

#! /bin/sh
TMP="/tmp"
VERSION="1.0.4-56528"

echo "Installing VMware Server for Ubuntu 6.06"

echo "Installing application requirements"
echo "---------------------------------------------------"
echo "Installing dev headers and xinetd..."
sudo apt-get -y install libx11-6 libx11-dev libxtst6 xlibs-dev xinetd wget
echo "---------------------------------------------------"

echo "Installing Linux Kernel headers..."
sudo apt-get -y install linux-headers-`uname -r` build-essential
echo "---------------------------------------------------"

echo "Installing build utilities..."
sudo apt-get -y install gcc binutils-doc cpp-doc make manpages-dev autoconf \
automake1.9 libtool flex bison gdb gcc-doc gcc-4.0-doc libc6-dev-amd64 lib64gcc1

# Install VMWare Server
echo "---------------------------------------------------"
echo "Uncompressing the VMware Server $VERSION archive..."

# Copy files to revelant locations
cp VMware-server-$VERSION.tar.gz $TMP
cp VMware-mui-$VERSION.tar.gz $TMP
sudo cp rundir.httpd.vmware /etc/init.d/

cd $TMP
tar -xzf VMware-server-$VERSION.tar.gz
rm VMware-server-$VERSION.tar.gz

echo "---------------------------------------------------"
echo "Installing VMware Server $VERSION..."
cd vmware-server-distrib
sudo ./vmware-install.pl

# Install the VMWare web management console
echo "---------------------------------------------------"

echo "Uncompressing the VMware MUI $VERSION archive..."

cd $TMP
tar -xzf VMware-mui-$VERSION.tar.gz
rm VMware-mui-$VERSION.tar.gz

echo "---------------------------------------------------"
echo "Installing VMware MUI $VERSION..."

# Configure the MUI rundir fix
sudo chmod 755 /etc/init.d/rundir.httpd.vmware
sudo ln -s /etc/init.d/rundir.httpd.vmware /etc/rc2.d/S90rundir.httpd.vmware

cd vmware-mui-distrib
sudo ./vmware-install.pl

echo "---------------------------------------------------"
echo "Cleaning up..."
cd $TMP
rm -rf vmware-server-distrib
rm -rf vmware-mui-distrib
echo "---------------------------------------------------"
echo "VMWare installation complete"

echo ""
echo "To administer VMWare you will need to either:"
echo "1. Enable root logon access (sudo passwd root)"
echo "2. Set set the uid of a designated user to 0 (sudo nano /etc/passwd)"
echo ""
echo ""

This is the script that handles installation of all the required dependencies, kernel sources and VMWare packages.

NOTE: You may need to tweak the TMP and VERSION variables to suit your server environment and downloaded VMWare version.

5. Finally enable execute permissions on this file and run the script:

chmod a+x install-vmware.sh./install-vmware.sh

The script will prompt you for your password in order to gain administrator privileges (sudo). It will then install all the required packages via apt before copying the rundir.http.vmware file into the correct location and running the VMWare installers.

For the most part the default VMWare install options can be accepted without much thought. The only settings that may need tweaking is the location of the VMWare images (default /var/lib/vmware/Virtual Machines) and the MUI's default login time of 60 minutes which seems a little long for my liking (15 minutes would seem more appropriate).

All going well at the end of the process you should have a fully functional VMWare Server with a web management console operating at https://server-ip-address:8333/. Whilst you cannot create or install virtual machines via the web console you can download the Windows or Linux VMWare Console installers which will get you going.

Kernel Updates

Be aware that each time you upgrade the Ubuntu kernel to a different version the VMWare kernel module must be recompiled. After upgrading your kernel and rebooting the system you will find VMWare fails to load. At this point do not panic, it only takes a few steps to rectify the situation.

1. First download the headers for your current kernel.

sudo apt-get install linux-headers-`uname -r`

2. Now run the vmware-config.pl script, accept the default settings and rebuild the VMWare kernel module.

sudo vmware-config.pl

Once this script completes VMWare should once again be operational. The only thing left to do is remove the old kernel headers if you feel you won't be needing them again:

sudo apt-get remove linux-headers-(old kernel version)

Google outflanks Sun with Android

David — Tue, 13 Nov 2007 23:44:44 +0000

Google recently released the highly anticipated Android mobile phone platform to developers. Android promises to be a more consistent and powerful environment for mobile applications compared to what currently exists in the fragmented mobile market. Whilst many people were disappointed that Android was not a Google-branded iphone; from a developers perspective if it can gain broad adoption it will make the developing powerful, Internet-centric mobile applications significantly easier.

One of the most interesting aspects of Android is that it is released under the Apache v2 software license. This license grants obligation-free use of the code to any party. This is different to other popular open-source licenses like the GPL which requires source-code modifications to be made publically available. In the competitive mobile phone market such an obligation is problematic which is why Sun releases the Java Mobile Edition (ME) under different open and closed source licenses.

A question that was hanging around Android was how Google had managed to release a Java mobile platform under the Apache license given that the licenses Sun release JavaME under are not compatible. Stefano Mazzocchi points out on his blog that Google have outflanked Sun by releasing a platform that supports the Java language but does not use Sun's Java compiler or the Java byte-code at its core. Instead Google have created Dalvik, a virtual machine released under the Apache license which understands how to compile Java source code into its own byte-code for execution.

This move outflanks Sun's licensing policies, essentially cutting them out of the Android equation. It is a gutsy move by Google but it does free them to focus on developing a platform they have complete control over rather than working in partnership with Sun. From the perspective of Java as a language this move wouldn't seem to pose any problems as the mobile and desktop/server worlds have always been quite distinct. Plus if anything Google's use of the Apache Harmony JavaSE libraries may actually make developing for the mobile and desktop more consistent than Sun's distinct JavaME and JavaSE implementations.

All things considered this news has made Android more interesting from my perspective. Before I heard this it was just another JavaME implementation but now it sounds like Google will have the capability to do some really interesting things. What has yet to be seen is what level of support (if any) this platform will have on the iPhone. Google and Apple have a strong relationship there and it would seem like Dalvik runtime would be a natural fit on the device if it is lightweight, fast and provides developers with the ability to write applications for both Android and the iPhone.

Apple purchases CUPS to ward off GPL3 requirements?

David — Fri, 13 Jul 2007 02:57:24 +0000

Today the lead developer of the CUPS (Common Unix Printing System) project announced that in February 2007 Apple purchased the CUPS source code and took him on as a staff member. CUPS is significant within the *NIX world because it is arguably the most well supported and feature-rich printing system available. Apple has used it within OSX from the outset and personally I feel it is the best implementation of CUPS available thanks to the Aqua interface and the fact that the majority of printers just work without any effort.

There would seem to be no immediate danger of CUPS code being completely closed sourced considering it is currently released under the GPL2 license. What the code purchase suggests is that Apple probably plans on internally re-licensing the code under a closed source license for many, if not all of the platforms that could make use of it, i.e. Mac, AirPort and the iPhone.

Dual licensing such as this is not uncommon by vendors who control the rights to GPL code. It is a good way of benefiting from an open source development model and community whilst still being able to ship a modified version of the code on closed devices or only binary form. License flexibility such as this will become increasingly important as the GPL3 license is adopted as it resolves many of the loopholes vendors used to ship GPL code in what effect were closed devices (Tivoization).

Now that Apple does have a formal stake in CUPS my biggest hope is that they can spare a graphic designer to give the website and the CUPS interface with a much needed aesthetic overhaul and maybe a better logo...

Eben Moglen on the threat of patent agreements

admin — Tue, 15 May 2007 22:37:34 +0000

Eben Moglen of the Software Freedom Law Centre is a great public speaker and he demonstrates this skill exceptionally well in his ability to answer what the risk to the Free Software community is when deals such as last years Novell - Microsoft agreement take place.

If you like his response to this quesiton then I believe you will find his interview on FLOSS Weekly about Free Software and the GPL not only good to listen to but educational at the same time.

Mount Microsoft makes more patent rumblings

admin — Mon, 14 May 2007 11:05:29 +0000

Microsoft's General Counsel Brad Smith placed the number of patent violations by Free & Open Source Software (FOSS) at 235 in an interview with Fortune magazine this week. Considering these suposed violoations cover everything from the kernel through to office applications I am somewhat surprised that the theoretical total is not more considering the huge number of patents Microsoft has recently aquired. Tim Bray of Sun's response in his blog was short but to the point, 'litigate or shut up'.

Unfortunately the chances of Microsoft actually litigating are slim to none considering the damage it would do to their image and the potential problems it would cause from counter-suits from the likes of IBM. Instead the threat of legal action looks like it will continue to remain just that in the vain belief that by simply placing a cloud over Free software's head it will deter existing Microsoft customers from jumping ship. Whether or not such a strategy will be successful only time will tell, but forcing potential customers to buy your product through fear of what would happen if they did not does not seem like the kind of image one would want to portray.

A funny little video from Novell

David — Tue, 20 Mar 2007 10:58:16 +0000

With Novell's Brainshare well underway they have released a very unique 'will it blend?' video onto YouTube. Ted Haeger's been talking about this for a while now and I can see why, it does a very good job of simulatanoeously being funny whilst remaining stuffy enough to be instantly identified as a Novell video. Sure it is no Apple advert but it is good to see even if you are just interested what would happen if you put a Windows CD, a Mighty Mouse and a helping of Red Bull into an industrial strength blender...

linux
novell

'The Code Linux' documentary

David — Sat, 17 Mar 2007 03:54:04 +0000

If you are new to the ideas of open source and Linux then this documentary is well worth watching:

The Code Linux - (available on Google Video)

Even if you are fairly versed in the subject matter it is good to see some prominent open source personalities such as Linus Torvalds, Jon "Maddog" Hall and Eric S Raymond. Well worth watching online or downloading for a rainy Sunday afternoon.

A great example of Windows' evolutionary drawbacks

David — Sat, 03 Feb 2007 01:11:28 +0000

If you have ever tried working through an NSA guide on securing Microsoft IIS you will appreciate how many internal systems exists under the hood of Windows and their many deficiencies. For those that have been fortunate to not go through the experience Richard Stiennon uses some compelling visuals by Sana Security to explain why Windows is less secure than Linux because of its long evolutionary history. This history has seen one set of functionality bolted in place over another with little or no thought to the clarity of the overarching system architecture. Consequently whilst Windows works what goes on under the hood is not pretty and very difficult to secure.