How to access ActiveRecord & Migrations outside of Ruby on Rails

Rails migrations are an extremely handy way of capturing changes to your database and rolling them back if they are incorrect.

Manipulating or querying a database is a pretty useful tool and ActiveRecord turns your tables into objects so I looked into how to access these features outside of a Rails environment.

I wanted to be able to either specify the database connection details directly in the script if I was running it outside of a Rails application, or to be able to bind into the Rails application's database.yml database configuration file.

To run it as a standalone script with the database details described inside the script run:

ruby play_with_the_database.rb
To run it within the context of a Rails application place the script inside a new directory called db/script and run:
ruby play_with_the_database.rb development
If you want to run the above script against the production database run:
ruby play_with_the_database.rb production
I'll show you the ruby code that I came up with and then explain the important bits below.
require 'active_record'
require 'pp'

database_type = ARGV[0]

if database_type.nil?
database_type = 'custom'
database_yaml = <<-YAML
#{database_type}:
adapter: oracle
database: database.andrewbeacock.com/beacock01
username: my_schema
password: my_schema
YAML
else
# if you have a Rails project then place your scripts in db/script
database_yaml = IO.read('../../config/database.yml')
end

class MyTable < ActiveRecord::Base; set_table_name :my_table; end

class Script < ActiveRecord::Migration
def self.run
execute('update my_table set updated = true')
pp MyTable.find(:all)
end
end

databases = YAML::load(database_yaml)
ActiveRecord::Base.establish_connection(databases[database_type])
Script.run
  • database_type - this is the RAILS_ENV that you want to run the database script against if you are running within a Rails application, it's empty at the start of the script and set to the value that you pass as the first argument on the command line. If an argument is present it looks up that value in the database.yml file otherwise it uses the inlined database connection details within the script.

  • class MyTable - any ActiveRecord object definitions go here allowing you to map objects to tables.

  • def self.run - Place the guts of your database script within this method so that it has access to all the ActiveRecord objects as well as any methods that migrations have access to (such as execute, create_table, add_column, remove_index, etc.).
This script was spawned from two things, a requirement to change the values in a database table without it being a fully tracked migration and a post on Dave Thomas's blog regarding Migrations Outside Rails.

Enjoy!

Technorati Tags: , , , , , ,

htop - a great alternative to top for Ubuntu Linux

I colleague recommended that I install htop the other day. It's an alternative to top which offers colour (ooh), a tree sort view (aah), and the ability to kill off processes within htop itself (very cool).

To install it, simply run:

sudo aptitude install htop
And to run it, simply type:
htop
This is what it looks like running (with the tree view selected):



And here are some of the help options:



Technorati Tags: , , , , ,

Upgrading to Ubuntu 7.10 (Gutsy Gibbon) causes snmpd errors

When Gutsy Gibbon was released back in October I was quick to upgrade my home Ubuntu server to this new release. The upgrade went smoothly apart from one failure - a strange error message when upgrading the SNMP support:

Starting network management services:invoke-rc.d: initscript snmpd, action "start" failed.
dpkg: error processing snmpd (--configure):
subprocess post-installation script returned error exit status 1
I didn't understand the error message and googling for it at the time brought up no solutions. I checked and found that snmpd was running successfully and all appeared fine. I recently applied some more updates and received the same error message so I investigated it again.

This time I found the reason, the SNMP package upgrade scripts don't stop snmpd before they perform the upgrade, so you need to stop snmpd before you update.

This is how I did it:
$ sudo /etc/init.d/snmpd stop
Stopping network management services: snmpd snmptrapd.
$ sudo dpkg --configure -a
Setting up snmpd (5.3.1-6ubuntu2) ...
Starting network management services: snmpd.
That's it, all updated and no errors, I do hope that the installation scripts get fixed for this soon...

Technorati Tags: , , , , ,

How to rollback Rails database migrations

When you perform database migrations in Rails you use the following command:

rake db:migrate
If you want to roll back your change to need need to find out what version your database is currently at and then roll back to a previous version:
rake db:migrate VERSION=<version to roll back to>
I've never liked this way to roll back, seems to much like hard work to me so I found a migration rollback script on the programmingishard site and simplified so that it only rollback by one version:
namespace :db do
  namespace :migrate do
    desc "Rollback the database schema to the previous version"
    task :rollback => :environment do
      previous_version = ActiveRecord::Migrator.current_version.to_i - 1
      ActiveRecord::Migrator.migrate("db/migrate/", previous_version)
      puts "Schema rolled back to previous verison (#{previous_version})."
    end
  end
end
To use this scriptlet, copy and paste the code into a file called db_rollback.rake and place it the lib/tasks directory within your Rails application.

To use it to roll back your most recent migration simply run:
rake db:migrate:rollback
Technorati Tags: , , , , , , ,

How to get the schema version number out of your Rails migrations

When you want to revert your Rails migration you need the version number so that you can perform the rollback:

rake db:migration VERSION=?
One way is to access the database and take a look in the schema_info table to find the version number.

An easier way is to add an additional task to Rake (found via the Quoted-Printable):
namespace :db do
desc 'Print the current database migration version number'
task :version => :environment do
puts ActiveRecord::Migrator.current_version
end
end
Copy this code into a file called db_version.rake and place it in the lib/tasks directory within your Rails application.

To find out the current version number, simply run:
rake db:version
Technorati Tags: , , , , ,

How to truncate a log file using Ubuntu Linux

Have you ever wanted to truncate a log file that was being written to by a running process without stopping the process to do so?

If so then all you need is the following to clear out the active log file:

cat /dev/null > some_log_file.log
There are some other great tips over on the Bash Cures Cancer blog.

Technorati Tags: , , , , ,

How to add system-wide environment variables to Ubuntu & Debian Linux

Following on from my last post on how to add shared libraries to Linux's system library path I wanted to share a related tip on system-wide environment variables.

Common reasons for doing this are setting your locale, setting the location of a Java installation or specifying the character set that your database is using (if you use Oracle then that would be the NLS_LANG environment variable). Debian actually has a policy against system-wide environment variables stating "a program must not depend on environment variables to get reasonable defaults" and suggests that all programs should have wrapper shell scripts to set the relevant variables.

Ignoring this sound advice I tried adding them to /etc/profile but this didn't work when ssh was used from a remote machine to execute some commands (I was using Capistrano).

I couldn't find a complete way to do it (it wasn't picked up by any start scripts in /etc/init.d) but I achieved the most success by placing my variables in /etc/environment:

LANG=en_GB
JAVA_HOME=/usr/local/src/java
Just remember that you don't need to export anything!

Technorati Tags: , , , , ,

How to add shared libraries to Linux's system library path

Sometimes in Linux when you install a new software package the instructions tell you to add a directory of shared libraries to your $LD_LIBRARY_PATH environment variable in your .bashrc.

You may have noticed that if you then create a shortcut icon on your desktop to this application it won't start because it can't find the libraries.
A typical solution is to write a wrapper shell script to set the LD_LIBRARY_PATH and then call that application.

Well, I've discovered how to add them to your system's library path allowing all environments to access them. Note: There are differences between Debian and Ubuntu (the two flavours of Linux that I'm familiar with).

Ubuntu
Create a new file in /etc/ld.so.conf.d/ called .conf

Edit the file and add a line per directory of shared libraries (*.so files), it will look something like:

/usr/lib/APPLICATION/lib
Reload the list of system-wide library paths:
sudo ldconfig
Debian
Edit /etc/ld.so.conf

Add a line per directory of shared libraries (*.so files) to the bottom of the file, it will look something like:
/usr/X11R6/lib
/usr/lib/APPLICATION/lib
Reload the list of system-wide library paths:
ldconfig
If you run your new application it should now work fine without you having to set any LD_LIBRARY_PATH environment variables.
If you still have problems you can obtain a list of the libraries that are on the system path by re-running the ldconfig command in verbose mode:
ldconfig -v

Technorati Tags: , , , ,

DESTRUCTOID's Team Fortress 2 for Beginners


Reverend Anthony over at the DESTRUCTOID gaming site has written a great set of beginner guides to the different classes of the rather excellent Team Fortress 2 game that came out of beta today. He's provided some tips for each class as well as a general post full of common tips.

I've found them quite invaluable in getting to know the game quickly!

Demoman
Engineer
Heavy Weapons Guy
Medic
Pyro
Scout
Soldier
Sniper
Spy
General Tips

Technorati Tags: , , , ,

A Capistrano recipe for restarting Apache 2 (on Linux)

Capistrano is the rather excellent tool for automating Rails application deployment. But it can do a lot more than just uploading Rails applications to live webservers. It's scripts are basically Rake scripts and so have the full power of Ruby behind then so you can pretty much write code to do anything that you want.

I wanted a way to reload the configuration for my Apache2 web server running on my home Ubuntu server. I found a post on Mongrel and Capistrano 2.0 by John Ward which showed a very nifty way to create Capistrano tasks for variants of the same base command.

My adapted version for controlling Apache is:

namespace :apache do
[:stop, :start, :restart, :reload].each do |action|
desc "#{action.to_s.capitalize} Apache"
task action, :roles => :web do
invoke_command "/etc/init.d/apache2 #{action.to_s}", :via => run_method
end
end
end
Add this code to your config/deploy.rb. This will add four new tasks to Capistrano which you can use to restart or reload Apache:
cap apache:stop
cap apache:start
cap apache:restart
cap apache:reload
Technorati Tags: , , , , , , ,

How to "Show Desktop" in Ubuntu Linux

Like Windows, Ubuntu (and pretty much every other Linux) has a "Show Desktop" icon on it's toolbar that minimizes all the running applications to the taskbar allowing full view of the desktop.

I discovered it last week by accident whilst I was attempting to pull off a difficult triple-press-perfect-timing-required IDE 'shortcut' ;)

The keyboard shortcut for "Show Desktop" is:

Ctrl-Alt-D
I also found a great linux desktop hotkeys article written by the Calgary Linux Users Group.

Technorati Tags: , , , ,

How to restore Windows XP's "Show Desktop" icon

I'm not sure of the exact details but it seems that on some installations of XP the "show desktop" icon is missing from the quick launch toolbar (next to the Start button). Most other applications are also available in the Start menus so it's pretty easy to restore them but not the Show Desktop icon.

The Techie Corner blog had the answer I was looking for - two different ways to restore the Show Desktop icon.

I preferred the second technique - throw the following command into Start ->Run:

regsvr32 /n /i:U shell32.dll
Your show desktop icon should have instantly appeared in the quick launch toolbar!

Technorati Tags: , , , ,

Manager Tools - my favourite podcast for over 2 years

Back in 2005 I blogged about the podcasts that I subscribe to and one of my favourites was Manager Tools.

Well two years later my list has changed a little (click on the 'podcasts' link on the left for a closer look) but Manager Tools is still my favourite podcast, the moment the show music starts to play in my car I know I'm in for an hour of energetic & informative advice regarding all aspects of management & leadership.

I'm amazed at how much time & effort Michael Auzenne & Mark Horstman put into producing a timely weekly show full of great content with such audio finesse as well.

I was recently very fortunate to get a sneak preview of some of their premium content and I must say if it's all of the same high quality (which with Mike & Mark is a given) then it's definitely worth the month fee.

They are consistantly the number 1 business podcast at PodcastAlley and have won the "Best Business Podcast" for two years in a row. If you only add one podcast to your itunes/podcatcher this week, make it Manager Tools, you won't regret it.

Technorati Tags: , , , ,

Having cron troubles with percent (%) signs in crontabs?

I recently wrote a cronjob which would save off my Ubuntu home server's current packages list every night to a dated file so that I have a history of the changes that I've made to my system. The entry in my crontab looked something like this:

# package lists (5am)
0 5 * * * dpkg -l > /var/log/packages/`date +%F`.txt
After a couple of days I decided to take a look at the files generated so far but to my disappointment I found none. After a quick search for dpkg in /var/log I found my problem reported in syslog:
... /USR/SBIN/CRON[28508]: ... CMD (dpkg -l > /var/log/packages/`date +)
I noticed that the command recorded to the logs was up to '`date +' - the next character - the percent sign (%) - was causing some kind of issue. A quick search threw up the Wikipedia page for Crontab which mentioned:
Another common mistake is to use unescaped % in your command; you have to escape them
All I had to do was change % to be \%. My working crontab entry now looks like this:
# package lists (5am)
0 5 * * * dpkg -l > /var/log/packages/`date +\%F`.txt
The funny thing was that most of the 'proper' Linux documentation for cron and crontab do not mention anything about percent signs (%) - I'm finding that I'm going to Wikipedia for answers more and more these days...

Technorati Tags: , , , ,

How to open Firefox Google searches in a new tab

I'm always using the built-in Google search box in Firefox to do my research. I used to click the 'New Tab' button (or CTRL-T) to open a fresh tab, then type in the box and hit return to open the results in my newly created tab.

I then found that it will open in a new tab automatically if you use Ctrl-Return rather than just Return when searching. That was a great improvement but if I forgot to hold down Ctrl I would wipe out my current tab with the Google search results.

Well I've found the ultimate solution from an 'Hacking Firefox' article on Computer World found via the excellent LifeHacker blog. It's an about:config value that makes the search results open in a new tab (in the same way that Ctrl-Return did).

To enable it, enter about:config in the location bar. Enter browser.search.openintab in the "Filter" text box and then double click the listed browser.search.openintab to change it's value from false to true:



Oh, and if you want the results to open in the current tab rather than a new tab for a particular search then you simply use Ctrl-Return instead of just Return! Joy!

Technorati Tags: , , , , ,

Getting Tomcat contexts to work in IntelliJ IDEA (and stop it starting Tomcat twice!)

If you run Tomcat within IDEA then you might have noticed a couple of things:

  1. Your application gets started, undeployed and then started again, increasing the start up time of Tomcat and reinitialising any load-on-startup servlets for a second time.

  2. Your carefully crafted <context> stanzas don't work as IDEA hasn't picked then up from either server.xml or META-INF/context.xml.
When you start Tomcat inside IDEA it copies the contents of your Tomcat conf directory to an IntelliJ system area (on Linux it's hidden in your home directory) and then runs Tomcat looking at those configuration files instead of the normal ones. In doing this it seems to ignore any context.xml files, any context stanzas of the server.xml and messes up any context files you might have in conf/Catalina/.

To get around this big mess and just use your Tomcat as if you ran it outside of IDEA you simply add a CATALINA_BASE environment variable to the configuration of Tomcat (within IDEA) pointing to your Tomcat directory:



Now when you start Tomcat within IDEA you should notice that your contexts are working correctly!

Technorati Tags: , , , ,

How to get the Home & End keys working in remote Linux shells & terminals

Have you ever SSH'ed onto a remote linux server only to find that your Home and End keys don't take you to the start and end of the line and instead give you a rather unhelpful "~"?

One solution is to use CTRL-A instead of Home and CTRL-E instead of End but I wanted to solve it properly.

The linux command line is managed by readline. The configuration file for controlling key bindings (amongst other things) is /etc/inputrc. In Ubuntu this file contains a number of commented out sections relating to the Home and End keys. To find out which keycodes are being sent to readline when Home & End are pressed I used od - a stdin dump utility which can display ASCII characters or backslash escapes:

$ od -c

<Home key pressed> ^[[7~
<End key pressed> ^[[8~

The key bits here are the 7 and 8, these need to be entered at the very end of /etc/inputrc as follows:
"\e[7~": beginning-of-line
"\e[8~": end-of-line
After you've made these changes logout and back in again and now your Home & End keys should be working perfectly!

Technorati Tags: , , , , , ,

Viewing Outlook vCalendar meeting invites in Thunderbird

I received an Outlook meeting invite in my email recently but because I use Thunderbird as my email reader I received the raw vCalendar data rather than a nicely formatted email with details of the meeting. After a quick google search I found a post on Daniel Tome's blog regarding Viewing Calendar Events in Mozilla Thunderbird.

His suggestion was to install a Thunderbird add-on called Lightning. This adds a calendar into Thunderbird and also adds support for understanding vCalendar events. More information can be found at Lightning's home page.



I installed the add-on and restarted Thunderbird and now the raw vCalendar email was transformed into a nicely formatted meeting invite - you even get the option to accept or decline the meeting. Just remember to right-click on the green "Install Now" button to download the add-on otherwise you will install it into firefox.

Thanks again Daniel!.

Technorati Tags: , , , , , , ,

Opening winmail.dat (TNEF) files in Thunderbird (on Ubuntu)

 
UPDATE: I have an improved version of the script available here.

I'm not going to repeat the many, many websites taking (and complaining) about Microsoft's proprietary e-mail attachment format called TNEF.

I'm assuming that if you are reading this then you have found that these fixes are not working for you (or not possible to enforce). I'll also assume that the LookOut Thunderbird Add-on by Aron Rubin is also not working correctly for you (this was my experience on Ubuntu Edgy Eft).

The best solution I could come up with was getting Thunderbird to run a script to unpack the winmail.dat extension into a folder on your Ubuntu desktop.

It relies on the tnef command-line program, so make sure that is installed first (it's bundled with Ubuntu):

sudo aptitude install tnef
Below is my little script, save it in a file called 'tnef.sh' somewhere and make sure it's executable (chmod +x tnef.sh) - or just download it here.
#!/bin/bash

LOCATION=~/Desktop/winmail.dat

mkdir $LOCATION
/usr/bin/tnef -C $LOCATION --save-body -f $1
Now find an email in Thunderbird with a winmail.dat attachment. Double click it and select to open it with the newly saved tnef.sh file:



Look on your desktop - there should be a 'winmail.dat' directory with the full contents of the attachment.

Double-clicking on any future winmail.dat file will result in the contents of the attachment to also be added to that directory.

Technorati Tags: , , , ,

How to stop schema.rb from being generated when running Rails Migrations

A couple of days ago I blogged about why rails migrations caused the database schema to be dumped into schema.rb. I've now found a way to stop it from happening without changing the schema_type or modifying any core Rails or Rake classes.

What I planned to do was override the db:migrate rake task to do the same as the default one bar the schema dump. My first attempt failed as it appears that rake will allow more than one task to have the same name, it runs then one after the other. The result of this was some strange migrations plus the delay due to dumping the schema!

After a little googling I found an excellent post on Matthew Bass's blog entitled Overriding existing Rake tasks. This described a similar situation and I've based my code on his advice.

I needed to delete the existing db:migrate task before adding my new one. You can't just call delete task on the Rake::Application class (there's no such method) so Matthew's suggestion was to add a delete_task method to the Rake::TaskManager module (that has access to the tasks) and then call this method before we define our new db:migrate task. The finished script is given below:

# add a delete_task method to the TaskManager and delete db:migrate
Rake::TaskManager.class_eval do
def delete_task(task_name)
@tasks.delete(task_name.to_s)
end

Rake.application.delete_task("db:migrate")
end

# define a new db:migrate that did the same as the old one bar the schema dump
namespace :db do
desc "Migrate the database through scripts in db/migrate. Target specific version with VERSION=x, don't run db:schema:dump at the end."

task :migrate => :environment do
ActiveRecord::Migrator.migrate("db/migrate/", ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
end
end
Technorati Tags: , , , , ,

Why do Rails Migrations cause the database schema to be dumped into schema.rb?

Have you ever wondered why rake db:migrate took so long to complete when you ran a simple migration script? The reason is that it's dumping your database's entire schema into db/schema.rb just like rake db:schema:dump does.

I had to get to the bottom of why this was happening and searching the internet was not providing any answers, so I took to searching the Rails codebase to see if I could find any clues.

databases.rake in /usr/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/tasks/ showed the root cause:

namespace :db do
desc "Migrate the database through scripts in db/migrate. Target specific version with VERSION=x"
task :migrate => :environment do
ActiveRecord::Migrator.migrate("db/migrate/", ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
Rake::Task["db:schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
end
If the 'schema_format' is set to':ruby' then run the rake task 'db:schema:dump' after we have finished running the migrations. But where is schema_format set? A look in config/environment.rb gave a clue:
  # Use SQL instead of Active Record's schema dumper when creating the test database.
# This is necessary if your schema can't be completely dumped by the schema dumper,
# like if you have constraints or database-specific column types
# config.active_record.schema_format = :sql
This configuration value is commented out and indicates that if this is enabled then the schema dumps will be in SQL format rather than the more portable (and in my mind preferable) Ruby format.

base.rb in /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/ gave the final clue:
    # Specifies the format to use when dumping the database schema with Rails'
# Rakefile. If :sql, the schema is dumped as (potentially database-
# specific) SQL statements. If :ruby, the schema is dumped as an
# ActiveRecord::Schema file which can be loaded into any database that
# supports migrations. Use :ruby if you want to have different database
# adapters for, e.g., your development and test environments.
cattr_accessor :schema_format , :instance_writer => false
@@schema_format = :ruby
So base.rb sets the schema_format variable to be ruby and therefore enables the automatic dumping of the schema after every migration.

Now I just have to figure out how to disable it...

Update: I've figured out a way to disable schema.rb from being generated.

Technorati Tags: , , , , ,

Undefined 'define_a_column' method when using Rail's Oracle OCI adapter?

One of the first hurdles I ran into when I started with Rails migrations was:

undefined method define_a_column' for class OCI8::Cursor'

It means that your LD_LIBRARY_PATH environment variable does not include the Oracle client binaries.

(I got my clue on how to solve this issue from this old defect ticket on the Rails Trac).

Technorati Tags: , , , ,

Where's dos2unix? In Ubuntu's tofrodos package!

I had a strange error recently to do with a bash script not running correctly. I couldn't see anything wrong with the file until I noticed that VIM was reporting that the file's type was DOS.

What I needed was to convert the downloaded file to UNIX file format using dos2unix but which package does it live in? A quick search of the Ubuntu packages database revealed tofrodos:

Converts DOS <-> Unix text files, alias tofromdos

DOS text files traditionally have CR/LF (carriage return/line feed) pairs as their new line delimiters while Unix text files traditionally have LFs (line feeds) to terminate each line.

Tofrodos comprises one program, "fromdos" alias "todos", which converts text files to and from these formats. Use "fromdos" to convert DOS text files to the Unix format, and "todos" to convert Unix text files to the DOS format.

This functionality is also available via the dos2unix/unix2dos symlinks.
To install it run:
sudo aptitude install tofrodos
And then use dos2unix and unix2dos to convert the file between the two different formats.

Technorati Tags: , , ,

Firefox hack to display "accesskey" keyboard shortcuts

I'm a big fan of using accesskeys in Firefox as I just love keyboard shortcuts! One problem is that most websites don't advertise the fact that they have accesskeys and what those shortcuts are.

After a little searching I found this forum post from back in 2004 with a hack to Firefox's UserContent.css file. I changed it slightly to show a nice purple "[shortcut key]" when it finds a keyboard shortcut. This is what it looks like when viewing the Wikipedia site:



To install this hack create a file in your profile's chrome directory called 'userContent.css' and add the following:

/* Display [] by the item with the accesskey */
a[accesskey]:after,
button[accesskey]:after,
input[accesskey]:after,
label[accesskey]:after,
legend[accesskey]:after,
textarea[accesskey]:after {
margin-left: 0.3em;
color: Plum;
content: "[" attr(accesskey) "]";
}

One other thing to remember is the accesskey enabler changed from Alt to Alt-Shift in Firefox2 (release notes):
Access key definitions provided by web pages can now be triggered using Alt+Shift+key on Windows, Ctrl+key on Mac OS X, and Ctrl+Shift+key on Unix.

Technorati Tags: , , , ,

Some useful Ruby & Rails firefox search plugins

I've found the following two Firefox search plugins useful and thought
I'd pass then on:

RubyInside

This plugin searches a massive list of sites (scroll to the comments for more details) maintained by Peter Cooper, a UK Rails developer who runs the excellent RailsInside website.

RailsHelp

This plugin was developed by Nick Cody to search the RailsHelp.com website.
Update - railshelp.com seems to be down at the moment...

Getting the Right Alt key (Alt GR) to work in Ubuntu Linux

I've been bothered for a while now that my right alt key labelled 'Alt GR' doesn't work in Ubuntu. It was clear that things weren't right when I ran 'xev' and pressed the two Alt keys - here is the output:

KeyPress event, serial 29, synthetic NO, window 0x3200001,
state 0x10, keycode 64 (keysym 0xffe9, Alt_L), same_screen YES,

KeyPress event, serial 29, synthetic NO, window 0x3200001,
state 0x10, keycode 113 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,

ISO_Level3_Shift? Not really the Alt_R I was looking for!
After reading this Ubuntu bug report I change my keyboard preferences to be "Alt and Meta are on the Alt keys" (found under System -> Preferences -> Keyboard)



Now when I run xev this is what I get:

KeyPress event, serial 30, synthetic NO, window 0x3200001,
state 0x10, keycode 64 (keysym 0xffe9, Alt_L), same_screen YES,

KeyPress event, serial 30, synthetic NO, window 0x3200001,
state 0x10, keycode 113 (keysym 0xffea, Alt_R), same_screen YES,

At last, a working Alt-R key!

Technorati Tags: , , , , ,

How to change your Rails database adapter from OCI to Oracle

I've started to use Rails migrations recently connecting to an Oracle database.

After reading many examples of how to configure the database.yml file on the Rails wiki (as well as ones from Oracle's own website), I ended up with something like:

development:
adapter: oci
host: database.com/mySID
username: schema_name
password: schema_name
This worked fine, but after researching other migration-related topics I read that oci was the 'old' way to connect to Oracle, the 'new' adapter to use is oracle.

A tutorial from Oracle showing usage of the new 'oracle' adapter (scroll down to the section entitled "Part 1: Of configuration, Cursors, and Rows") suggested that your config should look like this:
development:
adapter: oracle
host: DEBIAN
username: rails
password: rails
Note: DEBIAN was the name of their TNS entry specified in their local tnsadmin.ora file.

I didn't want to use a reference to a local tnsadmin.ora file on my machine, I wanted to specify the host and SID explicitly. After many failed attempts at specifying this in different formats I happened across this Rails patch ticket regarding the change from 'oci' to 'oracle' - the winning clue was:

To use the adapter with it's new name, simply change your database.yml file to include:
adapter: oracle
database: <your db name here>

Note that database is now used instead of host.

So the problem was that I was entering the database server details in the host field (like all the examples said to) but for a non-TNS named server you need to use the database field.

My fully working Oracle database.yml config entry now looks like this:
development:
adapter: oracle
database: database.com/mySID
username: schema_name
password: schema_name
Technorati Tags: , , , , ,

Manually setting the system date (and hardware clock) in Ubuntu Linux

I typed date on my Ubuntu server tonight at 23:06, this is what it reported from the system clock:

Tue May 29 22:56:41 BST 2007
I decided to also check the hardware clock (the one maintained by the CMOS/BIOS) by running hwclock:
Tue 29 May 2007 23:04:46 BST  -0.258052 seconds
Looks like I need to bring my server 'up to date'!

To update the system clock use the date command but also pass the date you want the clock set to (you need to be root/sudo to do this):
date 05292310
This reads as: "set the system date to the 5th month, the 29th day, the 11th hour (pm) and ten minutes". Remember the date format is American so month first then the day!

That sorted the system clock out, so I turned my attention to the hardware clock. This can be easily synchronised with the now correct system clock with the systohc option:
hwclock --systohc
Below is the hardware clock's date before and after the update compared to the system clock:
$ date
Tue May 29 23:14:12 BST 2007

$ hwclock
Tue 29 May 2007 23:13:02 BST -0.504456 seconds

$ sudo hwclock --systohc

$ hwclock
Tue 29 May 2007 23:14:27 BST -0.679613 seconds
Now both clocks are back in sync with the real time and each other. Of course this can also be solved with ntp but that's an article for some time in the future! :)

Note: for a more detailed explanation of clocks, locales and timezones please read LinuxSA's "Linux Tips - Linux, Clocks, and Time"

Technorati Tags: , , , , , , ,

My Scan UK Search Plugin for Firefox 2 & IE 7

When I was researching PC components back in September I spent a lot of time on Scan's excellent website looking up prices, specs, etc. A few weeks ago I stumbled upon a page describing how to create OpenSearch plugins for Firefox (which can also work in IE7).

It didn't look that hard so I picked Scan's website as my target. Here is the result:



This has been tested on Firefox2 and IE7 on WinXP and Firefox2 on Ubuntu. I would be very interested to know if it works on any other browsers or platforms...

More search engines can also be found at on the Firefox Search Engines page and at searchplugins.net.

Technorati Tags: , , , , , ,

Upgrading Ubuntu Server from 6.10 (Edgy Eft) to 7.04 (Feisty Fawn)

Since I reinstalled my webserver to run Ubuntu Server a couple of weeks ago, I was please to see that Feisty Fawn, the latest version of Ubuntu has been released.

Upgrading my server was easy, I simply followed the instructions on the Ubuntu website for performing the recommended network upgrade for Ubuntu servers.

It also cleaned up my manual installation of my wireless network drivers ensuring that I won't have problems when upgrading the kernel in the future.

The only hiccups I found were around the area of PHP - I use Cacti to monitor my server and network and it was having problems running it's CLI scripts. I had noticed during the upgrade that PHP5 was being installed so I thought it would be best to remove all trace of PHP4 and ensure that PHP5 was installed:

sudo aptitude remove php4-mysql
sudo aptitude remove php4-cli_
sudo aptitude remove php4_
sudo aptitude install php5


Note the "_" on the end of the package name, this also removed all the configuration files associated with that package.

That cleared my cacti issue up and now my server is running happily on the new Ubuntu!

Technorati Tags: , , , ,

How to mount a USB thumbstick drive using Ubuntu Server

I recently upgraded the OS of my linux webserver from Debian Sarge to Ubuntu Server (6.10 - Edgy Eft) (the semi-automatic upgrade from Debian Sarge to Etch messed up my wireless network card and I thought I might as well start again with Ubuntu this time...).

As my server only has a wireless card for it's network access and because the card uses a Broadcom chipset I had no network access after the installation of Ubuntu.

I transferred the required network drivers to a USB thumbstick drive and plugged it into the back of my server. I also use Ubuntu in it's desktop form and that auto-mounts any USB devices that are plugged in. My server didn't so I had to mount it manually.

Normally with Linux you need to know all sorts of information about the drive's file system type that you are mounting - not with Ubuntu, this was all I had to do to get the contents of the USB drive accessible in the /mnt directory:

mount /dev/sda1 /mnt

Technorati Tags: , , , , ,

A big thank you to RobinsReviews.com!

I received a surprise gift through the post on Saturday, a gift wrapped book off my Amazon Wish List from Robin of RobinsReviews.com.

Robin sent me the book after finding the Technorati Tags Bookmarklet on my blog - thanks again Robin I hope you find it very useful!

Technorati Tags: , ,

Problem performing an HTTP Redirect on a Motorola SLVR L7 mobile WAP browser

I'm blogging about this in the hope that it can help anyone else who happens to get stuck with this same issue.

On Friday I was caught out by a very 'interesting' bug in the mobile (WAP) internet browser of a Motorola SLVR L7 handset.

We were attempting to perform an HTTP redirect from one WAP site to another on the SLVR but were receiving a "404 Page Not Found" error message. When we looked at the URL that the phone was attempting to redirect to it was a combination of the first site with a bit of the query string of the second site tacked on the end:

First WAP site: http://www.first.com/Finder?type=holidays
Second WAP site: http://www.second.com/Holidays?title=Great Holiday+Location-Mar+2007

Result of redirect: http://www.firstsite.com/Finder-Mar+2007

We thought it might be the length of the URL, the number of parameters in the query string, or the fact that the sites were situated on different web servers. Mobile phone browsers are notorious for not fully supporting the HTTP stack and the Motorola SLVR was probably no exception. We resorted to slowly stripping out bits of the second URL hoping that at some point it would start to work.

We stripped all the parameters off the end of the second URL and that resulted in a successful redirection to the correct server. We then added the parameters back on and found that when we added the "title" parameter it failed. After a little more experimentation we discovered that the source of the problem was the word "Location" found in the value of the title parameter!

"Location" didn't work but "location" did...

Here's the reason why, take a look at this HTTP redirection header sent from the first server to the handset:

HTTP/1.x 302 Moved Temporarily
Date: XXXX
Server: XXXX
Location: http://www.second.com/Holidays?title=Great Holiday+Location-Mar+2007
...other HTTP headers...


The word "Location" occurs twice, once as a true HTTP header field and once as text in a URL.

Our guess is that the SLVR's browser code is a rather sloppy in how it detects and handles HTTP redirects and rather than looking for "Location" at the beginning of the line it matches the last one that it can find - hence the rather strange "-Mar+2007" on the end of the first URL!

Technorati Tags: , , , , ,

The current situation of REST applications accessed by a web browser

As regular readers will know I'm not one for link blogging but this recent post by Dave Thomas of The Pragmatic Programmers is the reason for me to do so.

This is one of the clearest, well written explanations of the current limitations of implementing REST-based applications that are accessed via a web browser that I have found.

This excerpt is one to to bear in mind before getting excited about the concept of REST:

How well do browsers play in a RESTful world?

Well, for a start, they really only use two of the HTTP verbs: GET and POST. Is this a problem? Yes and no.

If you want to know which bits of REST a browser currently supports and which bits it doesn't - read on...

Technorati Tags: , , ,

Screen capture a single window with Alt-Print Screen? - I never knew!

I've known about the Print Screen (Prt Scr) button taking a full screenshot and saving it in the clipboard for ages (and with Ubuntu it pop ups a handy "save screenshot" window as well).

I never knew that you could hold down the Alt button at the same time and get a screenshot of just the currently highlighted window!

You learn something new everyday - well I do at least...

More info: Windows Print Screen Key

Technorati Tags: , , , ,

March's AgileNorth - Applying Agile Software Practices to Consultancy Projects

This month's AgileNorth was a short presentation and following discussion on Applying Agile Software Practices to Consultancy Projects.

The 20-odd slides were presented by Simon Monk, CTO of Momote (Momote's wikipedia page). Momote offer products that allow the development of mobile applications that will run on any device from a consumer Java phone to a Windows Mobile PDA.

Simon has worked in an agile development way for some time and now wants to create a consultancy group that can work in an agile way. Momote are 6 months into offering consultancy services and have a small XP team for the main development of their MX platform. At the moment they have one consultant but are growing that team to three or four in the very near future.

Momote don't want to follow the traditional consultancy process which was described as:
* lengthy (and therefore costly) detailed requirements gathering
* Big Design Up Front - BDUF
* Severe project management - Gant charts, Microsoft Project, etc.
* Waterfall model of software projects - development, testing, delivery

They hope to offer a new way where they have happy customers & consultants, real & accurate project estimates and much more flexible requirements (so that the customer actually gets what they want rather than what was captured in the initial requirements capture session). They hope that the client still gets the security of fixed priced contract but with added flexibility and feedback that the agile approach can provide.

Momote wants to try and capitalise on the following agile practises:
* low cost estimates - requirements gathering takes about a day not weeks
* task estimates
* velocity & tracking
* flexible scope - customers are allowed two significant changes
* customer involvement

They have come up with a simple way to categorise projects to be able to give 'ball park' estimates to clients as part of their non-chargeable requirements capture phase:

CostSmallMediumLarge
Requirements2 days4 days5 days
Development10 days20 days40 days
Handover3 days4 days5 days


As part of the output of this phase they produce a five to six page requirements specification which includes the initial project plan on the back page. They then follow up with the client by the use of weekly web-based review meetings. These meetings allow the client to review the work done so far, see how the actual productivity of the consultant matches against the estimated throughput and provides an opportunity to change the work plan for the following week.

They have found that a minority of customers are "too busy" for the weekly update meetings and just want Momote to produce what was initially discussed, but so far most clients have been very accepting of the 'new way'. In fact it was mentioned in the discussion section after the presentation that the weekly meetings seem to be key to the whole process working well.

A few software tools were mentioned in the discussion that followed:

Trac
Trac is an enhanced wiki and issue tracking system for software development projects. Trac uses a minimalistic approach to web-based software project management.
I've blogged previously about:
Installing Trac on Linux (Debian Stable)
Creating a Trac instance on Debian Linux

Trac plugins

Confluence
Confluence is an enterprise wiki that makes it easy for your team to collaborate and share knowledge.

Jira
JIRA is a bug tracking, issue tracking, and project management application developed to make this process easier for your team. JIRA has been designed with a focus on task achievement, is instantly usable and is flexible to work with.

HttpUnit
HttpUnit emulates the relevant portions of browser behavior, including form submission, JavaScript, basic http authentication, cookies and automatic page redirection, and allows Java test code to examine returned pages either as text, an XML DOM, or containers of forms, tables, and links.

Watij
Watij (pronounced wattage) stands for Web Application Testing in Java. Watij is a pure Java API created to allow for the automation of web applications.


Technorati Tags: , , , ,

How to install a specific version of a Debian package using apt-get

At work we use a backported version of Subversion as the stable package is 1.1.4 and I wanted to install the latest one we could get a package for which was version 1.3.2. Along with the backported Subversion package is a package that contains a number of useful hook scripts called subversion-tools.

When I did an apt-get install subversion-tools Debian wanted to install the 1.1.4 stable release not the 1.3.2 backported release. By doing an apt-cache show subversion-tools I was able to see the two available packages.

These are the steps I went through to install the backported version of subversion-tools, although these steps will work when installing any specific version of a Debian package.

Before you install a specific version of a package you need it's complete version.
This will be displayed when you run:
apt-cache showpkg <package name>

e.g.
apt-cache showpkg subversion-tools

> Package: subversion-tools
> Versions:
> 1.3.2-5~bpo1(/var/lib/dpkg/status)
> 1.1.4-2 (/var/lib/apt/lists/ftp.uk.debian.org_debian_dists_stable_main_binary-i386_Packages)


To install the specific version simply run run:
apt-get install <package name>=<version>

e.g.
apt-get install subversion-tools=1.3.2-5~bpo1

Technorati Tags: , , , , , , ,

Using cvs2svn to migrate your repository from CVS to Subversion

This week I migrated our codebase from CVS to Subversion!

I'd previously downloaded the Tigris cvs2svn migration tool onto my build server, read the documentation and performed a test run of the migration last week so I was pretty confident that I would have no problems.

These are the steps that I took to perform the migration:
Notes: For simplicity I'll assume that your CVS repository root is /cvs and your Subversion directory is /svn. The cvs2svn tool creates a new Subversion repository when you run the cvs2svn migration tool.

  • Log in to the build server (a Debian Linux box) as root

  • Change directory to /cvs and find the name of the module you want to migrate to a Subversion repository, I'll use the module old in my example.

  • Choose a suitable name for your new Subversion repository (it doesn't have to be the same name as the CVS module) I'll use new in my example.

  • Run the following command, (it's best to redirect it's output to a log file to so you can check for errors at the end):

    cvs2svn -s /svn/new /cvs/old
This operation may take a while if the CVS repository is large, our migrated repository took about 30 minutes to migrate and resulted in about 3,500 revisions. I had no problems in the area of branches or tags but this is one issue that has cropped up when I migrated another repository over. You need to use the --force-tag and --force-branch options to specify how to deal with any problematic tags.

I performed the migration as the root user to ensure that we wouldn't have any problems with file permissions when reading or writing files, but we now need to change the permissions of the newly created Subversion repository to ensure that Apache has access to the files:

chown -R www-data:www-data /svn/new

Your new repository will now be available in the same way that you access your other Subversion repositories, just remember to use your newly migrated repository name!

Technorati Tags: , , , , , ,

Electronic voting machines hacked to give false voting results

I think this is a concern for all processes that are "going digital" - the opportunity for hacking and fraudulent activity to alter the state of the output. Check out this short demonstration video from Princeton University on installing rigged voting software onto a Diebold AccuVote-TS voting machine, one of the most widely deployed electronic voting platforms in the United States.

Technorati Tags: , , ,

How to restore a deleted and committed directory in CVS

I had to put my CVS administrator hat on this morning when a colleague realised that yesterday he had managed to delete and commit a directory full of files by accident. We needed to roll-back the clock to yesterday afternoon so to speak but CVS doesn't really give you any handy tools to do this.

I first logged onto the CVS server and browsed to the relevant directory (on the server the directory and files are not deleted, just marked as so) and took a look inside one of the ,v text files to find the time when the commit happened (where the file's state changed from Exp to dead).

After a bit of searching I found a very clear set of instructions on the Ximbiot CVS Wiki that detailed how to restore a deleted directory.

I followed those instructions and brought the directory (and files) back to life again!

Technorati Tags: ,

The latest VentureVoice podcast is a real treat!

I'm two-thirds of the way through listening to the latest VentureVoice podcast and I must say it's one of the most interesting and captivating things that I've listened to in a long time.

It's an interview with Fred Seibert, a life-long entrepreneur, a former jazz producer, the guy who introduced the animated MTV logo, turned Hanna-Barbera cartoons around and now runs his own online cartoon video network.

Greg Galant (the host of VentureVoice) seems to ask just the right questions and Fred describes his experiences in such an interesting way.

Download it and listen to it as soon as you can:

VV Show #43 - Fred Seibert of Frederator Studios and Next New Networks

Technorati Tags: , , , ,

How to change your default locale on Ubuntu Linux

One problem that has repeatedly cropped up when developing in Java is strange error messages in our unit tests for certain text manipulation tests when running on a freshly installed Ubuntu desktop.

They are all related to Ubuntu's default British locale: en_GB.UTF-8

This was causing files checked out of CVS to be in Unicode (UTF-8) format rather than ISO-8859-1 and so the British pound sign (£) was being encoded as a double-byte (rather than single-byte) character in the file.

To check which locale you currently have as your default just run: locale

Changing the default locale is a little different on Ubuntu compared to most Linux distros, these are the steps we needed to go through to get it changed:

Add the locale to the list of 'supported locales'
Edit /var/lib/locales/supported.d/local and add the following line:
en_GB ISO-8859-1

Regenerate the supported locales
Run sudo dpkg-reconfigure locales

Change the default locale
Edit /etc/environment and ensure the LANG and LANGUAGE lines read as follows:
LANG="en_GB"
LANGUAGE="en_GB:en"


UPDATE '09: An old collegue has suggested that this change should now be made in /etc/default/locale rather than /etc/environment - Thanks Guy!

Reboot!

Rerun locale to check that your default locale is now en_GB

Technorati Tags: , , , , ,

January's AgileNorth - Theory of Constraints (TOC)

The first AgileNorth of the new year was kicked off by an active session on the “Theory of Constraints” run by Kevin Rutherford and David Draper.

It was a re-run of a session from the Agile North conference - I'm not a Bottleneck, I'm a free man!

The Theory of Constraints (TOC) is a management philosophy that aims to continually achieve more of the goal of a system. Every system or process must have at least one constraint, which prevents the system from achieving a better result relative to it's goal. In order to increase the performance, these constraints must be identified and removed one by one.

There is a five-stage process to go through to follow TOC once you have clearly identify what the goal of the process is, e.g. better quality, more widgets, less wastage:


  1. Constraint
    Find the largest constraint that is holding you back from achieving that goal. Note: you are not trying to fix any and all problems or possible optimisations that you find, just the largest key bottleneck.
  2. Exploit
    Try to optimise the bottleneck to get maximum performance, e.g. remove any unnecessary tasks (timesheets, walking across the factory floor, etc.) from a worker who's duty has been highlighted as the bottleneck.
  3. Subordinate
    Move some of the constrained worker's tasks to other people/processes to free up the key worker to allow the bottleneck to be removed.
  4. Elevate
    If the above “free changes” (exploit and subordinate) don't help remove the constraint then introduce a “costly change” e.g. training, additional staff, etc.
  5. Repeat
    If the constraint has now moved, go back to step one and repeat the process.

The focus is always on one constraint at any one time – never attempt to address more than one problem area.

The main part of the evening was spent doing a practical exercise that demonstrated the Theory of Constraints quite clearly. The goal was to maximise the number of paper hats and boats pairs made in a short period of time. There were 6 of us attending so we formed a circle and were given different roles perform:

  • Analyst – received paper of differing sizes from the supplier and was told whether it would be a hat or a boat
  • Designer – would perform some of the critical early folds
  • Developer – did most of the folding of the paper
  • UI – added pictures to the sides
  • Tester – ensured that the made objects stood up, had the correct design, was neat and tidy, etc.
  • Production – checked quality and released 'product' - hat/boat pairs.

The first round started and we immediately noticed that all the paper was stocking up by the developer who was starting to rush and make mistakes, most of the rest of us were sat around doing nothing. Therefore the 'constraint' was the developer. We struggled to see how we could 'exploit' the developer so we 'subordinated' some tasks: we decided to ensure that it was clear whether the item was to be a hat or a boat (this had caused confusing during the first round) and would pass the paper to the developer in order, hat – boat – hat -boat to ensure enough 'product' was released.

On the second round we noticed that the bottleneck was still with the developer, the boats were taking nearly twice as much time to make as the hats. We decided to 'elevate' the constraint by 'training' the UI guy to take over the creation of the hats so that the developer could concentrate on the more complex boats. This time most of us were sat doing nothing for longer and yet we were able to get more products out the door – we were more productive whilst doing less work!

With the 'training' complete ;) we performed the third and final round and this time came out with some very positive results. This time the constraint had moved to the designer who was having to reject certain paper sizes due to it being too hard to make the hats and boats. This showed that if we had given the 'supplier' a template for the source paper we could have removed this bottleneck and sped up even further.

Here are the results of the three rounds:

Round123
source paper121012
completed products123
wastage/work in progress1062


It was a very interesting session and I think we all had some fun as well. It would have been good if we had more some spare time at the end to discuss possible ways in which you could spot the bottlenecks in a software environment. I enjoyed the active format and Kevin and David ran the session very well.

Technorati Tags: , ,

Problems with shell (sh) scripts on Ubuntu Edgy Eft? Find the fix here!

I use Ubuntu at work and recently upgrade from 6.0.6 (Dapper Drake) to 6.10 (Edgy Eft).

Today I found a problem with an SQL tool that we use - Oracle's SQLDeveloper. It runs from a shell script and was failing with a very strange error (it couldn't find a config file). After some head scratching and searching on the web I found the reason why:

Previous versions of Ubuntu (and probably most distributions of Linux) map sh to bash. Bash has more features and many scripts are written with Bash in mind.

Ubuntu 6.10 has changed this so that sh now uses dash - a faster-than-bash alternative that only supports what the original sh did. There is even an outstanding Ubuntu bug regarding this.

This was the reason why my script was failing - it was assuming a bash shell even though it mentioned #!/bin/sh at the top of the script, and dash just doesn't have bash's features.

The fix was simple:

In the /bin directory sh is linked to dash so you just need to update the link to point back to Bash:

cd /bin
sudo ln -sf bash sh


After that simple change SQLDeveloper fired right up!

Technorati Tags: , , , , , ,