Colour Linux & Cygwin console searches with Colourised Grep

Grep is a wonderful tool useful for searching for all kinds of stuff but sometimes what you are searching for gets lost:



To get your console shell to add a little colour to your grep search terms add the following to your .bashrc or .profile or the like:

export GREP_OPTIONS='--color=auto'
export GREP_COLOR='1;33'

Now if you perform the same search again your output should be a little clearer:



And as an added bonus this works on Cygwin (the Windows Unix shell) as well as Linux!

Technorati Tags: Cygwin, Linux, Console, Shell, Colour, Grep, Andrew Beacock

How to control another PC without a KVM using Synergy

If you sometimes have your laptop next to your desktop PC you might find that you are hopping from one keyboard/mouse/trackpoint to the other. After a while this gets a little uncomfortable so wouldn't it be cool if you could use your desktop PC's mouse and keyboard on your laptop? Synergy is one such utility, this is how the Synergy team describe it:

Synergy lets you easily share a single mouse and keyboard between multiple computers with different operating systems without special hardware. It's intended for users with multiple computers on their desk since each system uses its own display.

Download Synergy from sourceforge and install it as normal on both you desktop PC (the server) and your laptop (the client). On your desktop PC, start Synergy (Start -> Programs -> Synergy -> Synergy) you will be presented with the following screen:



Click "Share this computer's keyboard and mouse (server)" then click "Configure..." to set up the layout of the screens. Your now presented with this screen:



The first thing you have to set up are the "Screens", these are the computer names that you have given to your desktop and laptop, in my case "cubik" is my desktop and "laptop" is my laptop (funny that!):



To select which PC you are controlling at any one time synergy lets you simply move your mouse to the edge of the screen in the direction of the other PC, your mouse pointer will then disappear from one screen and magically appear on the other! To enable this you need to set up the "Links".

Links are the descriptions of the layout of your machines, i.e. is your laptop on the left or right of your main screen - in my case I have my laptop to the left:



Once you have made your choices via the drop-down lists click the little "+" sign to add that link. You now need to set up the opposite of that link so that you can get control back to your desktop again:



Click "Ok" and then click "Start" to start the server (your desktop PC) listening for connections. Over on the other PC (your laptop) start Synergy and click "Use another computer's shared keyboard and mouse (client)" and type the name of the other PC in the box, in my case I enter "cubik" as that's the name of my desktop PC. Now click "Start" to attempt to connect to your server.

If everything is ok then you should be able to move your mouse over to the left-hand side of your desktop PC's screen and the mouse (and therefore the keyboard) will hop over onto your laptop's screen so that you can control that instead. If you want to return control back to your desktop PC then simply move the mouse pointer to the right-hand side of the laptop screen and the mouse will jump back again!

If you get stuck for any reason on the wrong PC, just right-click on the Synergy icon in the system tray and choose "Quit".

One added bonus is that your copy buffer transfers over to the other PC, try it out - select some text on one screen, then to copy it, mouse over onto the other screen and to paste it. This even works when switching from one screen running Windows to another screen running Linux!

Technorati Tags: Synergy, KVM, VNC, Andrew Beacock

How to restore "Escape closes windows" shortcut in Pidgin (using Ubuntu Linux)

I use Gaim/Pidgin as my IM client as it can connect to all other IM services (AIM, MSN, Yahoo, GTalk, etc.) but keeps them all within one tidy app. My favourite feature is that when you want to close a conversation window you simply just hit 'Esc' and it's gone. A collegue asked recently why his Escape shortcut was not working any more after an upgrade to Ubuntu Gutsy.

It appears from Pidgin's release notes that this 'feature' was changed in version 2.0.0 (5/3/2007):

Removed "Escape closes windows;" default key binding is now Ctrl-W

This hadn't happened to my version (I'm also running Gutsy) but I had to get to the bottom of it. After a little searching in .purple (the home of Pidgin's user configuration) within my home directory I was able to find the following uncommented in my accels file:

(gtk_accel_path "
/Conversation/Close" "Escape")

In my collegue's ~/.purple/accels file he just had to change <Control>w to be Escape and then it all started working again!

Technorati Tags: Gaim, Pidgin, Andrew Beacock

How to use logrotate with Apache's Tomcat (and any other Java process)

Logrotate is an excellent utility for managing log files that can start grow into an unmanageble size, Tomcat's catalina.out is one such file.

Most of the solutions (scroll down to the 'Rotate the logs' section) for managing catalina.out, or other log files generated by Tomcat or any Java process for that matter, don't mention the copytruncate option:

Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one, It can be used when some program can not be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log file stays in place.

This basically means that you don't have to configure your logrotate script to restart Tomcat just so you can tidy up your log files - it trades off uptime against completeness of log records.

I found out about this from the Mission Data Blog in their post titled 'managing disk space with logrotate'. Is definitely worth taking a good look through the man page for logrotate to see if there are any other gems you can take advantage of for your setup.

Technorati Tags: logrotate, Tomcat, catalina.out, Andrew Beacock

An update to my Thunderbird TNEF script for opening winmail.dat

Back in July I blogged about Opening winmail.dat (TNEF) files in Thunderbird (on Ubuntu). Well a colleague at work has 'refined' my script so that rather than it create a folder on your desktop it simply opens Nautilus to display the contents of the attachment.

This is the new version of the script:

#!/bin/bash

LOCATION=/tmp/winmail_$$.dat

mkdir $LOCATION
/usr/bin/tnef -C $LOCATION --save-body -f $1

nautilus $LOCATION

The interesting part of this script for me is the $$ - this special script variable is the process ID (PID) of the currently running script, so it makes a temporary file called winmail_<current process id>.dat. Other than that it's pretty similar to my original script apart from opening Nautilus at the end.

Now it's even easier to open and extract files from winmail.dat files!

I found out the reason for the $$ from here.

Technorati Tags: winmail.dat, TNEF, Thunderbird, Ubuntu, Andrew Beacock

Poor performance after upgrading Ubuntu to 7.10 'Gutsy Gibbon'

I recently upgraded my desktop to Gutsy Gibbon using aptitude rather than the graphical Update Manager and noticed that the performance of the machine had dropped significantly.

Running htop highlighted the problem, evms and udevd were fighting for disk I/O, bringing the system to it's knees. After a quick search I found that most users can safely remove EVMS as it's normally used for RAID and other complex drive setups:

The Enterprise Volume Management System (EVMS) is a project that attempts to provide a single, unified system for managing storage and filesystems. Rather than managing individual disks, partitions and filesystems; LVM Physical Volumes, Volume Groups and Logical Volumes; RAID arrays (both software and hardware) etc. yourself, using EVMS you can address them in one consistent way.

The support for the EVMS package has been dropped by the Ubuntu team but if you didn't use the Update Manager to do the migration to Gutsy it won't have been removed.

To remove EVMS manually (please make sure you aren't currently using it as it could be dangerous to your data if you are and then you remove it) simply run:

sudo aptitude remove evms

This will remove the EVMS package and then rebuilt the kernel image so please be patient

You are then only a reboot away from a faster running system!

Technorati Tags: Upgrade, Ubuntu, Gutsy Gibbon, evms, udevd, Andrew Beacock

One suggestion re: Forbidden access via Apache Proxies

If you are getting the following error when configuring Apache 2.2's proxy support (mod_proxy):

HTTP error code 403 Forbidden

then you might want to take a look inside the error.log for your site. If you see an error message like this:

[warn] proxy: No protocol handler was valid for the URL /SomeUrl.
If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.

Then you might want to check that you have more than just mod_proxy enabled. Mod_proxy is more of a virtual module relying on other modules to perform the proxying work depending on the type required. A common mistake is to have enabled mod_proxy but not enabled mod_proxy_http.

With Ubuntu or Debian you can enable them with the following two commands:

a2enmod proxy
a2enmod proxy_http

Otherwise you will have to make manual symbolic links to them yourself.

Technorati Tags: Apache, mod_proxy, proxy, HTTP, 403, Forbidden, Andrew Beacock

A hidden gotcha regarding bash linked to sh (shell) in Ubuntu (Linux)

I was having some problems recently regarding settings inside a user's .bashrc file. When the user logged in their settings were not being picked up. I checked in /etc/passwd and saw that the user's shell was sh which was a symbolic link for bash. I assumed that because the shell was linked to bash it would have bash's behaviour - picking up .bash_profile, .bashrc, etc.

A search of bash's man page proved me wrong:

If bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible.

I.e. if it's called sh it works like sh!

Technorati Tags: bash, sh, Ubuntu, Linux, Andrew Beacock

htop - a great alternative to top for Ubuntu Linux

I colleague recommended that I install htop the other day. It's an alternative to top which offers colour (ooh), a tree sort view (aah), and the ability to kill off processes within htop itself (very cool).

To install it, simply run:

sudo aptitude install htop

And to run it, simply type:

htop

This is what it looks like running (with the tree view selected):



And here are some of the help options:



Technorati Tags: , htop, top, Ubuntu, Linux, Andrew Beacock

Upgrading to Ubuntu 7.10 (Gutsy Gibbon) causes snmpd errors

When Gutsy Gibbon was released back in October I was quick to upgrade my home Ubuntu server to this new release. The upgrade went smoothly apart from one failure - a strange error message when upgrading the SNMP support:

Starting network management services:invoke-rc.d: initscript snmpd, action "start" failed.
    dpkg: error processing snmpd (--configure):
     subprocess post-installation script returned error exit status 1

I didn't understand the error message and googling for it at the time brought up no solutions. I checked and found that snmpd was running successfully and all appeared fine. I recently applied some more updates and received the same error message so I investigated it again.

This time I found the reason, the SNMP package upgrade scripts don't stop snmpd before they perform the upgrade, so you need to stop snmpd before you update.

This is how I did it:

$ sudo /etc/init.d/snmpd stop
Stopping network management services: snmpd snmptrapd.
$ sudo dpkg --configure -a
Setting up snmpd (5.3.1-6ubuntu2) ...
Starting network management services: snmpd.

That's it, all updated and no errors, I do hope that the installation scripts get fixed for this soon...

Technorati Tags: Ubuntu, Gutsy Gibbon, Linux, Upgrade, SNMP, Andrew Beacock

How to truncate a log file using Ubuntu Linux

Have you ever wanted to truncate a log file that was being written to by a running process without stopping the process to do so?

If so then all you need is the following to clear out the active log file:

cat /dev/null > some_log_file.log

There are some other great tips over on the Bash Cures Cancer blog.

Technorati Tags: Truncate, Log, File, Ubuntu, Linux, Andrew Beacock

How to add system-wide environment variables to Ubuntu & Debian Linux

Following on from my last post on how to add shared libraries to Linux's system library path I wanted to share a related tip on system-wide environment variables.

Common reasons for doing this are setting your locale, setting the location of a Java installation or specifying the character set that your database is using (if you use Oracle then that would be the NLS_LANG environment variable). Debian actually has a policy against system-wide environment variables stating "a program must not depend on environment variables to get reasonable defaults" and suggests that all programs should have wrapper shell scripts to set the relevant variables.

Ignoring this sound advice I tried adding them to /etc/profile but this didn't work when ssh was used from a remote machine to execute some commands (I was using Capistrano).

I couldn't find a complete way to do it (it wasn't picked up by any start scripts in /etc/init.d) but I achieved the most success by placing my variables in /etc/environment:

LANG=en_GB
JAVA_HOME=/usr/local/src/java

Just remember that you don't need to export anything!

Technorati Tags: Environment, Variables, Ubuntu, Debian, Linux, Andrew Beacock

How to add shared libraries to Linux's system library path

Sometimes in Linux when you install a new software package the instructions tell you to add a directory of shared libraries to your $LD_LIBRARY_PATH environment variable in your .bashrc.

You may have noticed that if you then create a shortcut icon on your desktop to this application it won't start because it can't find the libraries.
A typical solution is to write a wrapper shell script to set the LD_LIBRARY_PATH and then call that application.

Well, I've discovered how to add them to your system's library path allowing all environments to access them. Note: There are differences between Debian and Ubuntu (the two flavours of Linux that I'm familiar with).

Ubuntu
Create a new file in /etc/ld.so.conf.d/ called .conf

Edit the file and add a line per directory of shared libraries (*.so files), it will look something like:

/usr/lib/APPLICATION/lib

Reload the list of system-wide library paths:

sudo ldconfig

Debian
Edit /etc/ld.so.conf

Add a line per directory of shared libraries (*.so files) to the bottom of the file, it will look something like:

/usr/X11R6/lib
/usr/lib/APPLICATION/lib

Reload the list of system-wide library paths:

ldconfig

If you run your new application it should now work fine without you having to set any LD_LIBRARY_PATH environment variables.
If you still have problems you can obtain a list of the libraries that are on the system path by re-running the ldconfig command in verbose mode:

ldconfig -v

Technorati Tags: LD_LIBRARY_PATH, Ubuntu, Debian, Linux, Andrew Beacock

A Capistrano recipe for restarting Apache 2 (on Linux)

Capistrano is the rather excellent tool for automating Rails application deployment. But it can do a lot more than just uploading Rails applications to live webservers. It's scripts are basically Rake scripts and so have the full power of Ruby behind then so you can pretty much write code to do anything that you want.

I wanted a way to reload the configuration for my Apache2 web server running on my home Ubuntu server. I found a post on Mongrel and Capistrano 2.0 by John Ward which showed a very nifty way to create Capistrano tasks for variants of the same base command.

My adapted version for controlling Apache is:

namespace :apache do
  [:stop, :start, :restart, :reload].each do |action|
    desc "#{action.to_s.capitalize} Apache"
    task action, :roles => :web do
      invoke_command "/etc/init.d/apache2 #{action.to_s}", :via => run_method
    end
  end
end

Add this code to your config/deploy.rb. This will add four new tasks to Capistrano which you can use to restart or reload Apache:

cap apache:stop
cap apache:start
cap apache:restart
cap apache:reload

Technorati Tags: Capistrano, Apache, Ubuntu, Ruby, Rails, Rake, John Ward, Andrew Beacock

How to "Show Desktop" in Ubuntu Linux

Like Windows, Ubuntu (and pretty much every other Linux) has a "Show Desktop" icon on it's toolbar that minimizes all the running applications to the taskbar allowing full view of the desktop.

I discovered it last week by accident whilst I was attempting to pull off a difficult triple-press-perfect-timing-required IDE 'shortcut' ;)

The keyboard shortcut for "Show Desktop" is:

Ctrl-Alt-D

I also found a great linux desktop hotkeys article written by the Calgary Linux Users Group.

Technorati Tags: Show Desktop, Ubuntu, Linux, CLUG, Andrew Beacock

Having cron troubles with percent (%) signs in crontabs?

I recently wrote a cronjob which would save off my Ubuntu home server's current packages list every night to a dated file so that I have a history of the changes that I've made to my system. The entry in my crontab looked something like this:

# package lists (5am)
0 5 * * * dpkg -l > /var/log/packages/`date +%F`.txt

After a couple of days I decided to take a look at the files generated so far but to my disappointment I found none. After a quick search for dpkg in /var/log I found my problem reported in syslog:

... /USR/SBIN/CRON[28508]: ... CMD (dpkg -l > /var/log/packages/`date +)

I noticed that the command recorded to the logs was up to '`date +' - the next character - the percent sign (%) - was causing some kind of issue. A quick search threw up the Wikipedia page for Crontab which mentioned:

Another common mistake is to use unescaped % in your command; you have to escape them

All I had to do was change % to be \%. My working crontab entry now looks like this:

# package lists (5am)
0 5 * * * dpkg -l > /var/log/packages/`date +\%F`.txt

The funny thing was that most of the 'proper' Linux documentation for cron and crontab do not mention anything about percent signs (%) - I'm finding that I'm going to Wikipedia for answers more and more these days...

Technorati Tags: Cron, Crontab, Percent, Linux, Andrew Beacock

How to open Firefox Google searches in a new tab

I'm always using the built-in Google search box in Firefox to do my research. I used to click the 'New Tab' button (or CTRL-T) to open a fresh tab, then type in the box and hit return to open the results in my newly created tab.

I then found that it will open in a new tab automatically if you use Ctrl-Return rather than just Return when searching. That was a great improvement but if I forgot to hold down Ctrl I would wipe out my current tab with the Google search results.

Well I've found the ultimate solution from an 'Hacking Firefox' article on Computer World found via the excellent LifeHacker blog. It's an about:config value that makes the search results open in a new tab (in the same way that Ctrl-Return did).

To enable it, enter about:config in the location bar. Enter browser.search.openintab in the "Filter" text box and then double click the listed browser.search.openintab to change it's value from false to true:



Oh, and if you want the results to open in the current tab rather than a new tab for a particular search then you simply use Ctrl-Return instead of just Return! Joy!

Technorati Tags: Tab, Search, Configuration, Firefox, LifeHacker, Andrew Beacock

Getting Tomcat contexts to work in IntelliJ IDEA (and stop it starting Tomcat twice!)

If you run Tomcat within IDEA then you might have noticed a couple of things:

1. Your application gets started, undeployed and then started again, increasing the start up time of Tomcat and reinitialising any load-on-startup servlets for a second time.


2. Your carefully crafted <context> stanzas don't work as IDEA hasn't picked then up from either server.xml or META-INF/context.xml.

When you start Tomcat inside IDEA it copies the contents of your Tomcat conf directory to an IntelliJ system area (on Linux it's hidden in your home directory) and then runs Tomcat looking at those configuration files instead of the normal ones. In doing this it seems to ignore any context.xml files, any context stanzas of the server.xml and messes up any context files you might have in conf/Catalina/.

To get around this big mess and just use your Tomcat as if you ran it outside of IDEA you simply add a CATALINA_BASE environment variable to the configuration of Tomcat (within IDEA) pointing to your Tomcat directory:



Now when you start Tomcat within IDEA you should notice that your contexts are working correctly!

Technorati Tags: IntelliJ, IDEA, Tomcat, Context, Andrew Beacock

How to get the Home & End keys working in remote Linux shells & terminals

Have you ever SSH'ed onto a remote linux server only to find that your Home and End keys don't take you to the start and end of the line and instead give you a rather unhelpful "~"?

One solution is to use CTRL-A instead of Home and CTRL-E instead of End but I wanted to solve it properly.

The linux command line is managed by readline. The configuration file for controlling key bindings (amongst other things) is /etc/inputrc. In Ubuntu this file contains a number of commented out sections relating to the Home and End keys. To find out which keycodes are being sent to readline when Home & End are pressed I used od - a stdin dump utility which can display ASCII characters or backslash escapes:

$ od -c

<Home key pressed>    ^[[7~
<End key pressed>    ^[[8~

The key bits here are the 7 and 8, these need to be entered at the very end of /etc/inputrc as follows:

"\e[7~": beginning-of-line
"\e[8~": end-of-line

After you've made these changes logout and back in again and now your Home & End keys should be working perfectly!

Technorati Tags: Home, End, Remote, SSH, Linux, Terminal, Andrew Beacock

Opening winmail.dat (TNEF) files in Thunderbird (on Ubuntu)

 
UPDATE: I have an improved version of the script available here.

I'm not going to repeat the many, many websites taking (and complaining) about Microsoft's proprietary e-mail attachment format called TNEF.

I'm assuming that if you are reading this then you have found that these fixes are not working for you (or not possible to enforce). I'll also assume that the LookOut Thunderbird Add-on by Aron Rubin is also not working correctly for you (this was my experience on Ubuntu Edgy Eft).

The best solution I could come up with was getting Thunderbird to run a script to unpack the winmail.dat extension into a folder on your Ubuntu desktop.

It relies on the tnef command-line program, so make sure that is installed first (it's bundled with Ubuntu):

sudo aptitude install tnef

Below is my little script, save it in a file called 'tnef.sh' somewhere and make sure it's executable (chmod +x tnef.sh) - or just download it here.

#!/bin/bash

LOCATION=~/Desktop/winmail.dat

mkdir $LOCATION
/usr/bin/tnef -C $LOCATION --save-body -f $1

Now find an email in Thunderbird with a winmail.dat attachment. Double click it and select to open it with the newly saved tnef.sh file:



Look on your desktop - there should be a 'winmail.dat' directory with the full contents of the attachment.

Double-clicking on any future winmail.dat file will result in the contents of the attachment to also be added to that directory.

Technorati Tags: winmail.dat, TNEF, Thunderbird, Ubuntu, Andrew Beacock