Migrating VMs from Redhat to Ubuntu

Recently I have installed Kubuntu onto my PC and had some Virtual Machines that I had used on a Centos machine under Virt-Manager/KVM that I needed to use on the Kubuntu system. I thought this would be a simple enough process but ran into some problems.

When trying to start up VMs in Virt-Manager on Ubuntu I got the following message:-

 error: Failed to start domain blackhat.example.net supported
 machines are 
...

To fix this I initially created a new VM to see what types it supported it does support Redhat VMs. Once this was created I viewed the XML for the image with the following command:-

 virsh dumpxml test

test being the name of the new VM I had created. The following line was relevant:-

 <type arch='x86_64' machine='pc-1.0'>hvm</type>  

In my other VM I saw the following:-

 <type arch='x86_64' machine='rhel6.2.0'>hvm</type>  

So the problem is machine='rhel6.2.0' to change this run the following command:-

 virsh edit blackhat.example.net  

Then change 'rhel6.2.0' to 'pc-1.0' it should then boot

ssh in a loop

Using ssh in a loop will break the loop to fix this use the -n flag, i.e:-

while read server
do
   echo $server
   ssh -n $server "ls -l /etc/rc.modules"
   echo
done < servers

Show vendor when searching through installed RPMs

This is done using the %{VENDOR} tag. For example:-

rpm -qa --qf '%{NAME}-%{VERSION}-%{RELEASE} %{VENDOR}\n'

Highlight all occurences of selected word in vi

To do this:-

:set hlsearch

To undo this:-

:nohlsearch

Show line numbers in vi

:set number

Rebuild RPM DB

This may need to be done in instances where simple commands such as rpm -qa hang. Steps:-

1. Kill all rpm and yum processes
2. yum clean all
3. rm -f /var/lib/rpm/__db*
4. rpm -v --rebuilddb
5. yum makecache

Mac Tips and Tricks

Recently I have been using a Macbook Pro for work (OS X 10.8.3), it's good but I have come across a few things that I am not used to and have discovered ways to fix them.  

Using Mac with regular UK keyboard
There are several problems:-
- Hash key in the wrong place
- @ key in the wrong place
- " key in the wrong place
- | symbol in the wrong place
- ~ key in the wrong place
- Home and End keys don't work
- Copy and paste are apple + c and apple + v

With regards to the standard keys there are several keyboard layouts you can download and use, the best of which I can find is the British (PC 105 alt) downloaded from the following location:-

http://liyang.hu/osx-british.xhtml

With regards to home and end keys see the following:-

http://lifehacker.com/225873/mac-switchers-tip--remap-the-home-and-end-keys

However the home and end keys still won't work in firefox! For this install the KeyFixer add-on:-

https://addons.mozilla.org/en-US/firefox/addon/keyfixer/?src=search

With regards to copy and paste keys, you can go to System Preferences > Keyboard > Keyboard Shortcuts and set Application Keyboard Shortcuts > All Applications > Add Shortcut

- Copy ^C
- Undo ^Z
- Cut ^X
- Paste ^V  

Scroll doesn't work the same way

This is an easy one to change. Go to System Preferences > Trackpad > Scroll and Zoom > Scroll Direction: natural - untick this.  

Useful Software to install

iTerm2 - pretty similar to terminator in linux (which can also be installed for Mac):-

http://www.iterm2.com/#/section/home

csshX - cluster ssh for Mac:-

http://code.google.com/p/csshx/

Tips and Tricks

Make ps more readable and put on separate lines

 ps -ef | grep java | tr " " "\n"  

Show hidden files with du

 du -sk .[!.]* *| sort -n  

Show just hidden files with du

 du -sh .[a-zA-Z]*  

Trim last character in vi

 :%s/.$//g  

With sed

 sed '$s/.$//'  

Awk, print all on one line with a comma seperating

 awk '{ printf "%s,", $1 }'  

Find command with regex

 find . -name \*[Ii]nstall\*[Ll]og\*  

Remove all blank lines with awk

 awk '/./' abc.txt  

Show octal permissions when doing an ls

for file in `ls`; do echo -n `stat -c %a $file`; echo -n " "; ls -ltd $file; done  

Using awk with a field seperator

 awk 'BEGIN { FS = "," } ; { print $1 }'  

Remove HTML tags from a file

 sed "s/<[^>]\+>//g" file  

Awk print from third column to the end

 awk '{ print substr($0, index($0,$3)) }'  

Passwordless ssh with Puppet

This can be achieved with the ssh_authorized_key resource type. Before doing this in puppet you need to generate a key on the host you want to ssh from. From within the user account you want the ssh access to occur from:-

 ssh-keygen  

Take a copy of the key text from /home/username/.ssh/authorized_keys. This should just be the key part not the ssh type or user and hostname. Also take a copy of the type and the user and hostname part.

Within puppet do the following:-

 ssh_authorized_key {  
  "andy@ahibbert":  
   ensure => present,  
   user  => andy,  
   type  => "ssh-rsa",  
   key  => "AAAAB....."  
 }  

In the above the details from the authorized_keys file are found:-

• andy@ahibbert - user and hostname I'm ssh'ing from
• type - the ssh type I am using
• key - the public ssh key

 This file should be saved with a .pp extension and then applied as follows:-

 puppet apply auto_ssh.pp  

You should know be able to ssh without a password

Get a list of files that exist on a website via curl and strip out HTML code

The following can be used to display a list of .csv.gz files that exist on a website and strips out all HTML code:-

 curl --silent http://www.theurl.com/thefiles/ | egrep -o "<a href=[^>]*>*.csv.gz"
 | sed 's/<a href=\"\([^"]*\).*/\1/g'  

The --silent flag in curl supresses the progress information and any error messages

Ruby - Check if a port is open

This requires the socket class so the following needs to be included at the top of the program:-

 require 'socket'  

The code here below can be used to see if linux is listening on a particular port.

 def port_open?(ip, port, timeout)  
  start_time = Time.now  
  current_time = start_time  
  while (current_time - start_time) <= timeout  
   begin  
    TCPSocket.new(ip, port)  
    return true  
   rescue Errno::ECONNREFUSED  
    sleep 0.1  
   end  
   current_time = Time.now  
  end  
  return false  
 end  

This can be called with the following:-

 port_open?(Socket.gethostname, 80, 10)  

Creating Directories With Chef

Creating directories using the automation tool Chef is fairly easy to do, unfortunately when creating a directory the correct permissions are only applied to the last directory. E.g. If you created /tmp/foo/bar only bar would have the correct permissions. This to me seems to be a bug and has been logged with Chef, but at the time of writing this; the version I am using which is 10 still has the bug. The bug is logged here:-

https://tickets.opscode.com/browse/CHEF-1327

This could be worked around in multiple ways, a possible solution is to create a method and call this for each level of the directory. For example:-

 def make_dir(dir_path)  
  directory dir_path do  
   owner username  
   group usergroup  
   mode "0755"  
   recursive true  
   action :create  
  end  
 end  

This saves us from calling the chef directory resource for each level of the directory. This could be called like so:-

 make_dir("/tmp/foo")  
 make_dir("/tmp/foo/bar")  

Hopefully this will be fixed in future versions but having the directory resource within a ruby method similar to the above will save lines of code

Source Code Formatting

I am using the following to format source code, linux commands etc:-

http://codeformatter.blogspot.co.uk

Creating a Custom Nagios/Opsview Plugin

This was done using Nagios with Opsview front end version 4.2.3

Scripts are generally stored in the following location:-

 /usr/local/nagios/libexec 

Write the script in your language of choice and ensure it is runable by the nagios user.

The script must exit and create either an OK, WARNING, CRITICAL or UNKNOWN status, with a text based message which can be displayed by Nagios.

Exit Codes

• 0 - This tells nagios that the check has passed and is OK
• 1 - This tells nagios that the check has a problem but is just a WARNING
• 2 - This tells nagios that the check has a problem that is CRITICAL
• 3 - This generally means there has been a problem running the check it will display UNKNOWN in nagios

Bash Script

The script could be done in Bash if so you just need to echo the message out and then exit with the relevant code. For example if the check is okay and you wish to exit do the following:-

 echo "The check has passed"  
 exit 0  

The script could also be written in other languages for example the same above in Python:-

 import sys  
 print 'The check has passed'
 sys.exit(0)

The same above in Ruby:-

 puts "The check has passed"  
 exit 0  

Once the script is thoroughly tested it needs to be linked to nagios. You should be aiming for a script which takes a short amount of time to run. i.e. under 10 seconds, although the timeout can be extended if neccessary. Since this is a local check a config file needs to be put in the nrpe_local directory. The contents of this file could be something like the following:-

 cat /usr/local/nagios/etc/nrpe_local/new_check.sh  

The service will now need restarting.

Opsview Configuration

Go into Settings > Advanced > Service Checks. Add a new check, the two fields you are interested in are plugin which should be check_nrpe and arguments which should be:-

 -H $HOSTADDRESS$ -c new_check  

Note that this can be tested via on the host with the following:-

 /usr/local/nagios/libexec/check_nrpe -H `hostname` -c new_check

If this works it should work okay in Opsview. If there is a timeout problem -t can be appended to the above with a figure e.g. 30, the default is 10.
You then need to associate the check with the relevant hosts. To do this go to Settings > Basic > Hosts, search for the host, double click to amend, go to the monitoring tab, expand the relevant service group and select the new check.

After this you need to update the configuration. To do this go to Settings > Configuration > Apply Changes. Click Reload Configuration, then once this is reloaded and the check run you should see the result.