Things I've found useful

This morning a discussion with a friend about various shells lead me to think it would be nice if my bash shell could tab complete hostnames from .ssh/known_hosts when I type ‘ssh <tab>’. I soon found this blog post which nicely documents how to do it. I made a directory in $HOME called .bash.completion and then added this to my .profile, which loops round any files in there, sourcing them individually:

if [ -d ${HOME}/.bash.completion ]; then
 for file in ${HOME}/.bash.completion/* ; do
   source $file
 done
fi

All sorted. However, it wasn’t long before I discovered that ‘ssh user@<tab>’ doesnt work, I tend to use this quite a lot so wanted to see if I could fix up the bash function to support that use case. Bit of hacking around and I’ve got it working, the replacement ssh-completion file is shown below:

# Add bash completion for ssh: it tries to complete the host to which you
# want to connect from the list of the ones contained in ~/.ssh/known_hosts
__ssh_known_hosts() {
    if [[ -f ~/.ssh/known_hosts ]]; then
        cut -d " " -f1 ~/.ssh/known_hosts | cut -d "," -f1
    fi
}
_ssh() {
    local cur known_hosts
    COMPREPLY=()
    cur="${COMP_WORDS[COMP_CWORD]}"
    known_hosts="$(__ssh_known_hosts)"
    if [[ ! ${cur} == -* ]] ; then
        if [[ ${cur} == *@* ]] ; then
            COMPREPLY=( $(compgen -W "${known_hosts}" -P ${cur/@*/}@ -- ${cur/*@/}) )
        else
            COMPREPLY=( $(compgen -W "${known_hosts}" -- ${cur}) )
        fi
    fi
    return 0
}
complete -o bashdefault -o default -o nospace -F _ssh ssh 2>/dev/null \
    || complete -o default -o nospace -F _ssh ssh

Happy.

Related posts:

1. Reading text files in bash
2. Setting a variable in bash if it is not defined
3. BASH assigning command output into an array

A new project I’ve just deployed onto live uses the fast and lightweight beanstalkd work queue.  As part of putting it into live I wanted to get at least some basic monitoring on the beanstalkd daemon.  All my servers run monit for keeping an eye on processes I care about, so that seemed a good place to start.

I had a bit of a look around the interwebs but couldn’t find any examples, so I set about putting something together myself. As it turns out beanstalkd has a nice simple text protocol, detailed in protocol.txt which is included in the source, and monit has the ability to send and expect arbitrary strings to a given port.  For starters I’ve set it up to issue a stats command and check for the expected response which is:

OK <bytes>\r\n

Adding this to my monit config file for beanstalkd gives me:

check process beanstalkd with pidfile /var/run/beanstalkd.pid
 start "/etc/init.d/beanstalkd start"
 stop "/etc/init.d/beanstalkd stop"
 if failed port 11300
   send "stats\r\n"
   expect "OK [0-9]{1,}\r\n"
 then alert

This is a pretty simple check but at least will enable monit to alert if beanstalkd isn’t listening or is returning garbage.

I think in the long term I’d like nagios to alert on  the current-jobs-delayed and possibly the current-waiting variables reported by the stats-tube command for a given tube, but that is for another day!

Another thought is possibly to have a cacti graph of the total-jobs field, it looks like a always increasing counter, so should be pretty trivial to hook up to cacti.

Related posts:

1. CDROM sub folders empty when they shouldn't be
2. hubs/switches/routers
3. Be 24 meg Thompson Speedtouch snmp enable

I seem to have a inherent disklike of Debian and the feeling appears to be mutual. It never makes my life easy. Just this morning I needed to install it on a Dell R210 rather than our usual Centos builds. The server is 15 odd miles away, so I took my standard route of PXE installing. After downloading the netboot.tar.gz and dropping the right files in place on my netboot server, I booted the R210 and began the install. Only a couple of screens in I was presented with this most unhelpful message. So I have to drive 30 miles to plug a usb stick into this machine to continue? That isn’t acceptable imho.

In my case it was the non free firmware for the Broadcom ethernet cards in the machine, I needed this package.

Turns out there is a fix. You just need to download the missing .deb, cpio it and cat it into the end of the initrd:

% mkdir /tmp/firmware
% cd /tmp/firmware
% wget http://ftp.uk.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-bnx2_0.28+squeeze1_all.deb
% cd /tmp
% find firmware | cpio -o | gzip -c > firmware.cpio.gz

Now cat the resultant firmware.cpio.gz onto the end of the existing initrd.gz, which for me was in /tftpboot/debian-installer/amd64/initrd.gz. So I ran this:

% cat /tmp/firmware.cpio.gz >> /tfpboot/debian-installer/amd64/initrd.gz

That will overwrite the initrd.gz without warning, so might want to take a backup of it first.

Related posts:

1. Hacking the SUSE boot.iso
2. Linux PPTP server
3. Vmware Server on Fedora Core 6

I quite often used to find myself wanting to print from my iPad, so when Apple announced AirPrint I thought things were looking good. According to the press release you would need either a HP printer or you could print to a shared printer on an existing mac. Sorted, I’ve got the latter of those, things were looking rosy.

Things took a turn for the worse when I read that they had pulled support for printing via shared printers and were only going to allow AirPrint to certain (at the moment only HP) printers.  I wanted the feature, but not enough to replace my trusty Lexmark.

Luckily there are people like this guy on the internet who took the time to figure out how it works and have since published a nice simple guide about how to setup a Linux box as a AirPrint server. Mainly thanks to Cups and some Avahi magic.

Sorted! And I’ve used it sufficiently to think spending an hour or so setting it up was worth while.

Related posts:

1. Linux PPTP server
2. Linux authentication from Active Directory
3. RedHat Enterprise Linux 5 Microsoft Active Directory Authentication

I use a Mac as my desktop and I’ve never successfully gotten the VMware Server console thing in the web interface to work. Which means if I need console access to a VM I have to boot a copy of windows somewhere and use IE in that.  This little trick gets me out of having to to that.

Add the following lines to the VMX config file for each VM you want access to, then you can just connect in with any VNC client and get the console. Use the host/ip address of the vmware server itself when you connect. I halted each of my VMs while editing that file because I wan’t to make sure VMware wasn’t going to write to the file as well.

RemoteDisplay.vnc.enabled = "TRUE"
RemoteDisplay.vnc.password = "password"
RemoteDisplay.vnc.port = "port"

If you want more than one VM to use VNC you have to use different port numbers.

Related posts:

1. Vmware Server on Fedora Core 6
2. Enabling vmware-tools clock sync from inside the guest
3. VMware any-any patch

So after a few hours getting Linux authenticating happily from Active Directory, I turned my attention to getting it all working with encryption.

Initially I had tried a telnet to port 636 (the LDAPS port) which failed, so it didnt look like my AD box was talking LDAPS. Somewhere to start I guess!

Enabling LDAPS in AD

I followed this article to the letter, which lead me on a slightly convoluted path because I ended up making another 2003R2 server to act as a Certificate Server, before finally getting it all working. I called upon the excellent ldap.exe again which is handy to prove its all working using SSL.

LDAP command line tools

Before poking around with actually making authentication work, I wanted to make the useful command line tools, like ldapsearch work with an encrypted connection. This turned out to be remarkably simple. You just have to change ldap:// to ldaps:// and if you are working with self-signed certificates (as I am) you need to add TLS_REQCERT never to /etc/openldap.conf, meaning my file now looked like this:


URI ldap://adserver.112.riviera.org.uk
BASE dc=112,dc=riviera,dc=org,dc=uk
TLS_REQCERT never


With that done, running the following command ought to return the same set of data as it did before you enabled encryption, but this time if you use Wireshark or something similar you should see no unencrypted LDAP traffic.


ldapsearch -x -LLL -E pr=200/noprompt -D "bindaccount@112.riviera.org.uk" -w $PASSWORD -s sub "(cn=*)" cn mail sn


nss_ldap config

Lastly you’ll actually want to configure the /etc/ldap.conf file which the nss_ldap packages use, this is also very simple. First, you have a choice to make, do you want pre LDAPv3 style SSL with LDAP over the top or to you want to use LDAPv3 only TLS. The library supports both, and they both work fine against Windows 2003R2 AD. The ssl off entry in the config file needs to be changed to one of:

• ssl on – Use pre LDAPv3 SSL
• ssl start_tls – Use LDAPv3 and greater TLS

As I said, either work with AD, so which you choose is up to you, or any local security polcies

With that changed, I also needed to add in a second line, to disable rootCA certificate checking, because of my self-signed cert. With that change made as well, my final /etc/ldap.conf file looked like:


host 192.168.254.14
base dc=112,dc=riviera,dc=org,dc=uk
binddn bindaccount@112.riviera.org.uk
bindpw $BINDPASSWORD
scope sub

ssl start_tls
tls_checkpeer no

timelimit 10
bind_timelimit 10
idle_timelimit 3600
nss_initgroups_ignoreusers root,ldap,named,avahi,haldaemon,dbus,radvd,tomcat,radiusd,news,mailman,nscd,gdm
nss_base_passwd dc=112,dc=riviera,dc=org,dc=uk?sub
nss_base_shadow dc=112,dc=riviera,dc=org,dc=uk?sub
nss_base_group dc=112,dc=riviera,dc=org,dc=uk?sub?&(objectCategory=group)(gidnumber=*)
nss_map_objectclass posixAccount user
nss_map_objectclass shadowAccount user
nss_map_objectclass posixGroup group
nss_map_attribute gecos cn
nss_map_attribute homeDirectory unixHomeDirectory
nss_map_attribute uniqueMember member


With this file changed, all your authentication traffic should now be running over an encrypted link. With a network sniffer, you can clearly see the differences between having ssl on and ssl start_tls set in the config file.

Hope that helps somebody, its the second time I’ve done this and last time I didnt write it down, hopefully next time I wont spend so long going over old ground!

Today I set out to get RHEL 5.x (Specifically 5.5) to authenticate from a Windows 2003 R2 Active Directory. I used plenty of web pages, none of which were 100% correct for my setup, so I thought I’d document exactly what I did here for my own future reference, if anybody else finds it useful, so much the better. To start with, thanks to the following pages, between them, they got me about 80% of the way there:

• Scott Lowe has a useful howto here. It is version 4 currently, click around his site to ensure he hasnt updated it before you use it as a reference.
• The second resource I used is here.

Required RPMs

You need a few packages installed, some of which are likely to be installed already, some not. The ones I have, with their versions are:

• nss_ldap-253-25.el5
• krb5-libs-1.6.1-36.el5_5.6
• openldap-2.3.43-12.el5_5.3
• ntp-4.2.2p1-9.el5.centos.2.1

I also found it useful to have openldap-clients-2.3.43-12.el5_5.3 installed as well, because that gives you ldapsearch, which is handy for debugging and testing things.

Names

In all of the following these names have been used:

• bindaccount – The name of the simple account inside AD for binding
• 112.riviera.org.uk – The DNS name of my domain
• ONEONETWO – The other (windows!?) name of my domain
• adserver.112.riviera.org.uk – the fqdn of my AD
• dc=112,dc=riviera,dc=org,dc=uk – My BaseDN, (I used the handy ldp.exe to find this, from this KB Article)

Active Directory

The blog post from Scott Lowe, explains about adding in the “Server for NIS” stuff into Windows, so I wont go into that here. Suffice to say, in any practical sense, you need it installed. One other thing to note is that the DNS service on the AD machines really likes to know about your new linux client. I created a A record with the associated PTR record for all the new Linux clients which were going to join the domain. Doing this before you start makes life easier later. I added a couple of test users into AD before starting to configure the Linux end too, the settings on ‘UNIX Attributes’ tab are shown below

Linux Setup

Before touching any config files make sure your client has its own IP and FQDN listed in /etc/hosts. Also make sure time is syncronised with AD, a default AD install puts a time server up for you, so you could use that, whatever you use, make sure they are in sync. The files I modified and what they ended up looking like are as follows, in all cases except for nsswitch.conf these are the entire files, so feel free to copy and paste and kill what is already there.

/etc/openldap/ldap.conf

This file configures the openldap clients, not strictly neccerssary, but useful to ensure you can talk ldap to your AD. Once this file is written you should be able to query AD with the following command

ldapsearch -x -LLL -E pr=200/noprompt -D "bindaccount@112.riviera.org.uk" -w $PASSWORD -s sub "(cn=*)" cn mail sn

I’ve put my password in a bash variable, just replace $PASSWORD with yours, if this fails then try the full command, which wont be using any defaults from /etc/openldap/ldap.conf:

ldapsearch -x -LLL -E pr=200/noprompt -h adserver.112.riviera.org.uk -D "bindaccount@112.riviera.org.uk" -w $PASSWORD -b "dc=112,dc=riviera,dc=org,dc=uk" -s sub "(cn=*)" cn mail sn

If that also fails you might want to try telneting to the LDAP port on the AD box, to see if that is open

My ldap.conf only has the following in it:


URI ldap://adserver.112.riviera.org.uk
BASE dc=112,dc=riviera,dc=org,dc=uk


/etc/krb5.conf


[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
default_realm = 112.RIVIERA.ORG.UK
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
forwardable = yes

[realms]
112.RIVIERA.ORG.UK = {
kdc = adserver.112.riviera.org.uk
admin_server = adserver.112.riviera.org.uk
default_domain = 112.riviera.org.uk
}

[domain_realm]
.112.riviera.org.uk = 112.RIVIERA.ORG.UK
112.riviera.org.uk = 112.RIVIERA.ORG.UK

[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}


/etc/ldap.conf


host 192.168.254.14
base dc=112,dc=riviera,dc=org,dc=uk
binddn bindaccount@112.riviera.org.uk
bindpw $BINDPASSWORD
scope sub
ssl no
timelimit 10
bind_timelimit 10
idle_timelimit 3600
nss_initgroups_ignoreusers root,ldap,named,avahi,haldaemon,dbus,radvd,tomcat,radiusd,news,mailman,nscd,gdm
nss_base_passwd dc=112,dc=riviera,dc=org,dc=uk?sub
nss_base_shadow dc=112,dc=riviera,dc=org,dc=uk?sub
nss_base_group dc=112,dc=riviera,dc=org,dc=uk?sub?&(objectCategory=group)(gidnumber=*)
nss_map_objectclass posixAccount user
nss_map_objectclass shadowAccount user
nss_map_objectclass posixGroup group
nss_map_attribute gecos cn
nss_map_attribute homeDirectory unixHomeDirectory
nss_map_attribute uniqueMember member


/etc/pam.d/system-auth


#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth sufficient pam_krb5.so use_first_pass
auth sufficient pam_ldap.so use_first_pass
auth required pam_deny.so

account required pam_unix.so
account sufficient pam_succeed_if.so uid < 500 quiet
account [default=bad success=ok user_unknown=ignore] pam_krb5.so
account [default=bad success=ok user_unknown=ignore] pam_ldap.so
account required pam_permit.so

password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok
password sufficient pam_krb5.so use_authtok
password sufficient pam_ldap.so use_authtok
password required pam_deny.so

session required /lib/security/pam_mkhomedir.so skel=/etc/skel/ umask=0022
session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_krb5.so
session optional pam_ldap.so


/etc/nsswitch.conf

I have not included all of nsswitch.conf here because I only changed the following three lines. Simply add ldap after files for each.


passwd: files ldap
shadow: files ldap
group: files ldap


/etc/samba/smb.conf


workgroup = ONEONETWO
security = ads
realm = 112.riviera.org.uk
use kerberos keytab = true
password server = adserver.112.riviera.org.uk


Finished!

When all that is done and working you should be able to run ‘getent passwd ‘ and have it return your passwd file entry from AD. You should also be able to su to the user, and login via either console or SSH. If the users home directory isnt created, pam should create it for you.

Next on my list is to make all this work with encrypted LDAP.

After nearly 7 weeks without having to suffer Windows in any of its incarnations, I returned to work today and had the pleasure of XP on my horribly clunky Thinkpad. Very shortly after booting it I was presented with the following error message.

What a wonderful dialogue box and what a ‘Useful tip’. A Google search suggests it is something to do with the Windows Bluetooth stack. Oh I dont fucking care you stupid program, I just wanted to message somebody, you worked 7 weeks ago, why are you arsing around now.

As an aside, I notice the 8.3 filename, which still at some level seems to exist in Windows XP. How quaint of it!

[root@sn-b07-a ~]# iscsiadm -m session
tcp: [9] 172.16.4.201:3260,1 iqn.1984-05.com.dell:powervault.6001c23000cd3f8300000000485684e3
tcp: [10] 172.16.4.202:3260,2 iqn.1984-05.com.dell:powervault.6001c23000cd3f8300000000485684e3
tcp: [11] 172.16.5.202:3260,2 iqn.1984-05.com.dell:powervault.6001c23000cd3f8300000000485684e3
tcp: [12] 172.16.5.201:3260,1 iqn.1984-05.com.dell:powervault.6001c23000cd3f8300000000485684e3
[root@sn-b07-a ~]# iscsiadm -m session -r 9 --rescan

From here.

Earlier today I was writing up a design in Microsoft Word for Mac 2011. When the inbuilt gamma checker highlighted the following word:

OK, I’ll give it that I, spreadsheet is probably correct, so I let it correct it and continued with my writing. Then a couple of seconds later I notice it has underlined the word again, but this time in red. This is the suggestion from the spelling checker when I right clicked on the word…

I’m at this point feeling stuck in some computer ‘helping’ me write hell!