I got this annoying SVN error today, that I hadn’t come across before. Even after resolving the conflict in the file, highlighted by lots of <<<<<<<, I still couldn’t get my commit to work!
idimmu@boosh:~/work/systems/trunk/dns$ svn ci idimmu.net -m “new funky domain”
svn: Commit failed (details follow):
svn: Aborting commit: ‘/home/rus/work/systems/trunk/dns/idimmu.net’ remains in conflict
rus@boosh:~/work/systems/trunk/dns$ svn ci idimmu.net -m “new funky domain”
Sending idimmu.net
Transmitting file data .
Committed revision 14281.
A lesson in regular svn updateing I guess 🙂
For more SVN advice, I recommend Version Control with Subversion by O’Reilly. It contains everything you need to know when using or managing SVN repositories.
I had a crazy weird bug today whilst running Puppet on one of our CentOS boxes where no packages were being installed. A quick investigation with yum yielded the following scenario!
Id I tried to use yum as root or with sudo it gave the following error ..
[idimmu@server ~]$ sudo yum check-update
rpmdb: Lock table is out of available locker entries
error: db4 error(22) from db->close: Invalid argument
error: cannot open Providename index using db3 - Cannot allocate memory (12)
Repository update is listed more than once in the configuration
Repository base is listed more than once in the configuration
Setting up repositories
https://www.mirrorservice.org/sites/mirror.centos.org/Null/updates/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Not Found
Trying other mirror.
Cannot open/read repomd.xml file for repository: update
failure: repodata/repomd.xml from update: [Errno 256] No more mirrors to try.
Error: failure: repodata/repomd.xml from update: [Errno 256] No more mirrors to try.
But worked fine if I did it as a non root user ..
[idimmu@server ~]$ yum check-update
Repository update is listed more than once in the configuration
Repository base is listed more than once in the configuration
Setting up repositories
Reading repository metadata in from local files
acl.i386 2.2.23-5.4.el4 base
apr-util.i386 0.9.4-22.el4 base
at.i386 3.1.8-82.el4 base
audit.i386 1.0.16-3.el4 base
I did a little bit of investigating with Google as to how yum works and discovered a few things..
https://www.mirrorservice.org/sites/mirror.centos.org/Null/updates/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Not Found
The Null in the URl stood out as an obvious case of “there’s something wrong here”, so looking at the yum.conf file ..
Basically RPM uses a Berkley database and due to too many instances of rpm dieing (the server is an old server) it left locks on the database that were never cleared. In order to fix the problem we have to remove the locks, by unfortunately, deleting the database and recreating it!
Backup the database directory /var/lib/rpm first:
tar cvzf rpmdb-backup.tar.gz /var/lib/rpm
Delete the locked Berkeley databases that rpm uses:
rm /var/lib/rpm/__db.00*
Force rpm to rebuild the databases
rpm --rebuilddb
Now, check the database to make sure it has been rebuilt correctly:
rpm -qa | sort
Problem solved, my database is no longer corrupt and Puppet is installing packages again!
O’Reilly’s Fedora Linux covers everything you need to know about dealing with Redhat systems and is a recommended read for anyone who wants to seriously get in to Redhat administration.
I recently needed to extract a file with the new, popular 7zip compression, which Ubuntu’s File Roller doesn’t support out of the box. Enabling 7zip support in File Roller was as simple as:
I managed to screw up my home machine when upgrading from Dapper to Hardy somehow, so decided to flatten and reinstall Hardy from scratch.
This is all well and good but I have a RAID 5 array on my machine which stores all my important documents and stuff (I really hate drive failure). Ubuntu didn’t suddenly detect my RAID array and figure everything out, which was not entirely unsurprising but was a little scary.
After some light reading, and a few sighs and heart flutters I worked out the magic runes to tell the system I have a RAID array, where it was and how to use it!
root@server:/root# mdadm -A /dev/md0 -v /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde
mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 4.
mdadm: added /dev/sdb to /dev/md0 as 1
mdadm: added /dev/sdc to /dev/md0 as 2
mdadm: added /dev/sdd to /dev/md0 as 3
mdadm: added /dev/sde to /dev/md0 as 4
mdadm: added /dev/sda to /dev/md0 as 0
mdadm: /dev/md0 has been started with 5 drives.
root@server:/root# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sda[0] sde[4] sdd[3] sdc[2] sdb[1]
976793600 blocks level 5, 128k chunk, algorithm 2 [5/5] [UUUUU]
unused devices:
root@server:/root# mdadm -Es
ARRAY /dev/md0 level=raid5 num-devices=5 UUID=19c77b42:c5e86159:c23e7b08:7f8ceabd
We recently had a few power outages at work, some scheduled, some not, and this played havoc with our xen servers.
One of the problems we had was that xend would not start (and thus xendomains would also not start).
Checking /var/log/xen/xend.log gave us the following snippet:
inst = XendNode()
File "/usr/lib/python2.5/site-packages/xen/xend/XendNode.py", line 164, in __init__
saved_pifs = self.state_store.load_state('pif')
File "/usr/lib/python2.5/site-packages/xen/xend/XendStateStore.py", line 104, in
load_state
dom = minidom.parse(xml_path)
File "xml/dom/minidom.py", line 1913, in parse
File "xml/dom/expatbuilder.py", line 924, in parse
File "xml/dom/expatbuilder.py", line 211, in parseFile
ExpatError: no element found: line 1, column 0
[2008-03-10 21:37:40 18122] INFO (__init__:1094) Xend exited with status 1.
A quick google of that error revealed several people that had come across the same problem, but no actual answer!
It looks like xen is having problems parsing an xml file, so some quick mental inspiration, and the find command, yielded /var/lib/xend/state/pif.xml which was a 0 byte file! A comparison to a working server showed that it should (or atleast could) contain this:
A copy and paste later and we had a working xend! However it refused to create any of the xenlets:
root@xen0:/etc/xen# xm create server0.cfg
Using config file "./server0.cfg".
Error: The privileged domain did not balloon!
Despite their being plenty of RAM!
root@xen0:/var/log/xen# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 7928 8 r----- 832.8
root@xen0:/var/log/xen# free
total used free shared buffers cached
Mem: 8119416 393028 7726388 0 11344 58832
-/+ buffers/cache: 322852 7796564
Swap: 15631224 0 15631224
An strace of the process revealed xen did think it had less memory available than it actually had ..
[2008-03-10 21:47:48 18620] DEBUG (__init__:1094) Balloon: 131064 KiB free; 0 to scrub;
need 524288; retries: 20.
As we had a working xend finally we decided to implement a technique we’d learned from working with Windows machines and rebooted the server. This magically fixed the memory issue, it would have been nice to know what actually caused it and if there was a proper fix though.
We have several PHP applications and libraries incorrectly packaged for i386 rather than all, which means they wont install to our nice new shiny AMD64 servers. Unfortunately we don’t have (or at least we can’t find) the original package sources, so we need to ‘frobble’ the packages and change the arch by hand!
Deb files are just an ar package so we must first extract it! (FYI ar is like tar, but for pirates .. 🙁 )
rus@absinthe:~/$ mkdir phplibs
rus@absinthe:~/$ mv phplibs_1.0-1_i386.deb phplibs
rus@absinthe:~/$ cd phplibs
rus@absinthe:~/phplibs$ ar xv phplibs_1.0-1_i386.deb
x - debian-binary
x - control.tar.gz
x - data.tar.gz
rus@absinthe:~/phplibs$ ls
control.tar.gz data.tar.gz phplibs_1.0-1_i386.deb debian-binary
The control.tar.gz is the one that contains all our precious arch information, so we must extract it!
rus@absinthe:~/phplibs$ tar -xzf control.tar.gz
rus@absinthe:~/phplibs$ ls
control control.tar.gz data.tar.gz phplibs_1.0-1_i386.deb debian-binary md5sums
open up the control file and edit in the arch you want (in our case, all)
Package: phplibs
Version: 1.0-2
Section: unknown
Priority: optional
Architecture: all
Installed-Size: 12
Maintainer: rus
Description: shared PHP libraries
Required system php libs
I’m going to increment the package version to -2 as well to signify that it’s just the packaging that has changed.
Now we delete the original control.tar.gz file and rebuild it.
rus@absinthe:~/phplibs$ rm control.tar.gz
rus@absinthe:~/phplibs$ tar -czf control.tar.gz control md5sums
We now just need to use ar again to create our debian package!
rus@absinthe:~/phplibs$ ar -r phplibs_1.0-2_all.deb debian-binary control.tar.gz data.tar.gz
ar: creating phplibs_1.0-2_all.deb
And there we have it, a shiny new package of arch all that can be installed anywhere!
Some of our PERL tools require some CPAN modules that are not part of the standard Ubuntu distribution. It’s obviously possible to install the module using CPAN but I like using deb packages where possible as then you only have one repository to manage. Fortunately with dh-make-perl it is possible to quickly turn any CPAN module in to a debian package!
First make sure dh-make-perl is installed
apt-get install dh-make-perl
Then download the PERL module you wish to package and extract it
Now run dh-make-perl on the directory to create the files needed for the package
rus@absinthe:~$ dh-make-perl Crypt-RC5-2.00/
cat: /etc/mailname: No such file or directory
Use of uninitialized value in concatenation (.) or string at /usr/bin/dh-make-perl line 527.
Found: Crypt::RC5 2.00 (libcrypt-rc5-perl arch=all)
Package does not provide a long description - Please fill it in manually.
Using maintainer: rus
Found changelog: Changes
Found docs: README
Using rules: /usr/share/dh-make-perl/rules.MakeMaker.noxs
Done
This will have created a debian directory
rus@absinthe:~/Crypt-RC5-2.00$ ls
Changes debian Makefile.PL MANIFEST RC5.pm README test.pl
rus@absinthe:~/Crypt-RC5-2.00$ ls debian/
changelog compat control copyright rules
Then cd in to the directory and run debuild to actually build the package
rus@absinthe:~$ cd Crypt-RC5-2.00/
rus@absinthe:~/Crypt-RC5-2.00$ debuild
This package has a Debian revision number but there does not seem to be
an appropriate original tar file or .orig directory in the parent directory;
(expected libcrypt-rc5-perl_2.00.orig.tar.gz or Crypt-RC5-2.00.orig)
continue anyway? (y/n) y
fakeroot debian/rules clean
dh_testdir
dh_testroot
# Add commands to clean up after the build process here
[ ! -f Makefile ] || /usr/bin/make realclean
dh_clean build-stamp install-stamp
dpkg-source -b Crypt-RC5-2.00
dpkg-source: warning: source directory './Crypt-RC5-2.00' is not - 'libcrypt-rc5-perl-2.00'
dpkg-source: building libcrypt-rc5-perl in libcrypt-rc5-perl_2.00-1.tar.gz
dpkg-source: building libcrypt-rc5-perl in libcrypt-rc5-perl_2.00-1.dsc
debian/rules build
dh_testdir
# Add commands to compile the package here
/usr/bin/perl Makefile.PL INSTALLDIRS=vendor
Checking if your kit is complete...
Looks good
Writing Makefile for Crypt::RC5
/usr/bin/make OPTIMIZE="-Wall -O2 -g"
make[1]: Entering directory `/home/rus/Crypt-RC5-2.00'
cp RC5.pm blib/lib/Crypt/RC5.pm
Manifying blib/man3/Crypt::RC5.3pm
make[1]: Leaving directory `/home/rus/Crypt-RC5-2.00'
touch build-stamp
fakeroot debian/rules binary
dh_testdir
dh_testroot
dh_clean -k
# Add commands to install the package into debian/ACKAGE_NAME here
/usr/bin/make test
make[1]: Entering directory `/home/rus/Crypt-RC5-2.00'
PERL_DL_NONLAZY=1 /usr/bin/perl "-Iblib/lib" "-Iblib/arch" test.pl
1..1
# Running under perl version 5.008008 for linux
# Current time local: Fri Feb 1 16:23:07 2008
# Current time GMT: Fri Feb 1 16:23:07 2008
# Using Test.pm version 1.25
ok 1
make[1]: Leaving directory `/home/rus/Crypt-RC5-2.00'
/usr/bin/make install DESTDIR=/home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl PREFIX=/usr
make[1]: Entering directory `/home/rus/Crypt-RC5-2.00'
Manifying blib/man3/Crypt::RC5.3pm
Installing /home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl/usr/share/perl5/Crypt/RC5.pm
Installing /home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl/usr/share/man/man3/Crypt::RC5.3pm
make[1]: Leaving directory `/home/rus/Crypt-RC5-2.00'
# As this is a architecture independent package, we are not
# supposed to install stuff to /usr/lib. MakeMaker creates
# the dirs, we delete them from the deb:
rmdir --ignore-fail-on-non-empty --parents /home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl/usr/lib/perl5
touch install-stamp
dh_testdir
dh_testroot
dh_installdocs README
dh_installchangelogs Changes
dh_perl
dh_link
dh_strip
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dpkg-gencontrol: warning: unknown substitution variable ${misc:Depends}
dh_md5sums
dh_builddeb
dpkg-deb: building package `libcrypt-rc5-perl' in `../libcrypt-rc5-perl_2.00-1_all.deb'.
dpkg-genchanges
dpkg-genchanges: including full source code in upload
dpkg-buildpackage (debuild emulation): full upload; Debian-native package (full source is included)
Now signing changes and any dsc files...
Could not find a signing program (pgp or gpg)!
debuild: fatal error at line 1174:
running debsign failed
rus@absinthe:~/Crypt-RC5-2.00$
You should then find a nice debian package a directory above!
rus@absinthe:~/Crypt-RC5-2.00$ ls .. | grep libcrypt | grep deb
libcrypt-rc5-perl_2.00-1_all.deb
One of the newer features to our site is an access control mechanism to force specific paths to only be delivered over SSL when our customers have particularly sensitive data. We already use Apache2 with mod_jk to talk to the Tomcat5.5 instance running our app so the only part left is to enable SSL!
First make sure mod_ssl is enabled:
root@reltest-tcj0:/var/log/apache2# a2enmod
Which module would you like to enable?
Your choices are: actions asis auth_anon auth_dbm auth_digest auth_ldap cache cern_meta cgid cgi dav_fs dav deflate disk_cache expires ext_filter file_cache headers imap include info jk ldap mem_cache mime_magic proxy_connect proxy_ftp proxy_http proxy rewrite speling ssl suexec unique_id userdir usertrack vhost_alias
Module name? ssl
This module is already enabled!
Then we configure mod_jk to pass it’s SSL environment variables to Tomcat by adding the following to apache2.conf
JkExtractSSL On
JkHTTPSIndicator HTTPS
JkSESSIONIndicator SSL_SESSION_ID
JkCIPHERIndicator SSL_CIPHER
JkCERTSIndicator SSL_CLIENT_CERT
Tell Apache2 to listen on the SSL port by editing ports.conf
Listen 443
We want to make sure we have the latest common CA certificates in order to establish a trusted root for our new shiny signed certificate!
apt-get install ca-certificates
If you have a lovely genuinely signed certificate like we do you might need to then add it’s intermediate certificate to the ca-certificates system. Move the certificate to /usr/share/ca-certificates then add it’s location to /etc/ca-certificates.conf
Now run update-ca-certificates to update the system’s certificate store (located in /etc/ssl/certs/ca-certificates.crt).
root@reltest-tcj0:/etc/apache2/sites-enabled# update-ca-certificates
Updating certificates in /etc/ssl/certs....done.
We want the same site to simply be available over SSL I’m going to duplicate the existing VirtualHost for that site specifying the use of port 80 for the original vhost and port 443 for the new one that uses SSL. The only change that needs to be made to the new vhost are the following SSL directives:
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain.com.key
SSLCACertificateFile /etc/ssl/certs/ca-certificates.crt
Obviously making sure the keys are in the right place!
And lastly make sure that NameVirtualHost settings exist for both port 80 and port 443!
Well, I guess I’ve already broken one of my New Years resolutions although I don’t feel too bad about it.
After a long stint of only drinking water in pubs (really bloody annoying) I had a diet coke .. not the end of the world. The spirit behind that resolution wasn’t really to completely give up caffeine anyway, but to stop having 8 cups of coffee a day, so I don’t feel too bad about it!
I accidently deleted my Pictures personal folder on my Vista laptop earlier and was stuck trying to recreate it. If I created a new folder and called it Pictures then it would turn in to a file, if I copied an existing folder and then renamed it to Pictures it, once again, would turn in to a file.
The trick to restoring it was to run the following command via the Run prompt in the Start Menu:
