Turning CPAN modules in to deb with dh perl make

Some of our PERL tools require some CPAN modules that are not part of the standard Ubuntu distribution. It’s obviously possible to install the module using CPAN but I like using deb packages where possible as then you only have one repository to manage. Fortunately with dh-make-perl it is possible to quickly turn any CPAN module in to a debian package!

First make sure dh-make-perl is installed


apt-get install dh-make-perl

Then download the PERL module you wish to package and extract it


[email protected]:~$ wget https://www.cpan.org/modules/by-module/Crypt/Crypt-RC5-2.00.tar.gz
[email protected]:~$ tar -pzxvf Crypt-RC5-2.00.tar.gz
Crypt-RC5-2.00/
Crypt-RC5-2.00/Changes
Crypt-RC5-2.00/Makefile.PL
Crypt-RC5-2.00/MANIFEST
Crypt-RC5-2.00/RC5.pm
Crypt-RC5-2.00/README
Crypt-RC5-2.00/test.pl

Now run dh-make-perl on the directory to create the files needed for the package


[email protected]inthe:~$ dh-make-perl Crypt-RC5-2.00/
cat: /etc/mailname: No such file or directory
Use of uninitialized value in concatenation (.) or string at /usr/bin/dh-make-perl line 527.
Found: Crypt::RC5 2.00 (libcrypt-rc5-perl arch=all)
Package does not provide a long description - Please fill it in manually.
Using maintainer: rus
Found changelog: Changes
Found docs: README
Using rules: /usr/share/dh-make-perl/rules.MakeMaker.noxs
Done

This will have created a debian directory


[email protected]:~/Crypt-RC5-2.00$ ls
Changes debian Makefile.PL MANIFEST RC5.pm README test.pl
[email protected]:~/Crypt-RC5-2.00$ ls debian/
changelog compat control copyright rules

Then cd in to the directory and run debuild to actually build the package


[email protected]:~$ cd Crypt-RC5-2.00/
[email protected]:~/Crypt-RC5-2.00$ debuild
This package has a Debian revision number but there does not seem to be
an appropriate original tar file or .orig directory in the parent directory;
(expected libcrypt-rc5-perl_2.00.orig.tar.gz or Crypt-RC5-2.00.orig)
continue anyway? (y/n) y
fakeroot debian/rules clean
dh_testdir
dh_testroot
# Add commands to clean up after the build process here
[ ! -f Makefile ] || /usr/bin/make realclean
dh_clean build-stamp install-stamp
dpkg-source -b Crypt-RC5-2.00
dpkg-source: warning: source directory './Crypt-RC5-2.00' is not - 'libcrypt-rc5-perl-2.00'
dpkg-source: building libcrypt-rc5-perl in libcrypt-rc5-perl_2.00-1.tar.gz
dpkg-source: building libcrypt-rc5-perl in libcrypt-rc5-perl_2.00-1.dsc
debian/rules build
dh_testdir
# Add commands to compile the package here
/usr/bin/perl Makefile.PL INSTALLDIRS=vendor
Checking if your kit is complete...
Looks good
Writing Makefile for Crypt::RC5
/usr/bin/make OPTIMIZE="-Wall -O2 -g"
make[1]: Entering directory `/home/rus/Crypt-RC5-2.00'
cp RC5.pm blib/lib/Crypt/RC5.pm
Manifying blib/man3/Crypt::RC5.3pm
make[1]: Leaving directory `/home/rus/Crypt-RC5-2.00'
touch build-stamp
fakeroot debian/rules binary
dh_testdir
dh_testroot
dh_clean -k
# Add commands to install the package into debian/ACKAGE_NAME here
/usr/bin/make test
make[1]: Entering directory `/home/rus/Crypt-RC5-2.00'
PERL_DL_NONLAZY=1 /usr/bin/perl "-Iblib/lib" "-Iblib/arch" test.pl
1..1
# Running under perl version 5.008008 for linux
# Current time local: Fri Feb 1 16:23:07 2008
# Current time GMT: Fri Feb 1 16:23:07 2008
# Using Test.pm version 1.25
ok 1
make[1]: Leaving directory `/home/rus/Crypt-RC5-2.00'
/usr/bin/make install DESTDIR=/home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl PREFIX=/usr
make[1]: Entering directory `/home/rus/Crypt-RC5-2.00'
Manifying blib/man3/Crypt::RC5.3pm
Installing /home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl/usr/share/perl5/Crypt/RC5.pm
Installing /home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl/usr/share/man/man3/Crypt::RC5.3pm
make[1]: Leaving directory `/home/rus/Crypt-RC5-2.00'
# As this is a architecture independent package, we are not
# supposed to install stuff to /usr/lib. MakeMaker creates
# the dirs, we delete them from the deb:
rmdir --ignore-fail-on-non-empty --parents /home/rus/Crypt-RC5-2.00/debian/libcrypt-rc5-perl/usr/lib/perl5
touch install-stamp
dh_testdir
dh_testroot
dh_installdocs README
dh_installchangelogs Changes
dh_perl
dh_link
dh_strip
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dpkg-gencontrol: warning: unknown substitution variable ${misc:Depends}
dh_md5sums
dh_builddeb
dpkg-deb: building package `libcrypt-rc5-perl' in `../libcrypt-rc5-perl_2.00-1_all.deb'.
dpkg-genchanges
dpkg-genchanges: including full source code in upload
dpkg-buildpackage (debuild emulation): full upload; Debian-native package (full source is included)
Now signing changes and any dsc files...
Could not find a signing program (pgp or gpg)!
debuild: fatal error at line 1174:
running debsign failed
[email protected]:~/Crypt-RC5-2.00$

You should then find a nice debian package a directory above!


[email protected]:~/Crypt-RC5-2.00$ ls .. | grep libcrypt | grep deb
libcrypt-rc5-perl_2.00-1_all.deb

Programming Perl Programming Perl is an awesome and recommended guide to Perl.

Apache2 with SSL and Tomcat5.5 on Ubuntu

One of the newer features to our site is an access control mechanism to force specific paths to only be delivered over SSL when our customers have particularly sensitive data. We already use Apache2 with mod_jk to talk to the Tomcat5.5 instance running our app so the only part left is to enable SSL!

First make sure mod_ssl is enabled:


[email protected]:/var/log/apache2# a2enmod
Which module would you like to enable?
Your choices are: actions asis auth_anon auth_dbm auth_digest auth_ldap cache cern_meta cgid cgi dav_fs dav deflate disk_cache expires ext_filter file_cache headers imap include info jk ldap mem_cache mime_magic proxy_connect proxy_ftp proxy_http proxy rewrite speling ssl suexec unique_id userdir usertrack vhost_alias
Module name? ssl
This module is already enabled!

Then we configure mod_jk to pass it’s SSL environment variables to Tomcat by adding the following to apache2.conf


JkExtractSSL On
JkHTTPSIndicator HTTPS
JkSESSIONIndicator SSL_SESSION_ID
JkCIPHERIndicator SSL_CIPHER
JkCERTSIndicator SSL_CLIENT_CERT

Tell Apache2 to listen on the SSL port by editing ports.conf


Listen 443

We want to make sure we have the latest common CA certificates in order to establish a trusted root for our new shiny signed certificate!


apt-get install ca-certificates

If you have a lovely genuinely signed certificate like we do you might need to then add it’s intermediate certificate to the ca-certificates system. Move the certificate to /usr/share/ca-certificates then add it’s location to /etc/ca-certificates.conf

Now run update-ca-certificates to update the system’s certificate store (located in /etc/ssl/certs/ca-certificates.crt).


[email protected]:/etc/apache2/sites-enabled# update-ca-certificates
Updating certificates in /etc/ssl/certs....done.

We want the same site to simply be available over SSL I’m going to duplicate the existing VirtualHost for that site specifying the use of port 80 for the original vhost and port 443 for the new one that uses SSL. The only change that needs to be made to the new vhost are the following SSL directives:


SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain.com.key
SSLCACertificateFile /etc/ssl/certs/ca-certificates.crt

Obviously making sure the keys are in the right place!

And lastly make sure that NameVirtualHost settings exist for both port 80 and port 443!


NameVirtualHost *:80
NameVirtualHost *:443

et voila.

Damn caffeine

Well, I guess I’ve already broken one of my New Years resolutions although I don’t feel too bad about it.

After a long stint of only drinking water in pubs (really bloody annoying) I had a diet coke .. not the end of the world. The spirit behind that resolution wasn’t really to completely give up caffeine anyway, but to stop having 8 cups of coffee a day, so I don’t feel too bad about it!

Restoring Vista personal folders

I accidently deleted my Pictures personal folder on my Vista laptop earlier and was stuck trying to recreate it. If I created a new folder and called it Pictures then it would turn in to a file, if I copied an existing folder and then renamed it to Pictures it, once again, would turn in to a file.

The trick to restoring it was to run the following command via the Run prompt in the Start Menu:


shell:Pictures

This then recreated the folder for me.

Apache2 ldap auth on Ubuntu Dapper and Feisty

As part of our internal office systems upgrade we have a shiny new LDAP server which we like to use as much as possible. One of the things we use it for is Apache user auth, mainly we control SVN with it so people can only commit to the projects they’re allowed to but we also use it so secure our system’s services from the developers that like to play wannabe sysadmin!

Unfortunately we are running several different flavors of Ubuntu in the office with slightly different Apache2 versions and thus LDAP requirements!

Ubuntu Dapper Drake (Apache 2.0)


AuthType basic
AuthName "BackupPC admin"
AuthLDAPUrl ldap://ldap-server:389/ou=people,dc=domain,dc=com?uid?sub
AuthLDAPGroupAttributeIsDN off
AuthLDAPEnabled on
Require group cn=systems,ou=groups,dc=domain,dc=com
AuthLDAPGroupAttribute memberUid

Ubuntu Feisty Fawn (Apache 2.22)


AuthType Basic
AuthName "SVN Repository"
AuthLDAPUrl ldap://ldap-server:389/ou=people,dc=domain,dc=com?uid?sub
AuthzLDAPAuthoritative On
AuthBasicProvider ldap
AuthLDAPGroupAttribute memberUid
AuthLDAPGroupAttributeIsDN off
Require ldap-group cn=developers,ou=groups,dc=domain,dc=com

Obviously you have to make sure you have the right LDAP modules enabled for each version of Apache2 but that’s all that is required to force Apache2 to authenticate against an LDAP group!

Ubuntu and webcams

I’ve had some debugging to do for work for a part of our site that uses webcams but I’ve been a bit hazey about starting it because the last time I plugged a webcam in to a Linux box, 100 years ago, it was a bit messy. I decided to risk it as I’m too lazy to go home and get my laptop, and .. it .. just .. worked ..

Once again Ubuntu impresses me!


[243052.596000] usb 2-7: new full speed USB device using ohci_hcd and address 3
[243052.804000] usb 2-7: configuration #1 chosen from 1 choice
[243052.972000] Linux video capture interface: v2.00
[243052.988000] quickcam: QuickCam USB camera found (driver version QuickCam USB 0.6.6 $Date: 2006/11/04 08:38:14 $)
[243052.988000] quickcam: Kernel:2.6.22-14-generic bus:2 class:FF subclass:FF vendor:046D product:0870
[243053.016000] quickcam: Sensor HDCS-1020 detected
[243053.024000] quickcam: Registered device: /dev/video0
[243053.024000] usbcore: registered new interface driver quickcam

I feel these office lights are harsh on my skin though 🙁

Using sed to replace all strings in a file

As part of our test environment rebuild one of the first things we tackled was moving the databases to the new virtual environment, this means changing the database address in a lot of config files, fortunately sed makes this job really easy!


sed -i s/olddatabase/newdatabase/g *.xml

We’re also using CNAMEs now for the addresses to make this change a lot easier next time the database moves hardware 🙂

Atom feeds with PHP 5 Dom and XSL

All blogs require silly amounts of feed generators, right? And this is a silly blog so requires a silly generator. The entire site is written using PHP5, and my automagic ‘datahandler’ activepage concept creates an XML document using DOM that then uses XSL as a templating engine, so I figured it wouldn’t be too hard to knock up a stylesheet to turn the default datahandler for the blog in to a nice atom feed! Just make sure you set the content-type to application/atom+xml when generate the page!


<?xml version="1.0" encoding="iso-8859-1"?>
<xsl:stylesheet version = "1.0" xmlns:xsl="https://www.w3.org/1999/XSL/Transform">
<xsl:output indent="yes" method="xml" encoding="iso-8859-1" omit-xml-declaration="yes" doctype-system="https://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd" doctype-public="-//W3C//DTD XHTML 1.1//EN" />
<xsl:template match="page">
<feed xmlns="https://www.w3.org/2005/Atom">
<link rel="alternate" type="text/html" href="https://www.idimmu.net/" />
<link rel="self" href="https://www.idimmu.net/blog/atom.php" />
<title>idimmu . net</title>
<link href="https://www.idimmu.net/"/>
<updated>
<xsl:value-of select="datahandler_blog/blog_list/blog/date/year"/>-<xsl:value-of select="datahandler_blog/blog_list/blog/date/month"/>-<xsl:value-of select="datahandler_blog/blog_list/blog/date/day"/>T<xsl:value-of select="datahandler_blog/blog_list/blog/date/hour"/>:<xsl:value-of select="datahandler_blog/blog_list/blog/date/minute"/>:<xsl:value-of select="datahandler_blog/blog_list/blog/date/second"/>Z</updated> <author> <name>idimmu</name> </author> <id>https://www.idimmu.net/</id> <xsl:apply-templates select="datahandler_blog"/> </feed> </xsl:template> <xsl:template match="datahandler_blog"> <xsl:apply-templates select="blog_list"/> </xsl:template> <xsl:template match="blog"> <entry> <title><xsl:value-of select="title"/></title> <link href="https://www.idimmu.net/{clonefakeurl}"/> <id>https://www.idimmu.net/<xsl:value-of select="clonefakeurl"/></id> <updated><xsl:value-of select="date/year"/>-<xsl:value-of select="date/month"/>-<xsl:value-of select="date/day"/>T<xsl:value-of select="date/hour"/>:<xsl:value-of select="date/minute"/>:<xsl:value-of select="date/second"/>Z</updated>
<content type="xhtml">
<div xmlns="https://www.w3.org/1999/xhtml">
<xsl:value-of select="bb_content" disable-output-escaping="yes"/>
</div>
</content>
</entry>
</xsl:template>
<xsl:template match="blog_list">
<xsl:apply-templates select="blog"/>
</xsl:template>
</xsl:stylesheet>

PHP Java Bridge in Ubuntu Gutsy with Lucene

The php/java bridge it a pretty awesome little protocol that basically lets us use java classes inside our own PHP applications! This lets you harness the awesome power of all the Java libraries that exist, including the popular Lucene search engine library.

I referenced two excellent blog entries here and here whilst implementing Lucene search for this blog, but I am writing up the experience anyway to compare issues and difficulties and enhance my understanding of the process.

To start with Java, Lucene and the bridge dependancies must be installed (remember to enable multiverse in your apt sources)


apt-get install sun-java6-jre sun-java6-jdk liblucene-java libitext-java
update-java-alternatives -s java-6-sun

Grab the php-java-bridge deb package from sourceforge and install it. The fact it is v4 does not reflect that it is only for PHP version 4! There are RPMs for version 5 which you could turn in to a deb package using alien but at the moment I am feeling lazy so I will see how version 4 works out first.


wget https://downloads.sourceforge.net/php-java-bridge/php-java-bridge_4.3.0-1_i386.deb
dpkg -i php-java-bridge_4.3.0-1_i386.deb

Apache should restart now, if not restart it yourself.

To check that it is working look at the output of phpinfo(), there should be a new shiny java section! Listing the running processes also is interesting!


root 20205 0.0 0.7 664520 15520 ? Sl 17:18 0:00 java -Djava.library.path=/usr/lib/php5/20060613+lfs
-Djava.class.path=/usr/lib/php5/20060613+lfs/JavaBridge.jar -Djava.awt.headless=true
-Dphp.java.bridge.base=/usr/lib/php5/20060613+lfs php.java.bridge.Standalone LOCAL:@java-bridge-4ee9 1

as does netstat


unix 2 [ ACC ] STREAM LISTENING 1913999 @java-bridge-4ee9

I think it gets started when apache starts, as java.so is loaded in to the PHP, I’m still investigating that.

As far as starting the Lucene development goes, this was a pretty good tutorial on how it all works and this site has some good Java example code that I used to work out how the PHP should work.

Below is my PHP Lucene test code, it just creates one document with a description then searches the index description for ‘idi test’ and outputs the match. It’s pretty rad!


java_require('/usr/share/java/lucene.jar');

$analyzer = new Java('org.apache.lucene.analysis.StopAnalyzer');
$writer = new Java('org.apache.lucene.index.IndexWriter', '/path/to/store/lucene/data/in', $analyzer, true);

$doc = new Java('org.apache.lucene.document.Document');
$field = new Java('org.apache.lucene.document.Field','description','idi data test',true, true, true);
$doc->add($field);

$writer->addDocument($doc);

$writer->close();

$indexer = new Java('org.apache.lucene.search.IndexSearcher','/path/to/store/lucene/data/in');
$parser = new Java('org.apache.lucene.queryParser.QueryParser','description',$analyzer);
$query = $parser->parse('rus test');

$hits = $indexer->search($query);

for ($i = 0; $i < $hits->length(); $i++) {
$found = $hits->doc($i);
print $i.".".$found->get('description');
}
?>

Now that it’s working I just have to incorperate it in to the site 🙂

Copying files between servers with netcat and tar

One of the quickest ways (faster than scp at any rate) of copying a large number of files between 2 servers is by abusing the awesome powers of Linux’s pipeing and netcat and tar!

Basically we set up netcat listening on the server you want the files copied too which pipes it’s output to tar which extracts anything sent to it.


[email protected]:/exports/archive# nc -l -p 7878 | tar -xzf -

Then we set up tar on the server we want to copy from, make it create a tarball and pipe it through a netcat which connects to the other server!


fee /home/shared/people # tar -cz MC | nc -q 10 tanglefoot 7878

When the copy has finished the sending instance of netcat will then exit!