Ajitabh Pandey's Soul & Syntax

Exploring systems, souls, and stories – one post at a time

Category: Tips/Code Snippets

  • Installing Oracle Instant Client for Mac OSX

    I came across few links for installing Oracle Instant Client on Mac OS X namely – this and this. But I don’t like to tinker with the System paths while installing any s/w manually. I did the same with perl CPAN modules, I install them locally in home directory and same with Python pip packages, but that’s another post for future. So I had to work out the following method for installing the Oracle Instant Client as I needed sqlplus client for connecting to my RDS Oracle instance.

    Download the following two files from http://www.oracle.com/technetwork/topics/intel-macsoft-096467.html

    instantclient-basic-macos.x64-11.2.0.4.0.zip
instantclient-sqlplus-macos.x64-11.2.0.4.0.zip

    These will be downloaded in the Downloads directory

    Unzip these two files in a single directory

    $ cd Downloads
$ unzip instantclient-sqlplus-macos.x64-11.2.0.4.0.zip
$ unzip instantclient-basic-macos.x64-11.2.0.4.0.zip
$ cd ~

    Create the required structure in the home directory and copy the files in place

    $ mkdir -p oracle/product/instantclient_64/11.2.0.4.0/{bin,lib,jdbc/lib,rdbms/jlib,sqlplus/admin,network/admin}
$ mv Downloads/instantclient_11_2/ojdbc* product/instantclient_64/11.2.0.4.0/jdbc/lib/
$ mv Downloads/instantclient_11_2/x*.jar product/instantclient_64/11.2.0.4.0/rdbms/jlib/
$ mv Downloads/instantclient_11_2/glogin.sql product/instantclient_64/11.2.0.4.0/sqlplus/admin/
$ mv Downloads/instantclient_11_2/*dylib* product/instantclient_64/11.2.0.4.0/lib/
$ mv Downloads/instantclient_11_2/*README product/instantclient_64/11.2.0.4.0/
$ mv Downloads/instantclient_11_2/adrci instantclient_11_2/genezi instantclient_11_2/sqlplus instantclient_11_2/uidrvci product/instantclient_64/11.2.0.4.0/bin/

    Make the following additions to your ~/.bashrc file –

    ORACLE_HOME="~/oracle/product/instantclient_64/11.2.0.4.0"
PATH=$ORACLE_HOME/bin:$PATH
alias sqlplus="DYLD_LIBRARY_PATH=$ORACLE_HOME/lib sqlplus"

    Now I can connect to the Oracle RDS instance using –

    $ sqlplus 'root@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=ajitabh-oracle-test.bkl9a9lqwsoxcd.us-west-1.rds.amazonaws.com)(PORT=1521))(CONNECT_DATA=(SID=ORCL)))'

SQL*Plus: Release 11.2.0.4.0 Production on Wed Aug 26 21:59:46 2015

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Enter password:

Connected to:
Oracle Database 11g Release 11.2.0.4.0 - 64bit Production

SQL>

    That’s it. More on exploring Oracle RDS later.

  • Build An MP3 Catalogue System With Perl – Conclusion

    In the last post we saw how to read ID3v1 and ID3v2 tags using perl. In this post we will continue our journey towards creating a simple catalog for the MP3 collection.

    Quickly Getting the Desired Information out of the MP3 – autoinfo()

    Usually in my catalog I am interested in the following information about an MP3 – Title, Artist, Album, Track, Year, Genre, Comment and the Artwork. However, I do not want to loop through all available information in my program to get this data. Fortunately the MP3::Tag module provides a autoinfo() function which gets almost all the information needed for us except the Artwork, which we may need to gather separately. The autoinfo() function returns the information about the title, album, artist, track, tear, genre and comment. This information is obtained from ID3v2 tag, ID3v1 tag, CDDB file, .inf file and the mp3 filename itself, where-ever it is found first. The order of this lookup can be changed with the config() command. I want to restrict my cataloging to only ID3v2  and ID3v1 tags.

    Following lines provides us with the needed information.

    $mp3->config("autoinfo", "ID3v2", "ID3v1");
my ($title, $track, $artist, $album, $comment, $year, $genre) = $mp3->autoinfo();

    Getting Artwork Information

    The artwork information is stored in the ID3v2 tag in a frame called APIC (stands for Attached PICture). This frame has _Data and MIME Type which we would need for our purpose. In order to extract this frame and its data we do not need to loop in through all the tags. The MP3::Tag module provides us with the get_frame() method, using which we can extract any frame directly like as shown below for artwork –

    my $apic_frame = $mp3->{ID3v2}->get_frame("APIC");
my $img_data = $$apic_frame{'_Data'};
my $mime_type = $$apic_frame{'MIME type'};

    This $img_data can be written out in a file and the $mime_type can be used as an extension. Thus we can extract the artwork from the MP3 file. The MIME type is something like “image/jpeg” and I have used the split function to get the string for the extension of the file.

    my ($mime1, $mime2) = split(/\//, $mime_type);
my $artwork_name = "artwork.$mime2";
open ARTWORK_FILE, ">$artwork_name" 
  or die "Error creating the artwork file";
binmode(ARTWORK_FILE);
print ARTWORK_FILE $img_data;
close ARTWORK_FILE;

    Generating the HTML using HTML

    This is a simple project so I have used HTML::Template module to generate HTML code to the standard output, which can then in turn be redirected to a file using shell redirection. For a making the table layout less cumbersome, I have used the purecss.io CSS framework. Here my HTML template code.

    my $template = <<HTML;
<html>
<head>
<title>My MP3 Catalog</title>
<link rel="stylesheet" href="http://yui.yahooapis.com/pure/0.5.0/pure-min.css">
</head>
<body>
<h1>My MP3 Collection</h1>
<table class="pure-table pure-table-horizontal">
    <thead>
        <tr>
            <th>Album Artwork</th>
			<th>Album</th>
            <th>Track</th>
            <th>Title</th>
            <th>Artist</th>
			<th>Year</th>
			<th>Genre</th>
			<th>Comment</th>
        </tr>
    </thead>

    <tbody>
		<!-- TMPL_LOOP NAME=SONGS -->
		<tr>
			<td><a src="<TMPL_VAR NAME=FILEPATH>"><img src="<TMPL_VAR NAME=IMG>" height="150" width="150"/></a></td>
			<td><!-- TMPL_VAR NAME=ALBUM --></td>
			<td><!-- TMPL_VAR NAME=TRACK --></td>
			<td><!-- TMPL_VAR NAME=TITLE --></td>
			<td><!-- TMPL_VAR NAME=ARTIST --></td>
			<td><!-- TMPL_VAR NAME=YEAR --></td>
			<td><!-- TMPL_VAR NAME=GENRE --></td>
			<td><!-- TMPL_VAR NAME=COMMENT --></td>
		</tr>
		<!-- /TMPL_LOOP -->
    </tbody>
</table>

</body>
</html>
HTML
my $tmpl = HTML::Template->new(scalarref => \$template);

    Complete Script

    The complete script is on github, you can have a look at –

    https://github.com/ajitabhpandey/learn-programming/blob/master/perl/id3-tags-manipulation/genCatalog.pl.

  • Build An MP3 Catalogue System With Perl – Basics

    My mp3 collection was increasing and I wanted to build a catalogue for the same. There are various steps in having even a simple catalogue system. In this post and a few posts that will follow, I will be explaining how to write such a system using perl as the programming language.

    MP3 Format and ID3 Tags

    MP3 is an audio coding format for digital audio. The audio data in this file is in a compressed format. The compression is a lossy compression, meaning the sound quality is not very clear. In spite of being a lossy format, it is one of the most popular format for audio streaming and storage. The mp3 file has built in bibliographic information such as title, artist, album. This information is stored in a field inside the file known as ID3 tag. Using this information, the MP3 players are able to display the Song Title, Album name and Artist name(s) etc.

    There are couple of versions of these ID3 tags in use. ID3v1 (1.1 being the last in version 1 series) and ID3v2 (ID3v2.4 being the latest version).

    Perl MP3::Tag Module

    The MP3::Tag module of perl can be used to read and write both the versions of the ID3 tag. Here are few of the sample perl programs to do that. This will help in understanding the usage of the modules before we proceed to the next steps.

    Below are couple of examples showing how to read ID3v1 and ID3v2 tags.

    #!/usr/bin/perl
#
# id3v1_read.pl
#
use 5.010;
use warnings;
use strict;
use MP3::Tag;

# set filename of MP3 track
my $filename = "your_mp3_file";

# create new MP3-Tag object
my $mp3 = MP3::Tag->new($filename);

# get tag information
$mp3->get_tags();

# check to see if an ID3v1 tag exists
# if it does, print track information
if (exists $mp3->{ID3v1}) {
  #$mp3->{ID3v1}->remove_tag();exit;

  say "Filename: $filename";
  say "Artist: " . $mp3->{ID3v1}->artist;
  say "Title: " . $mp3->{ID3v1}->title;
  say "Album: " . $mp3->{ID3v1}->album;
  say "Year: ". $mp3->{ID3v1}->year;
  say "Genre: " . $mp3->{ID3v1}->genre;
} else {
  say "$filename: ID3v1 tag not found";
}

# clean up
$mp3->close();

    ID3v2 tags are a bit more complex as they allow a lot more information to be stored in the MP3 file such as the album artwork etc. If you run the following script on one of your MP3 files it will print all the ID3v2 information to the screen. I have used the getc() function in order to allow you to observe the output and press <Enter> to proceed to the next set of key-value pair. After couple of keystrokes you will see there are lot of junk characters printed. These junk characters are nothing but the album artwork and following the junk characters is the MIME type of the artwork. In my case the MIME types were all “image/jpeg”.

    #!/usr/bin/perl
#
# id3v2_read.pl
#
use 5.010;
use warnings;
use strict;
use MP3::Tag;

# set filename of MP3 track
my $filename = "mp3_file_name;

# create new MP3-Tag object
my $mp3 = MP3::Tag->new($filename);

# get tag information
$mp3->get_tags();

# check to see if an ID3v2 tag exists
# if it does, print track information
if (exists $mp3->{ID3v2}) {
  # get a list of frames as a hash reference
  my $frames = $mp3->{ID3v2}->get_frame_ids();

  # iterate over the hash, process each frame
  foreach my $frame (keys %$frames) {
    # for each frame get a key-value pair of content-description
    my ($value, $desc) = $mp3->{ID3v2}->get_frame($frame);
    if (defined($desc) and length $desc) {
      say "$frame $desc: "; 
    } else {
      say "$frame :";
    }
    # sometimes the value is itself a hash reference containing more values
    # deal with that here
    if (ref $value eq "HASH") {
      while (my ($k, $v) = each (%$value)) {
        say "\n     - $k: $v";
      }
    } else {
      say "$value";
    }
    # allows to view each iteration
    getc(STDIN);
  }
} else {
  say "$filename: ID3v2 tag not found";
}

# clean up
$mp3->close();

    Next Steps

    Most of the current MP3 players read ID3v2 tags. It will be good to understand the structure of the ID3v2 tags, using one of the links I provided above. This will help you prepare for understanding the further articles in this series. In next part we will see how to extract desired information quickly and how to extract the artwork data from the MP3 file. Stay tuned for more.

  • Time-zone Setting in Linux and BSDs from Shell

    Often the default time-zone in a linux and bsd system does not match our preferences. On a system which we have installed ourself, we may have selected the appropriate time-zone during installation, but as systems administrators we often get our hands on a system which is pre-installed and after taking over we want to change the time-zone to something which we are comfortable understanding and co-relating various system events in the time of our comfort.

    The time-zone of  the system is determined by a file called “/etc/localtime”, which a binary file. In order to change the time-zone, we need to replace this file with an appropriate file of our time-zone. All time-zone files are found in “/usr/share/zoneinfo”.

    On some systems “/etc/localtime” is a copy and in some cases a hard link of one of the time-zone found in “/usr/share/zoneinfo” directory. In OpenBSD, “/etc/localtime” is a symlink to one of the files in “/usr/share/zoneinfo”. I prefer the symlink approach, you can pick any of the methods to make appropriate “/etc/localtime” file available.

    In order to change the time-zone of my system from UTC to IST, I did the following.

    $ sudo ln -sf/usr/share/zoneinfo/Asia/Kolkata /etc/localtime

    In case you accidentally delete the “/etc/localtime” file, the timezone of the system reverts to UTC and upon having the correct file present again, it will reflect the correct timezone again. See below (I did this on a RHEL 6 machine) –

    $ date
Wed Dec 18 13:56:05 IST 2013
$ sudo rm -f /etc/localtime
$ date
Wed Dec 18 08:36:29 UTC 2013
$ ls -l /etc/localtime
ls: cannot access /etc/localtime: No such file or directory
$ sudo ln -sf /usr/share/zoneinfo/Asia/Kolkata /etc/localtime
$ date
Wed Dec 18 14:07:03 IST 2013

     

  • Creating a DEB package for Oracle Java

    Linux based operating systems comes by default with Open JDK installations. Often it is necessary for various developmental needs to install the Oracle Java. Oracle only supplies two installations of Java for Linux based operating systems – a rpm and a tar.gz binary distribution.

    For RPM based distributions it is fairly easy to have the uniform setup on a large farm of servers, but on DEB based distributions a lot of manual work needs to be done at each host after unpacking the tar.gz file such as updating the alternatives etc to make the Oracle Java a default Java installation.

    I have been thinking about how to deploy Oracle Java SDK on multiple Debian servers in a standardized way i.e by creating a DEB file.

    A search on Debian Packages revealed the existence of a tool called – “make-jpkg”, a part of “java-package” package. This tool is capable of creating a DEB package from this tar.gz file. Here is how I created the package –

    $ sudo apt-get install fakeroot java-package

    I downloaded the Oracle Java Standard Edition  on my Linux Mint Debian Edition (LMDE) virtual machine.

    Next I ran the command –

    $ fakeroot make-jpkg --full-name "Ajitabh Pandey" --email "ajitabh@unixclinic.com" /home/ajitabhp/Downloads/jdk-7u25-linux-x64.tar.gz

    I got an error stating “No Matching Plugin was found”

    Just above this error message the plugin path was given for various plugins used by this utility – “make-jpkg”. The plugins were in /usr/share/java-package directory in the form of “.sh” files (shell scripts). I next opened each shell script to try and decipher them and found the possible problem in the plugins “/usr/share/java-package/oracle-j2*.sh”. The following pattern picked up from “/usr/share/java-package/oracle-j2sdk.sh” was the culprit –

    "jdk-7u"[0-9]"-linux-x64.tar.gz") # SUPPORTED
    j2se_version=1.7.0+update${archive_name:6:1}${revision}
    j2se_expected_min_size=180 #Mb
    j2se_priority=317
    found=true
    ;;

    If you see this pattern, it clearly indicates that it is only supporting up to update 9 of the JDK and the JDK I downloaded was update 25. So I added the following lines just below the above lines to add the support for my version.

    "jdk-7u"[0-9][0-9]"-linux-x64.tar.gz") # SUPPORTED
    j2se_version=1.7.0+update${archive_name:6:2}${revision}
    j2se_expected_min_size=180 #Mb
    j2se_priority=317
    found=true
    ;;

    There are two differences, one in first line and the second in second line. I am sure you will be able to spot them and decipher them.

    After making similar changes in the other oracle* files I ran the make-jpkg as mentioned above and it created a DEB file – oracle-j2sdk1.7_1.7.0+update25_amd64.deb, which is deploy-able with the usual “dpkg -i”.

    Changing the Default Java Installation to the Oracle Installation

    After installing the package, you may want to change some alternatives in order to make the Oracle JDK and associated JRE installations as defaults. As the default version of Java may be something different. You can check the current version of java and its provider by –

    $ java -version
    java version "1.6.0_24"
    OpenJDK Runtime Environment (IcedTea6 1.11.5) (6b24-1.11.5-1)
    OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
    $ file `which java`
    /usr/bin/java: symbolic link to `/etc/alternatives/java'
    $ ls -l /etc/alternatives/java
    lrwxrwxrwx 1 root root 46 Aug 16 20:18 /etc/alternatives/java -> /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java

    First find out the name of the available installations –

    $ /usr/sbin/update-java-alternatives -l
    j2sdk1.7-oracle 317 /usr/lib/jvm/j2sdk1.7-oracle
    java-1.6.0-openjdk-amd64 1061 /usr/lib/jvm/java-1.6.0-openjdk-amd64

    We will switch our default Java to “j2sdk1.7-oracle” with the following command –

    $ sudo /usr/sbin/update-java-alternatives -s j2sdk1.7-oracle

    Now you can check your Java version by

    $ java -version
    java version "1.7.0_25"
    Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
    Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
    $ file `which java`
    /usr/bin/java: symbolic link to `/etc/alternatives/java'
    $ ls -l /etc/alternatives/java
    lrwxrwxrwx 1 root root 41 Aug 16 20:22 /etc/alternatives/java -> /usr/lib/jvm/j2sdk1.7-oracle/jre/bin/java