Ajitabh Pandey's Soul & Syntax

Exploring systems, souls, and stories – one post at a time

Category: Tips/Code Snippets

  • Build An MP3 Catalogue System With Perl – Conclusion

    In the last post we saw how to read ID3v1 and ID3v2 tags using perl. In this post we will continue our journey towards creating a simple catalog for the MP3 collection.

    Quickly Getting the Desired Information out of the MP3 – autoinfo()

    Usually in my catalog I am interested in the following information about an MP3 – Title, Artist, Album, Track, Year, Genre, Comment and the Artwork. However, I do not want to loop through all available information in my program to get this data. Fortunately the MP3::Tag module provides a autoinfo() function which gets almost all the information needed for us except the Artwork, which we may need to gather separately. The autoinfo() function returns the information about the title, album, artist, track, tear, genre and comment. This information is obtained from ID3v2 tag, ID3v1 tag, CDDB file, .inf file and the mp3 filename itself, where-ever it is found first. The order of this lookup can be changed with the config() command. I want to restrict my cataloging to only ID3v2  and ID3v1 tags.

    Following lines provides us with the needed information.

    $mp3->config("autoinfo", "ID3v2", "ID3v1");
    my ($title, $track, $artist, $album, $comment, $year, $genre) = $mp3->autoinfo();

    Getting Artwork Information

    The artwork information is stored in the ID3v2 tag in a frame called APIC (stands for Attached PICture). This frame has _Data and MIME Type which we would need for our purpose. In order to extract this frame and its data we do not need to loop in through all the tags. The MP3::Tag module provides us with the get_frame() method, using which we can extract any frame directly like as shown below for artwork –

    my $apic_frame = $mp3->{ID3v2}->get_frame("APIC");
    my $img_data = $$apic_frame{'_Data'};
    my $mime_type = $$apic_frame{'MIME type'};

    This $img_data can be written out in a file and the $mime_type can be used as an extension. Thus we can extract the artwork from the MP3 file. The MIME type is something like “image/jpeg” and I have used the split function to get the string for the extension of the file.

    my ($mime1, $mime2) = split(/\//, $mime_type);
    my $artwork_name = "artwork.$mime2";
    open ARTWORK_FILE, ">$artwork_name" 
      or die "Error creating the artwork file";
    binmode(ARTWORK_FILE);
    print ARTWORK_FILE $img_data;
    close ARTWORK_FILE;

    Generating the HTML using HTML

    This is a simple project so I have used HTML::Template module to generate HTML code to the standard output, which can then in turn be redirected to a file using shell redirection. For a making the table layout less cumbersome, I have used the purecss.io CSS framework. Here my HTML template code.

    my $template = <<HTML;
    <html>
    <head>
    <title>My MP3 Catalog</title>
    <link rel="stylesheet" href="http://yui.yahooapis.com/pure/0.5.0/pure-min.css">
    </head>
    <body>
    <h1>My MP3 Collection</h1>
    <table class="pure-table pure-table-horizontal">
        <thead>
            <tr>
                <th>Album Artwork</th>
    			<th>Album</th>
                <th>Track</th>
                <th>Title</th>
                <th>Artist</th>
    			<th>Year</th>
    			<th>Genre</th>
    			<th>Comment</th>
            </tr>
        </thead>
    
        <tbody>
    		<!-- TMPL_LOOP NAME=SONGS -->
    		<tr>
    			<td><a src="<TMPL_VAR NAME=FILEPATH>"><img src="<TMPL_VAR NAME=IMG>" height="150" width="150"/></a></td>
    			<td><!-- TMPL_VAR NAME=ALBUM --></td>
    			<td><!-- TMPL_VAR NAME=TRACK --></td>
    			<td><!-- TMPL_VAR NAME=TITLE --></td>
    			<td><!-- TMPL_VAR NAME=ARTIST --></td>
    			<td><!-- TMPL_VAR NAME=YEAR --></td>
    			<td><!-- TMPL_VAR NAME=GENRE --></td>
    			<td><!-- TMPL_VAR NAME=COMMENT --></td>
    		</tr>
    		<!-- /TMPL_LOOP -->
        </tbody>
    </table>
    
    </body>
    </html>
    HTML
    my $tmpl = HTML::Template->new(scalarref => \$template);

    Complete Script

    The complete script is on github, you can have a look at –

    https://github.com/ajitabhpandey/learn-programming/blob/master/perl/id3-tags-manipulation/genCatalog.pl.

  • Build An MP3 Catalogue System With Perl – Basics

    My mp3 collection was increasing and I wanted to build a catalogue for the same. There are various steps in having even a simple catalogue system. In this post and a few posts that will follow, I will be explaining how to write such a system using perl as the programming language.

    MP3 Format and ID3 Tags

    MP3 is an audio coding format for digital audio. The audio data in this file is in a compressed format. The compression is a lossy compression, meaning the sound quality is not very clear. In spite of being a lossy format, it is one of the most popular format for audio streaming and storage. The mp3 file has built in bibliographic information such as title, artist, album. This information is stored in a field inside the file known as ID3 tag. Using this information, the MP3 players are able to display the Song Title, Album name and Artist name(s) etc.

    There are couple of versions of these ID3 tags in use. ID3v1 (1.1 being the last in version 1 series) and ID3v2 (ID3v2.4 being the latest version).

    Perl MP3::Tag Module

    The MP3::Tag module of perl can be used to read and write both the versions of the ID3 tag. Here are few of the sample perl programs to do that. This will help in understanding the usage of the modules before we proceed to the next steps.

    Below are couple of examples showing how to read ID3v1 and ID3v2 tags.

    #!/usr/bin/perl
    #
    # id3v1_read.pl
    #
    use 5.010;
    use warnings;
    use strict;
    use MP3::Tag;
    
    # set filename of MP3 track
    my $filename = "your_mp3_file";
    
    # create new MP3-Tag object
    my $mp3 = MP3::Tag->new($filename);
    
    # get tag information
    $mp3->get_tags();
    
    # check to see if an ID3v1 tag exists
    # if it does, print track information
    if (exists $mp3->{ID3v1}) {
      #$mp3->{ID3v1}->remove_tag();exit;
    
      say "Filename: $filename";
      say "Artist: " . $mp3->{ID3v1}->artist;
      say "Title: " . $mp3->{ID3v1}->title;
      say "Album: " . $mp3->{ID3v1}->album;
      say "Year: ". $mp3->{ID3v1}->year;
      say "Genre: " . $mp3->{ID3v1}->genre;
    } else {
      say "$filename: ID3v1 tag not found";
    }
    
    # clean up
    $mp3->close();

    ID3v2 tags are a bit more complex as they allow a lot more information to be stored in the MP3 file such as the album artwork etc. If you run the following script on one of your MP3 files it will print all the ID3v2 information to the screen. I have used the getc() function in order to allow you to observe the output and press <Enter> to proceed to the next set of key-value pair. After couple of keystrokes you will see there are lot of junk characters printed. These junk characters are nothing but the album artwork and following the junk characters is the MIME type of the artwork. In my case the MIME types were all “image/jpeg”.

    #!/usr/bin/perl
    #
    # id3v2_read.pl
    #
    use 5.010;
    use warnings;
    use strict;
    use MP3::Tag;
    
    # set filename of MP3 track
    my $filename = "mp3_file_name;
    
    # create new MP3-Tag object
    my $mp3 = MP3::Tag->new($filename);
    
    # get tag information
    $mp3->get_tags();
    
    # check to see if an ID3v2 tag exists
    # if it does, print track information
    if (exists $mp3->{ID3v2}) {
      # get a list of frames as a hash reference
      my $frames = $mp3->{ID3v2}->get_frame_ids();
    
      # iterate over the hash, process each frame
      foreach my $frame (keys %$frames) {
        # for each frame get a key-value pair of content-description
        my ($value, $desc) = $mp3->{ID3v2}->get_frame($frame);
        if (defined($desc) and length $desc) {
          say "$frame $desc: "; 
        } else {
          say "$frame :";
        }
        # sometimes the value is itself a hash reference containing more values
        # deal with that here
        if (ref $value eq "HASH") {
          while (my ($k, $v) = each (%$value)) {
            say "\n     - $k: $v";
          }
        } else {
          say "$value";
        }
        # allows to view each iteration
        getc(STDIN);
      }
    } else {
      say "$filename: ID3v2 tag not found";
    }
    
    # clean up
    $mp3->close();

    Next Steps

    Most of the current MP3 players read ID3v2 tags. It will be good to understand the structure of the ID3v2 tags, using one of the links I provided above. This will help you prepare for understanding the further articles in this series. In next part we will see how to extract desired information quickly and how to extract the artwork data from the MP3 file. Stay tuned for more.

  • Time-zone Setting in Linux and BSDs from Shell

    Often the default time-zone in a linux and bsd system does not match our preferences. On a system which we have installed ourself, we may have selected the appropriate time-zone during installation, but as systems administrators we often get our hands on a system which is pre-installed and after taking over we want to change the time-zone to something which we are comfortable understanding and co-relating various system events in the time of our comfort.

    The time-zone of  the system is determined by a file called “/etc/localtime”, which a binary file. In order to change the time-zone, we need to replace this file with an appropriate file of our time-zone. All time-zone files are found in “/usr/share/zoneinfo”.

    On some systems “/etc/localtime” is a copy and in some cases a hard link of one of the time-zone found in “/usr/share/zoneinfo” directory. In OpenBSD, “/etc/localtime” is a symlink to one of the files in “/usr/share/zoneinfo”. I prefer the symlink approach, you can pick any of the methods to make appropriate “/etc/localtime” file available.

    In order to change the time-zone of my system from UTC to IST, I did the following.

    $ sudo ln -sf/usr/share/zoneinfo/Asia/Kolkata /etc/localtime

    In case you accidentally delete the “/etc/localtime” file, the timezone of the system reverts to UTC and upon having the correct file present again, it will reflect the correct timezone again. See below (I did this on a RHEL 6 machine) –

    $ date
    Wed Dec 18 13:56:05 IST 2013
    $ sudo rm -f /etc/localtime
    $ date
    Wed Dec 18 08:36:29 UTC 2013
    $ ls -l /etc/localtime
    ls: cannot access /etc/localtime: No such file or directory
    $ sudo ln -sf /usr/share/zoneinfo/Asia/Kolkata /etc/localtime
    $ date
    Wed Dec 18 14:07:03 IST 2013

     

  • Creating a DEB package for Oracle Java

    Linux based operating systems comes by default with Open JDK installations. Often it is necessary for various developmental needs to install the Oracle Java. Oracle only supplies two installations of Java for Linux based operating systems – a rpm and a tar.gz binary distribution.

    For RPM based distributions it is fairly easy to have the uniform setup on a large farm of servers, but on DEB based distributions a lot of manual work needs to be done at each host after unpacking the tar.gz file such as updating the alternatives etc to make the Oracle Java a default Java installation.

    I have been thinking about how to deploy Oracle Java SDK on multiple Debian servers in a standardized way i.e by creating a DEB file.

    A search on Debian Packages revealed the existence of a tool called – “make-jpkg”, a part of “java-package” package. This tool is capable of creating a DEB package from this tar.gz file. Here is how I created the package –

    $ sudo apt-get install fakeroot java-package

    I downloaded the Oracle Java Standard Edition  on my Linux Mint Debian Edition (LMDE) virtual machine.

    Next I ran the command –

    $ fakeroot make-jpkg --full-name "Ajitabh Pandey" --email "ajitabh@unixclinic.com" /home/ajitabhp/Downloads/jdk-7u25-linux-x64.tar.gz

    I got an error stating “No Matching Plugin was found”

    Just above this error message the plugin path was given for various plugins used by this utility – “make-jpkg”. The plugins were in /usr/share/java-package directory in the form of “.sh” files (shell scripts). I next opened each shell script to try and decipher them and found the possible problem in the plugins “/usr/share/java-package/oracle-j2*.sh”. The following pattern picked up from “/usr/share/java-package/oracle-j2sdk.sh” was the culprit –

    "jdk-7u"[0-9]"-linux-x64.tar.gz") # SUPPORTED
    j2se_version=1.7.0+update${archive_name:6:1}${revision}
    j2se_expected_min_size=180 #Mb
    j2se_priority=317
    found=true
    ;;

    If you see this pattern, it clearly indicates that it is only supporting up to update 9 of the JDK and the JDK I downloaded was update 25. So I added the following lines just below the above lines to add the support for my version.

    "jdk-7u"[0-9][0-9]"-linux-x64.tar.gz") # SUPPORTED
    j2se_version=1.7.0+update${archive_name:6:2}${revision}
    j2se_expected_min_size=180 #Mb
    j2se_priority=317
    found=true
    ;;

    There are two differences, one in first line and the second in second line. I am sure you will be able to spot them and decipher them.

    After making similar changes in the other oracle* files I ran the make-jpkg as mentioned above and it created a DEB file – oracle-j2sdk1.7_1.7.0+update25_amd64.deb, which is deploy-able with the usual “dpkg -i”.

    Changing the Default Java Installation to the Oracle Installation

    After installing the package, you may want to change some alternatives in order to make the Oracle JDK and associated JRE installations as defaults. As the default version of Java may be something different. You can check the current version of java and its provider by –

    $ java -version
    java version "1.6.0_24"
    OpenJDK Runtime Environment (IcedTea6 1.11.5) (6b24-1.11.5-1)
    OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
    $ file `which java`
    /usr/bin/java: symbolic link to `/etc/alternatives/java'
    $ ls -l /etc/alternatives/java
    lrwxrwxrwx 1 root root 46 Aug 16 20:18 /etc/alternatives/java -> /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java

    First find out the name of the available installations –

    $ /usr/sbin/update-java-alternatives -l
    j2sdk1.7-oracle 317 /usr/lib/jvm/j2sdk1.7-oracle
    java-1.6.0-openjdk-amd64 1061 /usr/lib/jvm/java-1.6.0-openjdk-amd64

    We will switch our default Java to “j2sdk1.7-oracle” with the following command –

    $ sudo /usr/sbin/update-java-alternatives -s j2sdk1.7-oracle

    Now you can check your Java version by

    $ java -version
    java version "1.7.0_25"
    Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
    Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
    $ file `which java`
    /usr/bin/java: symbolic link to `/etc/alternatives/java'
    $ ls -l /etc/alternatives/java
    lrwxrwxrwx 1 root root 41 Aug 16 20:22 /etc/alternatives/java -> /usr/lib/jvm/j2sdk1.7-oracle/jre/bin/java

  • Nokia Asha 310 – Sync Contacts With Google Contacts using SyncML

    Nokia Asha 310 Image
    Nokia Asha 310

    Today I swapped my aging 4 year old Nokia 5800 Xpress Music with Nokia Asha 310 (Dual SIM with WiFI) from a local mobile store. The phone after the swap cost me Rs 3600/-.

    From last more than 8 months or so Android is my primary phone and prior to that for close to two year it was Blackberry. Along with these two devices I was using Nokia 5800 Xpress Music and another Nokia 500 for my second phone.

    The primary requirement I had was to ensure that I find some way of syncing contacts with google contacts as my android is configured to do so. This provides me a backup of all my contacts at google and will ensure that I can enter/edit my contacts at one devices and they will be available everywhere. Also google provides ability  to recover contacts (upto a certain amount of time) in case they are accidently deleted.

    In order to sync contacts I used SyncML. Support for which is enabled by google from quite some time.

    In order to configure Nokia Asha 310 to sync with google contacts, follow the following steps. We need to start with creating a personal configuration option first –

    • settings -> configuration -> personal configuration -> add new
      • Specify the account name. Anything will work here as it is just an identifier text entry, in case you are syncing from  multiple sources. My account name is “Gmail Contacts”.
      • server address should be https://m.google.com/syncml
      • username must be your username including the domain name (e.g something@gmail.com). If you are using google apps then you should use that domain instead of gmail.com.
      • password must be your password. You will be asked to enter it twice for verification.
      • contacts database -> database address is the one which is important for us as we will be syncing with it. For Gmail (and Google Apps) you should specify ‘contacts’ here. There is no need to specify username and password again here.
      • calendar database -> database address should have the value of ‘calendar’. In case you do not want to sync calendar you can leave it blank.
      • notes database -> database address should have the value of ‘notes’. In case you do not want to sync notes you can leave it blank.
      • use pref. acc. point – Set this to either ‘Yes’ or ‘No’. In case you set it to ‘No’ the you need to setup the access point values. I have set it to ‘Yes’, so that whatever my preferred access point at the time of sync is, will be used.

    NOTE Database names is a place where SyncML protocol lacks standardisation as different providers use different names, so you must know the database name in case you have a different provider.

    Next the sync settings can be configured as below –

    • Settings -> sync & backup -> sync with server -> sync settings
      • synchronised data – select the databaes which needs to be synced here. I have all three databases viz. contacts, calendar and notes selected
      • sync settings – Since this is a dual SIM phone, you will have to select either SIM1 or SIM2 and the select the configuration profile which you created above viz. ‘Gmail Contacts’.
      • automatic sync – This can be set to daily, weekly, monthly or off modes. In case you set it to off then you need to initiate sync manually when you wish to sync the databases.
      • rules for incoming – There are three possible values – Always allow, Always reject, Confirm first. The meanings are obvious. The first will always allow the incoming data and sync it, the second will always reject incoming data (in this case changes on your phone will be sent, but changes from google server will not be received) and confirm first will prompt for appropriate action each time. My setting is ‘Always allow’
    • Settings -> sync & backup -> sync with server -> sync now can be used to initiate the manual sync.

    Hope this post will help interested folks.