Monday, August 1, 2011

Pull Entire Directory Using SCP on a Mac

The following command can be used to fetch an entire directory including all the files from a remote host to a local directory.

$ scp -r username@host:/fully/qualified/path/on/remote/host/ /fully/qualified/path/on/local/host/

An alternative approach is to reference a directory relative to the user's account.

$ scp -r username@host:user's/path/on/remote/host/ user's/path/on/local/host/

Or using '.', the files will be copied to the current working direcory

$ scp -r username@host:user's/path/on/remote/host/ .

To use a wildcard to fetch only certain file types use the * syntax.

$ scp -r username@host:user's/path/on/remote/host/*.png .

Piece of Cake!!!

Thursday, July 28, 2011

Executing a Shell Script from Java

Using MySQL and Java? Check out an easier way: Yank

Today I figured out how to run a bash script from Java for the first time, and I wanted to jot down the essential steps needed to get it all working along with some things to watch out for. This is specific to mysqldump, but it should work for any script you have and want to run in the bash shell. This also demonstrates how to pass in an argument to the script.

Step 1: The first order of business in running a shell script from within a Java program is to setup a function that you can pass shell commands to. The following method takes a command as an argument and runs it in a bash shell. I put this method in a class called ExecUtils.

public static void execShellCmd(String cmd) {
        try {
            Runtime runtime = Runtime.getRuntime();
            Process process = runtime.exec(new String[] { "/bin/bash", "-c", cmd });
            int exitValue = process.waitFor();
            System.out.println("exit value: " + exitValue);
            BufferedReader buf = new BufferedReader(new InputStreamReader(process.getInputStream()));
            String line = "";
            while ((line = buf.readLine()) != null) {
                System.out.println("exec response: " + line);
            }
        } catch (Exception e) {
            System.out.println(e);
        }
    }

Step 2: Creating the script is pretty straight forward, but there are a few things to watch out for. For a bash script, you need #!/bin/bash on the first line. To pass an argument into the script, I used FILENAME=$1 and later used $FILENAME in my command. $1 represents the first argument. I also exit the script with the value returned from the mysqldump command.
#!/bin/bash
FILENAME=$1
exit mysqldump --user=username --password=pass --databases DB_XYZ | gzip -9 > temp/dump/$FILENAME
For this example, I placed my file, dump.sh in the directory '/temp'. One more thing to check is that your script is executable. Run this command in the terminal to make it executable:
sudo chmod 777 /temp/dump.sh

Step 3: Now just run the script through the execShellCmd method.
package mysql;

import java.util.Date;

import com.xeiam.utils.ExecUtils;

/**
 * This class demonstrates running a shell script from within Java
 */
public class MySQLDumpScript {

    public static void main(String[] args) {

        String fileName = "DUMPFILE.sql.gz";
        String shellCommand = "/temp/dump.sh " + fileName;
        ExecUtils.execShellCmd(shellCommand);
    }
}

You should now see a file called DUMPFILE.sql.gz in /temp/dump. Piece of Cake!!!

Executing mysqldump from Java and Pitfalls to Avoid

Using MySQL and Java? Check out an easier way: Yank

I just spent a lot of time figuring out how to run mysqldump from within my Java program. I came across several different problems. Each time it was not working, there was a different problem and solution, and never really easy to debug. Sometimes it would just not work at all. Sometimes it would create an empty file. In this blog, I document how I finally got it all to work and which pitfalls I fell into along the way.

Step 1: The first order of business in running mysqldump from within a Java program is to setup a function that you can pass shell commands to. The following method takes a command as an argument and runs it in a bash shell. I put this method in a class called ExecUtils.

public static void execShellCmd(String cmd) {
        try {
            Runtime runtime = Runtime.getRuntime();
            Process process = runtime.exec(new String[] { "/bin/bash", "-c", cmd });
            int exitValue = process.waitFor();
            System.out.println("exit value: " + exitValue);
            BufferedReader buf = new BufferedReader(new InputStreamReader(process.getInputStream()));
            String line = "";
            while ((line = buf.readLine()) != null) {
                System.out.println("exec response: " + line);
            }
        } catch (Exception e) {
            System.out.println(e);
        }
    }

Step 2: Now we just need to run the proper mysqldump command through our execShellCmd method. The following shellCommand String will backup the database DB_XYZ and write it to a file called /test.sql.gz.
private void mysqldump() {

        String shellCommand = "mysqldump --user=username --password=pass --databases DB_XYZ | gzip -9 > " + "/test.sql.gz";
        ExecUtils.execShellCmd(shellCommand);

    }

Pitfall 1: LOCK TABLES user permission. In order for the mysqldump command to work, the user which you define using the --user argument must have the LOCK TABLES permission. If that user does not have that privilege, you can add it with the following commands in MYSQL:

GRANT lock tables ON DB_XYZ.* TO 'username'@'localhost' IDENTIFIED BY 'pass';
GRANT lock tables ON DB_XYZ.* TO 'username'@'%' IDENTIFIED BY 'pass';
flush privileges;

Pitfall 2: False file permissions. In order for the /test.sql.gz to be written to disk, the user in which the Java program is running must have permission to write a file there. In this case: '/'.

Pitfall 3: mysqldump cannot be found on system PATH. In the command above I used the unqualified 'mysqldump' as the command to execute. You could just as easily replace 'mysqldump' with the fully qualified version: '/usr/local/mysq/bin/mysqldump', as in my case on my local machine. One possible problem with this approach though is that your Java code doesn't become portable to other platforms. For example, on one of my Linux machines, mysqldump is found here: '/usr/bin/mysqldump'. To be able to use just 'mysqldump' you have to make sure it is on the system PATH. This is how you do that...

On my Linux machine:
$ locate mysqldump
/usr/bin/mysqldump

OK, using the command 'locate' I determined that 'mysqldump' is located in the directory '/usr/bin'. Now let's check if '/usr/bin' is on PATH:
$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Yes, it is. No problem using 'mysqldump' directly.

On my Mac machine:
$ locate mysqldump
/usr/local/mysql/bin/mysqldump

OK, using the command 'locate' I determined that 'mysqldump' is located in the directory '/usr/local/mysql/bin'. Now let's check if '/usr/local/mysql/bin' is on PATH:
$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/git/bin

Nope. Now we need to add /usr/local/mysql/bin to PATH on Mac OS X (10.5 Leopard). Here's how I did that. I created a file in /etc/paths.d called mysqldump
$ sudo vim /etc/paths.d/mysqldump
and added the following text:
/usr/local/mysql/bin
and restarted my computer.

Now let's check again if '/usr/local/mysql/bin' is on PATH:
$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/git/bin:/usr/local/mysql/bin

Yup, all good!


Monday, May 9, 2011

Intel's Tri-Gate Process for Their Future Transistors

A Random Tomcat Error

Want to integrate charts into your webapp? Check out XChart.

Here's another nano-blog that will hopefully help someone solve a Tomcat error that I was dealing with today:

java.lang.NoSuchMethodException: org.apache.catalina.deploy.WebXml addServlet

The problem was that I was deploying catalina.jar within my .war file. Once I fixed up my build file to not include the catalina.jar file in the .war file, the error went away.

Saturday, May 7, 2011

Java One-Jar Example Using Ant and Eclipse

One-Jar is an amazingly clever and simple tool that let's you package an entire Java application containing external jar files all into one Jar. But just like anything, it's simple and easy to use only AFTER you figure out how to get it working. Having just worked through using One-jar, I thought I'd lay out the steps needed to successfully use One-jar in Eclipse using Ant.

Step 1: Get One-jar. Go to the One-jar download page on Source-Forge and download the "one-jar-ant-task-0.97.jar" file.


Step 2: Integrate this jar into Eclipse so that when running Ant, Ant has access to the one-jar Ant Task. If you don't do this, you may get and error like:

/eclipse/helloworkspace/HelloOneJar/build.xml:54: Problem: failed to create task or type one-jar
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any / declarations have taken place.

Move the jar to your Eclipse plugins folder. For me, that was in /eclipse/plugins. You can really put this anywhere you want, but it made sense for me to put it here as Eclipse stores a lot of jars it needs here.

In Eclipse, open preferences, drill down to Ant-->Runtime, highlight Ant Home Entries, and click "Add external JARs...". Find the one-jar JAR, and add it here.


Step 3: Create Ant build file. For demonstration and testing purposes, I created a tiny Java Project called HelloOneJar which contains a Hello class. All the Hello class does is log two messages using log4j. Here is a screenshot of the project structure and the Hello class:


build.properties

project.name=hello-onejar
lib.dir=lib
src.dir=src
build.dir=build
dist.dir=dist

build.xml

<?xml version="1.0"?>
<project name="hello-onejar" default="onejar" basedir=".">
 
    <taskdef name="one-jar" classname="com.simontuffs.onejar.ant.OneJarTask" onerror="report" />
 
    <property file="build.properties"/>
 
    <tstamp>
       <format property="timestamp" pattern="yyyy-MM-dd HH:mm:ss" />
    </tstamp>
 
    <path id="classpath">
     <fileset dir="${lib.dir}" />
    </path>

    <target name="clean">
        <echo>Cleaning the ${build.dir}</echo>
        <delete dir="${build.dir}"/>
        <delete dir="${dist.dir}"/>
    </target>

    <target name="init" depends="clean">
        <echo>Creating the build directory</echo>
        <mkdir dir="${build.dir}"/>
        <mkdir dir="${dist.dir}"/>
    </target>
 
    <target name="compile" depends="init">
        <echo>Compile the source files</echo>
        <javac srcdir="${src.dir}" destdir="${build.dir}" debug="on">
            <classpath refid="classpath"/>
        </javac>
    </target>

 <target name="mainjar" depends="compile">
       <jar jarfile="${dist.dir}/main.jar">
         <manifest>
            <attribute name="Built-By" value="${user.name}"/>
             <attribute name="Build-Date" value="${timestamp}"/>                 
             <attribute name="Main-Class" value="com.hello.Hello"/> 
           </manifest>
          <fileset dir="${build.dir}">
              <include name="**/*.class"/>
          </fileset>
        </jar>
 </target>

    <target name="onejar" depends="mainjar">
     
        <!-- Construct the One-JAR file -->   
  <one-jar destfile="${dist.dir}/${project.name}.jar">
         <main jar="${dist.dir}/main.jar">
         </main>
         <lib>
             <fileset dir="${lib.dir}" />
         </lib>
  </one-jar>
    </target>
 
</project>
The follwing parts are very important!

<taskdef name="one-jar" classname="com.simontuffs.onejar.ant.OneJarTask" onerror="report" />
without that line, the one-jar task is undefined. Remember to tell Ant where this class is in step 2!

<attribute name="Main-Class" value="com.hello.Hello"/>
without this in the manifest you may get an error like this when you run your jar:
Exception in thread "main" java.lang.Exception: hello-onejar.jar main class was not found (fix: add main/main.jar with a Main-Class manifest attribute, or specify -Done-jar.main.class=), or use One-Jar-Main-Class in the manifest
at com.simontuffs.onejar.Boot.run(Boot.java:327)
at com.simontuffs.onejar.Boot.main(Boot.java:168)

Step 4: Run the Ant build. In this example, I run the "onejar" task that I defined. This build file first creates a jar called main.jar that contains my application code. This jar has it's own manifest and could be ran separately. The onejar target wraps main.jar in hello-one-jar.jar and packages all the jar(s) in the lib folder inside. Both jars are created in the ./dist folder.

Step 5: Inspect the built jar. This is how I did this on my Mac. First I created a folder: /HelloOneJar and moved the hello-onejar.jar into it. Then, in terminal:
cd /HelloOneJar
jar -xvf hello-onejar.jar
Now the jar is unpacked and we can see what's in it. You can do the same for main.jar and verify that the manifest file was created correctly.

Step 6: Run the jar and make sure it all works. This is how I did this on my Mac. First I created a folder: /HelloOneJar and moved the hello-onejar.jar into it. Then, in terminal:
cd /HelloOneJar/
java -jar hello-onejar.jar

The following output appeared in Terminal:
0 [main] INFO com.hello.Hello - Hello.
1 [main] INFO com.hello.Hello - Bye Bye.

Exactly what I expected!

Piece of Cake!!

Thursday, May 5, 2011

Get Day of Week from Date Object in Java

To get the day of the week from a java.util.Date object, we can use the java.text.SimpleDateFormat class. First, create a Date object. Next, create a SimpleDateFormat instance using the getDateInstance() method, passing the String "E" or "EEEE" as the argument. Get the day of the week as a String using the format() method passing in the java.util.Date object. To get the weekday in numerical format, we can use the java.util.Calendar object's get method passing in Calendar.DAY_OF_WEEK.

Get Day of Week from Date in Java - Example Code

//
// The following example code demonstrates how to
// print out the Day of the Week from a Date object.
//
public class GetDayFromDate {

    public static void main(String[] args) {

        Date now = new Date();

        SimpleDateFormat simpleDateformat = new SimpleDateFormat("E"); // the day of the week abbreviated
        System.out.println(simpleDateformat.format(now));

        simpleDateformat = new SimpleDateFormat("EEEE"); // the day of the week spelled out completely
        System.out.println(simpleDateformat.format(now));

        Calendar calendar = Calendar.getInstance();
        calendar.setTime(now);
        System.out.println(calendar.get(Calendar.DAY_OF_WEEK)); // the day of the week in numerical format

    }
}

Here is the output of the example code:
Wed
Wednesday
4

Wednesday, May 4, 2011

Get Month from Date Object in Java

To get the month from a java.util.Date object, we can use the java.text.SimpleDateFormat class. First, create a Date object. Next, create a SimpleDateFormat instance using the getDateInstance() method, passing the String "MM", "MMM", or "MMMM" as the argument. Finally, get the month as a String using the format() method passing in the java.util.Date object.

Get Month from Date in Java - Example Code

import java.text.SimpleDateFormat;
import java.util.Date;

//
//The following example code demonstrates how to
//print out the Month from a Date object.
//
public class GetMonthFromDate {

    public static void main(String[] args) {

        Date now = new Date();
        SimpleDateFormat simpleDateformat = new SimpleDateFormat("MM"); // two digit numerical represenation
        System.out.println(simpleDateformat.format(now));

        simpleDateformat = new SimpleDateFormat("MMM"); // three digit abbreviation
        System.out.println(simpleDateformat.format(now));

        simpleDateformat = new SimpleDateFormat("MMMM"); // full month name
        System.out.println(simpleDateformat.format(now));

    }

}

Here is the output of the example code:
05
May
May

Monday, May 2, 2011

Port Forwarding (80 to 8080 for Tomcat) Using IPFW on Mac OSX

Here I show how easy it is to set up port forwarding (80 to 8080 for Tomcat) on Mac OSX using ipfw from the Terminal. IPFW is the built-in firewall of Mac OSX and we can quickly set up a firewall rule to allow port forwarding...


Step 1: View current firewall rules.
sudo ipfw show

Step 2: Add port forwarding rule (80 to 8080 for Tomcat)
sudo ipfw add 100 fwd 127.0.0.1,8080 tcp from any to any 80 in

If you want to remove your firewall rules run:
sudo ipfw flush


Piece of Cake!!!