magpiebrain

Sam Newman's site, a Consultant at ThoughtWorks

Updated to reflect some feedback and one example of using commons-exec as an alternative to the plain old Runtime.exec

Second Update to reflect use of shell-out – thanks Scott!

Basic

Making use of clojure.contrib.duck-streams:

(ns utils
 (:use clojure.contrib.duck-streams))

(defn execute [command]
  (let [process (.exec (Runtime/getRuntime) command)]
    (if (= 0 (.waitFor  process))
        (read-lines (.getInputStream process))
        (read-lines (.getErrorStream process)))))

...
user=> (execute "ls")
("MyProject.iml" "lib" "out" "src" "test")

It could be improved obviously – for example catching some of the potential IOExceptions that can result to rethrow additional information, such as the command being executed, or the ability to take a seq of program arguments.

Error & Argument Handling

This version adds some basic (and ugly) exception handling, and also handles spacing out arguments passed in (so passing "ls" "-la" gets processed into "ls -la"):

(defn execute
  "Executes a command-line program, returning stdout if a zero return code, else the
  error out. Takes a list of strings which represent the command & arguments"
  [& args]
  (try
    (let [process (.exec (Runtime/getRuntime) (reduce str (interleave args (iterate str " "))))]
      (if (= 0 (.waitFor  process))
          (read-lines (.getInputStream process))
          (read-lines (.getErrorStream process))))
    (catch IOException ioe
      (throw (new RuntimeException (str "Cannot run" args) ioe)))))

Using commons-exec

I had some problems with hanging processes, so knocked up a version using Apache’s commons-exec. This version has the added advantage of killing long-running processes, and I folded in Steve’s suggestion for a better way of splicing in the spaces in the command line args (see his comment). commons-exec is part of the special sauce inside Ant, so is a rock solid way of launching command-line processes (well, as rock solid as Java gets).

The use of the ByteArrayOutputStream is probably inefficient, and again, decent error handling is left as an exercise to the reader.

(defn alternative-execute
  "Executes a command-line program, returning stdout if a zero return code, else the
  error out. Takes a list of strings which represent the command & arguments"
  [& args]
  (let [output-stream (new ByteArrayOutputStream)
        error-stream (new ByteArrayOutputStream)
        stream-handler (new PumpStreamHandler output-stream error-stream)
        executor (doto
                  (new DefaultExecutor)
                  (.setExitValue 0)
                  (.setStreamHandler stream-handler)
                  (.setWatchdog (new ExecuteWatchdog 20000)))]
     (if (= 0 (.execute executor (CommandLine/parse (apply str (interpose " " args)))))
       (.toString output-stream)
       (.toString error-stream))))

Using clojure.contrib.shell-out

Many thanks to Scott for this. clojure.contrib supplies the very neat shell-out:

user=> (use 'clojure.contrib.shell-out)
nil
user=> (sh "ls" "-la")

I haven’t probed further to see if this deals with my hanging process problem, but it certainly doesn’t seem to have any support for killing timeout processes. If you’re worried about runaway tasks, the commons-exec version above might be the right choice for you.

About these ads

10 Responses to “Executing A Command Line Program With Clojure”

  1. Steve

    To splice together the command line args into one string you can simply write:
    (apply str (interpose ” ” args))

    Reply
  2. Scott

    I didn’t read your post, so feel free to delete this comment if it’s not relevant. But from a glance it appears you missed this one:

    (use ‘clojure.contrib.shell-out)
    (seq (.split (sh “ls”) “n”))

    (sh “ls” “-la”) is also valid.

    Reply
  3. Sam Newman

    Very relevant – thanks Scott. I’ve updated the article accordingly.

    Reply
  4. Asim Jalis

    Another approach is to wrap shell-out’s sh with Clojure’s future-call function.

    For example, here is a call to “p4 dirs //app/1*” which sometimes hangs for upto one minute when there is no network connectivity. Wrapped inside a future-call this cleanly times out after 1 second, and throws a java.util.concurrent.TimeoutException.

    (.get (future-call #(sh “p4″ “dirs” “//app/1*”)) 1 (java.util.concurrent.TimeUnit/SECONDS))

    You can abstract this out to create a sh with timeout.

    (defn sh-timeout [timeout-in-seconds & args]
    (.get
    (future-call #(apply sh args))
    timeout-in-seconds
    (java.util.concurrent.TimeUnit/SECONDS)))

    Reply
  5. Asim Jalis

    Also here is the use command to make sure “sh” is in scope for the method I just posted. Here is the complete source.

    (use ‘[clojure 1=”:only” 2=”[sh" language=".contrib.shell-out"][/clojure]])
    (defn sh-timeout [timeout-in-seconds & args] (.get (future-call #(apply sh args)) timeout-in-seconds (java.util.concurrent.TimeUnit/SECONDS)))

    Reply
  6. Asim Jalis

    (use ‘[clojure 1=”:only” 2=”[sh" language=".contrib.shell-out"][/clojure]])
    (defn sh-timeout [timeout-in-seconds & args]
    (.get
    (future-call #(apply sh args))
    timeout-in-seconds
    (java.util.concurrent.TimeUnit/SECONDS)))

    Reply
  7. Matthew

    In case anyone is reading this and scratching their head as I was – in Clojure 1.3 the exception thrown by Asim’s method is no longer a TimeoutException but a RuntimeException wrapping a TimeoutException.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Basic HTML is allowed. Your email address will not be published.

Subscribe to this comment feed via RSS

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: