Home
Academic
Articles
Musings
Reviews
Technical
For Fun
Personal
Contact

Website Generation using Clojure


June 24, 2009

This website was designed and handcoded in a day using my current language of choice Clojure. The only two tools that were used in the creation of this website was a text-editor loaded with the Clojure SDK, and Adobe Photoshop for creating the images. Clojure is a lisp-based language that targets the Java Virtual Machine. It supports direct access to Java, and as such takes full advantage of Java's enormous library. The fact that this final webpage is less than three pages in length is good evidence of Clojure's productivity.

Overview
The framework for this website consists of a few parts. At the lowest level, there is a simple s-expression to html converter. I think html is too verbose and loose, and I too often forget to include closing tags or surround the appropriate items with quotation marks.

The next level is to use the s-expression to html converter to layout my webpage. I personally use tables and spacer gifs, and avoid CSS where I can. I'll explain why later.

And to generate the content, I wrote a small utility function that will let me embed Clojure code into a text file. The website script reads in the file and displays text according to a default style, and runs any code that might be embedded in the file.

S-Expression to HTML converter
Instead of writing:

<html font="Georgia">
  <title>
    My ScratchPad
  </title>
</html>

I wish to write:

(html_with {"font" "Georgia"}
  (title
    (write "My ScratchPad")))

The basic function that will allow us to do this is "html_tag".

(defblockfn html_tag [tag attributes block]
  (write "<" tag)
  (doseq [[name value] attributes]
    (write " " name "=" \" value \"))
  (write ">")
  (block)
  (write "</" tag ">"))

"write" is just a function that prints its arguments out to the standard output stream. It's essentially a "print" without separating spaces between its arguments.
"defblockfn" is a small macro that allows me to define Ruby style functions that take blocks as parameters.

(defblockfn myfunction [arg block]
  (println arg)
  (block))

Expands to:

(defn myfunction* [arg block]
  (println arg)
  (block))
(defmacro myfunction [arg & tail]
  `(myfunction* ~arg (fn [] ~@tail)))

So now that we have "html_tag", we can write code like this:

(html_tag "html" {"font" "Georgia"}
  (html_tag "title" nil
    (write "My ScratchPad")))

It's closer to what I want but it's not quite there yet. I'll define some convenience functions:

(defblockfn html [block]
  (html_tag* "html" nil block))
(defblockfn html_with [attributes block]
  (html_tag* "html" attributes block))

(defblockfn title [block]
  (html_tag* "title" nil block))
(defblockfn title_with [attributes block]
  (html_tag* "title" attributes block))

Now I'm done. And I can use these functions to write code like I envisioned. It's a little unwieldy to write these convenience functions for every single html tag that I want to use, so I wrap everything in a doseq.

(doseq [[name tag] [['html "html"]
                    ['head "head"]
                    ['style "style"]
                    ['title "title"]
                    ['body "body"]
                    ['table "table"]
                    ['row "tr"]
                    ['col "td"]
                    ['span "span"]
                    ['h1 "h1"]]]
  (eval `(defblockfn ~name [block#]
           (html_tag* ~tag nil block#)))
  (eval `(defblockfn ~(symbol (str name "_with")) [attributes# block#]
           (html_tag* ~tag attributes# block#))))

If I need more tags later on, I'll just add it to the list.

Laying out the Website
Now that the s-expression to html library is done, I can proceed to layout the website. I'll create a new layout, to give you an impression of the process.

Say I want a three column table layout. And each column can be filled with whatever content I felt like adding that day.

(defn create_page [column1 column2 column3]
  (write_file "scratchpad/generated/programming_column_layout.html"
    (with-out-str
      (html
        (head
          (title "My 3 Column Webpage"))
        (table_with {"border" 1
                     "width" "100%"}
          (row (col (write (slurp column1)))
               (col (write (slurp column2)))
               (col (write (slurp column3)))))))))

In my REPL, I just type (create_page "column1.txt" "column2.txt" "column3.txt")
and it automatically generates programming_column_layout.html for me. The website framework is almost doen.

Embedding Code to Generate Content
The last part of the framework is to allow content pages to embed Clojure code inside them to control formatting and generate dynamic content. The goal is to allow me to write something like the following in column3.txt and have the website framework generate a times table for me.

Behold my awesome Times Table! And I didn't even need to write it out by hand!
embed:(table_with {"border" 1}
        (doseq [i (range 1 6)]
          (row (doseq [j (range 1 6)]
                 (col (write i "x" j "=" (* i j)))))))

The "parse_embedded_file" function enables me to do that.

(defn nonempty [string]
  (if (> (.length string) 0) string))

(defn read_reader [reader]
  (let [stringbuffer (StringBuffer.)
        buffer (make-array Character/TYPE 1024)]
    (loop [num_chars (.read reader buffer 0 (alength buffer))]
      (when (>= num_chars 0)
        (.append stringbuffer (String/valueOf buffer 0 num_chars))
        (recur (.read reader buffer 0 (alength buffer)))))
    (.close reader)
    (.toString stringbuffer)))

(defn take_form [string]
  (let [reader (PushbackReader. (StringReader. string))]
    [(read reader) (read_reader reader)]))

(defn split_at_embed [string]
  (let [[whole half1 half2] (re-find #"(?s)(.*?)embed:(\(.*)" string)]
    (if whole
      [(nonempty half1) half2]
      [string nil])))

(defn parse_embedded_file [file default]
  (let [string (slurp file)]
    (loop [[string form_string] (split_at_embed string)]
      (when string (default string))
      (when form_string
        (let [[form rest_string] (take_form form_string)]
          (eval form)
          (recur (split_at_embed rest_string)))))))

"parse_embedded_file" takes two arguments. The first argument is a path to the file. The second argument is a default function that takes a string. What "parse_embedded_file" does is read the file until it reaches a "embed:(" statement. All the text up until that statement is passed to the "default" function for processing. Then the code inside the embed:( statement is evaluated.

Now I want to be able to process content pages with embedded code inside my 3 column layout website. It requires just a small change to the "create_page" function.

(defn create_page [column1 column2 column3]
  (write_file "scratchpad/generated/column_layout.html"
    (with-out-str
      (html
        (head
          (title "My 3 Column Webpage"))
        (table_with {"border" 1
                     "width" "100%"}
          (row (col (parse_embedded_file column1 write))
               (col (parse_embedded_file column2 write))
               (col (parse_embedded_file column3 write))))))))

I don't wish to do any processing of text. I just want to write it out, so I just pass the "write" function to "parse_embedded_file".

Now we generate the webpage again and our embedded code has created a times table automatically for us. Final Column Layout

Afterward
All the code that I demonstrated fits in under three pages. And now I have a website where I can update it's entire layout by rewriting one function. And my content pages can directly generate it's own content as necessary using embedded code. This is just a small demonstration of the power of Clojure.

  -Patrick