Character Networks with Go and d3

I've had the opportunity to create a practical application in Go. I'm working on a research project in grad school that involves constructing character networks from literature text. After In order to get a grasp on what we might be able to "say" about networks in the corpus.

After pre-processing the text with Stanford's Named Entity Recognizer, I then searched the text for people, and began constructing the relationships. Details on the methods will be upcoming. First I wrote the processing code in R, and it took around 10 seconds to run. After rewriting in Go, the processing takes less than one second. I don't think it should be a surprise that it performs better. The important part is that this allows me to create a tool that will update parameters of the algorithm and update the network for display in real time.

Right now the display is being done in using D3's force layout. I think it should work out, but there are some kinks that I need to resolve first.

More to come.

WAID server

Bare Bones Martini App

Its pretty easy to add handlers to routes. And because the Handlers are simply functions of a specific type, it is easy to separate these into different files. Martini also has some helpers that make it trivial to deal with basic stuff like json request, and parameters. Below are a couple of route examples.

I'm using gorp to handle my database connections

// index route
m.Get("/entries", func(r render.Render) {
 r.JSON(200, entry.All(dbMap))
})

// create route
m.Post("/entries", binding.Json(entry.Entry{}), func(params martini.Params, e entry.Entry, r render.Render) {
 err := dbMap.Insert(&e)
 if err != nil {
  r.JSON(404, "Unable to update entry.")
  return
 }
 r.JSON(200, e)
})

Adding routes for other operations is pretty easy once the first couple are working.

Creating arbitrary HTTP request types.

Go's http library has some built in methods for sending GET and POST requests, but not for DELETE, PUT, or PATCH. I added a simple helper function that creates and sends a request of arbitrary type, then stuffs the response data into a passed interface

func jsonRequest(reqType string, path string, v interface{}) {
 var err error
 client := http.Client{}

 //prepare json data to send to the server
 jsonData, err := json.Marshal(v)
 doPanic(err)
 body := bytes.NewBuffer(jsonData)

 // create req uest
 req, err := http.NewRequest(reqType, fmt.Sprintf("%s%s", SERVER_URL, path), body)
 doPanic(err)
 resp, err := client.Do(req)
 doPanic(err)

 // error status code
 if resp.StatusCode >= 400 {
  panic(fmt.Sprintf("Request %d: %s:", resp.StatusCode, resp.Status))
 }

 // set returned "1data to interface, this will crash if the types aren't good
 entryData, err := ioutil.ReadAll(resp.Body)
 doPanic(err)

 // construct json data as requested struct
 json.Unmarshal(entryData, &v)
}

Go's json marshal and unmarshall made this very easy. A flaw I notice right now, is that the method assume that you'll want the submitted object to be modified by the server, this doesn't apply to DELETE requests (they should send back an empty response). But really this is still an exercise in learning Go for me.

Handing JSON payload with Martini

Initially I was trying to use Martini's form handler for the requests, but it was easier to write the client code for requests with a json payload. Luckily Martini also can handle JSON just as easily.

The net affect is that the client is much simpler, and separating the data handling and the interface has made adding new features easier.

Future Work

The database configurations is pretty funny. It's just a sqlit3 file in the user's home directory. I'd like to make that slightly more sophisticated. I'd also like to add some config files so that the user can, you know, configure that application without recompile. After those goals have been met. A deployment scheme is next.

WAID - What Am I Doing?

WAID repo

Recently my workplace started being serious about requiring time tracking. While I understand the numerous benefits, I'm not a fan of actually doing it. A way to turn it into an opportunity was to create my own time tracking application. So a few months ago I started waid. Waid (What am I doing?) as yet another command line time tracker. All that makes this project special to me is that I'm writing it, and I'm writing it in Go.

I've been interested in Go since I saw some talks from Rob pike at Google IO in 2012. However, At this point for me profess appreciation go for any thing other than Go's aesthetics would be bullshit. I haven't worked in the language long enough or written anything that compares to the size of applications I work on in Ruby. I like a lot of the ideas that Go incorporates, but my appreciation is superficial. Waid presented a perfect opportunity to do something not quite trivial in go.

At this point waid handles all the stuff I wanted it to do initially. It can: start and stop a time, add an entry, clear a list, list my time with a summary. And I've gained some valuable experience, and some basics of creating something larger than a hello world program in go. The verdict is that I have a long way to go until I understand how larger Go programs are put together, and what proper Go code looks like. But whatever, its a different paradigm, and I'll just need more experience.

Future work

Luckily there are some things that I'm looking to add/fix such as richer entry editing, and handling. Most especially I want to make a companion server application that keeps the entries the same across multiple computers.

I'm really excited. This will expose me to a lot of aspects of writing we applications in Go that I wasn't sure how to approach before. I'm looking forward to learning about stuff like: deployment, configuration, authentication, etc.

More to come.

Getting the Hang of Go

I'm starting to really enjoy writing go. I've been working on a command line app that I'll "release soon" (it's already free(dom) but I'll write something up about it), and I'm starting to get familiar with the basics of writing a non trivial application. I haven't had the opportunity to explore the parts of go that make it really awesome (concurrency in particular), but I already see some advantages.

The process of updating and install the source and binary across all the machines I've been using has been very easy. No need to write a gem file or any similar package. It's all built into the go command.

Now that I'm familiar with the typing system I can make changes to improve the program and with confidence expect the complier not to yell at me.

In short, go is fun so far, easy to create command line applications, even light weight ones. I'm definitely going to find more ways to use it.