January, 2014

Jan 14

Express.js dynamic route definitions

Express provides the bare necessities to bootstrap a web application and although due to its non-opinionated nature it is very flexible, bootstrapping a web application requires a bit of work, which isn’t available out of the box as with rails or another full stack web framework.

I’d like to have all my routes defined in a directory or resources and allow the url mappings to be defined in those files as well. There is some debate whether centralizing routing is beneficial, many frameworks (i.e. rails, play, etc…) do it. I think that there is a benefit to just writing a resource without having to also route it in a completely different file. Some java frameworks allow you to define url routes with annotations, though you write a resource, annotate it and you’re done. I personally like that style.

In order to make express follow some sort of convention by inferring the directory structure and allow you to define routings within the actual resource files, we came up with the following convention:

  1. The ‘routes’ directory will hold any .js files, which are basically resources. In those files, you can define your handlers and then provide a definition of how these handlers are routed.
  2. The directory can be arbitrarily nested
  3. There is convention of ‘routes/some/path/handler.js’ will be mapped to ‘/some/path/handler’ but can be overridden by doing your own route bindings
  4. Methods within the resource files can be mapped to the usual RESTful resource routing, though ‘some/path/handler’ with the HTTP methods, or you can follow the non-RESTful convention if needed, but modifying the path as needed, for example, you are more than welcome to do this: ‘some/path/handler/someMethod’.
    File-per-resource is nice, but sometimes you want might have resources that you want to group together.

Below is the code you need to bootstrap your application routing as described above. One can customize this as you wish, but this works for us right now. Also, there is a resource definition below that shows how the routing inference works and how you can define/customize routes.

Routes can be defined by using exports.routes in your resource file. The value can be on of the three below:

  1. A function with one argument will be called with that argument bound as the app object and you can do whatever route bindings you want yourself
  2. A function with no arguments will be called on to provide a data structure of bindings. The format of the data structure can be seen in the below resource files. Path is bound to an object of verb/handlers a. path can be relative or absolute. Absolute paths start with ‘/’, relatives do not b. path can be an underscore ‘_’, which basically a way to not specify any path for the resource and though it’ll be ‘path/to/resource’ with the proper verb dispatch
  3. You can also provide a data structure vs. a function that returns one. The only issue with that, is that you must then define your exports.routes after all the handlers, otherwise due to eager evaluation of javascript data structures, it’ll bind an undefined handler.

Jan 14

Cyclical dependency detection in the database

We recently had a need to find cyclical dependencies in our database. This happens to be a rather straightforward graph algorithm issue. The database foreign key constraints form a directed graph. Finding a cycle in a directed graph is mostly detecting an already visited node in a DFS algorithm (back-edge). We mark nodes as visited and if the ancestor of a node in the tree is already visited, then a back-edge (cycle) exists.

In order to do this on our own, we’d have to read the metadata from the database for each table, construct a directed graph using the foreign keys and then run the algorithm discussed above, rather straightforward. Most of the complexity comes from the cross cutting concerns of database metadata munging. We can easily accomplish all of the above using sqlalchemy and its ability to perform a topological sort on the reflected tables. Topological sort fails in there is a cycle detected and the exception thrown includes the nodes that produce the back-edge. Using this simple trick, we allow sqlalchemy to detect the cycles for us.

You’ll need to install sqlalchemy (and your db driver), networkx and graphviz (for visualization).