Posts Tagged: python

Uncategorized — 3 Comments
21
Jun 10

Lazy cheap flight calculations with priority queues

There is an interesting problem of utilizing priority queues to figure out the best price combination in a set of flight legs. The problem is as follows:

We need to calculate the cheapest combination of flight legs (connections) for a flight to a particular destination. We’re given a price ordered N set of flight legs and we need to find the winning combination. Each combination would be evaluate for eligibility and would either pass or fail, so the cheapest combination doesn’t necessarily reflect the cheapest possibly combination of prices from the legs. A black box predicate function is consulted to ensure the combination is eligible. This reflects various airline rules, like overlapping times, specials that are only available to certain people, routes, or connections.

Solution: A naive approach for say a two leg flight is to say construct a (n x m) ordered matrix and evaluate each priced ordered combination through the black box predicate routing until one passes. The problem with this approach is that we unendingly construct a full matrix when in many cases one of the first combinations is enough to present the cheapest “valid” price. The key to reducing this is to construct a lazy data structure which will prioritize the cheapest flights and can then be iterated to find one that’s valid. We do so at runtime while constructing matrix combinations. The solutions is generalized, so the same can be used for n leg flights.

The algorithm goes something like this…

Construct the first set of combinations which can reflect the cheapest flight. The first cheapest combinations is always n1 + m1. If that doesn’t pass, the next possible set of cheapest combinations is either n2 + m1 or n1 + m2. We then continue to n1 + ma and na + m1, where a is incremented until the end of the route leg set for either leg.

The worst case running time is quadratic O(n2), but because of the lazy data structure, the algorithm runs in rather constant time, depending on how lucky we are that the first few combinations will yield a “rule valid” price combination.

This problem idea came from reading The Algorithm Design Manual by Steven S. Skiena. I recommend this book for anyone wishing to delve into the world of more advanced algorithm design.

Here is the solution in python. You’ve probably noticed I’ve been using a lot of python. Besides the fact that I like the language, python is an incredibly good language for conveying algorithmic ideas in a concise but very readable way.

The only two functions that matter, are cheapest_price and _pick_combo, the rest are just auxiliary functions used to support an OO structure and running a sample.

  import heapq, random, time

  class Route(object):
      """docstring for TicketFinder"""
      def __init__(self):
          self.heap = []
          self.unique = dict()
          self.legs = []
          self.max_leg_len = 0
          self._counter = 0
          self._loop_counter = 0

      def add_leg(self, leg):
          leg.sort()
          self.legs.append(leg)
          leg_len = len(leg)
          if leg_len > self.max_leg_len:
              self.max_leg_len = leg_len

      def cheapest_price(self, pred_func=lambda x: True):
          for i in range(0, self.max_leg_len):
              combo = self._pick_combo(i, pred_func)
              if combo: return combo

      def print_stats(self):
          print("""Legs: %s
  Combos examined: %s
  Loops: %s
  """ % (len(self.legs), self._counter, self._loop_counter))

      def _pick_combo(self, curr_idx, pred_func):
          num_legs = len(self.legs)
          price_combo = [ leg[curr_idx] for leg in self.legs if not curr_idx >= len(leg) ]
          self._add_combo(price_combo)
          cheapest_price = self._eval_price_combo(pred_func)
          if cheapest_price: return cheapest_price
          for idx in range(1, self.max_leg_len-curr_idx):
              for j in range(0, num_legs):
                  if len(self.legs[j]) &lt= (curr_idx+idx): continue
                  combo = []
                  for k in range(0, num_legs):
                      self._loop_counter += 1
                      if j == k:
                          combo.append(self.legs[k][curr_idx+idx])
                      elif curr_idx &lt len(self.legs[k]):
                          combo.append(self.legs[k][curr_idx])
                  self._add_combo(combo)

              cheapest_price = self._eval_price_combo(pred_func)
              if cheapest_price: return cheapest_price

      def _add_combo(self, combo):
          self._counter += 1
          if len(combo) == len(self.legs) and not self.unique.has_key(str(combo)):
              heapq.heappush(self.heap, combo)
              self.unique[str(combo)] = True

      def _eval_price_combo(self, pred_func):
          for i in range(0, len(self.heap)):
              least_combo = heapq.heappop(self.heap)
              if pred_func(least_combo):
                  print("Winning combo: %s" % [ "%.2f" % l for l in least_combo ])
                  return sum(least_combo)
          return None


  ############### Samples below ##################

  def sample_run(num_legs, pred_func):
      print(("#" * 30) + " Sample Run " + ("#" * 30))
      route = Route()
      for i in range(0, num_legs):
          route.add_leg( [ random.uniform(100, 500) for i in range(0, 100) ] )

      start = time.clock()
      price = route.cheapest_price(pred_func)
      calc_time = time.clock() - start

      if price:
          print("Cheapest price: %.2f" % price)
      else:
          print("No valid route found")
      route.print_stats()
      print(("#" * 72) + "\n")

  if __name__ == '__main__':
      sample_run(2, lambda x: True)
      def pred(x):
          for price in x:
              if price &lt 150: return False
          return True
      sample_run(3, pred)

I haven’t thoroughly tested this for correctness besides numerous runs and some basic validation so let me know if you see anything apparently wrong here.

Running the above yields

    ############################## Sample Run ##############################
    Winning combo: ['103.62', '106.40']
    Cheapest price: 210.03
    Legs: 2
    Combos examined: 1
    Loops: 0

    ########################################################################

    ############################## Sample Run ##############################
    Winning combo: ['150.74', '150.25', '173.95']
    Cheapest price: 474.95
    Legs: 3
    Combos examined: 2852
    Loops: 8523

    ########################################################################

For the first sample run, we use a predicate function which yields True, so we never examine anything other than the first combo n1 + m1. For the second sample, I add a predicate function which only accepts any price combination where all legs are above $150. (Of course this is not anything resembling airline rules, just good enough to simulate some sample cases, where the first n combinations are rejected). In the second sample run, we utilized 3 legs and examined 2852 combinations before coming up with the winning leg combination for the route. Each price within the combination is the smallest possible price above $150 for each leg.

Uncategorized — No Comments
27
May 10

Random points in polygon generation algorithm

I needed to generate a set of random points within a polygon, including convex and concave. The need arouse in a geospatial domain where polygons are rather small (on a geo-scale) and wouldn’t span more than say 10 miles, though the benefit of employing more complex algorithms to deal with spheroid properties are negligible. Plane geometry provided enough to meet this requirement. Point-in-Polygon tests are rather simple and are used to test whether a point exists in a polygon. The test is performed using a Ray casting algorithm which test the intersections of a ray across the x-axis starting from the point in question.

Another concept is the Minimum Bounding Rectangle (Bounding Box), which is the minimal rectangle needed to enclose a geographical object (i.e. polygon).

So, one can generate random points within a polygon by…

  1. Generating a bounding box
  2. Generating a point within the bounding box. This is a simple algorithm.
  3. Using Point-in-Polygon to test whether this point exists within the polygon.

Because of the random sampling nature and false positives from step 2, which must be tested in step 3, the above must be performed in a loop until the Point-in-Polygon test passes.

This works quite well for generating test data, as there are no tight bounds on the performance characteristics of random generation. One could also use the above algorithm in production as long as the ration of polygon to bounding box is rather large, which is usually the case for convex polygons. The ratio might be too small convex polygons, though causing a more than acceptable number of false positives in step #2.

I’ve implemented this in the geo-utils python package and made available on github. Feel free to use and provide any feedback.

To utilize the geo-utils to generate random points within a polygon, you would do the following:

  from vtown import geo
  from vtown.geo.polygon import Polygon


  polygon = Polygon(  geo.LatLon(42.39321,-82.92114),
                      geo.LatLon(42.39194,-82.91669),
                      geo.LatLon(42.39147,-82.91796),
                      geo.LatLon(42.39090,-82.91974),
                      geo.LatLon(42.39321,-82.92114))

  point = polygon.random_point()

The above polygon is generated using lat/lon coordinates, but you can generate them using simple x/y coordinates with geo.Point(x,y)

Here are some code snippets from the implementation. I only pasted the relevant parts. For boilerplate and relevant data structures, see the geo-utils package.

class BoundingBox(object):

    def __init__(self, *points):
        """docstring for __init__"""
        xmin = ymin = float('inf')
        xmax = ymax = float('-inf')
        for p in points:
            if p.x < xmin: xmin = p.x
            if p.y < ymin: ymin = p.y
            if p.x > xmax: xmax = p.x
            if p.y > ymax: ymax = p.y
        self.interval_x = Interval(xmin, xmax)
        self.interval_y = Interval(ymin, ymax)

    def random_point(self):
        x = self.interval_x.random_point()
        y = self.interval_y.random_point()
        return Point(x, y)

class Polygon:
  ## __init__ omitted here...

  def contains(self, point):
        seg_counter = private.SegmentCounter(point)
        for i in range(1, len(self.points)):
            line = Line(*self.points[i-1:i+1])
            if seg_counter.process_segment(line):
                return True
        return seg_counter.crossings % 2 == 1

  def random_point(self):
        bb = BoundingBox(*self.points)
        while True:
            print("GENERATING RANDOM POINT...")
            p = bb.random_point()
            if self.contains(p):
                return p

class SegmentCounter(object):

    def __init__(self, point):
        self.point = point
        self.crossings = 0

    def process_segment(self, line):
        p, p1, p2 = self.point, line.point1, line.point2
        if p1.x < p.x and p2.x < p.x:
            return False

        if (p.x == p2.x and p.y == p2.y):
            return True

        if p1.y == p.y and p2.y == p.y:
            minx = p1.x
            maxx = p2.x
            if minx > maxx:
                minx = p2.x
                maxx = p1.x
            if p.x >= minx and p.x <= maxx:
                return True
            return False


        if ((p1.y > p.y) and (p2.y <= p.y)) \
                or ((p2.y > p.y) and (p1.y <= p.y)):
            x1 = p1.x - p.x
            y1 = p1.y - p.y
            x2 = p2.x - p.x
            y2 = p2.y - p.y

            det = numpy.linalg.det([[x1, y1], [x2, y2]])
            if det == 0.0:
                return True
            if y2 < y1:
                det = -det

            if det > 0.0:
                self.crossings += 1
Uncategorized — 2 Comments
17
May 10

Divide and conquer for exponentiation

Here is an awesome way to demonstrate divide and conquer algorithm performing exponentiation. Naive exponentiation algorithms xn would perform n-1 multiplications as n x n … x n-1. This has an algorithmic complexity of O(n) which of course scales poorly for any significantly large number. This is not even including the overhead of performing integer multiplication beyond CPUs capacity is slower than staying within the CPU integer range. Now, do that n times and you have a problem.

Logarithmic performance O(log n) is one of the best common algorithmic complexities there is (outside of constant complexity of course, which is rare). One can achieve calculating power by utilizing the power of logarithms, which are clearly apparent in divide and conquer problem solutions.

Logarithms grow very slow compared to number of inputs, though for a calculating a power of say n1000000, with the naive algorithm, you’d have to perform 999,999 multiplications. With a logarithmic complexity algorithm this drops to log21000000 = ceil(19.93) = 20 steps. 20 steps with a few extra operations for step compared to 1million multiplications.

Here is an example of both exponentiation algorithms, the logarithmic complexity and linear complexity (called naive), as well as built in python pow() function. Both our logarithmic power function and python’s built in one perform the same, where the naive linear function starts to truly deteriorate once any reasonable number is used as the exponent.

_Note: this function is recursive though you can run out of stack space for very large exponents (you can also easily reimplement it as recursion). On a system with a 1024 stack limit, this would mean your exponent would have to be above 21024 or

17976931348623159077293051907890247336179769789423065727343008 11577326758055009631327084773224075360211201138798713933576587 89768814416622492847430639474124377767893424865485276302219601 24609411945308295208500576883815068234246288147391311054082723 7163350510684586298239947245938479716304835356329624224137216

before you run out of stack space._

Here is a benchmarked python implementation. The relevant algorithm part is highlighted.

#!/usr/bin/env python
import math
import time
import sys

def power(b, e):
    """logarithmic divide/conquer algorithm"""
    if e == 0: return 1
    x = power(b, math.floor(e/2))
    if e % 2 == 0: return pow(x, 2)
    else: return b * pow(x, 2)

def naive_power(b, e):
    """linear power algorithm"""
    x = b;
    for i in range(1, e):
        x *= b
    return x

def perform(name, base, exp, pfunc):
    print("%s: %d^%d: %d" % (name, base, exp, pfunc(base, exp)))

if __name__ == '__main__':
    if len(sys.argv) != 3:
        sys.exit("You must provide a base and an exponent.  (Usage: exp.py base exp)")
    base = int(sys.argv[1])
    exp = int(sys.argv[2])
    for func in (power, naive_power, pow):
        print("Benchmarking %s..." % func.__name__)
        bench = []
        for i in range(0,5):
            start = time.time()
            ans = func(base, exp)
            end = time.time()
            bench.append(end-start)
        print("\tCalculated in: %s" % min(bench))
]]>

Running above to calculate 2200000

$ python exp.py 2 200000
Benchmarking power…
    Calculated in: 0.0042099952697753906
Benchmarking naive_power…
    Calculated in: 6.078423023223877
Benchmarking pow…
    Calculated in: 0.0041148662567138672

Hmmm, both pow() (python’s built in power) and power() (logarithmic complexity) calculated the power in 4 millis (above is in seconds) and our naive_power() function calculates the same result in 6 seconds.

I tried running the script to calculate 21000000, which calculated using logarithmic functions in 25 milliseconds and I killed the naive_power() calculation after a few minutes of impatiently waiting for it to complete.

Power to the logarithms!!! :-)

Uncategorized — 3 Comments
12
Apr 10

Python and mod_wsgi on Mac OS X Snow Leopard (oy vey)

I’ve been dabbling with Python Turbogears in the last week. Turbogears is a great framework so far. My biggest like is the loose coupling, allowing you to choose the best component for the job. I’ll write a more detailed blog about the other RAD web development frameworks out there sometime this week, but my biggest dislike of say Rails, Django, and some others, is how they tie you into using their integrated components. You can of course hackishly disable their use/dependency, but not without losing many other features of the framework. In my experience, these type of frameworks are great at getting something up quickly, but they suck when it comes to long term scalability and growth, as you basically end up rewriting the framework to integrate other 3rd party or your own components to the point that it marginalizes the benefits of the framework.

I found out that building Python on a Mac (OS X Snow Leopard) is nearly impossible. You can definitely compile it, but I needed it compiled to work with mod_wsgi, and various modules. I also needed to compile mod_wsgi against a particular version of apache, which required me to compile python as a universal binary to support i386 and x86_64 architectures. That’s where it started to get painful. Snow Leopard is distributed with Python 2.6.1 and supports a 3-way architecture (ppc7400, i386, and x86_64), but the latest python release is 2.6.5, so I tried to compile it. I used a myriad of options…

./configure --prefix=/usr/local/python-2.6.5 \
--enable-framework=/usr/local/python-2.6.5/frameworks \
MACOSX_DEPLOYMENT_TARGET=10.6 \
--enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk \
--with-universal-archs=3-way

I also tried to use intel for –with-universal-archs. Both CCFLAGS and LDFLAGS were being set correctly in the Makefile. I even tried setting them explicitly, with no luck: Python’s executable was compiling in all architectures except 64-bit. I wasn’t able to find any references to any such issue anywhere in the user forums and/or on the web. Every reference I saw in compiling python in 64-bit, I tried, with no luck. Evidentally, the Python distributed with Snow Leopard was compiled using some special compile procedures by Apple due to the fact that some packages lack 64-bit compatibility. I couldn’t find any reference to this procedure, nor did I really want to engage is such activity. Come on Python folks, WTF??? Can you either provide compilation instructions, or distribute MacPython as a universal binary including x86_64?

I ended up resorting (unhappily) to using Snow Leopard’s distributed Python and Mac’s web-sharing apache. I compiled mod_wsgi with:

./configure --with-apxs=/usr/sbin/apxs \
--with-python=/System/Library/Frameworks/Python.framework/Versions/2.6/bin/python

and voila, we have a working mod_wsgi install. I really dislike the fact that I wasted days trying to get it to work with Python 2.6.5 and custom apache install, but at least I have something that works now and hopefully won’t slow me down any more.

I’m loving python, as I have sporadically for quite a while, but I’m really missing the JVM with its problem-free installs, jar/war archives, and things just working.

Uncategorized — 18 Comments
30
Jul 09

Choosing a web development framework/toolkit

I’m sure I’m not the only lunatic that spends many hours well into the night thinking about web frameworks, but then again, maybe I am. This is all exacerbated by the fact that I work for startups, so requirements are much different that say someone working for an established corporation that has various standards and practices in place. I left the corporate world 4 years ago and haven’t looked back. I love the dynamics of the startup environments and my personality fits very well with its culture and pace.

So some of the questions I battle with are, which framework should I use for this new project, or am I using the right framework for my current project? Is the framework and language it’s written in supports writing applications in a powerful, flexible, fast, scalable way? A lot of the criteria I just listed are not as much framework as design and architecture of your applications and infrastructure, but frameworks can make it easier or harder to achieve such a desirable architecture.

The issue is not as pronounced for other non-web applications, mostly due to the fact that most Turing-complete languages are capable of performing the same job as any other, the only question is the programmers preference, proficiency, and the availability of some framework/abstraction to make your life a bit easier. In some languages, writing these abstractions is a breeze or in some instances they are not explicitly available because some or lost of boilerplate is reduced through various language idioms.

But web development is so complex these days, that simple abstractions are not enough. Anyone that thinks either hasn’t created a serious web application or posses some information that I’d be willing to pay to have:-) Sure, with the knowledge of HTTP and some gateway protocol, whether its CGI, Java Servlets, WSGI, etc…, one can do almost anything that’s possible on the web, but that’s a pretty bad criteria to have in the age of ever so complex applications/features. One doesn’t want to rewrite something from scratch. Authentications/Authorization for example, although many applications have a pretty custom authn/authz scheme, 80%+ of what’s needed is boilerplate that I nor any experienced application developer cares to reinvent. I’d rather be doing more challenging things, not sure about you.

So many frameworks conceal some amount of boilerplate from the developer through abstractions. Right now, there seems to be two kinds of web framework camp schools of thought:

  1. I’ve done this many times before, trust me, you don’t need anything else. I’ve extracted this, not made this up. The 20 applications that I’ve developed with this framework and extracted all of its generic concepts is all you’ll ever need. Here is my convention down your throat. You think you need XYZ? No dumb ass, you just have no clue and your brain has all the baggage of a previous framework. Come on, open your mind, do you really need XYZ? You do? Well #$%@ you, go use someone else’s framework.

  2. We don’t know what the developer wants, he might want bar or foo or barfoo or foobar or foobarbarfoobarbar, or what ever they wake up and desire that day. We’ll come up with abstractions that can be extended by other abstractions. But wait, what if the developer wants to extend those other abstractions, oh, well we must allow them to do so, so here are some more abstractions. Before you get started, here are 50 different things you must do, with xml or code bootstrapping.

The second camps sound nicer, more sane in some instances, and you’re overwhelmed with the flexibility that makes you believe that nothing is impossible. But that’s further from the truth.

I’ve mostly used the #2 frameworks, as with many years of development, I’ve developed quite a convention of my own. No, it’s not that I’m not open minded old timer that is scared of change, actually I love change so much that I find spending many unproductive nights hacking something in a completely different language/frameworks, exploring the ever so unpopular technologies, etc… But my conventions have grown empirically, though I’m not easily swayed to go back 10 years ago when I had no experience and relearn from the same mistakes I’ve already made to just come to most likely the same conclusions. People do this all the time, in every field and I don’t have grant money nor a big corporation blindly investing in my useless use of time.

I’m pretty big on DDD and OO and all the abstractions that come with it and I need a framework that allows me to do so, without forcing me to mix relational and OO concepts by forcing me to use a weak ORM, or no ORM, or an ORM that they choose for me. I also am more than capable of deciding whether I need a Repository data layer and not just a plain simple DAO layer. I know what I’m talking about, at least I think I do, so I don’t need any 20 year old telling me that using recordsets that masquerade as domain objects is just fine. Maybe for you, but I have a different opinion. It’s possible that your todo list will do just fine, but not my software that might start off with 10 domain objects concepts and grow larger as functionality is added. Ok, enough with my discontent with the 20 year old programmers, there are many of them that are brilliant, they’ll just have to learn (or not) as the time goes on.

Now with all that’s mentioned above, I also am very aware of over-engineering and aren’t we all so good at it. So, when I’m not in the mood to over-engineer something, I’d like a simple way of accomplishing a task. Without all the enterprise application pattern abstractions, etc… Sometimes I just want to create a quick prototype, start off simple, then grow it if needed into a production ready piece of software. Eventually through constant refactoring, I’ll add the necessary patterns/abstractions as my requirements grow or change.

With all the above rant, I’m yet to find a framework that can do this. Rails maybe comes close, only one problem. ActiveRecord is very limited and sucks. Besides the mapping limitations, you also bind your domain objects to the relational model and are constrained into modeling your domain in a unnatural way. This might be fine if you’re building a small app in its isolation, but it’s a huge technical debt if you’re building a service model on top of a persistence model that might be used/reused by other domains or services, etc… I want to have a consistent, coherent domain which is available as a service to other services, like webapps, background processes, etc… Good luck doing this with Rails. It is true that when I’m first starting my app this might not be a requirement, but like any other startup company, we have a vision and if that vision is realized, we’d rather not have that much technical debt to pay before we can move forward.

So after all the rant above, what is it that I want? Here is a list. It’s not comprehensive, just things I can think of at this time. I’ll update it as I think of anything else.

  1. Support for MVC (most frameworks have pretty decent support for that now)
  2. Extensible MVC (need to be able to extend the way controllers and views operate. Some frameworks do it by convention and limit you to a set of popular conventions.)
  3. Allows you to build your domain in isolation. (I want my domain model to be completely decoupled from any web technologies, persistence, etc… Just a plain OO domain model)
  4. Gives you very flexible persistence options. (I might decide to use a fully featured ORM (ActiveRecord is not fully featured), or I might want to use SQL, or heck, I might want to do both for efficiency or to scratch a morning itch, who cares about the reason, please let me choose. Oh, and one more thing, what if I don’t want to use a SQL database at all? I want to use a native XML store or better yet a key/value store. Even if it’s just to piss someone off, I want to do this and one should be able to accomplish that pretty easily. I’m not asking for a mapper for these stores, simply just don’t make your framework bound to some relational store though making the work of turning this dependency off a 5 hour chore.)
  5. Supports AJAX (I should be able to easily render JSON or XML views, without much plumbing or lots of mappings and annotations. The authentications and forms support should also expose some form of ajax compliant interfaces, so that forms and authentication can be done using ajax if I choose so. Be able to easily parse submitted data into some data structure and validate/synchronize it with the domain model.)
  6. Bindings (All frameworks have some sort of bindings. Some of them are limiting. I don’t want to create command objects to just simply bind the data and than synch with my domain model. If I have domain object graph(s), I should be able to bind it in the view layer. Bindings should be customizable. In Spring MVC for example, you can only bind one input control to a field or set of fields in the command object, but what if I want to bind 3 input fields that collectively represent one field in the domain object, I’m out of luck, unless I use javascript to first serialize those input fields into one field. That really sucks.)
  7. Support RESTful, stateless, and other web concepts in a straightforward way. (I want to be able to configure every part of HTTP and the web and make the application work, look, interact in my way that’s compatible with the web, not your way. Some component based frameworks make that harder that it should be, like the fact that they are inherently stateful by default. Some make it hard to support RESTful or custom URI schemes, because they transfer state through URL rewriting. All of these problems don’t exist in some frameworks, like Rails, Spring MVC, Grails, etc…, so I know it’s possible.)
  8. Validations (Most have fully fledged validation support, but I can’t say that it can’t be made a bit easier. I do like Spring’s flexible validation support.)
  9. Forms (This is a big one. Can you provide a flexible way of creating forms and layouts. I mean seriously, we’re developing forms today the same way we’ve developed forms 15 years ago. Every other aspect of development has moved on, but we’re still doing bullshit html form controls. XForms is a way out, but no browser support and pretty hard to integrate support from vendors like Orbeon and Chiba makes the standard useless. Can we either embrace it or come up with something else. Am I the only one that gets an anxiety attack every time I think about creating yet another interactive form that doesn’t do anything much differently than the form I created 4 months ago for a different project, though I either have to copy and paste all the cruft or start from the absolute scratch. Wow, that’s sad IMO.)
  10. Scalability. I know this one again is not up to the framework, but as I mentioned before, the framework can make it easier or harder to achieve. For example, inherently stateful frameworks that require either session affinity or replication of session state, make it very hard to horizontally scale. Yes, I know you can scale with replications tools available out there, but any synchronous replications is not linearly scalable. So any such frameworks makes it harder. There are many other criteria that can make a framework more scalable than others, but in general, statelessness, stability, and speed makes it viable for faster scalability tunes.)

Ok, that’s it for now, I need to vent before my brain allows me to think of other things I’ve encountered of my never ending framework journey. I’d really love for someone to just say, hey you’re wrong, there is such a tool(s), here it is. I’d be eternally grateful. Many will say that it’s useless to complain, if I see a need, help create the framework or functionality that you think is needed. I wish I had more time, until then, I’ll continue to grunt and develop my own inner frameworks to make things easier. One day, if I have time, maybe I’ll devote some time into making some existing open source framework better. I’ve had more time years ago and contributed to quite a few open source projects that I’m truly starting to miss it now. I still occasionally submit a patch or two to a framework I’m working on after fixing an issue or adding some feature, but at times I’m in such a hurry to move on to the next task, I don’t have the time to package or generalize it enough to make it useful for everyone else.

Right now, I’m working with Grails after starting a project in Spring MVC and not being able to deliver functionality as quickly as I wanted. I’ll have to live with some issues I found with Grails when I was using it about 4 months ago, like the fact that you must use hibernate or gorm, crappy groovy stack traces, etc… Hey, there is always something one might not like, but I really like Grails and am hoping that now that it’s in the hands of SpringSource, they’ll spruce up the documentation to be more like Spring’s awesome documentation and clean it up a bit.

Update: I wanted to reiterate a bit on the Grails in regards to isolated domain model. Grails does allow you to create and isolate your domain model and its persistence, unfortunately you have to twist it’s arm if it’s anything outside of GORM. You don’t have to add classes to domain directory, but wtf is it there for? Also, it would be nice if the grails team provided a way to specify which classes in the domain directory are persistent or not. I mean, a domain model != persistent entities. So transient classes and other domain artifacts should also be grouped together. Putting them into src/groovy sucks personally, because I have to navigate two directories now to look at what’s supposed to be a coherent domain model.