Feb 11

On startups and innovation

Throughout my career I’ve worked at numerous organizations, large and small. Last 5 years I’ve spent doing startups, the previous years were spent at large companies. I prefer small organizations, specifically startup environments, for reasons I’ll discuss later. There is something that can be learned, good and bad, from both types of organizations. I’ll start with some positives.


I’m going to use the word “startup” to not only refer to companies that are just launching or haven’t been around for a while. In my opinion startup is now synonymous with a company that operates like one. They are always in launch mode, always looking for new adventures and opportunities, and always innovating.

Not all small companies are startups. Many small companies have been around long enough to amass some or many startup antipatterns, which is in most cases rather detrimental to its growth. Large organizations can bear some of these antipatterns for years, as they lurke and hide behind large bureaucracies and/or residual revenues. Large organizations also have different priorities.

Startups are the innovation backbone in this country and others. Their beginnings usually involve individuals who have knowledge and experience in a particular area and/or they see a need to fill a gap. Sometimes it’s not a gap, in terms of a complete void, rather it’s a gap in terms of doing something better. These individuals are motivated by their idea and the vision of making a world (as they see it) a better place, glory, recognition, etc… There is also monetary motivation, but the product or service motivation is usually higher (at least in the ones that succeed). Also, there is drive and the drive has to be strong. I mean, in lots of cases, these individuals give up good paying jobs and benefits, put their quality of life and possibly the quality of life of others in jeopardy, without any guarantees, all for this dream. A lot of these folks have worked at other companies and they’ve decided that they can form a successful team and eventually bear the fruits of their labor (again not only in monetary compensation). Wow, that’s inspirational and also… “scary”.

During the initial stages of the startup, there is no time for bullshit. The bureaucracy and overhead that plagues established companies has to go. They have to get shit done, they have to do it good, and they have to do it fast. They don’t want to rush to sacrifice quality, since this is the reason they formed this alliance, to create a “quality” product, better than their competitors. This is also the time when creativity is at its highest. How else can you stay in business when you’re competing with a company that has 10 times as many employees, has been adding features for years, has an established brand, and more money.

Now the more technical side (I’m mostly writing about tech startups here, as this is what I’m more familiar with).

There are many things to do, design, architecture, programming, testing, deployment, etc… Choices need to be made and these choices have different priorities than in large companies. Stuff has to be done fast, so there is no time for the “Enterprise” stuff. Whatever that word at one point signified, it’s now become synonymous with “clueless pointy hairy bosses”. There is not time for commercial product evaluation, negotiations, and life cycles. Open Source Software usually rules in successful startups. It allows them to benefit from quality code with the ability to resolve bugs, add features and augment the software as they wish. They often built software using open source platforms and frameworks and themselves greatly contribute to open source. Many times the tools that they built, that aren’t specific to their business and don’t contain any intellectual property, are released to benefit the community. Now others can use that as a tool to benefit in their endeavor. The cycle continues.

I’m going to refrain from discussing specific examples of what I’d consider sensible choices. Every business and motivation has different priorities and in many cases what I would consider a suboptimal choice, would actually be the best choice for a task/dilemma at hand. One thing’s for sure though, open source software rules and beyond the brilliant people that make things happen, is a big contributor to the success of these companies.

Because people are smart, they have to get things done, and they put their blood and sweat into it, they take their time to create fun and productive environments. I mean, who wants to work a lot and not have fun doing it? Or who wants to give up the security of a full time job to fail?

Large companies

So I actually won’t talk about large companies here. I use the name mostly to refer to practices that are prevalent in larger organizations. I’m mostly going to look at the shift from an efficient, fun startup, to a bureaucratic mind draining organization that has lost its ways.

This happens all the time. The talent looses its interest and usually scatters to other companies or to start their own. The company is left with a product that’s aging and a leadership team that’s focused on maximizing the profit from the product it has. Nothing wrong with maximizing the profit, but not at the cost of stagnation. The talent drain is prevailing and the leadership for the lack of better judgment blames this on the lack of a process. “We’ve lost our ways, we’re not productive any more. We need to put a process in place that will get us going.” they say. I’m not going to judge these folks at this point, I mean their core competency isn’t innovation, it’s stability. The two are polar opposites.

Then havoc wreaks. Process after process is established; Feature after feature thoughtlessly gets added to the product; Documentation and processes take priority to common sense and productivity; Code base grows unmanageable; More time is spent talking about doing, than doing. Sometimes more and more people get hired, with less and less qualifications or for qualifications that aren’t critical to success. Sounds familiar?

This happens all the time. The bottom line is that most innovators don’t spend their lifetime chasing an idea (especially these days). They innovate and then when they don’t see any more room for innovation and fun, they leave. Yes, they have what some would call ADHD when it comes to stability and work ethics. They probably do, but in a good form, they are chasing their dreams and refuse to waste time on something that won’t make them happy.

This is not to say that innovators are better than “stabilizers” (I just made that one up). There is room for both, but stabilizers need to understand how to maintain the level of competitiveness and innovative spirit in the organizations. They should recruit and motivate the brightest, not the workaholics per say, not the conformists, but free spirited minds. These are the folks that make shit happen. These folks of course have to be kept in check by the stabilizers, but not by getting in their way. Actually, the total opposite, by getting out of their way. Give them room to do what they do best.


I don’t think it’s ever too late for a company to change its ways. Get back to a startup environment. Large organizations might have a hard time doing this. They have too many people and they can’t just start over. Small companies are great for this. They can start afresh. They can retain talent, pay them well, make them happy, and most of all, I’ll say it again, get out of their way. If you hired a development team to create a product, let them do it. The more shackles you put on them, the less likely they’ll succeed and the more likely they’ll leave ASAP. Intellectuals are always in demand and it really bothers me to see companies treat such employees as sweat shop workers. We pay you, you do as you’re told. That’s a recipe for disaster.

If your developers are more productive with language x and platform y, unless their choice is completely ridiculous, let them use it. They’ll thank you later, with a better quality product, more productivity and generally a more positive attitude.

Feb 11

Merge sort in lisp

I’m currently taking (online) MIT’s algorithms course. I needed a good brush-up on algorithms, starting with the basic theory. I’m going to jump in to help a buddy working on a graph database in lisp Vivace Graph, which should be an open source competitor to AllegroGraph. Allegro has probably the best graph db out there right now. There are some competitors in the java world, but they are either expensive or don’t provide enough features, or don’t provide APIs outside of java. Allegro has an extensive list of client libs. I’ll write up more on the graph db in some future, for now, I got back into lisp and implemented the simplest alg (merge sort) in it.

The reason I’m posting is to get some comments. I’d love to hear your thoughts on how a lisp nOOb can improve this 🙂

Dec 10

Rails custom validation before ActiveRecord typecasting

Rails 3 validation framework, extracted from ActiveRecord and now a part of ActiveModel is pretty sleek. As anything else rails (consider it good or bad) it offers sensible defaults. Most stuff in rails is also easily configurable/customizable. Validations is one such thing, but I think it needs more documentation.

I ran into a problem trying to validate a date field (defined in ActiveRecord) as:

Rails also doesn’t have a built-in date validation, but it’s easy enough to either provide your own validator method or create a reusable validator. I opted for the later, as I might need to reuse the same validator across the site.

So running this with any invalid date value, like say 11111111111 which gets parsed by Date.parse method didn’t yield an error, actually it never even invoked the validator.

After a bit of digging, I figured out that it has to do with the lifecycle of ActiveRecord, basically it tries to typecast the value to the value defined in migrations. Typecasting invalid values like above causes ActiveRecord to set the value to nil (not sure why yet, will dig into source code later). Because the validator isn’t invoked on nil values, unless you specifically tell it to not allow nil with :allow_nil => false, the validate_each method is bypassed. You can see the logic for yourself in the validate method.

In order to fix this issue, you have to access the raw value that’s kept around after the typecast. I turned this into a reusable class, though if the validator that needs to operate on the raw (uncasted) value, you just inherit from this class. Here is the class…

Now the DataValidator above just need to inherit from RawEachValidator and you’re all set.

Oct 10

Learning Haskell (Part 1)

So my friend Davis Ford decided to start a Haskell study group in Grosse Pointe, MI. Our first meeting was yesterday, and 4 people attended. This isn’t bad, considering that most IT folks don’t live nearby and the fact that it’s Haskell :-). I think we might be able to get 6+ at the next meeting and hopefully more at some point.

We had a really good start yesterday. We decided to read Ch. 1-4 of LearnYouAHaskell before the meeting and then discuss the relevant topics. Here are some of the n00b things we talked about:

  • function declarations and definitions
  • function argument destructuring and polymorphic dispatch
  • implemented recursive product function and quick sort
  • logging, which turned into having pure/impure function mixture

So our quicksort implementations turned out to be such:

One thing that I still haven’t concretely figured out is the fourth bullet point, logging. Basically I wanted to perform logging while say calculating a product. Evidently you can either have a destructive function, which performs/returns IO operation(s), or a pure function. Mixing both doesn’t seem to work.

Something as simple as:

printSomething' = do 
  putStrLn "test"
  return (1 + 1)

Didn’t work, yielding this error:

Occurs check: cannot construct the 
infinite type: b = IO b
      Expected type: b
      Inferred type: IO b
    In the first argument of return', 
    namely((product' xs) * x)'
    In the expression: return ((product' xs) * x)

I was able to find information discussing doing this through an IO Monad, but am yet to actually construct a working example. Seems like too much work for something as simple, but it might just be that I’m misunderstanding something due to lack of Haskell knowledge. Off to learn more this week and I’ll post a working hybrid function example once I get it worked out.

Oct 10

State machine with Clojure macros and runtime argument inference

I few years ago, before I delved into functional programming, I had a small stint with Flex/ActionScript. ActionScript is an imperative language very similar to Java. At the time, I needed a very simple state machine, which had a single path of execution (basically a chain of commands). The design included a chain object, which joined command objects and executed them sequentially as long as no exceptions where thrown. Because these chains where also used for transformations and had dependencies (one command might compute something that is needed by another command), the commands had to keep state, though a global context object was used to store/retrieve state. I’m sure there are other ways of designing such a system, but it turned out to be pretty maintainable and rather clean. One thing that bothered me at the time were the implicit dependencies amongst the command objects, which relied on certain context information to be there in forms of map keys, which means if a command changed how it stored a particular results, its dependents would have to be modified as well. Because of the lack of static typing and runtime inference (unless done at each command object level), there was no way to ensure that something wasn’t silently failing. The problem was due to utilization of map structures for context storage which besides not having any static typing abilities, also didn’t allow the chain invocations to perform runtime inference of argument matching. The implementation was very functional and wasn’t too badly designed, but definitely not very pretty.

I don’t have access to the exact code at this time, but below is a simple example that demonstrated similar issue in Java.

Running the above yields:

    result1: 1234
    result2: 2468

Besides the mandatory java ceremony, it’s also not apparent to me that this can be accomplished any better without the use of reflection, which of course would add yet more boilerplate.

Macros to the rescue. If any of you aren’t familiar with what makes lisp (besides its simple syntax allure) so powerful, you should familiarize yourself with macros. The example I give below doesn’t even make a dent into the possibilities of macros.

With a simple macro

The above can now be utilized with the following api…

Yielding (in SLIME REPL):

Executing first
Executing second: first arg
Executing third: second arg random-arg

Running a similar script with an added parameter in the third command that doesn’t exist in the chain

Throws an exception (in SLIME REPL):

'does-not-exist' argument is required!
  [Thrown class java.lang.Exception]

Of course the above can be much improved, one off the top of my head improvement is to allow default values for arguments that don’t exist and possibly be able to specify arguments who’s lack of should throw and exception an in other cases either bind a nil or default value. But the concise example demonstrates similar abilities of my java program but with added runtime argument name matching. I’d love to see if the same can be accomplished with a statically typed language with type compile time vs. runtime argument checking. Going to investigate this with Haskell and Scala this week.

Page 3 of 1312345...10...Last »