12
Jan 09

CodeMash 2009

Last week I attended my first CodeMash conference in Sandusky, OH. I must say, I was very impressed. Besides that fact that it’s a regional conferences with a limit of 500 participants, it had great tracks and a very cozy atmosphere that made one feel relevant. Bigger conferences usually attract many more participants to a track, which makes it hard to grab the attention of the speaker if one has a question. Not so at CodeMash, where tracks had an average of 20-30 participants.

My first track was on Thursday and I decided to attend Scala presentation by Venkat Subramaniam. Content aside, Venkat is an awesome speaker. Actually, awesome is an understatement. He communicates very well, has a great sense of humor, and is very thorough (he really knows his stuff). Throughout the sessions and conference in general, I took notes in the form of a Mind Map. This was my first shot at it, but I loved it. Instead of formal note taking I’m so used to, this was a refreshing experience. The mind map technique is described in many literatures, but I really learned about it from the Pragmatic Thinking and Learning book. Here is Venkat’s Scala talk as a Mind Map.

Scala Presentation MindMap (click on image to enlarge)

Scala


I used XMind to draw it, though at the conference I just drew on paper. I did an exhaustive search for Mind Mapping software a week ago and couldn’t find anything that was worth while, until a friend of mine David McKinnon recommended XMind. This software is awesome.


05
Jan 09

Pragmatic Thinking and Learning

I just finished the book titled Pragmatic Thinking and Learning.  Wow, am I fascinated.  This book is a concise overview of all the latest behavioral psychology and neurological studies as well as pragmatic advice on how to become a better software developer.  Although these techniques apply to just about anyone trying to improve their skills and/or just trying to become more productive, Andy does a good job relating to programmers, being that he's one.  I've read about lots of these topics in my adult lifetime, but this books organizes everything in one place, so you no longer need to go and read about bits and pieces of various topics.


There are two topics that really caught on to me (although all were very fascinating), Mind Maps and Managing Focus especially dealing with Context Switching.  I started to apply those right away.  There are others that I plan on incorporating into my plan in the following weeks and I'll discuss those as I do, but these are here to stay now.
Mind maps are a great way to represent your thoughts and really reinforce what you are learning/thinking.  They are non-formal, although I like the convention presented by the author.  There are many times when I understand a particular topic and have a mental image of how various subtopics are interrelated.  Although maps are hierarchical, you can have cross hierarchy and cyclical dependencies as well, but you might want to rethink the relationships if your map begins to look like a brain that just had to make 20 context switches in the last 10 minutes (more on that below).

Context Switching

I've always been aware of distractions, but somewhere along the lines of being instantly rewarded by satisfying clients/bosses with a constant barrage of IMs and Emails, we loose focus on important things.  By responding and completing these small tasks immediately, you loose focus and the items that are important.  So supposedly a 2 minute fix now turns into about a 20+ minute distraction because of the context switch phenomena described in the book and well backed up by many studies.  Your time might vary 🙂


I'm going to write up a better summary over the weekend, don't want to get distracted from focusing on important tasks at this time, but I would recommend the book to anyone, whether you admit in having issues with focusing and time management or not, and/or want to be more productive and learn and acquire skills faster and more efficiently.

02
Dec 08

Choosing best/efficient algorithm implementation for the task

So as I'm progressing studying collective intelligence concepts and algorithms, I couldn't help but think about the most efficient/performant (no such word, but you know what I mean) ways of implementing statistical algorithms that perform calculations on very large data sets .  As most programers proficient in a particular language(s) would do, is find a library or implement that algorithm in that particular technology, without looking outside of the box.  I'd probably implement most of these in some rather efficient language, but wait…


I was searching for some statistical/mathematical tools for prototyping and ran across R language.  So I decided to see how say python performs against R for the simplest of algorithms, mean.  I'm sure there is a way to tune the python code, but I even looked at some python statistical packages and they all implement the mean in the same way.  So here is a benchmark.  I hate to compare languages these ways, this means nothing, other than you should really look outside of the box for a tool that fits a particular problem.



#### Python with mean algorithm Time: 24.5236210823 seconds #####

import time

def mean (inlist):
 sum = 0
 for item in inlist:
     sum = sum + item
 return sum/float(len(inlist))

start = time.time()
result = mean([i for i in range(1,50000000)])
end = time.time()
print "Result: %s, Start: %s, End: %s, Time elapsed: %s\n" % (result,
start, end, end – start)

#### Python with using the R-lang interface.  It dispatches to R libs behind the scenes.  Time: 14.780577898 seconds #####

import rpy2.robjects as robjects
import time

r = robjects.r
start = time.time()
result = r.mean(robjects.FloatVector(range(1,50000000)))
end = time.time()
print "Result: %s, Start: %s, End: %s, Time elapsed: %s\n" %
(result[0], start, end, end – start)

#### R-lang using the mean function.  Time: 0.654 seconds (Yes, that's 654 milliseconds!!!) #####

print(system.time(print(mean(array(1:50000000)))))


Here I ran mean against an array of 50 million 1..50,000,000 numbers and I got some absolute astounding results.  I'm almost wondering if I'm completely missing something.


Update:

I changed the python implementation to use the scipy library, which also uses numpy.  Scipy has a mean function which can be passed a numpy array range.

result = scipy.mean(numpy.arange(1,50000000));
Time elapsed: 1.02892494202

Wow, way better.


28
Nov 08

Reflection with generics and annotation introspection

So I ran into an issue today that cost me an hour scouring documentation on reflection to see how it behaves with generics only to not find any sensible info and rather find some blog posts from folks that experienced similar issues. The basic idea is that reflection and generics or rather reflection and runtime type binding doesn’t work as some might expect. Well, at this time of the night, code speaks better than words to me, so here is a small example…

Compiling and running above with assertions enabled will throw the java.lang.AssertionError. Why one might as, well that’s because although the reflection package seems to locate the method using supertype arguments in place of actual types, this must yield some different runtime binding of that method, since introspecting for an annotation of that method yields nothing.

I’m not sure what the intention of this is and whether it’s my lack of understanding the reflection APIs and how they introspect runtime types. I’ll do more digging over the weekend and report if I find something interesting. For now the only way this can be resolved unfortunately is by writing a dynamic search method that will find the right method based on its name and argument types. The types can be supertypes, as we dynamically compare the method argument types to the required types using Class#isAssignableFrom(Class) method. Here is a util method that takes care of locating the annotation…


23
Nov 08

AlexBuild refactoring

So we're going through some major refactoring for AlexBuild. We quickly realized that the original syntax didn't have enough provisions to make Alex extensible for 3rd party plugins as well as making it easier for us to add various lifecycle strategies.

Here is what we have so far…

################################################

define project as {
 name: “some_widget”,
 version: “0.01-alpha”
}

set property src to “java/src”
set property build_dir to “target/classes”

define dependency ivy://commons-logging.commons-logging version 1.1.1 as logging for all
define dependency commons-lang.commons-lang version LATEST as lang for compile, test, package
define dependency file:///home/user/dependencies/spring-2.5.5.jar as spring-full for compile

compile java from “${src}” to “${build_dir}”
compile groovy from “groovy/src” to “target/groovy_classes”
compile resources from “resources” to “target/classes”

create jar named “CoolWidget.jar”
 from “${build_dir}” and “target/groovy_classes”
{
 manifest: “path/to/MANIFEST”,
 include: [ “**/*.class”, “**/*.xml”, “**/*.properties” ]
}

################################################

The basic idea is that each statement is implemented as an AlexPlug, which is an extensible set of interfaces that allow you to develop and extend how various statements in particular contexts are implemented.  Each AlexPlug will consume a set of parameters called AlexParams and arguments called AlexArgs.  The difference between AlexParams and AlexArgs is that AlexParams is a strongly typed set of parameters injected into the implementation of the plugin and AlexArgs is a JSON-like data structures which allows you to provide extensive loosely typed configuration for plugins.  Take a look at the "create jar" statement above.

I'm hoping to have the grammar redefined tonight with the refactoring complete sometime this week.  Once that's done, we can finally do our initial release.  I say finally because before this refactoring exercise, the code in the trunk was ready for the first alpha release, though it didn't provide any points of extensibility that didn't require tinkering with the grammar.  I believe this small delay is good and we can release the initial version with extensive extension APIs to allow for folks to write AlexPlugs.

More to come….

Page 10 of 13« First...89101112...Last »