IDEs and Emacs

So I am an emacs guy. I love emacs. I want to use emacs for everything I code. Unfortunately, emacs does not have great support for Java, which we are using for our project. In the past I have used Eclipse for Java, but this time through I thought I would try Intellij IDEA.

Intellij is a very good IDE, but it doesn't really understand the web framework we are using with our application, and so it tells us that our code is broken, even though it isn't. So for the first time ever, I decided to attempt to code a large Java project in emacs. It has been an interesting experience.

I find that I understand the application much better writing it in emacs. I have to manage all the imports myself and as such I have a much clearer understand about how the dependencies work. On the other hand, I definitely code Java slower in emacs.

I am still not sure which way I prefer for coding Java. I mean emacs Java features are terrible, but emacs general features are awesome.

Perhaps I will make another post later in the semester after I have made a more formal evaluation of the two in comparison to each other.

Roles

As we are moving into a new phase of development in our application, I find that our development roles are changing. As we started out work on this application, there was a clear division on labor. We each had our own part of the codebase that we owned.

Now that things are farther along, and many core features are nearing a complete state, we are having to shift roles working more intimately on each other codes.

This is an interesting transition for us and it is revealing a lot of issues we hadn't noticed before. For instance, until recently I only had a passing understanding how our database setup worked. Before now, I didn't need to know how it worked, it wasn't my code, I just used the API that my teammate made. Recently I had to work more directly with the DB code. I found myself spending quite a bit of time just working through the code before even getting to the issue, even though it had been in the application for a long time.

I think all of us are having this issue. It will be a little bit of a set back as we have to learn to work with code that we didn't right, but probably a good experience.

Polish

As of a few days ago our application is in a really nice state. Most of the features that we needed to implement are implemented. This really feels like a good place to be in. Now our development will probably shift to more candy features. Accounts, better GUI, more useful user feedback, etc.

I am finding it very hard, as we move into this set of feature development to stay really excited about these features. I find myself wishing to develop more large sweeping features, rather than these small ones. It is difficult to remind myself that it is these small features that will actually make the application successful. The devil is in the details, as the phrase goes.

It is a strange realization, that the use of a particular tool is not generally a function of its usefulness, but rather its polish. I mean, it still has to work, but it seems what really makes an application successful are these candy features.

Full working stack

As of a few days ago my team has assembled a full working stack for our project. This felt like a very good milestone for us. Prior to this we had a lot of the pieces together, but there was always some fear that assembling them would cause the entire thing to fall apart.

While this feels like a great victory for us, it is also quite scary. I feel like we are almost finished with the product, and yet, there is still so much time left in the semester. More so, there is still a lot of important pieces we need to put into place, particularly, we need to start building our data set to drive our application. While the core program is in a mostly functional alpha state, it is useless without data to drive it.

I guess this is the "Sag" that was discussed in class. I know I am starting to feel what that word means, in more than one way. So much done, but still much more to do.

Branching issues

As our project has matured somewhat we have run into some interesting issues with our VCS. We are using git for our VCS and it is great! One of it's killer features is the ability to create a lightweight branch very easily. This is, in my experience, much easier than the same process in SVN. This feature has proved to be very useful to us, as each of us can work on our own code with out mucking up the others, and still share it to the master repo so that others can pull it when they wish. However, we have used it so much that we have begun to experience branch cruft.

We now have many branches, many of which are technically dead, but it can be hard to tell which branch has the features you wish to pull. You can of course, examine the commit log for each branch, to see which one has been active recently, but this process seems very tedious.

The solution for this problem was trivial, just add a prefix to each branch for each week, so that we know which are active. However we only learned this simple technique after a week of dealing with many branches of unknown activity.

Bug Tracking Etiquette

As our project has grown from to a decent size, we have ended up making some minor mistakes and creating small bugs here and there. An issue which we are having to learn how to deal with is figuring out how to track all these bugs so that they don't get lost in the shuffle. The solution is simple of course, a bug tracker. Thankfully our repo management has a built in bug tracker that we have started to use.

More interestingly is the etiquette for bugs in code. Recently I discovered a trivial bug in a section of code that a teammate was responsible for creating. I thought that I understood his code well enough to make the change myself, but I hesitated on actually implementing the change. It occurred to me that changing another persons code, when that person is active on the project could be considered some how inappropriate. Instead of fixing the issue, I could just have easily opened a ticket in our bug tracking system for him to fix it, but then I must wait for him to notice the ticket, and take time to fix the problem. Neither of these alternatives is inherently better than the other, but it is an interesting dynamic that I had not encountered before.

I supposed the right answer on how to manage this stuff is very project and person dependent. It is surprising how much peoples interactions with each other and feelings play into software development. It is much more than just writing code in emacs.

On remembering to not be stupid

This post isn't a joke. I very frequently read scientific studies or articles or papers about the various things that people do that actually prevent them from being as happy and productive as they would like to be. Many of the things that I read about I actually do myself. Nevertheless, although I have amassed this great amount of knowledge about how to be productive and more happy, I put very little of it into effect.

For instance, I am by nature have a higher than average propensity for anxiety. Caffeine is highly correlated with an increase in anxiety (among several other negative things), and yet I still drink coffee several times a day. Sleep is another big one. Getting the correct amount of sleep is highly correlated with increase performance everywhere all the time in everything and yet I frequently get less than six hours of sleep. Why do I do this? Because even though I know this it feels like it isn't true. It feels like coffee helps me, rather than hurts me. It feels like working through the night helps me, rather than hurts me. So I need to remember to not listen to my feelings and not be stupid. This is especially important during very difficult school semesters.

So with that note, off to bed.

WTFM

RTFM or "Read The Fucking Manual" (http://en.wikipedia.org/wiki/RTFM) is a amusing, if vulgar, phrase that arose from people not reading the documentation for how something worked, and the complaining when it did not work or when they did not understand it.

Last week I spent literally twelve hours pouring over the code for a library that I was hoping to integrate with some code I was working. The library was very poorly documented, with some of the vague documentation contradicting other parts of the documentation. This reminded me of an alternative phrase that has popped up in recent times, WTFM or "Write the Fucking Manual".

With programming becoming a more and more common practice and more and more emphasis being placed into writing code very quickly (a perversion of Agile techniques), documentation of code is going be the wayside. This is very bad. Why is it bad? Two reasons, danger and waste.

It is dangerous to not write documentation because the other hot topic these days, is software security. When one fails to properly document a codebase the people who are working with the code base (either in library form or other developers working on a shared project) are much more liable to make errors that could introduce security bugs in the code base.

It is wasteful because as I already mentioned, I spent twelve hours reading the documentation for a library. The library was complex, but the actual API wasn't supposed to be complex, and I should have been able to adapt it to my code in a couple hours. Unfortunately there was some subtle strange behaviour that took my a long time to work out because of the terrible documentation.

This is a waste. That time I spent could have been used better, and many many people suffer from lack of good documentation on a daily basis. It is a tragic affair. So WTFM.

Technology Choices

This last week we were supposed to discuss our technology choices in our client meetings. In our meeting, we were short on time and didn't get to discuss them in as great detail as I would have liked, so I thought I talk about that a little bit here. I was in charge of researching web frameworks for our team, so that is what I will discuss.

We are creating a web based application, and for that we are using a web framework. I was considering for our team both Django and Play, one based on Python the latter based on Java/Scala. I prototyped a few toys in each of them, and decided on using Play in the end.

As I mentioned in a previous blog post, they are both very similar. Their overlap in feature set is probably close to 90%. So it was rather hard to make the decision.

In the end we decided on Play, mostly because we could write Java for it, which we are all very good at. It also was somewhat easier to create the types of web applications we are targeting with Play, while being easier to do other types of applications with DJango.

This was a bitter sweet choice for me. I really like Play. It is designed to be highly scalable and performant and (after reading some of the docs) is a joy to use. On the other hand, while being very familiar with Java I really don't care too much for Java. This is mostly a personal preference issue, as you can certainly create very well designed nice applications in Java (and just about anything else). On the other other hand, Play supports Scala, which I am quickly becoming very interested in, as I have developed a fondness for functional programming paradigms recently.

I am not sure that I will write any Scala in this class, but I can definitely see myself coming back to use Play with Scala in the future.

On Portablity of Modern Code

Recently I have noticed that there are a lot of programming languages targeting the JVM. Aside for Java, there is Clojure, Groovy, Scala, JRuby, Jython, and Rhino, as well as a handful of other less notable languages.

It is interesting how all of these languages flocked to the JVM to bootstrap themselves into all of the work that has been done on a powerful bytecode runtime. Without doing any extra work they get platform portability and runtime optimization (maybe, I am not totally sure about this dark magic.).

It is interesting to ponder what might happen if Oracle stopped developing the JVM, or closed source future updates to it. All of these separate entities would have to come together to maintain the runtime, since they are essentially dependent upon it.

Even more interesting is the fact that if a programming language isn't running on the JVM, it is pretty likely it is a scripting language. Ruby, Python, PHP, JavaScript all require no compilation to run. It seems that the ideals of creating portable code have now been met with either the JVM or a script.

On web frameworks

Over the past few days I have been working on selecting the web backend for our project. Last client meeting I had narrowed it down to django, a python based framework that is popular right now, and play, a Java/Scala based framework that is also becoming quite popular.

Evaluating which framework to use has been quite a challenge. They all seem to support all the features we care about. They are all expressive and "easy" to use. They all seem great.

As a team, we have a stronger background in Java, but we are also Computer Scientists and should be able to pick up a new (newish as we have some prior experience with Python) language in just a few days.

But web frameworks are much more that a library for a given language, they are a meta language unto themselves. The web has evolved in such a way that when we "surf the internet" we are using a number of technologies at once, many not designed to work with each other, connected together in strange an opaque ways. Web frameworks allow the programmer to once again be a programmer, rather than a Rube Goldberg machine master.

Further complicating the issue is the fact that are so many good web frameworks. The only way to truly compare one framework to another is to site down and write an app in both frameworks. Alas, this means learning two frameworks (or more), which is not a task that the busy student enjoys. But it must be done.

Are build systems bad?

The other day a friend and I were attempting to compile some Java code which we did not write ourselves. While we both are very experienced with the Java, neither of us had written or built any Java code in quite some time. Further we did not have Eclipse installed on the system we were working with, so we went about the daunting task of building the java program from the command line.

It took us a good ten minutes at least to get the code to finally compile. The experience reminded me just how frustrating it is to work with the Java build system. At one point in time, I knew the system in and out. I learned all the nuances of it just long enough to write a bash script to do it all for me.

When I come across issues like this I often think about how much some tools separate the programmer from what is actually happening with his code. Before I wrote the aforementioned bash script, if you had asked me to compile one of my Java projects without eclipse I would have had a very hard time, and I wasn't a novice Java programmer.

Java and Eclipse aren't the only examples of this, many many programming languages have complex build systems, and because we often use IDEs that build our software for us, we never actually understand how it works.

I can't decide if this is a good thing or a bad thing. Certainly automated building of software is good. But when I build software in C I write a makefile which runs commands that I understand (although there is some magic in the inference rules of POSIX make that I think most people gloss over). When I build Java in eclipse I am totally separated from the process.

Does the modern programmer need to know or care about this stuff? Or should he just spend his time on more interesting problems?