Testing Saves Time

Today while working on an Arraylist implementation in C and find myself working hard to find a very strange bug. As of now I still haven't found the cause, and will probably be working on it for some time more tonight.

I only found this bug because I decided to make heavy use of C assertions. This reminds me of just how important testing is the process of developing good software. Had I omitted these assertions, the code would have seemed to work correctly, only given me error ridden results, which usually take much longer to debug.

It is strange that there is so little desire to test first among programmers. Lack of testing invariably leads to long painful bouts of working through seemingly correct, but functionally wrong, code. Hopefully the ideas of software correctness and testing will take root more firmly in the future.

Iterators in C

Recently I have found myself implementing an Arraylist data structure in C. As I did this I contemplated how I might implement various standard OO mechanisms in C in support of this Arraylist. The most obvious mechanism to support for this data structure is the classic Iterator pattern. So I thought I talk about how iterators work in general and how they might work in C.

Iterators in supply an interface for traversing a Aggregate data structure, that is something that encapsulates 1 or more elements, in a way that decouples them from the client code. Most iterators provide a mechanism for at least four central operations, reset(), next(), isDone(), and getItem(). These functions allow for resetting the traversal to the beginning of the Collection, moving the iterator to the next element in the Collection, check to see if there are any more elements after the current element, and getting the element at the current location of iteration, respectively.

Iterators are very powerful as they provide a way to define different types of traversal over the same Collection, ways to store traversals for later, as well as many other neat tricks.

Usually when implementing an iterator in a standard OO language, you have the given Collection create some Iterator object that conforms to some Abstract Iterator interface (Or actual Interface rather than Abstract class in languages Java). In C we are not afford such luxuries as Objects.
What we could do is define the struct and members in a header file somewhere. We would set all of their values to be Function pointers. Any collection that wishes to provide an Iterator can create a struct of this type and set the pointers in it to point to static functions that are implemented internal to the Collection.

There are quite a few issues with this method however, for one it explicitly violates the intention of a static member by allowing references to them to leave the scope in which they were defined. But this is kind of how C works, Wild West Style.

I haven't finished all the implementation yet, so I can't speak to how practical this would be (I am sure that Google can tell me, but sometimes it is useful just to contemplate something.). It is an interesting task forcing OO concepts into non-OO languages.

Proposal Afterthoughts

Now that proposals have been picked and teams have been chosen, I have taken some time to reflect on the process as a whole. My proposal was not picked, which is okay. Having spent some time reflecting on Professor Ackley's feedback, it seems as though my idea may have been too much for this semester. I still believe that it is doable, inevitable even, but not in this context with this amount of time.

This is kind of how I felt about most of my ideas, that they wouldn't be feasible in eleven weeks. I had a few ideas that I liked a lot better than both of the ideas I ended up proposing, but felt that it was not realistic for us to complete them in such a short amount of time. In retrospect, I probably should have ignored this notion and proposed them anyway. I felt that if I wasn't sure I could bring an idea to what I considered a complete version in the given time frame, I should not propose it at all. But I failed to realized that even a partially functional prototype may be worthwhile, or maybe not. That is something that is a bit scary about this course, I don't now how much is enough for anything. I am not really complaining about that, in fact I think that it is structured that way intentionally, nevertheless it is still a bit nerve racking.

On another note, I felt like I learned a lot about pitching a project through this process. While I was generally satisfied with my presentation style, I didn't get any traction from my peers on my project. I think that the problem was two fold, my proposal sounded like it would be rather difficult, and I failed to go through enough refinement on the project itself. Having had more time to think about my proposal, I think I would change a lot of things about how I described the implementation. I mean, this is how I am about everything though. As soon as I get done with a given section of code I am ready to declare it a terrible failure and start again (Regardless of the fact that the code actually functions fine). Nevertheless, I think in this case my proposal could have been truly helped by some more thought and refinement. A lesson for next time I guess. As far as it being a difficult, or rather time consuming, task, I don't think there is much to be done about that.

I supposed I feel a little bit disappointed that my projects didn't get great responses from my peers, but I think I have learned from the experience and will be able to do better next time.

Composite Design Pattern

The Composite design pattern is a wonderful tool for dealing with sets of objects that can encapsulate other objects. Generally speaking, if you can describe a set of object compositions in terms of a tree, it is probably a good candidate for the Composite design pattern.

The idea is actually quite simple. You can have two types of object, a Leaf or primitive object, and a Composite object. The Leaf object does not encapsulate any other objects, but the Composite does.

An interface is provided that treats all the objects, Leaf or Composite, as the same type. Using this general interface, clients pass commands to the objects without respect to whether or not the given concrete object is a Leaf or a Composite. If the concrete object is a Leaf, then the command requested is executed directly. If it is a Composite, then the command is propagated to the children objects. This children, may be Leaf objects or they may be Composite objects themselves. In the latter case, the command is further propagated out through all of the Composite objects until it reaches all of the Leaf objects and thus traverses the entire tree of objects.

This pattern allows you to create very general code where the client of the interface you provide is spared any ugly details about what type of object it is dealing with. The pattern is often used in graphical applications, but is very general and can be applied to a large variety of problem domains.
It is a great pattern and one I will strive to use more frequently in my code.

An Aside on Specs

For this class, I ended up writing too proposals. These proposals weren't really specs, but they were something in-between a spec and a marketing document.

I have written a few specs in the past, and every single time I remember having to "update" the spec after the fact to deal with issues that we encountered during development. Which seems, somewhat counter to what a "spec" classically is, a description of the software system with exacting detail. The details are of course the problems, because as it turns out in reality the only true description of the software system or protocol or what have you, is the implementation of said system.

This isn't to say that specs are useless, they are incredible useful. They are a guide that should be followed within reason, and updated when they describe something that is either impossible or highly impractical.
I'll finish this aside with a email that I say on the Linux Kernel Mailing lists a while ago and I stumbled across again today.

From: Linus Torvalds

To: Arjan van de Ven Subject: Re: I request inclusion of SAS Transport Layer and AIC-94xx into the kernel Date: Thu, 29 Sep 2005 12:57:05 -0700 (PDT) Cc: Willy Tarreau , SCSI Mailing List , Andrew Morton , Linux Kernel Mailing List , Luben Tuikov , Jeff Garzik Archive-link: Article, Thread
On Thu, 29 Sep 2005, Arjan van de Ven wrote:

a spec describes how the hw works... how we do the sw piece is up to us ;)

How we do the SW is indeed up to us, but I want to step in on your first point.
Again.
A "spec" is close to useless. I have never seen a spec that was both big enough to be useful and accurate.
And I have seen lots of total crap work that was based on specs. It's the single worst way to write software, because it by definition means that the software was written to match theory, not reality.
So there's two MAJOR reasons to avoid specs:

• they're dangerously wrong. Reality is different, and anybody who thinks specs matter over reality should get out of kernel programming NOW. When reality and specs clash, the spec has zero meaning. Zilch. Nada. None.
  It's like real science: if you have a theory that doesn't match experiments, it doesn't matter how much you like that theory. It's wrong. You can use it as an approximation, but you MUST keep in mind that it's an approximation.
• specs have an inevitably tendency to try to introduce abstractions levels and wording and documentation policies that make sense for a written spec. Trying to implement actual code off the spec leads to the code looking and working like CRAP.
  The classic example of this is the OSI network model protocols. Classic spec-design, which had absolutely zero relevance for the real world. We still talk about the seven layers model, because it's a convenient model for discussion, but that has absolutely zero to do with any real-life software engineering. In other words, it's a way to talk about things, not to implement them.
  And that's important. Specs are a basis for talkingabout_ things. But they are not a basis for implementing software.

So please don't bother talking about specs. Real standards grow up despite specs, not thanks to them.

  Linus

Touchable

Having become largely disillusioned with my initial project, I decided to propose an entirely different product. Much of the feedback I initially received dealt with general issues that are applicable to many projects and I tried to address them here in this proposal. Admittedly, it is somewhat of a gamble to entirely change this late in the game, but I feel very strong about this new product so I believe it is worth the risk.

Feedback is welcome!

http://cs.unm.edu/~zdevex/touchable.pdf

Software Craftsmanship

I have often found myself thinking about the ideas of Software Craftsmanship, well before 460. The news outlets theses days are overrun with stories of poor software causing tremendous amounts of damage, both financial and otherwise. As someone studying computer science, I know all too well that we in the field are to blame for this current climate of dangerous programs.

There have been many many studies on proper coding practices, as well as proper project management. The strong consensus of these studies has shown that taking time to write quality (maybe not perfect) software is not only more secure and stable, but it also costs less money and time to create. It is better on all fronts, there is no downside. Of course one can not go overboard in this process, and spend weeks upon weeks writing ten lines of code, but simply writing code with reckless abandon is a known recipe for destruction.

Despite it being well known that there are practically no disadvantages to writing high quality code, people continue to hack things together in unmaintainable, unexpandable, insecure ways. I can't understand why we as humans do this. What is wrong with us? Are we so naive that we believe that we as programmers don't need to follow proper practices, because we think we are above these practices?

3 words

Primary: Mature, Responsible, Honest

Secondary: Creative, Adaptive, Hardworking

Proposal Review : Carpooling 505

Here is a link to the proposal I reviewed, Carpooling 505.

http://cs.unm.edu/~zdevex/proposalReview0.pdf 

Proposal

http://cs.unm.edu/~zdevex/proposal.pdf

Agile Reactions

Recently in my software engineering class we discussed the ideas of Agile Software Development. The discussion seemed to illuminate several things to me, primarily that there is this good idea called Software Requirements, there is this good thing called Agile Software Development, and there is this bad thing which is the common programmer's perception of each.

Often when I encounter people who advocate for the Agile style, they function as if Agile means the following.

• Don't comment your code (because your code is self documenting).
• Don't write specs, because that isn't flexible.
• Start coding right now (Like if you are still reading this sentence and not coding you aren't doing agile right).
• Don't write tests (Because hey I have madz hacking skillz, my code doesn't have bugs).

When I encounter people who advocate for the more classical approach (which is not too often these days) they seem to think the following.

• Plan everything before writing any code.
• Understand everything before doing anything.
• Never change from the spec/standard.

From our discussion, it really seems like real Agile and Requirements based development are two sides of the same coin, and not either of the two concepts I described above. Requirements does seem to focus more on, figure out what you are going to do before you start coding, while Agile does seem to emphasize flexibility more, but it seems like they are both reactions to the same thing, poor programming. You need to know what you are going to build before you build it, you need to write tests, plan, and define the problem space. At the same time you need to be flexible with your client and their needs, as much as is possible. You need to not spend all your time planning, but you do need to spend some of it planning. At the same time you do need to actual create a product at some point.

In reality it seems that Agile and Requirement are both abstractions on good design and engineering practice, which isn't to detract from them at all. After all this is Computer Science abstraction is our most powerful tool.