This is it. If everything goes according to plan, I will have finished all of undergraduate work before I go to sleep tonight. After this, I guess the "real" world. Though something tells me I've been in the "real" world all along.
I am pretty excited to move on, start new things, get some sleep. It has been a rough ride to get here. Only a few hours left to go.
I just came from the CS computer lab where we put together our final tarball to send to the TA. It was a good time. James brought donut holes and we sat around and talked and laughed while we compiled all the files together.
I am happy with our team, and the way things worked out.
So we placed second in the class. I have to honestly say that I am very surprised. It is not that I didn't think that our project was good, but I thought the other projects were amazing. Then again, those teams said similar things to me too.
It was a good feeling to see the work you've spent so much time on appreciated, and to have everything come together. I think we are all pretty happy with how it turned out.
We are currently in the process of preparing for our final presentations on Tuesday, and a seemingly trivial question is leaving us stumped, "What to wear?". We all don't really have any idea how to dress for these things. We know that how we present ourselves can often play a major role in how well our product is received, but we don't know what attire matches our goals. Our initial thought was of course "suits", but not everyone in the group owns a suit. On the other hand, the Google guys usually wear cargo shorts and t-shirt, the natural look of a young hacker these days. Is that what business people want to see?
During the process of deploying our application for on AWS, I ran into an interesting issue that we had yet to encounter during the entire development process, memory issues.
It turns out that the free tier of AWS gives you only 512M of memory. This is usually fine, unless you are using a memory hungry language like java (actually any garbage collected language will probably suffer performance issues with this little memory, there was a recent paper published on this. As it turns out the C people are right about garbage collection with respect to performance, however they are still wrong about a human being able to track garbage collection safely.). The last time we deployed the application, we didn't have any issues, but since then there have been some system updates, our code has changed, etc. We must have been very close to the limit last time, because the crash actually takes place on code compilation not runtime.
Anyway, to remedy this I had to upgrade our AWS server to the non-free instance. Thankfully this is only $0.044 per hour, so it won't break the bank.
So, this weekend we have been going through all the issues associated with actually deploying a production ready version of the application. This isn't the first time we have put the site up live, but this time we made extra sure that all the web related features we wanted worked correctly. Which, of course, they didn't.
By "web related features" I don't actually mean anything related to the code of the application, but rather the webserver, https, DNS, etc.
It is easy to see how so many servers can be misconfigured so easily, with the wealth of options and settings that one must tweak in order to get the behavior you desire.
I am uncertain how I will be able to actively participate in the final presentation of our project. For the first two trial presentations, I didn't say anything except my name. This isn't because I was not involved or didn't know what was going on, quite the contrary. However, we only have ten minutes and are already just about using up that time, and other people strongly wanted to be speakers.
In general, I don't really care too much about speaking, I just want things to go well. At the same time, being the only silent observer (Sonny drives the demo, so he isn't "observing") makes me look like the tag along member of the team.
But I keep coming back to the issue of time. I don't wish to "steal" parts from my teammates, and I certainly could add more to the content being presented, but where do we find the extra time.
The idea that we are currently floating, is to have me perhaps do the conclusion. But with the way things have gone, that is still basically nothing. Something like, "And that's Mechanapp, revolutionizing expert system based auto repair!". I mean, it will actually probably be at least three sentences, but it won't be much.
Hmm... what to do...
Currently our project has switched in to a mostly test mode. Almost all of the code that we are writing is just tests and bug fixes. We do have a minor issue that we hope to resolve tomorrow, two separate branches that need merging.
We diverged about a week ago to clean up the last two tasks in the application, and we have not had a chance to bring them back together yet. I don't anticipate any issues with the merge, but it is rather discomforting to not have it handled yet.
After this merge, I will sleep better. (Well, probably not, but one can hope...)
Group projects are inherently difficult. Many different people working on the same thing, with different skill levels, backgrounds, schedules, etc. It is hard. Often there is this idea that everyone must do an equal share of the load. I think that idea is silly and completely non-realistic. I have been on many group projects, in academia and otherwise, and I have never experienced people equally sharing the load. However, this does not upset me, rather I think it is the only group projects can in reality function.
When you start a group project, some people are going to have a higher skill level in the various areas of interest. This is natural, and even good. The best thing to do is then to divide those people up, based on their skills, and have each person work where he can do the most effective job. However, you will inevitably end up with multiple people working on the same task, with different skill levels. Expecting the less skilled person to do more or equal to than the more skilled person is insanity. Why would anyone expect that outcome? But if one is to take issue with this, then you will find yourself always dissatisfied with group performance. Rather, the metric by which you should be evaluating your compatriots should be their ability to perform the tasks they say they can, in the time frame they give. Perhaps you could have done it twice as fast, but that is not an issue as long as you are still working on other tasks than more work is still being done. The less skilled person, by performing the task becomes more skilled in the process, which helps the group as a whole.
To be more concrete, our project revolves heavily around web application architecture. This is an area that I am very familiar with, so I have more skill and understanding in the are than some of my teammates. As such, I have performed more work than some of my teammates. But I in no way feel that this means that they did not carry their share of the load. They did want they could, when they said they would, and I have been very satisfied. If our project had been focused on a different technology, say robotics, I would likely have been doing the least quantities of work rather than possibly the most, just because others would have had a better background in that area.
Modern applications are so complex and do so many things, that all but the largest of development efforts rely heavily on open source libraries to bootstrap themselves into a real working application. We are doing this in many different ways in our application. This includes grand large open source libraries, such as our framework, to smaller and perhaps more scary libraries like a google+ auth plugin (Not written by Google).
As the recent Heartbleed bug has show us, there are dangers to using such tools, but what else can a developer do? It would be some form of madness to constantly reinvent the wheel. Further code you write yourself is no less likely to not have bugs just as damning (often more likely).
How does one vet these libraries before including them in your application?
So this past week has been one of the busiest of my life. I have slept 15 hours in the past 4 days. I am not writing about this to garner a pity party, but rather to ask a question. What do you (whoever you are) do to stay productive when it seems like there is not enough time?
From the studies I have read, sleeping is always better than not sleeping, even when you are busy. But I don't know if the people who wrote these studies were computer scientist students also working 20+ hour a week jobs.
So when the rubber meets the road, I don't sleep and eat like crap. And I usually get everything done. What do you do when there isn't enough time?
As our project has become larger and larger I have noticed a much greater cost for adding new features into the codebase. This is almost always the case for any non-trivial project, however this cost seems particularly painful right now, perhaps due to the busyness of my life in other areas as well.
The latest feature that I have been working on adding to our application has taken much more time than I had thought to implement.
This is also one of my largest flaws, estimating how long it will take me to complete a task.
In the past few weeks, I have had a much greater desire to add new features to our application. Now that things are mostly together and I can see the product for what it is, it seems like it is hard not to see it for what else it could be. This is not to say that I am unhappy with our project I think we hit our goal and things came together better than I had expected.
At the same time, there are several things that I really want to add to our application. The rest of my team does thinks they would be great, but doesn't feel that they have time to add them. I have to say I rather agree, but it still irks me nonetheless. How does one "finish" a project?
As we get closer to the end of the project, more and more of the features that we wish to implement require less and less interesting work and more tedious work. This is not to say that the features themselves are boring, they are certainly not, they are quite cool to have in the application. However web applications involve so many moving parts, that in order to do them securely and correctly, you have to do a lot of boiler plate.
I find myself getting excited to work on a new feature, only to quickly lose interest as I realize all of the small indirectly related sub tasks that are much less interesting.
I wonder how one learns to make these small tasks more enjoyable, or if that is even possible?
This week my group had to do a mock presentation of our application to the rest of the class. It was a rather strange experience, in that there was not really enough time for the four of us to each speak in a fluid manner, leaving a single speaker and three generally silent members.
It felt strange to me, to be a silent figure at the front of the class. The role seems to imply some form of direct interaction upon my part with the presentation, although really there wasn't any. There was plenty of indirect interaction, through showing the body language of approval and interest in what my team member was talking about, but that was about it.
I was not dissatisfied with my team members presentation, on the contrary, I thought that it was very well done. It only felt as though there should have been more time, or I should have not been standing there. This isn't really a critique against the professor, because I am well aware that this situation occurs all the time. I just wish there was a more organic way to deal with this type of presentation.
Currently I am working on adding some new features to Mechanapp to allow for user login and persistent user session. Both of these items are in and of themselves not very complex or difficult to implement. However, implementing them in a good, safe, reusable manner is another story entirely.
User login, demands that we are sending user credentials back and forth on the network. This means that we must now start encrypting the traffic with TLS, as well as purchase a valid TLS cert (if we want anyone to actually use this application). Further both login and persistent user session (by which I mean tying state to a user id rather than a cookie), require that we introduce a second database that will be much more mutable than our previous one. Until now our architecture had remained pretty simple, but at this point it is getting more complex.
It is strange how such small features can have such a large ripple effect in the application.
Tomorrow we have our first demo presentation of our software. There has always been an age old question about these types of events, what should one wear? I know this sounds silly, but hear me out. Often at the large CS conferences, the main speakers will wear t-shirts and jeans, the formal attire of the programmer. Is this the way to go? Or are these people just being lazy? I often think that people who wore a suite to such events would either have to be James Bond cool or they would not be taken seriously. In our final meeting we will be presenting to business people, which lends itself to a more classically formal attire, but what do you wear for other CS people in a formal engagement?
Over this semester I have had a very interesting experience with meetings in my group. Generally our group has met at least once a week often more than that, and observing these meetings has been strange. Often it seems that we are get very little done during the course of the meeting. We discuss some high level topics and that is about it. It seems that most of the real discussion actually takes place in email chains and the like. This is not to say that the meetings are bad, on the contrary they are perhaps the some of the most enjoyable times during my week. It seems that our group has the fortunate (or unfortunate) issue of getting along too well. We get along so well during our meetings that is is hard to keep them from becoming just hang out sessions.
Reflecting upon this, I am not actually sure it is a bad thing. Certainly we have somewhat of a hard time accomplishing the tasks we wish to accomplish during the meeting, but they do get done shortly thereafter in emails and other forms of communications. Also I feel like the group bonding time is really good for our cohesiveness as a team.
So at this point we are pretty late in the game. There are only about 6 weeks left in this semester, and that is really not a lot of time. This puts us in a awkward position with our project, and by us, I mean me. There are several large new features that I would like to implement before the semester is done, but I don't know if we will have time to get them in place. I really feel that they are killer features, but with so little time, I feel that I don't want to lobby for help from other team members until I am at least sure I can get them off the ground.
This puts me in a very sticky situation. Work on new features that may never see the light of day, or focus on polish of the existing codebase?
So I am an emacs guy. I love emacs. I want to use emacs for everything I code. Unfortunately, emacs does not have great support for Java, which we are using for our project. In the past I have used Eclipse for Java, but this time through I thought I would try Intellij IDEA.
Intellij is a very good IDE, but it doesn't really understand the web framework we are using with our application, and so it tells us that our code is broken, even though it isn't. So for the first time ever, I decided to attempt to code a large Java project in emacs. It has been an interesting experience.
I find that I understand the application much better writing it in emacs. I have to manage all the imports myself and as such I have a much clearer understand about how the dependencies work. On the other hand, I definitely code Java slower in emacs.
I am still not sure which way I prefer for coding Java. I mean emacs Java features are terrible, but emacs general features are awesome.
Perhaps I will make another post later in the semester after I have made a more formal evaluation of the two in comparison to each other.
As we are moving into a new phase of development in our application, I find that our development roles are changing. As we started out work on this application, there was a clear division on labor. We each had our own part of the codebase that we owned.
Now that things are farther along, and many core features are nearing a complete state, we are having to shift roles working more intimately on each other codes.
This is an interesting transition for us and it is revealing a lot of issues we hadn't noticed before. For instance, until recently I only had a passing understanding how our database setup worked. Before now, I didn't need to know how it worked, it wasn't my code, I just used the API that my teammate made. Recently I had to work more directly with the DB code. I found myself spending quite a bit of time just working through the code before even getting to the issue, even though it had been in the application for a long time.
I think all of us are having this issue. It will be a little bit of a set back as we have to learn to work with code that we didn't right, but probably a good experience.
As of a few days ago our application is in a really nice state. Most of the features that we needed to implement are implemented. This really feels like a good place to be in. Now our development will probably shift to more candy features. Accounts, better GUI, more useful user feedback, etc.
I am finding it very hard, as we move into this set of feature development to stay really excited about these features. I find myself wishing to develop more large sweeping features, rather than these small ones. It is difficult to remind myself that it is these small features that will actually make the application successful. The devil is in the details, as the phrase goes.
It is a strange realization, that the use of a particular tool is not generally a function of its usefulness, but rather its polish. I mean, it still has to work, but it seems what really makes an application successful are these candy features.
As of a few days ago my team has assembled a full working stack for our project. This felt like a very good milestone for us. Prior to this we had a lot of the pieces together, but there was always some fear that assembling them would cause the entire thing to fall apart.
While this feels like a great victory for us, it is also quite scary. I feel like we are almost finished with the product, and yet, there is still so much time left in the semester. More so, there is still a lot of important pieces we need to put into place, particularly, we need to start building our data set to drive our application. While the core program is in a mostly functional alpha state, it is useless without data to drive it.
I guess this is the "Sag" that was discussed in class. I know I am starting to feel what that word means, in more than one way. So much done, but still much more to do.
As our project has matured somewhat we have run into some interesting issues with our VCS. We are using git for our VCS and it is great! One of it's killer features is the ability to create a lightweight branch very easily. This is, in my experience, much easier than the same process in SVN. This feature has proved to be very useful to us, as each of us can work on our own code with out mucking up the others, and still share it to the master repo so that others can pull it when they wish. However, we have used it so much that we have begun to experience branch cruft.
We now have many branches, many of which are technically dead, but it can be hard to tell which branch has the features you wish to pull. You can of course, examine the commit log for each branch, to see which one has been active recently, but this process seems very tedious.
The solution for this problem was trivial, just add a prefix to each branch for each week, so that we know which are active. However we only learned this simple technique after a week of dealing with many branches of unknown activity.
Recently I have noticed that there are a lot of programming languages targeting the JVM. Aside for Java, there is Clojure, Groovy, Scala, JRuby, Jython, and Rhino, as well as a handful of other less notable languages.
It is interesting how all of these languages flocked to the JVM to bootstrap themselves into all of the work that has been done on a powerful bytecode runtime. Without doing any extra work they get platform portability and runtime optimization (maybe, I am not totally sure about this dark magic.).
It is interesting to ponder what might happen if Oracle stopped developing the JVM, or closed source future updates to it. All of these separate entities would have to come together to maintain the runtime, since they are essentially dependent upon it.
Even more interesting is the fact that if a programming language isn't running on the JVM, it is pretty likely it is a scripting language. Ruby, Python, PHP, JavaScript all require no compilation to run. It seems that the ideals of creating portable code have now been met with either the JVM or a script.
Over the past few days I have been working on selecting the web backend for our project. Last client meeting I had narrowed it down to django, a python based framework that is popular right now, and play, a Java/Scala based framework that is also becoming quite popular.
Evaluating which framework to use has been quite a challenge. They all seem to support all the features we care about. They are all expressive and "easy" to use. They all seem great.
As a team, we have a stronger background in Java, but we are also Computer Scientists and should be able to pick up a new (newish as we have some prior experience with Python) language in just a few days.
But web frameworks are much more that a library for a given language, they are a meta language unto themselves. The web has evolved in such a way that when we "surf the internet" we are using a number of technologies at once, many not designed to work with each other, connected together in strange an opaque ways. Web frameworks allow the programmer to once again be a programmer, rather than a Rube Goldberg machine master.
Further complicating the issue is the fact that are so many good web frameworks. The only way to truly compare one framework to another is to site down and write an app in both frameworks. Alas, this means learning two frameworks (or more), which is not a task that the busy student enjoys. But it must be done.
The other day a friend and I were attempting to compile some Java code which we did not write ourselves. While we both are very experienced with the Java, neither of us had written or built any Java code in quite some time. Further we did not have Eclipse installed on the system we were working with, so we went about the daunting task of building the java program from the command line.
It took us a good ten minutes at least to get the code to finally compile. The experience reminded me just how frustrating it is to work with the Java build system. At one point in time, I knew the system in and out. I learned all the nuances of it just long enough to write a bash script to do it all for me.
When I come across issues like this I often think about how much some tools separate the programmer from what is actually happening with his code. Before I wrote the aforementioned bash script, if you had asked me to compile one of my Java projects without eclipse I would have had a very hard time, and I wasn't a novice Java programmer.
Java and Eclipse aren't the only examples of this, many many programming languages have complex build systems, and because we often use IDEs that build our software for us, we never actually understand how it works.
I can't decide if this is a good thing or a bad thing. Certainly automated building of software is good. But when I build software in C I write a makefile which runs commands that I understand (although there is some magic in the inference rules of POSIX make that I think most people gloss over). When I build Java in eclipse I am totally separated from the process.
Does the modern programmer need to know or care about this stuff? Or should he just spend his time on more interesting problems?
reset(),
next(), isDone(), and getItem(). These functions allow for
resetting the traversal to the beginning of the Collection, moving the
iterator to the next element in the Collection, check to see if there
are any more elements after the current element, and getting the
element at the current location of iteration, respectively.To: Arjan van de VenSubject: Re: I request inclusion of SAS Transport Layer and AIC-94xx into the kernel Date: Thu, 29 Sep 2005 12:57:05 -0700 (PDT) Cc: Willy Tarreau , SCSI Mailing List , Andrew Morton , Linux Kernel Mailing List , Luben Tuikov , Jeff Garzik Archive-link: Article, Thread
On Thu, 29 Sep 2005, Arjan van de Ven wrote:
a spec describes how the hw works... how we do the sw piece is up to us ;)How we do the SW is indeed up to us, but I want to step in on your first point.
Again.
A "spec" is close to useless. I have never seen a spec that was both big enough to be useful and accurate.
And I have seen lots of total crap work that was based on specs. It's the single worst way to write software, because it by definition means that the software was written to match theory, not reality.
So there's two MAJOR reasons to avoid specs:
So please don't bother talking about specs. Real standards grow up despite specs, not thanks to them.
- they're dangerously wrong. Reality is different, and anybody who thinks specs matter over reality should get out of kernel programming NOW. When reality and specs clash, the spec has zero meaning. Zilch. Nada. None.
It's like real science: if you have a theory that doesn't match experiments, it doesn't matter how much you like that theory. It's wrong. You can use it as an approximation, but you MUST keep in mind that it's an approximation.- specs have an inevitably tendency to try to introduce abstractions levels and wording and documentation policies that make sense for a written spec. Trying to implement actual code off the spec leads to the code looking and working like CRAP.
The classic example of this is the OSI network model protocols. Classic spec-design, which had absolutely zero relevance for the real world. We still talk about the seven layers model, because it's a convenient model for discussion, but that has absolutely zero to do with any real-life software engineering. In other words, it's a way to talk about things, not to implement them.
And that's important. Specs are a basis for talkingabout_ things. But they are not a basis for implementing software.
Linus
Recently in my software engineering class we discussed the ideas of Agile Software Development. The discussion seemed to illuminate several things to me, primarily that there is this good idea called Software Requirements, there is this good thing called Agile Software Development, and there is this bad thing which is the common programmer's perception of each.
Often when I encounter people who advocate for the Agile style, they function as if Agile means the following.
When I encounter people who advocate for the more classical approach (which is not too often these days) they seem to think the following.
From our discussion, it really seems like real Agile and Requirements based development are two sides of the same coin, and not either of the two concepts I described above. Requirements does seem to focus more on, figure out what you are going to do before you start coding, while Agile does seem to emphasize flexibility more, but it seems like they are both reactions to the same thing, poor programming. You need to know what you are going to build before you build it, you need to write tests, plan, and define the problem space. At the same time you need to be flexible with your client and their needs, as much as is possible. You need to not spend all your time planning, but you do need to spend some of it planning. At the same time you do need to actual create a product at some point.
In reality it seems that Agile and Requirement are both abstractions on good design and engineering practice, which isn't to detract from them at all. After all this is Computer Science abstraction is our most powerful tool.
#include <stdio.h>
#include <stdlib.h>
int
printMessage(char* message);
int
getName(char* message);
void
checkExit(int code);
int
getName(char* message)
{
char* name = malloc(sizeof(char)*10);
int i = 0;
int err = 0;
char c = 0;
while((c = getchar()) != '\n')
{
name[i] = c;
++i;
}
err = printf("%s %s\n", message, name);
free(name);
return err;
}
int
printMessage(char* message)
{
return printf("%s\n", message);
}
void
checkExit(int code)
{
if(code < 0)
exit(code);
else
return;
}
int
main(int argc, char* argv[])
{
int err = printMessage("What is your name?");
checkExit(err);
err = getName("Hello,");
checkExit(err);
return 0;
}
#include <stdio.h>
#include <stdlib.h>
int printmessage(char* message);
int getname(char* message);
void checkexit(int code);
int getname(char* message) {
char* name = malloc(sizeof(char)*10);
int i = 0;
int err = 0;
char c = 0;
for(; i < 10 && (c = getchar()) != '\n'; i++)
{
name[i] = c;
}
err = printf("%s %s\n", message, name);
free(name);
return err;
}
int printmessage(char* message) {
return printf("%s\n", message);
}
void checkexit(int code) {
if(code < 0)
exit(code);
else
return;
}
int main(int argc, char* argv[]) {
int err = printmessage("What is your name?");
checkexit(err);
err = getname("Hello,");
checkexit(err);
return 0;
}4c4,5
< int printmessage(char* message);
---
> int
> printMessage(char* message);
6c7,8
< int getname(char* message);
---
> int
> getName(char* message);
8c10,11
< void checkexit(int code);
---
> void
> checkExit(int code);
10c13,15
< int getname(char* message) {
---
> int
> getName(char* message)
> {
15c20
< for(; i < 10 && (c = getchar()) != '\n'; i++)
---
> while((c = getchar()) != '\n')
17a23
> ++i;
24c30,32
< int printmessage(char* message) {
---
> int
> printMessage(char* message)
> {
28c36,38
< void checkexit(int code) {
---
> void
> checkExit(int code)
> {
35,39c45,51
< int main(int argc, char* argv[]) {
< int err = printmessage("What is your name?");
< checkexit(err);
< err = getname("Hello,");
< checkexit(err);
---
> int
> main(int argc, char* argv[])
> {
> int err = printMessage("What is your name?");
> checkExit(err);
> err = getName("Hello,");
> checkExit(err);
20c20
< for(; i < 10 && (c = getchar()) != '\n'; i++)
---
> while((c = getchar()) != '\n')
22a23
> ++i;
if(a == b){
a++;
}
return true;
if(a == b)
a++;
return true;
The article on Wikipedia about software engineering seem to speak to me of one thing in general. Although people are being taught "Software Engineering", being hired as "Software Engineers", and writing books on the subject, people are still not quite sure what "Software Engineering is. Particularly, people seem to be unclear on how to separate it from Computer Science. At first glance, it seems to be some formalization or the practical application of Computer Science, but if this is the case does this mean that Computer Science is not in and of itself practically applicable? Perhaps...
And in other ways it seemed as though the article described Software Engineering as a commercialization of purely academic Computer Science.
Much of the article focused on discussing aspects of Programming that related to reusable, correct, code, such as testing, design, following standards, etc. Maybe the practices thought in Software Engineering could be summed up as teaching Computer Scientists how to not be lazy programmers? It does seem that this concept may be at the heart of the field. After all, an truly overwhelming amount of data shows how much of a negative effect that lazy and sloppy programming, testing, and documentation can have on a project, and these are areas that often even the most brilliant Computer Scientists struggle with.