And while I’m rambling on about music software, this is the coolest thing I’ve seen in a long while:
VoiceBand is software for the iPhone that lets you do multi-track recording using your voice as a controller for various instruments. Very clever indeed.
This is a lightly tweaked version of the report I submitted to the University after returning from Magnolia’s first-ever conference, in Basel, Switzerland. I realize I need to write up a more general-interest trip report, but have been dreadfully short on time since our return. I have, however, recounted our adventures often enough now that it should be a straightforward bit of writing; I’ll do it soon!
In September, my friend and coworker Jeff Snider and I attended the 1st conference for the Magnolia CMS, on which our Gato system is based, in Basel, Switzerland.
We presented two talks at the conference: Case Study: Magnolia at Texas State University (mostly Jeff’s work) and Surfacing External Data through Magnolia (mostly my work). Our talks were well received; people seemed to particularly enjoy the window into our implementation experiences.
Other presentations covered various open-source Magnolia modules that may be of use to us, Apache Sling (which looks like a great foundation for building web-service-based content applications), some basic details on clustered instances that will serve as a good starting pointing for setting up a clustered editing environment, and the new UI and architecture for the upcoming Magnolia 5. (We were unable to attend the last, since it was opposite our own, but the slides cover the information fairly well.)
The most valuable thing for us, however, was to get to meet the Magnolia staff and community. We were able to put faces to many of the names that pass across the user support list, and to discuss lots of aspects of Magnolia with the folks who are directly responsible for implementing it. (And one of them saved Sean during his presentation when he made a mistake in my data entry — thanks, Philipp!)
While the conference itself was not without the occasional hiccup — to be expected, given that this was the first time it had been held — the chance to build relationships within the community, learn from the other presentations, and to present Texas State’s experiences made the trip a worthwhile one.
Over the past several years at the University, we’ve had to hire for a variety of programming and programming-related positions. One of the interviewing practices we’ve adopted is to give candidates a programming assignment which they can complete at home, at their own pace using any resources they can muster. Because these conditions approximate the conditions under which they would normally be working, we find this to be a really good indicator or what sort of work they’ll do if hired. (Besides, it’s one of the practices from The Joel Test! We currently score about 8.5/12.)
One of the jobs we’ve been hiring for is what I refer to as the “Quality Assurance Czar” — someone who will spearhead and organize our efforts to make sure the code we’re fielding is as awesome as we want it to be. The programming test for this includes both writing a small program to compare two hands in poker and, more importantly, writing a suite of tests to verify that the program works correctly. (This is not the exact programming challenge we use. It was actually one we considered, but it’s a thornier problem than one would guess at first glance, and there are a number of published algorithms out there already. We decided to make up our own problem so that applicants couldn’t just google for an answer.)
A few months back, one of my good friends — let’s call him Larry — interviewed for this position. He did a fine job in the interview, and while his code was very good, it didn’t include the comprehensive testing we were looking for. He wrote a small PHP application that generated a couple of random hands and then compared them, relying on the person running the program to verify that the winning hand the program chose was correct. His solution was 100% accurate, but didn’t include any automated tests, as he had a hard time imagining what sort of testing would have been useful in that case. Here’s a note he sent after the position had been filled:
I was real proud of my [Poker] app, the one that was supposed to include a testing approach. Now I’ve tried Selenium, and I still don’t see how to test my app, because it has no user inputs.
What did you guys ever have in mind?
Okay, I can see these tests:
- entering the URL brings up the page.
- A refresh causes a change to the page (a new checkers arrangement).
But that don’t seem adequate.
He is, of course, completely correct. While this would test a few trivial aspects of the application’s functioning, it would leave the important and juicy bit of code, analyzing the poker hands, completely untested. (This is called “incomplete code coverage” in the argot.)
There are several things one can do to make this application more testable. First off, one needs to create a mechanism to feed the program a specific input. Once you can do that, you can give it a couple of predetermined hands and verify that it chooses the correct one. Doing this with a variety of different hands will demonstrate that the program is working correctly in those cases.
There are, however, about 2,400,000,000,000,000,000 combinations of two 5 card hands. For obvious reasons, we can’t test all of those. How do we choose which ones to check?
Let’s start with easy ones. Verify that a flush beats a pair. Verify that a full house beats a flush. Verify that four of a kind beats three of a kind. Verify that a pair of 8s with hearts and spades beats a pair of 8s with clubs and diamonds.
Once you have the basics covered, we want to look for “edge cases”. These are where our data bumps up against boundaries of some kind. They’re very important because “off by one” errors are very common in programming, and these tests help to expose those errors. In poker, for example, we would want to check that an “A2345” hand would count as a straight, and that a “10JQKA” would also count as a straight, but that “QKA23” would not.
By this point, we’ve accumulated perhaps a few dozen tests — a miniscule percentage of the possible total, but strategically enough chosen that we have confidence that things are working as they should be. If we later discover that we’ve missed something and that our program is inaccurate in some cases, we simply add another test that demonstrates the failure, and then work on our algorithm until all the tests are once again succeeding.
Next, it’s important to try invalid data and make sure that our program handles the error condition as we would expect. Asking the program to evaluate a hand with 5 aces of spades is meaningless, and should generate an appropriate message for the user. (Something like “You’re a dirty cheating dog” might be appropriate.) Having the 17 of acorns as one of the cards in your hand is similarly out of bounds, and should also trigger an appropriate error.
In Larry’s program, all of the logic was stored in a single PHP file. Though it’s a bit more work to set up, I would recommend using a Model-View-Controller design pattern, and splitting the “evaluate the poker hand” portion of the program from the “show the poker hand on screen” portion and the “handle user input” section. By doing so, we can test these portions of the code independently. If we know that the code that evaluates the hand does so correctly, but it’s showing the wrong answer on the screen, we have immediately narrowed the code that might contain the bug to just the bit that’s responsible for presentation.
Finally, once we’re using MVC, it becomes practical to use one of the many excellent unit testing frameworks that encapsulate a lot of hard-won experience doing effective testing. Since Larry was working in PHP, PHPUnit and SimpleTest appear to be decent options. Using an established testing framework has a number of advantages: the code for common testing tasks (like logging into a site) will already be written, it becomes easier to integrate our tests with other systems, test output can be reported in succinct, attractive forms, and other programmers will have an easier time understanding how our tests work.
So, in summary, we’ve updated our program to accept input, fed it a number of test cases and verified that it’s returning the results we expect, adopted MVC and a standard test framework. We now have both a high degree of confidence that our current code is working as expected and a “safety net”: if we decide to refactor the program or change the algorithm we’re using to compare hands, we’ll know immediately if anything we’ve done has broken it.
For a more comprehensive introduction to testing and using it to drive development, see this article. And if you’ve followed this, want to learn more, and are interested in a job, our QA Czar position is open once again!
Last October, a couple of my coworkers and I presented at [High Ed Web 2008]. The conference organizers have, at long last, posted the transcript and audio recording of our session. The quality of the transcript is fairly rough, and they didn’t include the visual aids (which, by the way, I put a good deal of time into converting to a format they’d asked for), but the audio quality is good. If you have an interest in our experiences with content management systems at Texas State University, give it a listen!
One of the things about writing software for a living is that customers are rarely happy when you tell them “It will be done when it’s done.” Even though development is fraught with uncertainty, it’s still incumbent upon us who practice this craft to be able to give people some idea of what kind of effort will be involved to implement a particular feature.
One way to do this is to have a project manager or senior programmer sit down, think really hard about it, and come up with numbers. This has the advantage of being expedient, but the disadvantage of being almost certainly wrong. The reasons for this are many: it’s hard to estimate a job accurately when you’re not the person doing the work. All of us as individuals possess blind spots. And programmers are notoriously optimistic about how hard it will be to do something. (Not as optimistic as marketing people are when faced with a programming task, but still…)
One of the approaches I’ve seen that seems to promise the best results over time is Evidence-Based Scheduling. By having programmers estimate the tickets they’ll be working on, then record how much time they actually spend on it, comparing one to the other, factoring in how much productive time they have during the day once meetings, email, and the ilk are all done, and doing a lot of math, one can come up with good quality estimations — especially as one accumulates data on which programmers tend to exaggerate and which think they can finish nearly anything before lunch today. This approach has the advantage of producing really good estimates, but requires detailed tracking both of estimates and actual work time to produce good results.
Somewhere in the middle is Planning Poker, an approach that allows teams to collaboratively create estimates with a minimum of fuss. The basic idea is that you get all your developers together, give each of them a deck of cards with a time estimates written on each card, and have everyone choose how long they think a given task will take by throwing the corresponding card on the table. If there’s not agreement, everyone discusses it, explains their reasoning, and then you do it again until there’s a consensus. (You can read more details here if you’re interested.)
When we’re planning what to do for the next phase of a project, we have to consider two things: how important an issue is, and how much work it will be to do the fix. The latter has, up to now, been done off the cuff by whomever of the developers happened to be in the room. But in the past week, we’ve added a field to our ticketing system for estimates and started doing Planning Poker sessions to capture estimates for the tasks ahead. These seem to be going well, and I’ve noticed some interesting things about them:
So after two sessions, I’m counting Planning Poker as a success, and have scheduled regular sessions to finish estimating our backlog of tickets and to handle new ones as they show up. Many thanks to Mike Cohn for the great writeup that got me excited about it!
I’ve long been fascinated by the prospect of wearable computing devices. Having ubiquitous access to information — movie times, news stories, restaurant reviews, product information, word definitions, novels, friends’ locations — seems like such a compelling idea that any talk of such things rivets me. As cellular phones have become more powerful and capable, they’ve become the de facto way for lots of people to access that information. I’m not much of a cell phone fan, but do feel like a comparative dullard when I don’t have an iPod Touch in my pocket.
Thus, MIT’s “Sixth Sense” system seems to me the most drool-worthy bit of technology I’ve seen in a long time. By combining a camera, a cell phone, and a small projector, they’re creating a smart system that watches what’s going on around you and spontaneously provides contextual information. Hold up a plane ticket, and it projects the flight’s status right on the ticket. Walk up to someone, and it recognizes the person’s face and provides name cues and information on that person’s interests. Hold up your fingers in the shape of a frame and it snaps a photo. It’s fascinating stuff, and is nicely demonstrated in this presentation from the 2009 TED conference, which makes me want to move to Boston and apply for admission at MIT.
This afternoon, I noticed that one of my Google Calendars had utterly vanished. I tried all the troubleshooting steps available in their help forum, but it was still gone. I was concerned that I wouldn’t be able to get any real help, based on some of the posts in the forum, but I sent them a note anyway:
I had a calendar named TRCC that is for my church. Today I was logged into
to Google Calendars, and was unable to enter events. I logged out and in
(and cleared cache and cookies), and when I logged back in, the calendar
had vanished altogether. It’s no longer displayed in my Google Calendar
interface, and the page where it is embedded throws an error too:
http://3riv.org/calendar (“Invalid calendar id:
5nuaouifu4j7ndhiunae5mmu3s@group.calendar.google.com“)It’s showing every sign of being gone altogether, though I certainly didn’t
delete it. 🙁
By the time I made it home, this was waiting in my inbox:
Hi Sean,
Thank you for your note. We’re sorry to hear that you’ve experienced the loss of your calendar’s data. We have reviewed your account, and noted that the calendar ‘TRCC’ had been deleted. Please be assured that we will never delete your calendar data. Your calendar may have been deleted by someone else who had ownership access to this calendar.
We have gone ahead and restored this calendar, and it should now appear in your calendar list. Please keep in mind that you can keep a backup of your events by exporting your calendar data. To do so, follow the instructions at http://www.google.com/support/calendar/bin/answer.py?answer=37111
Regards,
The Google Team
Sure enough, the Calendar is back intact. Thank you, Google Folk! You have impressed me once again.
My friend David Barnard, who runs the iPhone software company App Cubby, posts an interesting piece on the financial realities of running such a business. Well worth a read if you dabble in such things.
Several months ago, I noticed that one of the nearby offices was using a whiteboard mounted outside their office door to keep people informed of their comings and goings. “That’s good communication!” I thought to myself, “but my team is made up of geeks. Surely we can do something nerdier!”
Thus was born an experiment with Twitter that is yielding some good results. It began with our six-person development team: programmers who were interested in participating set up a twitter account and began posting what they were up to through the day. We subscribed to each other’s updates and easily keep a finger on the pulse of the team’s activity. (This was especially helpful when working off-site or stuck in meetings.) In addition to the standard Twitter functionality, we also wrote a little Rails app that uses Twitter’s APIs to build a replacement for the old-style In-Out board. People can pull up the status screen from a web browser or look at a monitor we stuck in the window of one of our cubicles to see at a glance who was around and what they were up to:
Status Board In Situ
After a month of voluntary participation, we had a meeting to discuss whether it was worth keeping up, and decided that the minimal effort required to improve our communication was well-spent. We’ve been using the system regularly for several months now. (One interesting side-effect that I’ve noticed is that it helps me to keep on task a little better: when I post that I’m working on ticket #726, I feel a little guilty if I’m not actually working on it!)
Of course, people walking by quickly noticed the screen and asked whether it could be expanded to include other people as well. At my boss’ requenst, we put together another status screen for use at the department’s front desk, where the people there can quickly ascertain whether a call can be transferred or a visitor directed back to someone’s office. It looks like this (with strategic blurring to protect the unwitting):
Front Desk Status
The color-coding is done by preceding a status update with “IN:”, “OUT:”, or “OFF:” (for those times you’re working but not at your desk), a feature of our status board app. While some of the folks around the department were initially a bit skeptical, they have quickly become enthusiastic once they use it for a day or two. The front-desk folks decided that they’d rather have timestamps than the “x hours ago” notation we’d originally used, so we tweaked the app to better meet their needs.
We’re still refining our software (I need to make the caching smarter), there are at least two other Status Board applications under development in the department (using Javascript and FLEX), and I now quickly hear about it if something goes wrong with the system. It should be interesting to see where this thing goes over the next few months! (If there’s interest, I can clean up the Rails code and open-source it once I finish the caching improvements.)
I’m currently at the High Ed Web 2008 conference in Springfield, MO. The conference is geared to web professionals in Higher Education, and is an interesting combination of marketers, technical folk, and the occasional vendor. There’s lots of good information to be had (as well as an immense amount of equally good food — who suspected that “travel is broadening” should be taken literally?), and all of us who are attending are getting a lot out of it.
Yesterday afternoon, James, Jeff and I did our planned presentation: University-Wide CMS Implementation: Failure, then Success. We had a lot of fun with it, taking a pretty irreverent approach to the history of our CMS project, and using some cool 3D timeline software to present our information in an interesting way.
Conference attendees have been using Twitter to communicate about the conference pretty extensively, so I told people at the start of our presentation that I wanted them to help me make all the people in other sessions feel as bad as possible for missing ours. Toward that end, I encouraged them to post the most outrageous lies about our presentation they could think of. Here’s the Twitter chatter that was posted during our presentation:
I’m having a great time getting to enjoy some of my work friends in a more social setting than usual, as well as meeting people from other Universities and Colleges around the country, getting to see Springfield, and taking photos with team members in luchador masks in front of local landmarks. We have another day and a half of conference to go, after which I’m looking forward to the chance to meet up with my Uncle Rick and some old friends who also live in the area.
Sidenote: before leaving for the trip, I told the kids that I was going to Springfield for a conference. “Cool!” said Liam. “You’ll get to meet the Simpsons!” I launched into an explanation of the fact that one of the running gags on the show is that they don’t ever let on where Springfield actually is, and how in the movie Ned Flanders points to the four adjacent states, Ohio, Nevada, Maine and Kentucky, which are of couse widely spread. Thus I wouldn’t actually get to meet the Simpsons, because the Springfield they live in is a fictional construct, not the one in Missouri.
Ten minutes later it occurred to me that I would also not get to meet them because they’re cartoons.
Yep, I’m a dork.