Building treehouses in code
I’ve been working on my CFUnited presentation titled “Reviving the Lost Craft of Writing Specifications” and thought I’d publish a teaser that might make it into the final version. This is raw, rough, uncut and unedited so beware.
//
Sometimes, a client knows what they want in the end, but really truly can’t tell you all the component pieces to get to that goal. One day, you get around to reading Robinson Crusoe and fall in love with living in a tree house. You know that a tree house is exactly what you need to make yourself happy forever and ever. So you hire an architect, buy them a copy of Robinson Crusoe and tell them that you want a tree house like the one in the book, only modernized. The iterative programming folks would have you climb up in the tree every day with the work crews while they saw and nail and screw and prune, asking you questions while you’re in the tree about attachment points, wind-shear factors and various things you know that Robinson didn’t think about when he was building his tree house because they weren’t in the book. After the structural builders are done, the trim guys are up in the house every day putting up crown molding and running pipes and lights and they keep asking you questions like, “Would you prefer the 15 watt halogen soft-white bulbs or the 45 watt incandescent natural glow bulbs in these pendant lights?” If you wanted to be wrapped up in that level of detail, you’d have built the damn thing yourself. Sometimes you just want someone else to execute your vision. You pick a dynamite tree-house builder, someone with dozens of houses under their belt, especially of the shipwrecked-looking-variety, you read them some passages from Robinson Crusoe and then tell them to call you when it’s done. And if you pick a really good builder, it’ll be just like you dreamed it would be.
To be the software developer equivalent of this latter type of builder, you need to do two things well: learn and communicate. The learning comes from your ability to rapidly acquire a deep and wide understanding of the business domain at hand. If it’s the futures trading market, you need to learn the difference between a pork belly and a cheddar barrel, a short sell and a long put. If it’s healthcare billing, you need to learn the difference between CPTs, DMEs, and CCIs. If it’s electronics manufacturing, you need to learn the difference between servo motors, teach pendants, PLBs and weld tips. In order to get that understanding of the domain, you need to be a good communicator, and get buddy-buddy with another good communicator who already is an expert of the domain you’re learning (generally called the “subject matter expert”). If you become a domain expert, you can translate the fuzzy, ambiguous ideas that your customer spouts out into a workable software design, and the whole time, poke holes in his harebrained schemes that won’t work because of domain rules.
Once you’re a really good builder, you can make customers feel involved in a process in which they are ultimately not involved by offering them lipstick-on-a-pig options, like with the style sheet. Should the logo be more “swooshy”? How about adding a horizontal rule between the sub-navs? Some customers dig this kind of interaction. Some don’t want to be bothered.
Getting to the point where you are a really good builder also has a drawback: You are opinionated and must express those opinions in the face of potential scorn. It goes something like this:
A not-so-experienced builder will dutifully take notes during meetings, distill the essence of a decision into the spec doc, send out the spec doc for review and ensure that the finished application follows the guidelines set out in the spec doc. If you’re building a social-networking app to let users watch paint drying, and your customer requires that all users log in just to view other people’s paint drying, you’ll spec it as such and build it as such.
But an experienced builder would be able to cross over into the business-decision making arena and question why the customer wants to force viewers of drying paint to log in. What’s the overriding business need for that? Won’t it place an onerous burden on the casual viewer who wants to check out some drying paint? The natural answer from the customer would be that they want to capture email addresses and some other contact information to sell other complimentary services (like watching grass grow) to people who watch paint dry. The experienced builder suggests a compromise: Let any visitor view drying paint, say, twice on the first day that they visit the site. If they want to view a third, they are gently prompted to register with just their email address. Or if they come back tomorrow and watch some more paint dry, we have a pretty good idea that they’re habitual boredom-seekers, so they certainly might be interested in watching grass grow or waiting for kettles to boil. Through a slight change in the requirements of the system, you’re collecting a fraction of the email addresses that the customer’s proposed way would have collected, but all of the emails collected in your way are pre-qualified. That is, they’re actually boredom junkies as opposed to just people who StumbleUpon the site. That, along with the fact that pre-registration is proven to turn off users, so some of those genuinely bored visitors might not have even viewed that FIRST drying paint if they were forced to enter their email address in the beginning. This kind of critical thinking and collaboration on your part is what separates the merely adequate from the really good application builders.
you’re probably not soliciting opinions, but i thought i’d give you mine anyway. too many metaphors! I liked the treehouse idea at the start, which then gets dropped for futures trading, healthcare billing etc in 2nd paragraph, then paint-drying web apps in the final paragraph.
Comment by d. — March 13, 2008 @ 12:00 am
I agree with the comments about metaphors. Stay with one theme. In the last example, instead of using the "watch grass grow", you could stick with the treehouse. Visitors can enter the treehouse deck and mark their name on the exterior wall once a day or paint the exterior wall. But if they want to get into the treehouse, they need to create a login account.
Comment by Lola LB — March 27, 2008 @ 12:00 am
Thanks so much for the comments! As I’ve been working on this material, the editorial review board here at Webapper has nixed this entire section for brevity’s sake. I’ve got to axe some more content in order to get through everything in the time alloted to me at CFUnited. The immortal advice of Strunk & White declares: "Make every word tell." So I’ve got some chopping to do.
Comment by Nat Papovich — March 27, 2008 @ 12:00 am
At first I was inclined to address your over-generalized stereotype of iterative development and your horrible metaphors, and to educate you on iterative software development. But I realized it would be a book – and you likely wouldn’t be interested anyway. I will ask, however, why don’t you talk about software development instead of metaphors? Being an engineering professional with experience of over three decades, three dozen clients, and projects from web apps to operating system to defense systems to critical medical systems (including implantable devices) – all with a track history of delivering as promised, you may wish to step outside of your narrow view of the world and find out why I take issue with your post. Perhaps you can learn something. Your choice. Cheers!
Comment by Mark Graybill — March 27, 2008 @ 12:00 am
Hi Mark –
Thanks for stopping by and for leaving the comments. This section has been edited out of my final presentation, which I’ll be delivering in June. I agree that it’s a bit over-generalized and uses poor metaphors. That’s why I put it up here – for comments just like yours!
I’ve read the iterative book you wanted to write and still don’t agree with most of the principles, as do many successful developers and project managers. Iterative works for some, it doesn’t work for others, and others (myself included) take what we like from the Agile manifesto and make our clients happy. And if you attend my presentation, you’ll notice that I don’t knock iterative development in my final version – I only take umbrage with iterative developers who knock everyone else.
I prefer to educate and recommend to developers who are looking for education and recommendations. I’m not trying, nor would I ever try, to influence people like you.
Comment by Nat Papovich — March 27, 2008 @ 12:00 am
Mark,
Perhaps you could explain why someone with your impeccable credentials has such a ghastly website?
http://www.mark.graybill.com/
I’m not saying anything, I’m just saying…
Dan Wilson
Comment by Dan Wilson — March 27, 2008 @ 12:00 am
Hahaha! Thats one of the worst websites I’ve ever seen – maybe its one of those joke sites. Mark are you playing a character – like a super-techy know-it-all whose code looks awesome but the final result of all his applications is something un-usable and un-friendly with error messages like "Doh! I thought I told you already, a valid email pls! snort!"?
Comment by Baz — March 27, 2008 @ 12:00 am
Ha! That website IS the worst website I have seen in a long time. Maybe, you should have spent 3 days in the 3 decades to learn some HTML and CSS. Or you could go out and hire an architect and give them a copy of Frontpage 101.
Comment by Paul — March 27, 2008 @ 12:00 am
Let’s not be too hard on him. He doesn’t claim to be a user experience guy, nor a designer, nor even a developer. He only claims to be an engineering professional.
But still, he just might want to consider a site update…
Comment by Nat Papovich — March 27, 2008 @ 12:00 am
The important thing about principles is to understand them well enough to apply them in a specific context. Some of the principles for instance were used on a Lockheed project I participated on, but due to the nature of the project not all of them were used and the ones that were used were translated for the project. On a Medtronic project, I saw a similar thing but different principles applied. At Vital, we applied Scrum and have achieved a level of success not before achieved. If you truly know software projects, you’ll understand the why behind iterative principles. If you knew what I knew about human cognition and social psychology, you would understand them even more.<br><br>It is important to think critically or your success will be limited. You taking issue with my website as a way to discredit me is an example of a lack of critical thinking. It is simply social play because your web persona is important to you. It isn’t to me. The website was put together years ago to simply publish information. It was my first and wasn’t impressive even when it was new more than a decade ago, and it certainly is a joke today. I’ve done a few websites since but they were business-oriented, not flashy in the manner you probably care about. I don’t do them anymore because they don’t pay enough, and I haven’t updated my website because I haven’t been using it. In fact, you see it today because I just renewed my Netidentity account last week to include the website because I’m back in consulting to have time to finish my Ph.d. and I will need it – and to update it. Any suggestions are welcome.<br><br>My message remains though. Critical thinking, slow to judge, and expanding your thinking to understand alternative approaches are keys to success as a software professional. My team at Vital Images did very well to raise their thinking to a new level, improve their professional skills, and expand their perspectives – and they are already producing amazing work.<br><br>Cheers!<br><br>-Mark
Comment by Mark Graybill — March 28, 2008 @ 12:00 am
Mark,
There’s no doubt that you have a lot of experience in this industry and we certainly welcome your views and your comments, provided they don’t just act as a way for you to bash the author without some constructive criticism like "you may wish to step outside of your narrow view of the world and find out why I take issue with your post", instead perhaps take a more collaborative approach and maybe outline a few of the many things you seem to take issue with. I would think that someone who knew what you knew about human cognition and social psychology would know more about online social collaboration, which is why most technical blogs are about, don’t just leave a bashing remark to make yourself feel better about yourself, someone with your credentials should not spend time with such pety activities, I think that’s why you were received by others in this manner. Perhaps you can learn something more about dealing with people. Your choice. Cheers!
Comment by phill.nacelli — March 28, 2008 @ 12:00 am
Judging by my last post, I must have issues with my editor as it died on me when I was posting.
Just to be clear, I am a software engineer not a web app developer or designer. I hate doing user interfaces. I find other things more challenging like C++ communications code that processes medical images so a doctor can in 5 minutes know what treatment to apply by looking at, for the first time ever, 4D images of a stroke patient’s brain with full blood profusion maps. I also find interesting writing C++ code to control robotics in an automated rapid fire non-line of sight cannon, or C code for pacemaker features. I sometimes like to do point of sale systems for fine dine restaurants because I usually get to eat free when I’m doing them. Other times I’ll write C algorithms to quality check MRI images or to calculate the geometry of air traffic data in a precision approach system. In my experience base, the project diversity and the team size variations is more than sufficient to understand agile development.
Now understanding from the cognitive and social psychological side of the problems with corporate software engineering, I understand iterative principles and how to apply them. I didn’t at first because I was focusing on how to spec out better and more comprehensive requirements up front. But I refused to just react and decided to think critically and to explore it. I ignored the agile “evangelists” because I was looking for where my practice could benefit. If indeed found prudent application and benefit.
I recommend gaining an understanding of iterative principles and see where their benefit may apply to your work. If you don’t think iterative principles apply to web development, there are those who would disagree and are accomplished at it. Instead of bashing, go seek them out. Perhaps there is something you can benefit from in whatever you do for a living.
The reason why I ended up at this site is because of a post in one of the agile forums taking issue with this blog. After reading it, I felt inclined to comment in the hope you might be interested in a new level of thinking. If you don’t think I have anything to offer because my website is crap, that’s your choice.
Comment by Mark Graybill — March 28, 2008 @ 12:00 am
I certainly could have phrased my original comment better. I left my superficial comment mostly because your self-aggrandizing original comment seemingly didn’t deserve a whole lot of time nor energy.
The angle I was coming from wasn’t just that your website was very poorly done both technically and aesthetically, but that you list such skills on your resume and appear to have very poor mastery of them. Thusly, you undermined your own credibility, I merely pointed it out.
I don’t have any firm opinions of Agile over Non-agile methodologies. Your original comment has done little in the way of providing any useful information that would steer me, or other information seekers towards your line of thinking. Simply claiming "I am smarter than you" as a retort to someone who doesn’t agree with your line of thinking doesn’t much model proper Critical Thinking ™ does it?
Dan Wilson
Comment by Dan Wilson — March 28, 2008 @ 12:00 am
Dan – BTW, I didn’t have a problem with your original comment. It was fine.
Apparently my comments didn’t bode well in this social scene, but they were on purpose. Mine was a response perhaps over-fitting for the kind of language in the original post – as others have noted in the past, it was my Mr. Hyde. It was also inappropriately projecting my own values. When I hear about someone’s experience and it is more than mine, I tend to listen to them and find out why they took issue with whatever the topic I’m communicating on – in the hope I’ll learn something.
Your comment about undermining my own credibility is based on false premises and fallacies, and is not applicable. If I was touting web app skills and I just finished my website to prove it – and that was the topic, then you have a valid point. Otherwise, it seems to me your point was an ad hominem response from your affect reaction to my post. Regardless, the situation underpinning your premise is not the case and I don’t do web programming (which is ironic given this website). Listing on my resume the use of a script or markup language in a project or an era doesn’t constitute me advertising I’m a web programmer. In fact, this exchange is simply social and has no bearing on the topic at all. But, for the sake of managing perceptions, you do have point that has been noted.
Interesting how you viewed my posts as alluding to being smarter. I never wrote what you quoted or used language close to it. I did explain my background to convey I was sufficiently experienced to know the difference because of the experience. I’m sure there are plenty who read this – including you – who have higher IQs or EQs and who organize their intelligence to do amazing things I cannot. Conveying I am smarter was never my intent or the message.
The original post was an affect diatribe of sorts, likely based on a general attitude against the iterative paradigm. The post was likely then borne of exploring justification of that attitude. However, it provided little reference to specific problems taken with Agile I could address. I was hoping my original post would draw out of Nat those specific issues outside of a metaphor, which could then be explored.
Again, this seems to be more rooted as a social debate, which I don’t know why I’m continuing in. I will say I intended to shake up the thoughts of Mr. Papovich. I will also contend some form of specification always needs to be provided. But specifications by themselves do not promise anything, and are inherently flawed for reasons few software types understand or care about. The foundation of Agile development is outlined in the Agile Manifesto (www.agilemanifesto.org) and should be the basis of instruction for executing an Agile project.
Dan, I’m curious why you capitalize and show a trademark for “Critical Thinking”. Can you explain?
Comment by Mark Graybill — March 28, 2008 @ 12:00 am
Agile software development is not at all about bothering the customer with implementation details. It is about getting out a usable system as far as possible, and then adding features in priority order.
Taking your tree house metaphor, the very first iteration could simply produce a platform in the tree, for example. Then the customer could decide whether next he wanted something to protect him from rain, something that protects him from wind, something that allows him to see inside the house when it’s dark etc.
Comment by Ilja Preuß — March 29, 2008 @ 12:00 am