I presented at the Philadelphia Knowledge Slam tonight on job satisfaction and Agile. It was a lot of fun! The hardest part was putting together a coherent presentation that fit within the strict 5 minute limit, with no slides allowed. There were 10 great presentations on a wide variety of topics: the songs of Robins, the latest innovations in genetic treatments for sickle cell disease, screenwriting, cultural myths and personal myths, baking, tips for networking, the mis-measuring of educational achievement, and more.
This was my first time going – Knowledge Slam is held the 3rd Wednesday of every month. Check out the Facebook page for more info.
Short clips of each presenter were recorded. Here’s mine, followed by my complete script.
About 4 years ago I read a book by Malcolm Gladwell called “Outliers: the Story of Success.” Buried in the middle of that book he wrote a few paragraphs that, for me, were the most important part of the story. He described the 3 things that make a job rewarding. The things that make you look forward to a day at work when you get up in the morning.
First is reward for effort – this means money of course, but it also means recognition. We want our boss and our co-workers to let us know we’re doing a good job.
Second is having challenging work – work that isn’t routine and boring, but isn’t so hard that it becomes frustrating. Work that’s in that sweet spot in between, where the work engages your skills and makes you feel that you are learning and growing.
So those first two are pretty straightforward. The third one is the most interesting to me: a rewarding job is one that gives you autonomy. You have a feeling of control over your work, and you feel that your actions and decisions are meaningful. You can make things happen without someone second-guessing you all the time. It’s the opposite of feeling like a cog in a machine.
This struck a chord with me because at the time I wasn’t really happy in my job. I create web sites and web applications for a living. I’ve been doing it since ancient times – the early 1990s – when the first web pages were painted on cave walls in bison blood. And I wasn’t alone in feeling this way. Job satisfaction surveys of Americans show that between half and three quarters of Americans are unhappy in their jobs. If you consider that we spend about half of our waking lives at work, that’s a depressing statistic.
So I decided it was time for a change, and I made a terrible, terrible decision – I went into management. I joined the ranks of the people who are ultimately responsible for all those unhappy workers. I figured, there must be a better way to do this. So I did my homework, and I started learning about this thing called Agile, with a capital A. It’s a way of managing work that originated in the software industry and has been spreading to other types of work. And it’s got a great name, who doesn’t want to be agile?
But I learned it’s more than just a buzzword. Learning and following Agile practices made me fall in love with my work all over again. I would need to talk for at least an hour to explain how it all works, but since I just have a few minutes, I’ll focus on the part that relates to this idea of autonomy. In a lot of workplaces, you have responsibility, and your boss has authority. You don’t have autonomy. Managers talk about being results-oriented, but most are really more focused on control. Since you don’t have autonomy, you may not be motivated to do great work, so you’re given more policies and procedures to follow. The end result is management gets work that meets a consistent but minimal level of quality, and you don’t get a whole lot of job satisfaction. The undercurrent here is a lack of trust.
So how does Agile fix this? First, it gets management’s focus where it should be: on results, not control. And it provides some new ways of measuring progress and results that don’t depend on micro-management. And second, it adjusts peoples’ roles, so you actually have authority over the things you are responsible for. It gives you autonomy. It’s really about training management to get out of the way for the day-to-day work, to foster a learning environment, and to step in only when help is needed. It means treating people like adults, and creating an environment of trust.
And when you have trust, great things can happen. People start working together and pooling their skills to solve problems. This happened recently at General Electric. They had a water heater that was made in China. Here in the US a team of engineers, factory line workers, even sales and marketing people, all got together and completely redesigned it. By pooling their skills and experience they came up with a new design that was so much less expensive to manufacture, GE moved the manufacturing for the water heater back to the US, creating jobs here, and lowered its retail price by $300.
At the end of the day, its not policies and procedures that get the credit for good work and great products, it’s enthusiastic and empowered people.
WordCamp NYC 2012 kickoff
09-Jun-2012 09:13, Canon Canon PowerShot SD780 IS, 3.2, 5.9mm, 0.04 sec, ISO 400
With 12 session tracks on Saturday, followed by unconference sessions on Sunday and encore performances of Saturday’s most popular presentations, WordCamp NYC last week was by far the biggest WordCamp I’ve attended. Then of course there’s the real reason people go – the after party on Saturday (there was also the sponsors and speakers party on Friday, which was a lot of fun too). At the parties I got to do what years of social training had previously convinced me was unacceptable: discuss code while drinking beer. I got to chat for a while with @garthkoyle (from Event Espresso), @jason_coleman (of Paid Memberships Pro fame), @vidluther (from zippykid), and @tinakesova (from Siteground).
Brad Williams speaking on WordPress security at WordCamp NYC
09-Jun-2012 09:37, Canon Canon PowerShot SD780 IS, 5.8, 17.9mm, 0.05 sec, ISO 640
My WordCamp NYC presentation on object oriented programming for WordPress plugins
My presentation was on object oriented programming for WordPress plugins (my slides are below). I decided to focus on OOP in general with PHP, as its simply a huge topic to try to cover in 30-40 minutes. The room was full and there were good questions at the end, so it went well. I was 1 of 3 presenters from WebDevStudios – Brad presented on WordPress security, and Eric on the rewrite API.
WebDevStudios WordCamp NYC dinner
Brad treated us to a great dinner afterwards (ribs!) and I stayed up too late with the WebDevStudios team Saturday night (including honorary team member for a night, Captin Shmit – yes that’s how he spells it). I found out at 2AM (when I checked the conference web site after we got back to the hotel) that the unconference presentation submission I made earlier in the day had been scheduled for Sunday morning.
Whiteboard from my WordCamp NYC unconference session on Agile project management
So on my way back to the conference Sunday morning I stopped at CVS to get post-it notes and dry erase markers, for doing an Agile project management workshop. Aside from scribbling some notes 20 minutes before the session started, I didn’t have a detailed plan, and it turned out to be a lot of fun. My session was right before lunch, which turned out to be great, as almost everyone stayed after the time was up, and we went for another half hour. I started the session by having everyone describe their client relationship and software development problems (in brief post-it note format) and collected those on one side of the white board. Then I had them describe the things they want to achieve in their business (also in post-it note form), and collected those on the other side of the board. Then we spent about an hour talking through how to get from one side of the board to the other. It was only enough time to scratch the surface of Agile practices, but what made the biggest impression on everyone is how almost all software development teams face the same challenges, and that there are ways to deal with them that are concrete, achievable, and rewarding. ContentRobot selected it as one of the best sessions.
To continue with the shameless self-promotion, here are some tweets about my talks:
And here are the slides from my OOP talk (the last half of the slideshow contains my slide notes).
If you want to see more, check out the WordCamp NYC 2012 site and ChrisDigital has a collection of links to other summaries, and slides.
In the last episode of WP Late Night, there was a brief debate about plugin size. Ryan expressed a preference for smaller plugins with tightly focused functionality. It’s natural to worry that, as the number of lines of code increases, so does the likelihood of bugs, and performance slowdowns.
This concern makes sense if you’re assuming plugin code is not very well organized and not very well tested. Unfortunately, that’s a safe assumption with many plugins. As plugin authors, we should have higher standards for our work. There are two things that come to mind:
- Optimizing the readability and maintainability of our code. This means writing “clean code” (the subject of my Philly WordCamp presentation). It reduces the likelihood of bugs, and makes our plugins easier to enhance and adapt to changing needs. It also makes it more likely that others will contribute to the project, or learn from it, and start writing better plugins themselves. These are some one of the main attractions for participating in the WordPress open source community, right?
- Optimizing for performance: one aspect of the “clean code” philosophy is to not prematurely optimize code for performance, especially when our guesses about how to do that are often wrong.
More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason – including blind stupidity.
Wulf, W. A. “A Case Against the GOTO,” Proceedings of the 25th National ACM Conference, August 1972, pp. 791-97.
The rule of thumb is to optimize for readability and maintainability first. If a performance problem comes up, it is likely stemming from a small area of the code, and you can focus your performance optimization efforts there. As one person put it: “There is far, far more money wasted on fixing and customizing and maintaining hard-to-read code than there is lost on inefficient code.”
There are many techniques involved with writing clean code. A foundational one is following the Single Responsibility Principle (SRP). Bob Martin has a very succinct definition of the SRP: “a class should have only one reason to change.” In his book Agile Software Development, he explains further:
If a class has more than one responsibility [more than one reason to change], then the responsibilities become coupled. Changes to one responsibility may impair or inhibit the ability of the class to meet the others. This kind of coupling leads to fragile designs that break in unexpected ways when changed.
If you follow the SRP, then it doesn’t matter how big your plugin is. What matters is how you use it.
The trick, of course, is figuring out what it means to have a single responsibility. WordPress itself helps you figure this out. When you call a WordPress hook or filter, it’s likely that you will want to create a class that implements what you intend to do for that hook or filter call. For example, if you call
add_shortcode, then you should have it instantiate a class that implements your shortcode. If that class needs to change, it will be only because your needs for the shortcode have changed. The shortcode logic is not tightly coupled to other parts of the code. Removing that coupling is an important step towards also removing that sinking feeling of fear when you start monkeying with the innards of some gigantic application.
Not every hook and filter call deserves its own class. Some are merely stepping stones to others and do not need their own class. For example, if you call
admin_menu simply for the sake of calling
add_options_page, one class is enough. Others may need more than one class to support them. But for getting your feet wet, having a class per hook or filter is a good place to start.
My Shashin plugin has a total of 55 classes and subclasses (you can see the code on GitHub). How can you find what you’re looking for in all those classes? It sounds horribly bloated for a WordPress plugin, right? It’s actually the opposite.
A coding habit that goes hand in hand with the SRP is the use of meaningful names. Each class in Shashin serves a specific purpose, and has a name that tells me what it does. If I need to make a change to the settings, I go to the Settings class; if I need to make a change to how album synchronizing is done with Picasa, I go to the PicasaSynchronizer class, etc. The majority of the classes are less than a couple hundred lines. With small, well-named classes and methods with clear purposes, when there is a bug, it’s usually not hard to find. And if I need to change something, I can make that change in one place with a greatly reduced fear of breaking something unrelated.
By using a class autoloader, such as mine, you can also save yourself the trouble of figuring out where to put
require_once statements, for loading your class files. With an autoloader, a class file is loaded only when “new” is called (so if you are worrying about performance with so many objects, they are only loaded when they are actually needed). How you keep track of object dependencies, and when and how you instantiate your classes, are what I’ll write about in my next post, which will cover using an injection container.
This past summer I started attending the Philly WordPress meetups, which led to an opportunity for me to speak at Philly WordCamp, which led to an amazing opportunity to work at WebDevStudios, with an amazing team. Today – Monday – was my first day on the job. I’m doing custom development work and soon I’ll get involved with project management. I’m starting with some enhancements to Baja.com. Friday was my last day at Penn, so my head is spinning a bit from the transition.
I’ve been part of the web team in Penn’s School of Medicine since 2004, and I’ve been Director for the past 2 years. My team had a lot of demands placed on them, with the need for projects outpacing what we could provide. One of the first things I did as Director was learn Agile practices so I could teach them to my team (and I brought in a scrum coach to help). These two graphics illustrate the two primary challenges we faced – not enough staff, and too much chaos:
Blue represents available SOMIS staff, red indicates how many people would be needed to complete the desired work
Illustration by Esther Derby showing a not very well functioning development process
For many months my work spilled over into nights and weekends as I tried to move things forward. I can’t say we entirely solved these problems, but we made a lot of progress, and got the wheels turning in higher levels of administration to address the situation (“turning the aircraft carrier,” as one of our project managers put it – change is not easy to implement in a huge institution). Deciding to leave was hard, but an opportunity to turn my WordPress plugin development hobby into a job, to work with Brad (@williamsba) and the WebDevStudios team, and having the flexibility of working at home… it was too good to pass up. I’m especially looking forward to having more time to code again. If the past 2 years have taught me anything, it’s that I have a passion for software development, and always striving to do it better.
Last week I gave a presentation to my team on dependency inversion and dependency injection in PHP. Dependency inversion is the “D” in the the SOLID object oriented design principles, and dependency injection is a design pattern for implementing it.
We previously learned the single responsibility principle (PDF) (the “S” in SOLID). To me, dependency inversion is the next logical topic. Single responsibility is about how to write objects that do one thing; dependency inversion is about how you wire those objects together into robust, working software.
This was a fun presentation, and I think the slides work well even without my narration. Several of the code examples are from the new version of Shashin, where I’m applying the principles. I’m still learning how to best apply them, so feedback in the comments section is welcome.
A couple weeks ago I gave a presentation to my team, based on Malcolm Gladwell’s Outliers: The Story of Success. I focused on his analysis of mitigated speech (when we downplay or sugarcoat the meaning of what we say, usually out of deference to authority). Mitigated speech has been a key factor in numerous airplane crashes, when the crew did not voice safety concerns clearly enough to their captain.
Gladwell’s examples from black box recordings are astonishing. In many cases everyone in the cockpit except the captain knew the plane was in serious trouble, but even then wouldn’t speak up forcefully (typically the captains were good pilots, but were exhausted and weren’t picking up the hints from their crews). They’re powerful examples of the importance of being assertive, and just how hesitant most of us are even when it’s obviously vital to speak up. And my presentation was, I hope, more entertaining then some stereotypical HR assertiveness training class.
The last couple slides show similar examples from The Clean Coder: A Code of Conduct for Professional Programmers, about the importance of saying no and not being passing aggressive.
Here are my slides (posted on SlideShare.net)
A standard practice of Agile programming, and good programming in general, is to separate your display code from your application logic. Bob Martin:
The ideal is for a source ﬁle to contain one, and only one, language. Realistically, we will probably have to use more than one. But we should take pains to minimize both the number and extent of extra languages in our source ﬁles.
Separating display code is central to the MVC pattern, which underlies almost all PHP frameworks (and frameworks in many other languages as well). WordPress is a blogging platform and arguably a CMS, but for plugin authors, it does not provide the typical tools of a development framework. If you want to separate out your display layer, you’re on your own.
As a result, very few plugin developers do. A common style for WordPress plugins is a single file, sometime running thousands of lines, that consists of cryptically named functions (often in no discernible order), which have the display code scattered around within the core application logic. This style might be acceptable for something small and simple. But once the functionality becomes complex, this style makes the code very difficult to understand for anyone other than the author (and the author is likely to find it hard to understand six months after finishing work on it). It also makes extending the functionality difficult. For example, adding support for a mobile UI would entail a major rewrite.
For a WordPress plugin, you can use output buffering to solve this problem. You can see an example in my ShashinMenuDisplayerAlbum class. The run() method triggers the display of the menu, and its HTML is in a separate file:
ob_start(); // start the output buffer require_once($this->relativePathToTemplate); // get the HTML template $toolsMenu = ob_get_contents(); // store it in a variable ob_end_clean(); // empty the buffer and turn it off
require_once call would include the file, evaluate any PHP code, and immediately output the HTML. Including the file with output buffering turned on allows you to instead store all the output in a variable. You can then control when it is displayed. With this approach you do not have to write all your HTML inline with your application logic.
The template file being included in my example does have PHP as well as HTML in it, so I am not meeting the ideal of complete separation. But as Bob Martin points out, such total separation isn’t always possible. In this case, the PHP code used in the template is the minimum necessary for the purpose of rendering the page (for example, it contains a foreach loop to display rows of data in a table).
Ron Jeffries wrote a post a couple days ago with a title indicating its about databases, but then he proceeds into a test-driven development (TDD) coding exercise. Halfway through, an ostensible reader interrupts:
Hey! I thought this was supposed to be about databases???
Hey, yourself. Try to pay closer attention. This is totally about databases.
Why is this even remotely about databases???
Well, here’s why:
We have just defined a core element of our member database record, namely the purchase count, and made sure that it works. Now when we read a member record in from our database, we can instantiate it into our class MemberRecord and send it messages to decide what to do. This is OO here, my young padawan, and that’s how we do it.
Yeah, sure, old man, but what about the membership number? That’s the key, according to the story, and you don’t even have it. And isn’t there just a small matter of actually storing and retrieving these babies?
Patience, youngling. Each thing comes in its time. Let’s see what the next tests bring us.
His post is a good illustration of the revelation I had when first learning Agile coding practices: they let you start in the middle, with the business logic. A common traditional approach is to instead define a database schema first, and then deal with the UI and business logic. There are 2 problems with this traditional approach 1. the database is the hardest layer to change, and your design is likely most volatile in its early stages, and 2. it makes it that much easier to end up tightly coupling the components of your application together.
By starting with the business logic and no supporting infrastructure other than your tests, you have no choice but to fully decouple the application logic from the UI and the database. Then when your boss says, “we need to make a mobile version of this app by Friday!” you can work on the new mobile UI design without having to cut through a tangled thicket of blended UI and business logic.
Having said that, there are challenges with mimicking complex interactions with the database and UI layers through unit tests. I consider myself a novice with Agile techniques, but I’ll share what I’ve learned so far:
- For my Shashin project, which pulls data from Picasa json feeds, I simply saved copies of some feeds to local files, and my tests can talk to those files. This allowed me to keep developing while offline (on a flight to Hawaii!) which is very cool.
- There are the object mother and test data builder patterns but both can entail a fairly heavy degree of typing when dealing with large or complex data sets.
- My team at Penn is currently trying out FitNesse, which lets you specify your inputs and expected outputs through simple wiki tables (which you can paste from a spreadsheet). This seems about as lightweight as you can get. Although it’s an acceptance testing tool (not a unit testing tool), it can still help drive your design. If you follow specification by example practices, and write your code so that it can take inputs and send outputs to a FitNesse test fixture OR a database and a form, then congratulations, you’ve decoupled your code from the database and UI!
Toppa.com has become a quiet place in recent months. I’ve been spending what time I have on coding Shashin 3, so I haven’t been blogging. I have been applying Agile coding practices to WordPress plugin development, and it’s more than I bargained for. I could write a book about it, but I’ll probably never have time, so I’ll see how far I can get with a series of blog posts instead. Here are some key points I hope to expand on in upcoming posts:
- Testing: a typical WordPress plugin is sprinkled with calls to custom WordPress functions and hooks, so how can you unit test anything? Also, PHPUnit is a command-line utility, so how can you test plugin code that’s meant to run from within WordPress, which is a web app? The answer to the first question is to pursue a combination of isolating your WordPress dependencies, and running them through a facade (my toppa-libs library provides one, but it’s coverage of WordPress functions is still limited). An answer to the second question is SimpleTest, which is designed to run tests through a browser. I’ve written a plugin that runs SimpleTest from within WordPress, so you can do integration testing: SimpleTest for WordPress (which I hope to have up at wordpress.org soon). Update 6/15: it’s now available at wordpress.org
- Which PHP version to use? PHP 5.3 provides a decently comprehensive object oriented toolkit. But WordPress understandably stays at least a couple years behind the current PHP version, given the widespread use of WordPress (not every hosting provider keeps up with the latest PHP releases). The upcoming WordPress 3.2 release will have PHP 5.2.4 as a requirement, so I’ve decided to have the same requirement in Shashin. That means I can use type hinting and autoloading, but not namespaces.
- Code organization: WordPress gives you custom hooks and functions, but how you organize your code is completely up to you. As a result, a lot of WordPress plugins are huge, single files that are a jumble of code for display and application logic. In Shashin I’m using output buffering to create pseudo-templates for display code, and for the application logic, I’m sticking to the single responsibility principle, which means I have small classes and small methods.
- Performance: given the processing power of modern computers, an Agile principle is that you code for readability and maintainability first, and worry about performance only when there is evidence that you may need to sacrifice some of that readability to improve performance. Sticking to this principle is a challenge with WordPress. WordPress sites are deployed in a huge variety of environments, with differences in site configurations, traffic, and the number and type of other plugins in use. This leads to a natural tendency to code defensively for performance. I’ve tried to approach this not by sacrificing readability, but by having a dependency injection model that’s careful to not make unnecessary copies of objects, and by organizing my code to minimize any non-essentials calls when particular hooks are invoked.
- WordPress itself will try to thwart you in writing Agile plugins, as there is nothing Agile in how WordPress itself is written (this is not a criticism of its creators, as only since PHP 5 could you even really consider an object oriented approach with PHP, and WordPress has been around since long before PHP 5 became commonly deployed). A key decision is how far you want to go in isolating your code from direct dependencies on WordPress functions and classes (as they are not written against any interfaces). The more direct your dependencies, the harder it is to unit test anything (and then you can’t use your code outside of WordPress either). There is also no consistent error handling – some WordPress functions may return a WP_Error object, or false, or null, or something else. None throw exceptions. You’ll need to investigate each individual function to find out what variable type its return value will be under different circumstances.
I hope to get SimpleTest for WordPress up at wordpress.org soon, so I’ll have more to say about testing in WordPress next.
Something that set today apart from yesterday is that I had opportunities to talk with several of the speakers. When I arrived, Chet Hendrickson and Ron Jeffries were sitting by themselves at a breakfast table, so I joined them. They were having a conversation about human nature, and whether it doomed us all to failure (heavy stuff for 8AM). By the end of the conversation we were discussing doing Agile in University settings… and unusual experiences with plumbing.
I had lunch with Eduardo Jezierski, who just before lunch gave a riveting presentation: Architecture and Agility with Lives at Stake. He had many stories to tell. One was about developing, in real-time, a system to save lives after the earthquake in Haiti. He had a product owner in a tent at the airport in Haiti, with a low-bandwidth connection, tweeting user stories as he received calls and people came to him with information. He was right near the runaway for days, with C-130s constantly landing with supplies and taking off again. They were able to broadcast SMS messages to people with cell phones, telling them to text a specific number if they were trapped or injured and needed help, or knew someone who did. His team was developing a system to get those incoming messages into an RSS feed that could be accessed by local responders. It was also fed back to the Haitian diaspora in the US, where volunteers who knew the neighborhoods used Google Maps to help pinpoint locations where people were trapped, based on descriptive information. He said in situations like this, they’ve had to get help for their programmers who develop post traumatic stress disorder. When you receive a message like “I’m pregnant, I’m trapped in a collapsed building, and I’m hurt” and you’re trying to develop code in real time to get that message where it needs to go… well, you can get just a bit stressed. I was fascinated by the work his organization did in Haiti and elsewhere. They have often turned into the facilitators between governments, NGOs, and local communities, as they try to leverage IT to solve local health problems and address emergencies.
The other talks I attended were also good. But it’s past my bedtime, so I’ll skip summarizing them for now, and close by mentioning I attended my first Philly WordPress Meetup after the conference. It was a great group. I met the organizer, Brad Williams, and we talked about the next Wordcamp Philly this Fall. It’s still a ways off, but I may give a talk about my current experiences with applying Agile coding practices to WordPress plugin development.