A Project Retrospective

About four and a half months ago, I took on a project to build a customer portal; it was a new project, for a new business. In the beginning, I was overwhelmed with nerdy excitement, excitement at the possibility which this project presented. If you’re a developer, your eyes are likely lighting up at having such a prospect.

A new project for a new business? No legacy code to deal with? No baggage from the past to have to work around? No maintaining an existing (read boring) codebase whilst the new, shiny, wonderful, all-singing-and-dancing^TM is built?

What a fantastic opportunity — even luxury — to have been given. And in part, you’d be right in thinking all of these thoughts. But, while it was an excellent opportunity having no legacy to contend with, it wasn’t without a flip-side.

Think about it. Greenfield projects are only green for a very short period. Eventually, they too become legacy projects. If we’re not careful in these situations, we can very easily make the mistakes of which we accuse others.

It can be easy to create a foundation which is easy to corrupt, whether by ourselves as we’re working through later revisions, or by those who take over the project after we’ve moved on.

So when I began, I tried to appreciate both the honour as well as the responsibility with which I’d been entrusted. I use those words with care. To provide further background, my remit was that this was a project which would not be a rapid turnaround, short-lived, agency-type project.

This was to be a long-lived project; one which would form the core of the business’ technology infrastructure, for so long as the business thrived and continued to exist. Given that, I began by considering, apart from the application’s architecture, one key requirement.

What Is My Continuous Delivery Approach?

From that, I had several further questions:

• How would environments be provisioned?
• What testing structure would the application be guided by?
• How would the application be deployed?

If you come from a background of smaller, shorter-lived projects, these questions may seem like expensive luxuries. But I don’t work in that space — a conscious choice. I believe in, and work in, long-term applications, which deliver value over the longer term. I believe in well thought out, well designed, yet practical, code.

Given that, these three questions aren’t luxuries — they’re necessities. It’s my belief that without these, any project, no matter how new, can rapidly descend into big balls of mud. These questions alone, however, won’t save a project from becoming a verbose bucket of drivel, suitable only for label as excrement.

But when taken seriously, they support the formation of a mindset which can lead to better applications, applications which either don’t go down the route of madness; or if they do, can be readily salvaged, and brought back on track, with a reduced investment.

Given that, I built a provisioning environment using Ansible and Vagrant. In this, I saw two advantages:

1. Minimise the Possibility of Environment-specific Bugs
2. Reduce - Even Eliminate - Delay for New Developers

Minimise the Possibility of Environment-specific Bugs

As all environments would be almost identical clones of each other, including the same versions of all packages, extensions, and libraries, then the possibility of bugs because of different environments was minimised.

That’s not to say that something couldn’t go wrong, such as the hardware which a node sat on being faulty, or a hardware (or software) router within the data centre or rack going down.

But it reduces the likelihood, as much as is practically possible, or an error owing to the environment. On this score, I’ve done well. I’m happy with the result, and aside from a minor oversight, has made deployment almost hassle-free.

Reduce - Even Eliminate - Delay for New Developers

How common is it, whether you’re just starting in a new company, or on a new project in your existing company, that time’s wasted before you can get underway adding features or fixing bugs? I’ve found it to be an all too common experience. But it doesn’t need to be that way.

Matter of fact, this type of delay can be all but eliminated. Given that Ansible’s being used as a provisioner, no developer needs to have special systems administration, or DevOps skills to get running. If you can run a terminal script, then you can be up and running inside 15 - 20 mins.

What’s more, it’s not only development where this comes in handy, but for setting up any other environment, whether that’s testing, staging, QA, or production. Through a well thought-out, and structured Ansible configuration, all of this is achievable.

On this score, at least at the start, I did well. However, over the course of the project, I became remiss in keeping it up to date.

I found out recently how much I’d varied from it, such that it isn’t a smooth process like it once was. This isn’t a disaster, though. But will take an investment to bring it back up to scratch.

Choice of Tools

Now let’s turn our attention to the choice of tools. This one, for a period I was a bit mixed in my feelings. But now I have no concerns or questions. Specifically, here’s the shortlist of the tools which I chose:

• PHP 5.6 (soon to be 7)
• Zend Expressive
• PostgreSQL 9.4
• Codeship
• Deployer

PHP

If you know me, then some, if not most of these, will come as no surprise. I’ve been developing, primarily, in PHP since I first came across it back in 1999 (if not before).

Zend Expressive? Given I’m a long-time advocate of Zend Framework, and regularly write about it, then it’s almost a no-brainer.

PostgreSQL

PostgreSQL, now there’s a different story. I’ve been a long-time MySQL user, like a lot of my peers and colleagues. But I’ve always had a bit of a nerdy love-affair for PostgreSQL. However I never really committed to using it, not fully.

I’d dabble with it, then back out if push came to shove. This time, however, I stuck with it. And I’m very — very — glad that I did.

Specifically, the reason that I’m enamoured of my choice is that it has support for Common Table Expressions, or With Queries. This is a feature which turned out to be critical - a feature not supported by MySQL.

If I’d been on MySQL, at least to the best of my knowledge, I’d either have to have written a lot of code, or to have ported the application to PostgreSQL in a hurry, resulting in greater development time and cost.

As far as I’m aware, only Microsoft SQLServer, Oracle and PostgreSQL support CTEs.

Codeship

Then there’s Codeship. I’ve been friends with the team there for some time now, first getting to know them some years ago, the first time I came across the service.

If you’re not familiar with it, it’s a managed deployment service, one complete with a well refined and polished interface, backed by a team of very competent and experience professionals.

It almost sounds like I’m on the payroll, no? Seriously, though, I’m not. I do write for them on a semi-regular basis. But that’s as far as it goes.

I rave about them because the service has always worked flawlessly for me, plus their technical support is top notch.

Through using the service, I’ve been able to create a hands-off fully-automated continuous deployment solution for the project.

Deployer

Then there’s deployer, a PHP-specific project for deploying applications. Honestly, I had no real reason to choose Deployer over other tools, such as Make, Ant, Phing, or Capistrano.

Each of these are excellent choices, very flexible in nature. I went with Deployer because it was PHP-specific, flexible, and accommodating. I found it easy to create the deployment steps (or tasks) which I required in a minimal amount of PHP code.

OK, there was a bit of newness which helped encourage me. But don’t conflate newness with shiny and pretty. I don’t like those kinds of projects for professional work.

Testing Regime

Besides the architecture, here’s where I feel most proud. Despite a few lapses here and there, the project was almost 100% driven by tests. For the last number of years, I’ve been ever more of the opinion that tests are essential.

I say ever more because I never started out that way. But thanks to lots of support from two mates, Tom Oram and Chris Hartjes, I’m an ardent believer in all forms of xDD; whether that be TDD, BDD, DDD, or some other *DD.

People may argue that a test-driven approach leads to a longer development time; and if they’re referring to the initial development period, then they’d be right.

But saying that it takes longer over the entire lifetime of the project is wrong. Test-driven code does take about 25 - 40% longer to produce. But that code often has 70 - 90% fewer bugs.

I’m happy to make that trade-off. The testing approach I took was using Codeception. If you’re not familiar with it, Codeception is a BDD-style testing framework that, out of the box, allows for rapid creation of unit, functional, and acceptance tests, all manageable through a command-line interface. It’s too much to talk about in one post.

If you’d like to know more, I strongly encourage you to check out the official documentation.

What’s more, Codeception has PhpUnit as its foundation — the gold standard for testing in PHP — and has a number of modules which provide a range of functionality, such as checking records in a database, files on filesystem, and for hooks into most of the popular PHP frameworks.

This made checking service configurations, and services I created in the Expressive app nearly painless. On this point, this was a win.

Client Communication & Managing Expectations

This is a big one and one that can’t be understated. While we can be masters of code, able to do the seemingly superhuman, we have to remember that what we do is meant to help people.

Whether that’s saving them time in their daily jobs, making certain processes quicker, easier, cheaper, or unnecessary, our applications are tools, and tools are designed to help people.

Given that, at some point, more often than not when you’re a freelancer, as I am, we have to interact with people. We have to discuss requirements, give status updates, point out omissions and defects, discuss timelines, and much, much more.

On this point, I’m going to give myself a mixed score. While I always focus on being to the point, and transparent, at all times, in all communications, I don’t feel that I communicated enough.

It’s my belief that I could have done more, could have gone further. At no time has anyone given me any indication that I was remiss.

But I feel that I was. So bearing this in mind, I’ll be communicating more regularly, more consistently in the future - until I feel that I’m regularly maintaining a baseline standard of quality.

Overall Feeling

My overall feeling, is that the project’s been handled better than I’d anticipated. Sure, there are areas which still need improvement. But, on the whole, it’s been a smooth development period. I’m keenly looking forward to the next project, as well as to see how this project evolves over time.

CC Image (background of main image) Courtesy of Don Merwin on Flickr