When I first started at Frogslayer, it was a different workplace than it is today – project teams were smaller and there was more variety in the process of how each project was managed. This story starts when I had just joined the team early in my career, eager to make an impact. I had a lot of new experiences in that first year, but none quite made the impression as my first code deployment. I wasn’t in charge of deploying the code, of course, but given our small team size, we all ended up touching similar parts of the code.
The Problem
As we approached our first milestone in the project, we refined our work and fixed bugs along the way. When we were ready, we pushed our deployment to staging. Now, we had QA deployments already, but staging was a special feeling. It was the first time the code was labeled as a candidate for production release – a huge deal for a new developer like me! Incidentally, that deployment was made on a Friday, which I wouldn’t recommend as a strategy for so many reasons (as I’ll outline below)…
The lead developer and I tested out the fresh deployment with a couple of logins and basic data access. We checked the dashboard, looked around, and generally agreed – “nice job!” As we prepared to log out for the day and get the weekend started, I noticed I had been logged out from the deployed site. I shrugged and logged back in, figuring I had misclicked somewhere. A few moments later, the lead developer asked, “did you just do something?” He had been logged out, too. That was enough to cause us more than a little concern.
If you guessed that it was an authorization state issue on the server, you are right! It was a quick fix and everything was back on track. But that momentary panic taught us some valuable lessons for code deployments.
Key Takeaways
1) Agree on an attainable goal.
Setting development goals for a milestone is crucial before any issues arise. For this deployment, we delivered multiple components, which could have been broken into smaller, more defined chunks that were easier to test individually. To ensure success, it may have been more beneficial to independently deliver and confirm that the authentication piece worked before delivering other pieces of the milestone. This approach of breaking down the work into manageable pieces promotes buy-in and ensures that everyone is on board with the goal and the breakdown of the work.
2) Execute – do the work.
To state the obvious, you have to get the idea from your head to the end user. After setting the plan for the milestone, the next step is to collaborate and identify any overlooked details that need to be addressed. Once your team is satisfied that all necessary details or information has been gathered, it’s time to develop the code, submit pull requests, and complete the review process to ensure the final product meets the established criteria.
3) Test the same way users will use the application.
To effectively identify and resolve any issues, it is essential to test the application in the same manner end-users will utilize it. This deployment was for a web application, so of course, multiple-user support is required. However, we didn’t discover the issue until we both tried to use the application at the same time. Up until this point, we didn’t have multiple users test it. In a simple scenario such as this, having two people testing would work, but for more complex cases, it’s essential that you have testing tools that simulate how actual users will utilize the software. Certain bugs only pop up when you have a really specific combination of usage.
4) Don’t rush it!
If you have a deadline and it seems a feature or piece of the milestone might not make it in time, don’t simply rubber-stamp an approval without verifying it meets the criteria you established. If you must, raise a yellow flag with your team and ensure the delay is clearly communicated to both the team and the client.
If we had headed out the door the moment we both were able to perform a single login, we would have missed this issue and it could have emerged during the client demo in the following week. Because we both investigated further after seeing that something might be wrong, we were able to quickly resolve the problem, which helped give us momentum going into the next sprint.
5) Celebrate accomplishments, even if there were bumps along the way.
It’s important to give credit where it’s due. Acknowledging individual and team successes through positive feedback helps build rapport and reinforce good habits. By celebrating accomplishments, you have a base of validation that you can build from when you might have to provide constructive criticism later on.
6) Go home – it’s Friday!
Deploying on Fridays is risky. If anything goes wrong, you’ll have to rush to fix issues before the end of the workday. Moreover, you might be tempted to skip testing, which can cause more issues. You can bet users in production won’t avoid using your code just because you deployed it on a Friday!
Additionally, going into the weekend after a new deployment means issues will be discovered while you’re not working and may not be addressed until the following week. If a Friday deployment is absolutely necessary, it will require extra care and attention in the DevOps processes to ensure safety and easy rollbacks.
Looking back, we really shouldn’t have deployed on a Friday late in the day. But if you happen to do that, once the deployment is done and tested, go home, get some rest, and be prepared to dive back in when you return to work the next week.
After a number of deployments, they will start to become routine. However, it can be helpful to get “back to the basics” and reflect on lessons learned from earlier in your career. By doing so, you may gain new insights, identify areas for improvement, and enhance your overall approach to software development, as I did for code deployments.
-
Andrew Locklair
