Relief. The first thing that comes to a developer’s mind when they hear the word “software release”, regardless of whether it’s manual or continuous deployment. That code they’ve been working on for ages is finally live. All bugs cleared. Releasing software means the job is done. And it’s well done.
Everyone’s happy to present their new product. But only those who have really worked on it know the huge amounts of effort it took to get there. Especially teams who still do manual deployment instead of continuous deployment.
Manual deployment is like riding on a rollercoaster. You never know what’s going to happen next. More importantly, you don’t know WHEN it’s going to happen.
Crisis hits when you’re least prepared
Something like this happened to Knight Capital in 2012. Knight Capital is a company that specializes in executing trades for retail brokers.
What seemed like a regular Wednesday morning turned into a real nightmare. Namely, the day started with a test run of the new feature of the company’s trading software.
At around 8 AM, the employees received 97 emails with the information that Power Peg, the defunct system they were using at the time, wasn’t set up properly. But, they didn’t take this warning very seriously.
The software was set to work only with a few stocks, with the buy/sell points well outside where the markets were trading at the moment to be certain that nothing would actually execute.
However, the unexpected happened — the New York Stock Exchange was opened for trading at 9 AM, after what the software started executing trades very quickly.
After 45 minutes of operating, Knight Capital’s software had executed 4 million trades, costing the company an enormous loss of $460 million. The company also got an additional fine of $12 million by the Securities Exchange Commission because of the violations of the financial risk management regulations.
What happened?
The developers from Knight Capital had eight servers to deploy to. However, one of the technicians didn’t copy the new code to one of them, so this server deployed an outdated version of the software. This human mistake led the company to bankruptcy in just 45 minutes.
Another reason for this gigantic disaster was that the flag they had previously used to set out Power Peg was repurposed in a new feature. This led the program to believe it was in a test environment and to execute trades as quickly as possible, regardless of other circumstances.
The Security Exchange Commission report stated that the company had no formal code review or QA process and that its processes weren’t set to check if the software was deployed accurately.
Why continuous deployment?
One thing that could have saved Knight Capital is continuous deployment.
Continuous deployment is a software releasing strategy where every code commit that passes the automated testing successfully is automatically released into production, making changes that the end-users can see and interact with.
What we can learn from the Knight Capital case study is that developing a great application isn’t enough. There are a lot of processes that follow it. It’s equally important to deliver it correctly. The engineers who didn’t copy the code to one of the servers aren’t to blame — it’s the entire process that failed. This process of doing things manually was going to fail, sooner or later. It just failed in the worst possible moment.
Every deployment process that relies on humans performing most of the tasks also brings great risk. Humans make errors naturally, and there’s no one to blame here.
In order to prevent this, deployments have to be automated and released from the risk of human errors. If Knight Capital’s team had had an automated deployment process that includes configuration, deployment, and test automation, this disaster wouldn’t have happened.
Companies that don’t want to repeat this mistake have to understand that it’s crucial to have a repeatable and secure software releasing process that minimizes human interventions.
Continuous deployment is the savior when it comes to releasing unchecked code in live software. It’s a sophisticated way to detect issues in new releases in real-time.
Continuous delivery vs. continuous deployment
People often mix continuous delivery with continuous deployment. However, they aren’t the same thing.
Continuous delivery is a software engineering concept where software is delivered in short sequences, making sure that software can be released at any time. The goals of continuous delivery are to build, test and release software faster and more often.
Continuous integration, delivery, and deployment make up the term continuous software development. This concept is based on Agile and DevOps principles.
Continuous integration is basically a combination of continuous delivery and continuous deployment — a strategy where the team develops, builds, and tests new code quickly through an automated process when only the good code gets released. Continuous delivery is what happens when a new code goes to QA for testing. It occurs in a production-like staging environment and changes are being manually released into production when the code is reviewed.
On the other hand, continuous deployment doesn’t happen in a production-like staging environment. In this case, the code isn’t manually reviewed — automated testing is integrated and goes through the entire release process. There’s no waiting for the code to pass the testing phase before it moves into live production.
The common thing for the two concepts is that they both operate in a real-time environment in order to detect code issues that weren’t noticed before deployment. Continuous deployment is the one that relies more on testing as it eliminates human intervention. Development teams can choose to use only one of the two approaches or combine them, depending on the type of the application they are creating and the users using it.
What are the benefits of continuous deployment?
Reduced risks. Automating the CI/CD pipeline means that the chances for errors to occur are significantly smaller. This frees up developers from a lot of time spent on reviewing code and fixing bugs, allowing them to focus on real product development.
No downtime when updating. Continuous deployment makes releasing updates much easier. When making changes in the application, the servers still have the software’s latest full version and are running it. Once updates are completed, the application is automatically updated with the changes visible for the users.
Users get features faster. Since updates are released automatically, they get to the users faster without wasting any time. This means that they can give feedback immediately, allowing developers to improve in real-time. The sooner they deploy, the sooner they can adapt to users’ feedback.
Continuous deployment completely changes development. Opening for continuous deployment means opening up to more chances for improvement.
Developers become more productive, creating a more stable product. At the same time, they are happier with their jobs as they won’t survive burnout every time a feature gets released.