Modern DevOps tools and practices are transforming how teams build and deploy software. In this post, we’ll explore the challenges we faced with our Jenkins-based CI/CD pipeline and the decision to migrate to GitHub Actions and AWS CodeDeploy. This switch not only streamlined our workflows but also addressed long-standing pain points that had accumulated over the years.
The Struggles with Jenkins
The Legacy Build/Deployment Pipeline:
Our Jenkins setup, which had been the foundation of our CI/CD pipeline for almost a decade, worked for a long time, but over the years, it became increasingly difficult to maintain. The pipeline was initially set up for development workflows for fewer developers. It wasn’t regularly updated, and the lack of investment in optimizing it led to a variety of challenges.
- Outdated Scripts: One of the biggest pain points was the build and deployment scripts themselves, which were stored inside Jenkins. They were not maintained, weren’t in source control and were not owned by the developers who could not access or change them. While they initially worked well, they became cumbersome to maintain as the team and requirements grew. This stagnation led to inefficiencies in the system, as we were unable to leverage new capabilities offered by modern CI/CD tools. We could have potentially fixed and optimized the scripts and our processes, but we determined this time investment would better be spent overhauling the entire pipeline.
- Server Maintenance: Another major issue was the Jenkins host itself. It required constant attention: managing disk space, Java, Maven, and other dependency updates. This manual maintenance became a bottleneck, especially when disk space filled up at the worst possible times. As a result, time that could have been spent on product development was instead spent dealing with operational headaches.
- Separate Build and Deployment Servers: The pipeline also relied on two separate Jenkins servers. One server was dedicated to builds and another to deployment. Each server was hosted on a different VPN. This setup added complexity, as developers had to switch between VPNs depending on the task, and coordination between the two servers was often cumbersome. While it provided some isolation, it increased operational overhead and slowed down workflow efficiency.
The Decision to Switch to GitHub Actions and CodeDeploy
Why GitHub Actions and CodeDeploy?
While evaluating alternatives, we considered continuing with Bitbucket Pipelines, which we had used for our test suite. Given our extensive integration with AWS, we ultimately chose to transition to GitHub Actions and AWS CodeDeploy. Several key factors guided this decision:
Platform Consolidation & Workflow Improvements
- Unified Platform with Larger Ecosystem: One of the biggest draws was GitHub’s integrated suite of tools for version control, project management, and issue tracking. Unlike Bitbucket, GitHub offered a more robust ecosystem that aligned with our needs. The ability to manage everything: code, issues, pull requests, and CI/CD, all within one platform streamlined our development process. We realized that moving everything to GitHub would reduce friction and improve overall workflow.
- Enhanced Action Capabilities: GitHub Actions offered more flexibility and capability compared to Bitbucket Pipelines. The native support for more flexible workflows and the rich marketplace of pre-built actions allowed us to tailor our CI/CD pipelines to our specific needs (enhanced parallelism, for example) more easily. We could leverage a wider variety of community-supported actions, making the whole process more efficient and scalable.
- Centralized CI/CD Pipeline: GitHub Actions brought both our version control and CI/CD pipeline into one cohesive workflow. This integration helped improve visibility and reduce overhead when managing deployments. This reduced bottlenecks and sped up our deployment process compared to the previous setup, where we relied on multiple separate systems.
Operational & Infrastructure Simplification
- Managed Infrastructure: GitHub Actions, being fully managed, eliminated the need for us to handle server maintenance. We no longer had to worry about managing the build infrastructure, dependencies, or runtime environments ourselves. This allowed us to focus more on our core development tasks rather than dealing with operational concerns.
- Seamless Authentication with OIDC: GitHub Actions uses OpenID Connect (OIDC) for secure authentication with AWS. GitHub can communicate directly with our AWS account without needing long-lived AWS credentials. This enhanced security and simplified the setup, as we didn’t have to worry about managing and rotating AWS credentials manually.
Deployment Strength & DevOps Scalability
- Unlocking Advanced Features with CodeDeploy: AWS CodeDeploy provided powerful deployment features that were previously out of reach for us. Features like blue/green deployments, which ensure zero-downtime releases and safer rollbacks, we knew would be a huge benefit. Additionally, with AWS autoscaling, we could scale our application seamlessly in response to changes in traffic, without manually managing the infrastructure. CodeDeploy also provided easy integration with Terraform, allowing us to manage our deployment infrastructure as code, further streamlining our DevOps workflows.
The Migration Process
Planning the Transition:
Migrating from a long-established Jenkins pipeline to GitHub Actions wasn’t without its challenges. We knew that we needed a well-thought-out plan to ensure a smooth transition. One of the key tasks was ensuring that our codebase, build scripts, and deployment strategies were properly prepared and compatible with the new tools.
- Updating Build Scripts: A major part of the migration was refactoring our existing build and deployment scripts to be compatible with GitHub Actions. This required significant changes to our deployment strategies, as we had to align them with the AWS CodeDeploy model. One important update was integrating deployment and tests into the same pipeline. In our previous setup, tests ran in a separate CI pipeline, but in our transition to GitHub Actions, we updated the scripts so that tests must pass before deployment.
- Addressing Dependency Management: Another critical aspect was managing dependencies like Java, Maven, and other necessary tools. In our Jenkins setup, we had manually managed environments and handled dependency updates ourselves. In contrast, GitHub Actions gave us a more streamlined approach to managing these dependencies in a clean, repeatable way using Docker containers or predefined actions. This greatly reduced the chances of discrepancies between builds and allowed us to maintain a more reliable pipeline.
Enhancing Observability
CodeDeploy deployment hooks allow you to execute scripts at different deployment life cycle events. These hooks provided us with better visibility into the deployment process. They allow us to track each stage more accurately and introduce additional validation. Additionally, we wanted to make it easier for developers to interact with the deployment process, so we created GitHub workflows that allowed them to trigger deployments to any environment directly from GitHub. These workflows give developers more control when needed and simplified the process for typical deployments.
Deployment Control
- Observability and Control with CodeDeploy: Once a deployment was triggered, developers could track the deployment status directly in AWS CodeDeploy, where they could monitor progress, perform manual steps if needed, and leverage advanced deployment features like rolling back a release or stopping a deployment if something went wrong. This increased the overall reliability and safety of our deployments.
- Deployment Approval and Change Management: Another adjustment made was introducing deployment approval through GitHub. After the build and test actions process successfully, a deployment approval step is triggered. Another developer has to sign off before production deployments can proceed. We have more work to do before we enter full continuous deployment, so this approval step adds an extra layer of safety and ensures that proper change management and auditing are in place.
- Onboarding the Team and Creating a Deployment Playbook:
To ensure the whole team could effectively work with the new tools, we created a comprehensive deployment playbook. This playbook detailed the processes for using GitHub Actions, and CodeDeploy, and provided step-by-step guides on how to interact with the pipeline, trigger deployments, and manage environments. We also set up training sessions to bring the team up to speed on the new technologies, ensuring everyone was comfortable with the transition. The playbook also contained details on when and how to rollback or fix forward. This made the adoption process much smoother and helped the team feel confident in using the new tools.
Testing the New System:
Before fully transitioning to the new pipeline, we ran extensive tests to ensure that everything worked as expected. This involved running integration tests and validating that the GitHub Actions workflow produced the desired outcomes. For a time, we kept the old pipeline active while we gained confidence in the new one. We verified that all workflows from the old pipeline could be executed in the new one, including rollback, fix-forward, and deploying specific revisions. We also ensured that pipeline performance was maintained – in fact, it improved. Build times were reduced by several minutes on the default GitHub runners and deployments no longer requiring a VPN connection.
The Results
Since making the switch, we’ve seen several key improvements in our workflow:
Deployment Speed & Reliability
- Faster and More Reliable Deployments: The integration between GitHub Actions and CodeDeploy has streamlined our deployment process. The automated pipelines run faster, with fewer manual interventions required. Additionally, AWS CodeDeploy’s unlocked advanced features, like blue/green deployments and autoscaling.
- Increased Observability and Reliability: The addition of deployment hooks in CodeDeploy has provided us with better visibility into the deployment process. We can now track the status of deployments more accurately through CodeDeploy’s lifecycle, ensuring that each step is completed successfully. This increased observability has improved our ability to troubleshoot and resolve issues quickly, leading to more reliable deployments.
Developer Experience & Workflow
- Improved Developer Productivity: With the shift to GitHub, developers can now focus more on writing code rather than managing multiple systems. The CI/CD pipeline is transparent, easy to understand, and fully integrated with the development environment. Developers can trigger deployments, observe progress, and intervene if needed more swiftly. This integration has simplified our workflow, making it more intuitive and less prone to human error.
- No More VPN Switching: One of the most significant improvements was eliminating the need to jump between different VPNs. Everything now runs smoothly from GitHub, and our deployment process no longer has those painful bottlenecks. We no longer have to switch between separate Jenkins servers or VPNs, and this has cut down on manual overhead and frustration during critical moments, such as rollbacks or urgent fixes.
- Clear Ownership: Housing the GitHub Action deployment scripts directly in the application repo gives developers full visibility and control over how deployments work. This creates clear ownership, encourages ongoing improvements, and keeps the deployment process closely aligned with the code it supports.
Operational Efficiency
- No More Managing Jenkins Servers: A huge benefit of switching to GitHub Actions and CodeDeploy is that we no longer need to manage additional servers running Jenkins. Previously, maintaining the Jenkins servers (ensuring they had enough resources, managing updates, and handling failures) was a constant operational overhead. Now, with GitHub Actions and CodeDeploy, we’re able to leverage managed services that automatically scale and require minimal maintenance, freeing up valuable engineering time. Our Jenkins host also ran builds and deployments more slowly than GitHub’s CI environment.
The shift from Jenkins to GitHub Actions and AWS CodeDeploy has been a transformative journey for our development and operations teams. While the transition took time and effort, the benefits in terms of reduced maintenance overhead, improved deployment speed, and a more cohesive development workflow have been well worth it. If you’re dealing with similar challenges, we highly recommend considering GitHub Actions and CodeDeploy as part of your DevOps toolkit.
