Tips for Choosing the Right CI/CD Tools

Build-Time Security

Securing Your CI/CD Pipeline

Download Now

Building an effective CI/CD pipeline can be a complex process with countless decisions that require a great deal of planning. Whether it’s a massive DevOps team or a single developer working alone, the more you can draw on practical, real-world knowledge in making decisions about CI/CD tools the better off you are. While highly experienced developers can pass along tips to less experienced team members, the constantly changing nature of DevOps means that even the most experienced developer can benefit. 

Like all workflows, CI/CD workflows are susceptible to security concerns, so it’s a best practice to integrate security into your DevOps world (something commonly known as DevSecOps). By pairing leading continuous integration tools with a cloud security and compliance solution like the Threat Stack Cloud Security Platform®, you can build security directly into the entire software development lifecycle. With security across the CI/CD pipeline, you can ensure that your team is developing more reliable and secure applications, without compromising your team’s efficiency.

In this post, we offer 50 tips offered up by a variety of industry experts as a good place for software engineers to start building a knowledge base. To make things easier, we’ve divided the list into the following categories, beginning with a few general tips that are useful no matter the team or project:

  1. General Tips for Choosing the Right CI/CD Tools
  2. Process and Features
  3. Tool Integration
  4. CI/CD Tool Research and Vetting
  5. Managed and Self-Hosted CI Services and Code Hosting Platforms
  6. Continuous Integration Testing Tools
  7. Open Source and Free-Tiered Tools
  8. Version Control
  9. Cloud Hosted CI vs. On-Premises
Disclaimer: Our intent in this post is to provide information that could help you add to your knowledge of CI/CD tools. None of the information is intended as an endorsement or criticism of any product or service. 

1. General Tips for Choosing the Right CI/CD Tools

There is no such thing as one size fits all with DevOps tools, so choose multiple CI tools to meet varied needs rather than a single solution.


“An organization could have multiple projects of different kinds, ranging from a monolithic big software system to a micro service-based architecture, and one CI tool may not be sufficient for all the scenarios. So, it is better to choose multiple CI tools for various needs instead of fitting all into one CI tool.”

— Sirish Kumar, Choosing the Right Continuous Integration Tool, DZone; Twitter: @DZone


It matters what other developers are using when you share commonalities in goals and ways of working, so tool popularity and DevOps reviews can help guide CI tool selection.


“So, if you can, look for a gauge of its popularity. This is harder for some products than others. But some good ways to do so are looking for the number of downloads on GitHub, activity on Twitter and social media, activity in product forums, and attention in more traditional media.”

— Mathew, How To Choose Your CI Tool, Sqreen; Twitter: @SqreenIO


Saving time and reducing errors must constantly be re-balanced, so consider the time investment needed for monitoring stack customization.


“Having the ability to customize a monitoring stack exactly to your requirements is an enormous advantage, but it comes at a price, and that price is the time required to make these customizations.”

— Brian Christner, The right tool for the job: Picking the right monitoring software, The New Stack; Twitter: @idomyowntricks


Chosen CI/CD tools must deliver real-time reporting and project management automation tool integration to be effective across different and multiple projects.


“The right test management tool for DevOps is one that enables agile teams to work collaboratively in the different areas described above — automated build, automated testing, and automated provisioning of infrastructure for deployment — in order to speed up the release of high-quality software. Doing that effectively means it should be able to integrate with other project management, issue tracking, and automation tools in your DevOps toolchain.”

— Sanjay Zalavadia, How to choose the best test management tool for DevOps, Zephyr; Twitter: @yourzephyr


If microservice architectures are your build goals as they are for many DevOps teams, CI/CD tool selection should be heavily weighted to their compatibility with those tools and processes.


“An increasing number of teams are opting to build applications as a collection of microservices, small and relatively simple features that can be combined into possibly several different applications.”

— Peter Varhol, Continuous delivery orchestration: How to choose the best suite for your pipeline, TechBeacon; Twitter: @pvarhol


Application hosting and data storage policies are major factors shaping project workflows, so they should be a primary driver of tool selection.

“The critical question to ask yourself before choosing or exploring a CI/CD tool solution is, what is your company policy on hosting your application and data storage? And is it required to be behind a firewall in your own private infrastructure?”

— Greg Sypolt, Choosing a CI-CD Tool, Saucelabs; Twitter: @saucelabs


Serverless technologies can streamline the CI/CD pipeline when microservice architectures are the driving force of the SDLC.


“One of the biggest benefits of using the serverless technologies is almost zero operational cost to use it. From the concept standpoint, it really can be compared to Gitops … where you can simply trigger some action, something happens behind the scenes, and you simply receive the desired result, or undesired result, probably, but you can work with that.

“Again, serverless technologies allow your organization, your team, and yourself to have almost zero extra operational charges. If you are a developer, you can simply focus on developing the code and pushing one magic button and everything will happen … and CI/CD systems that will be focused on serverless and are focused on serverless will globally satisfy this request, this demand.”

— Libby Clark, The Best CI/CD Tool For Kubernetes Doesn’t Exist, The New Stack; Twitter: @thenewstack


Tool support can be a defining criteria of CI tool choice if team members have limited practical experience with the tool that is chosen.


“Depending on which solution you choose, support may or may not be included. If something goes wrong during the process of setup, integration, or operation, it is important to know where you can turn for help. The duration between a query and a response from support could cause extensive delays in your project. It is also important that your solution has accessible documentation so that you can find answers to your specific issues, when necessary.”

— Johanna Luetgebrune, The Expert Guide To Continuous Integration, DeployBot Blog; Twitter: @DeployBotHQ


It takes careful planning to choose the right CI/CD tool, but the upfront time investment will pay off in many ways. That includes the time, cost, and work you will avoid if you are forced to migrate to a new one.


“Avoid marrying a tool, for tools are temporary. Well ideally speaking, CI tools should be.

“Why? It might not seem much now, but you’ll be stuck for years with that old tool because migrating to a new one is a huge job. Storing long shell commands in Jenkins? You’re doing it wrong!

“Instead, try creating small .sh files or .bat files, commit to your repo, and call them when needed. Use environment variables for saving credentials. This’ll help you to migrate to new tools very easily. Not only that, but you can run these files anywhere to carry out small tests.”

— Abhimanyu, 9 things I wish I knew about CI/CD pipelines during first year of my start-up, Test collab; Twitter: @TestCollab


It’s imperative to test out potential CI tools via a small test project before making the final decision.


“It’s not easy to decide on a CI tool by just reading about its features. The best way to understand its abilities and to see how it fits into an organization is to trial it by creating a small test project and monitoring how the chosen tool affects the speed and efficiency of the team.”

— Dorothy Norris,  CI at Your Service: Comparing Travis with CircleCI, Stratoscale; Twitter: @Stratoscale

2. Process and Features

The choice of communication tools should be a balance of integration options with the preference of your team and the way that they work.

“A communication tool is useful in a CI/CD pipeline because it enables an immediate reaction on your side. For example, if the CI/CD pipeline sends a notification to a communication tool (like chat or email) when something went wrong, or when a new version of software was deployed and it is ready for testing. I recommend Rocket.Chat because it is mature, open source, and has great integrations with other tools. I know many people prefer Slack.”

— Evgeny Mekhanikov, The CI/CD Infrastructure: My Recommended Tools, BlazeMeter; Twitter: @BlazeMeter


Not every feature or plugin available with a CI tool is meant to be used for every build. Choose among the available plugins with your CI tools carefully to avoid creating more work than is necessary.


“It can be tempting to use every feature and plugin of a CI system to manage builds, but this is usually counterproductive. In general, let the CI system handle the ‘where and when’ of building, but use your own scripts within the repository for the ‘how.’ This lets developers use the same scripts locally and makes it easier to switch to a different CI system in the future should that be desirable. It also means as much of the build process as possible is versioned along with the code, which makes it easier to build older versions.”

— Emanuel Borsboom. Continuous Integration: An Overview, FP Complete; Twitter: @FPComplete


While a robust set of plugins can provide greater functionality to a CI tool, it’s important to use them judiciously so as not to negatively affect workflows.


“To be sure, many of Jenkins’ 1,500 plugins provide functionality that not everyone needs. For example, PagerDuty or Azure Storage compatibility, because many users may not have a need for these functions.

“But the fact that you need plugins in Jenkins to do just about anything is problematic — and not only because it means software delivery teams have to spend time installing and configuring them before they can start working. The bigger issue at play here is that most of Jenkins’ plugins are written by third parties, vary in quality, and may lose support without notice.

Building a software delivery chain based on third-party plugins is not a good way to ensure availability or stability.”

— Micha Hernandez van Leuffen, The Many Problems with Jenkins and Continuous Delivery, The New Stack; Twitter: @thenewstack


Every CI/CD tool weighs its list of capabilities differently in terms of importance, so your own capability priorities list should drive your selection.


“Instead of thinking about what tools you need, think first about the capabilities you want, like ‘build pipeline for iOS,’ ‘build pipeline for Android,’ ‘mobile app distribution for both platforms,’ or ‘signing certificate security and management.’ That way, you’re focusing on how to move value through the system. The choice of tools will follow from that.”

— Kevin Casey, Getting started with CI/CD: 6 pitfalls to avoid, Enterprise Project; Twitter: @kevinrcasey


Easy script maintenance within the tool is imperative for teams with coders of differing levels of experience.


“Ease of developing and maintaining the script. Make sure you choose a tool that lets you create scripts and maintain them easily. That will save you a lot of time, especially when requirements change frequently, and you have to update scripts quite often.”

How to choose the test automation tool for your needs, Qualibrate; Twitter: @Qualibrate


Pipeline issues should be easily and clearly presented within the tool.


“Results should be easily importable via API or command line: You need to be able to see the results easily in the pipeline run log so that you can easily understand pipeline issues without going through different teams and tools.”

— Mantosh Singh, The rules of selecting tools for continuous delivery toolchains, Best DevOps; Twitter: @BestDevOps


Best practices are adopted because they work for almost everyone regardless of chosen process, which is why they should inform your tools selection.


“There is no exact set of tools that is suitable for every use case. You should first define your objectives and budget, then decide about short- and long-term tools and practices you want to adopt. But some basic principles do apply to every project. Everything as Code is one of them. If you write your whole Infrastructure as Code, then it will be very easy to spin up a new environment.”

— Dmitry Krasnov, Do I need CI/CD and what tool should I choose, Containerum; Twitter: @Containerumcom


Understanding how your DevOps team likes to work and the areas where they need to do things differently for better and faster outcomes will make it easier to determine ideal tools based on their compatibility with those goals.


“Ease of setup: Making the adoption of CI/CD frictionless is perhaps key to getting everyone on board. A huge learning curve and effort for setting up a CI/CD pipeline may be hard to justify when you’re already pinched for time.

“Build status: Since adopting CI/CD is about transparency and visibility, it’s also important to make it visible and transparent your way. It might look like a nice-to-have thingy, but getting notifications in your Slack channel, via email, or in other channels, may improve the flow of information considerably.”

What Is Continuous Integration and How to Benefit from It?, Nevercode; Twitter: @nevercodeHQ


The team and budget you want doesn’t always reflect the reality, so it’s imperative to make tool choices based on a holistic, real-world view of the CI/CD Lifecycle.


“Select automated deployment tools that support the languages developers work in, and that work well with the infrastructure stacks already in place for production. If developers on a specific project already use Jenkins, for example, see whether it can support other teams before you evaluate CI tool alternatives. If, however, the DevOps budget is modest and there isn’t enough expertise or time to learn complex new tools, seek out products that are easy to set up and configure.”

— Kurt Marko, Automated deployment tools mitigate the risk of human error in DevOps, TechTarget; Twitter: @DataCenterIT


Choose tool combinations that further the Infrastructure as Code (IAC) and Configuration as Code (CAC) advantages.


“Whether it is a paid-for technology or open source, be sure of the level of support you’ll be getting. Inadequate support can lead to delays that block the entire deployment pipeline — a situation that can be avoided through paid support services or with the help of a dynamic support community.”

— Sanil Pillai, Infrastructure and Configuration as Code come to the CI/CD rescue, SD Times; Twitter: @sdtimes


What you may need tomorrow should carry some weight when making decisions about what you need today, so choose infrastructure frameworks and platforms that have the flexibility to adapt to your changing needs.


“Flexible infrastructure options: Your CI platform should support your infrastructure, processing, and security requirements. If your builds are resource intensive, you should have the ability to buy bigger nodes. If you’re a Fintech company with strict compliance requirements or expect to move all your software development behind a firewall for any reason, you should make sure your CI provider has an enterprise version.”

— Pavan Belagatti, 10 Things to Consider While Choosing a CI Platform, Shippable Blog; Twitter: @BeShippable

3. Tool Integration

Security needs to be “baked in” to every part of the DevOps environment. That’s why you should consider whether security tools can be easily and seamlessly integrated into the CI/CD pipeline with your choice of CI/CD tools.


“Using a variety of application security testing tools at all stages of the CI/CD pipeline is the only way to make sure your application is always production-ready. But it can be difficult to manage the results coming in from all these tools.

“The easiest way to do this is to use an application vulnerability manager. This type of DevOps tool is not another testing tool. Rather, it combines the results and unleashes the true power and potential of your testing tools.

“CI/CD is an important step in the Agile and DevOps approach to software development. But security cannot be sacrificed in favor of speed. Rather, security must be integrated into each step of the process. Rapid and secure releases are the product of a truly successful CI/CD process, one that produces software that is valuable to end users and protects those users and your company from vulnerabilities.”

— Ken Prole, Don’t leave security behind in your CI/CD environment, Code DX; Twitter: @CodeDx


CI tools must work within a much broader and highly integrated DevOps process, so making sure that they integrate with the chosen application lifecycle management (ALM) tool set is one of many important considerations.


“Depending on your needs, you should make sure that the ALM tool you choose can integrate with the different CI servers that your development teams are using and that the reporting within the ALM tool can fuse the information from other sources with the build information. That way you can see all the changes in each build, which features have been added, and which issues have been resolved.”

What is Application Lifecycle Management, Inflectra; Twitter: @inflectra


Varied integration support and release process visualization are key components of a CI/CD platform.


“It’s fairly common for an organization’s software development lifecycle to rely on several integrated tools (such as issue boards and other discussion features). Find out what your teams are already using and whether the CI/CD solution you’re considering is supported. There’s been growing interest in built-in CI/CD, which means developers can spend less time stringing together their tooling and more on new features and improvements. Bringing all your tools under one product with one interface and datastore is also useful for things like cycle analytics, which can help to reduce the time between coming up with an idea and deploying it.

“One of the advantages of leveraging CI/CD is being able to see changes and new additions from the moment they’re created. Does your chosen solution offer Review Apps, so you can automatically check out a live preview of new code? You might also benefit from Deploy Boards, where you can watch a deploy roll out across pods and monitor the health and deployment status of each environment, all in one place. This makes it easier to spot problems and stop or roll back with one click. These are just a couple of features that can make a significant difference to your team’s efficiency.”

— Rebecca Dodd, What to look for in a continuous integration tool, GitLab Blog; Twitter: @gitlab


Keep in mind that not every plugin from every CI tool provider will integrate with every CI server whether it’s hosted or onsite.


“Because CI/CD systems are a coordination point for different environments and many other pieces of software, it is important to consider how easily different solutions integrate with other tools or systems you are using. Each continuous integration system has a different set of projects they support natively. Some systems are built with plugin frameworks that allow users to create or use extensions to enhance the functionality or interoperability of the platform. These are all points to consider when choosing a CI/CD system.”

— Justin Ellingwood, CI/CD Comparison: Using Managed Providers vs Self-Hosting, Digital Ocean; Twitter: @digitalocean


Robust and deep support is just as important a criterion as an intuitive UI to CI tool choice.


“Usability and support: Some tools offer great features and functionality, but they are buried under an interface that is anything but friendly. This hinders the success and usability of the tool, which is why it’s important to consider this aspect to make sure the user interface is easy to use and intuitive. Additionally, it is very important that the selected tool has a robust offering of training materials and available support for any incident you might experience.”

— Svitla Team, Continuous Integration Tools, Svitla Blog; Twitter: @SvitlaSystemsIn

4. CI/CD Tool Research and Vetting

Always see your top tool choices in action and ask the company reps every question you can think of.

“Once you’ve narrowed it down to your top 3–5 choices, set aside some time for live demonstrations of each. This will help you get an initial feel for the different software solutions and give you the opportunity to ask questions and narrow your list even further.

“Be sure to ask things like: What is the vendor’s methodology? Have they implemented the solution in similar companies? Is training for your team on using the system included? These are key things to consider when choosing the right CI enterprise software solution.”

— Cipher Team, 11 Steps to Choosing a CI Enterprise Software Solution, Cipher; Twitter: @CipherSysLLC


Gather research from actual open source tool users regarding its level of dependability.


“Projects in the hardening or enterprise stage have become mature technologies. The number of commits will signal the level of investment in a project. The type of commits tell a story, telling where the author(s) are trying to go with the code, revealing what they want to do by signaling interest in different features of the project. By now the initial excitement has died down and there is more demand for stability than new features. The initial development team may be working on other projects as it has developed a solid community  —  this is often a good sign of success of a project.”

— Matt O., How to Choose the Best Open Source Software, Towards Data Science; Twitter: @TDAtaScience


Tool comparisons should be based on a combination of your team needs and general best practices.


“The tool is not as important as the way you use it. You shouldn’t pick a tool and adapt your work around its capabilities. Instead, try to make the tool work for your needs as long as you follow best practices. Know your needs and know the tradeoffs and caveats of the tools. If you do that, you can choose the tool that will work best for you.”

— Sami Alajrami, CircleCI vs Google Cloud Build: How to choose the CI/CD tool that’s right for you, Praqma Blog; Twitter: @Praqma


A chosen CI tool should have an intuitive UI so that even novice team members can quickly grasp how to use it.


“Continuous Delivery (or Integration) is a powerful concept that works best when used by even the least technical team members. If your team is new to the CI/CD idea, consider it something that drastically reduces its learning curve by making things as easy as possible thanks to smart UI — not only the few who understand CI/CD ins and outs.”

— Pavan Belagatti, Which CI tool should we use, Quora


You need to weigh all the factors when evaluating on-premises source code hosting solutions including security, availability, budget, and more to ensure it’s the right choice for your project workflow and team.


“In addition to the license fees for your code hosting software, you’ll have to keep in mind that you need people and hardware to manage your installation and servers. Don’t forget these costs when comparing solutions.

“One way to significantly cut down on costs might be to go with an open source installation . . .. However, this comes with all the challenges that such a model brings with it (considering support, security updates, etc.).”

— Tobias Günther, On-Premise Source Code Management – 7 Git Hosting Platforms Compared, Tower; Twitter: @gittower


Budget, team size, tool preferences, and individual work processes need to be discussed thoroughly before vetting or choosing any CI tool or hosting platform.


“For solo developers, making this choice (between GitLab and GitHub) is a simple one — both of these hosting platforms offer free plans, so you can try out both and see which one suits you best. If you prefer to keep your repos private, and that’s a consideration for a lot of new programmers who may be shy about sharing their work — you’ll likely lean towards GitLab because it offers unlimited free repositories.

“On the other hand, GitHub is not only a well-designed software; it has also fostered a global community of programmers, so it’s easy to reach out to other developers for feedback or insights and see their own coding projects in process. If you’re still on the fence, it’s worth trying both to see which one best suits your tastes.

“If you’re working with a team, however, it’s worth your time to speak about these options together. Different programmers have their own opinions on which code development tools they prefer and why — and if you’re working in a startup, budget will likely also play a fairly significant factor in the decision.”

— Johanna Luetgebrune, GitLab vs. GitHub: What’s the Right Hosting Platform for Your Workflow?, DeployBot Blog; Twitter: @DeployBotHQ

5. Managed and Self-Hosted CI Services and Code Hosting Platforms

Choosing between onsite and hosted CI is just as crucial a decision as what CI server and software you choose.

“Is there a vendor who offers what you need? Is there a package tailorable to your unique situation? Will any of the default offerings fully cover your software needs both now and well into the future? If they won’t, you may find yourself playing musical chairs with vendors as the complexity and sophistication of your software grows. As with moving database vendors, moving CI providers may not be an inexpensive endeavor.

“Like most things, you need time to get acquainted with a set of tools and to build a set of workflows around them. To have to change them intermittently because they can’t grow with you may end up being a rather frustrating, not to mention expensive, undertaking.

“But there are other issues that can result from hosted solutions, even hosted open source solutions. Let’s take Jenkins, for example. While it may come with a near endless array of plugins that allow for all manner of functionality above and beyond the base, does the host provide it? You may find yourself in a position where you can only use the core functionality, plus a limited set of plugins. Or you may only be able to configure the tool within a predefined set of boundaries.

“Given that, hosting your CI solution, with GitLab or Bamboo for example, may be the right solution.

“Because when you host it yourself, you have the flexibility to choose any number of plugins and make all the configurations you need. You’re unconstrained by anyone else’s preconceived ideas of what you may or may not need.”

— Matthew Setter, The Pros and Cons of Hosted Versus On-Premise CI, CodeShip; Twitter: @settermjd

6. Continuous Integration Testing Tools


The more complex the software DevOps project, the greater the need for robust testing facilities within the chosen CI tool.


“Another advantage of hosted CI solutions is that they can provide much broader testing facilities. Browser and OS testing used to be a tedious affair — workstations and staff had to be dedicated to ensuring that bugs didn’t appear within a certain environment. In contrast, a hosted solution can maintain a set of cloud-based servers with different configurations for this purpose.”

— Vassili Van Der Mersch, Reach DevOps Zen with These Continuous Integration Tools, Nordic APIs Blog; Twitter: @nordicapis


Selecting a test management tool that integrates seamlessly with your CI tool is the first step to an efficient DevOps project process.


“While searching for the best test management platforms, look for a tool that provides integration with leading test automation tools like Selenium, TestingWhiz, Watir, JUnit, etc. since this can help you sync and run tests on a regular basis and also automate the entire process of managing tests, which can significantly reduce time and effort. Also check whether they seamlessly hook with Continuous Integration (CI) tools such as Jenkins and Bamboo to enhance the build release cycles and deployment automation.”

6 Features to Look for in an Ideal Test Management Tool, TestingWhiz; Twitter:  @itestingwhiz


While it’s just one of many tool decisions in the CI/CD pipeline, it’s important to make sure CD and automated testing tool integrations are a good match.


“For Continuous Integration testing, open source tools like Selenium and Appium are most popular for automating tests. Additionally, tools like CrossBrowserTesting can also be used to execute test automation and create an environment for continuous testing in the cloud.

“Although automation is essential for keeping up with the speed of a well-oiled CI process, it won’t always be the answer to testing. For brand new features, it’s best to resort to exploratory testing.”

— Alex McPeak, Understanding Continuous Integration Testing, CrossBrowser Testing

Testing tools are crucial to the CI pipeline, but a bad choice can slow the entire DevOps process.


“Some tools record actions and then create code or create a visual front end that allows programmers to ‘drop in’ to see the code behind the visualization. These offer the best of both worlds.

“The main issue here is that the people expected to learn the tool need to be willing and able to do that and have the time to do it. If the test process is behind, assigning testers to learn a new tool will add work, slowing the software-delivery process further.

“If the regression test process takes days or weeks to run, automating it, especially from the front end, will slow down testing more, creating a buildup of work until some break-even point is reached. Even after the break-even point, where the tool is no longer slowing the testers down, the old backlog will need to be cleared out.

“Of course, if the project is new, or if the company intends to hire a new person to do the test tool work, these objections might not apply. So, do the analysis of how the tool will be added to the team, what it will disrupt, who will do the work, and whether those people have the capability and time to do the work.”

— Mathew Heusser, How to choose a functional testing tool: 7 key considerations, TechBeacon; Twitter: @TechBeaconCom

7. Open Source and Free-Tiered Tools

Consider that some paid subscription-based tools limit concurrent builds, which can inflate costs for growing development teams.

“One important thing to keep in mind when deciding to use a paid subscription-based service is how the cost scales with your usage. There are a few factors that can affect cost. Particularly, some CI/CD SaaS services limit the number of build processes that can be run concurrently. All of this means that increased throughput will likely result in increased cost.

“If you expect to maintain a steady level of throughput (that is, you don’t expect to add significantly more developers, which would require additional CI/CD throughput), then perhaps limits on the number of concurrent build processes is not a concern for you. However, if you’re planning on adding more developers to your team, you’ll likely end up having more build/test jobs that need to be executed. Limits may hamper your team’s productivity.”

— Matt Burton, How to Choose a CI/CD Tool: Cost Scaling, Languages, and Platforms, and More, ParkMyCloud Blog; Twitter: @ParkMyCloud


Weigh the pros and cons of the potential loss of continued code development with open source software tools.


“Open source software is free to use (though you may need to pay for hosting). However, you always face the risk that the code may stop being actively developed, or that the developers may suddenly completely change everything.”

— Paul C, Choosing the Right CI/CD Tool, Functionize; Twitter: @Functionize


While free tiers for open source tools are a good opportunity for vetting a tool or for small builds, it’s important to take into account the cost in terms of time expenditure for setup.


“If you are running an open source project, you are in luck since ‘free for open source’ is kind of a must-have for all the CI solutions out there. If you are running a commercial or private project, most tools have a free tier, either provided in build minutes or number of builds per month. Pick what works best for you. I recommend Fire CI. Of course, I do since I built it.

“Open source solutions that you host yourself are free if you consider that your time setting it up and operating is ‘free.'”

— Louda Peña, What’s a good 100% free CI (continuous integration) and CD (continuous deployment) server?, Quora

Several CI tools are suitable for use as the “last mile” system where existing tools may not fit with a fully automated deployment pipeline.


“Something I seldom see addressed is how to actually get the software onto your servers. It’s assumed that if you’re doing continuous deployment, you already have the ‘deployment’ part taken care of. But even if this is the case, your existing tools may not necessarily fit with a fully automated deployment pipeline — especially if you’ve historically been on a fixed-window schedule. This is known as the ‘last mile’ problem, and it’s one of the hurdles we encountered on the Business Platform team.

“In simple single server cases, it’s possible to use your Bamboo (or Jenkins, etc.) agents themselves as the last mile system. By placing an agent on the target servers and binding that to that server’s deployment environment, you can script the deployment steps as agent tasks.”

— Steve Smith, Practical continuous deployment: a guide to automated software delivery, Atlassian; Twitter: @Atlassian


Remember that open source tools are not always the best choice and require significant vetting, investigation, and small-scale testing to be sure they are the best choice for your needs.


“Besides open source projects, several modern commercial software automation products are available including CircleCI, Codeship, and Shippable. Each of these has several different advantages and disadvantages for specific workflows. To really understand which will work for you, I’d encourage trying each of them specifically within your developer workflow to see how they work in your environment (how they work with your tools, your cloud platform, your container system, etc.).”

— Izzy Azeri, What is CI/CD?, Mabl; Twitter: @mablhq


The choice between open source vs. proprietary software can have serious repercussions in terms of maintenance without skilled people on the team.


“Surely, an open source CI can be enticing. Often being low cost (if not free), with customization possibilities and their extensive plug-in ecosystem, features almost everything you might want to install. On the other hand, this open source heaven might soon turn into a nightmare in the wrong hands.

“The reality might hit your team hard when it’s time for updating and maintaining all these open sourced extensions. One can say that all you need for the job is a highly skilled maintenance pro and time to modify, create patches, or create plugins. However, is the amount of overhead and development on hold really worth the stress, and can you really count it as saving money?”

How to do Continuous Integration Right, NeverCode; Twitter: @nevercodeHQ

8. Version Control


Version Control and Custom Script execution are two of the many key features that your CI/CD pipeline solutions must have to work across different projects.


“Whilst every SaaS or self-hosted solution on the market boasts myriad different feature sets, there are a handful of core features that are essential to operating a successful pipeline.

“Version control integration is the most crucial feature your chosen solution should possess. When I say integration, I mean it should poll your repository, or use webhooks to detect changes, which should then initiate a new build and subsequently trigger the rest of your CD pipeline process.

“Custom script execution within a pipeline step is a key feature, especially if you deliver a diverse number of projects.”

— Mark Sta Ana,  Made Tech’s Guide to Continuous Delivery Tools, Made Tech; Twitter: @madetech


Distributed version control systems are key to maximizing the use of CI tools and streamlining the CI pipeline.

“Distributed Version Control is an evolution of traditional SCM that lets developers run their own repositories locally, without any connection to the central server. Developers are no longer hindered by slow connection or VPNs, and this speeds up the development process.

“The move to DVCS has revolutionized the SCM landscape. Since its inception during the last decade, developers have been introduced to an entirely new set of tools and pushed branching and merging to a new level that lets developers — for the first time — work in a more agile way despite disconnected environments.

“No longer tied to a single server, teams can structure their development work using many repositories, which pull and push changes and help automate workflow.

“The CI workflow takes advantage of DVCS, adding feature-branch oriented development and the underlying distributed nature of the version control. It is now possible to easily ‘push’ changes to an integration server, which will detect and test new branches.”

— Pablo Santos, The Future of Continuous Integration, Code Magazine; Twitter: @CODEmagazine


While these three CI tools are among the most used by developers, they all offer tradeoffs in terms of usability, functionality, support, and costs.


“Jenkins, TeamCity, and Bamboo are all excellent tools for continuous integration, automated builds, automated testing, and continuous delivery. They all offer integrations that extend their capabilities further, though the number of integrations differs between tools. All three also offer great support and documentation, but again there are differences in depth and quality.”

— Ben Putano, CI/CD Tools Throwdown: Jenkins vs. TeamCity vs. Bamboo, DZone; Twitter: @DZone

9. Cloud Hosted CI vs. On-Premises


CI/CD workflows that use an edge cloud platform will need seamless compatibility with content delivery networks to avoid slowing the business’ web and application development.


“When considering an edge cloud platform for CI/CD, it’s vital to consider details beyond dollars per gb/s. The CI/CD development cycle depends on a CDN with control and configurability in order to properly mesh with your workflow. A CDN shouldn’t be a barrier to DevOps teams, but rather should be so responsive, configurable, and automated that it naturally becomes part of the development environment.”

Why your continuous integration and delivery practices need an edge cloud platform, Fastly; Twitter: @fastly


When comparing continuous integration tools, developers must determine whether on-premises vs. cloud CI management is worth the extra work and fits the goals for the team and the business.


“On-premises can be extra work that is not in line with your core business needs. Cloud options outsource the management of the CI tool to a third-party vendor. The cloud hosting company then handles uptime, support, and scaling of the CI tool, allowing your team to focus on core business needs. This can be a huge benefit for tight budget teams or smaller companies that need aggressive focus on product market fit goals.”

— Max Rehkopf, Continuous Integration Tools, Atlassian; Twitter:  @Atlassian


DevOps teams must weigh the potential downsides of on-premises hosting of CI against the benefits, including things such as the costs and the need for increased data security and control.


“Of course, there are hosted SaaS alternatives to Jenkins, which could be beneficial if you’re willing to pay a bit extra for someone else to maintain the software. Businesses tend to choose this option when they need a UI that is superior to the one that Jenkins offers. But a major benefit of self-hosting is that you have more control over your own data security.”

— Jordi Giménez, What Is Jenkins and Why Should You Be Using It?, BugFender Blog; Twitter: @bugfenderapp


Time investment can be as big a determiner of choosing cloud-hosted CI as costs, so project deadlines and scope will play a big part in CI tool choice.


“Setting up your own development environment from scratch can take weeks, if not months. A subscription, on the other hand, would let you provision the development environment within minutes, with minimal cost of ownership.

“This scenario highlights why so many organizations are moving development activities to hosted environments. The benefits of quick provisioning combined with minimal up-front payment and maintenance costs are game changers for developers who want to get going, and fast.”

Automate Your CI/CD Workflows with Oracle Developer Cloud Services: A Developer’s Guide, Oracle; Twitter: @Oracle

Build-Time Security

Securing Your CI/CD Pipeline

Download Now