Engineering Improvement Runbook | Engineering Automations
Carl Bergenhem
April 9th, 2024
Sleuth's Custom Automation Framework Deep-Dive Series
Tired of having to open up Jira or Linear just to move an issue from one column to the next every time you merge a pull request? Are you struggling to make sure that your team adheres to certain formatting when creating pull requests? These are all painful situations that we are sure many of you have experienced in the past, or are actively experiencing today.
With Sleuth’s Custom Automation Framework the pain associated with these scenarios are a thing of the past! This is thanks to the framework's ability to completely automate manual work, set up guardrails and best practices for your team, and send notifications along the way.
What is Sleuth’s Custom Automation Framework?
Sleuth Automations are a feature of Sleuth meant to help teams and organization drive improvements across their software delivery workflow. This includes automating manual work and reducing toil of individual contributors, setting up guardrails and best practices for teams and projects, and keeping teams and their members continuously informed on the things that matter as software gets pushed to various environments.
The power of Sleuth’s Custom Automations Framework comes from the fact that it is service agnostic, so the same Sleuth Automation can work with GitHub, Bitbucket, Gitlab, and Azure DevOps. Sleuth also connects to, and can automate interactions with, various other tools including issue trackers like Jira or Linear, observability and incident management tools like DataDog and PagerDuty, and much more.
Sleuth Automations are all built upon Sleuth’s Custom Automation Framework, which is a rich YAML-based rules engine. There is a long list of out-of-the-box Sleuth Automations found within the Sleuth Automations Marketplace, but in some scenarios there may be a need for a tailor-built Automation.
In the cases where something more bespoke needs to be created, developers can leverage the YAML-based rules framework to built a completely custom Sleuth Automation.
How Custom Sleuth Automations Work
As mentioned above, any Sleuth Automation is built on a YAML-based rules engine. Within each Sleuth Automation is a rule, or set or rules, that reads information from a certain source (a pull request or an issue for example) and does a certain action (e.g. “Send a message to a slack channel” or “add a comment to a pull request”).
Here’s an example of YAML used to comment on an issue whenever a pull request is deployed to a particular environment:
rules:- tool_add_comment_on_issue:actions:- notify_mentioned_issuesconditions:- environment='[[ Environment ]]'triggers: - deployed
In the above example, Sleuth waits for a particular trigger (which in this case is when a deploy happens), ensures that a condition has been met (the target environment is a previously configured environment like production), and a certain action is done (add a comment on any related issue(s)).
Two Ways to Use Custom Automations
Path 1: Sleuth Custom Automations from Marketplace
The Sleuth Marketplace lists a specific category for “Custom YAML Automations”, which provides a simple one-button install process for adding any custom automation. There’s no need for additional configuration in any other tools and there is no source code contributions needed.
A huge benefit of using the Marketplace Custom Automation is that any project or team in your Sleuth instance is able to install the same custom automation, making sharing the gains your team has made by implementing their tailored automation is just as easy as installing any out-of-the-box Sleuth Automation (a single button click). There’s no need to commit changes to your source code repository or copy fragile code from one project to the next.
Path 2: Adding a rules.yml file to your repositories
While most of what has been discussed so far uses the Sleuth application as the way to install and manage Sleuth Automations, there is also the option to save these rules in a single YAML file inside of any existing repository.
This can be done by adding all Sleuth Automations in a YAML file in a Sleuth-specific folder, .sleuth/rules.yml. Rather than install separate automations, all of the desired automations will live in this single file. When Sleuth detects this file it will keep all applicable automations in mind for the specific repository only.
Sharing any Custom Sleuth Automation will have to be done by copying the appropriate YAML snippets to a .sleuth.rules.yml file in another repository.
Building Blocks of Sleuth’s Custom Automation Framework
Any Custom Sleuth Automation is built in a similar way, using a mix of trigger, condition, filter, and action building blocks, all added together to automate various things throughout your software delivery workflow.
Each of these building blocks have a specific purpose when creating a Custom Automation:
trigger: The event that kicks off the rule of the Sleuth Automation. This an include a pull request getting merged, a deploy landing in an environment, or even a cron job that happens on regular intervals.condition: An expression that has to be true in order to execute the desiredaction.conditionscan take advantage of joined expressions by usingorandand.filter: An expression that helps filter a set of items like a set of pull requests or work in progress items (a specific thing in Sleuth).filtersare optional.action: The “thing” that the specific rule is supposed to do like “add a comment to a pull request”, “notify a slack channel/user”, “close a pull request”, and much much more. The heart of any Sleuth Automation.
There is a lot to unpack in each one of these. Luckily, below you can find a link to a resources where we dive deeper into each of these building blocks. Walking through each part of this series will enable anyone to become a Custom Automations master.
