Skip to main content
Shpend Kelmendi
Software Engineer & Architect

Shpend Kelmendi is a Software Engineer and Cloud Architect at Zühlke Engineering AG. His playground: Azure, DevOps, and cloud architecture. As a Microsoft Certified Trainer (MCT), he shares what he learns and helps others succeed in the cloud.

View all authors

Controlling Databricks Costs on Azure

· 7 min read
Shpend Kelmendi
Software Engineer & Architect

No one likes surprises on the bill. Whether it is at a restaurant with your date or in a company setting, an unexpectedly high invoice can quickly put us in unnecessary trouble. Explaining to leadership why costs suddenly went up can be okay when it was a conscious decision, but not when something was simply overlooked. That is exactly one of the biggest challenges for our Data Platform Engineering team: cloud and Databricks offer huge opportunities, but costs can rise sharply if you do not pay close attention. We often lack the clarity and early steering mechanisms needed to identify the main cost drivers and react in time. Our goal is to enable data teams to unlock the full potential of the platform without unpleasant end-of-month surprises from Finance. For us, success means no unexpected costs, cost awareness across all users, and clear, easy-to-manage processes for cost visibility and control. We evaluated multiple options and ultimately chose a solution that gives us this freedom without giving up control.

Merged PR 42: fix(Azure DevOps): Merge PR without prefix in commit message

· 3 min read
Shpend Kelmendi
Software Engineer & Architect

I use Azure DevOps across many customer projects. I really like it. One thing I didn't like is the message added to the beginning of the commit when you closed and merged your PR. You can see it above in the image or in my title. Not so nice to read, right? I agree.

Why is this important?

I like to have a clean commit history.
I like simple git graphs.
And I want to automate chores like deciding whether a new version is a patch, minor, or major increment.
That's why I like to use conventional commits which, enable semantic versioning (SemVer).

Conventional commit is a simple and lightweight convention for your commit messages. Here are some examples of conventional commit types and their impact on versioning:

TypeExampleMeaningVersion Impact
featfeat: login as guest with one-time passcodeNew featureMinor1.0.0 → 1.1.0
fixfix: correct device registration validation logicBug fixPatch1.0.0 → 1.0.1
chorechore: update minor versionMaintenance / small changesPatch or ignored (config dependent)
perfperf: import up to 10 projectsPerformance improvementPatch or ignored (config dependent)
refactor!refactor!: remove deprecated import endpointBreaking change indicated with !Major1.0.0 → 2.0.0

If you need support writing conventional commits, try out commitizen.

The "Merged PR NUMBER" prefix added by Azure DevOps after merging a PR is not helpful. So how can we fix this? Let's check our options.

Options for conventional commits

I know three options when you want to use conventional commits with Azure DevOps.

Manually

You can change the commit message by:

  1. Click on the complete button in your PR.
  2. Click "Customize merge commit message".
  3. Replace the commit message.
Manually change the commit message when merging a PR in Azure DevOps
Manually change the commit message when merging a PR in Azure DevOps

But this is not a reliable way in a team. That's why this solution didn't work for our team, and we would not recommend it.

Workaround: Custom Script

That's why we searched for alternatives. If you just want the commit history to be created correctly, John Reilly describes an alternative approach in his blog post, using the Azure DevOps API, Node.js, and TypeScript. He wrote a tool that uses the autocomplete feature to override the commit message before merging the PR.
So basically, what you have seen in the previous chapter by doing it manually.

Disable prefix in Azure DevOps Settings (New)

This problem was seen and reported in October 2018:

Feature request to disable or change the default commit message when merging a PR in Azure DevOps, created in 2018
Feature request to disable or change the default commit message when merging a PR in Azure DevOps, created in 2018

After endless complaints and a long waiting time, this feature has been implemented and released (5. March). You can disable the prefix message by:

  • Go to Project Settings
  • Select Repos → Repositories
  • Select a Repository
  • Disable "Include PR ID in the completion commit message title by default."

If you don't see this, you need to wait a few more weeks until this feature is available for you, too.

Conclusion

Based on the feedback it's not only me, struggling with this design decision. So thanks to the team who took the time to realise this. The feature is limited by disabling it per project or repository. You can not set this globally for all repositories in an organization.

Further Reading

The Kubernetes Matrix: Red Pill or Blue Pill?

· 7 min read
Shpend Kelmendi
Software Engineer & Architect
Joachim Jabs
Senior DevOps Engineer & Cloud Architect

The Red Pill and Blue Pill

If you haven't seen The Matrix, watch it. It's worth it.

The Matrix is a sci-fi action movie about a hacker named Neo who learns that reality is a simulation run by machines. With Morpheus and Trinity, he struggles to free humans from this illusion. However, he must first make a crucial decision:

Blue pill: comfortable illusion. Red pill: uncomfortable reality.

So what does this have to do with Kubernetes? Picking Kubernetes is the same kind of choice.

Take the Blue Pill if you thinkTake the Red Pill if you think
Kubernetes solves all my problems.Kubernetes is very powerful but adds complexity, which is not necessary for current requirements. Let's start simple.
It's the future - all big companies use it.Big companies have big problems and big teams. Your team with 5 developers has different needs than Google.
No vendor lock-in - we stay flexible.You trade cloud vendor lock-in for Kubernetes ecosystem lock-in. Plus managed k8s still ties you to a provider.
We need it to stay competitive.Competitive advantage comes from solving customer problems, not from adopting the latest technology hype.
It scales infinitely.It can scale, but you need the team, monitoring, and infrastructure to support it. And this is not cheap.
DevOps will be so much easier.DevOps becomes more complex. You need specialized knowledge and significantly more operational overhead.

Which one do you pick?