Featured image of post DevOps for Fun and Profit

DevOps for Fun and Profit

Pipelining all the things and streamlining my life

I’ve been thinking about making this post for a while. Well, since the start of January, at least.

Last year I studied for and passed my Cisco DevNet Associate certification after about 6 months of study. At the time, the main goal was to renew my CCNA which would have expired in June 2023 otherwise. I’d been working with Ansible and Git workflows at my job at the time which piqued my interest in DevOps.

During my studies, I also started a Python coding project called Clockbridge1 around this time which I used to practice certain concepts like CI/CD pipelines and building Docker images. That entire process was… an adventure because I opted to use Docker-in-Docker as my GitLab Runner executor for this project, which obfuscated a lot of issues I ran into in the build process. Since then, I’ve learned a lot about Docker in my day job through building and deploying a custom Flask application. I’ve also started using pipelines to streamline publishing to my blog, which brings me to the main topic of my post.

In January, I realised I’d been neglecting my blog for some time. One of the main reasons for this was because it was too much of a pain in the butt to update. For example, I have a text file on my server with the below commands in it which were required to build and upload the site:

1
2
3
4
5
6
7
8
# Test the website
$ hugo server --bind=0.0.0.0

# Build the website
$ hugo --config /opt/hugo/config.yaml --destination /var/www/html/blog -v -b https://blog.meta.id.au/

# Upload the built website to my server
$ scp -r ../blog/* meta@blog.meta.id.au:~

Whilst it doesn’t look like much, this was far from ideal. All the extra flags and things made it super annoying to build and remember what I was doing, especially as it would usually be months between posts. So, I decided that it was time to put my blog into our home GitLab instance and build a CI/CD pipeline for it to do all this for me. I don’t pretend any of this is good practice or even done well but it currently works and gives me something to build on as I develop my skills. That said, I’d absolutely welcome suggestions in the comments below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# Pull in theme submodule
variables:
  GIT_SUBMODULE_STRATEGY: recursive
  GIT_SUBMODULE_FORCE_HTTPS: "true"

stages:
  - build
  - test
  - deploy

build-job:
  stage: build
  script:
    - echo "Creating temp directory for build"
    - mkdir temp
    - echo "Building with Hugo"
    - hugo --destination temp/ -v -b https://blog.meta.id.au/
    - echo "Uploading build to radium to test build OK"
    - rsync -av --rsh=ssh temp/* meta@blog.meta.id.au:~/test

test-job:
  stage: test
  script:
    - curl https://test.meta.id.au --fail-with-body # Test if page loads/has content

deploy-job:
  stage: deploy
  environment: production
  script:
    - echo "Deploying to production..."
    - ssh meta@blog.meta.id.au cp -r ~/test/* ~/
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Only run if we're on main branch
      when: manual # Wait for user confirmation to run stage

This allows me to deploy to a test website before it appears on the main website so that if something has really gone awry, I can check it, though I do usually check locally on my build box before I push to GitLab as well. Additionally, it will only prompt me to deploy to production when committing to the main branch, which was really handy when I was testing the galleries function (as seen here.

One of the big things I’m not a fan of, and probably need to find a better method of doing, is the upload jobs. Currently the box where I deploy my Clockbridge builds and my GitLab Runner are one and the same - I don’t love this from a separation of concerns aspect and I’m currently working on building new servers for these purposes as well as deploying my own Kubernetes cluster as part of my study for the Certified Kubernetes Administrator exam that I am going to try and complete this year. This bit of work is also bundled in with a bunch of Ansible playbooks that I want to use to automate my homelab infrastructure, which is progressing slowly but surely.

This journey has been a lot of fun so far and I’ve lost many hours down the rabbit hole and discovering new tools and processes that I can take with me through my career. I look forward to posting in the future about my homelab which will hopefully be eventually built entirely with Terraform, Ansible, and Kubernetes.


  1. The main branch is, as of February 2024, a mess of half-finished code, the real meat is within the branches. ↩︎

comments powered by Disqus
Built with Hugo
Theme Stack designed by Jimmy