At the end of the year it’s now become somewhat of a tradition that I partake in the whole year in summary thing. I do find it interesting to read these posts, and I like doing my own since it’s a good way to reflect.
After a year of how-to type blog posts it feels odd, yet fun to be back talking in first person again. Today we’ll cover quite a lot of (varied) ground in a way I wouldn’t typically be comfortable with. But I’m hoping the informality works out. The following are some of the key things I learned in 2019 that I’ll go into more detail on very soon:
- Serverless technologies require a lot of knowledge, skill and patience
- Terraform is an amazing technology (and why)
- Single event logging is an amazing monitoring method (and why)
- Motivation for blogging is difficult (and how I regained mine)
- Why I changed the way I run my newsletter (yet it’s still hard)
It seems it’s going to be a pretty detailed post so let’s get to it…
2019: Success or Failure?
Before I get into the real details about 2019, I need to look back at the goal I set myself for 2019. Broadly speaking I like to have one goal that if completed would make my year a huge success. So in order to assess 2019 I need to dig that goal back out.
The goal for 2019 was to take my SaaS product Splitoo and make a huge investment in the technical aspects. I wanted better monitoring, better infrastructure, better continuous integration, better test coverage and just better tech in general. Overall I did get some big wins and made some big realisations, but let’s take a look…
From Heroku To Serverless API’s
Splitoo is hosted within Heroku as a single node application. Which means the front-end assets and the back-end API’s are served from the same host. The setup is simple — and it works.
But there are a few downsides to the current setup. The builds became exponentially slower, testing the app has gotten drastically harder, and over time the code has simply become more complex.
As I was working on Serverless a fair amount with my day job, I wanted to try and move the application into Serverless. My hope was that Serverless paradigm would simplify the application and make the app more cost effective to run and scale. But I soon learned the hard truth: Serverless might be simple, but it’s not easy. And I had to solve a few different problems:
- How to setup CI tooling for Serverless
- How to write serverless Infrastructure As Code
- How to effectively monitor Serverless
So I set to work laying the foundations for the migration.
Serverless CI Tooling (Github Actions)
There’s a lot of pre-requisites to running Lambda on AWS and one of them is that you’ll need a good CI tool. I liked the idea of container-based CI and I initially set about setting up my own cluster on EC2.
But after several expensive months we did shut the cluster down in favour of Github Actions. Github Actions has a generous free-tier which drastically reduced the CI costs.
Serverless Infrastructure As Code (Terraform)
Terraform Logo
Another key part of making the move was that I also needed to learn some element of Infrastructure as code. I’d worked a lot with CloudFormation in the past, and it did the job but it wasn’t particularly fun or exciting to work with.
Earlier on in the year I started working with Terraform a lot and very quickly fell in love with it. In fact I soon started kicking myself for not learning it earlier, and it’s what lead to writing the post: 5 Important Reasons To Learn Terraform Before Cloud Computing.
Not only did I need to learn Terraform, I also needed to learn how to migrate existing infrastructure, which took some time to understand. Which also lead to another post: 3 Steps To Migrate Manually Created Infrastructure To Terraform about how migrate existing infrastructure.
Monitoring Serverless
AWS Lambda Monitoring
With CI and Infrastructure As Code out of the way, I also realised that monitoring Serverless was not going to be simple either. Being quite unsatisfied with vanilla logging I stumbled the idea of one-per-service events and started experimenting with them.
So far the experiment with one-per-service logging has been a huge success. It even lead to me creating a small library to help with the logging tooling.
Routing To Serverless
And lastly the final hurdle in moving to Serverless was to setup the infrastructure to initiate it. And in my case, that something was a web API. I did initially try to a setup using AWS API gateway but found it very fiddly.
After some reading online I found out that Lambda also supported ALB’s now, so I tried to set that up. The ALB setup to me was much simpler than the others, and I got it working for my API’s quite quickly.
2019: A Success?
As you can see there was — a lot — to learn in order to make the goal that I had set. In fact, the sheer amount of learning was in fact the biggest problem. But, despite that I can still ask the question: Was 2019 a success? And the answer is… yes and no.
Whilst I gained a huge amount of knowledge in the areas I needed, I did drastically underestimate the amount of learning time it was going to take. So in the end I did do a lot less migrating of the infrastructure than I wanted. But hey, I think it was all worth it for the learning.
With the 2019 goal covered, I do want to talk about writing as I made some pretty big changes in 2019. And then finally we can cover the new goals I’m setting 2020.
Writing
Writing has been by far the greatest success of 2019 for me. For around 2 years I’ve seen a pretty meagre growth on my blog, which at times has been demoralising. In the middle of the year I posted very little new content. Around March time in a bout of desperation I decided to syndicate about 10 guest articles I’d written, publishing one per week for a few months in total. I expected almost nothing to happen as a result, but then this happened…
Growth In Traffic
Traffic Growth 2018-2019
My traffic started to budge upwards for the first time in nearly two years. The big difference was that some of the articles started to rank properly in Google and were getting consistent and passive traffic. That traffic was starting to compound and visitors were going up. Inspired by the growth I made some quick fixes for mobile and SSL which again saw the growth increase and metrics like bounce rate game down massively.
The growth continued.
The Cloud Native Blog Is Born
The traffic improvements gave me a huge boost of energy. In the wake of the success I decided to niche down the blog to a single topic: Cloud Native. Which meant I had to cut a lot of content. I chose three main areas to focus on increasing content in. And they are: Containers, Serverless and Infrastructure As Code. As I was making these changes to Cloud Native, my attention also turned to my newsletter.
A New Approach To The Newsletter
Since I launched the blog I’ve ran a newsletter. Previously the newsletter sent out the latest post on the following Monday. But the big probem with this approach for me was that since my posts were so broad (read: random) it was almost impossible for any reader to make sense of it.
So I also cut the old newsletter. I stopped sending automated emails in favour of a curated newsletter. And instead of just being my posts, I wanted to also feature posts from the community. So if I had no new posts for the week there would be content from the community.
With the new Cloud Native approach in mind, I re-launched the newsletter as the Cloud Native Software Engineering Newsletter. Which feels like a better and more consistent deal for the reader and is definitely more manageable.
And on that note, that pretty much covers the changes I made within my writing in the last year. And now we can move onto my favourite part of these posts, and that’s clarifying my goals for the coming year.
The Goals For 2020
I’ve pondered my goals for the next year a lot. And I’ve come to the conclusion I’d like to focus on the blog itself throughout 2020. I will be growing the user base and focusing on really improving the content as best as I can. That might mean investments in graphics, images, whatever makes the best content and experience.
Okay, so that’s a bit vague — what is the goal in real terms?
- Grow traffic for The Dev Coach to 10,000 visitors per month (by posting consistent high-quality articles).
- Increase sign-ups on the Cloud Native newsletter to 1,000 emails
And in addition to that, I will:
- Finish the three cornerstone articles: Serverless, Containers and Infrastructure As Code.
- Release at least three email (drip) courses (likely to be one for each of these cornerstone topics)
And that’s it.
Onto 2020
2019 has been a really turbulent year for me — but I’m keen for 2020 to be more consistent. Hopefully I can help you out through my writing. And lastly if you made it this far, thanks so much for reading and I’d love for you to check out the blog. If there’s anything I can do to help or if you have feedback, please get in touch as I do love to hear from readers.
Good luck with your 2020, and…
Speak soon Cloud Native friend!
- 2023 Summary: Data Driven Stories About The Cloud - December 31, 2023
- 2022 Summary: The Open Up The Cloud System - January 1, 2023
- Open Up The Cloud Newsletter #30 (January Recap 2022) - March 1, 2022