Serverless: An Ultimate Guide

Serverless, it’s the new hotness. But Serverless is also a very mystical, almost magical seeming technology. Nonetheless, understanding Serverless is an essential skill for any software engineer who is working in (or near) the cloud.

But learning Serverless isn’t straight-forward. Serverless is both a concept and a company. Not just that, but every Cloud Provider seems to define Serverless differently. It can get quite frustrating and confusing.

Over the past year I’ve been writing a lot about Serverless and wanted to bring everything I’ve written together into one easy place so you can fill in any gaps that you might have about what Serverless is, or how it works.

By the end of this article you should understand what Serverless is, the benefits, and the main tools that you should know when it comes to Serverless and much much more. 

Breaking Serverless Down

Before we jump into the details, let me give you an idea of how we’re going to break down this big old topic of Serverless…

Part 1: Understanding Serverless — In this first part we’ll discuss what Serverless is, why it’s useful, and some of the times that Serverless is not so useful, and why. 

Part 2: The Serverless Ecosystem — In the second part we cover how the Serverless ecosystem fits together to help you make decisions about things like tooling and cloud providers. 

Part 3: Thinking in Serverless — Now that we know what Serverless is and how it fits together we can tackle how to start thinking in a Serverless way. 

Part 4: Getting Started with Serverless — By now we’ll have an understanding of all the Serverless basics and start to discuss ways that you can start diving into Serverless, to learn more. 

Part 5: The Serverless Community — Lastly we discuss ways to continue your learning by engaging with the Serverless community to get the best knowledge available. 

And without further ado, let’s get to it…

Part 1: Understanding Serverless

We’ll being our tour of Serverless by digging into what Serverless is.

What is Serverless?

Serverless has many definitions, and when people refer to Serverless, they could mean any one of the following…

Serverless (as a concept) — Serverless can be defined purely as it’s characteristics, i.e any service which abstracts away server concerns, providing a customer with only computing features. Within this definition any service can be Serverless, such as a database or a file server.

Serverless (the company) — Confusingly there is also a company founded called Serverless Inc. They own serverless.com and they created the Serverless Framework. Serverless Inc is not related to any cloud provider in particular (we’ll dive more into Serverless Framework later on).

Serverless (as compute) — The last, but most common used definition is Serverless when applied to compute. Sometimes called “Functions as a Service” (FaaS) or simply “functions” for short. Serverless compute is where you upload code to a cloud provider and can execute it on demand without having to worry about the infrastructure concerns of that code.

Now I know what you’re thinking, that’s a lot of definitions. And it is. But for the sake of continuing our article we’re going to narrow our scope to the last definition and focus on Serverless as compute. After all, when you hear people refer to Serverless they often mean Serverless compute services, or functions.

Benefits of Serverless

Now we have a basic understanding of the different definitions of Serverless, let’s take a step back to consider why someone might want to use Serverless in the first place. What are the benefits of Serverless?

Less Infrastructure Maintenance — With a typical server setup for compute you’d need to maintain your underlying infrastructure. That would mean doing things like applying patches to operating systems in order to keep up with the latest security updates. Managing infrastructure manually can be a pain. But when using Serverless all of the underlying infrastructure is managed by the provider.

On-Demand Pricing — In a typical server setup you would pay for the cost of your server whether it’s used or not. With Serverless you only pay for the resources that you use when you need them, which means you shouldn’t ever be paying for resources to sit idle. This is a key aspect for Serverless technology.

Cloud Integrations — Serverless technologies are inherently Cloud Native, which means they come out-of-the-box with many other cloud provider integrations, such as encryption methods, easy access to databases, integration with event systems and more. In a server-oriented architecture you’d be responsible for doing a lot of this manually yourself.

But, as with all decisions there are trade-off’s and drawbacks. Let’s take a look at some of the drawbacks of Serverless…

Drawbacks of Serverless

Cloud Provider “Lock In” — Cloud lock-in is a huge, often controversial topic. Because serverless technology is a very high-level cloud provider service it creates the biggest disadvantage: vendor lock-in. Serverless technologies are by their nature not very portable. Moving between cloud providers isn’t straight-forward. This high degree of coupling can be scary for businesses who aim to reduce dependencies with single cloud providers.

With an understanding of what Serverless is now covered off, let’s turn our attention to the Serverless ecosystem, and let’s explore the various we have available to us when using and setting up with Serverless.

Part 2: The Serverless Ecosystem

Understanding Serverless also means understanding all the different third-party tools, and different Cloud Provider integrations. When it comes to picking the right setup, you’ll want to understand all the different options available to you. Let’s first take a look at an overview of the different main cloud providers and their Serverless offerings…

Choosing between AWS, GCP and Azure.

All of the big three cloud providers provide their own Serverless compute implementations:

Each Serverless implementation within the different Cloud Providers are different. Knowing which one to choose depends on not only the Serverless offering, but also the wider cloud platform. You’ll want to assess many different aspects of the cloud provider as a whole:

  • Does the functionality meet your needs?
  • Does the cloud provider meet your scaling requirements?
  • How easily can you attract software engineers with knowledge of that Cloud Provider?
  • Do the existing security services match your needs?

And then, once you’ve done your homework for which Cloud Provider to use, you’ll then want to explore the different Serverless frameworks.

Setting Up Serverless (Serverless Frameworks)

One thing to consider beyond just the cloud provider that you choose is how to setup your Serverless application. When it comes to setting up your Serverless application, you have quite a few different options…

  • Manual Provisioning — The most straight-forward setup is to simply go into your Cloud Provider and provision Serverless resources manually. It’s a primitive option, but it’s quick and it works. Useful if you’re just hacking around and learning Serverless.
  • Cloud Provider Infrastructure As Code (e.g. CloudFormation) — The next most obvious setup is to use the Cloud Providers built-in infrastructure as code tools. These tools are unlikely to give you any Serverless specific benefits, but it will work and it’s relatively simple.
  • Cloud Provider Serverless Framework (e.g. AWS SAM) — Some of the cloud providers have their own implementation of Serverless frameworks, in AWS that’s AWS SAM which helps to make working with Serverless easier.
  • Open Source Serverless Framework (e.g. Serverless Framework) — Another popular option is to use an independent third party Serverless framework tool, such as the Serverless Framework, and since Serverless Framework is so popular, we’ll cover it in the next section.
  • Open Source Infrastructure As Code (e.g. Terraform)  — You could setup your Serverless infrastructure using an open-source infrastructure as code tool.

Understanding differences between these Serverless approaches is difficult. Some give you nice extra’s like local development, but it’s at the cost of more complexity through an another layer of abstraction.

Choosing a Serverless setup is about balancing complexity such as added abstractions with the features you think you’ll really need. That said, unlike choosing a Cloud Provider, your serverless setup is not a permanent decision so don’t lose too much sleep making a decision. Jump in and start experimenting.

If you want to know more about the different options you’ve got for setting up Serverless on AWS, check out: Serverless on AWS Lambda: A Comprehensive Comparison Of Approaches (Serverless Framework vs SAM vs Terraform vs CloudFormation)

The Serverless Framework

Serverless Framework

Serverless Framework

As we talked about in the definitions at the start. When people refer to “Serverless” they sometimes mean Serverless Inc, the creators of the open source tool, Serverless Framework. So it helps to understand a bit more about what Serverless Framework is and how it fits into the Serverless ecosystem.

Serverless Framework is an open source tool that makes deploying and managing Serverless functions easier. It has integrations into all the main cloud providers so you can use the tool pretty much anywhere.

Serverless Framework helps you manage the infrastructure for your serverless projects, such as setting up HTTP routes to your functions easily. It also helps set up local development environments for you, and makes deployment of your functions easy.

To understand more about Serverless Framework, check out:  Serverless on AWS Lambda: A Comprehensive Comparison Of Approaches (Serverless Framework vs SAM vs Terraform vs CloudFormation) 

Part 3: Thinking In Serverless

Working with Serverless means learning to work with the concept, not applying old-school thinking to a new idea. Which is why now we’re going to take a look at some of the ways that you can start to “think” in Serverless.

We’ll look at the different concepts you need to know and consider, and we’ll also look at where Serverless might trip you up in ways you didn’t expect.

Concepts You Should Know About Before Learning Serverless

When it comes to learning Serverless you’re going to come into contact with many different concepts and ideas. It helps to have at least a surface level understanding of these ideas before diving into Serverless. But what are these ideas?

  1. Infrastructure As Code — The practice of storing your infrastructure code in source control (for more, check out the Infrastructure As Code Ultimate Guide).
  2. A Cloud Provider — You’ll need to know your managed infrastructure service well (for more, check out the AWS Ultimate Guide)
  3. Distributed Systems Concepts — You’ll need to understand the difficulties in having components communicating via a network.
  4. Software Pipelines — You’ll likely need an automated process for testing and packaging software components.
  5. Monitoring & Observability — You’ll need the ability and tools to remotely debug your Serverless application at some point.
  6. Microservices — An architecture practice of breaking down software components into independently deliverable parts.

Taking a look at some of these topics will help you start to navigate the world of Serverless and will greatly smooth out your journey.

To understand more about these concepts, check out:  The 6 Concepts You Need To Know Before Trying Serverless

Misconceptions of Serverless

When I was first learning Serverless there were a few ideas / concepts that I bumped into that weren’t very obvious at first. Let me take you through these as you might have the same misconceptions as I did. If you want to learn more, check out the article after the list for even more details.

  1. Functions scale independently — Under a decent amount of load, there are some ways that functions can impede the performance other totally unrelated functions. Having high load on an internal operations function could cause your production functions to error if you’re not careful. These failures are due to limits from the cloud providers on the number of concurrent executions that you can execute.
  2. Functions ramp to meet any load — Function scaling for most scenarios is quite straight-forward, if the cold-start of a function can be tolerated. However, when scaling demands are very quick, functions can in fact struggle to scale quick enough. I was wrong to think Serverless could meet any scaling pattern out-of-the-box, some tweaking and optimisations are often necessary.
  3. Functions can’t be DDOS’ed — You may think that because Serverless functions scale “infinitely” that they cannot be attacked by floods of requests, such as under a DDOS attack. But due to how Serverless functions scale they do have vulnerabilities, such as being exploited by high volume quick attacks.
  4. Functions don’t have memory leaks — Because sequential Serverless invocations re-use the same underlying containers Serverless functions can in fact have memory leaks. With Serverless you still have to be careful with how you use in-memory storage as you may mutate the state of your Serverless functions between executions.
To read more detail about these misconceptions, check out:  Misconceptions of Serverless: 5 Things You Thought AWS Lambda Did; But It Doesn’t.

Serverless & Event-Driven (Async)

SQS & Lambda Logos

SQS & Lambda Logos

Serverless architectures naturally lend themselves to event-driven (asynchronous) architecture. In an event-driven world software components react to events and emit events rather than issuing commands to other components. Events allow software components to be more de-coupled as no reference is made to the

One way of implementing event-driven architectures is with a queue. A queue is a software component that stores events to be processed at a later point in time. AWS for instance has the managed service SQS for this purpose. AWS Lambda can be easily configured to listen and poll events from an SQS queue.

But event-driven architectures come with added complexity. Business logic flows with events become more implicit as (theoretically at least) any software component can consume and process events, which can make it harder to understand business flows. Events also need additional concepts such as dead letter queues and retrying logic.

For more on how to implement AWS Lambda with SQS take a look at: AWS Queues And Lambda Processing: A Simple, Quick Walkthrough. 

Part 4: Getting Started With Serverless

Serverless Beginner Project Ideas

So after all this chat about Serverless, you might be itching to dive in. Getting hands-on is definitely one of the best ways to gain an understanding into how a technology works.

But of course you might be wondering what you should build? Serverless can be used for many different things, here are a couple of ideas that you can use to experiment with Serverless:

Operations Function — A function that performs tasks within your environment, such as security scanning. You could also try settingg your function to run on a schedule, such as every evening. Explore your cloud providers API / SDK’s to extract data about your environment.

HTML Website — There’s no reason you can’t use Serverless to render a static website. There are other options for static sites, but they can also be a great way to experiment with Lambda functions to explore how they work. Try extending your serverless function with a domain name and/or SSL certificate.

HTTP API — Lastly, a very common use case for Serverless is using functions to create API’s. You could setup your very own API, and as before try attaching a domain name to it and/or an API gateway or load balancer.

For a more in-depth run through of some example projects, check out: 3 Simple Serverless Beginner Projects Ideas 

Part 5: Serverless Community

Serverless Blogs / Podcasts

Naturally as you learn more about Serverless you’ll start to wonder about where to get information on to stay up-to-date.

Below is a list of some prolific writers and podcasters in the Serverless space…

Serverless Course

If you’re interested in learning more about Serverless I am currently in the process of putting together a free email course called Serverless with Lambda: Zero to Hero. You can already sign-up to register to receive the course when it’s released. Be sure to check it out…

Find Out More

Become A Serverless Pro

And that concludes our look at Serverless for today. I hope that helped to clear up some of the confusion around Serverless for you.

If you’re itching to learn more about Serverless you can take a look through some of the past articles I’ve written on the topics you’re most curious about.