How to Reduce Kubernetes Cost in AWS?

🎬S3:E03 Reducing your AWS bill

Hey, TechFlixers!

As more and more applications become cloud-native, one of the most important skills to have for software engineers is understanding how to design systems that are cost-effective.

Companies casually spend millions of dollars on cloud costs. Perhaps you can take a second look at the existing systems in your org to understand how you an optimize further.

In this episode, we focus on a specific cloud service for running Kubernetes and explore a real-world case study of how Adidas optimized its costs for the same.

---

🔦 Spotlight

📢 How Adidas reduced their Kubernetes cluster costs in AWS by 50%!

Adidas recently shared its journey of slashing the costs of its Kubernetes clusters on AWS by up to 50%. Here’s a quick summary.

1. Cheaper EC2 Instances with Karpenter

   Started leveraging Karpenter, a cluster autoscaler that optimizes node utilization by using various instance types and sizes. Karpenter's ability to use AWS spot instances—unused compute capacity offered at a lower price—was a game-changer. It ensures the lowest price and minimal risk of interruption by selecting the best spot instances available.

2. Automating VPAs with Kyverno

   To enhance resource utilization, Adidas automated the creation of Vertical Pod Autoscalers (VPAs) using Kyverno, a policy tool typically used for improving application security. This unconventional approach allowed them to dynamically adjust resource requests and limits for each workload, significantly optimizing their resource usage without manual intervention.

3. Scaling Down During Off-Hours

   They employed kube-downscaler to reduce compute hours during non-office hours, scaling applications down to minimal replicas when not in use. This scheduling tool efficiently scales applications based on predefined schedules, further cutting down unnecessary resource usage and costs.

4. Custom Metrics for Accurate Scaling

   Resource metrics don't always reflect the actual application load. To address this, they used KEDA (Kubernetes Event-driven Autoscaling), which scales applications based on various external metrics, such as those from Prometheus or Kafka. This approach allows for precise scaling, including scaling down to zero replicas when needed.

5. Ensuring Node Removal with Policy Enforcement

   Adidas faced issues with underutilized nodes not being removed due to stringent Pod Disruption Budgets (PDBs). They tackled this by enforcing Kyverno policies to ensure PDBs allowed for pod deletions.

Results

These measures resulted in a 50% reduction in monthly costs for development and staging clusters, with a significant 30% savings in CPU and memory across these environments.

Overwhelming? Worry not. Just keep the basic intuition of various possibilities. Make a note of the tools used by Adidas to explore further. Perhaps, they might fit into the context of your product’s architecture, too, and help you optimize costs.

Other interesting reads

• A Reddit thread about Kubernetes cost savings.

• Best practices for running cost-optimized Kubernetes applications on Google Kubernetes Engine.

• Optimize costs in Azure Kubernetes Service.

---

🚀 Power Up

Recently I stumbled upon an interesting question asked in Google Interviews.

“Given a 4GB binary file, find the shortest byte sequence that is not present in the file”.

Let's simplify and break it down step-by-step.

What Are We Dealing With?

1. Binary File

   It’s basically a file filled with a series of bytes.

2. Byte

   1. A byte consists of 8 bits. Each bit can be either 0 or 1.

   2. A byte can represent numbers from 0 to 255. Why? Because with 8 bits, you can have 2^8 (256) possible combinations, ranging from 00000000 to 11111111 in binary.

      Here are some examples of how binary numbers convert to decimal (base 10).

      1. 00000000 (binary) = 0 (decimal)

      2. 00000001 (binary) = 1 (decimal)

      3. 00000010 (binary) = 2 (decimal)

      4. 11111111 (binary) = 255 (decimal)

   3. So, a byte can have any value between 0 and 255.

The Goal

We need to find the shortest sequence of bytes that doesn't appear anywhere in this gigantic 4GB file.

How Do We Solve This?

Imagine our binary file contains this sequence:

[10, 20, 30, 40, 50, 60, 70, 80, 90, 100]

We look for the shortest sequence of bytes that isn't in this file.

• 1-byte sequences

  All possible 1-byte sequences are from 0 to 255. Since our file only has 10 unique bytes, there are many 1-byte values (like 1 or 255) that are missing.

• 2-byte sequences

  All possible sequences range from [0, 0] to [255, 255]. Again, since our file is small, there are many 2-byte sequences missing.

• 4-byte sequences

  Check sequences like [10, 20, 30, 40], [20, 30, 40, 50], etc. Eventually, we'll find a sequence like [1, 2, 3, 4] that's missing.

We have to extend this exact intuition to a 4GB sequence with ~4 billion bytes.

In simple brute force terms:

1. We generate the list of all possible n-byte sequences.

2. We also generate a list of all n-byte sequences present in the 4GB file.

3. We compare both these files to check if any sequence in the first list is missing in the second list.

4. For an n-byte sequence, the moment we find that something is missing, n will be the required length—the answer.

In a generic case, the answer can be directly considered as 4, as mathematically, the number of all possible 4-byte sequences (256^4 which is approx 4.2 billion) is much more than the 4-byte sequences you can make with a 4GB file (which approximately has 4 billion bytes).

To give a quick generic answer, the above math logic can be used to calculate for any given n-GB file. The answer is n where 256^n > number of bytes in the n-GB file.

But this may not hold for any special scenarios where byte sequences are repeating a lot and the shortest missing sequence can be lesser than 4.

Code Implementation

1. We need a general function to generate all n-byte sequences.

2. We need another function to read the binary file and “see” what sequences are there in the file.

3. We need a function that compares the above two lists to check if something is missing.

4. We need a small wrapper on this to perform the above three steps starting with n=1, 2, 3… and so on.

Sample Code

def read_n_byte_sequences(file_path, n):
    """Read n-byte sequences from the file and store them in a set."""
    seen_sequences = set()
    window = []

    with open(file_path, "rb") as file:
        while True:
            byte = file.read(1)
            if not byte:
                break
            window.append(byte[0])
            if len(window) > n:
                window.pop(0)
            if len(window) == n:
                seen_sequences.add(tuple(window))

    return seen_sequences

def generate_all_n_byte_sequences(n):
    """Generate all possible n-byte sequences."""
    sequences = []
    def generate_sequence(seq, depth):
        if depth == n:
            sequences.append(tuple(seq))
            return
        for i in range(256):
            seq.append(i)
            generate_sequence(seq, depth + 1)
            seq.pop()
    generate_sequence([], 0)
    return sequences

def find_missing_n_byte_sequence(file_path, n):
    """Find the shortest missing n-byte sequence in the binary file."""
    seen_sequences = read_n_byte_sequences(file_path, n)
    
    for seq in generate_all_n_byte_sequences(n):
        if seq not in seen_sequences:
            return seq

def find_shortest_missing_sequence(file_path, max_length=4):
    """Find the shortest missing byte sequence up to max_length bytes."""
    for n in range(1, max_length + 1):
        missing_seq = find_missing_n_byte_sequence(file_path, n)
        if missing_seq is not None:
            return missing_seq
    return None

# Example usage
missing_seq = find_shortest_missing_sequence('binary_file.bin')
print(f"The shortest missing byte sequence is: {missing_seq}")

Note that there can be further optimizations to this approach, but let’s not confuse ourselves for now.

Oh, btw, don’t worry about how you’d ever be able to write some clean code like above and solve this question in 30 minutes in a real interview.

Most people can’t. Most people who do clear the interview also can’t. It's not the first time they approached this question. Probably not even the second time. Regular practice is required to hone this, especially when you have to perform under time constraints in a high-pressure interview environment.

So, don’t get too demotivated when you find it hard. Even I can’t solve them. Just focus on the intuition of the logic and fundamental approach. Break things down into small blocks. Familiarize yourself with each block over time.

For example, it’s a lot easier to first learn how to generate all possible n-length sequences for a given n. That block of knowledge can be used in a “plug and play” manner across many different problems where you are required to generate such sequences to compare with something.

Until next time.