Complex Technologies are extension of Simple Ones

I have been learning System Design lately. And one of the thing I notice time and time again is how useful Algorithms, Data Structures and Design Patterns are.

For example: A message queue like Kafka is not only simple conceptually but scales to handle much large data.

This is not to say that once can go an build some things advanced by themselves.

There goes a lot into making a software. Every tiny improvement. Edge case and it’s bugfixes are a contribution of years of research.

But to know that something is an extension of simple concepts like Queue and Pub/Sub is humblings. That the tools that are built are just a combination of ideas that were arranged and executed well.

What does this mean practically?

Practically, this means we try to make things complex up front.

For example: CI/CD can be simple at the start too.

You don’t need to add to many things to just automate pushing something to an environment. You can add and update things overtime.

Similarly, you can let go of automation early on.

For example: calculating DORA metrics by hand