Lately I am thinking about this more and more:
What are the principles to write code that scales? ๐ค
Leaving aside the specific language, design patterns, and theory. What should a developer focus in practice when writing large software projects?
Some (initial) ideas:
- ๐ Local reasoning
- Modularity
- Refactoring
- ๐ฉ Error handling
- ๐งช Testability
- Configuration
- ๐ฌ Debugging/Observability
- ๐จ State management
- Data structures
Concepts to understand about error handling ๐ ๐ Error vs Exception ๐ Stack track ๐ Error object ๐ try/catch/finally ๐ Union types (Option/Either) Error handling is at the core of every program Back to refine and master the basics ๐
This is how it looks like in practice ๐
Local reasoning: the secret for software that scales
You work in a car factory. You role is to attach doors to the frame of the car:
- You expect the car's frame to be delivered to you (input)
- You perform your role (attach the door)
- You produce a car with doors (output)
In this process you don't need to bother with anything else: get the frame, attach the door, move to the next frame.
This is what makes factories scale: local reasoning.
It works the same for code:
Local reasoning: property of some code wherein the correctness of the code can be inferred locally (under specified assumptions) without considering prior application state or all possible inputs
(Source)
/*
* Can you tell what this function does, without reading any other part of the codebase?
* - Get a `card` and `amount`
* - Perform a payment to the given `card` for the given `amount`
* - It requires a `PaymentService` to work (e.g. Stripe)
* - It can fail if `card` is invalid or we are unable to check its validity
* - When successful, it returns the details of the payment (`Payment`)
* - It performs a side-effect (return type `Effect`)
*/
const makePayment = (params: { amount: number, card: Card }): Effect.Effect<
Payment,
InvalidCardError | CheckingCardError,
PaymentService
> => /// ...
There is more.
Every week I build a new open source project, with a new language or library, and teach you how I did it, what I learned, and how you can do the same. Join me and other 700+ readers.
Error handling
Error handling is not discussed enough ๐คจ
It's found and required in every software, but there is still a lot of confusion around it
In practice:
- Returns errors as values
- Propagate errors between functions
function getCreditCard(email: string): Effect.Effect<Card, CheckingCardError> {
/// ...
}
function getIsCardValid(card: Card): Effect.Effect<true, InvalidCardError> {
/// ...
}
/*
* `getCreditCard` and `getIsCardValid` define their own errors (on their return type).
*
* Since `makePayment` calls both `getCreditCard` and `getIsCardValid`
* those errors are propagated also in the return type of `makePayment` โ๏ธ
*/
function makePayment(params: { amount: number, card: Card }): Effect.Effect<
...,
InvalidCardError | CheckingCardError | ConnectionError
> {
/// ...
}
Testability
Tests cannot be an afterthought ๐๐ผโโ๏ธ
Writing testable software requires thinking about the structure of your code from the start
In practice:
- Pure functions: easy to (unit) test, same input always same output
- Dependency injection (manage effects): when side effects are necessary, inject mock implementations
it("should return a valid SubscribeResponse when request successful", async () => {
const response = await (
await Server.main.pipe(
Effect.provideService(
Request,
// ๐ API request mocked locally
new globalThis.Request("http://localhost:3000/", {
body: JSON.stringify({ email: "" }),
method: "POST",
})
),
// ๐ Configuration parameters mocked locally
Effect.provide(layerConfigProviderMock),
Effect.runPromise
)
);
// ๐ Check response to local mock value
expect(response).toStrictEqual(subscribeResponseMock);
});
Debugging/Observability
One thing is making code that compiles, another thing is verifying that it does what you expect (in every case).
Debugging is necessary: you cannot foresee everything that may happen.
When something doesn't work as expected, inspecting internal state at runtime becomes necessary ๐ฉ
In practice:
- Logging: not just text messages, but also timestamps, stack traces, relevant state, and more
- Metrics: inspect performance, count function calls, track external requests
- Tracing: complete overview of the lifetime of requests
Inspect every function call, how much time it takes, external requests, and more
State management
Values change: we want these changes to be performant, predictable, and easy to understand/visualize.
In practice:
- Data structures: optimize performance based on the most common operations
- State machines: define all possible states and make state transitions predictable and easy to visualize
You can read more about these topics below:
Regardless of programming language or area of expertise, these principles will always follow you everywhere ๐ป
These are underappreciated topics, I plan to cover each of them in more details (and in practice).
Stay tuned, this can become something big ๐คซ
See you next ๐