Things I learnt about weekly update emails

Recently I had the "pleasure" to work on something I've never worked on before. An organisation next door came asking for help. They wanted us to donate engineers helping them testing our code running in their binary. They were carrying out a huge architecture change, and would like to make…

Protecting a service against load

I've been talking with a friend of mine about interview questions. One question keeps popping up: what are the ways that we could protect a service against excessive traffic? What we refer to as "a service", is a typical micro service that serves user requests. A service usually runs on…

Gracefully shutdown an RPC service

RPC servers usually have several components: a thread pool (or green threads), local resources that are shared between requests (logging, metrics, throttling data, cache etc), and remote resources (databases and dependency systems). It is usually hard to cleanly shutdown a RPC server because the different types of resources involved. It…

A mutable object is equivalent to a state transfer API

When coding on ruaft [https://github.com/ditsing/ruaft] I adopted this "state transfer" style API to allow a graceful shutdown. To make the shutdown "safe" I insisted on taking the Raft instance away when the API is called, like the following. impl Raft { pub fn kill(self) { // do things.…

Read only requests in Raft (part 1)

In this post, we'll explain how Raft handles read only requests. To those who are unfamiliar, Raft is an algorithm that replicates a shared state to a group of servers using leader election. Raft maintains a change log that is consistent among all replicas. The log is append-only. Each change…

Object lifetime and threading

Last time we talked about object lifetime and ownership. Naturally scopes and objects form a tree hierarchy. The root of the tree is the scope where the program starts executing. Beyond the tree structure, we can pass information between scopes with the help of dynamic lifetime. Dynamic lifetime is hard…