Today I Learned
Short notes — smaller than a post, too useful to lose.
I was watching some breakdown of cloudflare latest new blog, here are some quick learning points
Vec<> which are linked list in Rust, allocates a certain amount of memory size (e.g say length of 8) before expanding dynamically. In this case, if the size is not fully used , you might have empty slots of reserved memory. In the normal case this might be okay but for cloudflare who stores ALOT of data , a single byte memory saving could represent GB to TB scale of memory savings. Refer to their diagram below!
Additionally, their problem has 2 caveats, (1) They are caching dns entry (2) Once the dns response is received , they never ever mutate it again
Solution:
Use a data structure that is generic and cannot grow in further capacity but store exactly what it needs to and not any extra space for future elements.
The solution they came up with is Box<[T]>. Basically get rid of having to store the extra capacity. Why Box and not Array or slices? Box<[T]> stores memory on the heap not the stack, so its size does not need to be known at compile time, additionally, it can live and be cleaned up approrpiately rather than by its scope.
Source: https://blog.cloudflare.com/dns-cache-memory-optimization-1111/
Sincerely,
Sean