Relative positional embeddings with RoPE
In language models, the order of tokens is of critical importance. This post explains RoPE: a technique that ensures that the pairwise token attention weights only depend on their relative position.
Using locality sensitive hashing to compactly represent k-mers
K-mers are frequently used in bioinformatics to profile and compare genomes. This article describes a technique for compactly representing long k-mers with locality-sensitive hashing.
Controlling a servo with an AVR microcontroller
Servos are useful small motors with applications in robotics, RC cars, and others. Here, we describe how to use an ATTiny44 to control its position.
