Relative positional embeddings with RoPE

Word order matters to a language model. In this post I'll explain RoPE: a technique that makes attention between two tokens depend on the distance between them instead of their absolute sentence position.

Word order matters to a language model. In this post I'll explain RoPE: a technique that makes attention between two tokens depend on the distance between them instead of their absolute sentence position.

K-mers are frequently used in bioinformatics to profile and compare genomes. This article describes a technique for compactly representing long k-mers with locality-sensitive hashing.

The Netherlands is very flat country, with most its area below sea level. So how flat is it really, and how to we keep the water at bay? Let's visualize the official "height" dataset of the Netherlands to find out!

Through some clever mathematical definitions, we can use OpenGL "point" primitives to draw any kind of vector shapes. This article explains how that's done and how it applies to my Google Summer of Code 2015 project.

Servos are useful small motors with applications in robotics, RC cars, and others. Here, we describe how to use an ATTiny44 to control its position.


Developed interpretable machine learning models for integrating multi-omic (genomic, transcriptomic, and proteomic) data from 10,000 participants in the "All of Us" biobank.
Led development of strain-level human microbiome analysis tools that enabled new insights into the role of the gut microbiome and recurrent UTIs. My work here contributed to my PhD at Delft University of Technology.
Developed web-based geographic information systems (GIS) tools using Django and PostgreSQL, securing multi-million euro infrastructure contracts with the government through improved collaborative project planning and design.