Learning notes
Notes from my learning journey
Over the years I have tried ideas in notebooks while reading, watching lectures, doing university labs or solving problems at work. These are those experiments, rewritten as short articles: one idea per note, explained plainly, with code that runs and the sources I learned from.
148 notes · 13 topics · 2017–2026 · dates show when I first explored each topic.
Computer Vision
Detection, segmentation, self-supervised features and the geometry of images.
Geospatial & GIS
Photogrammetry, LiDAR, rasters and polygons: putting pixels on the map.
3D Vision & Point Clouds
Points, voxels, meshes and the projections between 3D and 2D.
Signal & Image Processing
Waves, wavelets, Fourier transforms, filters and image statistics.
Deep Learning Foundations
Backpropagation, normalisation, losses and the plumbing of training.
NLP & LLMs
From bigram models to attention and GPT, built up from scratch.
Machine Learning
Clustering, trees and forests, SVMs, ensembles and the first steps of reinforcement learning.
Model Evaluation
Honest validation, thresholds, imbalance and reading errors beyond one number.
Statistical Inference
Bootstrap, permutation tests, multivariate tests, multiple testing and missing data.
Bayesian Methods
Priors, posteriors, credible intervals, Bayesian networks and Gaussian processes.
Computational Statistics
MCMC, random number generation, optimisation, EM and the numerics underneath.
Time Series & State Space
AR models, Kalman filters and smoothers, particle filters and hidden Markov models.
Programming & Data
Spark and SQL, algorithms, floating-point numbers and making code fast.
2026
- Monotonic constraints in gradient-boosted trees: building domain knowledge into the modelMachine Learning
- Grouped and spatial cross-validation: why random K-fold lies when samples clusterModel Evaluation
- Regression error analysis beyond RMSE: what each metric hides and how to look past itModel Evaluation
- The shoelace formula: the area of a polygon from its verticesGeospatial & GIS
- What's new in DINOv3: scaling self-supervised vision and fixing dense features with Gram anchoringComputer Vision
- DINOv2: learning visual features without labelsComputer Vision
- Sinkhorn–Knopp: balancing a matrix, and why it solves optimal transportComputational Statistics
- Polygons from detections: fixing invalid shapes and measuring how round or long they areGeospatial & GIS
2025
- Anchors and Region Proposal Networks: from a dense grid to a few good boxesComputer Vision
- Designing anchors for your own dataset: k-means, best possible recall and evolutionComputer Vision
- Feature Pyramid Networks: strong features at every scaleComputer Vision
- Stacking regression models: combining predictions without fooling yourselfMachine Learning
- Representing 3D data: points, voxels, octrees, meshes and depth maps3D Vision & Point Clouds
- Projecting point clouds to images: top-down rasters, spherical panoramas and the z-buffer3D Vision & Point Clouds
- Forest structure from above: from tree heights and crowns to DBH, basal area and Lorey's heightGeospatial & GIS
- Panoptic segmentation: things, stuff and one label per pixelComputer Vision
- From instance masks to polygons: contours, morphology and the usual pitfallsGeospatial & GIS
- Inside GPT-2 (124M): reading the weights and rebuilding the modelNLP & LLMs
- Vision Transformer: images as sequences of patchesComputer Vision
- Radial lens distortion: barrel, pincushion and how to model themSignal & Image Processing
- Sequence-to-sequence models: an encoder, a decoder and teacher forcingNLP & LLMs
- Choosing a C++ STL container: complexity, memory layout and the usual trapsProgramming & Data
- Tapping a CNN backbone: multi-scale feature maps from VGG and ResNetComputer Vision
- LSTMs: gates and a cell state that let gradients surviveNLP & LLMs
- Recurrent neural networks: a hidden state that carries the pastNLP & LLMs
- Word embeddings: from co-occurrence counts to word2vecNLP & LLMs
- Tokenization and byte-pair encoding: how text becomes integers for an LLMNLP & LLMs
- Overlapping polygons at scale: find pairs with an STRtree, then split the overlapsGeospatial & GIS
- Causal attention: how a GPT is stopped from reading the futureNLP & LLMs
- Building a small GPT from scratch, one improvement at a timeNLP & LLMs
- Multi-head attention: several small attentions in parallelNLP & LLMs
- Self-attention: queries, keys, values and the square-root scalingNLP & LLMs
- Preparing text for LLM training: sliding windows, batches and input embeddingsNLP & LLMs
2024
- Batch normalization: what it does, why it helps, and how it behaves at test timeDeep Learning Foundations
- A WaveNet-style language model: fusing context in a tree instead of all at onceNLP & LLMs
- Bigram language models: counting pairs, then learning them with a neural netNLP & LLMs
- Backprop by hand through an MLP language modelNLP & LLMs
- Getting initialization right: confident softmax, saturated tanh and Kaiming scalingNLP & LLMs
- An MLP language model: embeddings, context windows and honest evaluationNLP & LLMs
- Computational graphs: how a formula becomes something you can differentiateDeep Learning Foundations
- Backpropagation by hand, then automated: a micrograd walkthroughDeep Learning Foundations
- Training a tiny neural network from scratch (and the zero-grad bug)Deep Learning Foundations
- MSE, cross-entropy and focal loss: choosing a loss for regression and classificationDeep Learning Foundations
- Overlap-based losses: IoU, GIoU, DIoU, CIoU, Dice and TverskyDeep Learning Foundations
- The camera extrinsic matrix: rotation, translation and change of basisSignal & Image Processing
- Detecting small objects in very large images: tiling, clipping and stitchingComputer Vision
- Statistical models of natural images: from pixel histograms to the 1/f power spectrumSignal & Image Processing
- The discrete Fourier transform of images: magnitude, phase and shiftsSignal & Image Processing
- Gabor filters: local frequency and orientationSignal & Image Processing
- Sampling and aliasing: from the Nyquist rate to an aliasing attack on a CNN layerSignal & Image Processing
- Linear filters and the Gaussian blurSignal & Image Processing
- Reading drone image metadata: EXIF, XMP, GPS, gimbal angles and ground sampling distanceGeospatial & GIS
- Classic CNN architectures: LeNet, VGG, GoogLeNet and ResNetComputer Vision
- Counting dense objects with density and confidence mapsComputer Vision
- Spatial pyramid pooling: SPP, PSPNet's pyramid pooling module and SPPFComputer Vision
- Class activation maps: CAM and Grad-CAMComputer Vision
- Airborne LiDAR basics: returns, classes, ground filtering and DTM, DSM and CHMGeospatial & GIS
2023
- Colour spaces for image processing: RGB, HSV, CIELAB and YCbCrSignal & Image Processing
- Efficient input pipelines: keeping the GPU fed without fooling yourselfDeep Learning Foundations
- Parallel Python: threads, processes, asyncio and the GILProgramming & Data
- Profiling Python: finding where the time and the memory goProgramming & Data
- Forest stand delineation: turning canopy rasters into management polygonsGeospatial & GIS
- Clipping rasters and point clouds with polygons: windows, transforms, nodata and tilesGeospatial & GIS
- Colour transfer and histogram matching: making one image look like anotherSignal & Image Processing
- Contrast enhancement: histogram equalisation, CLAHE and gamma correctionSignal & Image Processing
- YOLOv1: object detection as one regression on a gridComputer Vision
- Dense detection heads: how a feature map becomes boxesComputer Vision
- Mean average precision: how object detectors are scoredComputer Vision
- Looking inside a CNN: first-layer filters, feature maps and layer-by-layer comparisonsComputer Vision
- Bounding-box formats: corners, centres, YOLO, Pascal VOC and COCO, and how to convert without bugsComputer Vision
- Nadir and off-nadir viewing: the viewing angle of every pixel in an aerial photoGeospatial & GIS
- Orthorectification and true orthophotos: why buildings and trees still lean in an orthomosaicGeospatial & GIS
- Relief displacement: why tall objects lean outward in aerial photosGeospatial & GIS
- Interpolating scattered data onto a grid: nearest, linear, cubic, RBF and IDWGeospatial & GIS
- Keypoints, descriptors and RANSAC: matching two images and estimating a homographyComputer Vision
2022
- Photometric colour augmentation: brightness, gamma, hue shifts and Fancy PCAComputer Vision
- Haze, the scattering model and the dark channel priorSignal & Image Processing
- Segmentation decoders: FCN skip fusion, U-Net, and how to upsample without checkerboardsComputer Vision
- 2D geometric transformations: from translation to homographies, and how to warp an image correctlyComputer Vision
- Thresholding an image: the mean, Otsu, Ridler–Calvard, Gaussian mixtures and adaptive thresholdsSignal & Image Processing
2020
- Check-then-act races: why a transaction alone doesn't stop overbookingProgramming & Data
- Entity linking: from spaCy spans to Wikipedia pages with a prior and a context modelNLP & LLMs
- Bag-of-words vs. LSTM for tweet sentiment: what each model can actually seeNLP & LLMs
- Natural language inference: the hypothesis-only baseline and cross-unigram featuresNLP & LLMs
- Functional dependencies, attribute closure and BCNF: normalisation without the hand-wavingProgramming & Data
- Rand index vs. adjusted Rand index: comparing a clustering with gold labelsModel Evaluation
- Multinomial Naive Bayes for text, and tuning it without fooling yourselfNLP & LLMs
- Inner vs left joins: where rows quietly go missingProgramming & Data
- Tf-idf: a tiny search engine and keyword extractorNLP & LLMs
- Training an RNN on one long time series: whole sequence, random windows, or stateful windowsDeep Learning Foundations
- Estimating the likelihood with a particle filter, and using it to fit a parameterTime Series & State Space
- Particle filters: sequential importance sampling, weight collapse and resamplingTime Series & State Space
- Gaussian process regression: the posterior via Cholesky, length scales, and periodic kernelsBayesian Methods
- Tabular Q-learning: the update, epsilon-greedy exploration, discounting, and slippery movesMachine Learning
- Learning noise variances in a state space model with EMTime Series & State Space
- The Kalman filter: predict, correct, and get the likelihood for freeTime Series & State Space
- Kalman smoothing: using later data to clean up earlier estimatesTime Series & State Space
- Structural time series: writing trend plus season as a state space modelTime Series & State Space
- Hidden Markov models: filtering, smoothing, Viterbi, and one-step predictionTime Series & State Space
- Long-range AR forecasts: why they fade to the mean, and when an AR model is stableTime Series & State Space
- Autoregressive models: least-squares fitting and choosing the order by validationTime Series & State Space
- Learning a Bayesian network's structure with hill climbing, and what 'equivalent' graphs areBayesian Methods
- Classifying with a Bayesian network: the Markov blanket, and naive Bayes as a tiny BNBayesian Methods
- Transforming random variables: the CDF method, the Jacobian, and the Cauchy distributionStatistical Inference
- Broadcast the small side: joining big data with a small lookup table in SparkProgramming & Data
- Kernel regression with several distances: summing vs multiplying kernelsMachine Learning
- CNNs aren't rotation invariant: what geometric augmentation can and can't fixDeep Learning Foundations
- What a Conv2D layer actually computes: multi-channel cross-correlationDeep Learning Foundations
- Which row had the max? The greatest-n-per-group pattern in SQL and SparkProgramming & Data
- Thinking in key-value pairs: aggregations with Spark RDDsProgramming & Data
- Bayesian AR(1) in Stan: why the mean gets lost when phi nears 1, and latent AR for countsBayesian Methods
- Class imbalance: baselines first, then class weights or undersamplingModel Evaluation
- Monte Carlo dropout vs. cross-validation: two different kinds of uncertaintyDeep Learning Foundations
- Padding and masking in sequence tagging: how a 99.97% accuracy can be fakeNLP & LLMs
- Gibbs sampling, from a normal model to a mixture with data augmentationComputational Statistics
- Posterior predictive distributions by simulation (and why plugging in a mean is wrong)Bayesian Methods
- Conjugate Bayesian linear regression: prior checks, posterior draws, and credible bandsBayesian Methods
- The Laplace approximation: a normal posterior from optim and the HessianBayesian Methods
- Posterior draws, transformed quantities, and equal-tail vs HPD intervalsBayesian Methods
- Grid approximation: the posterior of a one-parameter model by brute forceBayesian Methods
- Association rules with Apriori: support, confidence, lift, and describing clustersMachine Learning
- Classifying by distance to a subspace: a tiny SVD digit classifierMachine Learning
- The EM algorithm for missing data: a worked exponential exampleStatistical Inference
- K-means: what the SSE tells you, why the seed matters, and how to pick kMachine Learning
- Bootstrap and jackknife: standard errors, confidence intervals and bands around a curveStatistical Inference
- Permutation tests: p-values by shuffling, and what a p-value can't tell youStatistical Inference
- Gibbs sampling: deriving full conditionals for a random-walk smootherComputational Statistics
- Metropolis-Hastings: proposals, the Hastings correction, tuning and checking convergenceComputational Statistics
- Making random numbers: the inverse-CDF method and rejection samplingComputational Statistics
- Tuning one parameter: grid search, golden section, and why BFGS can get stuckComputational Statistics
- Maximum likelihood with optim(): log-likelihoods, gradients and gradient checksComputational Statistics
- Condition numbers: why solving the normal equations can failComputational Statistics
- Floating-point traps: equality, absorption and catastrophic cancellationProgramming & Data
2019
- The C parameter in soft-margin SVMs, and how to choose it honestlyMachine Learning
- Benjamini–Hochberg: controlling the false discovery rate when testing thousands of featuresStatistical Inference
- GAMs and the smoothing penalty: what lambda and effective degrees of freedom meanMachine Learning
- Decision thresholds and loss matrices: when 0.5 is the wrong cut-offModel Evaluation
- PCA vs ICA: uncorrelated is not the same as independentMachine Learning
- EM for a mixture of Bernoullis, and which log-likelihood to watchComputational Statistics
- Hotelling's T²: testing a mean vector, confidence ellipses and simultaneous intervalsStatistical Inference
- Finding multivariate outliers: Euclidean, standardized and Mahalanobis distanceStatistical Inference
- Random forests, out-of-bag error, and how boosting differsMachine Learning
- Missing data: MCAR, MAR, MNAR and why multiple imputation existsStatistical Inference
- The 0/1 knapsack problem three ways: brute force, dynamic programming, greedyProgramming & Data
- Least squares by hand: normal equations, QR, and what standardized residuals really areStatistical Inference
2017
- Minimum, maximum and mixed phase wavelets, and why bandwidth sets resolutionSignal & Image Processing
- Seismic wavelets: amplitude spectrum, phase spectrum and phase shiftsSignal & Image Processing
- Waves and the Fourier spectrum: amplitude, frequency and phaseSignal & Image Processing
No notes match that search.