A Byte of Coding Issue 354

G’day,

Another beautiful day on this spectacular green and blue marble we call home. Anyone else feel substantially better on sunny days? It’s been raining for the past week and it feels like a a weight has been lifted off of my shoulders this morning thanks to the sunshine and blue skies.

Anyway, here’s the issue.

One Billion Rows Challenge in Golang

Published: 19 February 2024

Tags: go, optimization

Shraddha Agrawal takes on a challenge that requires you to write a program that “reads the file [csv with one billion rows], calculates the min, mean, and max temperature value per weather station, and emits the results on stdout like this (i.e. sorted alphabetically by station name, and the result values per station in the format <min>/<mean>/<max>, rounded to one fractional digit)”.

Some highlights:

• using “Golang 1.21. All benchmarks are run on Apple M1 Pro, 2021 model with 16GB memory and 10 vCPU”

• “started with a rather unimpressive execution time of >6min and brought it down to 14s”

• includes optimizations for data structures, IO, parsing, and concurrency

Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch

Published: 18 February 2024

Tags: machine learning, ai, algorithms, python

Sebastian Raschka implements both LoRA and DoRA in PyTorch from scratch.

Some highlights:

• LoRA offers a more efficient alternative to computing weight updates for LLMs during training

• DoRA “can be seen as an improvement or extension of LoRA that is built on top of it”

• Full implementation and code available in the article

Gameplanning the SGE Apocalypse

Published: 19 February 2024

Tags: marketing, business, philosophy

Not really as technical as the typical articles that are featured in this newsletter, but still an interesting read (at least for me, because two of my businesses rely heavily on organic search). Erik Dietrich elucidates how to come out ahead if search engines start to replace traditional third-party information with their own AI generated responses.

Some highlights:

• trying to predict the future is a waste of time

• prioritize coming up with a plan that lets you come out ahead regardless of whatever search engines decide

• a number of different approaches on how to achieve the bullet point above