Jobs In Data logo

Jobs In Data

How to learn LLMs - a subjective roadmap

Updated on

In this article, I explore the pragmatic approach to getting into the field of Large Language Models from the perspective of an ML practitioner with most of my prior experience in tabular data. My ultimate objective was to train a useful model to predict a salary based on the job description, using the state-of-the-art technology available. Below, I present a collection of materials (ordered according to my thought-flow) that I gathered during my journey.

Summary - the curent GPT SOTA & Roadmap

Winning solutions' writeups in Kaggle NLP competitions 2022-2023

The free market - let the best model win. Thousands of programmers compete for fame on a reasonable-scale infrastructure. Models that win here are the true State of the Art (given the limitations of a single A100 GPU).

Papers on the most succesful models on Kaggle (the "free market") - working & cheap

Understanding the Foundations of Transformers

Understanding LLMs

  • Chatgpt
  • LLama - An open source and free for commercial applications model from Meta
  • Bard - Google response to ChatGPT powered by lamda

Other materials