Large Language Models (LLMs)
Large Language Models (LLMs)
A large language model is a generator you can talk to, and building with one means knowing where it came from, which ones you can run, and how to check and constrain what it says. The page goes in that order: the founding papers, the models and the instruction step, a dataset, the tools and articles for building, then evaluation practices, guardrails, reinforcement learning, metrics, and use cases. A second part on GPT follows the same path for that one family.
The same notes are in GPT, GPT3 is ZERO, ONE, FEW, NEURAL LANGUAGE GENERATION, Prompt, and Unlearning.
Papers
The story starts with two papers: one showed that generative pre-training teaches language understanding, and the next showed that scaling that recipe makes a model few-shot. Language understanding by generative pre-training - Alec et al. openAI, is the first.
The figure above is from that paper. The second step is LLM are few shot learners - scaling LLMs with data is enough to make them few shot.
Models
Once scale was shown to work, open and commercial models followed, and the practical question became which one you can use. Databricks Dolly is the open, commercially usable line: Version 1.0 introduces Dolly as the first open-source, commercially viable instruction-tuned LLM, Version 2.0 is the "Hello Dolly" post on democratizing the magic of ChatGPT with open models, and the weights are on Huggingface. Meta's LLaMA is a foundational 65-billion-parameter model released under a gated release, described as more efficient than, and competitive with, published models of similar size. Google's chat model was Bard, which now opens as Google Gemini. StabilityLM is the repository of Stability AI's StableLM language models. Vicuna and LLaMA are also named on this list.
Instructor
A raw model predicts text; following an instruction is a separate training step. Training language models to follow instructions with human feedback (using RLHF) is the paper for that step. The same idea applied to embeddings is the Instructor model, billed as one embedder for all tasks:
We introduce Instructor👨🏫, an instruction-finetuned text embedding model that can generate text embeddings tailored to any task (e.g., classification, retrieval, clustering, text evaluation, etc.) and domains (e.g., science, finance, etc.) by simply providing the task instruction, without any finetuning. Instructor achieves sota on 70 diverse embedding tasks!
Datasets
Instruction tuning needs instruction data, and the Dolly models above came with theirs. Databricks' 15K QA for Dolly 2.0 is the data folder of the Dolly repository, the set used for the model trained on the Databricks Machine Learning Platform.
Tools
With a model and data in hand, the next layer is the libraries that turn a model into an application.
The same notes are in Agents and RAG.
Scikit-LLM integrates LLMs into scikit-learn. LangChain is the framework most of the other tools here orbit; its documentation now leads with create_agent, a minimal, configurable agent harness composed from a model, tools, a prompt, and middleware. To learn it, An amazing tutorial is the archived crash course on building applications powered by large language models, and Patrick Loeber's matching video, "LangChain Crash Course - Build apps with language models", is in Youtube. The course walks through LangChain's parts in this order:
LLMs
Prompt Templates
Chains
Agents and Tools
Memory
Document Loaders
Indexes
For a shorter start, Langchain in 13 minutes is Rabbitmetrics' quickstart for beginners. ReAct & LangChain explains ReAct (Reasoning + Acting), the LLM chain framework behind much advanced reasoning, and how LangChain composes it.
LangChain also grew a visual layer. LangFlow, Medium, HuggingFace - is a UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows. The Medium post, Rodrigo Nader's "Language Models On Steroids", frames it by the success of ChatGPT, with 100 million users in its first two months.
Other tools connect a model to your data. PandasAI - PandasAI, asking data Qs using LLMs on Panda's DFs with two code lines. 𝚙𝚊𝚗𝚍𝚊𝚜_𝚊𝚒 = 𝙿𝚊𝚗𝚍𝚊𝚜𝙰𝙸(𝚕𝚕𝚖) & 𝚙𝚊𝚗𝚍𝚊𝚜_𝚊𝚒.𝚛𝚞𝚗(𝚍𝚏, 𝚙𝚛𝚘𝚖𝚙𝚝='𝚆𝚑𝚒𝚌𝚑 𝚊𝚛𝚎 𝚝𝚑𝚎 𝟻 𝚑𝚊𝚙𝚙𝚒𝚎𝚜𝚝 𝚌𝚘𝚞𝚗𝚝𝚛𝚒𝚎𝚜?') For data outside a data frame, LLaMa Index - LlamaIndex (GPT Index) is a project that provides a central interface to connect your LLM's with external data. LLM-foundry - LLM training code for Databricks foundation models using MoasicML.
For finding more, Awesome ChatGPT - Curated list of awesome tools, demos, docs for ChatGPT and GPT-3 is humanloop's curated list. To run a model on your own machine, GPT4 All Privacy-oriented software for chatting with large language models that run on your own computer. is nomic-ai's GPT4All, open source and available for commercial use.
To understand the model by building one, MinGPT - A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training. NanoGPT - The simplest, fastest repository for training/finetuning medium-sized GPTs.
The same kind of model can also see. Open-sourced codes for MiniGPT-4 and MiniGPT-v2 is the Vision-CAIR repository with the code for both, and each has a project page (https://minigpt-4.github.io, https://minigpt-v2.github.io/): MiniGPT-4 at ( https://minigpt-4.github.io and MiniGPT-v2 at https://minigpt-v2.github.io/.
Articles
Tools answer how; the articles explain why the models behave the way they do and how to steer them.
The same notes are in Prompt.
GPT4 can improve itself is the video on that claim. Lil Weng - Prompt Engineering is Lilian Weng's post on prompt engineering, also called in-context prompting: methods for steering an LLM toward a desired outcome without updating its weights, an empirical science whose effects vary a lot between models and need heavy experimentation. Chip Huyen - Building LLM applicaations for production is her post on taking those applications to production. How to generate text using different decoding methods for language generation with transformers is the Hugging Face post on decoding methods, the step that turns probabilities into text. (great) a gentle intro to LLMs and Langchain is the introduction to chaining LLMs, agents, and utilities with LangChain. LLMs can explain NN of other LLMs is OpenAI's work on language models explaining the neurons of language models. Fine tuning LLMs is a walkthrough of customizing Llama 3 8B for your needs. Model size vs Computer overhead - The trade-off between model size and compute overhead and reveal there is significant room to reduce the compute-optimal model size with minimal compute overhead.
Best Practices
A built application still has to be measured, and for RAG applications the judge is often another LLM. Best Practices for LLM Evaluation of RAG Applications is the Databricks post on evaluating retrieval-augmented generation with LLMs. Announcing MLflow 2.8 LLM-as-a-judge metrics and Best Practices for LLM Evaluation of RAG Applications, Part 2 is the follow-up: MLflow 2.8 automates evaluation with LLM judges to save time and cost, and it shows that data cleaning improves RAG performance.
Guardrails
Evaluation tells you what went wrong afterward; guardrails stop it at run time. GuardrailsAI is the Guardrails Hub from Guardrails AI.
Safeguarding LLMs with Guardrails is Aparna Dhinakaran's article, co-authored with Hakan Tekgul.
On a managed platform, Databricks GR - Implementing LLM Guardrails for Safe and Responsible Generative AI Deployment on Databricks.
Reinforcement Learning for LLM
Guardrails filter outputs; reinforcement learning from human feedback changes the model so it produces better ones in the first place.
The same notes are in RLHF.
RLHF: Reinforcement Learning from Human Feedback is Chip Huyen's explanation. Yoav on RL is the rl-for-llms gist. John Schulman - Reinforcement Learning from Human Feedback: Progress and Challenges.
Metrics
Whatever trained the model, its text output still needs a score.
The same notes are in LANGUAGE TRANSLATION, Perplexity, and Summarization.
For summaries, Understanding ROUGE - a family of metrics that evaluate the performance of a LLM in text summarization, i.e., ROUGE-1, ROUGE-2, ROUGE-L, for unigrams, bi grams, LCS, respectively.
Use Cases
The last step is putting a model to work beyond its context window. Enhancing ChatGPT With Infinite External Memory Using Vector Database and ChatGPT Retrieval Plugin gives ChatGPT external memory through a vector database and the retrieval plugin.
Guardrails come back as a use case too: Wenqi Glantz's "NeMo Guardrails, the Ultimate Open-Source LLM Security Toolkit", filed here as Safeguarding LLMs with Guardrails, explores the practical use cases of NeMo Guardrails, at https://towardsdatascience.com/safeguarding-llms-with-guardrails-4f5d9f57cff2
Research and Reviewer Agents is the use case for agents that do scientific research, review a manuscript or a claim, or do both.
GPT
GPT is the family most of the above grew from, so it gets its own path: the methods that led to it, the embedding tools built on it, the articles that explain and compare its alignment, the hackathons, and a catalog of assistants.
The same notes are in Chat Bots, Decoding Algorithms For NLP, GPT2, GPT3, Large Language Models (LLMs), Prompt, and Tokenization.
Precursor
Instruction-following GPT came from three steps, each building on the last:
Proximal Policy Optimization (PPO) - an RL algorithm, PPO is better than state-of-the-art approaches while being much simpler to implement and tune and is the default reinforcement learning algorithm at OpenAI.
Learning from human preference (human in the loop) - a method used to infer what humans want by being told which of two proposed behaviors is better.
instructGPT - arguably better at following user intentions than GPT-3 while also making them more truthful and less toxic, using human in the loop.
Tools
Beyond chat, GPT models also produce sentence embeddings for search. sentence embedding for semantic search is SGPT, GPT sentence embeddings for semantic search. GPT 3 Dense sentence embeddings is Nils Reimers asking whether OpenAI's GPT-3 text embeddings are really a new state of the art in dense text embeddings.
Articles
With the tools named, the articles explain how the model works and how its alignment compares. what is chatGPT doing and why does it work? is Stephen Wolfram's broader picture. Karpathy on building GPT is Andrej Karpathy's "Let's build GPT: from scratch, in code, spelled out." On alignment, Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study - answers with the PPO precursor above:
PPO is able to surpass other alignment methods in all cases and achieve state-of-the-art results in challenging code competitions.
Competitions
Hackathons show what people actually build with it. The GPT 4 Hackathon code results and the LangChain Gen Hackathon sheets list the results.
Virtual assistants
The same builds end up as assistants people share. flowGPT is an open platform to discover prompts and AI apps built by a global community, or to publish your own.
Deprecated links
These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.
An amazing tutorial in Youtube by Patrick Loeber. This address no longer opens: https://www.python-engineer.com/posts/langchain-crash-course/
(great) a gentle intro to LLMs and Langchain. This address no longer opens: https://towardsdatascience.com/a-gentle-intro-to-chaining-llms-agents-and-utils-via-langchain-16cd385fca81
Last updated
Was this helpful?