Popular LLM Models: A Comprehensive Overview¶

Table of Contents¶

Organization-Specific Guides
Evolution of LLMs
Model Categories
Major LLM Families
Detailed Model Comparisons
Performance Benchmarks
Deployment Considerations
Technical References

Organization Guides¶

Navigate to organization-specific sections for detailed model information:

Evolution of LLMs¶

Timeline of Major Models¶

timeline
    title LLM Evolution Timeline (2017-2026)

    2017 : Transformer (Vaswani et al.)
         : BERT Released

    2018 : GPT Released
         : RoBERTa

    2019 : GPT-2 Released
         : T5 Released

    2020 : GPT-3 Released (175B)
         : DALL-E (Vision+Language)

    2021 : PaLM 540B
         : BLOOM 176B

    2022 : ChatGPT Released
         : Llama Released (Meta)
         : Claude (Anthropic)

    2023 : GPT-4 Released
         : Llama 2 (70B)
         : Gemini Released
         : Mixtral MoE Models

    2024 : Llama 3 (405B)
         : GPT-4 Turbo
         : Claude 3 Family

    2025 : Llama 3.1 (405B)
         : Reasoning Models
         : Multimodal Integration

    2026 : Advanced Reasoning
         : Specialized Domain Models
         : Real-time Reasoning

Model Categories¶

By Architecture¶

graph TD
    A[LLM Models] -->|Decoder-Only| B[GPT-style]
    A -->|Encoder-Decoder| C[T5-style]
    A -->|Encoder-Only| D[BERT-style]

    B --> B1["GPT-2/3/4<br/>LLaMA<br/>Mistral"]
    C --> C1["T5<br/>BART<br/>mT5"]
    D --> D1["BERT<br/>RoBERTa<br/>ELECTRA"]

    A -->|Mixture of Experts| E["Mixtral<br/>GShard"]
    A -->|Vision+Language| F["CLIP<br/>LLaVA<br/>Gemini"]

By Application¶

Category	Models	Use Cases
General Purpose	GPT-4, Claude 3, Llama 3	Reasoning, writing, coding
Code Generation	Codex, Code Llama, DeepSeek Coder	Programming, debugging
Domain-Specific	BloombergGPT, LawGPT	Finance, legal, medical
Multilingual	BLOOM, mT5, XLM-R	Cross-language tasks
Vision+Language	Gemini, GPT-4V, LLaVA	Image understanding
Quantized/Edge	Llama 2-7B, Mistral-7B	Mobile, on-device

Major LLM Families¶

1. OpenAI's GPT Series¶

GPT-3 (2020)¶

Size: 175 billion parameters
Training Data: 570GB of text (Common Crawl, WebText2, Books, Wikipedia)
Context Window: 2,048 tokens
Key Innovation: Few-shot learning without fine-tuning

# Example API usage
import openai

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing"}
    ],
    temperature=0.7,
    max_tokens=500
)

print(response.choices[0].message.content)

Architecture Highlights: - Transformer decoder-only - Tokenization: BPE with 50,257 tokens - Training: 300B tokens from diverse sources - No task-specific fine-tuning required (prompt engineering instead)

Paper: Language Models are Unsupervised Multitask Learners

GPT-4 (2023)¶

Architecture: Not publicly disclosed (estimated 1-2 trillion MoE parameters)
Context Window: 8K / 32K / 128K (extended versions)
Key Improvements:
Better reasoning and problem-solving
Improved instruction following
Reduced hallucinations
Multimodal (vision + text)

# GPT-4 with Vision
response = openai.ChatCompletion.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://example.com/image.jpg"}
                }
            ]
        }
    ]
)

Technical Advances: - Constitutional AI for alignment - Reinforcement Learning from Human Feedback (RLHF) - Mixture of Experts for efficient scaling

2. Google's Model Family¶

BERT (2018)¶

Size: 12/24 layers, 110M/340M parameters
Architecture: Encoder-only (bidirectional)
Context: 512 tokens
Training: Masked Language Modeling (MLM) + Next Sentence Prediction (NSP)

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained("bert-base-uncased")

text = "The capital of France is Paris."
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)

# Last hidden state
embeddings = outputs.last_hidden_state  # [1, seq_len, 768]

Applications: Text classification, NER, semantic similarity

Paper: BERT: Pre-training of Deep Bidirectional Transformers

T5 (2019)¶

Size: 60M - 11B parameters
Architecture: Encoder-decoder (Seq2Seq)
Training: Denoising autoencoder objective
Unique: Treats all NLP tasks as text-to-text

from transformers import T5ForConditionalGeneration, T5Tokenizer

model = T5ForConditionalGeneration.from_pretrained("t5-base")
tokenizer = T5Tokenizer.from_pretrained("t5-base")

# Translation
input_text = "translate English to German: The quick brown fox"
input_ids = tokenizer.encode(input_text, return_tensors="pt")
outputs = model.generate(input_ids, max_length=50)
translation = tokenizer.decode(outputs[0], skip_special_tokens=True)

Task Format: - "translate English to German: ..." - "summarize: ..." - "question: ... context: ..."

Paper: Exploring the Limits of Transfer Learning

PaLM (2022)¶

Size: 540 billion parameters
Decoder-only architecture
Training: 780 billion tokens
Key Achievement: Emergent reasoning abilities at scale

Gemini (2023-2024)¶

Sizes: Ultra (2T), Pro (variants), Nano
Multimodal: Native text, image, audio, video
Context: 200K tokens (1M in extended versions)
Training: Mixture of modalities from inception

# Gemini API Example
import google.generativeai as genai

genai.configure(api_key="YOUR_API_KEY")
model = genai.GenerativeModel('gemini-pro-vision')

response = model.generate_content([
    "Describe this image",
    genai.upload_file("path/to/image.jpg")
])

print(response.text)

Paper: Gemini: A Family of Highly Capable Multimodal Models

3. Meta's Llama Series¶

Llama 1 (2023)¶

Sizes: 7B, 13B, 33B, 65B
Architecture: Decoder-only transformer
Training: 1.4T tokens from diverse sources
License: Open weights (for research)

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "meta-llama/Llama-2-7b-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "def fibonacci(n):"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
code = tokenizer.decode(outputs[0])

Llama 2 (2023)¶

Sizes: 7B, 13B, 70B (added from Llama 1)
Key Improvements:
Chat-optimized versions
Better instruction following
Safe by design (RLHF alignment)
License: Open weights (commercial friendly)

# Llama 2 Chat Format
prompt = """<s>[INST] <<SYS>>
You are a helpful, respectful assistant.
<</SYS>>

How do I fine-tune a language model? [/INST]"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=500)

Llama 3 (2024)¶

Sizes: 8B, 70B, 405B (flagship)
Improvements:
Extended context (8K tokens)
Better multilingual support
Improved reasoning
Grouped Query Attention (GQA)
Training: 15.6T tokens (15.6x more than Llama 2)

# Llama 3 405B Performance
# - MMLU: 92.9% (state-of-the-art)
# - Math: Significant improvement
# - Coding: Near GPT-4 levels

Architecture Details: - Rotary positional embeddings (RoPE) - Grouped query attention (GQA) for efficiency - Vocabulary size: 128,256 tokens - Flash Attention v2 for efficiency

Paper: The Llama 3 Herd of Models

4. Anthropic's Claude¶

Claude 1 (2023)¶

Training: Constitutional AI (RLHF with principles)
Focus: Safety, harmlessness, reduced hallucinations

Claude 3 Family (2024)¶

Opus: Largest, most capable
Sonnet: Balanced performance/cost
Haiku: Fastest, lightest

import anthropic

client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-3-opus-20240229",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Explain attention mechanisms in transformers"
        }
    ]
)

print(message.content[0].text)

Architecture Innovations: - Training-time constitution alignment - Extended context windows (200K tokens) - Better long-document reasoning - Reduced hallucinations through training approach

Key Benchmark Results:

Claude 3 Opus:
- MMLU: 88.7%
- HumanEval: 92%
- Math: Strong reasoning capabilities

5. Other Notable Models¶

Mixtral 8x7B/8x22B (2023-2024)¶

Architecture: Mixture of Experts (8 experts, 2 active)
Efficiency: 12B effective parameters, 8x7B physical
License: Open weights (Mistral AI)

# Mixtral routing example
# For each token, router selects top-2 experts from 8
# This allows model scaling without dense computation

# Effective parameters: 12.9B
# Throughput: Better than dense 70B

Characteristics: - Sparse activation (only 2/8 experts per token) - Better latency than dense equivalents - Competitive with much larger models

Code Llama (2023)¶

Variant of Llama 2 optimized for programming
Sizes: 7B, 13B, 34B
Training: 500B tokens from code datasets
HumanEval Score: 84.3% (34B)

# Code Llama Usage
prompt = "# Binary search implementation in Python"
# Model generates complete implementation

DeepSeek (2024)¶

Sizes: 7B, 67B
Focus: Cost-effective, high performance
MoE Variant: 16K experts with high sparsity
Notable: Sometimes outperforms larger models on benchmarks

Detailed Model Comparisons¶

Performance Matrix¶

graph TD
    A["LLM Performance Dimensions"] -->|Raw Capability| B["Reasoning"]
    A -->|Efficiency| C["Speed/Cost"]
    A -->|Specialization| D["Domain Expertise"]
    A -->|Alignment| E["Safety/Control"]

    B --> B1["GPT-4: Expert"]
    B --> B2["Claude Opus: Expert"]
    B --> B3["Llama 3 405B: Near-Expert"]

    C --> C1["Llama 3 8B: Excellent"]
    C --> C2["Mistral 7B: Excellent"]
    C --> C3["Mixtral 8x7B: Good"]

    D --> D1["Code Llama: Coding"]
    D --> D2["BloombergGPT: Finance"]
    D --> D3["LawGPT: Legal"]

    E --> E1["Claude 3: Strong"]
    E --> E2["Llama 2-Chat: Strong"]
    E --> E3["GPT-4: Strong"]

Benchmark Comparison¶

Model	MMLU	HumanEval	GSM8K	TruthfulQA	Size	Context
GPT-4	93.7%	92.0%	92%	59%	~2T MoE	128K
Claude 3 Opus	88.7%	92.0%	95%	63%	~200B	200K
Llama 3 405B	92.9%	90.2%	96%	58%	405B	8K
Llama 2 70B	82.9%	81.8%	56%	50%	70B	4K
Gemini Pro	71.8%	74.4%	46%	-	~1B	32K
Mistral 7B	60.0%	73.2%	28%	42%	7B	8K
Code Llama 34B	59.3%	84.3%	63%	-	34B	4K

Token Efficiency¶

# Tokens needed for common tasks

# MMLU Question (average)
tokens_needed = {
    "input": 400,        # Question + options
    "output": 50,        # Answer + reasoning
    "total": 450
}

# Code Generation (average)
code_task = {
    "input": 200,        # Prompt + context
    "output": 300,       # Generated code
    "total": 500
}

# Translation (average)
translation = {
    "input": 150,        # Source text
    "output": 160,       # Target text (varies by lang pair)
    "total": 310
}

# Long-form Content (per 1K words)
content = {
    "input": 100,        # Instructions
    "output": 1500,      # ~1K words generated
    "total": 1600
}

Performance Benchmarks¶

Inference Speed Comparison¶

xychart-beta
    title Inference Latency (ms per token) - Single GPU
    x-axis [GPT-4, Claude 3, Llama 405B, Llama 70B, Llama 13B, Mistral 7B]
    y-axis "Latency (ms/token)" 10 --> 200
    line [150, 140, 120, 45, 25, 15]

Cost-Performance Analysis¶

# Cost per 1M input tokens (approximate pricing 2024)

pricing = {
    "GPT-4": {
        "input": 30.0,
        "output": 60.0,
        "mmlu": 93.7,
        "cost_per_point": 0.032
    },
    "Claude 3 Opus": {
        "input": 15.0,
        "output": 75.0,
        "mmlu": 88.7,
        "cost_per_point": 0.096
    },
    "Llama 3 70B (Groq)": {
        "input": 0.59,
        "output": 0.79,
        "mmlu": 82.9,
        "cost_per_point": 0.007
    },
    "Mistral 7B (local)": {
        "input": 0.0,      # Self-hosted
        "output": 0.0,
        "mmlu": 60.0,
        "cost_per_point": 0.0
    }
}

# For many applications, Llama 3 70B or Mistral 7B
# provide best cost-performance balance

Reasoning Capability Progression¶

graph LR
    A["BERT<br/>Shallow NLU"] --> B["GPT-2<br/>Basic Generation"]
    B --> C["GPT-3<br/>Few-shot Learning"]
    C --> D["GPT-4<br/>Chain-of-thought Reasoning"]
    D --> E["Extended Reasoning<br/>o1-style Models"]

    style A fill:#ff9999
    style E fill:#99ff99

Deployment Considerations¶

Model Selection Decision Tree¶

graph TD
    A{Choose Model} -->|Budget: $$$<br/>Complex Reasoning| B["GPT-4 / Claude 3 Opus"]
    A -->|Budget: $$<br/>Good Performance| C["Claude 3 Sonnet<br/>Llama 70B"]
    A -->|Budget: $<br/>Speed Critical| D["Mistral 7B<br/>Llama 13B"]
    A -->|Budget: Free<br/>Self-hosted| E["Llama 3<br/>Open Source"]

    B --> B1["API Calls<br/>Batch Processing"]
    C --> C1["vLLM/TensorRT-LLM<br/>Self-hosted or API"]
    D --> D1["Edge Deployment<br/>Mobile/IoT"]
    E --> E1["Full Control<br/>Custom Training"]

Deployment Patterns¶

# Pattern 1: API-Based (GPT-4, Claude)
import openai

def api_inference(prompt):
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7
    )
    return response.choices[0].message.content

# Pattern 2: Local Inference (Llama, Mistral)
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="meta-llama/Llama-2-7b-hf",
    device=0  # GPU
)

def local_inference(prompt):
    return pipe(prompt, max_new_tokens=100)[0]["generated_text"]

# Pattern 3: Optimized Inference (vLLM)
from vllm import LLM, SamplingParams

llm = LLM(model="meta-llama/Llama-2-70b-hf")
sampling_params = SamplingParams(temperature=0.7, top_p=0.95)

def optimized_inference(prompts):
    outputs = llm.generate(prompts, sampling_params)
    return [output.outputs[0].text for output in outputs]

# Pattern 4: Edge Deployment (ONNX, Quantized)
import onnxruntime as ort

session = ort.InferenceSession(
    "llama-7b-quantized.onnx",
    providers=['CPUExecutionProvider']
)

def edge_inference(tokens):
    return session.run(None, {"input_ids": tokens})

Technical References¶

Key Papers¶

Foundational¶

Attention Is All You Need - Vaswani et al. (2017)
Introduces Transformer architecture
Foundation for all modern LLMs
BERT: Pre-training of Deep Bidirectional Transformers - Devlin et al. (2018)
Masked language modeling
Encoder-only architecture
Language Models are Unsupervised Multitask Learners - Radford et al. (2019)
GPT-2: 1.5B parameters
Few-shot learning without fine-tuning

Scale and Emergence¶

Language Models are Few-Shot Learners - Brown et al. (2020)
GPT-3: 175B parameters
Demonstrates in-context learning at scale
Scaling Laws for Neural Language Models - Kaplan et al. (2020)
Power-law relationships between scale and performance
Predicts model size/training compute tradeoffs
Emergent Abilities of Large Language Models - Wei et al. (2022)
Unexpected capabilities emerging at scale
Chain-of-thought prompting breakthrough

Optimization and Efficiency¶

The Llama 3 Herd of Models - Meta AI (2024)
405B model architecture and training
Grouped Query Attention improvements
Comprehensive evaluation
Mixtral of Experts - Mistral AI (2024)
Sparse Mixture of Experts
Efficient scaling without dense computation
LLaMA: Open and Efficient Foundation Language Models - Meta AI (2023)
Llama series introduction
Training efficiency improvements
Open weights commitment

Alignment and Safety¶

Constitutional AI: Harmlessness from AI Feedback - Bai et al. (2022)
Claude's training approach
Constitution-based alignment
Reduced human annotation needs
Training a Helpful and Harmless Assistant with RLHF - Christiano et al. (2019)
RLHF fundamentals
Preference learning

Multimodal¶

GPT-4V(ision) System Card
Vision integration with language models
Capabilities and limitations
Gemini: A Family of Highly Capable Multimodal Models - Gemini Team (2023)
Native multimodal (text, image, audio, video)
Training from inception with all modalities

Online Resources¶

Documentation and Guides¶

Hugging Face Model Hub - Model cards, weights, configs
LMSYS Chatbot Arena - Real-world model comparison
Ollama - Run models locally
vLLM Documentation - Efficient inference engine

Benchmarking¶

OpenCompass Leaderboard - Comprehensive model evaluation
BigCode Leaderboard - Code generation evaluation
HELM - Stanford's holistic evaluation

Community¶

r/LocalLLaMA - Local model discussion
Papers with Code - Implementation references
Hugging Face Forums - Community support

Model Selection Guide¶

By Use Case¶

1. Production Chat Applications¶

Best: GPT-4, Claude 3 Opus
Alternative: Llama 70B
Consideration: Cost vs. quality tradeoff

2. Code Generation¶

Best: GPT-4, Code Llama 34B
Alternative: Claude 3 Sonnet
Consideration: Llama 70B also strong for code

3. Document Processing¶

Best: Claude 3 Sonnet (200K context)
Alternative: Llama 70B or GPT-4 (128K context)
Consideration: Mixture of Experts for cost efficiency

4. Real-time Interactive Systems¶

Best: Mistral 7B, Llama 13B
Alternative: Gemini Nano (edge deployment)
Consideration: Quantization for mobile/IoT

5. Specialized Tasks (Domain-Specific)¶

Best: Fine-tuned Llama 7B/13B
Alternative: Domain-specific models (BloombergGPT, etc.)
Consideration: RAG + smaller model often better than raw LLM

6. Cost-Optimized Production¶

Best: Llama 70B (self-hosted)
Alternative: Mixtral 8x22B
Consideration: Batch inference, quantization

Key Takeaways¶

Model Hierarchy (2026):
Frontier: GPT-4, Claude 3 Opus (premium reasoning)
Strong Open: Llama 3 405B, Claude 3 Sonnet (balance)
Efficient: Llama 70B, Mixtral 8x22B (cost-effective)
Edge: Llama 7B/13B, Mistral 7B (mobile/IoT)
Trends:
Open models closing the gap with closed models
Mixture of Experts gaining adoption (efficiency)
Extended context windows (200K+ tokens common)
Specialized variants for domain-specific tasks
Reasoning models emerging (o1-style)
Practical Advice:
Start with API access (GPT-4, Claude) for prototyping
Migrate to open models (Llama, Mistral) for production
Use RAG + smaller model instead of larger LLM when possible
Quantize models for edge deployment
Fine-tune specific models for your domain
Future Outlook:
Reasoning abilities will become mainstream
Multimodal LLMs standard across all tiers
On-device inference improvements (better quantization, distillation)
Specialized models for vertical markets
Cost reduction through efficiency improvements