Skip to content

Popular LLM Models

Table of Contents

  1. Organization-Specific Guides
  2. Evolution of LLMs
  3. Model Categories
  4. Major LLM Families
  5. Detailed Model Comparisons
  6. Performance Benchmarks
  7. Deployment Considerations
  8. Technical References

-

Organization Guides

Navigate to organization-specific sections for detailed model information:

OpenAI

Openai Models - GPT-3, GPT-3.5, GPT-4, DALL-E

Anthropic

Anthropic Models - Claude Series

Google

Google Models - BERT, T5, PaLM, Gemini

Meta (Facebook)

Meta Models - Llama Series

Mistral AI

Mistral Models - Efficient & MoE

DeepSeek

Deepseek Models - Chinese & General

Kimi (Moon Shot)

Kimi Models - Chinese Focus

Evolution of LLMs

Timeline of Major Models

timeline
 title LLM Evolution Timeline (2017-2026)

 2017: Transformer (Vaswani et al.)
: BERT Released

 2018: GPT Released
: RoBERTa

 2019: GPT-2 Released
: T5 Released

 2020: GPT-3 Released (175B)
: DALL-E (Vision+Language)

 2021: PaLM 540B
: BLOOM 176B

 2022: ChatGPT Released
: Llama Released (Meta)
: Claude (Anthropic)

 2023: GPT-4 Released
: Llama 2 (70B)
: Gemini Released
: Mixtral MoE Models

 2024: Llama 3 (405B)
: GPT-4 Turbo
: Claude 3 Family

 2025: Llama 3.1 (405B)
: Reasoning Models
: Multimodal Integration

 2026: Advanced Reasoning
: Specialized Domain Models
: Real-time Reasoning

-

Model Categories

By Architecture

graph TD
 A[LLM Models] -->|Decoder-Only| B[GPT-style]
 A -->|Encoder-Decoder| C[T5-style]
 A -->|Encoder-Only| D[BERT-style]

 B --> B1["GPT-2/3/4<br/>LLaMA<br/>Mistral"]
 C --> C1["T5<br/>BART<br/>mT5"]
 D --> D1["BERT<br/>RoBERTa<br/>ELECTRA"]

 A -->|Mixture of Experts| E["Mixtral<br/>GShard"]
 A -->|Vision+Language| F["CLIP<br/>LLaVA<br/>Gemini"]

By Application

Category Models Use Cases
General Purpose GPT-4, Claude 3, Llama 3 Reasoning, writing, coding
Code Generation Codex, Code Llama, DeepSeek Coder Programming, debugging
Domain-Specific BloombergGPT, LawGPT Finance, legal, medical
Multilingual BLOOM, mT5, XLM-R Cross-language tasks
Vision+Language Gemini, GPT-4V, LLaVA Image understanding
Quantized/Edge Llama 2-7B, Mistral-7B Mobile, on-device

-

Major LLM Families

1. OpenAI's GPT Series

GPT-3 (2020)

  • Size: 175 billion parameters
  • Training Data: 570GB of text (Common Crawl, WebText2, Books, Wikipedia)
  • Context Window: 2,048 tokens
  • Key Innovation: Few-shot learning without fine-tuning
# Example API usage
import openai

response = openai.ChatCompletion.create(
 model="gpt-3.5-turbo",
 messages=[
 {"role": "system", "content": "You are a helpful assistant."},
 {"role": "user", "content": "Explain quantum computing"}
],
 temperature=0.7,
 max_tokens=500
)

print(response.choices[0].message.content)

Architecture Highlights:

  • Transformer decoder-only
  • Tokenization: BPE with 50,257 tokens
  • Training: 300B tokens from diverse sources
  • No task-specific fine-tuning required (prompt engineering instead)

Paper: Language Models are Unsupervised Multitask Learners

GPT-4 (2023)

  • Architecture: Not publicly disclosed (estimated 1-2 trillion MoE parameters)
  • Context Window: 8K / 32K / 128K (extended versions)
  • Key Improvements:
  • Better reasoning and problem-solving
  • Improved instruction following
  • Reduced hallucinations
  • Multimodal (vision + text)
# GPT-4 with Vision
response = openai.ChatCompletion.create(
 model="gpt-4-vision-preview",
 messages=[
 {
 "role": "user",
 "content": [
 {"type": "text", "text": "What's in this image?"},
 {
 "type": "image_url",
 "image_url": {"url": "https://example.com/image.jpg"}
 }
]
 }
]
)

Technical Advances:

  • Constitutional AI for alignment
  • Reinforcement Learning from Human Feedback (RLHF)
  • Mixture of Experts for efficient scaling

-

2. Google's Model Family

BERT (2018)

  • Size: 12/24 layers, 110M/340M parameters
  • Architecture: Encoder-only (bidirectional)
  • Context: 512 tokens
  • Training: Masked Language Modeling (MLM) + Next Sentence Prediction (NSP)
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained("bert-base-uncased")

text = "The capital of France is Paris."
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)

# Last hidden state
embeddings = outputs.last_hidden_state # [1, seq_len, 768]

Applications: Text classification, NER, semantic similarity

Paper: BERT: Pre-training of Deep Bidirectional Transformers

T5 (2019)

  • Size: 60M - 11B parameters
  • Architecture: Encoder-decoder (Seq2Seq)
  • Training: Denoising autoencoder objective
  • Unique: Treats all NLP tasks as text-to-text
from transformers import T5ForConditionalGeneration, T5Tokenizer

model = T5ForConditionalGeneration.from_pretrained("t5-base")
tokenizer = T5Tokenizer.from_pretrained("t5-base")

# Translation
input_text = "translate English to German: The quick brown fox"
input_ids = tokenizer.encode(input_text, return_tensors="pt")
outputs = model.generate(input_ids, max_length=50)
translation = tokenizer.decode(outputs[0], skip_special_tokens=True)

Task Format:

  • "translate English to German:..."
  • "summarize:..."
  • "question:... context:..."

Paper: Exploring the Limits of Transfer Learning

PaLM (2022)

  • Size: 540 billion parameters
  • Decoder-only architecture
  • Training: 780 billion tokens
  • Key Achievement: Emergent reasoning abilities at scale

Gemini (2023-2024)

  • Sizes: Ultra (2T), Pro (variants), Nano
  • Multimodal: Native text, image, audio, video
  • Context: 200K tokens (1M in extended versions)
  • Training: Mixture of modalities from inception
# Gemini API Example
import google.generativeai as genai

genai.configure(api_key="YOUR_API_KEY")
model = genai.GenerativeModel('gemini-pro-vision')

response = model.generate_content([
 "Describe this image",
 genai.upload_file("path/to/image.jpg")
])

print(response.text)

Paper: Gemini: A Family of Highly Capable Multimodal Models

-

3. Meta's Llama Series

Llama 1 (2023)

  • Sizes: 7B, 13B, 33B, 65B
  • Architecture: Decoder-only transformer
  • Training: 1.4T tokens from diverse sources
  • License: Open weights (for research)
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "meta-llama/Llama-2-7b-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "def fibonacci(n):"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
code = tokenizer.decode(outputs[0])

Llama 2 (2023)

  • Sizes: 7B, 13B, 70B (added from Llama 1)
  • Key Improvements:
  • Chat-optimized versions
  • Better instruction following
  • Safe by design (RLHF alignment)
  • License: Open weights (commercial friendly)
# Llama 2 Chat Format
prompt = """<s>[INST] <<SYS>>
You are a helpful, respectful assistant.
<</SYS>>

How do I fine-tune a language model? [/INST]"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=500)

Llama 3 (2024)

  • Sizes: 8B, 70B, 405B (flagship)
  • Improvements:
  • Extended context (8K tokens)
  • Better multilingual support
  • Improved reasoning
  • Grouped Query Attention (GQA)
  • Training: 15.6T tokens (15.6x more than Llama 2)
# Llama 3 405B Performance
# - MMLU
# - Math
# - Coding

Architecture Details:

  • Rotary positional embeddings (RoPE)
  • Grouped query attention (GQA) for efficiency
  • Vocabulary size: 128,256 tokens
  • Flash Attention v2 for efficiency

Paper: The Llama 3 Herd of Models

-

4. Anthropic's Claude

Claude 1 (2023)

  • Training: Constitutional AI (RLHF with principles)
  • Focus: Safety, harmlessness, reduced hallucinations

Claude 3 Family (2024)

  • Opus: Largest, most capable
  • Sonnet: Balanced performance/cost
  • Haiku: Fastest, lightest
import anthropic

client = anthropic.Anthropic()
message = client.messages.create(
 model="claude-3-opus-20240229",
 max_tokens=1024,
 messages=[
 {
 "role": "user",
 "content": "Explain attention mechanisms in transformers"
 }
]
)

print(message.content[0].text)

Architecture Innovations:

  • Training-time constitution alignment
  • Extended context windows (200K tokens)
  • Better long-document reasoning
  • Reduced hallucinations through training approach

Key Benchmark Results:

Claude 3 Opus:
- MMLU: 88.7%
- HumanEval: 92%
- Math: Strong reasoning capabilities

-

5. Other Notable Models

Mixtral 8x7B/8x22B (2023-2024)

  • Architecture: Mixture of Experts (8 experts, 2 active)
  • Efficiency: 12B effective parameters, 8x7B physical
  • License: Open weights (Mistral AI)
# Mixtral routing example
# For each token, router selects top-2 experts from 8
# This allows model scaling without dense computation

# Effective parameters
# Throughput

Characteristics:

  • Sparse activation (only 2/8 experts per token)
  • Better latency than dense equivalents
  • Competitive with much larger models

Code Llama (2023)

  • Variant of Llama 2 optimized for programming
  • Sizes: 7B, 13B, 34B
  • Training: 500B tokens from code datasets
  • HumanEval Score: 84.3% (34B)
# Code Llama Usage
prompt = "# Binary search implementation in Python"
# Model generates complete implementation

DeepSeek (2024)

  • Sizes: 7B, 67B
  • Focus: Cost-effective, high performance
  • MoE Variant: 16K experts with high sparsity
  • Notable: Sometimes outperforms larger models on benchmarks

-

Detailed Model Comparisons

Performance Matrix

graph TD
 A["LLM Performance Dimensions"] -->|Raw Capability| B["Reasoning"]
 A -->|Efficiency| C["Speed/Cost"]
 A -->|Specialization| D["Domain Expertise"]
 A -->|Alignment| E["Safety/Control"]

 B --> B1["GPT-4: Expert"]
 B --> B2["Claude Opus: Expert"]
 B --> B3["Llama 3 405B: Near-Expert"]

 C --> C1["Llama 3 8B: Excellent"]
 C --> C2["Mistral 7B: Excellent"]
 C --> C3["Mixtral 8x7B: Good"]

 D --> D1["Code Llama: Coding"]
 D --> D2["BloombergGPT: Finance"]
 D --> D3["LawGPT: Legal"]

 E --> E1["Claude 3: Strong"]
 E --> E2["Llama 2-Chat: Strong"]
 E --> E3["GPT-4: Strong"]

Benchmark Comparison

Model MMLU HumanEval GSM8K TruthfulQA Size Context
GPT-4 93.7% 92.0% 92% 59% ~2T MoE 128K
Claude 3 Opus 88.7% 92.0% 95% 63% ~200B 200K
Llama 3 405B 92.9% 90.2% 96% 58% 405B 8K
Llama 2 70B 82.9% 81.8% 56% 50% 70B 4K
Gemini Pro 71.8% 74.4% 46% - ~1B 32K
Mistral 7B 60.0% 73.2% 28% 42% 7B 8K
Code Llama 34B 59.3% 84.3% 63% - 34B 4K

Token Efficiency

# Tokens needed for common tasks

# MMLU Question (average)
tokens_needed = {
 "input": 400, # Question + options
 "output": 50, # Answer + reasoning
 "total": 450
}

# Code Generation (average)
code_task = {
 "input": 200, # Prompt + context
 "output": 300, # Generated code
 "total": 500
}

# Translation (average)
translation = {
 "input": 150, # Source text
 "output": 160, # Target text (varies by lang pair)
 "total": 310
}

# Long-form Content (per 1K words)
content = {
 "input": 100, # Instructions
 "output": 1500, # ~1K words generated
 "total": 1600
}

-

Performance Benchmarks

Inference Speed Comparison

xychart-beta
 title Inference Latency (ms per token) - Single GPU
 x-axis [GPT-4, Claude 3, Llama 405B, Llama 70B, Llama 13B, Mistral 7B]
 y-axis "Latency (ms/token)" 10 --> 200
 line [150, 140, 120, 45, 25, 15]

Cost-Performance Analysis

# Cost per 1M input tokens (approximate pricing 2024)

pricing = {
 "GPT-4": {
 "input": 30.0,
 "output": 60.0,
 "mmlu": 93.7,
 "cost_per_point": 0.032
 },
 "Claude 3 Opus": {
 "input": 15.0,
 "output": 75.0,
 "mmlu": 88.7,
 "cost_per_point": 0.096
 },
 "Llama 3 70B (Groq)": {
 "input": 0.59,
 "output": 0.79,
 "mmlu": 82.9,
 "cost_per_point": 0.007
 },
 "Mistral 7B (local)": {
 "input": 0.0, # Self-hosted
 "output": 0.0,
 "mmlu": 60.0,
 "cost_per_point": 0.0
 }
}

# For many applications, Llama 3 70B or Mistral 7B
# provide best cost-performance balance

Reasoning Capability Progression

graph LR
 A["BERT<br/>Shallow NLU"] --> B["GPT-2<br/>Basic Generation"]
 B --> C["GPT-3<br/>Few-shot Learning"]
 C --> D["GPT-4<br/>Chain-of-thought Reasoning"]
 D --> E["Extended Reasoning<br/>o1-style Models"]

 style A fill:#ff9999
 style E fill:#99ff99

-

Deployment Considerations

Model Selection Decision Tree

graph TD
 A{Choose Model} -->|Budget: $$$<br/>Complex Reasoning| B["GPT-4 / Claude 3 Opus"]
 A -->|Budget: $$<br/>Good Performance| C["Claude 3 Sonnet<br/>Llama 70B"]
 A -->|Budget: $<br/>Speed Critical| D["Mistral 7B<br/>Llama 13B"]
 A -->|Budget: Free<br/>Self-hosted| E["Llama 3<br/>Open Source"]

 B --> B1["API Calls<br/>Batch Processing"]
 C --> C1["vLLM/TensorRT-LLM<br/>Self-hosted or API"]
 D --> D1["Edge Deployment<br/>Mobile/IoT"]
 E --> E1["Full Control<br/>Custom Training"]

Deployment Patterns

# Pattern 1
import openai

def api_inference(prompt):
 response = openai.ChatCompletion.create(
 model="gpt-4",
 messages=[{"role": "user", "content": prompt}],
 temperature=0.7
)
 return response.choices[0].message.content

# Pattern 2
from transformers import pipeline

pipe = pipeline(
 "text-generation",
 model="meta-llama/Llama-2-7b-hf",
 device=0 # GPU
)

def local_inference(prompt):
 return pipe(prompt, max_new_tokens=100)[0]["generated_text"]

# Pattern 3
from vllm import LLM, SamplingParams

llm = LLM(model="meta-llama/Llama-2-70b-hf")
sampling_params = SamplingParams(temperature=0.7, top_p=0.95)

def optimized_inference(prompts):
 outputs = llm.generate(prompts, sampling_params)
 return [output.outputs[0].text for output in outputs]

# Pattern 4
import onnxruntime as ort

session = ort.InferenceSession(
 "llama-7b-quantized.onnx",
 providers=['CPUExecutionProvider']
)

def edge_inference(tokens):
 return session.run(None, {"input_ids": tokens})

-

Technical References

Key Papers

Foundational

Scale and Emergence

Optimization and Efficiency

Alignment and Safety

Multimodal

Online Resources

Documentation and Guides

Benchmarking

Community

-

Model Selection Guide

By Use Case

1. Production Chat Applications

Best: GPT-4, Claude 3 Opus
Alternative: Llama 70B
Consideration: Cost vs. quality tradeoff

2. Code Generation

Best: GPT-4, Code Llama 34B
Alternative: Claude 3 Sonnet
Consideration: Llama 70B also strong for code

3. Document Processing

Best: Claude 3 Sonnet (200K context)
Alternative: Llama 70B or GPT-4 (128K context)
Consideration: Mixture of Experts for cost efficiency

4. Real-time Interactive Systems

Best: Mistral 7B, Llama 13B
Alternative: Gemini Nano (edge deployment)
Consideration: Quantization for mobile/IoT

5. Specialized Tasks (Domain-Specific)

Best: Fine-tuned Llama 7B/13B
Alternative: Domain-specific models (BloombergGPT, etc.)
Consideration: RAG + smaller model often better than raw LLM

6. Cost-Optimized Production

Best: Llama 70B (self-hosted)
Alternative: Mixtral 8x22B
Consideration: Batch inference, quantization

-

Key Takeaways

  1. Model Hierarchy (2026):

  2. Frontier: GPT-4, Claude 3 Opus (premium reasoning)

  3. Strong Open: Llama 3 405B, Claude 3 Sonnet (balance)
  4. Efficient: Llama 70B, Mixtral 8x22B (cost-effective)

  5. Edge: Llama 7B/13B, Mistral 7B (mobile/IoT)

  6. Trends:

  7. Open models closing the gap with closed models

  8. Mixture of Experts gaining adoption (efficiency)
  9. Extended context windows (200K+ tokens common)
  10. Specialized variants for domain-specific tasks

  11. Reasoning models emerging (o1-style)

  12. Practical Advice:

  13. Start with API access (GPT-4, Claude) for prototyping

  14. Migrate to open models (Llama, Mistral) for production
  15. Use RAG + smaller model instead of larger LLM when possible
  16. Quantize models for edge deployment

  17. Fine-tune specific models for your domain

  18. Future Outlook:

  19. Reasoning abilities will become mainstream

  20. Multimodal LLMs standard across all tiers
  21. On-device inference improvements (better quantization, distillation)
  22. Specialized models for vertical markets
  23. Cost reduction through efficiency improvements