Ollama: Simple Local Model Running

Quick Facts

Aspect	Details
Purpose	Simple local LLM running
Models	100+ (Llama, Mistral, Phi, Gemma, etc.)
Installation	Download from ollama.ai
Best For	Local development, demos
Hardware	Any GPU or CPU

Installation & Usage

# Download from https://ollama.ai
# Then run:

ollama pull llama2          # Download model
ollama run llama2           # Interactive chat
ollama serve                # Start API server on port 11434

API Usage

import requests
import json

response = requests.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "llama2",
        "prompt": "What is machine learning?",
        "stream": False,
    }
)

print(response.json()["response"])

Docker

docker run -d -v ollama:/root/.ollama -p 11434:11434 ollama/ollama

Strengths

✅ Simplicity - One command setup
✅ Accessibility - Runs on consumer hardware
✅ No coding - Built-in interactive mode

Weaknesses

❌ Speed - Slower than optimized inference
❌ Production - Limited scaling capabilities

---

Use when: You want the simplest possible local LLM setup.