Ollama

Quick Facts

Aspect	Details
Purpose	Simple local LLM running
Models	100+ (Llama, Mistral, Phi, Gemma, etc.)
Installation	Download from ollama.ai
Best For	Local development, demos
Hardware	Any GPU or CPU

Installation & Usage

# Download from https://ollama.ai
# Then run:

ollama pull llama2 # Download model
ollama run llama2 # Interactive chat
ollama serve # Start API server on port 11434

API Usage

import requests
import json

response = requests.post(
 "http://localhost:11434/api/generate",
 json={
 "model": "llama2",
 "prompt": "What is machine learning?",
 "stream": False,
 }
)

print(response.json()["response"])

Docker

docker run -d -v ollama:/root/.ollama -p 11434:11434 ollama/ollama

Strengths

Simplicity - One command setup Accessibility - Runs on consumer hardware No coding - Built-in interactive mode

Weaknesses

Speed - Slower than optimized inference Production - Limited scaling capabilities

-

Use when: You want the simplest possible local LLM setup.