Skip to content

Packages & Organization: Building Professional Python Projects

Overview

Package structure determines project scalability: - Modules: Single .py files - Packages: Directories with __init__.py - Namespace packages: PEP 420 implicit packages - Best practices: Professional organization

Project Structure

Standard Project Layout

myproject/
  - pyproject.toml           # Modern project config
  - setup.py                 # Legacy setup
  - README.md
  - LICENSE
  - docs/
    - conf.py             # Sphinx config
    - index.rst
  - src/
    - myproject/
    - __init__.py
    - __main__.py      # CLI entry point
    - core.py
    - utils/
      - __init__.py
      - math.py
      - io.py
    - models/
    - __init__.py
    - neural_net.py
  - tests/
    - __init__.py
    - conftest.py         # Pytest config
    - test_core.py
  - requirements.txt        # Dependencies

init.py Patterns

Minimal init.py

# myproject/__init__.py
"""Package docstring."""

__version__ = "0.1.0"
__author__ = "Your Name"

Selective Exports

# myproject/__init__.py
"""MyProject - A powerful ML library."""

from .core import Model, Trainer
from .utils import load_data, preprocess

__version__ = "0.1.0"
__all__ = ['Model', 'Trainer', 'load_data', 'preprocess']

# Hide internals
# _internal is not exported with "from myproject import *"

Lazy Imports

# myproject/__init__.py
"""Lazy import to reduce startup time."""

def __getattr__(name):
    """Lazy import on attribute access."""
    if name == 'heavy_module':
        from . import heavy_module
        return heavy_module

    raise AttributeError(f"Module has no attribute {name}")

# Usage: myproject.heavy_module  # Imported only when accessed

Import Styles

Absolute Imports

# In myproject/models/neural_net.py
from myproject.core import Model      # Absolute import
from myproject.utils.math import sqrt # Absolute import

class NeuralNet(Model):
    pass

Relative Imports

# In myproject/models/neural_net.py
from ..core import Model              # Relative import
from ..utils.math import sqrt         # Relative import
from .layers import Dense             # Same-package import

class NeuralNet(Model):
    pass

Conditional Imports

# myproject/core.py
"""Core module with optional dependencies."""

try:
    import torch
    HAS_TORCH = True
except ImportError:
    HAS_TORCH = False

try:
    import tensorflow as tf
    HAS_TF = True
except ImportError:
    HAS_TF = False

def get_backend():
    if HAS_TORCH:
        return 'torch'
    elif HAS_TF:
        return 'tensorflow'
    else:
        return 'numpy'

Namespace Packages (PEP 420)

Implicit Namespace Packages

# No __init__.py needed!
mycompany/
  - ml/
    - models.py          # No __init__.py
  - data/
    - loader.py          # No __init__.py
  - web/
  - api.py             # No __init__.py

# All are part of 'mycompany' namespace
from mycompany.ml import models
from mycompany.data import loader
from mycompany.web import api

Entry Points

Command-line Entry Point

# pyproject.toml
[project.scripts]
myproject-cli = "myproject.cli:main"

# setup.py (legacy)
setup(
    entry_points={
        'console_scripts': [
            'myproject-cli=myproject.cli:main',
        ],
    }
)

# myproject/cli.py
def main():
    import sys
    # CLI implementation
    pass

if __name__ == '__main__':
    main()

Plugin System

# pyproject.toml
[project.entry-points."myproject.plugins"]
my_plugin = "my_plugin:Plugin"

# Discover plugins at runtime
def load_plugins():
    from importlib.metadata import entry_points

    eps = entry_points(group='myproject.plugins')
    plugins = {}

    for ep in eps:
        plugins[ep.name] = ep.load()

    return plugins

Best Practices

Do's

✓ Use relative imports within packages
✓ Organize by functionality, not file type
✓ Limit import depth (max 3 levels)
✓ Use __all__ to define public API
✓ Document package purpose in __init__.py

Don'ts

✗ Star imports (from module import *)
✗ Circular imports
✗ Deep nesting (package/sub/sub/sub/module)
✗ Hide important functions in init.py
✗ Inconsistent import style

Summary: Package Organization

Pattern Use Case
Flat structure Small projects
Functional packages Medium projects
Feature-based Large monolithic
Namespace packages Multi-repo systems