Packages & Organization¶
Overview¶
Package structure determines project scalability:
- Modules: Single.py files
- Packages: Directories with
__init__.py - Namespace packages: PEP 420 implicit packages
- Best practices: Professional organization
Project Structure¶
Standard Project Layout¶
myproject
- pyproject.toml # Modern project config
- setup.py # Legacy setup
- README.md
- LICENSE
- docs
- conf.py # Sphinx config
- index.rst
- src
- myproject
- __init__.py
- __main__.py # CLI entry point
- core.py
- utils
- __init__.py
- math.py
- io.py
- models
- __init__.py
- neural_net.py
- tests
- __init__.py
- conftest.py # Pytest config
- test_core.py
- requirements.txt # Dependencies
-
init.py Patterns¶
Minimal init.py¶
# myproject/__init__.py
"""Package docstring."""
__version__ = "0.1.0"
__author__ = "Your Name"
Selective Exports¶
# myproject/__init__.py
"""MyProject - A powerful ML library."""
from.core import Model, Trainer
from.utils import load_data, preprocess
__version__ = "0.1.0"
__all__ = ['Model', 'Trainer', 'load_data', 'preprocess']
# Hide internals
# _internal is not exported with "from myproject import *"
Lazy Imports¶
# myproject/__init__.py
"""Lazy import to reduce startup time."""
def __getattr__(name):
"""Lazy import on attribute access."""
if name == 'heavy_module':
from. import heavy_module
return heavy_module
raise AttributeError(f"Module has no attribute {name}")
# Usage
-
Import Styles¶
Absolute Imports¶
# In myproject/models/neural_net.py
from myproject.core import Model # Absolute import
from myproject.utils.math import sqrt # Absolute import
class NeuralNet(Model):
pass
Relative Imports¶
# In myproject/models/neural_net.py
from..core import Model # Relative import
from..utils.math import sqrt # Relative import
from.layers import Dense # Same-package import
class NeuralNet(Model):
pass
Conditional Imports¶
# myproject/core.py
"""Core module with optional dependencies."""
try:
import torch
HAS_TORCH = True
except ImportError:
HAS_TORCH = False
try:
import tensorflow as tf
HAS_TF = True
except ImportError:
HAS_TF = False
def get_backend():
if HAS_TORCH:
return 'torch'
elif HAS_TF:
return 'tensorflow'
else:
return 'numpy'
Namespace Packages (PEP 420)¶
Implicit Namespace Packages¶
# No __init__.py needed!
mycompany
- ml
- models.py # No __init__.py
- data
- loader.py # No __init__.py
- web
- api.py # No __init__.py
# All are part of 'mycompany' namespace
from mycompany.ml import models
from mycompany.data import loader
from mycompany.web import api
-
Entry Points¶
Command-line Entry Point¶
# pyproject.toml
[project.scripts]
myproject-cli = "myproject.cli:main"
# setup.py (legacy)
setup(
entry_points={
'console_scripts': [
'myproject-cli=myproject.cli:main',
],
}
)
# myproject/cli.py
def main():
import sys
# CLI implementation
pass
if __name__ == '__main__':
main()
Plugin System¶
# pyproject.toml
[project.entry-points."myproject.plugins"]
my_plugin = "my_plugin:Plugin"
# Discover plugins at runtime
def load_plugins():
from importlib.metadata import entry_points
eps = entry_points(group='myproject.plugins')
plugins = {}
for ep in eps:
plugins[ep.name] = ep.load()
return plugins
-
Best Practices¶
Do's¶
Use relative imports within packages
Organize by functionality, not file type
Limit import depth (max 3 levels)
Use __all__ to define public API
Document package purpose in __init__.py
Don'ts¶
Star imports (from module import *)
Circular imports
Deep nesting (package/sub/sub/sub/module)
Hide important functions in init.py
Inconsistent import style
Summary: Package Organization¶
| Pattern | Use Case |
|---|---|
| Flat structure | Small projects |
| Functional packages | Medium projects |
| Feature-based | Large monolithic |
| Namespace packages | Multi-repo systems |
-