Logging

Overview

SpeedPy uses django-structlog for structured logging. It provides consistent, machine-parseable log output with automatic request context binding.

Configuration

The logging setup in project/settings.py includes three formatters:

• plain_console — human-readable console output (used by default)
• json_formatter — JSON output for log aggregation services
• key_value — key=value format

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "json_formatter": {
            "()": structlog.stdlib.ProcessorFormatter,
            "processor": structlog.processors.JSONRenderer(),
        },
        "plain_console": {
            "()": structlog.stdlib.ProcessorFormatter,
            "processor": structlog.dev.ConsoleRenderer(),
        },
    },
    "handlers": {
        "console": {"class": "logging.StreamHandler", "formatter": "plain_console"}
    },
    "loggers": {
        "": {"handlers": ["console"], "level": LOG_LEVEL},
        "botocore": {"handlers": ["console"], "level": "WARNING", "propagate": False},
        "boto3": {"handlers": ["console"], "level": "WARNING", "propagate": False},
        "s3transfer": {"handlers": ["console"], "level": "WARNING", "propagate": False},
        "urllib3": {"handlers": ["console"], "level": "WARNING", "propagate": False},
    },
}

Log level

The root level comes from LOG_LEVEL, which defaults to DEBUG in development and INFO in production:

LOG_LEVEL = env.str("LOG_LEVEL", default="DEBUG" if DEBUG else "INFO").upper()

Set LOG_LEVEL=DEBUG on a deployed app when you need to debug something, and put it back afterwards.

Why boto and urllib3 are pinned

Those four loggers are held at WARNING independently of the root level, and that is deliberate rather than tidiness.

At DEBUG, botocore logs the full body of every HTTP request it sends. If your app sends mail through an AWS-backed provider — SES via Anymail, for example — that body is the whole email. In practice it means password-reset tokens (as base64 MIME anyone can decode), complete message bodies, and SigV4 Authorization headers written into your container log, where log aggregation and support access spread them further.

Lowering the root level alone would fix that today and quietly undo it the next time somebody sets LOG_LEVEL=DEBUG to investigate something unrelated — which is exactly the moment nobody is looking at boto output. Pinning each logger with propagate: False means raising the root level stays safe.

If you add another chatty client library that logs request bodies, pin it the same way rather than relying on the root level.

Check this if you upgraded from an older SpeedPy

Earlier versions set the root logger to DEBUG unconditionally and had no per-library pins. If your project/settings.py still reads "loggers": {"": {"handlers": ["console"], "level": "DEBUG"}} and you send mail through SES, your production log very likely contains real password-reset tokens. Apply the block above, then treat the existing log history as sensitive.

Middleware

The structlog request middleware is included to automatically bind request data (user, IP, request ID) to log entries:

MIDDLEWARE = [
    ...
    "django_structlog.middlewares.RequestMiddleware",
]

DJANGO_STRUCTLOG_CELERY_ENABLED = True
DJANGO_STRUCTLOG_COMMAND_LOGGING_ENABLED = True

Usage

import structlog

logger = structlog.get_logger(__name__)

def my_view(request):
    logger.info("processing_request", user_id=request.user.id)
    try:
        result = do_something()
        logger.info("processing_complete", result=result)
    except Exception:
        logger.exception("processing_failed")

Key practices: - Use structlog.get_logger(__name__) at module level - Pass context as keyword arguments, not in the message string - Use snake_case for event names - Use logger.exception() for errors (automatically includes traceback)

Celery Integration

Structlog is also configured for Celery workers in project/celeryapp.py via DjangoStructLogInitStep. This ensures Celery tasks get the same structured logging format.

from django_structlog.celery.steps import DjangoStructLogInitStep

app.steps["worker"].add(DjangoStructLogInitStep)

Switching to JSON in Production

To output JSON logs (for services like Datadog, Elastic, etc.), change the handler formatter:

"handlers": {
    "console": {"class": "logging.StreamHandler", "formatter": "json_formatter"}
},