> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lokpanchang.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Scheduler

> Celery and Beat: JIT planner, send_for_timezone, Redis locks, and the flow from 6:30 AM local to WhatsApp

Daily panchangam messages are sent at **6:30 AM in each user’s local timezone**. The application does not pre-schedule one task per user. Instead, **Celery Beat** runs a single recurring task every 10 minutes — the **JIT (just-in-time) planner** — which decides which timezones are about to hit 6:30 AM and enqueues one **`send_for_timezone`** task per such timezone. **Celery workers** run those tasks at the right moment (using a countdown), so each user gets their message at 6:30 AM local time. **Redis** is used as the message broker and for a per–timezone–date lock to avoid duplicate sends.

## Components

* **Celery** — Distributed task queue. Tasks are sent to a **broker** (Redis); workers pull and execute them.
* **Celery Beat** — Scheduler that, according to a fixed schedule, pushes task messages into the broker (e.g. “run `jit_planner` every 600 seconds”).
* **Redis** — Message broker (`CELERY_BROKER_URL`) and result backend (`CELERY_RESULT_BACKEND`). Also used for **locks** (e.g. `lock:{timezone}:{date}`) so only one send runs per timezone per day.
* **tasks.py** — Defines the Celery app, `jit_planner`, `send_for_timezone`, and the Beat schedule. Runs inside the **celery** (worker) and **beat** Docker services.

See [Backend](/backend) for how the Flask app and Celery relate; see [Quickstart](/quickstart) for bringing up the stack (including Redis, celery, beat).

## Flow: from 6:30 AM local to WhatsApp

1. **Beat** runs **`jit_planner`** every **10 minutes** (600 seconds).
2. **`jit_planner`** (in `tasks.py`):
   * Loads all **distinct timezones** that have at least one active user (`UserDetail.timezone` not null, `obsoleted_on > now`).
   * For each timezone, computes **local time** and checks whether **6:30 AM** (today or tomorrow in that zone) falls in the **next 15 minutes**.
   * If yes, tries to acquire a **Redis lock**: key `lock:{timezone}:{date}` (e.g. `lock:America/New_York:2025-02-25`), TTL 24 hours, `SETNX` (set only if not exists).
   * If the lock is acquired, enqueues **`send_for_timezone(timezone_name)`** with a **countdown** equal to the seconds until 6:30 AM in that timezone. The task is therefore executed at (or very close to) 6:30 AM local.
   * If the lock already exists, that timezone/date was already handled; skip.
3. When the countdown expires, a **Celery worker** runs **`send_for_timezone(timezone_name)`**:
   * Queries all active users with that timezone and valid location (phone, latitude, longitude).
   * For each user: loads `CityDetail` if `city_id` is set, calls **`generate_panchangam_with_timezone`** with the user’s coordinates, timezone, and **current local date** (so the panchangam is for “today” in their place), then calls **`send_panchangam_template`** to send the WhatsApp template message.
   * Logs success/failure per user and returns a short summary.

So: **Beat → jit\_planner every 10 min → for each TZ nearing 6:30 AM, lock + queue send\_for\_timezone with countdown → worker runs send\_for\_timezone at 6:30 local → panchangam generated and sent via WhatsApp.**

## JIT planner in detail

* **Why “JIT”?** — Only timezones that are about to hit 6:30 AM (within a 15-minute window) get a task. There is no fixed list of 6:30 AM UTC times; the planner recalculates using each timezone’s local time and DST.
* **15-minute window** — If the current time in a timezone is between 6:15 and 6:30 AM, the planner queues `send_for_timezone` with a countdown so the task runs at 6:30. This keeps scheduling simple and avoids missing a zone due to Beat’s 10-minute cadence.
* **Lock** — `redis_client.set(lock_key, 1, ex=86400, nx=True)`. Prevents two Beat runs (or a delayed run) from enqueueing two tasks for the same timezone/date. One send per timezone per calendar day.
* **Countdown** — `lead_seconds = (target_local - now_local).total_seconds()`; the task is enqueued with `countdown=lead_seconds`, so it runs at 6:30 AM in that timezone (worker clock is assumed correct; workers use UTC).

## send\_for\_timezone in detail

* **Input:** `timezone_name` (e.g. `Asia/Kolkata`).
* **Context:** Runs inside `flask_app.app_context()` so SQLAlchemy and the same `DATABASE_URL` as the web app are available.
* **Query:** Active users with `UserDetail.timezone == timezone_name` and `obsoleted_on > now`, with phone, latitude, longitude.
* **Per user:** Get `CityDetail` for `user.city_id` (for location name in panchangam). Call **`get_user_local_time(timezone_name)`** to get “now” in that timezone and pass it as **`target_date`** to **`generate_panchangam_with_timezone`**, so the panchangam is for the user’s current local date. Then **`send_panchangam_template`** (see [Core Panchangam calculation](/panchangam/core-calculation) and [Backend](/backend)).
* **Errors:** Exceptions (e.g. WhatsApp API errors) are logged; optional handling for “disable user” is documented in code. The task still returns a summary (success/error counts).

## Beat schedule (full)

Defined in **`tasks.py`** as **`celery.conf.beat_schedule`**:

| Schedule key             | Task                                  | When                     |
| ------------------------ | ------------------------------------- | ------------------------ |
| `jit-planner`            | `tasks.jit_planner`                   | Every 600 s (10 min)     |
| `collect-system-health`  | `tasks.collect_system_health_metrics` | Every 300 s (5 min)      |
| `update-daily-metrics`   | `tasks.update_daily_metrics_task`     | At 5 min past every hour |
| `cleanup-old-metrics`    | `tasks.cleanup_old_metrics`           | Daily at 02:00 UTC       |
| `generate-weekly-report` | `tasks.generate_weekly_report`        | Mondays at 09:00 UTC     |
| `backup-daily-logs`      | `tasks.backup_logs_task`              | Daily at 00:00 UTC       |
| `upload-logs-to-s3`      | `tasks.upload_logs_to_s3_task`        | Sundays at 02:00 UTC     |

The one that drives daily panchangam is **`jit-planner`**. The others are for metrics, health, log backup, and S3 upload (see [Logging](/logging)).

## Configuration and deployment

* **Broker / backend:** `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND` (e.g. `redis://redis:6379/0`). Same Redis is used for locks via **`redis_client`** in `tasks.py`.
* **Celery app:** `celery = Celery('tasks', broker=..., backend=...)` with `timezone='UTC'`, `enable_utc=True`, and `broker_transport_options={"visibility_timeout": 86400}` so long-countdown tasks are not reclaimed by the broker too early.
* **Flask app context:** A minimal Flask app is created in `tasks.py`, bound to the same `db` and `DATABASE_URL`, and used only to run tasks inside `with flask_app.app_context():`.
* **Docker:** The **celery** service runs `celery -A tasks.celery worker --loglevel=info --concurrency=1`. The **beat** service runs `celery -A tasks.celery beat` with a schedule file under `/tmp` (persisted via volume `beat_schedule`). Both need the same codebase, `DATABASE_URL`, and Redis URL.

## Changelog

03/20/2026

* Replaced legacy references to `plan_todays_broadcasts` with `jit_planner` in manual and test entry points.
* Confirmed active scheduler configuration already uses `tasks.jit_planner` in `tasks.py` (no production scheduling behavior change).
* Updated the `/test-planner` debug route in `app.py` to trigger `jit_planner` instead of the obsolete planner task name.
* Updated scheduler test utilities (`test_scripts/quick_test.py`, `test_scripts/test_scheduler_manually.py`) to import and call `jit_planner`.
* Verified `plan_todays_broadcasts` is no longer present in active code paths; remaining mentions are historical log artifacts only.
* Confirmed `/test-planner` is not referenced elsewhere in code, so it functions as a manual/debug endpoint only.

## Summary

* **Beat** fires **`jit_planner`** every 10 minutes.
* **`jit_planner`** finds timezones whose local time is 6:15–6:30 AM, acquires a Redis lock per timezone/date, and enqueues **`send_for_timezone(timezone)`** with a countdown to 6:30 AM local.
* **Workers** run **`send_for_timezone`** at that time, generate panchangam for each user in the timezone (using their local “today”), and send the WhatsApp template message.
* **Redis** is the broker and the store for the per–timezone–date lock so each user gets one panchangam per day at 6:30 AM local.