Shared hosting's process limit makes your watchdog kill its own child

Symptom

A background collector quietly dies every 30–60 minutes. No traceback. The log stops mid-line.

Cause

fork: Resource temporarily unavailable

Restricted shells on shared hosting set a very low per-user process limit (ulimit -u). And the watchdog wrapper I wrote ran this on every loop:

while [ "$(ps aux | grep -c '[j]ob_name')" -gt 0 ]

One ps, one grep, one worker. Every poll burns three process slots → limit exceeded → the child gets killed.

The wrapper was killing the process it existed to protect.

Iron rules

1. No ps / grep polling in background watchdog scripts
2. Prevent concurrent runs with a lock file
3. Long jobs: pile all arguments into one process. Don't split into batches and launch several.
4. Partial save per item → progress survives a crash. This saved the day more than once.

What it looks like once you're over the limit

When fork starts failing, you can't even open a new SSH session. Shells already attached can't run commands.

★ If the number of standing cron jobs is already over the process limit, the moment a couple of batches overlap they push each other out. Nobody counts them at registration time because each one is just one.

★ Batches run one at a time, sequentially. If "it seems stuck" keeps happening, suspect this.

Image conversion tools need -limit thread 1 + MAGICK_THREAD_LIMIT=1 for the same reason. Without it, everything fails while reporting "done"; always count the output files.

Three more remedies

1. A one-time pre-check before starting — if ps -u "$(id -un)" | wc -l is over a threshold (say 12), skip this run. Count once at startup, not in a polling loop
2. nice -n 10 — lower the batch's priority so SSH stays responsive
3. Check for duplicate cron registrations — crontab -l | grep -c <name> should be 1. The same entry registered twice fills the limit twice as fast

---

---