How to monitor VICIdial SFTP pushes with append-only JSONL

01 / 08

How this guide watches the push script

Identify the actual delivery job and log path, then read its documented fields and expected events. A JSON Lines file contains one JSON value per line.

Check syntax and access separately from remote receipt. Rotation changes configuration, so test it on a disposable log before applying a policy.

• Record the actual log path and documented event schema.
• Use a dedicated read-only monitor account.
• Compare local outcomes with recipient-side confirmation.

02 / 08

Step 1 — Confirm the log exists

This guide does not install a delivery job. Look up the actual JSONL path in the deployed push-job configuration; the example path below is illustrative, not a stock VICIdial path. Inspect file metadata privately.

sudo ls -l -- '<ACTUAL_JSONL_PATH>'

03 / 08

2. Reproduce the six run-health cases locally

This teaching schema contains exactly run_id,at and event. at is timezone-aware UTC; event is started,completed or failed. Each run needs one start and one terminal result in order. completed means the local job reported completion; it is not recipient-side integrity confirmation.

The fixed as-of time is2026-11-07T12:00:00Z. These fixture runs are already past their expected completion deadline; the model is not an in-progress polling policy. A fresh log can still contain a failed or missing-terminal run.

Run the complete model below. It rejects stale history, missing completion, explicit failure, duplicate terminal events and invalid JSON. These are real local checks of fictional lines, not a transfer test.

1. Copy the complete fixed Python model into a local terminal or file and run it with Python3. Keep its fictional lines and fixed as-of time unchanged for this first check; it does not read your delivery log.
2. Read each named case in the actual stdout: complete must pass the local contract; missing,failed,duplicate,stale and malformed JSON must each produce the stated rejection category. Check the process exits successfully after all assertions.
3. Compare the results with the retained local recording. If any assertion differs,save the exact code,stdout and Python version and stop before adapting this reader to a real job.
4. Only after the six cases agree,document the real producer’s field adapter,cadence and completion deadline. Keep recipient-side file/hash confirmation separate from the log contract.

python3 - <<'PYMODEL'import jsonfrom datetime import datetime,timezoneASOF=datetime.fromisoformat("2026-11-07T12:00:00+00:00")def check(lines):    runs={}; times=[]    try:        for line in lines:            row=json.loads(line)            if set(row)!={"run_id","at","event"}: return "invalid schema"            rid,event=row["run_id"],row["event"]            if not isinstance(rid,str) or not rid or event not in {"started","completed","failed"}:return "invalid schema"            at=datetime.fromisoformat(row["at"].replace("Z","+00:00"))            if at.tzinfo is None or at>ASOF:return "invalid time"            events=runs.setdefault(rid,[])            if events and at<events[-1][1]:return "out of order"            events.append((event,at)); times.append(at)    except (ValueError,TypeError,KeyError):return "invalid JSON/schema"    if not times or (ASOF-max(times)).total_seconds()>3600:return "stale"    for events in runs.values():        names=[e[0] for e in events]        if names.count("started")!=1 or names[0]!="started":return "invalid start"        if sum(e in {"completed","failed"} for e in names)>1:return "duplicate terminal"        if "failed" in names:return "failed run"        if names[-1]!="completed":return "missing completion"    return "pass local log contract"def row(event,at="2026-11-07T11:45:00Z"):    return json.dumps(dict(run_id="fixtureA",at=at,event=event))cases=[("complete",[row("started"),row("completed")],"pass local log contract"),       ("missing",[row("started")],"missing completion"),       ("error",[row("started"),row("failed")],"failed run"),       ("duplicate",[row("started"),row("completed"),row("completed")],"duplicate terminal"),       ("stale",[row("started","2026-11-07T09:00:00Z"),row("completed","2026-11-07T09:01:00Z")],"stale"),       ("bad-json",["not-json"],"invalid JSON/schema")]for name,lines,expected in cases:    actual=check(lines); assert actual==expected,(name,actual)    print("PASS",name,"->",actual)print("PASS 6 fictional monitor cases; no SFTP transfer or recipient receipt tested.")PYMODEL

View a still screenshot

Observed local fictional model

Open to enlarge

You should seeSyntax alone is no longer the test: run identity, state, freshness and completion are explicit.

04 / 08

3. Read the actual adapted log after its deadline

Have the job owner verify the adapter and choose a stable completed window of at most5MB. This reader expects the exact teaching schema; it does not guess the producer’s fields. Change the maximum age only from the documented cadence and alert policy.

Run after the relevant job’s expected completion deadline plus its agreed grace. A run still legitimately in progress must not be declared a delivery failure by this snapshot reader. If an active writer or rotation changes the window, take a new approved snapshot rather than editing or truncating the live log.

Record the reader’s exit/category, as-of time and next owner. Nonzero results go to the delivery/log owner; independently ask the recipient to confirm the selected files and hashes. Never disable delivery by merely turning off this monitor.

1. With the job owner,select the actual read-only log snapshot,verify its adapter emits exactly run_id/at/event and record the expected completion deadline plus grace. Confirm the chosen window is stable and no larger than the reader’s5MB bound.
2. Replace only the reader’s approved snapshot path and reviewed freshness/deadline settings. Execute once as the authorized monitor account after that window’s deadline;do not reuse the fictional fixture result as evidence for this run.
3. Record the complete stdout/stderr,exit category,as-of time and snapshot identity. For a missing/unreadable file,failed run,missing terminal,duplicate or stale result,hand off that specific boundary to its owner;do not truncate the log,widen permissions or suppress the failure.
4. For a passing local log contract,obtain the independent recipient-side selected-file/hash confirmation before claiming delivery. For a failed or changing snapshot,retain the observation and ask the owner for a new approved window after correction;do not disable the delivery job to silence this monitor.

(  LOG_PATH='/REPLACE/WITH/APPROVED/STABLE/JSONL'  MAX_AGE_SECONDS=3600  python3 - "$LOG_PATH" "$MAX_AGE_SECONDS" <<'PYLOG'import json,sys,pathlibfrom datetime import datetime,timezoneASOF=datetime.now(timezone.utc)MAX_AGE=int(sys.argv[2])if MAX_AGE<=0: raise SystemExit("STOP: freshness must be positive")def check(lines):    runs={}; times=[]    try:        for line in lines:            row=json.loads(line)            if not isinstance(row,dict) or not isinstance(row.get("at"),str): return "invalid schema"            if set(row)!={"run_id","at","event"}: return "invalid schema"            rid,event=row["run_id"],row["event"]            if not isinstance(rid,str) or not rid or event not in {"started","completed","failed"}:return "invalid schema"            at=datetime.fromisoformat(row["at"].replace("Z","+00:00"))            if at.tzinfo is None or at>ASOF:return "invalid time"            events=runs.setdefault(rid,[])            if events and at<events[-1][1]:return "out of order"            events.append((event,at)); times.append(at)    except (ValueError,TypeError,KeyError):return "invalid JSON/schema"    if not times or (ASOF-max(times)).total_seconds()>MAX_AGE:return "stale"    for events in runs.values():        names=[e[0] for e in events]        if names.count("started")!=1 or names[0]!="started":return "invalid start"        if sum(e in {"completed","failed"} for e in names)>1:return "duplicate terminal"        if "failed" in names:return "failed run"        if names[-1]!="completed":return "missing completion"    return "pass local log contract"path=pathlib.Path(sys.argv[1])try:    if path.stat().st_size>5_000_000:        raise SystemExit('STOP: log exceeds the reviewed5MB bound; ask owner for a bounded completed window')    with path.open(encoding='utf-8') as log:        status=check(line for line in log if line.strip())except OSError as error:    raise SystemExit('STOP: '+type(error).__name__+'; check the actual path/account with its owner')print(status)raise SystemExit(0 if status=='pass local log contract' else 1) PYLOG)

You should seeAn operational result has a documented schema, deadline, freshness rule and response owner.

05 / 08

4. Confirm the file and directory access boundary

Inspect the file and parent directory ownership, permissions and applicable access controls. Test the monitoring account on a disposable copy; do not attempt to append to a live operational log.

stat -c '%U %a %n' /REPLACE/WITH/ACTUAL/LOG

06 / 08

5. Test rotation against the writer’s actual reopen behavior

Plan a policy for the actual log path and the writer’s reopen behavior. Test the policy against a disposable file before deploying it; rotation and retention are configuration changes.

/var/log/vicidial-sales-sftp/push.jsonl {  weekly  rotate 12  missingok  notifempty  compress  delaycompress}

07 / 08

6. Compare stable read evidence without hiding failures

During a quiet test window, record the hash and count, read the file, then compare them again. An active writer or rotation can legitimately change the file during the test; investigate timing rather than attributing a change to the reader.

sha256sum /REPLACE/WITH/ACTUAL/STABLE/LOGwc -l /REPLACE/WITH/ACTUAL/STABLE/LOG

08 / 08

7. Debug the test policy and preserve delivery ownership

Use logrotate debug mode on the actual test policy and inspect its output without suppressing errors. Restore the prior policy through the same configuration process if the test fails.

Disabling monitoring does not stop delivery. Independently disable the delivery job if transfers must stop.

logrotate --debug /path/to/test-policy