​
🧭 System Architecture

​
🗂️ Defect Module Categories

The SYIA Defect System handles different modules organized into three main categories:

​
🗂️ Symbols & Sets

  • V\mathcal{V} = set of active vessels, indexed by IMO number.
  • rr = defect record.
  • y=year(reportedDate)y = \text{year}(\text{reportedDate}) = reporting year.
  • t0t_0 = current run date at midnight.
  • Signore={"closed","inactive","draft","invalid","rejected","cancelled"}\mathcal{S}_{ignore} = \{\text{"closed"},\text{"inactive"},\text{"draft"},\text{"invalid"},\text{"rejected"},\text{"cancelled"}\} = ignored statuses.

​
🔄 Common Processing Logic

​
1. Data Retrieval & Cleaning

  • Query MongoDB collections for inspection and report defects.
  • Restrict to active vessels (status=ACTIVEstatus = ACTIVE).
  • Normalize keys (case folding, unify field names).
  • Parse dates: reportedDate, targetDate, closingDate, nested findingsTargetDate, correctiveAction.targetDate.
  • Lowercase all statuses.
  • Replace parse failures (NaT) with None.
  • Standardize nested JSON structures (findings, correctiveAction).

​
2. Overdue Detection

For any candidate date dd with status ss: Overdue⁡(d,s):=(d≠NaT∧d<t0∧s∉Signore∧s≠"closed").\operatorname{Overdue}(d,s) := (d \neq \text{NaT} \land d < t_0 \land s \notin \mathcal{S}_{ignore} \land s \neq "closed"). Applied at three levels:
  • Parent record (targetDate).
  • Findings (findingsTargetDate).
  • Corrective Actions (CA.targetDate).

​
3. Parent Status Promotion

status←{"open",if parent = closed and any finding = open"overdue",if any finding/CA is overduestatus,otherwise.status \gets \begin{cases} \text{"open"}, & \text{if parent = closed and any finding = open} \\ \text{"overdue"}, & \text{if any finding/CA is overdue} \\ status, & \text{otherwise}. \end{cases}

​
4. Status → Color Mapping

status_colors = {
  "closed": "green",
  "open": "blue",
  "review in progress": "orange",
  "overdue": "red",
  "cancelled": "gray",
  "inactive": "lightgray",
  "rejected": "cyan"
  # ... extended variants handled
}

​
📊 Aggregations

​
Yearly Bar Data

For each year yy:
  • Total defects: Ty=∣{r:year(r)=y}∣T_y = |\{ r : year(r)=y \}|
  • Open defects: Oy=∣{r:year(r)=y∧status∉Signore}∣O_y = |\{ r : year(r)=y \land status \notin \mathcal{S}_{ignore} \}|
Output series: {(y,Ty)},{(y,Oy)}\{(y,T_y)\}, \{(y,O_y)\}.

Sample Defects Chart

​
Summary Counts

Let CsC_s = count of rows by status. After filtering ignored:
  • status_counts={s:Cs∣s∉Signore}status\_counts = \{ s : C_s \mid s \notin \mathcal{S}_{ignore} \}
  • Pending=∣{r:status∉Signore}∣Pending = |\{ r : status \notin \mathcal{S}_{ignore} \}|

Sample Output Table

​
🧮 Common Output Structures

  • Tabular rows: [Report Date, Reference No (link), Status, Description, Target Date, Closing Date, Closing Note, Source]
  • Charts: Yearly bar plots (Total vs Open).
  • Summary JSON: status counts, pending counts, chart data.
  • Lineitems: nested findings + corrective actions with overdue annotations.

​
⚙️ Algorithm

  1. Load environment + set t0t_0.
  2. Fetch active vessels.
  3. Retrieve raw defect data (inspection/report collections).
  4. Normalize fields, lowercase status, parse datetimes.
  5. Build findings + corrective actions arrays.
  6. Apply overdue detection + parent status promotion.
  7. Aggregate yearly bars and status counts.
  8. Format outputs (tables, charts, summaries).
  9. Assign colors with status_colors.
  10. Upsert into MongoDB component collections.

​
✅ Robust Engineering

  • Idempotent upserts (update_one(..., upsert=True)).
  • Consistent normalization: statuses always lowercase, dates coerced.
  • Error handling: try/except per vessel; log errors without halting pipeline.
  • Deterministic sort order: (reportedDate desc, status_color, targetDate, referenceNo).
  • Test mode override for fixed t0t_0 values.

​
📊 Worked Example

Suppose a record has:
  • Parent status = open, targetDate=None.
  • One finding with findingStatus=open, findingsTargetDate=t_0-1 day.
Then:
  • Overdue(findingsTargetDate,open)=TrueOverdue(findingsTargetDate,open) = True
  • Finding → overdue.
  • Parent → overdue.
  • Assigned color = red.