Proof-of-Delivery Capture with OCR for a Haulage Firm

The most important lesson of the project came early: the quality of the photograph mattered more than the choice of model. Drivers photograph notes at an angle, in shadow under a trailer, or in low light at the end of a shift. Before any reading takes place, an OpenCV stage corrects skew, flattens shadows and adjusts exposure, so the document model sees something close to a scanned page.

That clean-up step lifted accuracy more than any model change we tried, and it is why the system copes with the photos drivers really take rather than the photos a demo would assume.

• Perspective and skew correction
• Shadow removal and exposure balancing
• Consistent, scan-like images for the reading stage

Printed fields such as the customer, the note number and the planned load are read by a document model in Azure Document Intelligence. Handwritten quantities and signatures behave very differently, so they are verified in a separate stage that produces its own confidence score for each value.

Keeping the stages apart means a faint signature does not drag down confidence in a perfectly legible customer name, and the system can say precisely which field it is unsure about. That precision is what makes the review queue fast: the administrator is shown the one value that needs a human, not the whole document.

Reading a note is only half the job. Each extracted note is reconciled against the planned delivery held in the transport system — the expected customer, date and pallet count. When everything agrees, the delivery is released for invoicing with no human input; this now happens for 91% of notes.

When something differs — eighteen pallets read where twenty were planned, or a signature the model cannot confirm — the note is flagged as an exception rather than invoiced. Discrepancies are resolved while the delivery is fresh in everyone's memory, instead of surfacing weeks later as a disputed invoice.

• Automatic reconciliation against planned deliveries
• Pallet-count and signature exceptions caught before invoicing
• Matched notes released straight to the transport system

Exceptions appear in a review queue that shows the photograph and the extracted fields together, with low-confidence values highlighted. One click corrects a value and releases the invoice. Administrators who once spent their days typing now spend them handling exceptions and disputes — the work that genuinely needs their judgement.

Behind the queue, a FastAPI service and Redis queues keep processing steady during the end-of-day rush when most photos arrive, and PostgreSQL stores every note, field and decision. Any note can be found by customer, date or driver within seconds.