The supplier review is on Thursday. One supplier insists it shipped on time, and your receiving log says the pallets landed two days late. Another delivery arrived complete but a third of it failed inspection, and nobody can agree whether that counts as a miss. A third supplier is quoting a confirmed date your own team apparently moved by email three weeks ago, so the conversation is now about whose date is correct instead of why the line went short. Meanwhile an OTIF number in a spreadsheet is drifting down, and you are not sure you could defend it if the supplier pushed back hard.
This is the everyday version of a supplier scorecard problem. It rarely feels like a data problem. It feels like arguing about the past instead of fixing the future. A useful supplier scorecard and OTIF report should let your team act before the next quarterly business review, not just narrate what went wrong after the quarter has closed. The job is to turn purchase orders, promised dates, actual receipts, quality holds, claims, and escalation notes into one view that procurement and operations trust enough to make decisions from.
This guide is written for a procurement lead, supply chain director, category manager, or operations head who needs supplier performance to become visible, defensible, and connected to follow-up. You may already have plenty of reports. The problem is usually not the absence of numbers. It is that supplier conversations still run on anecdotes, disputed dates, and spreadsheets someone stitched together the night before.
What the scorecard is actually there to change
It helps to be honest about what a supplier scorecard is for. It is not there to produce a percentage. It is there to change three things: which suppliers get pressure, which get help, and where your team spends its follow-up time. If the scorecard does not change any of those, it is a report nobody acts on.
A scorecard that earns its place should let you answer practical questions every week and every quarter. Which suppliers are hitting their commitments and which are drifting. Which misses are the supplier's fault and which are yours. Which delivery problems are turning into quality problems, short shipments, or claims. And which suppliers need a conversation now rather than a note in the next review pack.
The hard part is credibility. The moment an OTIF definition is unclear or a number cannot be traced back to a receipt, the supplier argues, your own team hedges, and the scorecard loses authority. So the real work sits underneath the chart: clear rules, linked evidence, honest attribution, and a place for the resulting actions to live. Get that right and the percentage almost takes care of itself.
Follow one late pallet from the dock to the QBR
Before choosing software or adding AI, trace how one late delivery actually moves through your process today. Not the version in a procedure document. The messy version.
In most teams it looks something like this. A shipment arrives at the dock. Receiving scans it in, but the goods receipt gets posted in the system a day or two later, so the timestamp is already off. Someone notices the quantity is short and raises it with the supplier by email. The supplier replies that they shipped in full and blames the carrier. A quality hold is opened separately because a few cartons are damaged, and that lives in the quality system, not the delivery report. Weeks later, an analyst exports purchase orders, goods receipts, and confirmations into a spreadsheet, adjusts a few dates by hand, and pastes the result into the scorecard. By the time the supplier sees it at the review, the whole meeting is spent debating definitions instead of fixing performance.
The point of tracing this is not to blame anyone. It is to see where the information gets lost. Almost every supplier scorecard problem comes from a handoff: a date leaves its source system, loses context, gets copied somewhere else, picks up a manual adjustment, and then gets discussed by people who do not know how much checking happened underneath. The workflow breaks because supplier reporting is treated as a monthly project instead of something captured as the work happens.
Settle what OTIF means before you build a single chart
OTIF looks simple. A delivery is on time and in full, or it is not. In practice, almost every word in that phrase hides a decision, and the suppliers you score will find every one of those decisions the first time the number goes against them. Settle the rules first, write them down, and share them with suppliers before you publish anything.
On time, measured against which date?
This is the fight that sinks most scorecards. On time against the date you requested is a harder test than on time against the date the supplier confirmed. Suppliers will always argue for the confirmed date. Your team often wants the requested date because that is what the business actually needed. Both are legitimate, and you have to pick one, name it, and be consistent. You also have to decide the tolerance window. Is exactly on the day the only pass, or do you allow a day either side? Do early deliveries count as on time, or as a miss because they clog receiving and tie up cash? Many teams forget that an early window matters as much as a late one.
In full, measured at what level?
In full can mean the whole order arrived, or every line arrived, or the delivered value matched the ordered value. Line level is the honest test and the harshest one, because a single short line fails the order. You also need a rule for over-shipments and for partial deliveries that are later completed. If a supplier sends ninety percent now and the rest in three days, is that one miss, two lines, or a pass once completed?
Does quality belong inside OTIF or beside it?
A delivery can arrive on time and in full and still be wrong. Cartons are damaged, a lot fails inspection, documentation is missing so the goods cannot be released. Some teams fold this into a stricter measure, sometimes written as on time, in full, and error free. Others keep quality as a separate dimension so a delivery miss and a quality miss do not get blurred. Either choice works. Not choosing is what causes trouble, because a quality failure then either disappears from the score or double counts.
| Dimension | The decision you have to make | Common trap |
|---|---|---|
| On time | Requested date or confirmed date, and the tolerance window each side | Measuring against requested date while the supplier only ever agreed to a confirmed date |
| Early delivery | Whether an early receipt is a pass or a miss | Treating early as free, then absorbing the receiving and inventory cost quietly |
| In full | Order level, line level, or value level, and how partials are treated | Order-level scoring that hides repeated short lines on critical parts |
| Receipt timestamp | Dock arrival, goods receipt posting, or put-away time | Using posting time, so slow receiving looks like supplier lateness |
| Quality | Inside a stricter OTIF measure or a separate dimension | Quality holds sitting outside delivery data, so supplier impact is understated |
Separate supplier misses from your own changes
This is the difference between a scorecard suppliers respect and one they dismiss. If your team moved a delivery date late, closed the dock for stocktake, changed the order after confirmation, or posted the receipt three days after the truck arrived, that is not a supplier failure. Score it as one and you lose the room. The supplier stops trusting the number, and worse, your own team stops trusting it too, because everyone knows some of those misses are self-inflicted.
Fair attribution does not require a heavy process. It requires an exception reason on every miss and a short list of who caused it. A miss is either supplier caused, internally caused, a carrier or transport issue, a demand or order change, or a data error such as a late receipt posting. Once you split misses this way, two useful things happen. Supplier conversations get sharper because you only pressure suppliers on the misses that are genuinely theirs. And your own operation gets a to-do list, because the internally caused misses are problems you can fix without a single supplier email.
The minimum version worth shipping
You do not need a large procurement platform to make this work. You need clear rules, source data you can trust, and somewhere for the actions to live. A first version that genuinely changes supplier conversations usually has four moving parts and nothing more.
The first is a scorecard view that shows OTIF and its dimensions by supplier, and lets you cut it by category, site, lane, or product family, using the rules you settled above. The second is a drill-through that shows the evidence behind any miss: the late, short, damaged, rejected, or disputed deliveries that make up the number, each linked to a source record. The third is a supplier issue log with an owner, a root cause, a next action, and an escalation status. The fourth is a review pack or supplier conversation note that is generated from the same underlying data, so the QBR is not a separate rebuild. Everything else is an improvement you can add later once these four earn their keep.
Build the evidence trail so the scorecard survives an argument
Supplier scorecards live or die on date discipline and record matching. For every miss, the team should be able to say which date counts, where it came from, when it changed, and who caused the failure. Without that trail, OTIF becomes a debate. With it, the supplier can still disagree with the decision, but not with the facts.
The data usually sits in four places, and the work is connecting the minimum needed to defend a miss rather than integrating everything at once.
| Source | What you pull from it | Why the scorecard needs it |
|---|---|---|
| Procurement and PO system | Requested date, confirmed date, quantity, revisions, category, supplier | Establishes the commitment the delivery is measured against |
| Receiving, warehouse, or ERP | Actual receipt, shortages, overages, holds, put-away timing | Establishes what actually arrived and when |
| Quality and claims | Defects, rejects, damages, non-conformances, corrective actions | Captures the misses that on-time-in-full alone hides |
| Supplier messages and ASN | Shipment notices, tracking, explanations, escalation notes | Supplies the context and the supplier's own version of events |
The most common data failure is the receipt timestamp. Dock arrival, the goods receipt posting, and the put-away time can be days apart. If your OTIF calculation reads the posting time, a slow receiving team will quietly make good suppliers look late. Decide which timestamp counts, and if there is a known lag in how receipts get posted, correct for it before you blame anyone.
Turn the findings into a supplier issue log
A scorecard tells you where performance dropped. It does not, on its own, fix anything. The bridge between the number and the improvement is a supplier issue log: a short, live record of what went wrong, who owns it, and what happens next. Without it, the same misses get rediscovered every quarter and nothing changes between reviews.
Keep the log lean. Each entry needs the supplier or segment, the evidence behind the issue, the issue itself, and the next action with an owner. The example below shows what a working log looks like partway through a month.
| Supplier segment | Evidence | Issue | Next action and owner |
|---|---|---|---|
| Packaging suppliers | Three receipts after the confirmed date | Late reason is inconsistent across the shipments | Category manager to confirm the date rule before the QBR note |
| Ingredient suppliers | Rejected lot tied to a quality hold | OTIF miss driven by a quality failure, not delivery timing | Quality owner to attach the non-conformance evidence |
| Contract manufacturers | ASN quantity differs from the receipt | Short shipment needs a dispute review before it is scored | Procurement to update the status after the supplier replies |
The value of the log is continuity. When the review happens, you are not reconstructing what went wrong from memory and email threads. You are reading a record that has been building all month, with the evidence already attached.
A worked example: 120 suppliers, one QBR
Say a food and beverage distributor with 120 active suppliers runs a quarterly review with its top 15 by spend. The scorecard shows an overall OTIF of 91 percent, which looks respectable, so nobody expects a difficult meeting. Then two things surface once the evidence is attached.
First, the headline number is measured against the confirmed date, but three of the top suppliers had their requested dates missed by a week and simply reconfirmed to a later date each time. Against the requested date, their real performance is closer to 78 percent. Neither number is wrong. They answer different questions, and the distributor had been quietly reading the flattering one. Second, one ingredient supplier scores 96 percent on delivery but sits behind four quality holds in the same period, none of which touched the OTIF figure because quality was tracked in a separate system. On paper that supplier is a star. In reality it is causing more disruption than a supplier with a lower delivery score.
When the team splits the misses by cause, roughly a third turn out to be internal: late order changes, a dock closed for a stocktake, and receipts posted days after arrival. That is uncomfortable, but useful. The supplier conversations get shorter and more honest, because the distributor only presses on the misses that are genuinely the supplier's, and it walks away with its own list of fixes. This scenario is illustrative, not a real client result, but the shape of it is what a useful scorecard exposes: the average was hiding both the date question and the quality question.
Segment suppliers so an average does not hide the misses
A single portfolio-wide OTIF number is almost always misleading. A miss on a commodity carton and a miss on a critical, single-source component are not the same event, yet an average treats them as equal. The scorecard becomes more honest the moment you segment it.
Segment by spend, by criticality, and by category. A supplier of a critical part running at 90 percent may deserve more attention than a commodity supplier running at 80, because the downside of a stockout is far worse. Weighting the score by spend or by criticality, or simply showing the critical suppliers on their own view, stops the important misses from being averaged into invisibility. This is also where the scorecard starts to guide procurement strategy rather than just grade the past, because it shows where a second source or a safety-stock buffer is worth the cost.
Set a weekly cadence, not a monthly scramble
The workflow needs a predictable rhythm, or every review becomes a fresh negotiation over deadlines and formats. The trick is to capture performance evidence continuously and reuse it, so the weekly operating decision and the quarterly supplier conversation draw from the same place instead of two separate rebuilds.
| Cadence | What happens | Output |
|---|---|---|
| Daily to weekly | New misses land in the issue log as receipts and holds happen, tagged with a cause | A live list of open supplier issues, not a month-end surprise |
| Weekly review | Procurement and operations clear exceptions, chase disputed dates, assign next actions | Decisions made while the problem is still fresh and fixable |
| Monthly | Scorecard refreshed, internal versus supplier misses reconciled, trends checked | A defensible OTIF view by supplier and segment |
| Quarterly review | QBR pack generated from the same data, with evidence and open actions attached | A supplier conversation about improvement, not definitions |
The exact intervals matter less than the discipline. What you are trying to avoid is the pattern where the whole period is spent collecting and reconciling data, and there is no energy left for the conversation the reporting was supposed to support.
Where AI helps inside the workflow
AI is useful here once the rules and the evidence are in place. If your OTIF definition is vague and your sources are inconsistent, AI mostly helps you produce confident-looking confusion faster. Used with a clear structure around it, it takes a lot of the manual reconstruction off the team.
The practical uses are narrow and real. It can read supplier explanations spread across emails, notes, and the issue log and group them into consistent reason themes, so you can see whether lateness is a capacity problem, a transport problem, or a planning problem. It can summarize repeated misses and quality incidents for a supplier ahead of a review, pulling the history into one place. It can draft a first-pass QBR commentary, a corrective-action request, or an internal escalation note with the source links attached. And it can flag conflicting dates or missing evidence that a human needs to resolve before the scorecard goes out.
What AI should not do is score a supplier on its own or invent a reason where the evidence is thin. The rule is simple: AI prepares the review, and your team owns the definitions and the decisions.
Where human judgment has to stay
Some parts of this workflow should never be automated away, because they are exactly where credibility is won or lost. Your team, not a model, decides what OTIF means and whether the definition changed. Your team decides whether a miss is genuinely the supplier's or your own, which is a judgment call as often as it is a data lookup. Your team decides how to weight a critical supplier against a commodity one. And your team owns the tone of the supplier conversation, because a scorecard used as a weapon damages a relationship you may depend on when supply gets tight.
The goal is not to remove judgment from supplier management. It is to stop spending that judgment on copy-paste, date chasing, and version control, and save it for the decisions that actually move performance.
Traps that keep the scorecard ignored
The same failures show up again and again, and most of them are avoidable once you know to look for them.
Measuring against the wrong date
If suppliers only ever agreed to a confirmed date and you score them against the requested date without saying so, the first review turns into an argument they are right to have. Pick the date rule deliberately, write it down, and tell suppliers before you publish.
Letting receiving lag count as supplier lateness
When the receipt posting time is treated as the arrival time, a slow dock makes good suppliers look bad. This is one of the most common and least visible ways a scorecard loses credibility, because the supplier knows the truck was on time and you cannot explain the gap.
Hiding quality inside a clean delivery number
A supplier that delivers on time but fails inspection repeatedly looks fine on OTIF alone. If quality is not connected to the scorecard, your worst supplier can look like your best one.
Averaging away the misses that matter
A single OTIF percentage across every supplier and part treats a commodity miss and a critical-component miss as equal. Segment by criticality and spend, or the important failures disappear into the mean.
Building the scorecard once a month from scratch
If the number is rebuilt from exports the night before the review, it is stale and fragile by the time anyone sees it. Capture the evidence as it happens and let the review read from it.
Publishing a score with nowhere for the action to go
The most quietly fatal trap is a scorecard that produces a number and no follow-up. If misses do not become owned issues with a next action, the same problems return every quarter and the supplier learns that the scorecard has no teeth.
Pick tools after the rules are settled
There is no single correct tool stack for supplier scorecards, and the right answer depends on how many suppliers you have, how clean your source systems are, and how much you want to own internally. A spreadsheet over clean ERP and quality exports is a perfectly good first version if the supplier count is small and the definitions are clear, though it will strain on version control and manual matching. A structured database or lightweight app makes intake, ownership, and issue status much clearer without a large platform project. A BI layer over a warehouse suits teams whose sources are already consistent and who want repeatable views. Dedicated supplier performance modules inside larger procurement suites can fit teams that want a system built around this specifically.
Whichever you choose, the tool does not settle the questions that actually matter. Definitions, attribution rules, owners, cadence, and where the actions live are decisions the implementation has to make, and no software makes them for you. Pick the tool after those are clear, not before.
A first version to ship in four weeks
If the current process is messy, do not try to rebuild everything at once. Start narrow enough to ship and valuable enough to matter, and let it prove itself before you widen it.
A sensible first month starts by picking one supplier segment, category, or site and settling the scorecard rules before touching a chart: requested date, confirmed date, receipt date, the quantity rule, the quality rule, the exception reasons, and the escalation route. The first real output is a supplier performance record that carries the OTIF result, the evidence links, an issue owner, a root-cause note, a next action, and a review status. In the following two weeks, connect the minimum PO, receiving, quality, claims, and ASN data needed to defend that record, and let AI help normalize supplier explanations and draft the review notes while your team keeps the definitions and decisions. By the fourth week, procurement and operations should be able to run one supplier review straight from the scorecard, the evidence drill-through, and the issue log, rather than from a spreadsheet built the night before.
Keep going if the scorecard is changing supplier conversations and cutting down date debates. Stop or narrow it if the OTIF rules are still being argued, because that is the signal the definitions are not settled yet and no amount of tooling will fix it.
How Ubisar would implement this workflow
This is a practical month inside the way Ubisar works: pick one workflow that is slowing the business, fix the data and tools around it, and ship something the team actually uses. For a supplier scorecard, week one settles the rules and builds the first performance record for one segment. Weeks two and three connect the minimum procurement, receiving, quality, and supplier-message data needed to defend a miss, with AI helping to summarize repeat misses and draft the review notes. By week four, one supplier review runs from the scorecard, the evidence, and the issue log instead of a manual rebuild, and you keep or narrow the work based on whether it changed the conversation.
If supplier performance is eating your team's time and losing arguments it should be winning, Ubisar can help you settle the definitions, build the evidence trail, and add AI where it improves the work without taking over the judgment. The AI, Data and Tech Implementation service covers exactly this kind of month, and the contact page is the place to start if you want to talk through your own supplier data.
Useful next links
Supplier performance connects directly to exceptions, reorder risk, and freight execution, so a few adjacent guides are worth reading alongside this one.
- Read the supply chain exception reporting guide when supplier misses need to feed the daily operating queue.
- Read the inventory reorder visibility guide when supplier lead time and OTIF performance are driving inventory risk.
- Read the freight booking and document workflow guide when supplier performance problems show up as shipment and document noise.