Duplicate Jobs
Branch if this deal was seen before
The Deduplicate node checks whether the thing that just came in is one you've already worked, and splits the job two ways: brand new, or a repeat of something already in the system. Wire the repeat path away from the expensive step and you stop paying to analyze the same deal twice.
How it works
The same application shows up more than you'd think — a broker resends it, an applicant follows up, a file gets forwarded twice. Without a check, each arrival runs the full analysis again and costs again. Deduplicate looks up whether a job with the same identity (for lenders, the business's tax ID) has already come through, and routes accordingly.
It surfaces rather than silently discards. Because the identity is read off the document and reading can occasionally misfire, the node never just drops a job — it sends repeats down a path you control and hands along the earlier job it matched (when it ran, its status, a link to the prior result) so a person can confirm the match is real before anything is skipped.
When a repeat arrives carrying a document the first one didn't (the classic case: a broker resends the deal to add the signed application their first email forgot), the node attaches that new file onto the earlier job so it isn't lost, and flags the alert with how many new documents came in and what they were — giving you {data.newDocCount} and {data.newDocTypes} to put in the duplicate notice. It attaches and surfaces only; it does not re-run the analysis on the new document.
It knows the difference between a genuine repeat and your own deliberate re-run of a job, so re-running something on purpose isn't mistaken for a duplicate. Point the 'new' path at your analysis and the 'duplicate' path at a quick heads-up to your team, and repeats resolve themselves without burning a second analysis.
Common use cases
Real situations where this node earns its place in a flow.
Don't underwrite the same deal twice
When a broker resends an application you've already analyzed, route it to a heads-up instead of running — and paying for — a second underwriting pass.
Catch duplicate submissions
Spot when the same business comes in again by its tax ID and surface the earlier job, with a link to the prior result, so a person can decide.
Skip repeats, keep humans in the loop
Send first-time deals to analysis and repeats to your team, so nothing is silently dropped on a mis-read identity.
Duplicate Jobs in a workflow
A simple example of where this node sits in a flow.
What you set up
- Put it after Extract, so the identity it matches on has been read off the document.
- Leave it on the default (the deal key / tax ID) or point it at another field.
- Wire 'new' to your analysis and 'duplicate' to a notice or a status — don't leave the duplicate path unwired, or a repeat ends quietly with no word to anyone.
Common questions
Won't my own re-runs look like duplicates?
No. A deliberate re-run shares the identity of the job it came from, and the node excludes that lineage, so re-running on purpose routes as new — only a separate resubmission counts as a duplicate.
What if the identity was read wrong and two different deals collide?
That's why it surfaces instead of dropping. A match sends the job down a path you control with the prior job attached, so a person can confirm it's genuinely the same before skipping anything.
Does it save the cost of reading the document too?
No — it runs after Extract, because that's where the identity is read. It saves the later analysis pass, not the extraction. Matching on an identity known before a document is read would be a separate setup.
Settings you configure on this node, stored under data.config in the workflow JSON. Leave a field unset to use its default.
keyFieldDotted path to the identity to match on (e.g. application_fields.ein), resolved from the extracted data / lookups. Empty uses the job's stamped dealKey (normalized EIN).
Type: stringmatchStatusesWhich prior-job statuses count as a match, comma-separated. Excludes failed/cancelled by default so a dead prior doesn't block a resubmission.
Type: stringDefault:queued, extracting, scrubbing, analyzing, awaiting_input, done, rejectedwithinDaysOnly match priors created within this many days; empty matches any time.
Type: number
How this node appears inside a workflow's nodes[] array (the shape you see in the Developer export / import). A full workflow is { nodes: [...], edges: [...] }.
{
"id": "dedupe",
"type": "card",
"position": {
"x": 160,
"y": 120
},
"data": {
"type": "dedupe",
"kind": "processor",
"title": "Duplicate Jobs",
"subtitle": "Branch if this deal was seen before",
"config": {
"keyField": "",
"matchStatuses": "queued, extracting, scrubbing, analyzing, awaiting_input, done, rejected",
"withinDays": "0"
}
}
}