Skip to content

Remove Duplicates

Node: removeDuplicates · Full type: n8n-nodes-base.removeDuplicates · Version: 2

Operations

removeDuplicateInputItems

Parameter Type Default Details
compare options "allFields" allFields, allFieldsExcept, selectedFields
options collection {} keys: disableDotNotation, removeOtherFields, scope, historySize

removeItemsSeenInPreviousExecutions

Parameter Type Default Details
logic options "removeItemsWithAlreadySeenKeyValues" removeItemsWithAlreadySeenKeyValues, removeItemsUpToStoredIncrementalKey, removeItemsUpToStoredDate
options collection {} keys: disableDotNotation, removeOtherFields, scope, historySize

clearDeduplicationHistory

Parameter Type Default Details
mode options "cleanDatabase" cleanDatabase
options collection {} keys: disableDotNotation, removeOtherFields, scope, historySize

Common Parameters

Parameter Type Default Details
fieldsToExclude string ""
fieldsToCompare string ""
dedupeValue string "" required
incrementalDedupeValue number ""
dateDedupeValue dateTime ""

Parameter Details

options children:

Parameter Type Default Details
disableDotNotation boolean false
removeOtherFields boolean false
scope options "node" workflow, node
historySize number 10000

Example

NODE "removeDuplicates" AS "Remove Duplicates" {
  operation: "removeDuplicateInputItems",
  dedupeValue: {{ $json.dedupeValue }}
}