setDiff

Description

Returns the set difference of two arrays, producing a new array with elements from array1 that are not in array2.

• Duplicates are discarded when computing the difference.
• Supported element types include string, bool, number, interval, timestamp, regexp, and enum.

Syntax

Like many functions in DataPrime, setDiff supports two notations, function and method. These interchangeable forms allow flexibility in how you structure expressions.

Function notation

setDiff(array1: array<T>, array2: array<T>): array<T>

Method notation

(array1: array<T>).setDiff(array2: array<T>): array<T>

Arguments

Name	Type	Required	Description
array1	array<T>	true	The base array to compare
array2	array<T>	true	The array containing values to exclude from array1

Example

Use case: Identify unauthorized IP addresses

Suppose you have aggregated a list of IP addresses that accessed different paths. Consider the following input:

{
    "path": "/home",
    "ip_addresses": ["156.76.87.4", "156.76.12.4", "156.74.1.4"]
},
{
    "path": "/checkout",
    "ip_addresses": ["156.76.87.4"]
}

By applying setDiff, you can compare the observed IP addresses against a known allow list to identify unauthorized addresses.

Example query

Function notation

create unauthorized_ip_addresses from setDiff(ip_addresses, ["156.76.12.4", "156.76.87.4"])

Method notation

create unauthorized_ip_addresses from ip_addresses.setDiff(["156.76.12.4", "156.76.87.4"])

Example output

The result will include a new field unauthorized_ip_addresses containing only the addresses not in the allow list:

{
    "path": "/home",
    "ip_addresses": ["156.76.87.4", "156.76.12.4", "156.74.1.4"],
    "unauthorized_ip_addresses": ["156.74.1.4"]
},
{
    "path": "/checkout",
    "ip_addresses": ["156.76.87.4"],
    "unauthorized_ip_addresses": []
}