Copy as Markdown[Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs-docusaurus.kinsta.page%2Fdataprime%2Fuser-guide%2Fusing-dataprime%2Fnormalize_data.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.)[Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs-docusaurus.kinsta.page%2Fdataprime%2Fuser-guide%2Fusing-dataprime%2Fnormalize_data.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.)

# Using DataPrime to clean and normalize data with functions

## Goal[​](#goal "Direct link to Goal")

By the end of this guide you should be able to use the [`firstNonNull`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/general/firstnonnull/.md), [`ipInSubnet`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/ip/ipinsubnet/.md), [`parseInterval`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/time/parseinterval/.md), and [`urlDecode`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/url/urldecode/.md) functions to clean, transform, and standardize inconsistent or encoded data in your logs.

## Why it matters[​](#why-it-matters "Direct link to Why it matters")

Raw logs are rarely consistent. Fields can be missing, renamed, or encoded—making it hard to search, group, or build alerts. These functions let you create structure out of inconsistency, enriching your logs without requiring upstream changes.

***

## `firstNonNull` – Use the first available non-null value[​](#firstnonnull--use-the-first-available-non-null-value "Direct link to firstnonnull--use-the-first-available-non-null-value")

### Description[​](#description "Direct link to Description")

Use [`firstNonNull`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/general/firstnonnull/.md) to return the first non-null value from a list of scalar fields. Ideal for normalizing inconsistent key names or fallback values.

### Syntax[​](#syntax "Direct link to Syntax")

```
firstNonNull(value: any, ...values: any): any
```

### Example – Normalize user identifiers across schemas[​](#example--normalize-user-identifiers-across-schemas "Direct link to Example – Normalize user identifiers across schemas")

#### Sample data[​](#sample-data "Direct link to Sample data")

```
{ "userId": "123", "user_id": null, "user_identifier": null }

{ "userId": null, "user_id": "456", "user_identifier": null }

{ "userId": null, "user_id": null, "user_identifier": "789" }
```

#### Query[​](#query "Direct link to Query")

```
choose canonical_user_id from firstNonNull(userId, user_id, user_identifier)
```

#### Result[​](#result "Direct link to Result")

```
{ "canonical_user_id": "123" }

{ "canonical_user_id": "456" }

{ "canonical_user_id": "789" }
```

Use this when multiple versions of a field may exist, and you want to unify them under a single key.

***

## `ipInSubnet` – Check if an IP address belongs to a given subnet[​](#ipinsubnet--check-if-an-ip-address-belongs-to-a-given-subnet "Direct link to ipinsubnet--check-if-an-ip-address-belongs-to-a-given-subnet")

### Description[​](#description-1 "Direct link to Description")

[`ipInSubnet`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/ip/ipinsubnet/.md) tests whether an IP address is within a given CIDR block. Useful for filtering internal vs external traffic, identifying environments, or tagging traffic by region.

### Syntax[​](#syntax-1 "Direct link to Syntax")

```
ipInSubnet(ip: string, ipPrefix: string): bool
```

### Example – Filter logs by internal IP range[​](#example--filter-logs-by-internal-ip-range "Direct link to Example – Filter logs by internal IP range")

#### Sample data[​](#sample-data-1 "Direct link to Sample data")

```
{ "ip": "10.8.0.8" }

{ "ip": "192.168.1.45" }

{ "ip": "8.8.8.8" }
```

#### Query[​](#query-1 "Direct link to Query")

```
filter ipInSubnet(ip, '10.0.0.0/8')
```

#### Result[​](#result-1 "Direct link to Result")

```
{ "ip": "10.8.0.8" }
```

Only logs where the IP is within `10.0.0.0/8` are retained. Useful for segmenting or excluding traffic.

***

## `parseInterval` – Convert a string duration to an interval type[​](#parseinterval--convert-a-string-duration-to-an-interval-type "Direct link to parseinterval--convert-a-string-duration-to-an-interval-type")

### Description[​](#description-2 "Direct link to Description")

With [`parseInterval`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/time/parseinterval/.md) you can parse duration strings like `"2d3h"` into an `interval` so you can perform time-based arithmetic (e.g., add to a timestamp). Enables accurate time math for scheduling, tracking, and alerts.

### Syntax[​](#syntax-2 "Direct link to Syntax")

```
parseInterval(string: string): interval
```

### Example – Add a duration to a start time[​](#example--add-a-duration-to-a-start-time "Direct link to Example – Add a duration to a start time")

#### Sample data[​](#sample-data-2 "Direct link to Sample data")

```
{

  "timestamp": 1728763337,

  "job_name": "BUILD_VIDEO",

  "completed_in": "35m10s"

}
```

#### Query[​](#query-2 "Direct link to Query")

```
create completed_time from addTime(timestamp, parseInterval(completed_in))
```

#### Result[​](#result-2 "Direct link to Result")

```
{

  "timestamp": 1728763337,

  "completed_in": "35m10s",

  "completed_time": 1728782447  // +2110 seconds

}
```

Now the log includes a concrete end time, useful for SLA tracking or time-based filtering.

***

## `urlDecode` – Decode URL-encoded values[​](#urldecode--decode-url-encoded-values "Direct link to urldecode--decode-url-encoded-values")

### Description[​](#description-3 "Direct link to Description")

Decode percent-encoded characters (e.g., `%20`) back into human-readable form with [`urlDecode`](https://docs-docusaurus.kinsta.page/dataprime/language-reference/functions-reference/url/urldecode/.md). Use when logs contain encoded query strings or URLs.

### Syntax[​](#syntax-3 "Direct link to Syntax")

```
urlDecode(string: string): string
```

### Example – Decode a URL-encoded name field[​](#example--decode-a-url-encoded-name-field "Direct link to Example – Decode a URL-encoded name field")

#### Sample data[​](#sample-data-3 "Direct link to Sample data")

```
{

  "query_string": "name=Tom%20Kosman&b=c&c=d"

}
```

#### Query[​](#query-3 "Direct link to Query")

```
extract query_string into query_params using kv(pair_delimiter='&', key_delimiter='=')

| replace query_params.name with urlDecode(query_params.name)
```

#### Result[​](#result-3 "Direct link to Result")

```
{

  "query_params": {

    "name": "Tom Kosman",

    "b": "c",

    "c": "d"

  }

}
```

Now your logs contain readable names and values, improving searchability and visualization.

***

## Common pitfalls[​](#common-pitfalls "Direct link to Common pitfalls")

* `firstNonNull` only works on scalar values (not arrays or objects).
* `ipInSubnet` requires valid CIDR strings like `'192.168.0.0/24'`.
* `parseInterval` fails silently if the input format is invalid (`1s1d` is not allowed).
* `urlDecode` works only on strings—decoding must be done field by field.