# Understanding logs Copy as Markdown[Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs-docusaurus.kinsta.page%2Fdataprime%2Fuser-guide%2Ffoundations%2Funderstanding-logs.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.)[Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs-docusaurus.kinsta.page%2Fdataprime%2Fuser-guide%2Ffoundations%2Funderstanding-logs.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.) ## Goal[​](#goal "Direct link to Goal") By the end of this guide, you should be able to: * Copy the full nested JSON path for any field * Understand how user data is stored inside the `$d`, `$m`, `$l` objects * Use the `choose` and `filter` functions to view and narrow down logs by specific fields ## Why It Matters[​](#why-it-matters "Direct link to Why It Matters") Before you can investigate issues, correlate events, or create dashboards, you need to know how to explore raw logs. This guide will help you extract meaningful structure from JSON events and use DataPrime’s core tools to begin transforming and filtering logs effectively. *** ### Use `choose` to select specific fields[​](#use-choose-to-select-specific-fields "Direct link to use-choose-to-select-specific-fields") If you want to see just what you need, use the `choose` command: ``` choose k8s ``` Or to access multiple items ``` choose k8s, user_id ``` This will return JSON objects with only the `k8s` and `user_id` fields. > Pro tip: You can also rename fields or compute new ones using `as`: ``` choose (response_bytes / 1024) as kb_returned ``` *** ### Understand the structure: `$d` is your user data[​](#understand-the-structure-d-is-your-user-data "Direct link to understand-the-structure-d-is-your-user-data") All user log data lives under the `$d` namespace. This includes fields like `user_id`, `cluster_name`, and nested structures like `http.request.headers`. In this log, the "root" level of the log is `$d`. ``` { "user_id": 11, "cluster_name": "frank", "http": { "request": { "headers": { "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36", "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8", "accept_language": "en-US,en;q=0.5", "accept_encoding": "gzip, deflate, br", ... } } } } ``` So you can access `user_agent` with `$d.http.request.headers.user_agent`.\*\* But `$d` is the default namespace, so it can be omitted entirely\*\*, `http.request.headers.user_agent` Think of `$d` as the core payload of each log. Metadata like severity or timestamp is stored separately under `$m`. Note Your field names in `$d` are your own, so they don't necessarity match what are in these docs. Don’t forget that `$d` is not strictly needed in most cases. If it’s left out you will query your user data. To query metadata `$m`, labels `$l`, and parameters `$p`, those access mechanisms are required. ### Copy a full nested JSON path[​](#copy-a-full-nested-json-path "Direct link to Copy a full nested JSON path") Want to build a query on a deeply nested field? * Right-click on a key in the expanded log. * Select **Copy complete JSON path**. * This will copy the full path (e.g., `http.request.headers.user_agent`) which you can use in your query. ![complete JSON path](/assets/images/copyJSONpath-56b6edcadce36bc7549177ff319108e8.webp) This is especially useful when dealing with structured logs or observability pipelines. ## Accessing data[​](#accessing-data "Direct link to Accessing data") When writing queries in DataPrime, you’ll often reference keys inside your logs. These can be accessed in two ways: Use dot notation for clean, readable paths when your key names don't contain special characters. Say your JSON object looks like this: ``` { "k8s": { "container": { "id": 1 } } } ``` You can access the `id` using the flat dot notation and the `choose` command. ``` source logs | choose k8s.container.id ``` Or you can use bracket notation ``` source logs | choose $d['k8s']['container']['id'] ``` Using bracket notation is required to access a key containing spaces, symbols, dots (`.`), or any other special character. This log contains characters in its keys that [break dot notation](https://docs-docusaurus.kinsta.page/dataprime/cookbook/accessing_special_chars/.md): ``` { "k8s": { "container": { "id": 1 } }, "k8s.container.id": 2, "user name": "Rick", "DataPrime 🤘": "Rocks" } ``` **query** ``` source logs | choose $d['k8s.container.id'], $d['user name'], $d['DataPrime 🤘'] ``` **result** ``` { DataPrime 🤘: Rocks k8s.container.id: 2 user name: Rick } ``` ### You can also mix and match both styles as needed[​](#you-can-also-mix-and-match-both-styles-as-needed "Direct link to You can also mix and match both styles as needed") ``` source logs | choose $d['k8s'].container['id'] ``` **or** ``` source logs | choose k8s.container['id'] ``` These examples just prove the point, but use whatever best suits your use cases and preferences. ### **Use `filter` to focus your search**[​](#use-filter-to-focus-your-search "Direct link to use-filter-to-focus-your-search") Once you know the path to a key, narrow your logs with a `filter`: ``` source logs | filter http.status_code == 500 ``` This returns only logs where the HTTP status code equals `500`. You can combine with `choose` for a compact view: ``` source logs | filter http.status_code == 500 | choose timestamp, http.path, http.status_code ``` *** ## Common Pitfalls[​](#common-pitfalls "Direct link to Common Pitfalls") * **Forgetting the `$d` context**: If your field isn’t found, make sure it’s inside the `$d` object unless it’s a label (`$l`) or metadata (`$m`). * **Copying partial paths**: Always use “Copy complete JSON path” to avoid malformed queries. * **Too many fields?** Use `choose` aggressively to reduce noise.