> ## Documentation Index
> Fetch the complete documentation index at: https://docs.serval.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Splunk Observability Cloud
> Splunk Observability Cloud (formerly SignalFx) is Splunk's metrics, APM, and alerting platform - Serval connects to your realm with an org access token so workflows can read and manage detectors, incidents, alert muting rules, events, SLOs, dashboards, and teams.
## About Splunk Observability Cloud
Splunk Observability Cloud (formerly SignalFx) is Splunk's observability platform covering infrastructure monitoring, APM, synthetics, and detector-based alerting. Serval connects to a Splunk Observability Cloud organization using its **realm** (the regional instance identifier, e.g. us0, us1, eu0, jp0, au0) and an **org access token**, stored as a secret and attached to every request Serval makes on your behalf. An Instance Name distinguishes multiple connections on the same team.
**Authentication:** Org access token with the API authorization scope (custom header authentication). The token is stored as a secret and injected on every request as the X-SF-TOKEN header. There is no OAuth flow and no token refresh - the token is used as-is until you rotate it by editing the connection.
**Data sync:** On-demand only. Serval calls the Splunk Observability Cloud API when a workflow step runs - there is no background sync.
This integration is for **Splunk Observability Cloud**, not Splunk Enterprise / Splunk Cloud Platform (the log-search and SIEM product). The two products have separate APIs and credentials.
## What the Splunk Observability Cloud integration enables
| Capability | Description |
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| Manage detectors | List, create, update, enable, disable, and validate detectors, and read a detector's events and incidents. |
| Read and clear incidents | List active incidents (triggered alerts), read a single incident, and clear incidents. |
| Manage alert muting rules | Create, read, update, delete, and end alert muting rules. |
| Query events | Search custom and detector events with the events v2 API. |
| Manage SLOs | Create, read, search, validate, update, and delete service level objectives. |
| Manage dashboards and charts | Full CRUD on dashboards, dashboard groups, and charts. |
| Manage teams | List teams, read team details, and manage team membership. |
| Generic API access | Call any endpoint in the bundled spec (31 typed paths across the families above) with full control over the request. |
Anything defined in the bundled families of the [Splunk Observability Cloud API](https://dev.splunk.com/observability/reference) can be accessed through Serval.
Operations that Splunk restricts to **session tokens** (a logged-in admin's user token) are not supported, because Serval holds only your org access token. This mainly affects the Splunk-side Integrations API (managing Splunk's own notification integrations), org token management, and organization membership. Everything in the capability table above works with an org access token.
## Get your credentials
Serval needs two pieces of information from Splunk Observability Cloud: your **realm** and an **org access token** with the API authorization scope. See [Splunk: Create and manage authentication tokens](https://help.splunk.com/en/splunk-observability-cloud/administer/authentication-and-security/authentication-tokens/org-access-tokens) for the official guide.
Log in to Splunk Observability Cloud and open **Settings > View Profile > Organizations**. The realm is listed there, and it is also visible in your browser URL (e.g. app.us1.observability.splunkcloud.com means realm us1).
Navigate to **Settings > Access Tokens**, select **New Token**, give it a name (e.g. "Serval Integration"), and choose the **API** authorization scope. Copy the token value.
API-scoped access tokens carry an RBAC authorization role: power, usage, or read\_only. Reads work with read\_only; creating or updating detectors, muting rules, SLOs, and dashboards, and clearing incidents require the power role. Grant the token the role your workflows need.
Getting the realm right matters more than anything else on this page. Your token only exists in your organization's realm, so a us1 organization's token will never work against the us0 API domain.
## Connect in Serval
All three fields are required.
Enter your realm identifier, e.g. "us0", "us1", "eu0", or "jp0". Serval derives the API domain from it: api.\.observability.splunkcloud.com. Anything other than a lowercase alphanumeric identifier is rejected.
Paste the org access token you created. This is a password-type field.
Give the connection a display name, e.g. "Production". Required to distinguish between multiple Splunk Observability Cloud connections.
Click **Submit** to create the connection. Serval stores the token as a secret and verifies the connection with a health check against your realm.
When editing an existing connection, any field you leave blank keeps its current value - only fields you fill in are overwritten. To rotate just the access token, fill in only that field. For security, the existing token value is never displayed in the edit form.
## Verifying the connection
When the connection is created, Serval runs a health check that confirms your token is valid and your realm is reachable. The integration also includes four named health-check workflows you can run at any time:
* **Validate Splunk Observability Cloud API Connection** - lists detectors to confirm the realm and token work together.
* **Check Splunk Observability Incidents Access** - verifies Serval can read incidents (triggered alerts).
* **Check Splunk Observability Dashboards Access** - verifies Serval can read dashboards.
* **Check Splunk Observability Teams Access** - verifies Serval can read teams.
## Gotchas and troubleshooting
The realm field is free text - it is not auto-detected. If your organization is on us1 but you enter us0, every API call fails authentication because your token only exists in your realm. Check **Settings > View Profile > Organizations** in Splunk Observability Cloud, or your app URL, and enter the matching realm.
Serval expects an **org access token** (Settings > Access Tokens) with the API authorization scope. A User API access token (session token) also works against the API but expires and is tied to a person, so don't use one for the connection. Conversely, Splunk endpoints that demand a session token (the Splunk-side Integrations API, org token management, organization membership) are out of scope for this integration.
An API-scoped access token carries an RBAC authorization role (power, usage, or read\_only). If workflows get 403s on writes - creating detectors, clearing incidents, managing muting rules - while reads work, the token likely has read\_only or usage authorization. Recreate or re-scope the token with the power role.
Several retrieve endpoints (e.g. GET /v2/detector/\{id}) return 200 with an empty body instead of 404 when the object doesn't exist, and list endpoints return 200 with count: 0 for no matches. Workflows should check the response payload rather than relying on an error being thrown.
Retrieve operations return at most 10,000 objects regardless of paging, even if your organization has more. Filter with the name/tag query parameters to narrow results instead of paging through everything.
GET /v2/incident returns a JSON array of incidents, unlike most other list endpoints in the API which return a \{count, results} envelope. Workflows iterating incidents should handle the array shape.
***
Need help? Contact **[support@serval.com](mailto:support@serval.com)** for assistance with your Splunk Observability Cloud integration.