Skip to main content

Set Up Splunk Data Source Connector for AI Discovery

Overview​

The Cequence AI Gateway can discover AI agents, remote MCP servers, and LLM providers by querying Splunk SIEM for Palo Alto Next-Generation Firewall logs using the Splunk REST APIs. This guide explains the configuration required to enable this data source connector.

This configuration guide covers:

  • Splunk Enterprise standalone
  • Splunk Cloud Platform
  • Standard Splunk Add-on for Palo Alto Networks ingestion
  • Cribl-based ingestion into Splunk

Compatibility Summary​

The connector uses Splunk REST search jobs and authenticates with:

Authorization: Bearer <token>

It does not currently support username/password authentication or session-key authentication.

Splunk Enterprise standalone is supported. Required configuration includes token authentication, REST API endpoint and Splunk index names hosting the Palo Alto NGFW log data. Before you start confirm that the REST API is reachable, token authentication works, and the indexes are searchable.

Splunk Cloud Platform is supported. Required configuration includes token authentication, the REST API endpoint, and Splunk index names hosting the Palo Alto NGFW log data. Before you start, confirm that the Splunk Cloud search REST API endpoint is reachable, search API allowlisting is in place, token authentication works, and the required Palo Alto data is available.

Splunk Enterprise search head cluster deployments are not supported in this release.

Ingestion intermediaries like Cribl are supported only when the final events in Splunk are searchable, and the Palo Alto log data indexes can be mapped along with the sourcetypes, URL filter, and fields.

How the Connector Uses Splunk​

The connector uses the Splunk management/search REST API. It creates search jobs, polls for completion, retrieves results, and deletes the completed job.

The REST operations used are:

  • POST /services/search/jobs
  • GET /services/search/jobs/{sid}
  • GET /services/search/jobs/{sid}/results
  • DELETE /services/search/jobs/{sid}

The default source profile expects Palo Alto logs with:

  • Index: pan_logs
  • Traffic sourcetype: pan:traffic
  • URL/threat sourcetype: pan:threat
  • URL event filter: log_subtype=url
  • Key fields: url, dest, src_ip, app, category, user, http_user_agent, bytes, referer, and cert_cn

Custom indexes, sourcetypes, URL filters, and field names are supported through advanced configuration.

Prerequisites​

1. Enable Splunk REST Search API Access​

Requirement​

The connector must reach the Splunk search REST API over HTTPS.

Splunk Enterprise Standalone​

  1. Identify the Splunk search head management URL, for example:

    https://splunk.example.com:8089

  2. Ensure that the Cequence AI Gateway Control Plane IP(s) provided to you are allow listed on your firewall so that it can reach the Splunk host on the appropriate TCP port (8089 in the above example).

  3. Ensure the Splunk endpoint presents a valid TLS certificate signed by a public CA.

  4. This URL including the port is required for configuring the connector.

Splunk Cloud​

  1. Identify the Splunk Cloud search API endpoint:

    https://<deployment-name>.splunkcloud.com:8089

  2. Add the Cequence AI Gateway Control Plane IP(s) provided to you to the Splunk Cloud search-api allowlist. Use Admin Config Service if your organization manages allowlists self-service. If ACS is not available, open a Splunk Support case to enable REST API access and provide the Cequence AI Gateway Control Plane IP(s).

  3. This URL including the port is required for configuring the connector.

Additional Documentation​

2. Assign a Least-Privilege Search Role​

Requirement​

The service account used by the connector must be able to run searches against the Palo Alto index. Administrative privileges are not required. Create a role with least-privileges (read-only) and scoped to the Palo Alto index.

Splunk Enterprise Standalone & Splunk Cloud​

  1. Create a dedicated Splunk service account for the connector.

  2. Create a dedicated role, for example:

    cequence_aig_reader

  3. Grant the role the search capability.

  4. Add the Palo Alto index, such as pan_logs, to the role's searchable indexes.

  5. Assign the connector service account to the role.

  6. Confirm the service account can run:

    | tstats count WHERE index=pan_logs sourcetype=pan:* BY index, sourcetype

Notes​

The service account does not need the SPL delete capability. The connector uses HTTP DELETE to clean up its search job and does not run the SPL delete command against indexed data.

Additional Documentation​

3. Create a Splunk Bearer Token​

Requirement​

The connector requires a valid Splunk authentication token and sends it as a bearer token in the HTTP Authorization header. Create the token only after the dedicated service account has the read-only, index-scoped role described in the previous section.

Splunk Enterprise Standalone​

  1. Enable token authentication on the Splunk deployment.

  2. Create a static authentication token for the service account.

  3. Set an explicit expiration date. When no expiration time is set, Splunk uses the default global expiration of +30d.

  4. This token is required for configuring the connector.

Splunk Cloud​

  1. Create or request a static Splunk authentication token that can be used as Authorization: Bearer <token> for REST API calls.

  2. Set an explicit expiration date, or request a non-expiring static token if your security policy allows it. Confirm that the non-expiring static token is not governed by the Splunk Cloud default global expiration duration.

  3. This token is required for configuring the connector.

Notes​

The connector does not perform a SAML login flow and needs a Splunk bearer token. Interactive tokens issued using a SAML login flow are currently not supported since it requires an additional refresh process to mint a replacement token.

Additional Documentation​

4. Identify the Palo Alto Index​

Requirement​

The connector searches an index containing Palo Alto Traffic and URL/threat events. The default index is pan_logs.

Splunk Enterprise Standalone & Splunk Cloud​

  1. In Splunk Web, go to Settings, then Indexes.

  2. Identify the current index with the Palo Alto NGFW log data before creating a new one.

    You can run a manual Splunk search for likely Palo Alto sourcetypes to identify the index:

    | tstats count WHERE index=* (sourcetype=pan:* OR sourcetype=*palo*) BY index, sourcetype
    | sort - count

    If your Palo Alto NGFW logs were ingested using the standard Splunk Add-on for Palo Alto Networks, you can narrow the search:

    | tstats count WHERE index=* (sourcetype=pan:traffic OR sourcetype=pan:threat) BY index, sourcetype
    | sort - count

  3. If you use the default index name, no advanced configuration for the connector is required.

  4. If you use another index, advanced configuration for the connector is required to override the index name for the connector to search. If you have set up Splunk to use multiple indexes for the Palo Alto NGFW logs, you can configure a comma-separated list to override the index names in the connector advanced configuration.

  5. After confirming the index names, verify that the chosen indexes contains Palo Alto data. Replace pan_logs if you selected a different index:

    | tstats count WHERE index=pan_logs sourcetype=pan:* BY index, sourcetype

Additional Documentation​

5. Palo Alto Traffic and URL/Threat Logs​

Requirement​

The connector requires searchable Palo Alto Traffic logs as the bare minimum. URL Filtering logs is required for richer URL/category/decryption coverage. In the default Splunk Add-on path:

  • Traffic events use sourcetype pan:traffic.
  • URL Filtering events use sourcetype pan:threat.
  • URL Filtering events are selected with log_subtype=url.

The connector fails if it cannot find at least one destination-identifying field. At least one of these fields must be populated:

  • url
  • dest

The SPL aggregation also uses src_ip, so Palo Alto source IP data should be present in this field.

The connector expects the following default field map:

  • url
  • dest
  • app
  • category
  • user
  • http_user_agent
  • bytes
  • referer
  • cert_cn

Splunk Enterprise Standalone & Splunk Cloud​

  1. For Splunk Add-on-compatible data:

    Verify sourcetypes:

    | tstats count WHERE index=pan_logs sourcetype=pan:* BY index, sourcetype

    Verify URL Filtering events:

    index=pan_logs sourcetype=pan:threat log_subtype=url | stats count

    Check field availability with the default field names:

    index=pan_logs (sourcetype=pan:traffic OR sourcetype=pan:threat)
    | head 5000
    | fieldsummary
    | search field IN (url, dest, src_ip, app, category, user, http_user_agent, bytes, referer, cert_cn)

  2. For custom intermediary pipelines such as Cribl that send to Splunk:

    Identify the sourcetypes if you know the index:

    | tstats count WHERE index=<palo_alto_index> BY index, sourcetype
    | sort - count

    Identify the sourcetypes if you do not know the index or sourcetypes:

    | tstats count WHERE index=* (sourcetype=pan:* OR sourcetype=*palo* OR sourcetype=*cribl*) BY index, sourcetype
    | sort - count

    If the sourcetype names are not enough to distinguish Traffic from URL/threat logs, sample the likely sourcetypes and inspect fields such as log_subtype, type, subtype, url, dest, app, and category:

    index=<palo_alto_index> sourcetype=<candidate_sourcetype>
    | head 100
    | table _time sourcetype log_subtype type subtype url dest app category

    Inspect the actual field names that exist in the selected index and sourcetypes:

    index=<palo_alto_index> (sourcetype=<traffic_sourcetype> OR sourcetype=<url_or_threat_sourcetype>)
    | head 5000
    | fieldsummary
    | sort field

    Sample representative events to identify which fields carry the destination host or URL, source IP, App-ID, URL category, user, User-Agent, byte count, Referer, and certificate common name:

    index=<palo_alto_index> (sourcetype=<traffic_sourcetype> OR sourcetype=<url_or_threat_sourcetype>)
    | head 100
    | table _time sourcetype *

  3. If your verification confirmed the use of Splunk Add-on-compatible sourcetypes, URL Filtering events, and field names, no advanced configuration for the connector is required.

  4. For custom intermediary pipelines such as Cribl capture the sourcetypes and custom field names for use in the connector advanced configuration. At a minimum, identify a field that maps to url or dest, and confirm source IP data is available in the src_ip field.

Cribl Support Boundary​

Cribl is supported only as a preprocessing and forwarding layer. The connector reads from Splunk, not Cribl. Discovery quality depends on the final Splunk events retaining or mapping the required Palo Alto fields.

Additional Documentation​

Optional Requirements​

Optional requirements improve discovery completeness, attribution, and confidence. Missing optional configuration will not block discovery if mandatory data plumbing is correct.

1. Enable Palo Alto URL Filtering Logs​

Benefit​

URL Filtering logs improve URL category, destination, and AI traffic visibility.

Steps​

  1. In PAN-OS, go to Objects, then Security Profiles, then URL Filtering.

  2. Create or edit a URL Filtering profile.

  3. Set the AI-related categories, such as artificial-intelligence, to a log-producing action such as alert.

  4. Attach the URL Filtering profile to the outbound Security policy rule.

  5. Ensure the policy forwards Threat logs to Splunk.

  6. Commit the PAN-OS configuration.

Additional Documentation​

2. Enable SSL Forward Proxy Decryption for AI Categories​

Benefit​

Decryption is not mandatory, but it is the main difference between basic destination-level reporting and higher-confidence specific-agent and MCP classification.

With decryption, Palo Alto logs can expose:

  • Full URL paths
  • User-Agent headers
  • Referer headers
  • Better Layer 7 App-ID classification, including mcp

Steps​

  1. Define the AI categories or destinations approved for decryption.

  2. Create or identify a forward-trust CA certificate.

  3. Deploy the forward-trust CA certificate to managed client systems.

  4. Create an SSL Forward Proxy decryption policy.

  5. Scope the policy to approved AI categories or destinations.

  6. Exclude pinned, sensitive, or policy-exempt destinations.

  7. Attach a decryption profile for certificate and cipher controls.

  8. Commit the PAN-OS configuration.

Additional Documentation​

3. Enable HTTP Header Logging​

Benefit​

The connector uses User-Agent and Referer signals for specific agent classification when they are available. Without header logging, discovery can still classify based on host, App-ID, category, and path, but specific client identification is weaker.

Steps​

  1. In PAN-OS, edit the URL Filtering profile used for outbound AI traffic.

  2. Enable HTTP header logging for supported headers.

  3. Include User-Agent and Referer where allowed by policy.

  4. Ensure the relevant traffic is decrypted. HTTP headers are not visible in encrypted HTTPS traffic without decryption.

  5. Commit the configuration.

Additional Documentation​

4. Enable User-ID​

Benefit​

User-ID allows discovery to attribute AI agent and MCP activity to users. Without User-ID, discovery falls back to source IP attribution.

Steps​

  1. Configure a user mapping source, such as:

    • GlobalProtect
    • User-ID agent
    • Syslog user mapping
    • XML API user mapping

  2. Enable User Identification on trusted source zones.

  3. Commit the PAN-OS configuration.

Additional Documentation​

AI Discovery by Palo Alto Feature Coverage​

The connector is designed to produce degraded discovery when optional Palo Alto enrichment is missing. It does not require SSL decryption or User-ID to run, provided the mandatory Splunk data plumbing and destination fields are present.

Traffic Logs Only​

Expected discovery behavior:

  • AI destinations may be detected by destination host or App-ID when available.
  • User attribution is available only if the user field is populated.
  • MCP visibility is limited.
  • Encrypted HTTPS sessions may lack URL path and User-Agent evidence.
  • Findings may skew toward likely or candidate rather than confirmed.

Traffic Logs and URL Filtering Logs Without Decryption​

Expected discovery behavior:

  • AI provider visibility improves through URL category and URL Filtering records.
  • Destination-level findings can still be useful.
  • Decryption coverage is low or zero.
  • Specific-agent classification is limited when User-Agent and Referer are absent.
  • MCP classification is limited because URL path and Layer 7 mcp App-ID signals are usually unavailable without decryption.

URL Filtering Logs With SSL Decryption​

Expected discovery behavior:

  • URL paths become available.
  • AI API path evidence improves confidence.
  • MCP path evidence becomes available.
  • app=mcp classification is more likely because the firewall can inspect Layer 7 traffic.
  • MCP findings can move from candidate or likely to confirmed when multiple signals are present.

Decryption With User-Agent and Referer Header Logging​

Expected discovery behavior:

  • This is the strongest configuration for specific-agent classification.
  • User-Agent and Referer can identify specific clients such as CLI tools, IDEs, desktop apps, or SDKs.
  • The connector resolves client signals first, then adds host, path, category, App-ID, or certificate evidence.
  • Verified identifying signals can produce confirmed findings.
  • Provisional signals are guarded so a lone unverified signal does not overstate confidence.

User-ID Enabled​

Expected discovery behavior:

  • User rows identify users according to the selected identity mode.
  • Findings can be grouped by user.
  • Without User-ID, the discovery uses source IP attribution.

Cribl-Based Ingestion​

Expected discovery behavior:

  • Discovery quality depends on what Cribl preserves or transforms.
  • If Cribl preserves the Splunk Add-on-compatible schema, advanced configuration is not needed.
  • If Cribl drops URL paths, User-Agent, Referer, category, App-ID, or user fields, discovery can still run when mandatory fields remain, but attribution and confidence degrade.

Validation​

Click "Test data source connector" before enabling the connector, especially if you have advanced configuration in place.

Basic Configuration​

Tests Splunk plumbing to ensure that token, TLS, and REST connectivity are valid.

Advanced Configuration​

Tests Splunk plumbing to ensure that token, TLS, and REST connectivity are valid.

Tests index names, sourcetypes, field checks and data availability.

Further Reading​

Last updated on