Setup — Install Splunk and Load the Tutorial Data
Everything in this guide runs against a sample dataset called index=tutorial. Do this once, before anything else.
1. Download Splunk Enterprise
Splunk offers a free trial of Splunk Enterprise, which is what every exercise in this guide assumes you're running. Download the Windows installer below, or visit splunk.com/download for macOS/Linux.
- Run the downloaded .msi file.
- Accept the license agreement.
- Choose "Local System" account when prompted (default is fine for a learning install).
- Set an admin username and password — you will use this to log into Splunk Web every time, so write it down somewhere safe.
- Finish the install. It will automatically open Splunk Web in your browser at http://localhost:8000, or you can navigate there manually once setup completes.
Tip: If port 8000 is already in use by something else on your machine, the installer will tell you during setup — you can change it, but localhost:8000 is the default everyone's instructions (including this guide) assume.
2. Download the tutorial data
This is the sample dataset — fictional company "Buttercup Games" — that every single exercise in this guide is written against. It's a single zip file containing several log file types.
Tip: Don't unzip this file. Splunk reads it directly as a .zip during the upload step below — unzipping it first is unnecessary and won't cause anything to work better.
3. Create the tutorial index
Before uploading any data, you need a destination for it: an index named tutorial. Every search example in this guide starts with index=tutorial — if you name it something else, you'll need to mentally substitute your own index name in every single search from here on, so it's worth just matching the name exactly.
- Log into Splunk Web (http://localhost:8000).
- Click Settings (top right) → Indexes.
- Click New Index (top right of the indexes list).
- Index Name: type tutorial exactly, all lowercase.
- Leave every other setting at its default — Index Data Type: Events, Max Size, Home Path, etc. all default values are fine for a learning environment.
- Click Save.
Exercise: Confirm the index exists
Go back to Settings → Indexes and look for tutorial in the list. It should show 0 events for now — that's expected, you haven't loaded any data into it yet.
4. Upload tutorialdata.zip into the tutorial index
Now you'll point Splunk at the zip file and tell it specifically to load that data into the tutorial index you just created — not the default index, which is a mistake that quietly breaks every exercise later if it happens.
- Click Settings → Add Data.
- Click Upload (the "I want to upload a file" option, not monitor or forward).
- Click Select File, browse to wherever you saved tutorialdata.zip, and select it.
- Click Next at the bottom of the page.
- On the "Set Source Type" screen, Splunk will try to auto-detect the format. You can leave this as-is and click Next — the three sourcetypes used in this guide (access_combined_wcookie, vendor_sales/vendor_sales, www1/secure) are bundled inside the zip with their formats already defined.
- On the "Input Settings" screen, find the Index dropdown. Change it from default to tutorial.
- Click Review, confirm the settings look correct (Index: tutorial), then click Submit.
Warning: The single most common setup mistake: forgetting to change the Index dropdown away from default before submitting. If every search in this guide comes back with zero results, this is the first thing to check — go to Settings → Indexes, look at how many events default has versus tutorial, and re-upload correctly if needed.
5. Verify the data loaded correctly
Before trusting any exercise in this guide, confirm the data is actually there and Splunk can find it.
Tip: Set your time range to All time as a habit for the rest of this guide, every time a search comes back empty before you assume something is broken.
Exercise: Run your first search
Go to the Search & Reporting app, paste this into the search bar, and set the time range (top right of the search bar) to All time. You should see thousands of events appear. If you see zero, check the warning above about the Index dropdown, and double-check the time range — almost every "zero results" problem in this entire guide traces back to one of those two things.
index=tutorial
Search Fundamentals — table, rename, fields, dedup, sort
Your first real SPL commands: turning a messy raw event into a clean, readable report.
The dataset you'll use throughout this guide
index=tutorial contains three sourcetypes. Knowing their shape up front saves a lot of guessing later.
| Sourcetype |
What it contains |
Quoting needed? |
| access_combined_wcookie |
Web traffic — clientip, method, productId, status, bytes |
No |
| vendor_sales/vendor_sales |
Vendor transactions — VendorID, Code, AcctID |
Yes — wrap in quotes |
| www1/secure |
Secure site activity — clientip, action, categoryId, itemId |
Yes — wrap in quotes |
Warning: Sourcetype names containing a slash (/) must be wrapped in double quotes — e.g. sourcetype="vendor_sales/vendor_sales". Forgetting this is the single most common syntax mistake newcomers make.
table and rename
table picks and orders the columns you want to see. rename changes a column's display label without touching the underlying field name or data — the data never changes, only what you see.
Exercise: Build a clean web traffic report
Run this first and just look at the raw results — don't build anything yet, get a feel for what fields exist.
index=tutorial sourcetype=access_combined_wcookie
| head 5
index=tutorial sourcetype=access_combined_wcookie
| table _time, clientip, method, productId, status, bytes
You picked 6 fields out of a much messier raw event, displayed as clean columns in the order you listed them.
index=tutorial sourcetype=access_combined_wcookie
| table _time, clientip, method, productId, status, bytes
| rename clientip as "Visitor IP", method as "HTTP Method", productId as "Product", status as "Response Code", bytes as "Response Size"
Every column header except _time now has a friendly name. Try renaming _time to "Visit Time" as well — type it yourself rather than copy-pasting.
The single most useful distinction in all of SPL
It's easy to assume any command's effect shows up on the Statistics tab. It doesn't. Splunk's commands split into two groups, and mixing them up causes confusion for weeks if it isn't corrected early.
| Group |
Commands |
Behavior |
| Display-only |
table, fields, dedup, sort, rename |
Change what you SEE or how results are arranged. Do NOT populate the Statistics tab. |
| Transforming |
stats, top, rare, chart, timechart |
Actually aggregate or summarize data. DO populate the Statistics tab automatically. |
Tip: Quick check: after running a search, look at the Statistics tab next to Events and Visualization. Empty or greyed out → display-only command. Populated with a summary table → transforming command.
fields, dedup, and sort
fields trims which fields are kept in the underlying result set — but on its own it changes nothing visible in the raw Events display. dedup keeps only the first event per unique value of a field. sort orders results (a leading minus sign means descending).
Tip: Most finished, real-world reports look roughly like this: filter → trim → dedup → sort → table → rename.
Exercise: Trim, deduplicate, and order a result set
Run it, then check the Fields sidebar — only your 3 chosen fields are listed there now. Add | table clientip, method, productId to actually see it.
index=tutorial sourcetype=access_combined_wcookie
| fields clientip, method, productId
What do you expect to see change in the Events tab? Click to check.
index=tutorial sourcetype=access_combined_wcookie
| dedup clientip
| table _time, clientip, method, productId
Only the first event for each unique clientip is kept — compare the result count to a plain search without dedup.
index=tutorial sourcetype=access_combined_wcookie
| dedup clientip
| table _time, clientip, method, productId
| sort - _time
Descending (newest first). Remove the minus sign and rerun to see ascending order instead.
top, rare, and stats
Unlike fields/dedup/sort, these three ARE transforming commands — the Statistics tab populates the moment you use any of them. top finds the most common values (sorted descending, with an automatic percent column, top 10 by default). rare is its mirror image — least common values, same shape, opposite order. stats is the most flexible of the three; top and rare are really just shortcuts for specific stats patterns underneath.
Exercise: Compare top, rare, and stats on the same field
Most common product viewed.
index=tutorial sourcetype=access_combined_wcookie
| top productId
index=tutorial sourcetype="vendor_sales/vendor_sales"
| rare Code
Least common transaction code. Rare events often matter more in real investigations than common ones — a rare status code or login pattern is often where problems hide.
index=tutorial sourcetype=access_combined_wcookie
| stats count by productId
| sort - count
Run this and compare it directly to the top productId result above — they should show identical products in identical order. top is doing exactly what you just built manually with stats and sort.
From real-world questions to SPL queries
In a real SOC, nobody asks you to 'run a top.' They ask a business or security question, and you decide which command answers it. The pattern: read the question in plain English, decide whether you're counting, finding extremes, or just cleaning up a list, then pick the matching command.
| The question sounds like... |
Use this |
| "What's the most common / top / most frequent...?" |
top |
| "What's the least common / rarest...?" |
rare |
| "How many of each...?" / needs grouping by 2+ fields |
stats count by |
| "Just show me a clean list, no duplicates, sorted, readable" |
fields + dedup + sort + table + rename |
| "Only include records where X" |
Add a filter right after sourcetype=, before any other command |
Exercise: Translate three real questions into SPL
"Who visits us the most?" — a single winner, so limit=1.
index=tutorial sourcetype=access_combined_wcookie
| top limit=1 clientip
| rename clientip as "Most Frequent Visitor", count as "Number of Visits"
index=tutorial sourcetype=access_combined_wcookie status!=200
| stats count by productId
| sort - count
"How many requests failed (non-200) per product?" — notice we filter with status!=200 BEFORE piping into stats. Filter early, aggregate after — this pattern shows up constantly.
index=tutorial sourcetype="vendor_sales/vendor_sales"
| fields VendorID, Code, AcctID
| dedup VendorID
| sort - AcctID
| table VendorID, Code, AcctID
"Which vendors had the highest account numbers?" — no counting language at all, so reaching for stats/top here out of habit would be the wrong instinct.
Geo & Single-Value Commands
Turning an IP address into a country, and building the classic SOC attack map.
iplocation — where is this traffic actually coming from?
Takes an IP address field and looks it up against a built-in geolocation database (no install needed), adding location fields to each event. By default it adds City, Country, lat, lon, and Region.
Warning: Private/internal IP addresses (10.x.x.x, 192.168.x.x) get NO fields added at all — not blank values, simply nothing, since they're not in the geolocation database. If a search seems to silently 'lose' events after iplocation, check whether those clientips are internal.
Tip: This is the command behind 'where is this attack coming from?' When a SOC analyst sees a spike in failed logins from an unfamiliar IP, iplocation turns a meaningless number into a country and city.
Exercise: Add location data to web traffic
Run it. To request every available field, not just the defaults, try | iplocation allfields=true clientip.
index=tutorial sourcetype=access_combined_wcookie status>=400
| iplocation clientip
| table clientip, status, City, Country
geostats — plotting points on a map
Aggregates geographic data for a map visualization. Works almost exactly like stats — same aggregation functions — but expects latitude/longitude fields and accepts only ONE by-clause field, unlike stats which allows several.
Tip: This is how a SOC builds the classic 'attack map' — a sudden cluster of activity from a country with no normal business presence is a fast visual red flag.
Exercise: Build a point map of traffic by country
Switch to the Visualization tab → Cluster Map. Points appear sized and colored by count.
index=tutorial sourcetype=access_combined_wcookie
| iplocation clientip
| geostats count by Country
geom — shading entire regions
Adds polygon/shape data so Splunk can render a choropleth map — entire regions shaded based on a metric, rather than individual plotted points. Splunk ships two built-in shape files: geo_us_states and geo_countries.
- geostats plots individual POINTS on a map (e.g. one dot per city)
- geom shades entire REGIONS based on a metric (e.g. an entire country colored by total traffic)
Exercise: Build a choropleth of traffic by country
Switch to Visualization → Choropleth Map. Countries appear shaded by relative traffic volume.
index=tutorial sourcetype=access_combined_wcookie
| iplocation clientip
| stats count by Country
| geom geo_countries featureIdField="Country"
addtotals — a lightweight sanity-check command
Adds a total — either as a new field per row (the default) or as a new summary row at the bottom. If you don't specify which fields to sum, ALL numeric fields get summed automatically.
Warning: addtotals defaults to row=true. If you wanted one grand-total row instead, you must explicitly set row=f col=t.
Exercise: Add row and column totals
Row totals (default). A new Total field appears on each row.
index=tutorial sourcetype="vendor_sales/vendor_sales"
| stats count by Code, VendorID
| addtotals
index=tutorial sourcetype=access_combined_wcookie
| stats count by status
| addtotals row=f col=t labelfield=status
Column total instead — explicitly set row=f col=t. One new summary row appears at the bottom.
Filtering & Enrichment
eval, search vs where, fillnull, and turning raw codes into friendly labels with lookups.
eval — the most versatile command in SPL
Creates NEW fields by calculating, transforming, or deriving values from existing ones — arithmetic, string manipulation, conditionals, type conversion, all live here.
Tip: eval is how raw log noise becomes an analyst-readable severity label. Real SOC dashboards almost never show raw status codes to a triage analyst.
Exercise: Build a severity classification from scratch
A brand-new field, bytes_kb, calculated rather than extracted from raw data.
index=tutorial sourcetype=access_combined_wcookie
| eval bytes_kb=bytes/1024
| table clientip, bytes, bytes_kb
index=tutorial sourcetype=access_combined_wcookie
| eval traffic_type=if(status>=400, "Error", "Normal")
| table clientip, status, traffic_type
Every event now gets labeled Error or Normal — a derived classification that didn't exist in the raw log.
index=tutorial sourcetype=access_combined_wcookie
| eval severity=case(status>=500, "Critical", status>=400, "Warning", status<400, "OK")
| table clientip, status, severity
case() builds multi-tier classification in one line, evaluated top to bottom — first matching condition wins.
search vs where
These look similar but behave differently in two concrete, checkable ways.
|
search |
where |
| Wildcards |
Supported directly, e.g. status=4* |
NOT supported — where evaluates expressions, it doesn't pattern-match raw text |
| CIDR / IP range |
Supported directly, e.g. clientip="10.10.10.0/24" |
Needs a function like cidrmatch() |
| Typical use |
Filtering raw field values, early, for performance |
Comparing fields, or filtering on a field eval just created |
Tip: Simple rule: filtering on a raw field, or want wildcards/CIDR → search. Filtering on something eval just calculated, or comparing two fields → where.
Exercise: See where search wins, and where where is required
Direct CIDR match, no extra function needed.
index=tutorial sourcetype=access_combined_wcookie
| search clientip="10.10.10.0/24"
index=tutorial sourcetype=access_combined_wcookie
| eval bytes_kb=bytes/1024
| where bytes_kb > 50
You CANNOT do this with search — bytes_kb doesn't exist until eval creates it mid-pipeline.
fillnull — making blank cells look intentional
Replaces empty/null field values with a default — either 0 (the default if unspecified) or any string you choose.
Warning: fillnull can't act on a field that is null in EVERY single event, because Splunk doesn't consider a field to exist in the schema at all unless it has at least one non-null value somewhere in the result set.
Exercise: Replace blank geo fields with a readable label
iplocation leaves blanks for internal IPs. fillnull value="Unknown" turns those blanks into a clear label.
index=tutorial sourcetype=access_combined_wcookie
| iplocation clientip
| table clientip, City, Country
| fillnull value="Unknown"
Lookups — bolting a friendly label onto raw codes
A lookup is a separate reference table (usually a CSV) that maps a raw value to a friendlier one. Three pieces are involved: the lookup table FILE (the CSV), the lookup DEFINITION (tells Splunk the file exists and which column is the key), and the AUTOMATIC LOOKUP (wires the definition to a sourcetype so the new field just appears).
| Code |
category_name (a worked example) |
| L |
Software License |
| B |
Board Game |
| D |
Digital Download |
| F |
Figurine and Collectible |
| N |
Novelty Item |
Warning: Leaving output fields blank makes Splunk treat every non-match column as an implicit output, which can create lookup reference cycles. Always fill it in explicitly. Also: automatic lookups only take effect on searches run AFTER the automatic lookup is saved — re-run, don't just wait.
Tip: Pick a small, closed-set field for your first lookup — something like a 10-value transaction code, not a field with hundreds of distinct values.
Exercise: Build a lookup end to end
Test it manually before automating. Then re-run the same search with no | lookup line at all — status_description should still appear automatically once the automatic lookup is saved.
index=tutorial sourcetype=access_combined_wcookie
| lookup http_status_lookup status OUTPUT status_description
- Settings → Lookups → Lookup table files → Add new. Destination app: search. Upload the CSV with a destination filename that matches exactly.
- Settings → Lookups → Lookup definitions → Add new. Type: File-based. Select the uploaded CSV from the dropdown.
- Settings → Lookups → Automatic lookups → Add new. Apply to: sourcetype. Lookup input fields: status = status. Lookup output fields: status_description = status_description — fill this in explicitly, never leave it blank.
Correlating Events
transaction vs stats — the most conceptually subtle decision in SPL, and why a brute-force attack is a sequence, not a single event.
What a transaction actually is
A transaction is a group of conceptually related events treated as one unit — every web request a single visitor made during one browsing session, or every step of a single purchase. transaction marks a series of events as interrelated based on a shared piece of common information.
Warning: A brute-force login attempt isn't one event — it's dozens of failed-login events that, taken together, represent one attack. A SOC analyst doesn't care about event #47 in isolation; they care that one IP made 40 failed attempts in 90 seconds, then succeeded.
Grouping events by field, then by field and time
Warning: Events MUST be sorted in descending chronological order (newest first) before maxspan or maxpause are used, or the command silently returns INCORRECT results — not an error, just wrong groupings.
Exercise: Build transactions with increasing precision
Simplest transaction — one shared field, no time constraint, groups ALL events from that IP across the entire search window.
index=tutorial sourcetype=access_combined_wcookie
| transaction clientip
index=tutorial sourcetype=access_combined_wcookie
| transaction clientip host
Multiple fields — events must share BOTH to be grouped, a tighter result.
index=tutorial sourcetype=access_combined_wcookie
| transaction clientip maxspan=30s maxpause=5s
maxspan limits the TOTAL span (first to last event); maxpause limits the GAP between any two consecutive events.
Defining a transaction by its content, not just time
Sometimes a transaction's boundaries are defined by content rather than duration — e.g. a session starts on a login event and ends on a logout event, regardless of how long that takes.
Tip: Attack chains are usually defined by startswith/endswith logic, not fixed time windows — regardless of whether the chain took 30 seconds or 30 minutes.
Exercise: Group by content boundaries
Note duration is filtered AFTER transaction, since transaction itself creates that field.
index=tutorial sourcetype=access_combined_wcookie
| transaction JSESSIONID clientip startswith="view" endswith="purchase"
| where duration>0
transaction vs stats — the decision that matters most
stats calculates statistical values on events grouped by field values, and DISCARDS the original events. transaction also groups events by field values, but does not compute statistics beyond duration — it KEEPS the raw events and their original field values, combined together.
| Use transaction when... |
Use stats when... |
| A shared ID alone isn't enough to tell transactions apart |
You have a genuinely unique identifier and just need numbers |
| A transaction's boundaries are defined by CONTENT, not just an ID |
You don't need to see the original raw events afterward |
| You need the actual combined raw text of the grouped events |
Performance matters — stats is faster in most cases |
Warning: In cases where either command could solve the problem, stats is usually faster, especially in distributed search environments. Don't reach for transaction by default — reach for it when stats genuinely can't express what you need.
Tip: A genuinely cool capability stats cannot match: transaction recognizes transitive relationships — if events A and B share a field, and B and C share a different field, transaction can chain A, B, and C together even though A and C share nothing directly.
Knowledge Objects & Governance
Naming, permissions, field extraction, aliases, and the tags/event-types layer a real team depends on.
What counts as a knowledge object
If you've built and saved anything in Splunk, it was probably a knowledge object: saved searches and reports, alerts, dashboards, lookups, field extractions, field aliases, calculated fields, tags, event types, macros, workflow actions, and data models.
Tip: A consistent naming convention helps a team differentiate between similar reports and identify what team owns it, what technology it involves, and what it's designed to do.
A real naming formula worth adopting
Applied example: a SOC alert checking failed logins on the Windows platform over a 1-hour window might become SOC_alert_windows_auth_1h_FailedLogons.
| Component |
What it captures |
| Group |
The team that owns this object (e.g. SOC, NetSec, IAM) |
| Search type |
alert, report, or summary-index-populating |
| Platform |
The technology/system the search targets |
| Category |
The concern area (e.g. auth, malware, network) |
| Time interval |
The window the search runs over, if scheduled |
| Description |
1–2 words, meaningful, succinct |
Warning: All objects within a knowledge object CATEGORY must have unique names. If two objects share a name, only ONE is actually applied — silently, with no warning. A shadowed duplicate can mean your team THINKS something is running, when an outdated version is actually executing.
Permissions — three sharing levels
- By default, only admin and power roles can share or promote knowledge objects
- Admins can change permissions on ANY object. Power users can only change permissions on objects THEY own
- If neither Read nor Write is checked for a role, that role cannot see or use the object at all
| Level |
Who can see / use it |
| Private |
Only the object's owner — no one else |
| App |
Anyone with Read permission, but only while using the same app the object was created in |
| Global (All apps) |
Anyone with Read permission, across every app in the deployment |
Tip: The standard production pattern: Read on, Write off, for the Everyone role. Broad usability, controlled modification.
Orphaned objects — a real operational failure mode
An orphaned knowledge object occurs when its owner account becomes invalid — e.g. an analyst leaves the team.
Warning: The Reassign Knowledge Objects page (Settings → All Configurations) cannot reassign objects that are both orphaned AND privately shared. This is a strong argument for not leaving important detection logic private long-term.
The Field Extractor — when Splunk doesn't already know your fields
Real-world log sources often don't come with built-in extraction rules. Entry path: run a search, click All Fields in the sidebar, click Extract New Fields.
| Ask yourself |
Answer |
| Is the data structured (table-like, consistent field order)? |
YES → Delimiters. NO → Regular Expression. |
| Even if delimiter-separated, is field POSITION consistent? |
If NO, use Regular Expression with Required Text instead. |
Warning: Delimiter extraction assumes every event has the same number of fields in the same order. If positions don't line up, the extraction can silently pull the wrong value into the wrong slot.
Field aliases
An alias is an alternate name for a field that already exists — it does NOT replace or remove the original. This solves a real problem: different sourcetypes often use different names for the same concept.
Warning: Two hard limits: you can't alias a calculated field, event type, tag, or lookup-added field. And one alias name can only map to ONE original field. Never name an alias the same as an internal field like _time.
Exercise: Alias clientip to src_ip
Both columns show identical values, and the original clientip field is still there, untouched.
index=tutorial sourcetype=access_combined_wcookie
| table clientip, src_ip
- Settings → Fields → Field Aliases → Add new.
- Source type: access_combined_wcookie. Existing field name: clientip. New alias name: src_ip.
Calculated fields
A calculated field performs a calculation using values of fields already present in events, computed at search time — a reusable shortcut for an eval expression.
Warning: You can't scope a calculated field to an ALIASED source/host/sourcetype — put conditional logic inside the eval expression using if() instead.
Exercise: Build a calculated field with coalesce()
This picks whichever field actually has a value, in priority order — the fix when an alias can't map one name to two fields.
- Settings → Fields → Calculated Fields → Add new. Field name: unified_ip. Eval expression: coalesce(clientip, src_ip).
Tags and event types
|
Tag |
Event Type |
| Bound to |
ONE single key=value pair, no wildcards |
A full search expression — many fields, wildcards, conjunctions |
| Think of it as |
A sticky note on a specific field value |
A saved, reusable search condition |
| Search syntax |
tag::fieldname=tagvalue |
eventtype=MyEventType |
Warning: Event type definitions CANNOT contain a pipe or a subsearch — they categorize raw events, they don't transform or summarize them.
Tip: Layer them: a broad event type (any error) tagged error, and a narrower one (status>=500) tagged BOTH error and critical. Searching tag=critical finds only the narrow set; tag=error finds both.
Exercise: Build and search an event type
Search the saved category directly by name.
eventtype=server_errors
- Run index=tutorial sourcetype=access_combined_wcookie status>=500.
- Save As → Event Type → name it server_errors, optionally tag it critical.
Automation, Workflow, and Data Models
Macros, clickable workflow actions, data model hierarchies, and the industry-wide vocabulary called CIM.
Macros — stop retyping the same SPL fragment
A macro is a reusable chunk of SPL you insert into any search by name. It can be any part of a search — an eval statement, a search term, a whole pipeline.
Warning: A macro with no arguments and a macro with one argument are genuinely different definitions — you cannot just add an argument to an existing zero-argument macro. Backtick (`) and single-quote (') look similar but are completely different keys.
Exercise: Build macros with zero, one, and two arguments
Invoke it with BACKTICKS, not single quotes.
`server_errors_macro`
- Settings → Advanced Search → Search Macros → New. Name: server_errors_macro. Definition: index=tutorial sourcetype=access_combined_wcookie status>=500.
`status_filter(500)`
`status_filter(404)`
A one-argument macro named status_filter(1), with definition index=tutorial sourcetype=access_combined_wcookie status=$status_code$ — note the (1) in the name itself, which is part of the macro's identity.
Workflow actions — making field values clickable
Workflow actions create HTML links that run searches in external engines, generate HTTP POST requests, or launch secondary Splunk searches using a field value from a selected event.
| Type |
What it does |
Real-world shape |
| GET |
Opens an external resource, passing a field value in the URL |
Right-click an IP → jump to a WHOIS lookup |
| POST |
Sends field values as form data to an external system, silently |
Right-click an error → auto-file a ticket |
| Search |
Launches a new Splunk search using a field value |
Right-click an IP → see all its other activity |
Warning: Variables in GET URIs are automatically URL-encoded. If a field value is itself already a full HTTP address, use $!fieldname$ instead of plain $fieldname$ to prevent escaping.
Exercise: Build a GET workflow action for WHOIS lookups
Run a search, click a clientip value in any event — your new WHOIS action appears in the field menu.
- Settings → Fields → Workflow Actions → New. Apply to: field clientip. Action type: Link.
- URI: https://www.whois.com/whois/$clientip$. Link method: get.
Data models — the menu that Pivot reads from
When you select a dataset for Pivot, the unhidden fields you define for that dataset are exactly the fields you get to choose from in Pivot. A data model IS the menu Pivot reads from.
Warning: To accelerate a data model, it needs at least one root EVENT dataset, or a root SEARCH dataset using ONLY streaming commands. Root TRANSACTION datasets cannot be accelerated.
Exercise: Build a root event object and a child dataset
- Settings → Data Models → New Data Model. Add Dataset → Root Event. Constraints: index=tutorial sourcetype=access_combined_wcookie. Add fields: clientip, status, method.
- Add Dataset → Child. Object Name: Server Errors. Additional Constraint: status>=500 — this automatically inherits everything from the parent, you don't redefine clientip or method.
CIM — the same techniques you already know, at industry scale
Each data model in the Common Information Model defines the least common denominator of a domain of interest — the smallest, common set of fields every log of that type should be normalized down to, so they can all be searched and correlated the same way.
- This is exactly what field aliases and tags/event types already do, pre-built by Splunk and shared as a common vocabulary across the security industry
- Dashboards in CIM-compliant apps (Enterprise Security, PCI Compliance) display ONLY data normalized to CIM's tags and fields
- CIM data models do NOT tag your data automatically — they define a target schema and expect you to map your own data to it
Warning: If the CIM expects a string but your field is numeric, don't alias directly — alias to an intermediate name first, then use a lookup to produce the correctly typed final field.
Tip: A SOC ingesting logs from three vendors normalizes all of them to the same CIM field names — one search, one dashboard works across every vendor at once.
SOC Capstone — Investigate, Classify, Report
An end-to-end walkthrough: locate, classify, reconstruct, report — exactly how a real investigation runs.
Your role
You are the on-shift SOC analyst for Buttercup Games. Leadership has asked for a single incident review dashboard covering web traffic for today's review window. Work through this the way a real investigation runs: locate, classify, reconstruct, report. Each phase builds on the previous one.
Phase 1 — Locate: where is the traffic coming from?
Before deciding what's worth investigating, you need geographic context. An IP address with a 3-digit country code means nothing to a manager — a country name does.
Tip: Write down the top 2 countries by error count. One of these becomes your focus for the rest of the investigation.
Exercise: Find your top error-traffic countries
Add location data to all web traffic.
index=tutorial sourcetype=access_combined_wcookie
| iplocation clientip
| table _time, clientip, status, City, Country
index=tutorial sourcetype=access_combined_wcookie status>=400
| iplocation clientip
| stats count by Country
| sort -count
Find which countries generate the MOST error traffic — your first real lead. Save as a report named "Error Traffic by Country".
Phase 2 — Classify: turn raw codes into analyst-readable severity
Numbers don't triage themselves. Before you can prioritize, every event needs a severity label a human can scan quickly.
Warning: Everything from this point forward should work FROM this filtered, classified list — not from the full raw dataset.
Tip: Did fillnull catch any blank City/Country values? Internal/non-routable IPs showing up in public web traffic is sometimes worth flagging on its own.
Exercise: Build the Critical Severity Events report
Save this as a report named "Critical Severity Events".
index=tutorial sourcetype=access_combined_wcookie status>=400
| iplocation clientip
| eval severity=case(status>=500, "Critical", status>=400, "Warning")
| where severity="Critical"
| fillnull value="Unknown" City, Country
| table _time, clientip, status, severity, City, Country
Phase 3 — Reconstruct: what actually happened, session by session?
A single Critical event rarely tells the whole story. Real incidents are sequences — you need to see what an IP did before and after the event that caught your attention.
Tip: Decision check: could this summary have been built with stats alone, without transaction? Write 1–2 sentences justifying your answer.
Exercise: Reconstruct sessions for your critical clients
Pick one clientip from Phase 2 and reconstruct its full session behavior.
index=tutorial sourcetype=access_combined_wcookie clientip="PASTE_YOUR_IP_HERE"
| transaction clientip maxspan=30s maxpause=5s
| table _time, duration, eventcount, status
index=tutorial sourcetype=access_combined_wcookie status>=500
| transaction clientip maxspan=30s maxpause=5s
| stats avg(duration) as avg_duration, avg(eventcount) as avg_events, count as session_count by clientip
| sort -session_count
A summary across ALL clientips in your Critical list, to see if the pattern is isolated or widespread. Save as "Session Reconstruction - Critical Clients".
Phase 4 & 5 — Build the dashboard, write the summary
Assemble everything into one dashboard a manager or shift lead could open and immediately understand.
- Create a new dashboard named "Web Traffic Incident Review", with a short description of what it covers.
- Add all reports from Phases 1–3. Arrange panels in investigative order: geographic overview first, severity breakdown second, session reconstruction last.
- Write a 4–6 sentence incident summary as if handing this to your manager at shift handoff — what you found, where it's from, how severe it is, whether the session reconstruction supports your initial suspicion. Use your own real numbers.
Self-check before you submit
- Does your dashboard tell a clear story in the order someone would actually investigate it?
- Did you use iplocation, eval/case, where, fillnull, AND transaction somewhere — not just one or two?
- Can you explain, out loud, WHY you chose transaction over stats (or vice versa) at the point you made that choice?
- Does your incident summary use real numbers from YOUR results, not guesses?
Command Reference
search (Filtering)
Filters events using raw field matching, wildcards, and CIDR ranges.
The implicit command at the start of every query. Matches raw field values directly — supports wildcards (status=4*) and CIDR notation (clientip="10.0.0.0/24") natively. Best for filtering on values exactly as they appear in the log, early in the pipeline, for performance.
where (Filtering)
Filters using evaluated expressions — required for fields eval just created.
Evaluates a boolean expression rather than pattern-matching raw text. Does NOT support wildcards or CIDR the way search does. Use it when comparing two fields to each other, or filtering on a field that eval created earlier in the same pipeline.
table (Display)
Displays chosen fields as a clean grid, in the order you list them.
A display-only command — it shows the fields you pick as columns, in the order given. Does not populate the Statistics tab. Often added after fields, dedup, or sort to actually see what those commands did.
fields (Display)
Trims the underlying field set, mostly invisible until you add table.
Trims which fields are kept in the result set. Changes the sidebar's Interesting Fields list, but the Events tab still shows full raw text — the trimming is invisible until combined with table, or with a transforming command.
dedup (Display)
Keeps only the first event per unique value of a field.
Removes duplicate events based on one or more fields, keeping the first (most recent, by default sort order) match. A display-only command — check the Statistics tab won't reflect it; pipe into table to see the effect directly.
sort (Display)
Orders results. A leading minus sign means descending.
Sorts results by one or more fields. sort field is ascending; sort - field is descending. Order matters for downstream commands like transaction, which require descending chronological order to behave correctly.
rename (Display)
Changes a column's display label only — the underlying data is untouched.
Purely cosmetic: relabels a field's header text for display. The original field name still works in later pipeline stages exactly as before — rename does not propagate the new name backward or forward into the field's identity.
top (Transforming)
Most common values for a field, with automatic count and percent.
A transforming command — populates the Statistics tab automatically. Returns the top 10 most frequent values by default, sorted descending, with count and percent columns added. Functionally a shortcut for stats count by field | sort - count.
rare (Transforming)
Least common values for a field — the mirror image of top.
Same output shape as top, but sorted ascending by count instead of descending. Rare values are often more interesting in security work than common ones — a rare status code or login pattern is frequently where real problems hide.
stats (Transforming)
Flexible aggregation — count, sum, avg, and more, grouped by any fields.
The most flexible transforming command in SPL. Supports any number of BY-clause fields and a wide range of aggregation functions (count, sum, avg, min, max, dc, and more). top and rare are really just shortcuts for specific stats patterns.
chart (Transforming)
Builds a chart-ready table: one row-split, one column-split field.
Like stats, but capped at two BY-clause fields specifically structured for charting — one field becomes the row-split (x-axis categories), the other becomes the column-split (separate data series).
timechart (Transforming)
Time-series aggregation — _time is always the row-split automatically.
Builds a time-series table. _time is automatically the first column and row-split; only one additional BY-clause field (the column-split) is allowed. Use span= to control the time bucket size, e.g. span=1h.
eval (Calculation)
Creates new fields by calculating or deriving values from existing ones.
The most versatile command in SPL — handles arithmetic, string manipulation, conditionals (if, case), and type conversion. eval only calculates; it doesn't filter. Pair it with where afterward if you need to remove non-matching results.
fillnull (Calculation)
Replaces empty/null values with a default — 0, or any string you choose.
Fills blank field values, defaulting to 0 if no value is specified, or fillnull value="X" for a custom fill. Cannot act on a field that is null in every single event in the result set, since Splunk doesn't consider such a field to exist in the schema at all.
case (Calculation)
Multi-tier conditional logic inside eval — first match wins.
An eval function: case(condition1, value1, condition2, value2, ...) evaluates conditions top to bottom and returns the value for the first one that's true. Note: the function name CASE() is itself case-sensitive when used for forced case-sensitive matching elsewhere.
coalesce (Calculation)
Returns the first non-null value from a list of fields.
An eval function, not a standalone command — used inside eval or a calculated field definition. coalesce(fieldA, fieldB) returns whichever field actually has a value, useful when the same concept is split across differently-named fields.
lookup (Enrichment)
Adds fields from an external CSV by matching on a key field.
Manually invokes a lookup definition, matching a key field against a reference CSV and pulling in output columns. Once an automatic lookup is configured for the same definition, this manual invocation becomes unnecessary — the fields just appear.
iplocation (Enrichment)
Adds City, Country, lat, lon, and Region from an IP address field.
Looks up an IP field against Splunk's built-in geolocation database. Private/internal IPs (10.x.x.x, 192.168.x.x) get no fields added at all — not blank values, simply nothing — since they aren't present in the database.
geostats (Enrichment)
Aggregates geographic data for point-based map visualizations.
Works like stats but expects latitude/longitude fields and accepts only one BY-clause field. Built specifically to feed map visualizations like Cluster Map — plots individual points, e.g. one dot per city.
geom (Enrichment)
Adds polygon/shape data for choropleth (region-shaded) maps.
Adds featureCollection and geom fields containing JSON polygon data, used to shade entire regions (countries, states) on a Choropleth Map visualization, rather than plotting individual points the way geostats does.
addtotals (Calculation)
Adds a row or column total — defaults to a per-row total field.
Defaults to row=true, adding a Total field summing all numeric fields in each row. Set row=f col=t for a single grand-total summary row instead. Restrict which fields get summed by listing them explicitly, e.g. addtotals count_*.
transaction (Correlation)
Groups conceptually related events into one unit, keeping raw events.
Groups events sharing one or more fields (optionally constrained by maxspan, maxpause, or startswith/endswith content boundaries) into a single combined event. Unlike stats, it keeps the original raw events rather than discarding them — at a real performance cost.
head (Display)
Returns only the first N results.
Limits output to the first N events (or results, if used after a transforming command). Commonly used early in exploration — head 5 — to preview raw data before building a real query.