SPLUNK ADMIN

What happens when hot buckets are rolled to warm?

- Hot buckets also roll to warm automatically when the indexer is restarted - Hot and warm buckets are stored in the db directory for the index - Hot buckets are renamed when rolled to warm

Strict Time-based Retention Policies. What are issues to consider?

- Splunk freezes entire buckets, not individual events - If a bucket spans more than one day, you can't meet the 90 day requirement

Free License

-Disables alerts, authentication, clustering, distributed search, summarization, and forwarding to non-Splunk servers -Allows 500MB/day of indexing and forwarding to other Splunk instances

Enterprise Trial License

-Downloads with product -Features same as enterprise except for 500 MB per day limit -Only valid for 60 days, after whcih one of the other 3 license types must be activated -*Sales trial license* is a trial Enterprise license of varying size and duration

System Admin vs. Data Admin SYSTEM ADMIN responsibilities:

-Install, configure, and manage Splunk components -Install and manage Splunk apps -Manage Splunk licensing -Manage Splunk indexes -Manage Splunk users and authentication -Manage Splunk configuration files -Monitor MC and respond to system health alerts

Enterprise License

-Purchased from Splunk -Full functionality for indexing, search head, deployment server, etc. -Sets the daily indexing volume -No enforcement license, allows users to keep searching even if you are in a license violation period.

What DOES NOT count against your Daily License quota?

-Replicated data (Index Clusters) -Summary Indexes -Splunk internal logs (_internal, _audit, etc. indexes) -Structural components of an index (metadata, tsidx, etc.)

Forwarder License

-Sets the server up as a heavy forwarder -Applies to non-indexing forwarders -Allows authentication, but no indexing

System Admin vs. Data Admin DATA ADMIN responsibilities:

-Work with users requesting new data sources -Document existing and newly ingested data sources -Design and manage inputs UFs/HFs to capture data -Manage parsing, event line breaking, timestamp extraction -Move configuration through non-production testing as required -Deploy changes to production -*Manage Splunk configuration files

What are indexes? The system admin can

-create new indexes -Control which indexes users can access

Index Time Precedence Order

1. etc/system/local 2. etc/apps/search/local 3. etc/apps/unix/local 4. etc/apps/search/default 5. etc/apps/unix/default 6. etc/system/default

LICENSE WARNINGS and VIOLATIONS How many warnings on a Free license is a violation?

3, in a rolling 30-day period

LICENSE WARNINGS and VIOLATIONS How many warnings on an enforced Enterprise license is a violation?

5, in a rolling 30-day period

What is a bucket?

A bucket is a directory containing a set of raw data and associated index files *buckets have a maximum data size and a time span

Why create indexes?

Access control - segregate events into separate indexes to limit access by Splunk role Retention - Policy is set per index - Separate events into different indexes based on desired retention time

What counts as Daily License quota?

All data from all sources that is indexed -It is the data (full size) that flows through the parsing pipeline, per day -It is not the amount of storage used by the indexes

Index Time Merging of Configurations If there are conflicts, which setting is used?

If there are conflicts, the setting with the highest precedence is used.

How many Master input configuration models exist in memory at run time?

Only one. Regardless of the number of inputs.conf files in various apps or system path.

License Pooling What is a Pool?

Pools allow licenses to be subdivided and assigned to a group of indexers -can be created for a given stack -Warnings and violations occur per pool

How do you reload configuration files after editing? Splunk Web CLI

Restarting Splunk by: - Splunk Web: Settings > Server controls > Restart Splunk - CLI: splunk restart

Where are all .conf files documentation and examples found?

SPLUNK/HOME/etc/system/README

When adding a license, Licenses are stored under...

SPLUNK_HOME/etc/licenses

Overridding Defaults There are default settings in ...

SPLUNK_HOME/etc/system/default SPLUNK_HOME/etc/apps/search/default

At ________, Splunk scans the entire time range on a bucket name to determine whether or not to open the bucket and search its events

Search time

Searching / Indexing / Inputs. Which one? Allow users to submit search requests using SPL

Searching

Searching / Indexing / Inputs. Which one? Consolidate results and render visualizations of results

Searching

Searching / Indexing / Inputs. Which one? Distribute search requests to the indexers

Searching

Searching / Indexing / Inputs. Which one? Seach time knowledge objects and stored on the search heads. IE. field extractions, alerts and dashboards

Searching

Managing Licenses Where do you go and what are the steps?

Select Settings > Licensing 1. Designate the license server type - Master or slave 2. Change license group 3. Add a license 4. Check license alerts and violations 5. View stacks 6. Edit and add pools

Splunk admins can create indexes from Splunk Web. How is this done?

Settings > Indexes > New Index

LICENSE WARNINGS and VIOLATIONS What happens to Splunk Enterprise 6.5.0 and later during the violation period?

Splunk Enterprise 6.5.0 and later provides warnings but it does not disable search during the violation period. *Prior versions of Splunk would disable search*

Preconfigured Indexes _internal

Splunk indexes its own logs and metrics from its processing here

Preconfigured Indexes _audit

Splunk stores its audit trails and other optional auditing information

Can splunk users specify the index to search?

Yes

Each configuration file governs....

a particular aspect of Splunk functionality

LICENSE WARNINGS and VIOLATIONS If the indexing exceeds the the allocated daily quota in a pool... What happens next?

an alert is raised in Messages (pool warning) on any page in Splunk Web *The daily license quota resets at midnight*

An index stores events in ___________

buckets

Preconfigured Indexes _thefishbucket

contains checkpoint information for file monitoring inputs

Preconfigured Indexes _main

default index for inputs, located in the defaultdb directory

Preconfigured Indexes _summary

default index for summary indexing system

What does the Last Chance Index default to?

defaults to Empty

What are indexes? Splunk stores the input data as _______ in indexes

events

Where are configuration changes saved?

in .conf under SPLUNK_HOME/etc/

Index Time Precedence - Adding an App If two or more apps at the same level of precedence have conflicts between them, the conflicts are resolved in .....

lexicographical order by app directory name

The correct method to override these settings is to do so in the ___________ directory at the same scope

local The correct method to override these settings is to do so in the local directory at the same scope

Index Time Merging of Configurations _________ always takes precedence over _________

local, default Local always takes precedence over default.

Splunk admins can create indexes from CLI. How is this done?

splunk add index <index_name>

This command shows on-disk configuration for requested file

splunk btool conf-name list [options]

How does metrics data count against a license? And where does it draw from?

Metrics data counts against a license at a fixed 150 bytes per metric event *Metrics data draws from the same license quota as event data*

What is running splunk btool useful for

Checking the configuration scope and permission rules

When Splunk starts, what happens to configuration files?

Configuration files are merged together into a single run time model for each file type.

What is Last Chance Index?

Gives the ability to define a last chance index for events destined for non-existent indexes.

Searching / Indexing / Inputs. Which one? Receive, index, and store incoming data from forwarders

Indexing

Searching / Indexing / Inputs. Which one? Reside on dedicated machines

Indexing

Searching / Indexing / Inputs. Which one? Search data in response to requests received from the search heads

Indexing

Searching / Indexing / Inputs. Which one? Best practice data collection method

Inputs

Searching / Indexing / Inputs. Which one? Requires minimal resources and typically installed on the machines that produce the data

Inputs

Searching / Indexing / Inputs. Which one? Splunk instances that monitor configured inputs and forward the data to the index

Inputs

What happens when Last Chance Index setting is not defined or is empty?

It will drop such events

Index Time Merging of Configurations What is the result if there are no duplicate stanzas or common settings between the files?

The result is the union of all files

When hot buckets reach their max size or time span, what happens?

They are closed and converted to warm status

T or F .conf files are text files and are case sensitive

True

T or F Splunk ships with some indexes already installed?

True

T or F? The files in the "default" directory must never be modified.

True

Buckets on Different Storage Systems. What is a best practice?

Use a high performance file system to store indexes. *The bucket span and storage type can affect search performance.

When a warm bucket rolls to cold, what happens?

the entire bucket is moved maintaining its name

Preconfigured Indexes _introspection

tracks system performance, Splunk resource usage data, and provides MC with performance data