Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Vincent Bumgarner

• Publisher: Packt Publishing
• Year: 2013
• eBook ISBN: 9781849693288
• Edition: 1
• Topic: Computer Science
• Subtopic: Information Technology
• Index: Computer Science

In Detail

Splunk is a data collection, indexing, and visualization engine for operational intelligence. It's a powerful and versatile search and analysis engine that lets you investigate, troubleshoot, monitor, alert, and report on everything that's happening in your entire IT infrastructure from one location in real time. Splunk collects, indexes, and harnesses all the fast moving machine data generated by our applications, servers, and devices - physical, virtual, and in the cloud.

Given a mountain of machine data, this book shows you exactly how to learn to use Splunk to make something useful from it. Depending on your needs, you can learn to search, transform, and display data, or learn to administer your Splunk installation, large or small.

"Implementing Splunk: Big Data Reporting and Development for Operational Intelligence" will help you get your job done faster, whether you read from the beginning or jump to what you need to know today. New and experienced users alike will find nuggets of wisdom throughout.

This book provides you with valuable examples and step-by-step instructions, showing you how to take advantage of everything Splunk has to offer you, to make the most out of your machine data.

"Implementing Splunk: Big Data Reporting and Development for Operational Intelligence" takes you on a journey right from inception to a fully functioning implementation of Splunk. Using a real-world data walkthrough, you'll be shown how to search effectively, create fields, build dashboards, reports, and package apps, manage your indexes, integrate into the enterprise, and extend Splunk. This practical implementation guide equips you with high-level knowledge for configuring, deploying, extending, and integrating Splunk. Depending on the goal and skills of the reader, enough topics are covered to get you on your way to dashboard guru, app developer, or enterprise administrator. This book uses examples curates reference, and sage advice to help you make the most of this incredibly powerful tool.

Approach

A step-by-step practical implementation tutorial that equips you with high-level knowledge of configuring, deploying, extending, and integrating Splunk to bring machine-generated operational intelligence (?)to your advantage.

Who this book is for

The book targets professionals and organizations who want to implement or have already implemented Splunk for log analysis and indexing. Analysts and IT staff for end-to-end investigation, performance monitoring, and so on will also learn from the practical examples. It would even help managers to build reports and summarize the health, performance, and activity of their IT infrastructure and business. You will also find it helpful as a technical administrator, consultant, or end user.

This book aims to be useful to Splunk users of all levels, from complete newbie to seasoned user. The book assumes that you have access to a copy of Splunk, ideally not in production. Many examples also assume your user has admin rights.

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

---

Table of Contents

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. The Splunk Interface

Logging in to Splunk

The Home app

The top bar

Search app

Data generator

The Summary view

Search

Actions

Timeline

The field picker

Fields

Search results

Options

Events viewer

Using the time picker

Using the field picker

Using Manager

Summary

2. Understanding Search

Using search terms effectively

Boolean and grouping operators

Clicking to modify your search

Event segmentation

Field widgets

Time

Using fields to search

Using the field picker

Using wildcards efficiently

Only trailing wildcards are efficient

Wildcards are tested last

Supplementing wildcards in fields

All about time

How Splunk parses time

How Splunk stores time

How Splunk displays time

How time zones are determined and why it matters

Different ways to search against time

Specifying time in-line in your search

_indextime versus _time

Making searches faster

Sharing results with others

Saving searches for reuse

Creating alerts from searches

Schedule

Actions

Summary

3. Tables, Charts, and Fields

About the pipe symbol

Using top to show common field values

Controlling the output of top

Using stats to aggregate values

Using chart to turn data

Using timechart to show values over time

timechart options

Working with fields

A regular expression primer

Commands that create fields

eval

rex

Extracting loglevel

Using the Extract Fields interface

Using rex to prototype a field

Using the admin interface to build a field

Indexed fields versus extracted fields

Indexed field case 1 – rare instances of a common term

Indexed field case 2 – splitting words

Indexed field case 3 – application from source

Indexed field case 4 – slow requests

Indexed field case 5 – unneeded work

Summary

4. Simple XML Dashboards

The purpose of dashboards

Using wizards to build dashboards

Scheduling the generation of dashboards

Editing the XML directly

UI Examples app

Building forms

Creating a form from a dashboard

Driving multiple panels from one form

Post-processing search results

Post-processing limitations

Panel 1

Panel 2

Panel 3

Final XML

Summary

5. Advanced Search Examples

Using subsearches to find loosely related events

Subsearch

Subsearch caveats

Nested subsearches

Using transaction

Using transaction to determine the session length

Calculating the aggregate of transaction statistics

Combining subsearches with transaction

Determining concurrency

Using transaction with concurrency

Using concurrency to estimate server load

Calculating concurrency with a by clause

Calculating events per slice of time

Using timechart

Calculating average requests per minute

Calculating average events per minute, per hour

Rebuilding top

Summary

6. Extending Search

Using tags to simplify search

Using event types to categorize results

Using lookups to enrich data

Defining a lookup table file

Defining a lookup definition

Defining an automatic lookup

Troubleshooting lookups

Using macros to reuse logic

Creating a simple macro

Creating a macro with arguments

Using eval to build a macro

Creating workflow actions

Running a new search using values from an event

Linking to an external site

Building a workflow action to show field context

Building the context workflow action

Building the context macro

Using external commands

Extracting values from XML

xmlkv

XPath

Using Google to generate results

Summary

7. Working with Apps

Defining an app

Included apps

Installing apps

Installing apps from Splunkbase

Using Geo Location Lookup Script

Using Google Maps

Installing apps from a file

Building your first app

Editing navigation

Customizing the appearance of your app

Customizing the launcher icon

Using custom CSS

Using custom HTML

Custom HTML in a simple dashboard

Using ServerSideInclude in a complex dashboard

Object permissions

How permissions affect navigation

How permissions affect other objects

Correcting permission problems

App directory structure

Adding your app to Splunkbase

Preparing your app

Confirming sharing settings

Cleaning up our directories

Packaging your app

Uploading your app

Summary

8. Building Advanced Dashboards

Reasons for working with advanced XML

Reasons for not working with advanced XML

Development process

Advanced XML structur...

Table of contents

• Implementing Splunk: Big Data Reporting and Development for Operational Intelligence