[Splunk] – Basic search fields and commands

For those who are new to Splunk, let’s look at the necessary commands to use in the basic field.

Field

A field is a very powerful function that makes log search convenient, and it is composed of [field name = value] and can be used for search.

If you don’t use a field, it checks to see if the searched string is included in all areas, but if you specify a field, you can search logs that contain the characters you want within that field.

For example, if you want to see the 404 status code of the web server, input only 404 and then perform a search. Can’t get results. Only the log whose index field is book and the referer address starts with http://www.ipbalance and whose status value is 404 is output.

 

index=”book” referer=http://www.ipBalance* status=”404″

index=web sourcetype=access_combined clientip=10.*

(index=web OR index=security) NOT status=200  <– Showing data with not having status=200 

(index=web OR index=security) status!=200 <– Showing data with not value is 200

index=web (status=500 OR status=503 OR status=505)

 

 

Field values are not case-sensitive, but field names are case-sensitive. If the field contains an IP address, it can be searched in CIDR format.

Pipe

When searching for logs, there are cases in which very complex conditions are applied and searched in order to understand in detail. And use pipelines when field breaks aren’t always extracted the way you want them to.

The example below extracts a log with a status value of 404 among logs where the index field is web and the sourcetype indicating the data format is cookie.

After that, the result extracted through the pipeline is outputted by the number of URLs.

 

index=”web” sourcetype=”cookie” status=”404″ | stats count by url

more example

index=”web” sourcetype=”cookie” status=”404″
| stats sum(cost) as total by productID
| where total > 100

 

Table

Combined with the field name, the search results are displayed in a table format.

index=”web” sourcetype=”cookie” | table Userip method, productId, status

 

Rename

Change the field name to something else.

It can be used when you want to give meaning to a field name required in the log or write Hangul in the field.

If you want to include spaces in the field name you want to convert, just add quotation marks (“).

index=”web” sourcetype=”cookie” | table IP, method, productId, status | rename IP AS UserIP, method AS “Get Or Post”, productId AS ProductID, status AS “Web Status”

index=web status IN (“500” OR “503” OR “505”)
| fields -status
| stats count by status
| rename status as  “HTTP Status”, count as “Number of Events”

 

Dedup

It is used to remove duplicates from search results.

If you remove the duplicates of the specified field, even though the values of other fields are not duplicates, if you remove the previous duplicated result, the seats will also disappear.

index=”web” sourcetype=”cookie” status=”404″ | dedup ProductID

 

Sort

Sort the search results.

The + (default) option is in ascending order, and the – option is in descending order.

index=”web” sourcetype=”cookie” status=”404″
| table date_hour, Userip, productId, method, status
| sort date_hour, -method

 

Fields

It is used to include or remove specific fields from search results.

index=”web” sourcetype=”cookie” status=”404″
| fields productid

index=”web” sourcetype=”cookie” status=”404″
| fields – productid

index=web status IN (“500” OR “503” OR “505”)
| fields -status
| stats count by status

 

Stats

Calculate statistics using various statistical functions.

stats [count|dc|sum|avg|list|value] by [Field name]

index=”web” sourcetype=”cookie”
| stats sum(bytes), avg(bytes), max(bytes), median(bytes), min(bytes) by Userip

By using the stats command and the sort command, you can check the senders who sent the most packets in a given time period in descending order.

index=”web” sourcetype=”cookie”
| stats sum(bytes) by Userip
| sort -sum(bytes)

 

Top

toplimit=<number> [showperc=T/F] [showcount=T/F] [useother=T/F] Field1, field2, byfield

index=”web” sourcetype=”cookie” status=”404″ Userip=”108.33.46.67″
| top limit=10 showperc=T showcount=F date_hour by Userip

another example

index=”web” sourcetype=”cookie” status=”404″
| top limit=5 showcount=T showperc=F Userip
| rename clientip AS “Source IP”, count AS “access_count”

 

Rare

rare limit=<number> [showperc=T/F] [showcount=T/F] [useother=T/F] Field1, field2, by field

 

eval

eval [formula]

index=network sourcetype=cisco_wsa_squid
| stats sum(sc_bytes) as Byte by usage
| eval bandwidth = Bytes/1024/1024

 

erex

erex / easy, not require RegEx knowledge, requires sample data to generate RegEx

index=games sourcetype=SimCube
| erex Character fromfileld=_raw examples=”pixie, Kooby

 

rex

rex / require RegEx knowledge, not require sample data, when possible use rex

index=games sourcetype=SimCube
| rex field=_raw “^[^’\n]*'(?P<User>[a-zA-Z0-9_.-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9.]+)”

index=games sourcetype=SimCube
| rex field=_raw “^[^’\n]*'(?P<User>[a-zA-Z0-9_.-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9.]+)’\s[a-zA-Z:]+'(?P<Character>[a-zA-Z0-9.-]+)”

 

Calculated field

index=network sourcetype=cisco_wsa_squid
| stats sum(sc_bytes) as Byte by usage
| eval bandwidth = Bytes/1024/1024