Splunk
From Wikipedia, the free encyclopedia
Splunk is a search engine for IT data. It is used to search large volumes of application, server (computing) and network events. Splunk is not an automated monitor, but an interactive search tool with plotting and SQL report capabilities designed to let human users recognize patterns, find problems, and connect seemingly unrelated events. The name "Splunk" is a reference to data mining.
Its back end builds an index of syslog entries, log files, messages from application servers, and other machine-generated data. It does this by tailing live log files, named pipes, IP sockets and batch-loading archive directories. Indexing is done through admin-configured secure remote data collection, rather than by crawling other hosts.
Besides search, Splunk's other difference from traditional monitoring and reporting tools is that it automatically figures out any data format it encounters, rather than requiring pre-configured templates (there are some pre-trained patterns, for consistent naming of popular formats such as "linux_messages_syslog.") Splunk can be further trained against local data, e.g., "This example file's source type is log4j." If Splunk does not recognize a source type, it will create a placeholder name for it and reverse-engineer its event format.
[edit] Web 2.0 components
Splunk has been categorized as a Web 2.0 (or sometimes "Enterprise 2.0") software for two reasons. First, its indexed events can be tagged, anonymized, and shared with the user community (somewhat like Flickr photos) as a means of sharing knowledge about IT troubleshooting among different sites. Splunk Base is an open community database of user-contributed log excerpts that have been collaboratively tagged in a folksonomy and annotated in a wiki. For privacy and security, users can selectively anonymize fields in their events before posting them. A user queries Splunk Base by clicking on search results being investigated.
Also, Splunk's front end is a web-based search interface that uses Ajax on top of a SOAP API. There are three user interfaces: an Ajax GUI, a traditional search engine text box, and a Unix command-line interface. All use the same API.
