Splunk SIEM
Splunk Enterprise: Architecture, Features, and Capabilities
What Is Splunk Enterprise?
Splunk Enterprise is a data platform designed to help businesses manage big data and analyze machine data. It can be deployed on-premises or in the cloud via the Splunk Cloud Platform. Key features include data visualization, performance metrics, data collection, real-time search, indexing, KPI tracking, reporting and monitoring.
The solution allows administrators to:
- Create custom dashboards, add visualizations, and create custom forms for data input.
- Continuously monitor operational events for anomalies and receive alerts in real time.
- Leverage workload management to reserve storage capacity for high-priority tasks.
- Perform large-scale data clustering.
Splunk Enterprise provides integration with a large variety of third-party platforms, making it easy to ingest data into the platform from existing data sources and applications.
Splunk Enterprise Features
Following are the key features of Splunk Enterprise.
Collect and Index Data
Splunk collects data from virtually any source and location. It transforms logs into metrics and freely analyzes and correlates data without the constraints of existing database structures. It can also pull data from relational databases and data warehouses.
Workload Management
Splunk's workload management feature provides a policy-based mechanism to reserve system resources (such as CPU and memory) for workload collection and retrieval based on organizational priorities. This allows administrators to group workloads and reserve system resources for higher priority workload groups.
Search, Analyze and Visualize
Splunk provides a search processing language that enables simple searches as well as advanced data exploration. It provides a graphic UI that is accessible for business users with no technical expertise, as analysts and advanced users. Rich visualizations make the results easy to understand for any audience.
Monitor, Alert and Report
Splunk provides thresholds for monitoring events and proactively warns of potential problems when data passes the threshold. Alerts can generate a notification, initiate an application, or a custom action. Teams can use custom dashboards to organize and share data, or embed it in other applications.
Machine Learning Toolkit (MLTK)
Splunk MLTK allows teams to create custom machine learning models, providing a Smart Assistant with an easy-to-use graphic UI. The assistant guides users through the steps of creating a machine learning algorithm, creating Search Processing Language (SPL) queries in the background.
MLTK extends the Splunk platform by filtering out noise through outliers and anomaly detection, predictive analytics, and clustering algorithms. It makes it easy to use machine learning in production to process data and extract insights from it.
Apps and Premium Solutions
Splunk offers several solutions for security, IT and DevOps, which can help businesses derive more value from their data and react to important events. For example, Splunk provides Splunk Enterprise Security, a security information and event management (SIEM) system, and Splunk SOAR, which automatically responds to security incidents.
Splunk Enterprise In-Depth
Splunk Enterprise Architecture and Process
You install Splunk Enterprise on a host as a process called splunkd. This is a distributed server written in C/C++ that can ingest, process, and index large volumes of data, as well as handle search requests. The splunkd architecture uses several processes called pipelines, each made up of several processes which perform different operations on streaming data.
Key concepts in the architecture:
Pipelines—threads inside the splunkd process. Each pipeline has its own XML configuration.
Processors—reusable C/C++ functions that perform an operation on the data stream.
Queues—used to pass data from one pipeline to another.
Web server—splunkd runs a web server using SSL on port 8089 by default, and another web server on port 8000 without SSL.
Splunk Enterprise Metrics
Splunk Enterprise enables users to collect, investigate, monitor, and share metrics in real-time. Service engineers, IT staff, and system administrators can use this functionality to track various aspects of the ecosystem, tracking infrastructure, business applications, and security systems.
You can store metrics data by using metric indexes. In Splunk, each index type is optimized to store and retrieve metric data. This functionality lets you run metrics-specific commands on certain metric data points within the associated metric indexes.
For example, you can run the mstats command to apply aggregate functions like average, count, rate, and sum to these data points. This command can help you isolate and correlate issues from various data sources.
Analytics Workspace
Splunk’s Analytics Workspace provides a user interface (UI) that lets users monitor and analyze metrics and other time series without relying on SPL queries. It facilitates rapid identification and responds to anomalies or issues in your data.
Analytics Workspace lets you choose data sources to create interactive charts in your workspace. You can then apply aggregations and filters to gain insight into your metrics and system performance.
Functions, operations, and actions
Splunk’s Analytics Workspace includes a set of analytic operations and functions to help you gain insights from data. Available operations depend on your data source, and all functions generate SPL in the background. Here are common operations:
Aggregations—can summarize data points into meaningful values.
Time shifts—can modify the time range of a series.
Splits—show results for a specific dimension.
Filters—either exclude or include specific results.
Once you have sufficiently refined your data, you can use the Analytics Workspace to perform certain actions. For example, you can create a dashboard that monitors or shares your findings. You can also configure an alert to notify you of specific behavior in your data.
Monitoring Splunk Enterprise
Splunk Enterprise is a distributed system which requires monitoring to ensure all components are working properly. You can monitor the system via a monitoring console and a rest-based component monitoring tool.
Monitoring Console
The Splunk Enterprise Monitoring Console is a search-based tool that shows the current topology of the Splunk Enterprise system, and performance of individual components. You can use it to monitor anything from simple single-instance deployments to multi-site indexer clusters.
It provides several dashboards that let you give you visibility into:
Search and indexing performance
Resource usage
License usage
Splunk component monitoring API
Splunk Enterprise provides a REST API, accessible at the /server/health/splunkd endpoint, which lets you access the health of Splunk components programmatically. It uses a tree structure that reports the health of individual features, providing a continuous view of deployment health. The same health report is also provided by the Splunk Web interface.
Securing the Splunk Platform
Splunk Enterprise gives you three ways to protect the platform and data from unauthorized access:
Role-based access control (RBAC) which can limit who can access what in Splunk Enterprise.
Defining certificates and SSL encryption for configurations, stored data, and data ingestion.
Defining SSL encryption for both Splunk Web (external communication) and connections between splunkd instances (internal communication).
Obfuscating login credentials.
Hardening splunkd instances by ensuring hosts have a secure configuration, properly managing credentials, and configuring encryption.
Managed Detection & Response
Splunk Security with BlueVoyant
Quickly scale your security operations across your environments without the need to invest in additional hardware or software.
Additional Readings
Splunk SIEM
Understanding Splunk Cloud: Capabilities and Related Solutions
Splunk SIEM
Splunk Enterprise Security: Use Cases, Features, and Process
Splunk SIEM
What is Splunk Phantom (Renamed to Splunk SOAR)?
Splunk SIEM
8 Splunk Security Solutions and How to Secure Splunk Data
Splunk SIEM
Splunk Security Cloud: Product Editions and Professional Services
Splunk SIEM
Splunk User Education and Role-Based Access Control (RBAC)
Splunk SIEM
Part 1: CI/CD Pipelines Efficiently Delivers the Most Accurate and Updated Security Content
Splunk SIEM
Part 2: How Workflow Influences CI/CD Process