The most underutilized cybersecurity tool is the Security Information and Event Management (SIEM) system. I’ve never seen any other vendor solution be purchased as much, and left so underutilized. To that point, I’ve never even seen a SIEM being used to the effectiveness that the business purchased and planned for it to be used. 

It’s not your fault.

Often the marketing campaigns will focus on next-generation correlation and security event escalation, anomaly detection, endpoint activity, and user behavior anomaly. They will show beautiful visual dashboards and graphics. Seldom is that level of high-fidelity detection and alerting making it to a single pane dashboard where a junior level analysts await. 

Problems

From my experience consulting with Minnesota companies to improve their SIEM implementation I can separate the problems into two buckets; strategic and technical.

Strategic

Correlation Philosophy​​​​​​​

This is a security philosophy issue. One strategy believes they should collect all the logs they can and then out of the data lake fish for some good correlation alerts. 

The other philosophy starts with examining the gaps between current security and the control capabilities, then create alert gaols that identify malicious attempts in these gaps. 

What is your security correlation philosophy? 

If you want to create a logging strategy for specific alerts based on existing security control gaps, I can help you. If you want to create alerts out of logs, then I cannot help you. (I actually could, but it won’t be valuable.)

Purpose

The discussion of ‘logs‘ has really grown over the last few years. We’ve learned that ‘log management‘ can take on many different definitions. As companies mature their information security programs, they need log management for a very specific purpose – security events.

The purpose of these security events is to detect and correlate events in real-time and bring to the surface activities that need immediate investigation. This definition has driven the label of ‘security information and event management‘. This scope for log management no longer needs old data based on this definition. Cybersecurity analysts are not going to be looking at data from last September, they are focused on now. So that means our log retention time period can be greatly reduced. 

What happens when we reduce log retention time?

Two things; you save money and you break compliance requirements. That’s fine for security, but what about compliance? Who’s supporting their technical objectives? Should they now sponsor their own log management solution?

This drives a critical question to wrestle through before improving and streamlining your SIEM;

Who are currently depending on our current log management infrastructure, and what are they using it for?

Technical

Log Formats

The truth is, at a very basic level there is not an accepted standard for log message formats. That means that different assets will send their log messages to the collections system in different formats. Because of this, the log messages need to be parsed before being ingested by the log correlation engine.

Parsing is an activity that reorders the data fields in the log file to a common format that can be stored, queried, and read. Often SIEM’s will come with parsing scripts for specific technologies. Time needs to be invested to not only relate your log sources to the correct parsing engines, but time and effort also need to be spent validating it’s done correctly. 

Bad logs can wreak havoc on a logging database. The result of bad parsing can be having date fields being stored in username fields, and the data not being updated to the universal time of the event collectors. Windows Event ID’s being stored as date fields. It’s a mess. Logs from assets that are not parsed correctly provide zero value to your system, and will actually do more harm than good. 

Key Asset Ingestion

Did you know that when a user logs into a Windows domain only a single Active Directory Domain Controller will store the login event?(That hurts discovering on your own, let me tell you). That means that for you to provide full integrity of domain login events, you need to be logging every single domain controller. 

When you’re building alerts from ideal security events based on residual risk and gaps you will need to draw out what assets are required to achieve the event notification and escalation. You will then need to dive into that manufacturer’s configuration information, or software and figure out what log sources contain this data, what event ID’s are required, and how to ship (or pull) the logs into the SIEM. This can be frustrating and time-consuming.

If you grew into your security role from being a network expert, it’s going to be painful (not impossible) to get the right logs out of IIS, and Apache Web servers. If you’re a Windows guy or gal, it’s painful to get the right logs out of pFSense and Palo Alto. 

Log Collection Conflicts

What if you mature to the point that you have multiple log management systems, one for each specific purpose of; application, compliance, and security event management?

What if two or more of those systems require the same asset logs?

If the asset source can ship Syslog (a log file format) to multiple destinations, you’re in luck. But when you need security logs from a source that requires an agent installation, this becomes a conflict. Often times specific SIEM vendors will sell ‘agents‘ you install on your log sources. Again, the marketing will say ‘you don’t need to, the logs can be collected without an agent‘. Sometimes that’s true, but not if they are critical log sources. My point is this, if you need to install a vendor-specific agent on an asset you will disable this asset from reporting logs to other management solutions. 

One way we’ve used to overcome this challenge is by leveraging an open source agent, such as ‘Beats’.  We then have the open source agents send their logs to a log agent, parser, and shipper. This shipping agent, such as LogStash, can then ship the specific logs required to the correct log management destinations. A solution like this can meet the requirements of multiple log collection destinations without tying up the log source. 

Log Choking

Most vendor SIEM marketing material will include some kind of ‘funnel’ to help the client understand how all these messages are being ingested and filtered down to ‘security events‘. But do you know what happens when you pour much into a funnel? Have you heard that gurgling sound? They choke and spit up. 

That means when you send too much data to a log management system, critical events from critical log sources do not make it to the critical part of being correlated for security investigations. 

Quick free tip for you, don’t ingest database logs.

I once thought this was a great idea, and worked with an Oracle DBA team. I thought this was a great way to get much-needed visibility into the database security and provide the DBA’s a value add by giving them assess to their logs. Choke and fail. The amount of logs coming out these databases is absolutely enormous. One single database created more logs than all the other assets currently logging put together! There are ways to improve this, but it goes back to my earlier point – it’s really hard for a security analyst to configure an Oracle database to only send the appropriate security logs we’re looking for. 

Conclusion

Security information and event management (SIEM) is a powerful tool to leverage and mature your cybersecurity program. To do it in the most right way requires starting with a vision and specific security philosophy on how this technology should be used. 

The right SIEM, coupled with the right process, and the right people can greatly reduce the risk to our business and decrease costs. 

To do this successfully, be familiar with the common challenges and problems others run int while building their security log management system. 

Partnering

If you are in Minnesota and want help implementing, or reviving your SIEM solution consider giving us a call.

We can help you take your SIEM from zero to sixty.

We’ve helped other Minnesota companies, we’re familiar with the space and can save you time and money by overcoming the common challenges with building a SIEM that provides great value for years to come.  

Asher Security is a Minnesota Cybersecurity Consulting and Advisory company focused on serving Minnesota companies to help lower risk and decrease cost by partnering to mature the information security program.