Werner Schmidt
Enterprise Networking and Security Expert

SIM, SEM, SIEM, what does it mean?

Before I begin, can I mention my rant for acronyms? Not that long ago, DDI was created. It’s an acronym of acronyms, that stands for DNS, DHCP and IPAM.

Moving on, lets get to the confused field of SIM, SEM, SIEM and others. I’ll start with the purist definitions and then get into what people want and ask for versus what they ought to need. I won’t expand too much on what a log message is. Suffice it to say that it might be a syslog entry, but in fact can come from numerous sources including flat files, various agents, etc.

SIM - Security Information Management
SEM - Security Event Management
SIEM - Security Information and Event Management

Might as well add in here LM which stands for log management. These terms are bantered about by various manufacturers and even research groups like Gartner who haven’t quite come up to speed on how to really differentiate the terms, let alone the solutions, or better yet the requirements and needs. This has led to the problem of trying to find the best ideal solution for an indeterminate problem. Unfortunately, this has led to a lot of misunderstanding and hype around SIEM, a hybrid by definition of SIM and SEM. Like a multifunction copier, scanner and printer, make sure you really need all the functions and that the “blend” is right for you.

Ultimately these solutions are used to deal with various functions or capabilities such as:
Alerting - An automated response or alert to a single message, several messages or correlated event.
Retention - The ability to retain in full or summarized form messages from a period of time ideally related to compliance retention concerns.
Compliance - Ability of the system to create standardized reports for compliance or audit concerns.
Reporting - The ability to create useful reports that go beyond just compliance driven reports.
Normalizing - Taking similar messages from disparate systems and noting how different terms for things like properties are the same. For instance, knowing that bytes sent, sent bytes, traffic sent, etc., are all similar terms.
Correlation - Combining several different events, messages, threats, etc. into a single incident or event

In the classic purest view, LM/SIM solutions collect, store, alert and report on the data. Ideally these systems are tailored for long retention periods. Better solutions tend to be indexed, they usually offer stock reports for compliance, and often times have additional and extensive reports. Due to their nature, they are excellent aggregators and repositories of messaging.

SEM solutions by and large use a rolling shorter window of log messages, normalize it, correlate it and then attempt to do some kind of automated alerting and perhaps trouble ticketing. A SEM solution goes through the reams of log messages trying to find and summarize the most important information. They usually offer stock reports for compliance. Tend to have elaborate messaging and alerting. However, their goal is to process logs with the intent of creating alerts from correlated events.

Therefore, when we look at weaknesses:
SIM - Not ideal for complex alerting and not good for security (aka incident) reporting, trending or dashboards.
SEM - Not ideal for long term collection and storage or detailed searching and reporting.

Since SIM tends to be better at log collections, it can be used to drive or feed SEM solutions. SIEM solutions attempt to do both, but frankly it is a massive task and if the goal is long term retention and detail message logs, almost all of them will fall apart in a one size fits all approach. If you think databases and know how a single SQL search on poor indexing can take down a database, you get the idea. A hybrid system must make sure there is enough horsepower to constantly normalize, correlate and create incidents real time. It’s akin to trying to mix up operational database base needs with longer term data mining, the two are in direct conflict.

This is also a case of needing to walk before running. A good SIEM cannot be built upon a poor foundation of SIM. A strong SIM can feed one or multiple SEM/SIEM solutions. SEM/SIEM solutions require training. They need to be taught about correlation that matters in your environment and just as importantly trained to minimize false positives. SIM is easy to implement and deploy, SEM/SIEM takes time. That’s not to say SEM/SIEM is bad, it can be impossible to deal with reams of information coming from a SIM and most folks do ignore it until they need to deconstruct an incident.

The point is, it’s important to understand where your needs are and what solution or solutions will meet your need. There is some good news here. Mid-market customer can often times buy a single solution that can meet SIM/SEM/SIEM needs. Large enterprises should focus on buying both solutions and using an enterprise SIM to feed one or multiple SEM/SIEM solutions. Mid market customer will probably find our
LogLogic SEM solutions meet their needs. Larger customers will be thrilled with our LogLogic SIM solutions to feed other SEM and compliance solutions including LogLogic SEM. Smaller customers may find that the LogLogic virtual appliance can meet logging needs when less than 40 devices are involved.