Event Log: Leveraging Events and Endpoint Logs for Security

An event log is a file that contains information about usage and operations of operating systems, applications or devices. Security professionals or automated security systems can access this data to manage security, performance, and troubleshoot IT issues.

In the modern enterprise, with a large and growing number of endpoint devices, applications and services, it is no longer possible to manage security and IT operations with network monitoring alone. Event logs, and in particular endpoint logs, are of critical importance.

In this page you will learn about:

Introduction to Event Logs and Security Logs

Events that occur in end-user devices or IT systems are commonly recorded in log files. Operating systems record events using log files. Each operating system uses its own log files, and applications and hardware devices also generate logs. Security teams can use security logs to track users on the corporate network, identify suspicious activity and detect vulnerabilities.

Most security and IT organizations find that systems generate more log information than they can process. Event and log management tools help analyze logs, monitor important events recorded in logs, and leverage them to identify and investigate security incidents.

Key concepts of log management

  • Log—raw data stored by a computer system.
  • Events—something that occurs somewhere on a network or computer system, can be extracted from log data.
  • Incidents—events that are identified as possible security breaches. This can include unauthorized access to data or IT systems, violation of security policies, etc.

Using Endpoint Logs for Security

With the growing use of endpoint devices, many of which are laptops, phones or other mobile devices, endpoint logs are becoming more important for security. Attackers who gain access to an endpoint device can use it to penetrate your network. Therefore, it’s essential to collect data from endpoint logs and identify malicious or unauthorized activity.

Windows Event Logs

The Windows operating system logs actions of software or hardware components. Administrators can access this information to detect and troubleshoot issues. Six default categories are used to classify events:

  • Application log—events logged by applications. Developers determine the events logged by their application. The application can log information from several sources. It is important to note the source alongside the event ID.
  • System log—events logged by the operating system. For example, issues experienced by drivers during the startup process.
  • Security log—events related to security, including login attempts or file deletion. Administrators determine which events to enter into their security log, according to their audit policy.
  • Directory service log—records active directory operations like authentication and modification of privileges. Only available on domain controllers.
  • DNS server log—records DNS activity. Only available on DNS servers.
  • File replication service log—records domain controller replication, only available on domain controllers.

Windows contains a system called Event Viewer, which can be used to view event logs across all the above categories. Event Viewer displays information about an event, including the date and time, username, computer, source, and type.

User Execution Attack Technique
Image source: Microsoft

Linux Event Logs

The Linux operating system stores a timeline of events related to the server, kernel, and running applications. The main log categories are:

  • Application logs
  • Event logs
  • Service logs
  • System logs

There are several ways to view logs in Linux:

  • Access the directory cd/var/log. Specific log types are stored in subfolders under the log folder, for example, var/log/syslog.
  • Use the dmseg command to browse through all system logs
  • Use the tail command, which displays the last lines written to a certain log file, where problems are usually found. For example tail -f /var/log/syslog prints the next line written to the file, letting you follow changes to the syslog file as they happen.

Following are commonly used Linux log files:

  • /var/log/syslog or /var/log/messages—general system activity logs. Used to detect problems that may occur during startup or to isolate application service errors. RedHat-based systems store information in the messages folder while Debian-based systems store them in the syslog folder.
  • /var/log/auth.log or /var/log/secure—all authentication and authorization logs. Used to investigate failed login attempts. RedHat-based systems store these in the auth.log folder while Debian-based systems store them in the secure folder.
  • /var/log/kern.log—kernel activity logs, including custom kernels.
  • /var/log/faillog—failed login attempts.
  • /var/log/maillog or var/log/mail.log—logs related to mail servers. Used to track issues like emails tagged as spam, and suspicious use of postfix or smtpd.

iOS Logging

iOS does not log events, however it does log application crash reports. iOS 10.0 and later offers an API that can be used to log application events. You can use crash reports and the logging API to find and investigate errors generated by your applications, either during development or in production.

iOS devices come with their own security features, implemented in both hardware and software. The logging API provides access to data generated by these security features. They include:

  • Data encryption—protects both personal and business data from unauthorized use.
  • App security—verifies the security of iOS apps.
  • Network security—provides developers with protocols for secure authentication and encrypted data transmission.
  • Apple pay—iOS devices can be used to pay securely. Identifiable information is not collected.
  • Internet services—iOS provides a variety of secure services, like iMessage and iCloud Backup.
  • User password management—authentication methods such as password autofill keychain. Apps cannot access this information without user permission.
  • Device controls—management tools that enable actions like remotely wiping stolen devices.
  • Privacy controls—users can decide which apps access what information.

Android Logging

Android offers a platform that provides access to all system and applications logging, including logs from the kernel driver, C, C++, and Java classes. The logging platform provides applications for viewing and filtering log messages.

Android Log Types

  • Application log—an android application uses the android.util.Log class to create log messages. Applications can set log levels or message “severity”, and descriptive tags, to enable log filtering and alerting.
  • Event log—messages are created using the android.util.EventLog class, which uses binary-formatted log messages. Log entries are made up of binary tag codes, binary parameters and a log message string. Message codes are stored in /system/etc/event-log-tags.
  • System log—classes in the Android framework use the system log to separate their messages from application logs. Android classes perform logging using the android.util.Slog class.

Logging sensitive data

  • Pay attention to how your organization handles user data. In Android, logs are shared to applications with READ_LOGS permission, meaning you can unintentionally leak user data to other apps.
  • Keep user data to a minimum. You can do this by avoiding the storage or transmission of Personally Identifiable Information (PII). External components should not access user data if they have no reason to do so.
  • If applications need access to data, provide direct access via the client. In this way, data does not have to be transmitted to other servers.

Additional Logs You Should Consider Monitoring

Beyond the common log sources mentioned above, there are many more enterprise systems and security tools that generate logs. All of them might have security implications. However, it’s crucial to prioritize logs for monitoring by analysts, since many organizations have limited security manpower.

Following is a list of most of the common log and information sources you may encounter in your organization. Select the most important sources your security team will regularly monitor.

Logs from Security Controls Network Logs Non-Log Infrastructure Information Non-log Business Information
  • IDS
  • Endpoint security (Antivirus, anti-malware)
  • Data Loss Prevention
  • VPN concentrators
  • Web filters
  • Honeypots
  • Firewalls
  • Routers
  • Switches
  • Domain controllers
  • Wireless access points
  • Application servers
  • Databases
  • Intranet applications
  • Configuration
  • Locations
  • Owners
  • Network maps
  • Vulnerability reports
  • Software inventory
  • Business process mappings
  • Points of contact
  • Partner information

How a SIEM Helps Make Sense of Security Event Data

Most mid-to-large organizations find it difficult to store and manage their log files, because modern IT systems generate gigabytes of logs every day. Security Information and Event Management (SIEM) systems can help.

SIEM tools ingest logs and apply correlation rules to find anomalies. SIEMs are a powerful tool that can package logs for the consumption of security analysts, and help identify actual security incidents among huge amounts of innocuous events.

Next- generation SIEMs with advanced analytics capabilities take this one step further, stitching together logs from disparate sources to find unknown threats. For an example, see Exabeam Advanced Analytics.

To learn how SIEM can help you manage logs and extract security value out of them, see our in-depth guide on Leveraging Events and Logs for Security.