Task 1: Logs as Evidence of Historical Activity
The room starts by explaining why logs are so important. Logs keep a record of all actions on a system, which is key for figuring out what happened during unusual or suspicious activities. By analyzing logs, we can answer essential questions like:
- What happened?
- When did it happen?
- Where did it happen?
- Who was involved?
- Were their actions successful?
- What was the result?
This introduction helps you see the value of keeping and analyzing logs for better system security.
Task 2: Types, Formats, and Standards
In this task, we dive into the different types of logs systems generate, depending on the services they run. For example:
- Application Logs: Track the status, errors, and other details of specific applications.
- Audit Logs: Record activities that are important for regulatory compliance.
- Security Logs: Keep track of security events like logins or firewall activity.
- Server Logs: Include system, event, error, and access logs.
- System Logs: Capture system-level details like kernel activities or boot sequences.
- Network Logs: Record network traffic and connections.
- Database Logs: Track database queries and updates.
- Web Server Logs: Log web requests, URLs, response codes, etc.
We also explore log formats. I originally thought all logs were the same, but this task shows how they differ:
- Semi-Structured: Like Syslog, which is widely used for system and network logging.
- Windows Event Log (EVTX): Microsoft’s log format for Windows systems.
- Structured Logs: These follow a standard format, such as:
- JSON: Easy to read and compatible with modern programming languages.
- W3C ELF: Used for web server logs, particularly on IIS (Microsoft’s web server).
- XML: Flexible and customizable for different systems.
- Unstructured Logs: Free-form text logs, such as:
- NCSA CLF: A common web server log format used by Apache.
- NCSA Combined Log: An extended version with additional fields like referrer and user agent (used by Nginx).
We also learned about standards like the Syslog Protocol and Common Event Expression (CEE), which help create and manage log data more effectively.
Task 4: Collection and Management of Logs
Log collection is crucial for analyzing logs properly. It involves gathering logs from various sources like servers, software, and databases.
Good log management ensures that logs are stored securely and can be retrieved quickly when needed. Centralizing logs is essential for easy access, analysis, and fast incident response. Centralized systems often come with features like real-time detection and automatic notifications.
Task 5: Managing Storage, Deletion, and Retention
I was wondering how we could store so many logs, especially in a big company, and this task answered my question. There are different types of storage:
- Hot Storage: For logs from the last 3-6 months, allowing real-time queries.
- Warm Storage: For logs from 6 months to 2 years, accessible but not as fast.
- Cold Storage: For older logs (2-5 years), archived and used mainly for historical analysis.
Task 6: Log Analysis – How to Do It
Finally, we learned how to analyze logs effectively. Since logs come from different sources and in different formats, raw logs aren’t easy to work with. That’s where the log analysis process comes in. It includes:
- Parsing: Breaking down logs into manageable pieces.
- Normalization: Standardizing data from different sources to make comparisons easier.
- Sorting: Organizing logs by time, severity, or other factors to spot trends or anomalies.
- Classification: Categorizing logs based on severity, source, or other characteristics.
- Enrichment: Adding context to logs, like geographical data or user information, to make them more meaningful.
- Correlation: Connecting related events to see patterns or relationships.
- Visualization: Using graphs or charts to make data easier to understand.
- Reporting: Summarizing the log data to help decision-making or meet compliance needs.
Conclusion
I loved this room! Logs can seem intimidating to most cybersecurity students because we don’t hear about them enough, and they look difficult to read or analyze. But this room made logs easy to understand and even fun. Logs are definitely something I want to explore more, especially since they’re fundamental to SOC analysis. Highly recommended!
