Systems constantly generate information about their own activity.

Examples:

• Users logging in
• Services starting
• Files being modified
• Processes executing
• Network connections occurring
• Scheduled tasks running

This information is known as telemetry.

Telemetry provides visibility into what is happening on a system and is one of the most important resources available to Operators.

Without telemetry:

• Troubleshooting becomes difficult
• Investigations become guesswork
• Security incidents become harder to detect
• Attack chains become difficult to reconstruct

---

Why This Matters

Nearly every defensive activity relies on telemetry.

Examples:

When an Operator asks:

"What happened?"

Telemetry is often where the answer begins.

---

What Is Telemetry?

Telemetry is information generated by a system that describes activity occurring within that system.

Examples:

Telemetry acts as a historical record of system activity.

---

Linux Logging

Linux does not rely on a single logging mechanism.

Instead, multiple components may generate logs.

Common sources include:

• Applications
• Services
• Authentication systems
• The kernel
• Scheduled tasks
• Security tools

Many of these logs ultimately become accessible through a small number of logging systems.

---

Journal

Modern Linux distributions commonly use:

systemd-journald

to collect and store system logs.

The resulting log database is commonly referred to as:

The Journal

The journal records activity from:

• Services
• Applications
• The operating system
• The kernel

---

Why The Journal Matters

Many important events appear within the journal.

Examples:

• Service startup
• Service shutdown
• Service failures
• System boot events
• System shutdown events
• Application errors

Questions Operators commonly ask:

• Why did a service fail?
• When did the system reboot?
• What happened before the crash?

The journal often contains the answer.

---

Viewing Journal Data

A common tool used to inspect journal data is:

journalctl

Example:

journalctl

Displays journal entries.

Example:

journalctl -u ssh

Displays entries related to the SSH service.

Operators are not expected to memorize commands at this stage.

The objective is simply understanding where information originates.

---

Syslog

Historically, Linux systems relied heavily on:

syslog

for logging.

Many systems still use:

rsyslog

or compatible implementations.

Syslog commonly stores logs in:

/var/log

Examples:

/var/log/syslog
/var/log/messages

The exact location depends on the distribution.

---

Authentication Logs

Authentication activity is particularly important.

Questions Operators commonly ask:

• Who logged in?
• When did they log in?
• Did authentication fail?
• Was root accessed?

Many Linux systems record authentication activity within:

/var/log/auth.log

or:

/var/log/secure

depending on the distribution.

---

Service Logs

Services frequently generate their own telemetry.

Examples:

/var/log/nginx/
/var/log/apache2/
/var/log/mysql/

These logs may contain:

• Connections
• Errors
• Requests
• Authentication attempts
• Operational events

Service logs are often invaluable during investigations.

---

Kernel Messages

The Linux kernel also generates telemetry.

Examples:

• Hardware events
• Driver events
• Boot activity
• System failures

Common sources include:

dmesg

and:

journalctl -k

Kernel telemetry can provide important context during troubleshooting and forensic investigations.

---

Telemetry Is Evidence

For Operators, telemetry is more than logging.

Telemetry is evidence.

Examples:

Many investigations consist of collecting and correlating telemetry from multiple locations.

---

Building Timelines

One of the most important uses of telemetry is timeline construction.

Example:

09:00 User Login
        ↓
09:05 SSH Connection
        ↓
09:10 Suspicious Script Executes
        ↓
09:11 Service Modified
        ↓
09:15 Malware Launches

By combining telemetry from multiple sources, Operators can reconstruct system activity.

This concept becomes extremely important later in:

• Blue Team Foundations
• Incident Response
• Digital Forensics

---

Common Investigation Questions

Examples:

Many investigations begin by answering these questions.

---

Operator Perspective

When approaching an unfamiliar Linux system, Operators commonly ask:

Authentication

• Who logged in?
• Were there failed login attempts?
• Was root used?

Services

• What services started?
• What services failed?
• Were services modified?

System Activity

• Was the system rebooted?
• Were errors recorded?
• Were warnings generated?

Security

• Is logging enabled?
• Are logs intact?
• Do events appear suspicious?

Investigations

• What happened?
• When did it happen?
• Who performed it?
• What evidence exists?

These questions form the foundation of nearly every investigation.

---

Telemetry Limitations

Telemetry is powerful, but it is not perfect.

Examples:

• Logs may be deleted
• Logging may be disabled
• Storage limits may remove older events
• Some activity may never be logged

For this reason, Operators should never rely on a single telemetry source.

Multiple sources should be correlated whenever possible.

---

Key Takeaways

• Telemetry provides visibility into system activity.
• Linux generates telemetry from many sources.
• The journal is a major source of system information.
• Syslog remains common on many systems.
• Authentication logs provide evidence of login activity.
• Service logs provide application-specific evidence.
• Kernel logs provide operating system and hardware information.
• Telemetry is one of the primary sources of evidence during investigations.
• Multiple telemetry sources should be correlated when building timelines.
• Understanding telemetry is foundational for incident response, threat hunting, and digital forensics.

The next and final Linux basics lesson explores shell types.