Detecting a Multi-Host Attack Chain with Splunk

Introduction

Security monitoring becomes significantly more challenging when an attack moves across multiple hosts, operating systems, and network segments. Detecting an isolated malicious command is one thing; reconstructing how an attacker progressed from an initial foothold to privileged access on another system requires visibility across the entire attack chain.

In this project, I built a multi-host SOC lab to simulate and detect an end-to-end attack against a segmented Windows and Linux environment. The attack originates from a Kali Linux machine, compromises a Windows employee workstation, escalates privileges to SYSTEM, obtains protected credentials, and uses the compromised workstation as a pivot to reach an internal Ubuntu server. The final stage abuses a deliberately misconfigured Linux file capability to escalate privileges from a service account to root.

The offensive actions are only one half of the project, I also enriched it with Windows Sysmon and Linux auditd whose provided telemtry is later ingested into Splunk, where I later develop and tune SPL detections for individual stages of the attack.

The detections I mentioned identify suspicious SYSTEM-level PowerShell execution, SSH remote port forwarding, successful lateral SSH access, execution of a capability-enabled binary, and the resulting transition to root privileges.

Finally, the telemetry is correlated across hosts to show the whole picture. The goal of the project is to demonstrate a practical SOC workflow: understand attacker behavior, collect the right data, develop detections around evidence, reduce nose, and eventually correlate events into a coherent security investigation.

Lab architecture

The lab consists of three virtual machines distributed across separate network segments. The Kali Linux host represents the external attacker, the Windows Employee workstation acts as the initial target and later as a pivot host, while the internal Ubuntu server hosts the final target and the Splunk environment used for detection and investigation.

Scenario premise - Physical access (assumed)

The adversary gains temporary physical access to the corporate environment and plugs an unauthorized device to an exposed employee-network Ethernet port. Insufficient Network Access Control (NAC) and... oops, the device gets acccess to the employee network. And that's where our attack begins.

Initial Network Topology

At the beginning of the simulation, the attacker's Kali Linux host and the Windows Employee workstation share the 172.16.50.0/24 network segment. Kali can communicate with the Employee workstation, but has no direct access to the internal 192.168.56.0/24 network.

The Employee workstation is connected to both network segments: 172.16.50.0/24 on the attacker-accessible side and 192.168.56.0/24 on the internal side. This will make the Windows Employee particularly interesting from an attacker's perspective - but the attacker doesn't know that yet...

1. Reconnaissance & Target Selection

With access to the employee network established, the attacker still has no credentials, no compromised endpoint, and no knowledge of the internal infrastructure. The first step is therefore pretty obvious: find out what is actually reachable.

Initial reconnaissance focuses on discovering live hosts and exposed services within the accessible network. At this point, attacker is not trying to exploit anything yet, but to understand the available attack surface and identify the most promising initial target.

Rather than poking at every exposed service blindly, attacker followed the strongest lead from the reconnaissance results. Nmap gave us something much more useful than just an open port - the identified HttpFileServer instance provided a specific product and version to investigate further. Time to take a closer look.

2. Initial Access - Shadow IT backfires

A closer look at the exposed service revealed something much more interesting: the Employee workstation was running Rejetto HFS as an unauthorized file-sharing service. Shadow IT is known to introduce security gaps into the corporate networks, as these services are often unmanaged and may operate without the knowledge of system administrators.

The HFS fingerprint gave a concrete product and version to examine. Initially the local Exploit-DB database was searched and a remote command execution exploit targetting Rejetto HttpFileServer 2.3.x. was found.

The exploit generated a request for the target HFS instance, but producing a payload in itself doesn't prove successful code execution. To verify the exploit independently, test was repeated using a simple file-creation payload. If command execution succeeded, a proof file should appear on the Employee workstation.

The attacker could not verify the expected effect on the target, and a direct check on the Windows workstation confirmed that the expected artifact had not been created.

Since the initial PoC could not be reliably validated, attacker returned to vulnerability research, which soon led to CVE-2024-23692 and a different proof-of-concept targeting the exposed HFS service.

As before, the attacker started exploitation attempt with a direct command execution test. whoami was his first choice - a minimal test that would both confirm remote code execution and reveal the security context of the compromised process.

With RCE confirmed, the next step was to understand the compromised host, rather than immediately attempting further expoitation.

The results confirmed access to DESKTOP-KBDL1T4 under a standard user context. Attacker, not finding any immediately available useful administrative privileges, shifted the focus toward local misconfigurations and services executing with higher privileges.

InterfaceAlias   IPAddress       PrefixLength
--------------   -------------   ------------
Ethernet 3       172.16.50.20    24
Ethernet 2       192.168.56.20    24
Ethernet         10.0.2.15        24

The compromised workstation could now become a gateway to the internal network.

Before exploring the newly discovered network, the attacker continued enumerating the compromised workstation for potential privilege escalation opportunities.

Service enumeration revealed one service that stood out from the standard Windows components: EmployeeUpdaterService, its name suggested an organization-specific utility and therefore making it an interesting target for closer analysis.

SERVICE_NAME: EmployeeUpdaterService

START_TYPE         : 3  DEMAND_START
SERVICE_START_NAME : LocalSystem
BINARY_PATH_NAME   : C:\Windows\System32\cmd.exe /c "C:\Program Files\EmployeeTools\updater.cmd"

This changed the direction of the attack. The cusom service executed as LocalSystem, while its command path points to script under C:\Program Files\EmployeeTools. If that script could be modified by the compromised employee account, the existing employee-level access could potentially be escalated to SYSTEM.

The next step was to inspect the permissions of the referenced script.

3. Windows Privilege Escalation

Further inspection of EmployeeUpdaterService focused on the script which its command path points to. Although the service itself executed as LocalSystem, this alone did not represent a vulnerability, the critical loophole appeared when attacker investigated updater script permissions:

BUILTIN\Users:(M)
SERVICE_START_NAME: LocalSystem
BINARY_PATH_NAME: C:\Windows\System32\cmd.exe /c "C:\Program Files\EmployeeTools\updater.cmd"

With the privilege boundary confirmed, the next step was to validate whether control over updater.cmd could actually be converted into SYSTEM-level command execution.

WRITE_OK

The success write test confirmed that the compromised employee context could modify a file used by a service running as LocalSystem. The attacker could therefore affect code executed across the privilege boundary.

After the write access was confirmed, the attacker modified updater.cmd to record the identity under which the privileged execution path ran. If the script was executed by the service as expected, the resulting file would provide direct evidence of SYSTEM-level execution.

@echo off
echo Service executed as: %USERNAME% > C:\ProgramData\EmployeeTools-result.txt
whoami >> C:\ProgramData\EmployeeTools-result.txt

The payload was intentionally minimal, its only purpose was to record the security context under which updater.cmd was executed, providing a simple way to verify whether the privilege escalation reached SYSTEM.

Service executed as: DESKTOP-KBDL1T4$
nt authority\system

At this point, the attacker had successfully converted the initial employee-level compromise into SYSTEM-level execution on the Windows workstation.

4. Internal Network Discovery

Earlier enumeration revealed that the compromised workstation was connected to the previously unknown 192.168.56.0/24 network. With SYSTEM-level execution now established, the attacker returned to this finding and began investigating the internal segment.

The ARP cache was inspected to identify systems the workstation had already communicated with on the internal network.

Interface: 192.168.56.20 --- 0xf
  Internet Address      Physical Address      Type
  192.168.56.10         08-00-27-78-1c-ff     dynamic

After identifying specific host, the attacker shifted from network discovery to targeted service enumeration to determine what was exposed on 192.168.56.10.

4.1 Targeted Service Enumeration of 192.168.56.10

Because 192.168.56.10 was not directly reachable from Kali, service enumeration was performed through the compromised Employee workstation. A small PowerShell TCP scanner was used to test selected ports from the workstation's internal network interface.

$ports = @(21,22,25,53,80)

foreach ($p in $ports) {
    $c = New-Object System.Net.Sockets.TcpClient
    $r = $c.BeginConnect("192.168.56.10",$p,$null,$null)

    if ($r.AsyncWaitHandle.WaitOne(1000)) {
        if ($c.Connected) {
            Write-Output "$p OPEN"
        } else {
            Write-Output "$p CLOSED"
        }
    } else {
        Write-Output "$p TIMEOUT"
    }

    $c.Close()
}

21 TIMEOUT
22 OPEN
25 TIMEOUT
53 TIMEOUT
80 TIMEOUT

The discovery of an accessible SSH port justified expanding research to additional infrastructure and management ports.

$ports = @(110,139,443,445,3306,5432,8000,8080,8089,9997)

foreach ($p in $ports) {
    $c = New-Object System.Net.Sockets.TcpClient
    $r = $c.BeginConnect("192.168.56.10",$p,$null,$null)

    if ($r.AsyncWaitHandle.WaitOne(1000)) {
        if ($c.Connected) {
            Write-Output "$p OPEN"
        } else {
            Write-Output "$p CLOSED"
        }
    } else {
        Write-Output "$p TIMEOUT"
    }

    $c.Close()
}

110    TIMEOUT
139    TIMEOUT
443    TIMEOUT
445    TIMEOUT
3306   TIMEOUT
5432   OPEN
8000   OPEN
8080   TIMEOUT
8089   OPEN
9997   OPEN

Open ports alone were not sufficient to identify the underlying services. The next step was therefore to fingerprint the exposed endpoints and determine the role of the internal host.

4.2 Service Fingerprinting and Host Identification

After several reachable ports were identified, the investigation moved toward understanding what was actually running on the host. Each exposed endpoint was examined individually to identify the associated service and gather information about the underlying system.

The SSH service was examined first because it represented a potential path for later lateral movement.

$c = New-Object System.Net.Sockets.TcpClient("192.168.56.10",22)
$s = $c.GetStream()
$s.ReadTimeout = 3000
$b = New-Object byte[] 1024

try {
    $n = $s.Read($b,0,$b.Length)
    if ($n -gt 0) {
        [Text.Encoding]::ASCII.GetString($b,0,$n)
    }
} catch {
    Write-Output "NO_BANNER"
}

$c.Close()

SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.18

With the host identified as Ubuntu, attention shifted to the remaining exposed ports.

Port 8000 was examined first because it commonly exposes HTTP-based applications and could potentially reveal the role of the server through response headers or content.

$c = New-Object System.Net.Sockets.TcpClient("192.168.56.10",8000)
$s = $c.GetStream()
$s.ReadTimeout = 3000

$req = [Text.Encoding]::ASCII.GetBytes(
    "GET / HTTP/1.0`r`nHost: 192.168.56.10`r`n`r`n"
)

$s.Write($req,0,$req.Length)

$b = New-Object byte[] 4096

try {
    $n = $s.Read($b,0,$b.Length)

    if ($n -gt 0) {
        [Text.Encoding]::UTF8.GetString($b,0,$n)
    } else {
        Write-Output "NO_RESPONSE"
    }
}
catch {
    Write-Output "NO_RESPONSE"
}

$c.Close()

HTTP/1.1 303 See Other
Location: http://192.168.56.10/en-US/
Server: Splunk

With both OpenSSH and Splunk Web identified, the remaining open ports could be assessed in context, though these were not treated as confirmed until independently verified.

With Splunk Web already confirmed, TCP/8089 was analyzed next to determine whether it exposed another Splunk component.

Rather than relying on the port number alone, the service was fingerprinted at the TLS layer. A TLS connection was established from the compromised Employee workstation and the certificate presented by the service was inspected.

$c = New-Object System.Net.Sockets.TcpClient("192.168.56.10",8089)

try {
    $s = New-Object System.Net.Security.SslStream(
        $c.GetStream(),
        $false,
        ({$true} -as [Net.Security.RemoteCertificateValidationCallback])
    )

    $s.AuthenticateAsClient("192.168.56.10")

    Write-Output "TLS_OK"
    Write-Output "PROTOCOL: $($s.SslProtocol)"
    Write-Output "CERT: $($s.RemoteCertificate.Subject)"

    $s.Close()
}
catch {
    Write-Output "TLS_ERROR"
    Write-Output $_.Exception.Message
}

$c.Close()

TLS_OK
PROTOCOL: Tls13
CERT: O=SplunkUser, CN=SplunkServerDefaultCert

After the attacker identified Splunk-related service on TCP/8089, attention shifted to TCP/5432. Although the port is commonly associated with PostgreSQL, the service was treated as unverified until protocol-level evidence could confirm the hypothesis.

$c = New-Object System.Net.Sockets.TcpClient("192.168.56.10",5432)
$s = $c.GetStream()
$s.ReadTimeout = 3000

# PostgreSQL SSLRequest: length=8, code=80877103
$b = [byte[]](0,0,0,8,4,210,22,47)
$s.Write($b,0,$b.Length)

try {
    $r = $s.ReadByte()

    if ($r -eq 83) {
        Write-Output "POSTGRESQL_DETECTED"
        Write-Output "SSL_SUPPORTED"
    } elseif ($r -eq 78) {
        Write-Output "POSTGRESQL_DETECTED"
        Write-Output "SSL_NOT_SUPPORTED"
    } else {
        Write-Output "UNKNOWN_RESPONSE: $r"
    }
} catch {
    Write-Output "NO_RESPONSE"
}

$c.Close()

POSTGRESQL_DETECTED
SSL_SUPPORTED

Port 9997 was additionally confirmed as reachable using the same low-level TCP validation method described for the preceding services.

The validation results show that the Ubuntu server exposes a relatively small and clearly defined set of services. The identified attack surface consists primarily of OpenSSH, PostgreSQL, and the Splunk infrastructure, including Splunk Web, the Management API, and the TCP receiving service.

Although these services increase the server's network exposure, their presence alone does not establish an immediately exploitable entry point. The results therefore provide a useful baseline for comparison with the Employee workstation, where a broader and potentially more promising attack surface was identified during reconnaissance.

5. Establishing Access to the Internal Ubuntu Server

Through the previous enumeration, the attacker confirmed that the internal Ubuntu server exposed an OpenSSH service on TCP port 22. However the host was not directly accessible from Kali because it was located behind the compromised workstation. So it was time to figure out how to get there.

5.1 Discovery of SSH Authentication Material

With SSH identified as a potential access path, one of the next steps of reconnaissance was to determine whether the compromised Employee workstation contained any authentication material that could be used against the internal Ubuntu server.

Earlier service enumeration had already identified C:\Program Files\EmployeeTools as a directory of interest. The EmployeeUpdaterService discovered during that stage referenced C:\Program Files\EmployeeTools\updater.cmd, which had previously been analyzed as part of the privilege escalation process.

The attacker therefore returned to the same directory and enumerated its contents, this time looking for files that could be relevant to accessing the internal server.

Volume in drive C has no label.
Volume Serial Number is 429E-106C

Directory of C:\Program Files\EmployeeTools

08/31/2026  02:47 PM    <DIR>          .
08/27/2026  08:38 AM    <DIR>          ..
08/31/2026  02:47 PM               154 backup.ini
08/31/2026  02:23 PM               411 backup_ed25519
09/07/2026  09:57 AM               554 updater.cmd
               3 File(s)          1,119 bytes
               2 Dir(s)  41,231,888,384 bytes free

The directory contained two files of particular interest: backup.ini and backup_ed25519, that suggested the presence of SSH authentication material.

The attacker first attempted to read the suspected private key using the current employee-level access.

KEY_READ_DENIED
Access to the path 'C:\Program Files\EmployeeTools\backup_ed25519' is denied.

The attempt failed due to insufficient permissions, confirming that the compromised employee account could not directly acccess the private key.

Then it was attempted to inspect the permissions of the suspected SSH private key using the compromised employee account and icacls command.

Successfully processed 0 files; Failed processing 1 files

Even the file permissions could not be inspected from the current employee context.

Since the private key could not be accessed with the current employee privileges, the attacker returned to previously identified EmployeeUpdaterService privilege-escalation path. The service executed updater.cmd as LocalSystem, providing a way to access the protected authentication keys with elevated privileges.

Extraction and Validation of the SSH Private Key

As demonstrated previously, an attempt to read:

C:\Program Files\EmployeeTools\backup_ed25519

resulted in an Access is denied error because the file was accessible only to privileged security principals, including NT AUTHORITY\SYSTEM.

To bypass this restriction, the previously identified EmployeeUpdaterService was used. The compromised Employee account was able to modify:

C:\Program Files\EmployeeTools\updater.cmd

while the corresponding scheduled task executed this script with NT AUTHORITY\SYSTEM privileges. Contents of updater.cmd were modified so that, when executed by the scheduled task, it copied the protected SSH private key to a location accessible from the compromised context:

The distinction is important: the Employee account did not gain direct access to backup_ed25519. Instead the attacker-controlled updater.cmd was executed by the scheduled task as SYSTEM, therefore the copy operation inherited SYSTEM privileges and was able to read the previously inaccessible private key.

After these operations, the otherwise proected SSH private key was available at:

C:\Users\Public\key.txt

The presence of the copied file was verified with:

Test-Path "C:\Users\Public\key.txt"

which returned:

True

The next step was to extract its contents from the Windows host while preserving the original binary/text representation.

To avoid corruption caused by quoting and command-enconding issues encountered with the PoC, the contents of key.txt were represented as Base64 and transferred in that form. The resulting Base64 data was saved locally on the Kali system as:

key.b64

It was then decoded into the reconstructed SSH private key:

base64 -d key.b64 > backup_ed25519

The resulting file was 411 bytes in size, matching the size of the original backup_ed25519 file observed on the Windows host.

Finally, the integrity and format of the extracted key were validated by deriving its corresponding public key:

ssh-keygen -y -f backup_ed25519 > backup_ed25519.pub

The command completed successfully and produced a valid ssh-ed25519 public key.

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEAbKACZX7/fEzS34kNCJAqYVWhLCMuP7urXChkWb1i2 root@soc-splunk

The root@soc-splunk value does not indicate that the recovered private key provides SSH access as root. It is only a comment associated with the SSH key and does not determine which account accepts the key for authentication.

The actual target account had been identified from backup.ini, which contained:

[backup]
server=192.168.56.10
port=22
username=backupsvc
identity_file=C:\Program Files\EmployeeToolackup_ed25519
destination=/home/backupsvc/backups

5.2 Establishing the Pivot

At this point, the SSH private key had been successfully obtained and the corresponding configuration identified the target as:

backupsvc@192.168.56.10:22

However, possessing valid SSH credentials was not sufficient to establish a connection. The kali attacker machine did not have a direct route to the 192.168.56.0/24 network where the Ubuntu server was located.

The compromised Windows Employee workstation, on the other hand, was connected to this internal network and could communicate with the Ubuntu server.
This made the Employee worstation a suitable pivot host.

−

100%

+Reset view

Drag to pan · Scroll to zoom · Reset to fit

Chisel was used to establish a reverse tunnel through the compromised Employee workstation. This allowed the Kali attacker machine to use the dual-homed workstation as a pivot and reach the otherwise inaccessible Ubuntu server on the 192.168.56.0/24 internal network.

Preparing the Chisel Pivot

Before establishing the tunnel, three separate services were used on the Kali attacker machine.

Together, these three terminals provided the tunnel endpoint, the Chisel client delivery mechanism, and the command-execution channel required to configure the Employee workstation as a pivot host.

5.3 SSH Access Through the Pivot

With the Chisel tunnel established, the Kali attacker machine could route traffic through the compromised Employee workstation and reach the previously inaccessible Ubuntu server on the internal 192.168.56.0/24 network. Using the previously obtained SSH private key, the attacker authenticated to the Ubuntu server as backupsvc.

88 updates can be applied immediately.
1 of these updates is a standard security update.
To see these additional updates run: apt list --upgradable

Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status

*** System restart required ***
Last login: Tue Sep 15 13:04:55 2026 from 192.168.56.20
backupsvc@soc-splunk:~$

5.4 Backupsvc Enumeration and Privilege Escalation

After successfully accessing the internal Ubuntu server through the Chisel pivot, the attacker began enumerating the system from the context of the backupsvc account. The objective was to identify the account's privileges, accessible resources, and potential paths for further escalation.

With the current user and system context confirmed, the next step was to assess the privileges available to the backupsvc account.

The backupsvc account was not a member of any additional groups, and enumerating its sudo privileges required a password. This indicated that no immediate passwordless sudo-based privilege escalation was available.

With no immediate sudo-based privilege escalation path identified, the attacker proceeded to enumerate the backupsvc home directory for potentially useful files, configuration data and artifacts from previous activity.

Among the files identified in the backupsvc home directory, .bash_history was particularly valuable because it could reveal previously executed administrative or maintenance commands.

whoami
hostname
sudo cat /tmp/backup_ed25519
sudo ssh -i /tmp/backup_ed25519 backupsvc@192.168.56.10
exit
whoami
hostname
id
ip addr
sudo -l
echo "=== IDENTITY ==="
whoami
id
echo "=== SUID ==="
find / -perm -4000 -type f 2>/dev/null
echo "=== CAPABILITIES ==="
getcap -r / 2>/dev/null
echo "=== CRON ==="
cat /etc/crontab 2>/dev/null
ls -la /etc/cron.d /etc/cron.daily 2>/dev/null
echo "=== SYSTEMD ==="
systemctl list-timers --all --no-pager 2>/dev/null
echo "=== WRITABLE /OPT ==="
find /opt -writable -ls 2>/dev/null
echo "=== WRITABLE /USR/LOCAL ==="
find /usr/local -writable -ls 2>/dev/null
whoami
id

The shell history showed that several common Linux privilege-escalation vectors had previously been investigated, including SUID binaries, Linux capabilities, scheduled tasks, systemd timers, and writable directories. Among these, the getcap enumeration produced a particularly interesting finding.

The attacker repeated the Linux capabilities enumeration to determine whether it exposed any interesting privileged binaries.

Among the identified binaries, one was of particular interest to the attacker:

/opt/company-tools/backup-helper cap_setuid=ep

It stood out because it was a custom company utility configured with cap_setuid=ep, allowing the process to manipulate its user idenitity.

To determine whether the capability could be abused, the attacker inspected the binary, its permissions, and its available functionality.

Futher inspection revealed that the source code of the custom utility was also accessible, allowing the attacker to directly review how the privileged capability was used.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

int main(int argc, char *argv[]) {
    if (argc == 2 && strcmp(argv[1], "--run") == 0) {
        printf("Running company backup as UID %d\n", geteuid());
        return 0;
    }

    if (argc == 2 && strcmp(argv[1], "--maintenance") == 0) {
        if (setuid(0) != 0) {
            perror("setuid");
            return 1;
        }

        execl("/bin/bash", "bash", "-p", NULL);
        perror("execl");
        return 1;
    }

    printf("Company Backup Helper\n");
    printf("Usage: %s --run\n", argv[0]);
    return 0;
}

Consequently, this resulted in privilege escalation to Root.

backupsvc@soc-splunk:~$ /opt/company-tools/backup-helper --run
Running company backup as UID 1001

backupsvc@soc-splunk:~$ /opt/company-tools/backup-helper --maintenance
root@soc-splunk:~# whoami
root

root@soc-splunk:~# id
uid=0(root) gid=1001(backupsvc) groups=1001(backupsvc)

5.5 Attack Path Summary

This completed the attack path. Starting from the external attacker machine, the Employee workstation was identified and compromised, privileges were escalated to SYSTEM, and the workstation was subsequently used as a pivot into the isolated internal network.

Using the recovered SSH credentials, the attacker gained access to the Ubuntu server as backupsvc. Local enumeration revealed a misconfigured custom utility with the CAP_SETUID capability, ultimately enabling privilege escalation to root.

The next section examines the same sequence of events from the defender's perspective using telemetry collected and analyzed in Splunk.

6. Splunk Detection & Investigation

This section focuses on identifying, investigating, and correlating the attack activity using the telemetry available in Splunk.

6.1 Monitoring Architecture and Data Collection

The diagram below presents the monitoring configuration and telemetry collected from each monitored host.

−

100%

+Reset view

Drag to pan · Scroll to zoom · Reset to fit

The Employee workstation forwards Windows Security, System, Application, and Sysmon telemetry to Splunk Enterprise over TCP port 9997 using the Splunk Universal Forwarder.

On the Ubuntu server, Linux audit telemetry from /var/log/audit/audit.log is ingested locally into the linux_security index with the linux:audit sourcetype. These data sources provide the telemetry used throughout the following detection and investigation sections.

To verify that telemetry from both monitored hosts was successfully ingested, the following SPL query was executed:

index=*
| stats count by host, source, sourcetype
| sort host source

The results confirm successful ingestion of all intended telemetry.

6.2 Detection of the Initial Windows Compromise

The investigation was initiated after an employee reported unusual activity in an unauthorized HFS application running on the Employee workstation. The application, deployed as Shadow IT without formal IT approval, displayed unexpected requests containing command-like parameters and encoded PowerShell content.

The SOC began investigating the workstation in Splunk to determine whether the suspicious requests had executed commands on the system.

No corresponding Sysmon activity was identified for the same date as the HFS application-log evidence. The investigation window was therefore expanded to determine whether equivalent suspicious activity had been recorded earlier that week.

The following search examined Sysmon process-creation events for processes launched directly by the HFS executable:

host="DESKTOP-KBDL1T4"
sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
| rex field=_raw "<EventID>(?<EventID>\d+)</EventID>"
| search EventID=1
| rex field=_raw "<Data Name='Image'>(?<Image>[^<]*)</Data>"
| rex field=_raw "<Data Name='CommandLine'>(?<CommandLine>[^<]*)</Data>"
| rex field=_raw "<Data Name='ParentImage'>(?<ParentImage>[^<]*)</Data>"
| rex field=_raw "<Data Name='User'>(?<User>[^<]*)</Data>"
| where like(lower(ParentImage), "%hfs.exe%")
| table _time User ParentImage Image CommandLine
| sort _time

To reduce the 84 matching process-creation events into a more interpretable view, the results were aggregated by process relationship and command type.

...
| eval ProcessChain=ParentProcess." → ".ChildProcess." → ".Execution
| stats count as Events earliest(_time) as FirstSeen latest(_time) as LastSeen by ProcessChain
| convert ctime(FirstSeen) ctime(LastSeen)

The results show hfs.exe → cmd.exe → Encoded PowerShell, providing a concise view of the process relationship associated with the suspicious HFS activity.

6.3 Investigation of Windows Privilege Escalation

Following confirmation of command execution through the HFS process, the investigation continued by examining subsequent activity on the Employee workstation for evidence of privilege escalation.

Sysmon process creation events were analyzed to determine whether the suspicious activity was followed by a change in execution context or by processes running under a more privileged account.

host="DESKTOP-KBDL1T4"
sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
| rex field=_raw "<EventID>(?<EventID>\d+)</EventID>"
| search EventID=1
| rex field=_raw "<Data Name='Image'>(?<Image>[^<]*)</Data>"
| rex field=_raw "<Data Name='CommandLine'>(?<CommandLine>[^<]*)</Data>"
| rex field=_raw "<Data Name='ParentImage'>(?<ParentImage>[^<]*)</Data>"
| rex field=_raw "<Data Name='User'>(?<User>[^<]*)</Data>"
| rex field=_raw "<Data Name='IntegrityLevel'>(?<IntegrityLevel>[^<]*)</Data>"
| where NOT like(lower(User), "%splunkforwarder%")
| eval Activity=case(
    like(lower(ParentImage), "%\\hfs.exe")
        AND like(lower(CommandLine), "%powershell%"), "HFS spawned encoded PowerShell",
    User="NT AUTHORITY\\SYSTEM"
        AND like(lower(CommandLine), "%employee-update-trigger.cmd%"), "Update trigger executed as SYSTEM",
    User="NT AUTHORITY\\SYSTEM"
        AND like(lower(CommandLine), "%updater.cmd%"), "Updater executed as SYSTEM"
)
| where isnotnull(Activity)
| where _time >= strptime("2026-09-29 08:19:45", "%Y-%m-%d %H:%M:%S")
| eval Command=case(
    Activity="HFS spawned encoded PowerShell", "cmd.exe -> powershell.exe -enc",
    Activity="Updater executed as SYSTEM", "cmd.exe /c updater.cmd",
    Activity="Update trigger executed as SYSTEM", "cmd.exe /c employee-update-trigger.cmd"
)
| stats earliest(_time) AS Time
        values(User) AS User
        values(IntegrityLevel) AS IntegrityLevel
        values(Command) AS Command
        by Activity
| sort 0 Time
| eval Time=strftime(Time, "%H:%M:%S.%3N")
| table Time Activity User IntegrityLevel Command

The results revealed a clear change in execution context. The initial HFS-related command execution occured under the Employee account with Medium integrity. Shortly afterward, updater.cmd and employee-uodate-trigger.cmd were executed as NT AUTHORITY\SYSTEM with System integrity. This confirmed that the compromise had progressed from user-level command execution to SYSTEM-level execution on the Employee workstation.

6.4 Investigation of Pivoting and Internal Network Activity

Following confirmation of SYSTEM-level execution, the investigation continued by reviewing subsequent process creation events on the Employee workstation. The results contained a large volume of encoded PowerShell activity, indicating continued command execution after the initial compromise.

Among these events, a particularly suspicioous curl.exe execution was identified. The command attempted to retrieve an executable named chisel.exe from 172.16.50.10 over TCP port 8082and write it to C:\ProgramData.

host="DESKTOP-KBDL1T4"
sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
| rex field=_raw "<EventID>(?<EventID>\d+)</EventID>"
| search EventID=1
| rex field=_raw "<Data Name='Image'>(?<Image>[^<]*)</Data>"
| rex field=_raw "<Data Name='CommandLine'>(?<CommandLine>[^<]*)</Data>"
| rex field=_raw "<Data Name='ParentImage'>(?<ParentImage>[^<]*)</Data>"
| rex field=_raw "<Data Name='User'>(?<User>[^<]*)</Data>"
| where _time >= strptime("2026-09-29 08:21:18", "%Y-%m-%d %H:%M:%S")
| table _time User ParentImage Image CommandLine
| sort _time

The executable name provided the first direct indication that the attacker was deploying Chisel. This finding was then used as an indicator to perform a focused search for additional Chisel-related activity on the workstation.

host="DESKTOP-KBDL1T4"
sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
| rex field=_raw "<EventID>(?<EventID>\d+)</EventID>"
| search EventID=1
| rex field=_raw "<Data Name='Image'>(?<Image>[^<]*)</Data>"
| rex field=_raw "<Data Name='CommandLine'>(?<CommandLine>[^<]*)</Data>"
| rex field=_raw "<Data Name='ParentImage'>(?<ParentImage>[^<]*)</Data>"
| rex field=_raw "<Data Name='User'>(?<User>[^<]*)</Data>"
| where like(lower(Image), "%chisel%")
    OR like(lower(CommandLine), "%chisel%")
    OR like(lower(CommandLine), "%8001%")
    OR like(lower(CommandLine), "%8082%")
| table _time User ParentImage Image CommandLine
| sort _time

The focused search confirmed repeated attempts to retrieve chisel.exe from 172.16.50.10:8082 using curl.exe, with PowerShell acting as the parent process. This established that Chisel was deliberately introduced onto the compromised Employee workstation.

Since the download alone does not demonstrate that the tool was successfully executed or used for pivoting, the investigation proceeded to search for subsequent execution of chisel.exe and its associated command-line arguments.

host="DESKTOP-KBDL1T4"
sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
| rex field=_raw "<EventID>(?<EventID>\d+)</EventID>"
| search EventID=1
| rex field=_raw "<Data Name='Image'>(?<Image>[^<]*)</Data>"
| rex field=_raw "<Data Name='CommandLine'>(?<CommandLine>[^<]*)</Data>"
| rex field=_raw "<Data Name='ParentImage'>(?<ParentImage>[^<]*)</Data>"
| rex field=_raw "<Data Name='User'>(?<User>[^<]*)</Data>"
| where like(lower(Image), "%\\chisel.exe")
| table _time User ParentImage Image CommandLine
| sort _time

The expanded investigation window revealed that chisel.exe was executed on September 30, demonstrating that the malicious activity continued beyond the initial compromise observed on September 29.

The recorded command line showed Chisel operating in client mode and connecting to 172.16.50.10:8001. The argument R:2222:192.168.56.10:22 configured reverse port forwarding from TCP port 2222 to SSH on 192.168.56.10:22.

This suggested that the attacker may have been attempting to establish a pivot through the compromised Employee workstation. However, the Chisel execution alone did not confirm that the tunnel was subsequently used to access the internal host.

6.5 Investigation of SSH Access to the Internal Server

The Chisel configuration identified 192.168.56.10:22 as the intended destination of the reverse tunnel. To determine whether the suspected pivot was successfully used, the investigation shifted to telemetry collected from the internal Ubuntu server. The analyst searched the available Linux security telemetry for SSH-related activity occuring after the Chisel tunnel was established on September 30.

index=linux_security
| search "sshd"
| table _time host sourcetype _raw
| sort _time

The audit records contained several SSH-related events. The most relevant events for determining whether interactive access occured are summarized below.

The USER_START event showed that an SSH session was successfully opened for the backupsvc account. This was followed by a successful USER_LOGIN event associated with the interactive terminal /dev/pts/0. Both events recorded 192.168.56.20 as the source address.

index=linux_security
| search ("backupsvc" AND "sshd")
| rex field=_raw "type=(?<EventType>[A-Z_]+)"
| rex field=_raw "acct=\"(?<Account>[^\"]+)\""
| rex field=_raw "AUID=\"(?<AuditUser>[^\"]+)\""
| rex field=_raw "addr=(?<SourceIP>[0-9.]+)"
| rex field=_raw "terminal=(?<Terminal>[^ ]+)"
| rex field=_raw "res=(?<Result>[a-z]+)"
| where EventType="USER_START" OR EventType="USER_LOGIN"
| table _time EventType Account AuditUser SourceIP Terminal Result
| sort _time

These events confirmed successful interactive SSH access to the internal Ubuntu server using the backupsvc account.

Correlation with the previously identified Chisel activity linked the successful SSH session with the reverse tunnel configured on the Employee workstation. The relevant Windows and Linux events were aligned chronologically to reconstruct this sequence.

6.6 Investigation of Post-Compromise Activity on the Ubuntu Server

Having confirmed successful SSH access to the Ubuntu server, the investigation continued by examining activity associated with the backupsvc session. The objective was to determine what actions were performed after authentication.

index=linux_security
earliest="09/30/2026:19:28:22"
| search "backupsvc"
| table _time _raw
| sort _time

Review of the resulting events identified several activieties relevant to the investigation:

• 20:18:12 - a failed sudo attempt was recorded for backupsvc.

• 20:31:45, 20:41:56 and 20:42:37 - /opt/company-tools/backup-helper was executed repeatedly under the backupsvc audit identity.

• The backup-helper executions were associated with the audit key capability_lpe

The repeated execution of a custom binary associated with the capability monitoring warranted further investigation. The analyst therefore narrowed the search to backup-helper activity to examine the following actions.

index=linux_security
earliest="09/30/2026:20:30:00"
latest="09/30/2026:20:45:00"
| search "backup-helper"
| table _time _raw
| sort _time

The audit trail established the following privilege-escalation sequence:

1. Execution - At 20:42:37.305, backupsvc executed /opt/company-tools/backup-helper --maintenance.

2. Privilege transition - Two milliseconds later, the same process performed a successful setuid system call.

3. Root privileges confirmed - AUID="backupsvc" preserved the originating audit identity, while UID="root" and EUID="root" confirmed that the process had transitioned to root privileges.

This confirmed successful privilege escalation from the compromised backupsvc session to root.

6.7 Attack Timeline and Event Correlation

7. Splunk Detections and Alerts

Following reconstruction of the attack timeline, the strongest indicators idenitified during the investigation were converted ito reusable Splunk detections and alerts. The objective was to detect similar malicious activity automatically without requiring an analyst to manually reconstruct the entire attack sequence.

7.1 Suspicious HFS Child Process / Encoded PowerShell

Sysmon process creation events were used to detect PowerShell execution with encoded command-line arguments. The detection focuses on powershell.exe launched with -enc or -EncodedCommand, providing a reusable indicator of potentially obfuscated command execution.

The detection was operationalized as a scheduled Splunk alert that runs every five minutes and evaluates the previous five-minute window. The alert triggers when at least one matching event is detected and records the result in Splunk's Triggered Alerts.

7.2 Chisel / Reverse Tunnel Detection

A dedicated Splunk detection was created to identify Chisel-based tunneling activity on the Employee workstation. The detection analyzes Sysmon process creation events and searches for Chisel running in client mode with reverse port-forwarding arguments. During the attack, this behavior was observed when the compromised workstation established a reverse tunnel used to access the internal Ubuntu server over SSH.

The search was operationalized as a scheduled Splunk alert running every five minutes against the preceding five-minute window. Any matching event triggers the alert and is added to the Triggered Alerts list for investigation.

The detection is intentionally tool-specific and provides a high-confidence indicator for the Chisel-based tunneling activity observed during the attack. Because it relies on the Chisel executable name and characteristic command-line arguments, alternative tunneling tools or renamed binaries may bypass this rule.

7.3 SSH Access Detection

Another Splunk detection was created to identify successful SSH access to the internal Ubuntu server. The search analyzes Linux audit events generated by sshd and extracts the authenticated user, source IP address, and terminal associated with each successful login as shown below:

The SPL query responsible for identifying this activity is shown below:

To operationalize the detection, the search was also configured as a scheduled alert running every five minutes against the preceding five-minute window. Any matching SSH login is added to the Triggered Alerts list for investigation.

7.4 Linux Privilege Escalation Detection

The final Splunk detection focuses on the Linux privilege escalation performed on the internal Ubuntu server. Linux audit telemetry captured a successful setuid operation initiated by /opt/company-tools/backup-helper, showing that the original backupsvc session transitioned to root privileges, as shown below:

The SPL query used to isolate and extract the relevant privilege transition fields is shown below:

As with the previous detections, the search was configured as a scheduled alert running every five minutes. Due to the confirmed privilege transition from backupsvc to root, the alert was assigned Critical severity and added to the Triggered Alerts list when matched.

8. Detection Summary and MITRE ATT&CK Mapping

The detections developed throughout the investigation were mapped to the MITRE ATT&CK framework to show their coverage across the observed attack chain. The table below summarizes each detection and its corresponding ATT&CK technique.

9. Conclusion

This lab demonstrated a complete attack-and-detection workflow across a segmented Windows and Linux environment. From the attacker perspective, the assessment went from initial reconnaissance and compromise of the Employee workstation to privilege escalation, internal network pivoting, SSH access, and root compromise of the Ubuntu server.

From the defender perspective, Splunk was used to centralize and analyze Windows and Linux telemetry, reconstruct the attack timeline, correlate activity across multiple hosts, and develop scheduled detections for key stages of the attack. The resulting detections demonstrate how individual security events can be transformed into meaningful indicators of compromise and actionable alerts.

Overall, the lab shows the value of combining offensive security techniques with defensive monitoring to understand both how an attack progresses and how the same activity can be identified through centralized security telemetry.