Let's go in depth: Wireshark TLS Decryption Guide For Java

Introduction

Sometimes during project development, it is necessary to use tools like Wireshark to analyze the underlying network communication. However, as SSL/TLS has become the standard for secure network communication, all data is encrypted, making it impossible to directly observe the actual payload.

To decrypt TLS traffic in Wireshark, we need a way for the client or server to export the master secret key after the SSL/TLS certificate exchange. According to the TLS standard, once the certificate exchange and verification are complete, the communication channel switches from asymmetric encryption (RSA/ECDH) to symmetric encryption (AES/GCM, AES/CBC, etc.).

The rationale for this fallback is that public key encryption is computationally expensive and unnecessary for encrypting the bulk of the data. Symmetric encryption is much faster and provides the same level of security for ongoing communication. By capturing the master secret key, we can decrypt the encrypted TLS traffic in Wireshark and inspect the actual content exchanged between client and server.

This document explains how TLS encryption/decryption works in a Java client and how to use the extract-tls-secrets-4.0.0.jar agent to export TLS secrets for analysis, both standalone and with a Payara application server.

TLS Decryption & extract-tls-secrets Usage in Java

When a Java application (e.g., HttpsURLConnection, SSLSocket) communicates over HTTPS/TLS:

Component	Responsibility
OS (Kernel/TCP Stack)	Handles TCP/IP: packet receipt, checksum, segmentation, reassembly. Does not decrypt TLS.
JVM TLS Library (JSSE / Conscrypt / BouncyCastle)	Performs TLS handshake, key derivation, encryption, decryption, integrity verification. Produces plaintext for the app.
Application Server / Client App	Receives plaintext data after JVM decryption, executes business logic.
TLS Agent (extract-tls-secrets)	Hooks into JVM TLS APIs to capture pre-master/master secrets for external decryption. Does not modify payload.

This swim lane diagram clearly shows who does what at each step of TLS communication between a Java client/server and the network, highlighting the role of each component in encryption, decryption, and secret logging.

Decryption is performed by the JVM TLS library.

· The TLS Agent only logs secrets.

· The OS only handles transport of encrypted bytes.

· The Application Server works purely with plaintext after decryption.

Step-by-Step Explanation:

1. OS Lane

o Receives encrypted TCP segments from the network.

o Validates checksums and TCP flags.

o Reassembles TLS records into a continuous stream for the JVM.

2. JVM TLS Library Lane

o Reads encrypted bytes from the OS.

o Performs TLS handshake (ClientHello, ServerHello).

o Validates server certificates.

o Generates pre-master and master secrets.

o Expands keys and decrypts ApplicationData records.

o Verifies integrity and produces plaintext for the application server.

o Encrypts response data before sending back to OS.

3. TLS Agent Lane

o Hooks into the JVM TLS library.

o Captures pre-master and master secrets during handshake.

o Logs secrets to a file for use with Wireshark.

o Does not perform decryption or modify TLS data.

4. Application Server Lane

o Receives plaintext HTTP requests from JVM.

o Parses headers and body, validates request data.

o Executes business logic.

o Generates HTTP response, which is then encrypted by JVM before being sent to the OS.

`extract-tls-secrets` Overview

Decrypt HTTPS/TLS connections on-the-fly. Extract the shared secrets from secure TLS connections for use with Wireshark. Attach to a Java process on either side of the connection to start decrypting.

· Java agent to extract TLS secrets from running JVM processes.

· Can be used standalone (attach to HttpURLConnectionExample) or with application servers like Payara.

· Output secrets can be used by Wireshark to decrypt TLS traffic.

Using extract-tls-secrets Standalone and Run with TLS Agent

Download this extract-tls-secrets-4.0.0.jar from https://repo1.maven.org/maven2/name/neykov/extract-tls-secrets/4.0.0/extract-tls-secrets-4.0.0.jar

Attach on startup

Add a startup argument to the JVM options: -javaagent:<path to jar>/extract-tls-secrets-4.0.0.jar=<path to secrets log file>

For example to launch an application from a standalone java file run:

java -javaagent:/path/to/extract-tls-secrets-4.0.0.jar=/path/to/secrets.log HttpURLConnectionExample

· /path/to/secrets.log will contain TLS session secrets.

· These can then be configured in Wireshark to decrypt the traffic.

Using extract-tls-secrets with Payara Server and Run with TLS Agent

JVM Startup Option : Captures TLS secrets for all JVM-initiated connections after startup.

Add the Java agent in Payara JVM options:

asadmin create-jvm-options "-javaagent:/path/to/extract-tls-secrets-4.0.0.jar=/path/to/secrets.log"

Hot-Attach to Running Payara

Only captures new TLS sessions after attachment. No runtime toggle; to “disable,” restart JVM without -javaagent.

Attach agent to running process:

java -jar /path/to/extract-tls-secrets-4.0.0.jar <PID> /path/to/secrets.log

TLS secret key logs

TLS 1.3 (with traffic secrets)

In modern TLS 1.3, tools like export-tls-secrets or SSLKEYLOGFILE produce logs with named traffic secrets. Example (secrets.log):

TLS 1.3 no longer uses a single “master key.” Instead, it derives multiple secrets (handshake, application traffic, etc.) from the initial key exchange.

Each line has: <Secret_Type> <ClientRandom> <Secret_Value_Hex>

Examples of Secret_Type:

CLIENT_HANDSHAKE_TRAFFIC_SECRET
SERVER_HANDSHAKE_TRAFFIC_SECRET
CLIENT_TRAFFIC_SECRET_0
SERVER_TRAFFIC_SECRET_0

TLS 1.2 and earlier (RSA / Master Secret)

Older SSL/TLS (RSA key exchange) used a single Master-Key.

TLS 1.2 (RSA) logs include Session-ID and Master-Key.
The Master-Key is used to derive session keys for encryption.
Only one line per session; no multiple traffic secrets like TLS 1.3.

Example (secrets.log):

Using TLS Secrets in Wireshark

Open Wireshark and open .pcap file.
Go to: Edit → Preferences → Protocols → TLS

o Upload (Pre)-Master-Secret log filename.

o Set (Pre)-Master-Secret log filename to the secrets log path.

TLS packets will now decrypt automatically.

TLS Version	Log Style	Example
TLS 1.3	Traffic secrets per direction/stage	CLIENT_HANDSHAKE_TRAFFIC_SECRET <ClientRandom> <HexKey>
TLS 1.2/SSL	Single master key per session	RSA Session-ID:<id>\nMaster-Key:<hex>