Understanding AES modes and their CTF-relevant weaknesses - ECB, CBC, CTR, and more.

AES Modes of Operation

AES itself is a secure block cipher, but the mode of operation determines how blocks are chained. CTF crypto challenges almost always attack the mode, not AES itself.

AES Fundamentals

Block size: always 128 bits (16 bytes)
Key sizes: 128, 192, or 256 bits
AES processes exactly one 16-byte block at a time

For messages longer than 16 bytes, a mode of operation is required.

Visual Comparison of AES Modes

Modes of Operation — Modes of operation for block ciphers

Mode	Parallelizable	Deterministic	Use Case	Vulnerability
ECB	Yes	Yes (Bad!)	Legacy only	Pattern leakage
CBC	No	Yes (with IV)	Standard	Padding oracle
CTR	Yes	No (with nonce)	Streaming	Nonce reuse
GCM	Partial	No (with nonce)	Authenticated	Nonce reuse

Mode: ECB (Electronic Codebook)

Each 16-byte block is encrypted independently with the same key.

P_i \xrightarrow{\;\mathrm{AES}_K\;} C_i \quad (i = 1,2,\ldots)

Fatal Flaw

Identical plaintext blocks produce identical ciphertext blocks. Patterns in plaintext survive into ciphertext.

Classic ECB Attack: Detect Mode

# Send two identical 16-byte blocks
payload = b'A' * 32
# If any 16-byte block in ciphertext is repeated → ECB
ciphertext = oracle(payload)
blocks = [ciphertext[i:i+16] for i in range(0, len(ciphertext), 16)]
if len(blocks) != len(set(blocks)):
    print("ECB detected!")

ECB Byte-at-a-Time Oracle Attack

If you control part of the plaintext prepended to a secret:

def ecb_byte_at_a_time(oracle):
    secret = b''
    block_size = 16  # detect this first

    for i in range(256):   # for each byte position
        # Craft input that aligns known data to block boundary
        padding = b'A' * (block_size - len(secret) % block_size - 1)

        # Target block (contains one unknown byte)
        target = oracle(padding)[: ((len(secret) // block_size) + 1) * block_size]

        # Try all 256 possible byte values
        for byte in range(256):
            test = oracle(padding + secret + bytes([byte]))
            if test[:len(target)] == target:
                secret += bytes([byte])
                break

    return secret

Mode: CBC (Cipher Block Chaining)

Each plaintext block is XOR'd with the previous ciphertext block before encryption.

\begin{aligned} C_1 &= \mathrm{AES}_K(P_1 \oplus \mathrm{IV}) \\ C_2 &= \mathrm{AES}_K(P_2 \oplus C_1) \\ C_3 &= \mathrm{AES}_K(P_3 \oplus C_2) \end{aligned}

Bit Flipping Attack

Flipping a bit in C[i] flips the corresponding bit in P[i+1] after decryption. No key required.

# Goal: change P[1][j] from 'X' to 'Y'
# Flip bit: C[0][j] ^= ord('X') ^ ord('Y')

ciphertext = bytearray(oracle_encrypt(payload))
target_block = 0   # modify ciphertext block 0 to corrupt plaintext block 1
target_byte  = 6   # byte position within block

ciphertext[target_block * 16 + target_byte] ^= ord('X') ^ ord('Y')
# Now decrypt → P[1][6] == 'Y'

IV = Key Attack

Some implementations reuse the key as the IV (very common CTF mistake):

# If IV == Key, you can recover the key:
# Encrypt: P → C
# Decrypt C1 || 0..0 || C1
# XOR P'[0] ^ P'[2] = K

c1 = ciphertext[:16]
crafted = c1 + b'\x00' * 16 + c1
decrypted = oracle_decrypt(crafted)
key = xor(decrypted[:16], decrypted[32:48])

Mode: CTR (Counter)

A keystream is generated by encrypting a nonce+counter, then XOR'd with plaintext. CTR turns AES into a stream cipher - decryption uses the same operation as encryption.

\text{Keystream}_i = \mathrm{AES}_K(\mathrm{nonce} \,\|\, \text{counter}_i), \qquad C_i = P_i \oplus \text{Keystream}_i

Nonce Reuse Attack

If the same nonce (counter starting point) is reused across two messages, their keystreams are identical:

C_1 = P_1 \oplus \mathrm{KS}, \quad C_2 = P_2 \oplus \mathrm{KS} \;\Rightarrow\; C_1 \oplus C_2 = P_1 \oplus P_2

Knowing one plaintext recovers the other (and the keystream):

c1 = bytes.fromhex("...")
c2 = bytes.fromhex("...")

# If P1 is known (e.g. from context)
ks = xor(c1, known_p1)
p2 = xor(c2, ks)

Mode Detection (CTF Technique)

def detect_mode(oracle):
    # oracle(plaintext) returns ciphertext
    ciphertext = oracle(b'A' * 48)
    blocks = [ciphertext[i:i+16] for i in range(0, len(ciphertext), 16)]

    if blocks[1] == blocks[2]:
        return "ECB"
    else:
        return "CBC (or other chained mode)"

Common CTF AES Scenarios

Scenario	Mode	Attack
Identical ciphertext blocks	ECB	Byte-at-a-time oracle
"Decrypt any token except admin"	CBC	Bit flipping
Encrypt-then-decrypt with same key	CBC	Padding oracle
Two ciphertexts, same key	CTR with nonce reuse	XOR ciphertexts
Fixed IV = key	CBC	Key recovery

Quick Python Helpers

from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad

def aes_ecb_encrypt(key, plaintext):
    cipher = AES.new(key, AES.MODE_ECB)
    return cipher.encrypt(pad(plaintext, 16))

def aes_cbc_encrypt(key, iv, plaintext):
    cipher = AES.new(key, AES.MODE_CBC, iv)
    return cipher.encrypt(pad(plaintext, 16))

def xor(a, b):
    return bytes(x ^ y for x, y in zip(a, b))

Last updated on Aug 1, 2025

AES Modes of Operation

On this page