[Airflow] - docker-compose.yml FERNET KEY

Hello everyone.

I’m deploying Airflow in Docker Compose and using the attribute AIRFLOW__CORE__FERNET_KEY. This enables Airflow to encrypt sensitive information in the database. So, if your database ever gets exposed or goes public without your permission, your data remains safe because no one will be able to read it without the key.

And how does it works? Fernet is an implementation of symmetric authenticated cryptography

Lets break it down this sentence. I ask to ChatGpt explain and his gave me it:

“Symmetric Cryptography is the process of encrypting and decrypting information using the same key. Think of it like a physical safe: you use the exact same key to lock your document inside and to unlock it later.

However, standard symmetric cryptography has a major vulnerability: an attacker can intercept the message in transit and alter its contents. Even if they cannot read the encrypted message, they can modify its bits, causing you to decrypt corrupted or malicious data.

This is where the Authenticated part comes in.

Authenticated Cryptography adds a “tamper-proof seal” to your encrypted message (usually using a mechanism called HMAC).

Before sending the message, the system generates a unique signature based on the encrypted content and your secret key. When the recipient receives the message, they verify this signature first. If an attacker changed even a single character of the ciphertext in transit, the signature will not match, and the system will immediately reject the message without decrypting it.

In summary: Fernet doesn’t just keep your data secret (encryption); it also proves that the data hasn’t been modified by anyone else (authentication).”

Ok. It give us some briefing of how Fernet works. Now, I’ve tested it. The test was consulting the airflow database with fernet key enable and fernet key unable.

I was looking up what the behavior of Airflow would be if AIRFLOW__CORE__FERNET_KEY is not declared in airflow.cfg. Gemini told me that Airflow generates a random Fernet Key to encrypt the data even if I don’t explicitly set it. However, in the Airflow docs, I couldn’t find anything about this explicitilly.

1. Without declaring a Fernet Key

• The steps of this test was declaring AIRFLOW__CORE__FERNET_KEY = ’’ in .env file, after that, I have run a command to create a connection:

docker exec -it airflow-airflow-webserver-1 airflow connections add test_conection_v2 \
    --conn-type generic \
    --conn-password "test_fernet"

• Then, I logged into the Airflow database and ran a SELECT query to see if the connection password was encrypted, and it was. So, by default, airflow encrypt the password even if I don’t declare it.

2. With declaring a Fernet Key
   • Now I’ve declared a Fernet key. It’s important to mention that a Fernet key needs to be 32 bytes (256 bits). You can easily generate one using Python’s cryptography library. To compare this with the unencrypted connection from my previous test, I kept the Docker volumes, restarted the containers, and created a new connection using the exact same process as in TEST 1. Here is the SELECT query showing the results: