We’re back with part three of a close look at how 1Password works. So far we’ve seen how the Two-Secret Key Derivation (2SKD) process is used to unlock macOS clients, and how the Encrypted Master Key (EMK) does the same under Windows. In both cases, we end up with a decrypted master key, the “sym key” in the account’s first keyset. As I’ve said in both prior segments, this key then lets us descend into the vault and decrypt everything else.

Well, now it’s finally time to get to the fun stuff!

Steampunk 1Password

How exactly the vaults are set up is a bit complicated, but it’s all for a very good reason. Rather than jumping straight into the technical bits, let’s look at the system from a different angle.

Imagine that you have a bunch of passwords you need to keep track of. Because they get changed from time to time, you don’t write them into a book (because you’d also like to keep them neatly sorted), so instead you put them on index cards. Being security conscious, you don’t simply leave these cards on your desk – you put them into a drawer and lock it. (Extra points if it’s made with dark wood and lots of gleaming brass).

Now imagine that you have a different set of passwords that you need to share with your coworkers. So you get another drawer, put the passwords in there, and give everyone on the team a key to that drawer. And so on.

Card Catalogs FTW!

Card Catalogs FTW!

With me so far? Good. Because here’s where it gets interesting.

What if one of your team members is out on vacation when you pass out the keys? You obviously can’t just leave the key on their desk… So instead, you order a bunch of little toy banks. These all have a combination lock to open the door, and a coin slot on the top. Everyone on the team gets one of these, each with their own combination.

Now, when you need to distribute keys for a new drawer, you just drop the keys through each person’s coin slot, and now they have access.

But what if you forget your combination? Well, naturally, you write it down, and seal that in an envelope (so you’ll know if someone’s tampered with it), and then lock that in a drawer in your desk.

So now our system looks like this:

Old-School Password Management System

Old-School Password Management System

  • Master key: unlocks desk drawer
  • Sealed envelope: protects backup copy of combination
  • Box: Contains individual drawer keys
  • Coin slot: lets team members add keys to your box
  • Combination: unlocks the desktop box
  • Drawer key: unlocks drawers, and thus, individual cards (passwords)

So now we’ve replicated 1Password in the Real World. But how does it really work, on the computer?

Actual Vault Structure

I’ve just told you! This structure is exactly how 1Password vaults work. If you recall, the Master Password eventually decrypts a master key, an AES encryption key stored in the primary vault’s first keyset entry. That key corresponds to the key you use to unlock your desk. Here’s a full mapping of the (only slightly stretched) metaphor:

Steampunk Action 1Psssword Action
Desk Key Unlocks Desk Master Unlock Key Decrypts 1st keyset AES key
Sealed Envelope Tear open Master Key Decrypts 1st keyset RSA private key
Key Box (toy safe) Contains keys Keyset Contains keys
Coin Slot Others add new keys RSA Public Key Others encrypt new keys for you
Combination Lock Opens keybox RSA Private Key Decrypts vault keys, other keysets
Drawer Key Opens drawer, reveals passwords Vault Key Decrypts vault items (passwords)

Let’s see how this analogy maps up visually:

Steampunk meets Cipherpunk

Steampunk meets Cipherpunk

Keysets

Every account includes at least one, and possibly more, keysets. These are stored (predictably) in the keysets table of the 1Password database. The first keyset entry for each account is used to decrypt the others.

In the macOS example, we looked at the enc_sym_key item for that first keyset. It was encrypted using the Master Unlock Key (key id “mp”). In all the other keysets, that key is encrypted using the RSA key that we’re about to decrypt.

Here’s what that first keyset looks like, showing all the critical components:

Keyset 1:
{
    "encrypted_by": "mp", 
    "enc_pri_key": {
        "enc": "A256GCM", 
        "data": "V1sG-80rSSwRw7vpHj6dH159IF-35WulxFh_LzJVvu1wb4GXmc7aZzhyMWAx9kLN4t9ZXiY_gdgg2HgAQXGAT90tWrg6SPW6v-uIWWlYAcDKdBefJtQGJSjlCUgrZnMp3mSyxWfKsfiHQcD6-uhi9x6HIDUjAqUWBM11Bx0v5XNRKImgKDQZbkhwOysufLw41OpkdrSSSmb-IEyiPfFwZwnyYaswTVuASjgp6nuxjzfNk-PkvoblmzQ_0c5GGrLdnY2QNMsjYPFQw01bNfBHBww50jnuNM0bN5E279M6pKCH-y1k1Jm2xrYFhMSH87IqeSvOWadcopian8swGzAtgvCoB_STMrlDF5gvDd5i0VKMtbJkuYAME1H7OxEls9x4Yjq1_l4DND1dNoSEEFgjMusMitDzCWqR09B2BkBGUSPQNThm5zdfhuZTSyO8VC9o8K7rDMhczr-u_cLUimXUMjz_7m5UEFRBcpJDntvG_J5dfYrD3YIzA6um_pWkkg_GEORJapsIVVN1FtoV-_h6JcwX0GOjouSdTxTOH24-7qybNy0qYqW30YyFSWlROPhjYrntWDTt2MpA4Qsjg4EmrLkmlWH6oOLG70lJkXUp4qIRTyUC9wsCmobL9DM0eknLptNAQfHwI_227xbVjhfx1vy5FlTOjlvJ5JYhZVpsKvB1KKM7ENf0vJIxGFWgJeedw_R9nZbFg_6eVTzVYk9ndPrJQVvX3Dmn3fUIzQYnj4uEEOZ0TnEYhk5fU20ILQBJyVa1rrFdSyT9m5hVphCx56gHM_HpRvxx3-7kr6Qtx-yxA9JEUI_Gxx8JeYPY9cjqh0i6PGN3f8VbM-a6_XfDsPfH5I2yGsSMdI2W1nV7fXCaedfIr0QBMnvzRgQdhZtdZtN-Q2rUkwL_Erc8PYkO_eNrdKok4D8sL5MIryJ00sWBe6QjefWYV7Xu_rtbtI5rBxMQ9n_Hfoq9kCizYh1z-25Dki5GwgyfCoOXLsP9MTai92afuXFbPLwmJ_YCB2UHgYUX-V8uMvfVroTfjlqgkF6gtvI5eOpbpYsb6B4q2hC4GgyzhlqQ053hgRy4noRcqDl2-75K1qhgnGD-kfA1oO1GtF3RIjDeswU7IoH58f-QMvM868Fkx9-DjW8VTDEUPrrGVhVvG4KHUVh_hiVqE5N-azAJJZ8gBAQ_-5dCdkKA_9P4Wi3UOydCoTZWCn1wJ5eAUn2DrcfQ-VtBVhoqdhn5LVhTbT2LmSfwVzDRyKhCdWbWZjutkHl64RCen-MEa_cBL6351U3fskOUTQAgbS_UmQ5JVdcJBSZD3ilVEVsbVlg-jIfz2Ray0T0jnhfWkw91fDs6R9_eSMTkxNPCrQI0RH3t0V_2m_WPV7Eym-3twnebnZGnHJuvyLBDW35m7sUlMdx8eifOw_l2eH8r9ZddIWFXhkBa0AyfSywuDjAYnmnAkCESEYXiWTWMhywmLFDTuUoAhcz4F5sz7fkNFx8pwSqAqtSBHEW53Vvwuy2c57L8MARM00-HR4qDygxoypJFATny5SmznhpYbCmT7A9fwKQLWNQbiUAdQfDI-ccM", 
        "cty": "b5+jwk+json", 
        "iv": "x_pZCisivs-aCINbqS4fLQ==", 
        "kid": "qn8uimc4l7sofa26yivex24j7q"
    }, 
    "pub_key": {
        "key_ops": [
            "encrypt"
        ], 
        "e": "AQAB", 
        "kty": "RSA", 
        "alg": "RSA-OAEP", 
        "n": "xA6dAIu2_S9Ia_xRkodmvBv9w4pMyjE7FFAiXKTcQJS8d1RLkY82hwghBa6YK7V28_-S0Hfe2_NecesRMCpDf03kl1SClJkl8bJpJ0AwZFhvj6JO1JUZAj8o06OpgUCij_Jt8YSiu8bQIXgH5bEEkZ3oBx1OyozgqCo6JBa7cQVlv2LGV25YnqIbzOTof8YBZMNM0GuzPQQDxJUEB4ktmKekFjtDvHzAUmtMEgGYpbgXl4AmRAbHlYPpepSBplqXSrJfxVfEgftAJudjQsrMr_uVNX5TYGgFJDqUzkiXBXEUFy22GqcIArLLiOtvUwEU843wYpLtSPN-A20YLfSTCw", 
        "ext": true, 
        "kid": "qn8uimc4l7sofa26yivex24j7q"
    }, 
    "enc_sym_key": {
        "enc": "A256GCM", 
        "p2c": 100000, 
        "alg": "PBES2g-HS256", 
        "cty": "b5+jwk+json", 
        "iv": "2FF8mtGD55z84h9jMtWAyQ==", 
        "p2s": "cA4f6QY7wwUoclj74RMvUg==", 
        "data": "8OjOA2NqUZZGxXD4r-z4QUfxjvuk23_i0DFAcYxx1r84hmsG1KV1G9iKBZd-kFpfzDgciJD3h8d91OT9D6F8KVqvdmx_q649mWEhiWwVcmRlKRVzgj-eZunS1XHxwHYDhvNvdzKUpNdAp7EKsQCRpiJJ3-eTndQBFMdyeCwkxnqMkuGW326P_mjW5yp_qYpGc4HgpY-_3aEhKimKVGJuxL4I5U5LU2ZFVNNhRIxkjJShtEwtXcTaVwH6", 
        "kid": "mp"
    }, 
    "uuid": "qn8uimc4l7sofa26yivex24j7q"
}

We start with the last key, “enc_sym_key.” Using the MUK from previous installments, we again decrypt this key:

$ python gcm_decrypt.py 
Enter AES-256 key (hex or base-64 encoded)
 --> 6VIFjzKXaHctk44NXr8hOenpgwxWgZebtOlGkPwGK+Y=

Enter IV (hex or base-64): 2FF8mtGD55z84h9jMtWAyQ==

Enter ciphertext (hex or base-64): 8OjOA2NqUZZGxXD4r-z4QUfxjvuk23_i0DFAcYxx1r84hmsG1KV1G9iKBZd-kFpfzDgciJD3h8d91OT9D6F8KVqvdmx_q649mWEhiWwVcmRlKRVzgj-eZunS1XHxwHYDhvNvdzKUpNdAp7EKsQCRpiJJ3-eTndQBFMdyeCwkxnqMkuGW326P_mjW5yp_qYpGc4HgpY-_3aEhKimKVGJuxL4I5U5LU2ZFVNNhRIxkjJShtEwtXcTaVwH6

Output:

{
    "alg": "A256GCM",
    "ext": true,
    "k": "Sco1rWpdmrLiAeZNtwAlCQsMMqN46AnyGasaMu3EqlQ=",
    "key_ops": [
        "decrypt",
        "encrypt"
    ],
    "kid": "qn8uimc4l7sofa26yivex24j7q",
    "kty": "oct"
}

Now we have the AES key designated “qn8u….47jq”. This key is then used wherever an encrypted item is found with a matching encryption key id, and using the 256-bit AES GCM algorithm. The immediate example is the “enc_pri_key” in the same keyset:

$ python gcm_decrypt.py 
Enter AES-256 key (hex or base-64 encoded)
 --> Sco1rWpdmrLiAeZNtwAlCQsMMqN46AnyGasaMu3EqlQ=
Enter IV (hex or base-64): x_pZCisivs-aCINbqS4fLQ==
Enter ciphertext (hex or base-64): V1sG-80r (.... see above for full data field ....)-ccM

Output:

{
    "alg": "RSA-OAEP",
    "d": "XcdvqfcqjGi1h5GloyVJKulotMPOf1iVHd5G0XG6ONnsXFfh3bpXJrfos8MT3rRqNcQmAbmUzDjZEDyUeCl_J8GmegxeeZ3X3Iiua6v0ecsjcdz9QAohcEWtza4XQlAcciZQGJqNDKzImXnErUXDHbQebGjEa3Z_b3DjZqfI-QH5DYDMh5W61L7Ky8_8kc54A9EtJupqtKZwYnBtazzLTcl82APkyQ71aN9kD-iO8qA3lAQGkUykRBa_8TF0tDCGgFfW7ijcGk06NGsYex9ir_n8fYZOP3LXahEMO5_3j4vIixmktFpI8IhtdQNvXqYir3JyB3WOmszr5XC4VasAYQ",
    "e": "AQAB",
    "key_ops": [
        "decrypt"
    ],
    "kid": "qn8uimc4l7sofa26yivex24j7q",
    "kty": "RSA",
    "n": "xA6dAIu2_S9Ia_xRkodmvBv9w4pMyjE7FFAiXKTcQJS8d1RLkY82hwghBa6YK7V28_-S0Hfe2_NecesRMCpDf03kl1SClJkl8bJpJ0AwZFhvj6JO1JUZAj8o06OpgUCij_Jt8YSiu8bQIXgH5bEEkZ3oBx1OyozgqCo6JBa7cQVlv2LGV25YnqIbzOTof8YBZMNM0GuzPQQDxJUEB4ktmKekFjtDvHzAUmtMEgGYpbgXl4AmRAbHlYPpepSBplqXSrJfxVfEgftAJudjQsrMr_uVNX5TYGgFJDqUzkiXBXEUFy22GqcIArLLiOtvUwEU843wYpLtSPN-A20YLfSTCw",
    "p": "0It5RDblXwYnJg-xuBrww6bxNr11x8ILCEVojwuAaNFegAqwPHbUw4nekx5mML30HltVgg3i3bi0ITLdHVqvdy9zUetTsEhsYlk9Zq8ox6nGQ9qEa-Hnu4YCB5Uh5iHMBZyUlmjRUPh1V7NcyafzjgJSin8-Me_DKrHxdalU6y8",
    "q": "8Kuuf2mnoNK1skNuxJU38Q6HC6cq9JoHN1U5dYKIcAXd0B1wEqHGcbo8UyviftfdPRy2fomKu1-c0uWcOzBZmlV4SkQ-_TwxcFPgTVcrhuAESHERZIYJuIr6JENoD7iph_BGOF-ftVGBULT7fFRH47t0jPkfTolXeC2tLIsQbuU"
}

Now that I’ve decrypted the private key, I can use this to decrypt other keysets:

Keyset 2:
{
    "encrypted_by": "qn8uimc4l7sofa26yivex24j7q", 
    [.....]
    "enc_sym_key": {
        "enc": "RSA-OAEP", 
        "data": "jwXCx1Qer2eCkJa4hRqplAxOw65uemSROLXpp6QIVrsa81q2czxyTinF6trlrQa-Q2M8633xOydEDvJp9pwHWhIHtiyzwwkYy0SCJjp4wF-4qLv-EWx5CS8b3G5m9vEhW11UOQ0aV95JLQLLVj4o4FR6O8z_fc8cz6SOvBiwyNmoRn18lUoJA1jLEMtGjTxvTxiY0hkyyedllXteNwG0wa-nIYV9VhjhHU6Ibh_anDnbb1Eq_rYoHoOHj9756oj8X-NcWIpvuEnH47C-RxS0bU7bbjQNHPeNPJLEMsTzJvUKmqlqYxA391T4UwRX81biu4TsQp9-SE-TWYgSTWx4QA==", 
        "cty": "b5+jwk+json", 
        "kid": "qn8uimc4l7sofa26yivex24j7q"
    }, 
    "uuid": "s08414l8481og4hk36hw8jshn0"
}

Which decrypts to:

{
    "alg": "A256GCM",
    "ext": true,
    "k": "E8nTamfBwcgmCR3cq5Z5b5ssaRvj2xMVc_10NdTZhkE=",
    "key_ops": [
        "decrypt",
        "encrypt"
    ],
    "kid": "s08414l8481og4hk36hw8jshn0",
    "kty": "oct"
}

and so forth.

Vaults

Vaults are actually spread across three different tables in the 1Password database. Keys are stored in the “vault access” table, information about each vault (like name, etc.) in the “vaults” table, and individual entries are in the “items” table.

The keys are decrypted with the private keys stored in the keysets, while vault information and and items are decrypted by the vault keys.

Vault Structure Details

Vault Structure Details

A vault key entry looks like this:

{
    "cty": "b5+jwk+json",
    "data": "nIqHzfOGVlyYTot8niXg5-Xc09GgZqDSK9CSwDRz5IEWO1wD9tT0mMxERRPEW2Um8SRpTI_CeKCWcI4S6YYyukfO0ylTkWgTsKb2tUSDZa0UHhTPQNO9TdeiKOc-34dhJRBcM-fysD17Qka8v2rrYqpj66G3fmDIyXizYTCRXtcl8KwsLhD0e_YWpPgFJxbRoBAxuu4gNJwF_EUxpZtPBy4gd42SKvdysd2xt6V-4D2sT7R9UTdyopIAuosnCI8eayBju55GCO6JXmzZWCEB_j8ItDFTfFbq3BEpzQR8gLukOFD33IaK3UrXkErtHph3IU-3lGdV-ITx0s6Q_vDuYw==",
    "enc": "RSA-OAEP",
    "kid": "qn8uimc4l7sofa26yivex24j7q"
}

The vault information entry:

{
    "cty": "b5+jwk+json",
    "data": "k9ilYNPPLH24pdkvUIALswSHSEpAfIn_hvFyd65osu208FKwvUlDw8t5LfhUvp8GrkWhxfycqMPlONUe8zl3gv-6wA4PTH-fP56SWaz-MOkrfTucnPSKCWZRrLAquM6bPxGDAmwKJYVFVqGpSEtK8ypVQ3lIe8hqwtZPILYVG6lh6kv-z7j0",
    "enc": "A256GCM",
    "iv": "py0VvhU4S0lsVp3HCWPVBQ==",
    "kid": "r07y2eh9nj8vjf20g6a9vpbkv7"
}

and an example password item:

{
    "details": {
        "cty": "b5+jwk+json",
        "data": "McvWj2omT-hVTd1x9cSpZpgmxKephqcmr7eNy2rO2gDlQ-PO8IEWxmhiGtlHHskD2j5SO9yWYNYWj1CBjphBmUyT7RAn6z-Eq_0Y0FuCIcxOKDlNJsSeDb2573E2tv3Rsj2bffitwTsKerTJscs8k5RofXAo-Qg_S_caFcKopXIaj47_sa_y-g==",
        "enc": "A256GCM",
        "iv": "oWvPO1H5ff8cuXR3ZrzFMA==",
        "kid": "r07y2eh9nj8vjf20g6a9vpbkv7"
    },
    "item_num": "items.1",
    "overview": {
        "cty": "b5+jwk+json",
        "data": "DBhEc4m6GsGIUKpJfcbscxgzCiaNheXa7y4OkVXHiarQXrSm5mxJwEPr7ZRsomB3Ub4nvGuxjjlGYOB2TZ4vL52oxhVAUBfPragCRcp28pp_mOLi7Ldp7XHeUh4=",
        "enc": "A256GCM",
        "iv": "k-WuwHdm0I7GrEus5zsWvw==",
        "kid": "r07y2eh9nj8vjf20g6a9vpbkv7"
    },
    "vault": "vault1"
}

Let’s decrypt that password item! First, we need to get the vault key, using the RSA key we decrypted above (for “qn8u”):

$ python rsa_decrypt.py 
Enter RSA private key (as json, decrypted from keyset): {"key_ops": ["decrypt"], "e": "AQAB", "d": "Xcdvqfcqj [.... this gets kind of long .... ], "kty": "RSA"}

Enter ciphertext (base64): nIqHzfOGVlyYTot8niXg5-Xc09GgZqDSK9CSwDRz5IEWO1wD9tT0mMxERRPEW2Um8SRpTI_CeKCWcI4S6YYyukfO0ylTkWgTsKb2tUSDZa0UHhTPQNO9TdeiKOc-34dhJRBcM-fysD17Qka8v2rrYqpj66G3fmDIyXizYTCRXtcl8KwsLhD0e_YWpPgFJxbRoBAxuu4gNJwF_EUxpZtPBy4gd42SKvdysd2xt6V-4D2sT7R9UTdyopIAuosnCI8eayBju55GCO6JXmzZWCEB_j8ItDFTfFbq3BEpzQR8gLukOFD33IaK3UrXkErtHph3IU-3lGdV-ITx0s6Q_vDuYw==

{
    "alg": "A256GCM",
    "ext": true,
    "k": "N5UH1HxXJgtTSrvMHWStrEnuiHiq9Q1Vf064XlCYfgg=",
    "key_ops": [
        "decrypt",
        "encrypt"
    ],
    "kid": "r07y2eh9nj8vjf20g6a9vpbkv7",
    "kty": "oct"
}

The value for “k” is the actual AES key, base-64 encrypted. Now we can decrypt the vault information:

$ python gcm_decrypt.py 
Enter AES-256 key (hex or base-64 encoded)
 --> N5UH1HxXJgtTSrvMHWStrEnuiHiq9Q1Vf064XlCYfgg=

Enter IV (hex or base-64): py0VvhU4S0lsVp3HCWPVBQ==

Enter ciphertext (hex or base-64): k9ilYNPPLH24pdkvUIALswSHSEpAfIn_hvFyd65osu208FKwvUlDw8t5LfhUvp8GrkWhxfycqMPlONUe8zl3gv-6wA4PTH-fP56SWaz-MOkrfTucnPSKCWZRrLAquM6bPxGDAmwKJYVFVqGpSEtK8ypVQ3lIe8hqwtZPILYVG6lh6kv-z7j0

Output:

{
    "avatar": "",
    "desc": "unk-b64-blob",
    "name": "Test vault: vault1",
    "type": "P",
    "uuid": "ixaw6slq5k7c7d71lwzkh87qy1"
}

The password entry itself has two different encrypted fields – the overview and details fields. These are encrypted separately, for two reasons. First, the “really sensitive” data only gets decrypted when the user specifically asks for it. Second, when the user opens a vault, they expect to see at least some information about every item (name, description, etc.) so they know which item contains what they’re looking for. By keeping the overview information separate, and smaller, it’s much easier for the client to decrypt a lot of these at once for display in the client.

First, the overview data:

$ python gcm_decrypt.py 
Enter AES-256 key (hex or base-64 encoded)
 --> N5UH1HxXJgtTSrvMHWStrEnuiHiq9Q1Vf064XlCYfgg=

Enter IV (hex or base-64): k-WuwHdm0I7GrEus5zsWvw==

Enter ciphertext (hex or base-64): DBhEc4m6GsGIUKpJfcbscxgzCiaNheXa7y4OkVXHiarQXrSm5mxJwEPr7ZRsomB3Ub4nvGuxjjlGYOB2TZ4vL52oxhVAUBfPragCRcp28pp_mOLi7Ldp7XHeUh4=

Output:

{
    "ainfo": "account-name",
    "title": "My test!",
    "url": "https://example.com"
}

And now the actual item:

$ python gcm_decrypt.py 
Enter AES-256 key (hex or base-64 encoded)
 --> N5UH1HxXJgtTSrvMHWStrEnuiHiq9Q1Vf064XlCYfgg=

Enter IV (hex or base-64): oWvPO1H5ff8cuXR3ZrzFMA==

Enter ciphertext (hex or base-64): McvWj2omT-hVTd1x9cSpZpgmxKephqcmr7eNy2rO2gDlQ-PO8IEWxmhiGtlHHskD2j5SO9yWYNYWj1CBjphBmUyT7RAn6z-Eq_0Y0FuCIcxOKDlNJsSeDb2573E2tv3Rsj2bffitwTsKerTJscs8k5RofXAo-Qg_S_caFcKopXIaj47_sa_y-g==

Output:

{
    "fields": [
        {
            "name": "username",
            "type": "T",
            "value": "user"
        },
        {
            "name": "password",
            "type": "P",
            "value": "password"
        }
    ]
}

And that’s how 1Password encrypts your data!

We’ve covered how Windows and macOS clients unlock, and now the vault structure. Next time we’ll discuss (briefly) how local vaults change things, and then I’ll try to wrap it up.