You are currently viewing How To Manage Encryption In Cloud Storage

How To Manage Encryption In Cloud Storage

  • Post author:
  • Post category:GCP

How To Manage Encryption In Cloud Storage

Hello Everyone

Welcome to CloudAffaire and this is Debjeet.

In the last blog post, we have discussed Access Logs in Cloud Storage.

https://cloudaffaire.com/how-to-enable-and-view-access-logs-in-cloud-storage/

In this blog post, we will discuss encryption in Cloud Storage.

Encryption In Cloud Storage:

Cloud Storage always encrypts your data on the server-side, before it is written to disk, at no additional charge. Besides this standard behavior, there are additional ways to encrypt your data when using Cloud Storage.

  • Server-side encryption: encryption that occurs after Cloud Storage receives your data, but before the data is written to disk and stored.
    • Customer-supplied encryption keys: You can create and manage your own encryption keys for server-side encryption, which act as an additional encryption layer on top of the standard Cloud Storage encryption.
    • Customer-managed encryption keys: You can generate and manage your encryption keys using Cloud Key Management Service, which act as an additional encryption layer on top of the standard Cloud Storage encryption.
  • Client-side encryption: encryption that occurs before data is sent to Cloud Storage. Such data arrives at Cloud Storage already encrypted but also undergoes server-side encryption.

Google-managed encryption keys:

Cloud Storage manages server-side encryption keys on your behalf using the same hardened key management systems that we use for our own encrypted data, including strict key access controls and auditing. Cloud Storage encrypts user data at rest using AES-256. There is no setup or configuration required, no need to modify the way you access the service, and no visible performance impact. Data is automatically and transparently decrypted when read by an authorized user.

Customer-supplied encryption keys:

As an additional layer on top of Google-managed encryption keys, you can choose to provide your own AES-256 key, encoded in standard Base64. This key is known as a customer-supplied encryption key. If you provide a customer-supplied encryption key, Cloud Storage does not permanently store your key on Google’s servers or otherwise manage your key. Instead, you provide your key for each Cloud Storage operation, and your key is purged from Google’s servers after the operation is complete. Cloud Storage stores only a cryptographic hash of the key so that future requests can be validated against the hash. Your key cannot be recovered from this hash, and the hash cannot be used to decrypt your data.

When you apply a customer-supplied encryption key to an object, Cloud Storage uses the key when encrypting:

  • The object’s data.
  • The object’s CRC32C checksum.
  • The object’s MD5 hash.

Cloud Storage uses standard server-side keys to encrypt the remaining metadata for the object, including the object’s name. This allows you to read and update general metadata, as well as list and delete objects, without needing the customer-supplied encryption key.

For example, if an object is encrypted with a customer-supplied encryption key, the key must be used to perform operations on the object such as downloading or moving it. If you attempt to read the object’s metadata without providing the key, you receive metadata such as the object name and Content-Type, but not the object’s CRC32C checksum or MD5 hash. If you do supply your key with the request for the object metadata, the object’s CRC32C checksum and MD5 hash are included with the metadata.

Restrictions:

  • Cloud Storage Transfer Service, Cloud Dataflow, and Cloud Dataproc do not currently support objects encrypted with customer-supplied encryption keys.
  • You cannot use the Google Cloud Console to download objects that are encrypted with a customer-supplied encryption key. Similarly, when you use the Google Cloud Console to upload an object, you cannot encrypt the object with a customer-supplied encryption key. You can perform these actions with customer-managed encryption keys.
  • To use customer-supplied encryption keys with gsutil, you must have gsutil 4.18 or later.
  • You can only set customer-supplied encryption keys on individual objects. You cannot set a default customer-supplied encryption key for a bucket.
  • If you are performing a compose operation on objects encrypted by customer-supplied encryption keys, the component objects must be encrypted by the same key, and you need to provide the key with the compose request. The resulting composite object is encrypted by the same key.

Customer-managed encryption keys:

As an additional layer on top of Google-managed encryption keys, you can choose to use keys generated by Cloud Key Management Service. Such keys are known as customer-managed encryption keys. If you use a customer-managed encryption key, your encryption keys are stored within Cloud KMS. The project that holds your encryption keys can then be independent from the project that contains your buckets, thus allowing for better separation of duties.

When you apply a customer-managed encryption key to an object, Cloud Storage uses the key when encrypting:

  • The object’s data.
  • The object’s CRC32C checksum.
  • The object’s MD5 hash.

Cloud Storage uses standard server-side keys to encrypt the remaining metadata for the object, including the object’s name. Thus, if you have sufficient permission, you can perform actions such as reading most metadata, listing objects, and deleting objects even after you’ve disabled or destroyed the associated customer-managed encryption key.

Restrictions

The following restrictions apply when using customer-managed encryption keys:

  • Cloud SQL exports to Cloud Storage and Dataflow do not currently support objects encrypted with customer-managed encryption keys.
  • You cannot use the JSON API Copy Object method when the source object is encrypted with a customer-managed encryption key or when the destination object would become encrypted by a customer-managed encryption key. Use the Rewrite Object method instead.
  • You cannot use the JSON API Compose Object method when one or more of the source objects are encrypted with a customer-managed encryption key.
  • You cannot encrypt an object with a customer-managed encryption key by updating the object’s metadata. Include the key as part of a rewrite of the object instead.
  • You must create the Cloud KMS key in the same location as the data you intend to encrypt. For example, if your bucket is located in US-EAST1, any Cloud KMS key encrypting objects in that bucket must also be created in US-EAST1.
  • You cannot specify a customer-managed encryption key as part of a Storage Transfer Service transfer, and any such keys on source objects are not applied to the transferred objects. Set a default customer-managed key on your bucket prior to performing the transfer.

Note: Because most metadata can be read regardless of the encryption method, do not include sensitive information in your object and bucket metadata or names.

How To Manage Encryption In Cloud Storage

Encryption using customer-supplied encryption key.

Encryption using customer-managed encryption key.

Hope you have enjoyed this article. In the next blog post, we will discuss Access Control in cloud storage.

All the public cloud providers are changing the console user interface rapidly and due to this some of the screenshots used in our previous AWS blogs are no longer relevant. Hence, we have decided that from now onwards most of the demo will be done programmatically. Let us know your feedback on this in the comment section.

To get more details on cloud storage, please refer below GCP documentation.

https://cloud.google.com/storage/docs/

https://cloud.google.com/storage/docs/encryption/

https://cloud.google.com/storage/docs/encryption/default-keys

https://cloud.google.com/storage/docs/encryption/customer-managed-keys

https://cloud.google.com/storage/docs/encryption/client-side-keys

https://cloud.google.com/storage/docs/encryption/using-customer-supplied-keys

https://cloud.google.com/kms/docs/creating-keys