How To Enable Object Versioning in Cloud Storage
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed how to create a bucket and object in cloud storage using gsutil.
https://cloudaffaire.com/how-to-create-a-bucket-and-object-in-cloud-storage-using-gsutil/
In this blog post, we will discuss how to enable object versioning in cloud storage.
Object Versioning in Cloud Storage:
You can enable Object Versioning to protect your Cloud Storage data from being overwritten or accidentally deleted. Enabling Object Versioning increases storage costs, which can be partially mitigated by configuring Object Lifecycle Management to delete older object versions.
When you enable object versioning, Cloud Storage creates a noncurrent version of an object each time you perform an overwrite or delete of the live version, as long as you do not specify the generation number of the live version. Noncurrent versions have below properties:
- Noncurrent versions retain the name of the object, but are uniquely identified by their generation number.
- Noncurrent versions only appear in requests that explicitly call for object versions to be included.
- Noncurrent versions of objects exist independently of any live version.
Note: You permanently delete versions of objects by including the generation number in the deletion request or by using Object Lifecycle Management.
Cloud Storage uses two properties that together identify the version of an object. One property identifies the version of the object’s data; the other property identifies the version of the object’s metadata. These properties are always present with every version of the object, even if Object Versioning is not enabled.
- generation: Identifies the content (data) generation, and updates when the content of an object is overwritten. There is no relationship between the generation numbers of unrelated objects, even if the objects are in the same bucket.
- metageneration: Identifies the metadata generation, and increases every time the metadata for a given content generation is updated. metageneration is reset to 1 for each new generation of an object. The metageneration property has no meaning without the generation property and should be used only in conjunction with it. In other words, it is meaningless to compare metadata generations of two versions that have different data generations.
How To Enable Object Versioning in Cloud Storage:
Step 1: Create a bucket and upload an object in your bucket.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
######################################## ## Object Versioning In Cloud Storage ## ######################################## ## Create a bucket in cloud storage (bucket name needs to be globally unique) gsutil mb -c standard -l asia-south1 -p cloudaffaire gs://cloudaffaire_bucket ## Upload an object into your bucket ## Create a file for upload echo "this is my object version one" > cloudaffaire_object.txt ## Upload an object into the bucket gsutil cp cloudaffaire_object.txt gs://cloudaffaire_bucket ## View generation and metageneration for your object gsutil stat gs://cloudaffaire_bucket/cloudaffaire_object.txt ## gsutil stat url... ## Note the generation and metageneration for the object ## Generation: 1577366178757673 ## Metageneration: 1 |
Note: Though versioning is not enabled, generation and metageneration number exist for your object. generation and metageneration numbers are created for each object irrespective of versioning is enabled or disabled.
Step 2: Enable object versioning for your bucket.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
## Enable object versioning for your bucket gsutil versioning set on gs://cloudaffaire_bucket ## gsutil versioning set ## gsutil versioning get url... ## View generation and metageneration for your object post versioning gsutil stat gs://cloudaffaire_bucket/cloudaffaire_object.txt ## Note the generation and metageneration for the object does not changed ## Generation: 1577366178757673 ## Metageneration: 1 |
Step 3: Edit your object’s metadata.
|
1 2 3 4 5 6 7 8 9 |
## Edit the metadata associated with an object gsutil setmeta -h "content-type:json" gs://cloudaffaire_bucket/cloudaffaire_object.txt ## View generation and metageneration for your object post versioning gsutil stat gs://cloudaffaire_bucket/cloudaffaire_object.txt ## Note the generation for the object does not changed but metageneration got incremented by 1 ## Generation: 1577366178757673 ## Metageneration: 2 |
Note: Since only metadata got changed, generation id remains the same, only metageneration number got change.
Step 4: Upload a new version of your object.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
## Upload a new version of the object cat /dev/null > cloudaffaire_object.txt && echo "this is my object version two" > cloudaffaire_object.txt ## Upload an object into the bucket gsutil cp cloudaffaire_object.txt gs://cloudaffaire_bucket ## Note the generation and metageneration for the object got changed ## Generation: 1577366880531707 ## Metageneration: 1 ## list both live and noncurrent versions of an object and view their generation numbers gsutil ls -a gs://cloudaffaire_bucket ##noncurrent => gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366178757673 ##live => gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366880531707 ## View generation and metageneration for your live object to confirm gsutil stat gs://cloudaffaire_bucket/cloudaffaire_object.txt |
Note: Due to the new version upload, your object previous version moved to noncurrent version.
Step 5: Delete the current live version of your object and disable versioning in your bucket.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
## Delete the current live object gsutil rm gs://cloudaffaire_bucket/cloudaffaire_object.txt ## list both noncurrent versions of an object and view their generation numbers gsutil ls -a gs://cloudaffaire_bucket gsutil ls gs://cloudaffaire_bucket #list only current version (no current version exist) ##noncurrent => gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366178757673 ##noncurrent => gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366880531707 ## Disable object versioning for your bucket gsutil versioning set off gs://cloudaffaire_bucket |
Step 6: Restore a noncurrent version of your object.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
## Restore any one of the noncurrent object version using generation number gsutil cp gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366178757673 gs://cloudaffaire_bucket/cloudaffaire_object.txt ## List live object gsutil ls gs://cloudaffaire_bucket ## list both noncurrent versions of an object and view their generation numbers gsutil ls -a gs://cloudaffaire_bucket ##noncurrent => gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366178757673 ##noncurrent => gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366880531707 ##live => gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577367857987078 ## View generation and metageneration for your live object to confirm gsutil stat gs://cloudaffaire_bucket/cloudaffaire_object.txt ## Generation: 1577367857987078 ## Metageneration: 1 |
Step 7: Clean up
|
1 2 3 4 5 6 7 |
## Delete all the objects using generation number gsutil rm gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366178757673 gsutil rm gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577366880531707 gsutil rm gs://cloudaffaire_bucket/cloudaffaire_object.txt#1577367857987078 ## Delete the bucket gsutil rb gs://cloudaffaire_bucket |
Hope you have enjoyed this article. In the next blog post, we will discuss the object lifecycle in cloud storage.
To get more details on cloud storage, please refer below GCP documentation.
https://cloud.google.com/storage/docs/
https://cloud.google.com/storage/docs/gsutil