How To Create A Composite Object In Cloud Storage
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed Object Transcoding in Cloud Storage.
https://cloudaffaire.com/what-is-object-transcoding-in-cloud-storage/
In this blog post, we will discuss the Composite Object in Cloud Storage.
Composite Object:
Composite objects are useful for making appends to an object, as well as for building an object using components that you’ve uploaded in parallel. The compose operation creates a new composite object whose contents are the concatenation of a given sequence of source objects. The source objects all must:
- Have the same storage class.
- Be stored in the same Cloud Storage bucket.
- NOT use customer-managed encryption keys.
Object composition can be used for uploading an object in parallel: divide your data into multiple chunks, upload each chunk to a distinct object in parallel, compose your final object, and delete any temporary source objects.
How To Create A Composite Object In Cloud Storage:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
######################################## ## Composite Objects In Cloud Storage ## ######################################## ## Prerequisite: gsutil/gcloud installed and configured ## https://cloudaffaire.com/gcloud-installation-and-configuration/ ## Create a bucket in cloud storage (bucket name needs to be globally unique) gsutil mb -c standard -l asia-south1 -p cloudaffaire gs://cloudaffaire_bucket_co ## Create two files that will act as source object. echo "hello from source object one" > cloudaffaire_source_object_one.txt && echo "hello from source object two" > cloudaffaire_source_object_two.txt ## Upload both the object in your bucket gsutil cp cloudaffaire_source_object_one.txt gs://cloudaffaire_bucket_co && gsutil cp cloudaffaire_source_object_two.txt gs://cloudaffaire_bucket_co ## Create a composite object out of the two source objects gsutil compose gs://cloudaffaire_bucket_co/cloudaffaire_source_object_one.txt \ gs://cloudaffaire_bucket_co/cloudaffaire_source_object_two.txt \ gs://cloudaffaire_bucket_co/cloudaffaire_composite_object.txt ## Check the content of the new composite object (includes content of both source objects) gsutil cat gs://cloudaffaire_bucket_co/cloudaffaire_composite_object.txt ## hello from source object one ## hello from source object two ## View the metadata associated with a normal object gsutil stat gs://cloudaffaire_bucket_co/cloudaffaire_source_object_two.txt ## View the metadata associated with a composite object gsutil stat gs://cloudaffaire_bucket_co/cloudaffaire_composite_object.txt ## Component-Count: 2 ## Hash (crc32c): PRP77Q== ## Delete all the objects gsutil rm gs://cloudaffaire_bucket_co/cloudaffaire_source_object_one.txt && gsutil rm gs://cloudaffaire_bucket_co/cloudaffaire_source_object_two.txt && gsutil rm gs://cloudaffaire_bucket_co/cloudaffaire_source_object_one.txt ## Delete the bucket gsutil rb gs://cloudaffaire_bucket_co |
Hope you have enjoyed this article. In the next blog post, we will discuss Access Logs in cloud storage.
All the public cloud providers are changing the console user interface rapidly and due to this some of the screenshots used in our previous AWS blogs are no longer relevant. Hence, we have decided that from now onwards most of the demo will be done programmatically. Let us know your feedback on this in the comment section.
To get more details on cloud storage, please refer below GCP documentation.
https://cloud.google.com/storage/docs/
https://cloud.google.com/storage/docs/composite-objects