How To Create A Bucket and Object In Cloud Storage Using gsutil
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed key concepts of cloud storage.
https://cloudaffaire.com/cloud-storage-in-gcp/
In this blog post, we will discuss how to create a bucket and object in cloud storage using gsutil.
How To Create A Bucket and Object In Cloud Storage Using gsutil:
Step 1: Install and configure gsutil.
|
1 2 3 4 5 6 7 8 9 10 |
################################################ ## Create a cloud storage bucket using gsutil ## ################################################ ## Install and configure gsutil ## gsutil comes as part of cloud sdk and by default installed during gcloud installation ## You can follow below link to install and configure gcloud ## https://cloudaffaire.com/gcloud-installation-and-configuration/ ## Check gsutil version gsutil version |
Step 2: Create a bucket using gsutil mb command.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
## Create a bucket (name needs to be globally unique) gsutil mb -c standard -l asia-south1 -p cloudaffaire gs://cloudaffaire-bucket #create a bucket with storage class nearline => gsutil mb -c nearline gs://cloudaffaire-bucket #create a bucket with retention period 36 months => gsutil mb --retention 36m gs://cloudaffaire-bucket ## gsutil mb [-b ## -b = Specifies the uniform bucket-level access setting. Default is "off" ## -c = Specifies the default storage class. Default is "Standard". ## -l = Can be any supported location. Deafult is US. ## -p = Specifies the project ID under which to create the bucket. ## -retention = Specifies the retention policy. Default is no retention policy. ## url = gs:// |
Step 3: Create an object using gsutil cp command.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
## Create some files for upload echo "this is my 1st object" > cloudaffaire-object-one.txt ## Upload an object into the bucket gsutil cp cloudaffaire-object-one.txt gs://cloudaffaire-bucket #upload the file inside a directory => gsutil cp cloudaffaire-object-one.txt gs://cloudaffaire-bucket/directory #upload all files of a directory => gsutil cp -r directory gs://cloudaffaire-bucket ## gsutil cp [OPTION]... src_url... dst_url ## -a = Sets named canned_acl when uploaded objects created. ## -A = Copy all source versions from a source buckets/folders. ## -c = If an error occurs, continue to attempt to copy the remaining files. ## -D = Copy in "daisy chain" mode, copying between two buckets by hooking a download to an upload, via the machine where gsutil is run. ## -e = Exclude symlinks. When specified, symbolic links will not be copied. ## -I = Causes gsutil to read the list of files or objects to copy from stdin. ## -j = Applies gzip transport encoding to any file upload whose extension matches the -j extension list. ## -J = Applies gzip transport encoding to file uploads. ## -L = Outputs a manifest log file with detailed information about each item that was copied. ## -n = No-clobber. When specified, existing files or objects at the destination will not be overwritten. ## -p = Causes ACLs to be preserved when copying in the cloud. ## -P = Causes POSIX attributes to be preserved when objects are copied. ## -R, -r = Causes directories, buckets, and bucket subdirectories to be copied recursively. ## -s = The storage class of the destination object(s). ## -U = Skip objects with unsupported object types instead of failing. ## -v = Requests that the version-specific URL for each uploaded object be printed. ## -z = Applies gzip content-encoding to any file upload whose extension matches the -z extension list. ## -Z = Applies gzip content-encoding to file uploads. ## src_url = source url ## dst_url = your bucket url |
Step 4: List your bucket/objects using gsutil ls command.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
## List providers, buckets, or objects gsutil ls #list bucket details => gsutil ls -L #list bucket deatils recursively with size => gsutil ls -lrh gs://cloudaffaire-bucket ## gsutil ls [-a] [-b] [-d] [-l] [-L] [-r] [-p proj_id] url... ## -l = Prints long listing (owner, length). ## -L = Prints even more detail than -l. ## -d = List matching subdirectory names instead of contents, and do not recurse into matching subdirectories even if the -R option is specified. ## -b = Prints info about the bucket when used with a bucket URL. ## -h = When used with -l, prints object sizes in human readable format (e.g., 1 KiB, 234 MiB, 2 GiB, etc.) ## -p = Specifies the project ID to use for listing buckets. ## -R, -r = Requests a recursive listing, performing at least one listing operation per subdirectory. ## -a = Includes non-current object versions / generations in the listing (only useful with a versioning-enabled bucket). ## -e = Include ETag in long listing (-l) output. |
Step 5: Rename your object using gsutil mv command.
|
1 2 3 4 5 6 |
## Move/rename objects gsutil mv gs://cloudaffaire-bucket/cloudaffaire-object-one.txt gs://cloudaffaire-bucket/cloudaffaire-object-new.txt ## gsutil mv [-p] src_url dst_url ## All options that are available for the gsutil cp command are also available for the gsutil mv command ## (except for the -R flag, which is implied by the gsutil mv command). |
Step 6: Display the content of your object using gsutil cat command.
|
1 2 3 4 5 6 |
## Concatenate object content to stdout gsutil cat gs://cloudaffaire-bucket/cloudaffaire-object-new.txt ## gsutil cat [-h] [-r] url... ## -h = Prints short header for each object. ## -r = Causes gsutil to output just the specified byte range of the object. |
Step 7: Delete your object using gsutil rm command.
|
1 2 3 4 5 6 7 8 |
## Remove objects gsutil rm gs://cloudaffaire-bucket/cloudaffaire-object-new.txt ## gsutil rm [-f] [-r] url... ## -f = Continues silently (without printing error messages) despite errors when removing multiple objects. ## -I = Causes gsutil to read the list of objects to remove from stdin. ## -R, -r = Causes bucket or bucket subdirectory contents (all objects and subdirectories that it contains) to be removed recursively. ## -a = Delete all versions of an object. |
Step 8: Delete your bucket using gsutil rb command.
|
1 2 3 4 5 |
## Remove buckets gsutil rb gs://cloudaffaire-bucket ## gsutil rb [-f] url... ## -f = Continues silently (without printing error messages) despite errors when removing buckets. |
Hope you have enjoyed this article. In the next blog post, we will discuss versioning in cloud storage.
To get more details on cloud storage, please refer below GCP documentation.
https://cloud.google.com/storage/docs/
https://cloud.google.com/storage/docs/gsutil