How to download an object from Amazon S3 using AWS CLI

gcptutorials.com AWS

This post explains how to download objects from AWS S3 bucket using aws s3api get-object command.

aws s3api get-object

aws s3api get-object retrieves objects from Amazon S3 Bucket. To use this command you must have read access to the object.

Topics

Download complete object from S3 Bucket
Download specific byte range of object
Download specific version of object
Download object if modified since a given time
Download object if Etag is different
Download object if Etag is same

Prerequisite

To run the below commands you need to have AWS CLI installed and configured, if AWS CLI is not installed or not configured follow below posts to complete the setup.

[On Windows]

1. How to install AWS CLI on windows
2. How to configure AWS CLI

[On Ubuntu]

1. How to install AWS CLI on Ubuntu
2. How to configure AWS CLI

Command to download an object from S3 using AWS CLI.

           
  
  aws s3api get-object --bucket "BUCKET_NAME" --key "OBJECT_KEY" "NAME_OF_OUTPUT_FILE"

Download S3 Object

The following example downloads an object with name sample_object1.txt from folder dir in S3 bucket test-bucket-001 and saves the output to the local file sample_object1.txt.

          
      
  aws s3api get-object --bucket test-bucket-001 --key dir/sample_object1.txt sample_object1.txt

Download specific byte range from a S3 Object

Following example command downloads first 500 bytes of an object with name sample_object1.txt from folder dir in S3 bucket test-bucket-001 and saves the output to the local file sample_bytes.txt.

          
      
  aws s3api get-object --bucket test-bucket-001 --key dir/sample_object1.txt --range bytes=1-500 sample_bytes.txt

Download specific version of S3 Object

By default, the GET action returns the current version of an object. To return a different version, use the below command. This command saves specific version of object sample_object.txt to the local file sample_object.txt . Replace abcdef12345 with the object version Id.

   
  
  aws s3api get-object --bucket test-bucket-001 --key dir/sample_object.txt --version-id abcdef12345 sample_object.txt

Download S3 object only if modified since given timestamp

Use the command below to download an object if it has been modified since a certain timestamp. This command generates a 304 (not modified) error if object is not modified since given timestamp. Replace 2022-10-04T04:42:00.00z with the required timestamp.

Run 1 : Object not modified since given timestamp

      
  
  aws s3api get-object --bucket test-bucket-001 --if-modified-since 2022-10-04T04:42:00.00z --key dir/sample_object.txt sample_object.txt

Output

  
  
  An error occurred (304) when calling the GetObject operation: Not Modified

Run 2: Object modified since given timestamp

  
  
  aws s3api get-object --bucket test-bucket-001 --if-modified-since 2022-10-04T04:42:00.00z --key dir/sample_object.txt sample_object.txt

Output

   
  {
      "AcceptRanges": "bytes",
      "LastModified": "2022-10-04T04:45:41+00:00",
      "ContentLength": 18,
      "ETag": "\"111111111111111\"",
      "VersionId": "111111111111111",
      "ContentType": "text/plain",
      "Metadata": {}
  }

Download S3 object only if Etag is different

To download the object only if object ETag is different from the specified in the command. Etag is a hash of the object. The ETag reflects changes only to the contents of an object, not its metadata. Below command return a 304 (not modified) error if the provided ETag and Object ETag are same, else it downloads the object. Replace 111111111111111 with the Etag.

Run 1 : Etags are same

  
  
  aws s3api get-object --bucket test-bucket-001 --if-none-match 111111111111111 --key dir/sample_object.txt sample_object.txt

Output

   
  An error occurred (304) when calling the GetObject operation: Not Modified

Run 2: Etags are different (Object content updated)

    
  
  aws s3api get-object --bucket test-bucket-001 --if-none-match 111111111111111 --key dir/sample_object.txt sample_object.txt

Output

   
  {
      "AcceptRanges": "bytes",
      "LastModified": "2022-10-04T04:45:41+00:00",
      "ContentLength": 18,
      "ETag": "\"111111111111111\"",
      "VersionId": "111111111111111",
      "ContentType": "text/plain",
      "Metadata": {}
  }

Download S3 object only if Etag is same

Use below command to download the object only if object ETag is same as specified in the command. Below command return a a 412 (precondition failed) if the provided ETag and Object ETag are different, else it downloads the object. Replace 111111111111111 with the Etag.

Run 1 : Etags are different

   
  
  aws s3api get-object --bucket test-bucket-001 --if-match 111111111111111 --key dir/sample_object.txt sample_object.txt

Output

   
  An error occurred (PreconditionFailed) when calling the GetObject operation: At least one of the pre-conditions you specified did not hold

Run 2: Etags are same

       
  aws s3api get-object --bucket test-bucket-001 --if-match 111111111111111 --key dir/sample_object.txt sample_object.txt

Output

   
  {
      "AcceptRanges": "bytes",
      "LastModified": "2022-10-04T04:45:41+00:00",
      "ContentLength": 18,
      "ETag": "\"111111111111111\"",
      "VersionId": "111111111111111",
      "ContentType": "text/plain",
      "Metadata": {}
  }

Category: AWS