Android Application Integrate to Amazon S3 Tutorial

This tutorial explains how to integrate Amazon S3 to your Android Application to upload files, download files or fetch the list of files from your S3 Bucket

7

This tutorial explains how to integrate Amazon S3 to your Android Application to upload files, download files or fetch the list of files from your S3 Bucket.

Introduction

Amazon S3, Amazon Simple Storage Service provides cloud storage service. Its offered by Amazon Web Services.It is highly scalable, reliable, fast, inexpensive data storage infrastructure. Amazon provides AWS Mobile SDK. Developer can use AWS Mobile SDK in Android App to upload files to S3 Bucket, download files from S3 and also retrieve list of files from S3 Bucket.



In this tutorial, will will learn how to integrate Amazon S3 to Android Application. also we will learn how to upload files from Android Application to S3 bucket, download files from S3 bucket to Mobile device and Also display list of files that stored in Amazon S3 bucket by using AWS MObile SDK. SDK has Transfer Utility class to transfer data in an asynchronous process. First, we need to create an account on Amazon S3 and setup Amazon Cognito Identity to authenticate your app with AWS.

AWS Credentials

To access Amazon service in Android Application, we need to get AWS Credentials using Amazon Cognito Identity as credential provider. It allows to set permission for Amazon service which be accessed by users.

Create an Identity Pool

To set up Amzon Cognito, we need to create an identity pool to store user identity data to your account. We will use one Identity pool per Application. Every identity pool is a configurable IAM role that monitors Amazon Mobile Analytics and allows developer to access AWS services.

    Login to Amazon Cognito and click on the Manage federated identities button
    amazons3-min
    To create a new identity,

  • Enter the identity name in the Identity pool name input box
  • Select the check box to enable access to unauthenticated identities.
  • Click on the Create Pool button to create identity pool.
    createpool-min
  • Your Cognito identities require access to your resources, hence click on the allow button
    allow-min

Configure IAM role

We need to configure bucket to store data and set access permission of that bucket to developer by configuring its IAM role.

  • We need a bucket, so create a new bucket
  • Go to Identity and Access Management Console and click on Roles in the left-hand pane.
    welcome_identity_access_management-min
  • Click on the Identity pool name for unauthenticated users.
    cognito-min
  • Create Inline Policy, click on the Create Role Policy button.
    create_role_policy-min
  • To set the permission, click on Select button.
    setpermision-min
  • Select Amazon S3 from drop down list, select All Actions(*) for Actions and enter arn:aws:s3:::bucket_name/* as Amazon Resource Name (ARN).
    editpermision-min
  • Click on the Add Statement button to add the permission then click on Next Step button.
    editpermision_next_step-min
  • Click on Apply Policy button to apply policy.
    reviewpolicy-min

 

Create a new project

We are going to create a new android project. Go to File ⇒ New ⇒ New Projects in Android studio.
 

Add AWS Mobile SDK

To integrate Amazon S3 to Android Application, we need to add AWS Mobile SDK in the dependencies.

......
    compile 'com.amazonaws:aws-android-sdk-core:2.2.13'
    compile 'com.amazonaws:aws-android-sdk-cognito:2.2.13'
    compile 'com.amazonaws:aws-android-sdk-s3:2.2.13'
    compile 'com.amazonaws:aws-android-sdk-ddb:2.2.13'
......

 

Set Permission in Manifest

To access Amazon services, we need to add permission and service in the application. AndroidManifest.xml file look like this:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="in.studytitorial.amazons3tutorial">
    <uses-permission android:name="android.permission.INTERNET"/>
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">

        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <service android:name="com.amazonaws.mobileconnectors.
            s3.transferutility.TransferService" android:enabled="true"/>
    </application>

</manifest>

 

Create Layout

We will upload file to S3 bucket and download file from S3 bucket and also fetch all file list that stored to S3 bucket. All these functionalities perform on click button, so we need to add three button the activity_main.xml file.

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:paddingBottom="@dimen/activity_vertical_margin"
    android:paddingLeft="@dimen/activity_horizontal_margin"
    android:paddingRight="@dimen/activity_horizontal_margin"
    android:paddingTop="@dimen/activity_vertical_margin"
    tools:context="in.studytitorial.amazons3tutorial.MainActivity">

    <Button
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:id="@+id/uploadSd3"
        android:text="Upload S3 File"
        android:onClick="uploadFileToS3"/>

    <Button
        android:layout_marginTop="@dimen/activity_horizontal_margin"
        android:layout_below="@+id/uploadSd3"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:id="@+id/downloadSd3"
        android:text="Download S3 File"
        android:onClick="downloadFileFromS3"/>

    <Button
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_below="@+id/downloadSd3"
        android:id="@+id/fetchFileToS3"
        android:text="Fetch File From S3"
        android:onClick="fetchFileFromS3"
        android:layout_marginTop="@dimen/activity_horizontal_margin"/>

</RelativeLayout>

amazon-min

 

Initialize Credential Provider

To make request to use Amazon Service, We will need to initialize AWS credentials in the Android Application by using CognitoCachingCredentialsProvider object. We need to pass your COGNITO_IDENTITY_POOL Id in this object.

public void s3credentialsProvider(){

        // Initialize the AWS Credential
        CognitoCachingCredentialsProvider cognitoCachingCredentialsProvider = new CognitoCachingCredentialsProvider(
                getApplicationContext(),
                "us-east-1:521c4556-b969-456d-93ec-a3fbc8bfe37f", // Identity Pool ID
                Regions.US_EAST_1 // Region
        );
        setAmazonS3Client(cognitoCachingCredentialsProvider);
    }

 

Create Amazon S3 Client

We will create a new method createAmazonS3Client() and get AWS Credential provider as a parameter. With in this method, we will create AmazonS3Client() constructor and pass the AWS Credential Provider and set the region of your S3 bucket.

/**
 * Create a AmazonS3Client constructor and pass the credentialsProvider.
 * @param credentialsProvider
 */
public void createAmazonS3Client(CognitoCachingCredentialsProvider credentialsProvider){

    // Create an S3 client
    s3Client = new AmazonS3Client(credentialsProvider);

   // Set the region of your S3 bucket
   s3Client.setRegion(Region.getRegion(Regions.US_EAST_1));
}

 

Call TransferUtility Class

To upload files from Android device to S3 Bucket and download files from S3 bucket to Android device, we need to use TransferUtility class. Pass the Amazon S3 client and the application context to the Transfer Utility.

public void setTransferUtility(){

   transferUtility = new TransferUtility(s3Client, 
           getApplicationContext());
}

 

Upload File to S3

To upload the file to S3 by using TransferUtility class. We will pass the MY_BUCKET, OBJECT_KEY and MY_FILE to upload() method of TransferUtility class and return it to the TransferObserver object.

  • MY_BUCKET: is the name of S3 bucket, where we want to upload file
  • Key: is the File name
  • MY_FILE: file path
public class MainActivity extends AppCompatActivity {

    AmazonS3 s3Client;
    String bucket = "studtytutorial";
    File uploadToS3 = new File("/storage/sdcard0/Pictures/Screenshots/Screenshot.png");
    TransferUtility transferUtility;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        // callback method to call credentialsProvider method.
        s3credentialsProvider();

        // callback method to call the setTransferUtility method
        setTransferUtility();

    }

    public void s3credentialsProvider(){

        // Initialize the AWS Credential
        CognitoCachingCredentialsProvider cognitoCachingCredentialsProvider = 
            new CognitoCachingCredentialsProvider(
                getApplicationContext(),
                "us-east-1:521c4556-b969-456d-93ec-a3fbc8bfe37f", // Identity Pool ID
                Regions.US_EAST_1 // Region
        );
        createAmazonS3Client(cognitoCachingCredentialsProvider);
    }

    /**
     *  Create a AmazonS3Client constructor and pass the credentialsProvider.
     * @param credentialsProvider
     */
    public void createAmazonS3Client(CognitoCachingCredentialsProvider 
        credentialsProvider){

        // Create an S3 client
        s3Client = new AmazonS3Client(credentialsProvider);

        // Set the region of your S3 bucket
        s3Client.setRegion(Region.getRegion(Regions.US_EAST_1));
    }

    public void setTransferUtility(){

        transferUtility = new TransferUtility(s3Client, getApplicationContext());
    }

    /**
     * This method is used to upload the file to S3 by using TransferUtility class
     * @param view
     */
    public void uploadFileToS3(View view){

        TransferObserver transferObserver = transferUtility.upload(
                bucket,          /* The bucket to upload to */
                "Screenshot.png",/* The key for the uploaded object */
                uploadToS3       /* The file where the data to upload exists */
        );

        transferObserverListener(transferObserver);
    }

 

TransferObserver Listener

To know the status of uploading and downloading the file, we need to set transfer listeners. Amazon SDK provides TransferObserver class which have various listener methods to track the status of the upload or download. So, implement and override onStateChanged(), onProgressChanged(), onError() methods of TransferObserver class.

 /* This is listener method of the TransferObserver
  * Within this listener method, we get status of uploading and downloading file,
  * to display percentage of the part of file to be uploaded or downloaded to S3
  * It displays an error, when there is a problem in  
  * uploading or downloading file to or from S3.
  * @param transferObserver
  */

 public void transferObserverListener(TransferObserver transferObserver){

     transferObserver.setTransferListener(new TransferListener(){

       @Override
       public void onStateChanged(int id, TransferState state) {
          Toast.makeText(getApplicationContext(), "State Change" + state,
                 Toast.LENGTH_SHORT).show();
       }

       @Override
       public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
           int percentage = (int) (bytesCurrent/bytesTotal * 100);
           Toast.makeText(getApplicationContext(), "Progress in %" + percentage, 
              Toast.LENGTH_SHORT).show();
       }

       @Override
       public void onError(int id, Exception ex) {
           Log.e("error","error");
       }

      });
}

 


Amazon_upload_file_progress-minamazon_upload_0_percentage-minamazon_progress_100-minamazon_upload_completed-min

 

Download Files

To download file from S3, we need to call the download method of the TransferUtility class and return TransferObserver object which will track the status of the download. Pass the MY_BUCKET, OBJECT_KEY and MY_FILE to the download() method.

/**
 *  This method is used to Download the file to S3 by using transferUtility class
 * @param view
 **/
 public void downloadFileFromS3(View view){

     TransferObserver transferObserver = transferUtility.download(
             bucket,     /* The bucket to download from */
             "Screenshot.png",    /* The key for the object to download */
             downloadFromS3        /* The file to download the object to */
     );
     transferObserverListener(transferObserver);
}

 


Amazon_upload_file_progress-minamazon_upload_0_percentage-minamazon_progress_100-minamazon_upload_completed-min

Get All file file List

To get list of file name that stored to the S3 bucket, we will use listObjects object of S3Client. We will call it in a asynchronously. All file file list stored in a String object.

public void fetchFileFromS3(View view){

   // Get List of files from S3 Bucket
   Thread thread = new Thread(new Runnable(){
   @Override
   public void run() {

      try {
         Looper.prepare();
         listing = getObjectNamesForBucket(bucket, s3Client);

         for (int i=0; i< listing.size(); i++){
                 Toast.makeText(MainActivity.this, 
                     listing.get(i),Toast.LENGTH_LONG).show();
         }
         Looper.loop();
         }
      catch (Exception e) {
          e.printStackTrace();
          Log.e("tag", "Exception found while listing "+ e);
      }

   }
 });
  thread.start();
}

    /**
     * @desc This method is used to return list of files name from S3 Bucket
     * @param bucket
     * @param s3Client
     * @return object with list of files
     */
    private List<String> getObjectNamesForBucket(String bucket, AmazonS3 s3Client) {
        ObjectListing objects=s3Client.listObjects(bucket);
        List<String> objectNames=new ArrayList<String>(objects.getObjectSummaries().size());
        Iterator<S3ObjectSummary> iterator=objects.getObjectSummaries().iterator();
        while (iterator.hasNext()) {
            objectNames.add(iterator.next().getKey());
        }
        while (objects.isTruncated()) {
            objects=s3Client.listNextBatchOfObjects(objects);
            iterator=objects.getObjectSummaries().iterator();
            while (iterator.hasNext()) {
                objectNames.add(iterator.next().getKey());
            }
        }
        return objectNames;
    }

 


amazon_fetch_file_list1amazon_fetch_file_list1-min

 

Final Code


public class MainActivity extends AppCompatActivity {

    AmazonS3 s3Client;
    String bucket = "studtytutorial";
    File uploadToS3 = new File("/storage/sdcard0/Pictures/Screenshots/Screenshot.png");
    File downloadFromS3 = new File("/storage/sdcard0/Pictures/Screenshot.png");
    TransferUtility transferUtility;
    List<String> listing;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        // callback method to call credentialsProvider method.
        s3credentialsProvider();

        // callback method to call the setTransferUtility method
        setTransferUtility();

    }

    public void s3credentialsProvider(){

        // Initialize the AWS Credential
        CognitoCachingCredentialsProvider cognitoCachingCredentialsProvider = 
                new CognitoCachingCredentialsProvider(
                getApplicationContext(),
                "us-east-1:521c4556-b969-456d-93ec-a3fbc8bfe37f", // Identity Pool ID
                Regions.US_EAST_1 // Region
        );
        createAmazonS3Client(cognitoCachingCredentialsProvider);
    }

    /**
     *  Create a AmazonS3Client constructor and pass the credentialsProvider.
     * @param credentialsProvider
     */
    public void createAmazonS3Client(CognitoCachingCredentialsProvider 
                                             credentialsProvider){

        // Create an S3 client
        s3Client = new AmazonS3Client(credentialsProvider);

        // Set the region of your S3 bucket
        s3Client.setRegion(Region.getRegion(Regions.US_EAST_1));
    }

    public void setTransferUtility(){

        transferUtility = new TransferUtility(s3Client, getApplicationContext());
    }

    /**
     * This method is used to upload the file to S3 by using TransferUtility class
     * @param view
     */
    public void uploadFileToS3(View view){

        TransferObserver transferObserver = transferUtility.upload(
                bucket,     /* The bucket to upload to */
                "Screenshot.png",    /* The key for the uploaded object */
                uploadToS3       /* The file where the data to upload exists */
        );

        transferObserverListener(transferObserver);
    }

    /**
     *  This method is used to Download the file to S3 by using transferUtility class
     * @param view
     **/
    public void downloadFileFromS3(View view){

        TransferObserver transferObserver = transferUtility.download(
                bucket,     /* The bucket to download from */
                "Screenshot.png",    /* The key for the object to download */
                downloadFromS3        /* The file to download the object to */
        );
        transferObserverListener(transferObserver);
    }

    public void fetchFileFromS3(View view){

        // Get List of files from S3 Bucket
        Thread thread = new Thread(new Runnable(){
            @Override
            public void run() {

                try {
                    Looper.prepare();
                    listing = getObjectNamesForBucket(bucket, s3Client);

                    for (int i=0; i< listing.size(); i++){
                        Toast.makeText(MainActivity.this, listing.get(i),Toast.LENGTH_LONG).show();
                    }
                    Looper.loop();
                    // Log.e("tag", "listing "+ listing);
                }
                catch (Exception e) {
                    e.printStackTrace();
                    Log.e("tag", "Exception found while listing "+ e);
                }

            }
        });
        thread.start();
    }

    /**
     * @desc This method is used to return list of files name from S3 Bucket
     * @param bucket
     * @param s3Client
     * @return object with list of files
     */
    private List<String> getObjectNamesForBucket(String bucket, AmazonS3 s3Client) {
        ObjectListing objects=s3Client.listObjects(bucket);
        List<String> objectNames=new ArrayList<String>(objects.getObjectSummaries().size());
        Iterator<S3ObjectSummary> iterator=objects.getObjectSummaries().iterator();
        while (iterator.hasNext()) {
            objectNames.add(iterator.next().getKey());
        }
        while (objects.isTruncated()) {
            objects=s3Client.listNextBatchOfObjects(objects);
            iterator=objects.getObjectSummaries().iterator();
            while (iterator.hasNext()) {
                objectNames.add(iterator.next().getKey());
            }
        }
        return objectNames;
    }

    /**
     * This is listener method of the TransferObserver
     * Within this listener method, we get status of uploading and downloading file,
     * to display percentage of the part of file to be uploaded or downloaded to S3
     * It displays an error, when there is a problem in  uploading or downloading file to or from S3.
     * @param transferObserver
     */

    public void transferObserverListener(TransferObserver transferObserver){

        transferObserver.setTransferListener(new TransferListener(){

            @Override
            public void onStateChanged(int id, TransferState state) {
                Toast.makeText(getApplicationContext(), "State Change" 
                        + state, Toast.LENGTH_SHORT).show();
            }

            @Override
            public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
                int percentage = (int) (bytesCurrent/bytesTotal * 100);
                Toast.makeText(getApplicationContext(), "Progress in %" 
                        + percentage, Toast.LENGTH_SHORT).show();
            }

            @Override
            public void onError(int id, Exception ex) {
                Log.e("error","error");
            }

        });
    }
}






  • sachin jaiswal

    I am able to upload to my bucket, for keeping in a folder inside a bucket, do I just have a specify a path and it will create a structure or first I have to create a folder structure in bucket then give path to that folder while uploading in the bucket?

    • You can create a folder in the bucket. You need to set the permission as a public

      • sachin jaiswal

        can i create folder in bucket from android app?

        because my requirement is : some set of images need to be pushed into kept in one folder and another in different folder.

        • sachin jaiswal

          I am able to create folder at run time as well, I made code changes and it is working…
          TransferObserver transferObserver = transferUtility.upload(
          bucket, /* The bucket to upload to */
          “image/”+uploadToS3.getName(), /* The key for the uploaded object */
          uploadToS3 /* The file where the data to upload exists */
          );
          image/”+uploadToS3.getName(),
          this allows me to create folder image and put image inside it

          • We can create runtime but some time folder permission not set as a public then it can not upload file

  • Abhey

    I was trying to make the same project as you have uploaded to store,download and fetch files from amazon s3. I want to download the files after fetching the whole list of files uploaded and then selecting the file that needs to be downloaded. Can you please suggest me a way to select the file from the whole list so that i can pass it directly as the key in download method.”

    • I have already explained, how to fetch all files from Bucket, you can display object as list. I have explained how to download file. You need to implement listview or recyclerview to display file name as list and call download method on click item.