How to Download a Folder from AWS S3 Bucket

avatar
April 3, 2024

Whether you’re new to Amazon S3 or an experienced user, downloading a whole folder or bucket from AWS S3 can seem like a challenge sometimes. In this article, we’re going to explain what folders and buckets are, and share a convenient way of downloading content from them so you can streamline your workflow. 

What is the Difference Between Bucket and Folder in AWS S3?

Amazon Simple Storage Service, or Amazon S3, isn’t like most file systems. It doesn’t have the common “directory -> sub-directory -> files” hierarchical structure. Instead, it uses “buckets” and “objects” and organizes them in a flat structure. It does, however, allow you to group objects or files by giving them a common prefix to make it easier for you to understand the system. 

Objects are stored inside a “bucket.” There is no limit to how many objects you can upload in it and you can have up to a total of 100 buckets in a single account. 

The main difference between a folder and a bucket is that the former doesn’t really exist in the AWS S3 system. All files in a bucket can be grouped by entering slashes (“/”) in their names. However, this only gives the appearance of a hierarchy so you can easily organize objects by using folders.

Buckets, on the other hand, are what contain objects or files.

2 Methods to Manage S3 Buckets

There are two ways to download or upload files to an S3 bucket:

  1. AWS Command-Line Interface (CLI) – The official way to upload files to a bucket is to use either the Amazon S3 console or the AWS CLI. The former has a size limit of 160 GB, and the latter can be a bit complex, especially if you’re not familiar with command-line interfaces. The AWS CLI also lets you download a folder, but again, many users will find it complicated.

    You can learn more about the AWS CLI on Amazon’s official page. If you enjoy a good challenge, then feel free to watch this tutorial of how to upload files via AWS CLI or read this helpful Stack Overflow guide.

  2. A Third-Party Software – An easier way to download an entire folder from S3 is to use a third-party app called Commander One. It’ll help you connect to Amazon S3 as well as cloud storage services and remote servers like FTP and SFTP.

Best Way to Download Amazon S3 Bucket on macOS

Commander One is much more than just a file manager – it’s an S3 browser for Mac that makes it extremely easy to upload and download all files from an S3 folder or buckets in your AWS account. Its user-friendly interface eliminates the need to learn and remember commands. Moreover, it allows you to access specific S3 buckets, and easily manage all your online files, folders, and archives as if they’re stored on your local drive. 

The dual-pane interface of Commander One is demostrated

Not only can you download a folder from an S3 bucket, but there are plenty of other things that are possible with Commander One. Here are some of the ways you can interact with folders in Commander One:

That’s not all. Here are a few more things you can do in Commander One:

  • Upload files of large sizes to S3.
  • Upload files in batches.
  • Work with archive files using  Commander One. Supported formats include .zip, .rar, .apk, .war, and more.
  • Browse and interact with files in an S3 bucket without downloading them to your computer.

How to Download a Folder From S3 using Commander One

Commander One is the easiest way to download a folder from S3. Don’t believe us? Here’s a step-by-step guide that shows you just how simple the process is:

1

First, go to the Amazon site and log in with your details.

2

Download Commander One on your Mac.

3

Navigate to the “Connections” manager in the app.

The Connections manager of the Commander One is highlighted
4

Click on Amazon S3.

Amazon S3 is highlighted in the Connections Window of the app
5

Enter your credentials, I.e., your Access Key and Secret Key. Both of these were assigned to you when you registered on AWS.

Click on Connect to add your AWS S3 account as a local drive.

Amazon S3 credentials are added and the Connect button is highlighted

Now you’re ready to interact with the contents of S3 bucket. You can download and manage them using options like hotkeys, drag-and-drop, copying and pasting, and more.

When you download files from Amazon S3 bucket, a small window will pop up showing you the ongoing progress.

How to Upload Files to AWS S3 from FTP/SFTP Server

Commander One is a versatile app. Apart from downloading files and data from an AWS S3 bucket, you can also use it to transfer a file from S3 to FTP server. Once you’ve mounted the FTP server and your S3 account as local drives, you can move files between them as you would on any local drive.

Frequently Asked Questions

When using Commander One, 5 TB is the maximum size of a file you can upload to an Amazon S3 bucket. Keep in mind, though, that the S3 console has a limit of 160 GB. To bypass this limit, you can use the Amazon CLI, but it’s not the most user-friendly method. That’s why we recommend using Commander One as it’ll easily let you upload and download an entire S3 bucket.

Yes, Commander One allows you to transfer files between different Amazon S3 accounts and buckets. In fact, you can also move your files from S3 to cloud storage services like Google Drive and OneDrive, and remote servers like FTP and SFTP.

Cookie
Electronic Team uses cookies to personalize your experience on our website. By continuing to use this site, you agree to our cookie policy. Click here to learn more.