After my earlier posts on invalidating Amazon CloudFront objects, I thought it would be important to see if an Amazon S3 object existed before trying to invalidate it. With the 1,000 request limit on invalidation requests before Amazon charges you for them, this seemed to be a prudent thing to do. So, I turned to the Amazon Web Services ASP.NET SDK to help me out with it. This is what I came up with:
public bool S3ObjectExists(string bucket, string key)
{
using (AmazonS3Client client = new AmazonS3Client(this._awsAccessKey, this._awsSecretKey))
{
GetObjectRequest request = new GetObjectRequest();
request.BucketName = bucket;
request.Key = key;
try
{
S3Response response = client.GetObject(request);
if (response.ResponseStream != null)
{
return true;
}
}
catch (AmazonS3Exception)
{
return false;
}
catch (WebException)
{
return false;
}
catch (Exception)
{
return false;
}
}
return false;
}
I decided that if I found a valid ResponseStream on the S3Response, then I had a valid object. All I’m checking on is the object key itself, i.e. an image path in S3. Another note here is I’m checking for three different exceptions but returning false for all 3. The reason I have this coded this way for now is I wanted to see what different exceptions GetObject might throw depending on what was wrong with the request. This was done purely for testing purposes and will probably be changed in the future. For instance, I discovered that AmazonS3Exception is thrown when the object isn’t there. WebException is thrown when the object is there, but the request cannot be completed. I’m still in the testing phase with this, but I hope this helps some other Amazon Web Service developers out there.
I always enjoy learning what other people think about Amazon Web Services and how they use them. Check out my very own tool CloudBerry Explorer that helps manage S3 on Windows . It is freeware. http://s3.cloudberrylab.com/
Hi Andy,
I checked it out the other day actually. I’m developing some of my own tools because I needed to push up a ton of images and do it based on some internal business rules. Your app is pretty good though. I used it to copy some images from one bucket to another. It worked pretty good except for the permissions didn’t get copied over. The meta data did though. Thanks!
Nice code sample, but i think that simple HTTP HEAD request should be more efficient and faster way to check object existence. What do you think?
An HTTP GET request may be quicker and more efficient, but I didn’t really run any tests comparing the two.
i think you should check the metadata using GetObjectMetadataRequest instead. This way you don’t have to download the whole file.
an example is given here:
http://stackoverflow.com/questions/3526585/determine-if-an-object-exists-in-a-s3-bucket-based-on-wildcard