s3 getobject body not closed

[thomasf]

I don't see that file handles are closed anywhere.

[thomasf]

I needed to be able to use my own S3 client instance so I could provide my own configuration for the client and use fake version of it it for tests.

I just added tracking of all GetObject bodies and a close method I can call from my code for now...

I can confirm that when I run the Close() method after one of the zipfile operations the response bodies are still not closed. I am doing zip.GetCentralDirectory() and lots of zipfile.ReaderForRecord(...) calls

// S3ObjectFetcher implements the Fetcher interface from cloudzip.zipfile.
type S3ObjectFetcher struct {
	client  S3Client
	bucket  string
	key     string
	closers []io.Closer
}

// NOTE: This is only here because it looks like the upstream library doesn't
// close file handles.
//
// Will wait for discussion on https://github.com/ozkatz/cloudzip/issues/10
// before taking any other action.
func (s *S3ObjectFetcher) Close() error {
	var closersWithErrors []io.Closer
	var errs []error
	for _, c := range s.closers {
		err := c.Close()
		if err != nil {
			errs = append(errs, err)
			closersWithErrors = append(closersWithErrors, c)
		}
	}
	s.closers = closersWithErrors
	return errors.Join(errs...)
}

func (s *S3ObjectFetcher) Fetch(ctx context.Context, startOffset *int64, endOffset *int64) (io.ReadCloser, error) {
	response, err := s.client.GetObject(ctx, &s3.GetObjectInput{
		Bucket: aws.String(s.bucket),
		Key:    aws.String(s.key),
		Range:  http_range.BuildRange(startOffset, endOffset),
	})
	if err != nil {
		return nil, err
	}
	s.closers = append(s.closers, response.Body)
	return response.Body, nil
}

[thomasf]

ReaderForRecord should return a io.ReadCloser instead of a Reader, if there is some underlying storage type that does not require closing they should probably just use be wrapped with io.NopCloser().

GetCentralDirectory should make sure it closes Body after it has read what it needs.