|
| 1 | +NIO Filesystem Provider for Google Cloud Storage (Alpha) |
| 2 | +======================================================== |
| 3 | + |
| 4 | +Implementation of Java 7 `java.nio.file.FileSystem` for |
| 5 | +[Google Cloud Storage](https://cloud.google.com/storage/). |
| 6 | + |
| 7 | +This library allows you to use the standardized Java file system API |
| 8 | +for interacting with Google Cloud Storage. |
| 9 | + |
| 10 | +[](https://travis-ci.org/GoogleCloudPlatform/gcloud-java) |
| 11 | +[](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) |
| 12 | +[]( https://img.shields.io/maven-central/v/com.google.cloud/gcloud-java-nio.svg) |
| 13 | +[](https://www.codacy.com/app/mziccard/gcloud-java) |
| 14 | +[](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) |
| 15 | + |
| 16 | +- [Homepage](https://googlecloudplatform.github.io/gcloud-java/) |
| 17 | +- [API Documentation](http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/cloud/storage/package-summary.html) |
| 18 | + |
| 19 | +> Note: This client is a work-in-progress, and may occasionally |
| 20 | +> make backwards-incompatible changes. |
| 21 | +
|
| 22 | +Quickstart |
| 23 | +---------- |
| 24 | +If you are using Maven, add this to your pom.xml file |
| 25 | +```xml |
| 26 | +<dependency> |
| 27 | + <groupId>com.google.cloud</groupId> |
| 28 | + <artifactId>gcloud-java-nio</artifactId> |
| 29 | + <version>0.2.3</version> |
| 30 | +</dependency> |
| 31 | +``` |
| 32 | +If you are using Gradle, add this to your dependencies |
| 33 | +```Groovy |
| 34 | +compile 'com.google.cloud:gcloud-java-nio:0.2.3' |
| 35 | +``` |
| 36 | +If you are using SBT, add this to your dependencies |
| 37 | +```Scala |
| 38 | +libraryDependencies += "com.google.cloud" % "gcloud-java-nio" % "0.2.3" |
| 39 | +``` |
| 40 | + |
| 41 | +Example Applications |
| 42 | +------------------- |
| 43 | + |
| 44 | +* [`Stat`](../../gcloud-java-examples/src/main/java/com/google/cloud/examples/nio/Stat.java) |
| 45 | +shows how to get started with NIO. |
| 46 | + |
| 47 | +* [`ParallelCountBytes`](../../gcloud-java-examples/src/main/java/com/google/cloud/examples/nio/ParallelCountBytes.java) |
| 48 | +efficiently downloads a file from Google Cloud Storage. |
| 49 | + |
| 50 | +* [`ListFileSystems`](../gcloud-java-nio-examples/README.md) illustrates how |
| 51 | +NIO can add Google Cloud Storage support to some legacy programs, without |
| 52 | +having to modify them. |
| 53 | + |
| 54 | + |
| 55 | +Authentication |
| 56 | +-------------- |
| 57 | + |
| 58 | +See the [Authentication](https://github.com/GoogleCloudPlatform/gcloud-java#authentication) |
| 59 | +section in the base directory's README. |
| 60 | + |
| 61 | +About Google Cloud Storage |
| 62 | +-------------------------- |
| 63 | + |
| 64 | +[Google Cloud Storage][cloud-storage] is a durable and highly available |
| 65 | +object storage service. Google Cloud Storage is almost infinitely scalable |
| 66 | +and guarantees consistency: when a write succeeds, the latest copy of the |
| 67 | +object will be returned to any GET, globally. |
| 68 | + |
| 69 | +See the [Google Cloud Storage docs][cloud-storage-activation] for more details |
| 70 | +on how to activate Cloud Storage for your project. |
| 71 | + |
| 72 | +About Java NIO Providers |
| 73 | +------------------------ |
| 74 | + |
| 75 | +Java NIO Providers is an extension mechanism that is part of Java and allows |
| 76 | +third parties to extend Java's [normal File API][java-file-api] to support |
| 77 | +additional filesystems. |
| 78 | + |
| 79 | +Getting Started |
| 80 | +--------------- |
| 81 | +#### Prerequisites |
| 82 | + |
| 83 | +For this tutorial, you will need a [Google Developers |
| 84 | +Console](https://console.developers.google.com/) project with "Google Cloud |
| 85 | +Storage" and "Google Cloud Storage JSON API" enabled via the console's API |
| 86 | +Manager. You will need to [enable |
| 87 | +billing](https://support.google.com/cloud/answer/6158867?hl=en) to use Google |
| 88 | +Cloud Storage. [Follow these |
| 89 | +instructions](https://cloud.google.com/docs/authentication#preparation) to get |
| 90 | +your project set up. You will also need to set up the local development |
| 91 | +environment by [installing the Google Cloud SDK](https://cloud.google.com/sdk/) |
| 92 | +and running the following commands in command line: `gcloud auth login` and |
| 93 | +`gcloud config set project [YOUR PROJECT ID]`. |
| 94 | + |
| 95 | +#### Installation and setup |
| 96 | +You'll need to obtain the `gcloud-java-nio` library. |
| 97 | + |
| 98 | +There are two ways to use this library. |
| 99 | + |
| 100 | +The recommended way is to follow the [Quickstart](#quickstart) section to add |
| 101 | +`gcloud-java-nio` as a dependency in your code. |
| 102 | + |
| 103 | +The second way is more complicated, but it allows you to add Google Cloud |
| 104 | +Storage support to some legacy Java programs. This approach is described in the |
| 105 | +[gcloud-java-nio-examples README](../gcloud-java-nio-examples/README.md). |
| 106 | + |
| 107 | +#### Accessing files |
| 108 | + |
| 109 | +The simplest way to get started is with `Paths` and `Files`: |
| 110 | + |
| 111 | + Path path = Paths.get(URI.create("gs://bucket/lolcat.csv")); |
| 112 | + List<String> lines = Files.readAllLines(path, StandardCharsets.UTF_8); |
| 113 | + |
| 114 | +If you know the paths will point to Google Cloud Storage, you can also use the |
| 115 | +direct formulation: |
| 116 | + |
| 117 | + try (CloudStorageFileSystem fs = CloudStorageFileSystem.forBucket("bucket") { |
| 118 | + Path path = fs.getPath("lolcat.csv"); |
| 119 | + List<String> lines = Files.readAllLines(path, StandardCharsets.UTF_8); |
| 120 | + } |
| 121 | + |
| 122 | +Once you have a `Path` you can use it as you would a normal file. For example |
| 123 | +you can use `InputStream` and `OutputStream` for streaming: |
| 124 | + |
| 125 | + try (InputStream input = Files.openInputStream(path)) { |
| 126 | + // ... |
| 127 | + } |
| 128 | + |
| 129 | +You can also set various attributes using CloudStorageOptions static helpers: |
| 130 | + |
| 131 | + Files.write(csvPath, csvLines, StandardCharsets.UTF_8, |
| 132 | + withMimeType(MediaType.CSV_UTF8), |
| 133 | + withoutCaching()); |
| 134 | + |
| 135 | +Limitations |
| 136 | +----------- |
| 137 | + |
| 138 | +This library is usable, but not yet complete. The following features are not |
| 139 | +yet implemented: |
| 140 | + * Listing all the buckets |
| 141 | + * Resuming upload or download |
| 142 | + * Generations |
| 143 | + * File attributes |
| 144 | + * (more) |
| 145 | + |
| 146 | +Some features are not on the roadmap: this library would be a poor choice to |
| 147 | +mirror a local filesystem onto the cloud because Google Cloud Storage has a |
| 148 | +different set of features from your local disk. This library, by design, |
| 149 | +does not mask those differences. Rather, it aims to expose the common |
| 150 | +subset via a familiar interface. |
| 151 | + |
| 152 | +**NOTE:** Cloud Storage uses a flat namespace and therefore doesn't support real |
| 153 | +directories. So this library supports what's known as "pseudo-directories". Any |
| 154 | +path that includes a trailing slash, will be considered a directory. It will |
| 155 | +always be assumed to exist, without performing any I/O. This allows you to do |
| 156 | +path manipulation in the same manner as you would with the normal UNIX file |
| 157 | +system implementation. You can disable this feature with |
| 158 | +`CloudStorageConfiguration.usePseudoDirectories()`. |
| 159 | + |
| 160 | +#### Complete source code |
| 161 | + |
| 162 | +There are examples in [gcloud-java-examples](../gcloud-java-examples/src/main/java/com/google/cloud/examples/nio/) |
| 163 | +for your perusal. |
| 164 | + |
| 165 | +Java Versions |
| 166 | +------------- |
| 167 | + |
| 168 | +Java 7 or above is required for using this client. |
| 169 | + |
| 170 | +Versioning |
| 171 | +---------- |
| 172 | + |
| 173 | +This library follows [Semantic Versioning](http://semver.org/). |
| 174 | + |
| 175 | +It is currently in major version zero (``0.y.z``), which means that anything |
| 176 | +may change at any time and the public API should not be considered |
| 177 | +stable. |
| 178 | + |
| 179 | +Contributing |
| 180 | +------------ |
| 181 | + |
| 182 | +Contributions to this library are always welcome and highly encouraged. |
| 183 | + |
| 184 | +See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` |
| 185 | +[shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/blob/master/contributing/readme.md#how-to-contribute-to-gcloud) |
| 186 | +for more information on how to get started. |
| 187 | + |
| 188 | +Please note that this project is released with a Contributor Code of Conduct. |
| 189 | +By participating in this project you agree to abide by its terms. See |
| 190 | +[Code of Conduct][code-of-conduct] for more information. |
| 191 | + |
| 192 | +License |
| 193 | +------- |
| 194 | + |
| 195 | +Apache 2.0 - See [LICENSE] for more information. |
| 196 | + |
| 197 | + |
| 198 | +[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md |
| 199 | +[code-of-conduct]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CODE_OF_CONDUCT.md#contributor-code-of-conduct |
| 200 | +[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE |
| 201 | +[TESTING]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/TESTING.md#testing-code-that-uses-storage |
| 202 | +[cloud-platform]: https://cloud.google.com/ |
| 203 | + |
| 204 | +[cloud-storage]: https://cloud.google.com/storage/ |
| 205 | +[cloud-storage-docs]: https://cloud.google.com/storage/docs/overview |
| 206 | +[cloud-storage-create-bucket]: https://cloud.google.com/storage/docs/cloud-console#_creatingbuckets |
| 207 | +[storage-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/cloud/storage/package-summary.html |
| 208 | +[cloud-storage-activation]:https://cloud.google.com/storage/docs/signup?hl=en |
| 209 | + |
| 210 | +[java-file-api]: https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html |
0 commit comments