【问题标题】：Unable to Verify that GCS bucket and PKIX path building failed Errors in Creating and staging GCP Dataflow template无法验证 GCS 存储桶和 PKIX 路径构建失败创建和暂存 GCP 数据流模板时出错
【发布时间】：2020-11-07 19:50:10
【问题描述】：

我正在使用以下命令在云存储中创建和暂存 gcp 数据流模板：

mvn -X compile exec:java -Dexec.mainClass=main.java.TemplatePipeline -Dexec.args="--runner=DataflowRunner --project=hv-hcap-development --stagingLocation=gs://my-bucket/staging --templateLocation=gs://my-bucket/templates/template1"

构建失败并出现以下错误

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.563 s
[INFO] Finished at: 2020-07-17T17:32:05-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project dataflow-template: An exception occured while executing the Java class. Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions): InvocationTargetException: Unable to verify that GCS bucket gs://bucket-df-job exists. sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project dataflow-template: An exception occured while executing the Java class. Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
    at org.codehaus.mojo.exec.ExecJavaMojo.execute (ExecJavaMojo.java:311)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:224)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:149)
    at main.java.TemplatePipeline.main (TemplatePipeline.java:95)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:149)
    at main.java.TemplatePipeline.main (TemplatePipeline.java:95)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.RuntimeException: Unable to verify that GCS bucket gs://bucket-df-job exists.
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.verifyPathIsAccessible (GcsPathValidator.java:86)
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.validateOutputFilePrefixSupported (GcsPathValidator.java:53)
    at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions (DataflowRunner.java:274)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:149)
    at main.java.TemplatePipeline.main (TemplatePipeline.java:95)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alerts.getSSLException (Alerts.java:198)
    at sun.security.ssl.SSLSocketImpl.fatal (SSLSocketImpl.java:1967)
    at sun.security.ssl.Handshaker.fatalSE (Handshaker.java:331)
    at sun.security.ssl.Handshaker.fatalSE (Handshaker.java:325)
    at sun.security.ssl.ClientHandshaker.serverCertificate (ClientHandshaker.java:1689)
    at sun.security.ssl.ClientHandshaker.processMessage (ClientHandshaker.java:226)
    at sun.security.ssl.Handshaker.processLoop (Handshaker.java:1082)
    at sun.security.ssl.Handshaker.process_record (Handshaker.java:1010)
    at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:1079)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1388)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1416)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1400)
    at sun.net.www.protocol.https.HttpsClient.afterConnect (HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect (AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0 (HttpURLConnection.java:1340)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream (HttpURLConnection.java:1315)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream (HttpsURLConnectionImpl.java:264)
    at com.google.api.client.http.javanet.NetHttpRequest.execute (NetHttpRequest.java:113)
    at com.google.api.client.http.javanet.NetHttpRequest.execute (NetHttpRequest.java:84)
    at com.google.api.client.http.HttpRequest.execute (HttpRequest.java:1012)
    at com.google.auth.oauth2.UserCredentials.refreshAccessToken (UserCredentials.java:203)
    at com.google.auth.oauth2.OAuth2Credentials.refresh (OAuth2Credentials.java:157)
    at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata (OAuth2Credentials.java:145)
    at com.google.auth.oauth2.UserCredentials.getRequestMetadata (UserCredentials.java:281)
    at com.google.auth.http.HttpCredentialsAdapter.initialize (HttpCredentialsAdapter.java:91)
    at com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize (ChainingHttpRequestInitializer.java:52)
    at com.google.api.client.http.HttpRequestFactory.buildRequest (HttpRequestFactory.java:88)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest (AbstractGoogleClientRequest.java:422)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed (AbstractGoogleClientRequest.java:541)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed (AbstractGoogleClientRequest.java:474)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute (AbstractGoogleClientRequest.java:591)
    at com.google.cloud.hadoop.util.ResilientOperation$AbstractGoogleClientRequestExecutor.call (ResilientOperation.java:171)
    at com.google.cloud.hadoop.util.ResilientOperation.retry (ResilientOperation.java:67)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.getBucket (GcsUtil.java:521)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.bucketAccessible (GcsUtil.java:509)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.bucketAccessible (GcsUtil.java:482)
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.verifyPathIsAccessible (GcsPathValidator.java:83)
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.validateOutputFilePrefixSupported (GcsPathValidator.java:53)
    at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions (DataflowRunner.java:274)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:149)
    at main.java.TemplatePipeline.main (TemplatePipeline.java:95)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:748)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild (PKIXValidator.java:450)
    at sun.security.validator.PKIXValidator.engineValidate (PKIXValidator.java:317)
    at sun.security.validator.Validator.validate (Validator.java:262)
    at sun.security.ssl.X509TrustManagerImpl.validate (X509TrustManagerImpl.java:330)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted (X509TrustManagerImpl.java:227)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted (X509TrustManagerImpl.java:132)
    at sun.security.ssl.ClientHandshaker.serverCertificate (ClientHandshaker.java:1671)
    at sun.security.ssl.ClientHandshaker.processMessage (ClientHandshaker.java:226)
    at sun.security.ssl.Handshaker.processLoop (Handshaker.java:1082)
    at sun.security.ssl.Handshaker.process_record (Handshaker.java:1010)
    at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:1079)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1388)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1416)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1400)
    at sun.net.www.protocol.https.HttpsClient.afterConnect (HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect (AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0 (HttpURLConnection.java:1340)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream (HttpURLConnection.java:1315)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream (HttpsURLConnectionImpl.java:264)
    at com.google.api.client.http.javanet.NetHttpRequest.execute (NetHttpRequest.java:113)
    at com.google.api.client.http.javanet.NetHttpRequest.execute (NetHttpRequest.java:84)
    at com.google.api.client.http.HttpRequest.execute (HttpRequest.java:1012)
    at com.google.auth.oauth2.UserCredentials.refreshAccessToken (UserCredentials.java:203)
    at com.google.auth.oauth2.OAuth2Credentials.refresh (OAuth2Credentials.java:157)
    at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata (OAuth2Credentials.java:145)
    at com.google.auth.oauth2.UserCredentials.getRequestMetadata (UserCredentials.java:281)
    at com.google.auth.http.HttpCredentialsAdapter.initialize (HttpCredentialsAdapter.java:91)
    at com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize (ChainingHttpRequestInitializer.java:52)
    at com.google.api.client.http.HttpRequestFactory.buildRequest (HttpRequestFactory.java:88)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest (AbstractGoogleClientRequest.java:422)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed (AbstractGoogleClientRequest.java:541)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed (AbstractGoogleClientRequest.java:474)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute (AbstractGoogleClientRequest.java:591)
    at com.google.cloud.hadoop.util.ResilientOperation$AbstractGoogleClientRequestExecutor.call (ResilientOperation.java:171)
    at com.google.cloud.hadoop.util.ResilientOperation.retry (ResilientOperation.java:67)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.getBucket (GcsUtil.java:521)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.bucketAccessible (GcsUtil.java:509)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.bucketAccessible (GcsUtil.java:482)
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.verifyPathIsAccessible (GcsPathValidator.java:83)
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.validateOutputFilePrefixSupported (GcsPathValidator.java:53)
    at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions (DataflowRunner.java:274)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:149)
    at main.java.TemplatePipeline.main (TemplatePipeline.java:95)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:748)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build (SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild (SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build (CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild (PKIXValidator.java:445)
    at sun.security.validator.PKIXValidator.engineValidate (PKIXValidator.java:317)
    at sun.security.validator.Validator.validate (Validator.java:262)
    at sun.security.ssl.X509TrustManagerImpl.validate (X509TrustManagerImpl.java:330)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted (X509TrustManagerImpl.java:227)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted (X509TrustManagerImpl.java:132)
    at sun.security.ssl.ClientHandshaker.serverCertificate (ClientHandshaker.java:1671)
    at sun.security.ssl.ClientHandshaker.processMessage (ClientHandshaker.java:226)
    at sun.security.ssl.Handshaker.processLoop (Handshaker.java:1082)
    at sun.security.ssl.Handshaker.process_record (Handshaker.java:1010)
    at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:1079)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1388)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1416)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1400)
    at sun.net.www.protocol.https.HttpsClient.afterConnect (HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect (AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0 (HttpURLConnection.java:1340)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream (HttpURLConnection.java:1315)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream (HttpsURLConnectionImpl.java:264)
    at com.google.api.client.http.javanet.NetHttpRequest.execute (NetHttpRequest.java:113)
    at com.google.api.client.http.javanet.NetHttpRequest.execute (NetHttpRequest.java:84)
    at com.google.api.client.http.HttpRequest.execute (HttpRequest.java:1012)
    at com.google.auth.oauth2.UserCredentials.refreshAccessToken (UserCredentials.java:203)
    at com.google.auth.oauth2.OAuth2Credentials.refresh (OAuth2Credentials.java:157)
    at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata (OAuth2Credentials.java:145)
    at com.google.auth.oauth2.UserCredentials.getRequestMetadata (UserCredentials.java:281)
    at com.google.auth.http.HttpCredentialsAdapter.initialize (HttpCredentialsAdapter.java:91)
    at com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize (ChainingHttpRequestInitializer.java:52)
    at com.google.api.client.http.HttpRequestFactory.buildRequest (HttpRequestFactory.java:88)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest (AbstractGoogleClientRequest.java:422)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed (AbstractGoogleClientRequest.java:541)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed (AbstractGoogleClientRequest.java:474)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute (AbstractGoogleClientRequest.java:591)
    at com.google.cloud.hadoop.util.ResilientOperation$AbstractGoogleClientRequestExecutor.call (ResilientOperation.java:171)
    at com.google.cloud.hadoop.util.ResilientOperation.retry (ResilientOperation.java:67)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.getBucket (GcsUtil.java:521)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.bucketAccessible (GcsUtil.java:509)
    at org.apache.beam.sdk.extensions.gcp.util.GcsUtil.bucketAccessible (GcsUtil.java:482)
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.verifyPathIsAccessible (GcsPathValidator.java:83)
    at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.validateOutputFilePrefixSupported (GcsPathValidator.java:53)
    at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions (DataflowRunner.java:274)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:149)
    at main.java.TemplatePipeline.main (TemplatePipeline.java:95)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
    at java.lang.Thread.run (Thread.java:748)
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

我已经从终端登录到 GCP 使用 gcloud auth application-default login 和 export GOOGLE_CLOUD_CREDENTIALS=${PATH}

另外，我已经使用 cacerts 添加了谷歌云证书 sudo keytool -import -trustcacerts -file /path/to/google.cer -alias google -keystore $JAVA_HOME/jre/lib/security/cacerts 并验证它存在

非常感谢任何解决此问题的帮助。

【问题讨论】：

标签： java maven google-cloud-platform google-cloud-dataflow pkix

【解决方案1】：

这可能有两个原因。

要么没有按名称命名的存储桶 -bucket-df-job。如果是，则创建存储桶
无论您使用什么服务帐号并已分配给GOOGLE_APPLICATION_CREDENTIALS，该服务帐号是否有权访问 GCS 存储桶？如果没有，请授予访问权限并重新尝试构建数据流模板。

【讨论】：

感谢@bigbounty 的回复。该存储桶存在于我的帐户中，并且在权限选项卡下我可以看到我的服务帐户。此外，我已将存储桶操作的所有权限授予服务帐户。不幸的是，它仍然显示相同的错误。我创建了一个 python 版本，它可以在相同的资源和凭据下正常工作。

【解决方案2】：

除了确保您输入了正确的存储桶名称/路径之外，我还会确保：

您已启用所有必需的 API。目前包括：Cloud Dataflow、Compute Engine、Stackdriver Logging、Cloud Storage、Cloud Storage JSON、BigQuery、Cloud Pub/Sub、Cloud Datastore 和 Cloud Resource Manager
您通过创建具有 Project > Owner 角色的服务帐户正确设置了身份验证。

请参阅此link，以确保您满足所有要求，因为您正处于数据流的创建阶段。

【讨论】：

谢谢@eyoto。我已确保所有这些都已设置，但不确定为什么它不适用于 Java 和 Maven 代码。我为同一件事（创建和暂存数据流模板）的替代版本的 python 代码工作正常。如果这是我认为的帐户或权限问题，它也不会起作用。

【解决方案3】：

编辑-2

我终于可以解决这个错误，现在我可以从 mvn 和 Eclipse 运行我的 Dataflow 运行程序。我做了以下事情：

我发现我在 pom 中的 SDK 版本到处都是，因此，将 directrunner 和 dataflowrunner 版本合并到 2.29.0。
我手动将 ServiceAccountCredentials 范围设置为“https://www.googleapis.com/auth/cloud-platform”，我将其删除并仅添加了 GOOGLE_APPLICATION_CREDENTIALS 作为运行时环境变量，并为我的 serviceaccount 键分配了缩小范围用于数据流作业的执行。
为了运行 mvn compile exec:java，在我的脚本中添加了 GOOGLE_APPLICATION_CREDENTIALS 以便为每个会话运行

编辑

我可以克服凭据错误，但是由于以下错误，我的工作再次失败：

java.io.IOException: Error matching file spec gs://gce_lab/staging/nashorn-BJZNQ7N8Lsfq-WSM0IMsRCwFMC3RIxBOEjrlB1YwKOw.jar

查看之前的一些 WARNs，我发现我所有复制到暂存位置的操作都失败了……不过，我仍在尝试调试。

我正在使用以下内容：

<dependency>
  <groupId>org.apache.beam</groupId>
  <artifactId>beam-sdks-java-core</artifactId>
  <version>2.33.0</version>
</dependency>
<dependency>
  <groupId>org.apache.beam</groupId>
  <artifactId>beam-runners-google-cloud-dataflow- java</artifactId>
  <version>2.33.0</version>
</dependency>
<dependency>
  <groupId>org.apache.beam</groupId>
  <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
  <version>2.33.0</version>
</dependency>

@StackOverFlowUser - 我试图从 Eclipse 运行我的 DataFlow 管道并遇到完全相同的错误，尽管设置了正确的 SA 和其他细节。当我在 DataflowPipelineOptions 参考中添加以下 2 个参数时，它对我有用：

pipelineOptions.setPathValidatorClass(NoopPathValidator.class);
pipelineOptions.setTemplateLocation("gs://<template_lcoation>"); // all 3 -> temp, stage and template location.

【讨论】：