【问题标题】:Submit-AdlJob fails with Method not Found when running in Azure Automation Runbook在 Azure 自动化 Runbook 中运行时,Submit-AdlJob 失败并出现 Method not Found
【发布时间】:2018-05-17 06:10:04
【问题描述】:

我正在尝试通过 Azure 自动化服务安排我之前在 Azure Data Lake Analytics 上运行的一些 U-SQL 脚本(我也尝试在 ADF 中执行此操作,但它也对我不起作用,我'将起草另一个问题来解决该路径)。

这是基于MS Docs Guidance 的简化运行手册代码,但对我来说仍然存在错误:

$Conn = Get-AutomationConnection -Name AzureRunAsConnection
Connect-AzureRmAccount -ServicePrincipal `
 -Tenant $Conn.TenantID `
 -ApplicationId $Conn.ApplicationID `
 -CertificateThumbprint $Conn.CertificateThumbprint

$adla = "example_adla01"

$script = @"
@a  = 
    SELECT * FROM 
    (VALUES
        ("Contoso", 1500.0),
        ("Woodgrove", 2700.0)
    ) AS D( customer, amount );
OUTPUT @a
TO "/data.csv"
USING Outputters.Csv();
"@

Submit-AdlJob -AccountName $adla -Script $script -Name "Demo"

正在调用的命令的版本。

CommandType  Name            Version   Source
-----------  ----            -------   ------                                    
Alias        Submit-AdlJob   4.2.3     AzureRM.DataLakeAnalytics   

此作业在我的本地 Powershell ISE 和 Cloud Shell 中运行良好;但在 Azure 自动化服务测试运行程序中失败并出现此错误(已更新以包含更详细的错误):

Submit-AzureRmDataLakeAnalyticsJob : A command that prompts the user failed because the host program or the command type does not support user interaction. The host was attempting to request confirmation with the following message: 

Method not found: 'Newtonsoft.Json.Serialization.IAttributeProvider 
Newtonsoft.Json.Serialization.JsonProperty.get_AttributeProvider()'.
At line:23 char:1

+ Submit-AzureRmDataLakeAnalyticsJob -AccountName $adla -Script $script ...    
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotImplemented: (:) [Submit-AzureRmDataLakeAnalyticsJob], HostException
+ FullyQualifiedErrorId : HostFunctionNotImplemented,Microsoft.Azure.Commands.DataLakeAnalytics.SubmitAzureDataLakeAnalyticsJob

我已经导入了我认为正确的模块,但无法找到与 JSON 相关的任何其他模块,或者根本无法调用 JSON 序列化程序的原因,否则这是我站起来的全新自动化实例今天早上。

命令的调试输出:

7:04:53 PM - SubmitAzureDataLakeAnalyticsJob begin processing with ParameterSet 'SubmitUSqlJob'.

7:04:53 PM - using account id 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'...
[Common.Authentication]: Authenticating using Account: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', environment:  'AzureCloud', tenant: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 

04/23/2018 19:04:54:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenForClientHandler: === Token Acquisition started:
    Authority: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/
    Resource: https://management.core.windows.net/
    ClientId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    CacheType: Microsoft.Azure.Commands.Common.Authentication.AuthenticationStoreTokenCache (1 items)
    Authentication Target: Client


Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Looking up cache for a token...
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - TokenCache: An item matching the requested resource was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: 59.9101559033333 minutes left until token in cache expires
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: A matching item (access token or refresh token or both) was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 :

04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenHandlerBase: === Token Acquisition finished 
successfully. An access token was retuned:
    Access Token Hash: //removed
    Refresh Token Hash: [No Refresh Token]
    Expiration Time: 04/23/2018 20:04:48 +00:00
    User Hash: null


Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54:  - AuthenticationStoreTokenCache: Serializing token cache with 1 items.

[Common.Authentication]: Authenticating using Account: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', environment: 'AzureCloud', tenant: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 
04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenForClientHandler: === Token Acquisition started:
    Authority: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/
    Resource: https://datalake.azure.net/
    ClientId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    CacheType: Microsoft.Azure.Commands.Common.Authentication.AuthenticationStoreTokenCache (1 items)
    Authentication Target: Client
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Looking up cache for a token...
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: No matching token was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Storing token in the cache...
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: An item was stored in the cache

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 :

04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenHandlerBase: === Token Acquisition finished successfully. An access token was retuned:
    Access Token Hash: //removed
    Refresh Token Hash: [No Refresh Token]
    Expiration Time: 04/23/2018 20:04:53 +00:00
    User Hash: null

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54:  - AuthenticationStoreTokenCache: Serializing token cache with 2 items.
[Common.Authentication]: Authenticating using Account: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', environment: 'AzureCloud', tenant: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 

04/23/2018 19:04:54: 1cdd67fa-ce99-4931-b1aa-4ced126fa018 - AcquireTokenForClientHandler: === Token Acquisition started:
    Authority: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/
    Resource: https://datalake.azure.net/
    ClientId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    CacheType: Microsoft.Azure.Commands.Common.Authentication.AuthenticationStoreTokenCache (2 items)
    Authentication Target: Client


Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Looking up cache for a token...
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - TokenCache: An item matching the requested resource was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: 59.9828124866667 minutes left until token in cache expires
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: A matching item (access token or refresh token or both) was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 :

04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenHandlerBase: === Token Acquisition finished successfully. An access token was retuned:
    Access Token Hash: //remove
    Refresh Token Hash: [No Refresh Token]
    Expiration Time: 04/23/2018 20:04:53 +00:00
    User Hash: null

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54:  - AuthenticationStoreTokenCache: Serializing token cache with 2 items.

【问题讨论】:

  • 您能否提及您使用的是哪个版本的 cmdlet?要获取此信息,您可以运行 Get-Command Submit-AdlJob -All。另外,您能否在运行脚本时尝试通过在脚本的最后一行添加-Debug 来获取调试输出? (如果您在此处共享调试输出,您可能希望省略任何敏感细节,如身份验证令牌、路径名等。)
  • @MattH 我已经更新了问题以包含附加信息,然后按照 Joo Wan Ro 的建议删除和导入。

标签: powershell json.net azure-data-lake u-sql azure-automation


【解决方案1】:

除了 Matt 的评论,您能否尝试通过删除所有模块并仅导入 AzureRM.DataLakeAnalytics 来刷新模块?导入 AzureRM.DataLakeAnalytics 应该会自动导入 AzureRM.Profile,这是唯一的依赖模块。

【讨论】:

  • 我可能误解了你的建议,我删除了一堆模块并只导入 AzureRM.DataLakeAnalytics,但给了我一个错误,我需要先将 AzureRM.Profile 导入我的帐户。 “此模块具有此帐户中不存在的依赖项。必须存在所有依赖项才能导入此模块。依赖项:AzureRM.Profile (≥ 4.6.0)”
  • 请先尝试导入 AzureRM.Profile。在 Powershell 中,它会自动导入 AzureRM.Profile。
  • 我清除了模块并重新导入了 AzureRM.Profile 和 AzureRM.DataLakeAnalytics 并得到了与原始帖子中相同的错误。
【解决方案2】:

尝试删除(或重命名)您在 C:\Windows\Assembly\GACC:\ 中的所有 newtonsoft.json.dll 文件Windows\Assembly\GAC_MSIL 目录。

【讨论】:

  • 这是 Azure 自动化中的运行手册,我遇到了问题,我不确定在哪里访问底层主机以清除 GAC,或者即使可能。
【解决方案3】:

自从我打开这个问题后,似乎发生了变化,现在我得到了一个不同的(更有用的错误消息)。

Environments                                                                                                            
------------                                                                                                            

{[AzureUSGovernment, AzureUSGovernment], [AzureChinaCloud, AzureChinaCloud], 
[AzureGermanCloud, AzureGermanCloud], [A...
Submit-AzureRmDataLakeAnalyticsJob : The user is not authorized to perform this operation on storage.

At line:21 char:1
+ Submit-AzureRmDataLakeAnalyticsJob -AccountName $adla -Script $script ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : CloseError: (:) [Submit-AzureRmDataLakeAnalyticsJob], CloudException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.DataLakeAnalytics.SubmitAzureDataLakeAnalyticsJob

我必须添加 Azure 自动化 RunAs 帐户才能访问基础 Azure Data Lake Storage 帐户,因为虽然它从订阅的参与者继承了对计算的访问权限,但它没有获得对 Azure 数据的默认权限Lake Storage 帐户。

我是通过 Azure Data Lake“添加用户向导”完成的

我现在看到 Azure 自动化用户能够针对 Azure Data Lake Analytics Blade 中的作业管理器触发作业。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2019-07-03
    • 2015-08-11
    • 2020-09-16
    • 2019-03-15
    • 2018-03-07
    • 1970-01-01
    • 2020-11-24
    • 2020-07-24
    相关资源
    最近更新 更多