【发布时间】:2022-06-13 04:54:58
【问题描述】:
我们有一个夜间进程(全部在 Azure VM 上运行),它主要是一组用于在我们的 TEST 环境中更新数据库的 powershell 脚本。我们经常遇到如下问题。我们有大约 25 个数据库要更新,在此过程中的某个随机时间点,我们会在尝试使用 Azure 托管身份验证时遇到零星错误。下面显示了它如何在一个 DB 上完美运行,但随后在下一个 DB 上失败,即使从同一 Azure VM 上的同一进程运行并连接到源 VM 的托管标识配置为具有完整 sysAdmin 的同一 SQL 托管实例权利。
实际上,我之前曾被告知,“这些服务并不那么可靠,因此您只需将所有此类逻辑包装在重试逻辑中,并希望您最终能走运”。我们看到这些问题的频率让我想知道是否还有更多的事情发生,是否有人看到过这个或有其他想法?如果这些服务的性质使得它每次都不能可靠地工作,你会认为他们会在核心库中添加重试逻辑,例如 Microsft.Data.SqlClient 或底层 Azure.Identity 库 vs期望开发人员在逻辑链中检测到这些问题并做出响应?
无论如何,我不确定是否有其他人看到过这个,或者我只是不走运:)
[06:51:56][Step 5/6] ========================================================================
[06:51:56][Step 5/6] Updating DB zzzzz_Regression (zzzzzRegression) on server xxxxx-tcc-sql.2222.database.windows.net (xxxxx-TCC-SQL)
[06:51:56][Step 5/6] ========================================================================
[06:51:57][Step 5/6] 6:51:56 AM:Using env file from command line: .\Package\BuildScripts\Configs\mmmmmTestAzure.fmenv.xml
[06:51:57][Step 5/6] 6:51:57 AM:Loaded environment: mmmmmTestAzure
[06:51:57][Step 5/6] 6:51:57 AM:Applying change scripts.
[06:52:09][Step 5/6] 6:52:09 AM:Creating functions.
[06:52:27][Step 5/6] 6:52:27 AM:Creating triggers.
[06:52:34][Step 5/6] 6:52:34 AM:Creating stored procedures.
[06:54:08][Step 5/6] 6:54:07 AM:Database FM3OMS_Regression upgraded successfully.
[06:54:08][Step 5/6] Inializing Report Urls on Database Id zzzzzRegression
[06:54:14][Step 5/6] ========================================================================
[06:54:14][Step 5/6] Updating DB yyyyy (yyyyy) on server xxxxx-tcc-sql.2222.database.windows.net (xxxxx-TCC-SQL)
[06:54:14][Step 5/6] ========================================================================
[06:54:14][Step 5/6] 6:54:14 AM:Using env file from command line: .\Package\BuildScripts\Configs\mmmmmTestAzure.fmenv.xml
[06:54:15][Step 5/6] 6:54:14 AM:Loaded environment: mmmmm
[06:54:15][Step 5/6] 6:54:14 AM:Applying change scripts.
[06:54:28][Step 5/6] 6:54:28 AM:Creating functions.
[06:54:52][Step 5/6] 6:54:52 AM:Creating triggers.
[06:55:10][Step 5/6] 6:55:03 AM:Creating stored procedures.
[06:57:39][Step 5/6] 6:57:39 AM:Database yyyyy upgraded successfully.
[06:57:42][Step 5/6] Inializing Report Urls on Database Id yyyyy
[06:58:18][Step 5/6] Initialize-FMReportUrls : 6:58:07 AM:Using env file from command line:
[06:58:18][Step 5/6] .\Package\BuildScripts\Configs\mmmmmTestAzure.fmenv.xml 6:58:13 AM:Loaded environment: mmmmmTestAzure
[06:58:18][Step 5/6] System.AggregateException: One or more errors occurred. ---> Azure.Identity.CredentialUnavailableException:
[06:58:18][Step 5/6] ManagedIdentityCredential authentication unavailable. No Managed Identity endpoint found. at
[06:58:18][Step 5/6] Azure.Identity.ManagedIdentityClient.<AuthenticateAsync>d__12.MoveNext() --- End of stack trace from previous location
[06:58:18][Step 5/6] where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at
[06:58:18][Step 5/6] System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at
[06:58:18][Step 5/6] Azure.Identity.ManagedIdentityCredential.<GetTokenImplAsync>d__9.MoveNext() --- End of stack trace from previous
[06:58:18][Step 5/6] location where exception was thrown --- at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex)
[06:58:18][Step 5/6] at Azure.Identity.ManagedIdentityCredential.<GetTokenImplAsync>d__9.MoveNext() --- End of stack trace from previous
[06:58:18][Step 5/6] location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
[06:58:18][Step 5/6] task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at
[06:58:18][Step 5/6] Azure.Identity.ManagedIdentityCredential.<GetTokenAsync>d__7.MoveNext() --- End of stack trace from previous location
[06:58:18][Step 5/6] where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at
[06:58:18][Step 5/6] System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at
[06:58:18][Step 5/6] System.Threading.Tasks.ValueTask`1.get_Result() at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.ActiveDirectoryAuthenticationProvider.<AcquireTokenAsync>d__19.MoveNext() --- End of inner
[06:58:18][Step 5/6] exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
[06:58:18][Step 5/6] at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) at
[06:58:18][Step 5/6] System.Threading.Tasks.Task`1.get_Result() at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.SqlInternalConnectionTds.GetFedAuthToken(SqlFedAuthInfo fedAuthInfo) at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.SqlInternalConnectionTds.OnFedAuthInfo(SqlFedAuthInfo fedAuthInfo) at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream,
[06:58:18][Step 5/6] BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream,
[06:58:18][Step 5/6] BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK) at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword,
[06:58:18][Step 5/6] SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover, Boolean
[06:58:18][Step 5/6] isFirstTransparentAttempt, Boolean disableTnir) at
[06:58:18][Step 5/6] Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword,
[06:58:18][Step 5/6] SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential
[06:58:18][Step 5/6] credential, TimeoutTimer timeout) at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer
[06:58:18][Step 5/6] timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString
[06:58:18][Step 5/6] newSecurePassword, Boolean redirectedUserInstance) at
【问题讨论】:
-
看起来它正在返回某种错误,而不仅仅是超时。您可以使用 Fiddler 或 Wireshark 来查看确切的响应吗?
-
您可以在连接字符串中添加额外的参数以重试:
Connection Timeout=30;ConnectRetryCount=3;ConnectRetryInterval=10; -
谢谢,我们将尝试连接字符串的选项。由于该问题只能随机重现,因此不确定我是否/何时能够确认它解决了该问题,但绝对值得一试!谢谢!
标签: sql-server azure azure-managed-identity azure-sql-managed-instance