azure hdinsight 创建具有附加存储帐户的集群答案

【问题标题】：azure hdinsight create cluster with additional storage accountazure hdinsight 创建具有附加存储帐户的集群
【发布时间】：2016-07-08 21:01:14
【问题描述】：

我正在尝试使用 azure powershell 自动化 HDINSIGHT 集群。

我正在使用来自官方文档https://github.com/Azure/azure-content/blob/master/articles/hdinsight/hdinsight-hadoop-create-linux-clusters-azure-powershell.md的这个模板

如何为我的集群设置额外的存储帐户？你有什么主意吗？文档提到参数 -AdditionalStorageAccounts 没有示例

$resourceGroupName = "<ResourceGroupName>"      # Provide the Resource Group name
$storageAccountName = "<StorageAcccountName>"   # Provide the Storage account name
$containerName = "<ContainerName>"              # Provide the container name
$storageAccountKey = Get-AzureStorageAccountKey -Name $storageAccountName -ResourceGroupName $resourceGroupName | %{ $_.Key1 }

# Set these variables
$clusterName = $containerName                   # As a best practice, have the same name for the cluster and container
$clusterNodes = <ClusterSizeInNodes>            # The number of nodes in the HDInsight cluster
$credentials = Get-Credential -Message "Enter Cluster user credentials" -UserName "admin"
$sshCredentials = Get-Credential -Message "Enter SSH user credentials"

# The location of the HDInsight cluster. It must be in the same data center as the Storage account.
$location = Get-AzureRmStorageAccount -ResourceGroupName $resourceGroupName -StorageAccountName $storageAccountName | %{$_.Location}

# Create a new HDInsight cluster
New-AzureRmHDInsightCluster -ClusterName $clusterName -ResourceGroupName $resourceGroupName -HttpCredential $credentials -Location $location -DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" -DefaultStorageAccountKey $storageAccountKey -DefaultStorageContainer $containerName  -ClusterSizeInNodes $clusterNodes -ClusterType Hadoop -OSType Linux -Version "3.2" -SshCredential $sshCredentials

【问题讨论】：

标签： powershell azure azure-hdinsight

【解决方案1】：

这是一个来自 C# 库的示例，我已经有大约 2 年没有使用此代码了，所以 api 可能已经改变，但曾经可以工作，希望对你有所帮助。

// PROVIDE THE CERTIFICATE THUMBPRINT TO RETRIEVE THE CERTIFICATE FROM THE CERTIFICATE STORE 
var store = new X509Store(StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
var cert = store.Certificates.Cast<X509Certificate2>().First(item => item.Thumbprint == thumbprint);

// CREATE AN HDINSIGHT CLIENT OBJECT
var creds = new HDInsightCertificateCredential(Guid.Parse(subscriptionid), cert);
var client = HDInsightClient.Connect(creds);
client.IgnoreSslErrors = true;


// the location of additional-libs that will get pulled into the the env on create
string hiveAddtionalLibContainer = "additional-hive-lib";
var hiveAdditionalLibStorage = new WabStorageAccountConfiguration(storageaccountname, storageaccountkey, hiveAddtionalLibContainer);


// PROVIDE THE CLUSTER INFORMATION
var clusterInfo = new ClusterCreateParametersV2()
{
    Name = clusterName,
    Location = location,
    DefaultStorageAccountName = storageaccountname,
    DefaultStorageAccountKey = storageaccountkey,
    DefaultStorageContainer = clusterName,
    UserName = username,
    Password = password,
    ClusterSizeInNodes = clustersize,
    Version = "3.2",     
    ClusterType = Microsoft.WindowsAzure.Management.HDInsight.ClusterProvisioning.Data.ClusterType.Hadoop,

};

// add more storage
clusterInfo.AdditionalStorageAccounts.Add(new WabStorageAccountConfiguration(storageaccountnameAdd1, storageaccountkeyAdd1));

client.CreateCluster(clusterInfo);

【讨论】：

谢谢，我正在尝试使用 azure powershell 找到解决方案。再次感谢
我从头开始重新创建资源、存储帐户实体等。如果我们设置不同于集群名称的存储 blob 容器名称，似乎集群会失败..

【解决方案2】：

看来我们应该为集群和默认容器指定相同的名称，以便正确创建您的 HDInsight 集群。

罗伯托

【讨论】：

【解决方案3】：

以下几行创建一个存储帐户，并将其添加为附加存储帐户

# create a storage account
$additionalStorageAccountName = $token + "store2"
New-AzureRmStorageAccount -ResourceGroupName $resourceGroupName -StorageAccountName $additionalStorageAccountName -Location $location -Type Standard_LRS
$additionalStorageAccountKey = (Get-AzureRmStorageAccountKey -Name $additionalStorageAccountName -ResourceGroupName $resourceGroupName)[0].Value

# Specify the additional storage account
$config = New-AzureRmHDInsightClusterConfig
Add-AzureRmHDInsightStorage -Config $config -StorageAccountName "$additionalStorageAccountName.blob.core.windows.net" -StorageAccountKey $additionalStorageAccountKey

# Create a new HDInsight cluster with an additional storage account
New-AzureRmHDInsightCluster `
    -ClusterName $clusterName `
    -ResourceGroupName $resourceGroupName `
    -HttpCredential $credentials `
    -Location $location `
    -DefaultStorageAccountName "$defaultStorageAccountName.blob.core.windows.net" `
    -DefaultStorageAccountKey $defaultStorageAccountKey `
    -DefaultStorageContainer $defaultStorageContainerName  `
    -ClusterSizeInNodes $clusterNodes `
    -ClusterType Hadoop `
    -OSType Linux `
    -Version "3.4" `
    -SshCredential $sshCredentials `
    -Config $config

【讨论】：