获取我的数据块工作区中所有笔记本的列表答案

【问题标题】：Get list of all notebooks in my databricks workspace获取我的数据块工作区中所有笔记本的列表
【发布时间】：2021-11-25 11:44:59
【问题描述】：

如何获取工作区中所有笔记本的列表并将它们的名称以及完整路径存储在 csv 文件中，我尝试使用 Databricks CLI 选项，但似乎没有递归操作。

databricks workspace list

【问题讨论】：

标签： powershell command-line-interface databricks azure-databricks

【解决方案1】：

正如我们在代码中看到的，没有递归选项： https://github.com/databricks/databricks-cli/blob/master/databricks_cli/workspace/cli.py (def ls_cli)

示例解决方案是在python中导入cli并扩展它：

from databricks_cli.sdk import ApiClient
from databricks_cli.sdk import service


host = "your_host"
token = "your_token"

client = ApiClient(host=host, token=token)
objects = []
workspace = service.WorkspaceService(client)

def list_workspace_objects(path):
    elements = workspace.list(path).get('objects')
    if elements is not None:
        for object in elements:
            objects.append(object)
            if(object['object_type'] == 'DIRECTORY'):
                list_workspace_objects(object['path'])


list_workspace_objects("/")

print(objects)

【讨论】：

谢谢，正是需要的！

【解决方案2】：

您可以直接使用下面的代码。注意：测试代码

    from pyspark.sql.types import IntegerType
    from pyspark.sql.types import *
    from pyspark.sql import Row
    import base64
    import requests
    import json
    
    databricks_instance ="databricks Instance"
    
    url_list = f"{databricks_instance}/api/2.0/workspace/list"
    url_export = f"{databricks_instance}/api/2.0/workspace/export"
    
    
    payload = json.dumps({
      "path": "/"
    })
    headers = {
      'Authorization': 'Bearer token',
      'Content-Type': 'application/json'
    }
    
    response = requests.request("GET", url_list, headers=headers, data=payload).json()
    notebooks = []
    
    # Getting the all notebooks list for given notebooks.
    
    def list_notebooks(mylist):
      for element in mylist['objects']:
        if element['object_type'] == 'NOTEBOOK':
          notebooks.append(element)
        if element['object_type'] == 'DIRECTORY':
          payload_inner = json.dumps({
            "path": element['path']
          })
          response_inner = requests.request("GET", url_list, headers=headers, data=payload_inner).json()
          if len(response_inner) != 0:
            list_notebooks(response_inner)
      return notebooks
    
    result = list_notebooks(response)
    print(result[0])

【讨论】：