File Explorer Plugins

File Explorer plugins allow you to import an entire folder of assets at once into the project of your choice, from a private bucket.

Creating a File Explorer Plugin

First, add an integration from Hub to the private bucket you'd like to navigate with the File Explorer. You may find instructions on doing so here.

Following the steps outlined in this section, create a new plugin from the UI, choosing "File Explorer" as the plugin type.

Then, create and run a Python script using the FileExplorerPlugin class you can find in our ango Python package under ango.plugins.

You will need to add the ango package to your Python environment by running

pip install ango

Here is the class's documentation, and an example:

FileExplorerPlugin

Parameters:

• id: string

  • The plugin's ID. You may obtain this ID from the plugin's information box in the Development section of the Plugin page.

• secret: string

  • The plugin's secret. You can think of this as a private key you'll need to be able to connect your script to the plugin. You may obtain this secret from the plugin's information box in the Development section of the Plugin page.

• callback: Callable[[str, dict], Tuple[str, BytesIO]]

  • The callback function. This function will be run whenever a user runs this plugin.

Callback Function

Parameters:

• **data: dict

  • folder: str

  • projectId: str

  • integrationId: str

  • bucket: str

    • The ID of the project for which the plugin was run.

  • apiKey: str

  • orgId: str

  • runBy: str

    • The user ID of the user running the plugin.

  • session: str

  • logger: PluginLogger

  • batches: List[str]

  • files

  • upload

  • project

  • scroll_token

  • configJSON: str

    • The config JSON your users will pass to you through the Config JSON text field when running the plugin. Warning: the JSON will be passed as a string so you will have to destringify it. Example code to obtain the original JSON as a Python object:

  def sample_callback(**data):
      config_str = data.get('configJSON')
      config = json.loads(config_str)

Sample Python Script

This script allows your users to navigate the file structure of a private bucket from AWS S3, from Ango Hub's UI, and to them import an entire folder of valid assets into the project of their choice.

import boto3 as boto3
from ango.sdk import SDK
from ango.plugins import FileExplorerPlugin, run


HOST = '<YOUR HOST>'
PLUGIN_ID = '<YOUR PLUGIN ID>'
PLUGIN_SECRET = '<YOUR PLUGIN SECRET>'

PAGE_SIZE = 50


def sample_callback(**data):
    # Extract input parameters
    bucket = data.get("bucket")
    folder = data.get("folder", "")
    files = data.get("files", None)
    upload = data.get("upload", False)
    project_id = data.get("projectId", None)
    integration_id = data.get("integrationId", None)
    scroll_token = data.get("scrollToken", None)
    api_key = data.get("apiKey")
    logger = data.get("logger")

    logger.info("Plugin session is started!")

    sdk = SDK(api_key=api_key, host=HOST)
    integration = sdk.get_integrations(integration_id)

    region = integration.get("region")

    client = boto3.client('s3', region_name=region, aws_access_key_id=integration.get("publicKey"),
                          aws_secret_access_key=integration.get("privateKey"))
    s3paginator = client.get_paginator('list_objects')

    path = folder

    if upload:
        page_iterator = s3paginator.paginate(Bucket=bucket,
                                             Prefix=path,
                                             Delimiter='/',
                                             PaginationConfig={'PageSize': 1000})
        files_to_upload = []
        if files:
            for key in files:
                url = "https://%s.s3.%s.amazonaws.com/%s" % (bucket, region, key)
                files_to_upload.append({"data": url, "externalId": key})
        else:
            for page in page_iterator:
                for content in page.get("Contents", []):
                    key = content["Key"]
                    if key[-1] != "/":
                        url = "https://%s.s3.%s.amazonaws.com/%s" % (bucket, region, key)
                        files_to_upload.append({"data": url, "externalId": key})
        response = sdk.upload_files_cloud(project_id=project_id, assets=files_to_upload, integration_id=integration_id)
        logger.info("Plugin session is ended!")
        if response.get("status", "") == "success":
            return {"status": "success"}
        else:
            return {"status": "fail", "error": response}

    else:
        page_iterator = s3paginator.paginate(Bucket=bucket,
                                             Prefix=path,
                                             Delimiter='/',
                                             PaginationConfig={'PageSize': PAGE_SIZE, 'StartingToken': scroll_token})

        folders = []
        files = []
        page_iter = iter(page_iterator)
        page = next(page_iter)
        new_scroll_token = None
        files_page = page.get("Contents", [])
        folders_page = page.get("CommonPrefixes", [])
        if files_page is not None:
            for content in files_page:
                key = content["Key"]
                if key[-1] != "/":
                    files.append(key)
            if len(files_page) == PAGE_SIZE:
                new_scroll_token = page.get("Contents", [{}])[-1].get("Key", None)

        if folders_page is not None:
            for content in folders_page:
                key = content["Prefix"]
                folders.append(key)

        return {"folders": folders, "files": files, "scrollToken": new_scroll_token, "success": True}


if __name__ == "__main__":
    plugin = FileExplorerPlugin(id=PLUGIN_ID,
                                secret=PLUGIN_SECRET,
                                callback=sample_callback)
    run(plugin, host=HOST)

plugin-samples/file_explorer_plugin.py at main · angoai/plugin-samplesGitHub

Running the File Explorer

If you have added this plugin from the Directory, from the dashboard of the project where you'd like to run the script, enter the Settings tab and then the Plugin section. Then click on Open on the File Explorer plugin you need. A dialog will pop up.

If you have created this plugin yourself, from the Development section of the Plugins page, click on Open on the File Explorer plugin you need. A dialog will pop up.

Project: The project in which you wish to import the assets.

Bucket Name: The name of the bucket you have integrated with Hub, which contains the assets to be imported. Ensure this is typed correctly.

Integration: The integration you set up in the previous step.

Folder: Once all other fields are filled, this button will activate. By clicking it, a dialog will pop up. From the dialog, you will be able to select a folder from your bucket, and by clicking on "Import" you will instantly add all valid assets in that folder to the selected project on Hub.