> ## Documentation Index
> Fetch the complete documentation index at: https://docs.skypoint.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Azure Data Lake Storage Gen2

> Connect Azure Data Lake Storage Gen2 to import data into Skypoint AI

<Tabs>
  <Tab title="Import data">
    ## Overview

    You can make Azure Storage the foundation for building enterprise data lakes on Azure with the help of [Azure Data Lake Storage Gen2](https://azure.microsoft.com/en-in/pricing/details/storage/data-lake/). You can use SkyPoint AI's built-in connector for importing data from Azure Data Lake Storage Gen2.

    This document will guide you through the process of connecting Azure Data Lake Storage Gen2 to Skypoint AI.

    ## Prerequisite

    You will need the following details to configure and import data using Azure Data Lake Storage Gen2 connector:

    * Storage account name
    * Account key details
    * Storage path.

    ## Import data using Azure Data Lake Storage Gen2 connector

    Follow the below steps to create a new dataflow for the Azure Data Lake Storage Gen2 import connector:

    1. Go to **Dataflow > Imports**.
    2. Click **New dataflow**.

    The **Set dataflow name** page appears.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Setdataflowname.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=53fd9f811af0ec666fb63872033fbf5a" alt="Set dataflow name" style={{border: "1px solid black"}} width="729" height="356" data-path="doc_snippets/ADLSGen2_Setdataflowname.png" />

    3. In the **Set dataflow name** page, type Dataflow name in the **Name** text area.
    4. Click **Next**.

    The **Choose connector** page appears.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Chooseconnector.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=1294e04f97e88ec17d2360d280419b8c" alt="Choose connector" style={{border: "1px solid black"}} width="1286" height="605" data-path="doc_snippets/ADLSGen2_Chooseconnector.png" />

    <Note>
      1. Azure Data Lake Storage Gen2: Use this to import each CSV file as an entity.
      2. Azure Data Lake Storage Gen2 v2: Use this to import multiple CSV files in a folder as a single entity.
    </Note>

    ## To choose Azure Data Lake Storage Gen2 connector

    1. In the **Choose connector** page, select **Azure Data Lake Storage Gen2** connector.

    <Note>
      You can use the Search feature to find the connector. Also, the Azure Data Lake Storage Gen2 connector is available under both Analytics and Cloud categories.
    </Note>

    The **Set dataflow name** page appears.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Setdataflowdisplayname.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=5920ea503eb9d75b1026d5ecd72257af" alt="Set dataflow display name" style={{border: "1px solid black"}} width="465" height="557" data-path="doc_snippets/ADLSGen2_Setdataflowdisplayname.png" />

    2. Type a **Display Name** for your dataflow in the text area.
    3. Type a **Description** for your dataflow in the text area.
    4. Click **Next**.

    The **Configuration** page appears.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_ADLSConfiguration.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=81f15c3e2dc284d40a638dc858953503" alt="ADLS Configuration" style={{border: "1px solid black"}} width="724" height="487" data-path="doc_snippets/ADLSGen2_ADLSConfiguration.png" />

    ## To configure Azure Data Lake Storage Gen2

    Follow the below steps to configure the connection to **Azure Data Lake Storage Gen2**:

    1. Type the **Storage account name** in the text area.
    2. Type the **Account key** in the text area.
    3. Click the **Folder icon** in the **Storage path** text area.

    Once you select your Storage path, the **Table Details** columns appear.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Configurationtabledetails.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=91950e1daba16fa543c65eaa132070e3" alt="Configuration table details" style={{border: "1px solid black"}} width="1347" height="616" data-path="doc_snippets/ADLSGen2_Configurationtabledetails.png" />

    <Note>
      Upon selecting Azure Data Lake Storage Gen2 v2, **Table Details** appear as per the image below.
    </Note>

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_v2_Configurationtabledetails.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=31cc2049bb569c2e5ce0b603c4d4ecca" alt="v2 Configuration table details" style={{border: "1px solid black"}} width="1564" height="491" data-path="doc_snippets/ADLSGen2_v2_Configurationtabledetails.png" />

    4. Enter the **Table Details** to process the data.

    | Item                | Description                                                                                                                    |
    | :------------------ | :----------------------------------------------------------------------------------------------------------------------------- |
    | Purpose             | Option to assign a purpose (Data or Metadata) for each table.                                                                  |
    | Data                | Loads customer data.                                                                                                           |
    | Metadata            | Loads Metadata.                                                                                                                |
    | File Name           | Displays the name of the file that you imported.                                                                               |
    | Folder Name         | Displays the name of the folder that are present in the selected folder. (Applicable only for Azure Data Lake Storage Gen2 v2) |
    | Table Name          | Displays the imported table name.                                                                                              |
    | Datetime format     | Displays a number of **Datetime Formats** and Skypoint AI is set to automatically detect them.                                 |
    | Delimiter           | Displays available separators for the variables in the imported data.                                                          |
    | First Row as Header | Check the box for the system to automatically collect the data according to the Header Contents.                               |
    | Advanced Settings   | Select the options to fine tune the Import process with minute details.                                                        |

    5. If necessary, apply the **Advance settings** to modify the default settings.

    The **Advanced settings** pop-up appears.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Advancesettings.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=f387de6302b4d7053ff4e2168df4e602" alt="Advanced settings" style={{border: "1px solid black"}} width="320" height="368" data-path="doc_snippets/ADLSGen2_Advancesettings.png" />

    <Note>
      Advanced settings allow you to modify the default settings. It gives more flexibility to apply advanced use cases. However, the default settings are adequate to perform the task.
    </Note>

    | If you want to                                                                 | Then                                                                                                           |
    | :----------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------- |
    | Modify data types such as fixed or variable data types.                        | Select from the **Compression type**. It allows you to reduce the size of data by removing the number of bits. |
    | Change the delimiter                                                           | Click **Row delimiter**. By default, a column delimiter is selected, and each row is separated with a comma.   |
    | Change information or instruction                                              | Choose from the **Encoding** list. By default, UTF-8 encoding is selected.                                     |
    | Modify the escape character such as backslash (\\) or slash (/)                | Select from the **Escape character**.                                                                          |
    | Apply different quote characters such as Single quote (') or Double quote ("). | Select from the **Quote character**.                                                                           |

    6. Click **Save**.

    ## Run, edit, and delete the imported data

    Once you save the connector, the Azure Data Lake Storage Gen2 connector gets displayed in the list of tables created in the Dataflow page.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Output.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=81651c4aebf935417a51989222d727b1" alt="Output" style={{border: "1px solid black"}} width="1652" height="495" data-path="doc_snippets/ADLSGen2_Output.png" />

    | Item              | Description                                                                                                         |
    | :---------------- | :------------------------------------------------------------------------------------------------------------------ |
    | Name              | Displays the name of the imported Dataflow.                                                                         |
    | Type              | Displays connector type symbol.                                                                                     |
    | Status            | Indicates whether the data is imported successfully.                                                                |
    | Tables Count      | Displays the number of tables.                                                                                      |
    | Created Date      | Displays date of creation.                                                                                          |
    | Last refresh type | Displays the refresh value. After the last data refresh, it will indicate whether the value is Full or Incremental. |
    | Updated Date      | Displays last modified date.                                                                                        |
    | Last Refresh      | Displays the latest refresh date, which updates each time you refresh the data.                                     |
    | Group by          | Option to view the items in a specific Group (For example, name, type, status).                                     |

    * Select the horizontal ellipsis in the **Actions** column and do the following:

    | If you want to                       | Then                                                                                               |
    | :----------------------------------- | :------------------------------------------------------------------------------------------------- |
    | Modify the Dataflow                  | Select **Edit** and modify the Dataflow. Click **Save** to apply your changes.                     |
    | Execute the Dataflow                 | Select **Run**.                                                                                    |
    | Bring the data to its previous state | Select **Rollback**.                                                                               |
    | Delete the Dataflow                  | Select **Remove** and then click the **Delete** button. All tables in the data source get deleted. |
    | See the run history of the Dataflow  | Select **Run** history.                                                                            |
  </Tab>

  <Tab title="Setup guide">
    ## Overview

    This document will help you gather all credentials for connecting Azure Data Lake Storage Gen2 with Skypoint AI.

    ## Prerequisite

    You must have an [Azure](https://azure.microsoft.com/en-us/free/) account.

    ## To Create a Storage Account

    Follow below steps to create a storage account:

    1. Log in to **Microsoft Azure**.
    2. Go to **Azure services > Storage accounts**.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Azurehomepage.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=8f019f89b5d3a4f75567afca8f12d75f" alt="Azure homepage" style={{border: "1px solid black"}} width="936" height="247" data-path="doc_snippets/ADLSGen2_Azurehomepage.png" />

    3. Click **Create**.

    ### Creating Storage account name

    1. Go to **Create a storage account > Basics**.
    2. Fill the information to create a new **Storage account**.
       * Type a desired **Storage account name** in the text area.

    <Note>
      The name must be unique among all existing storage account names in Azure. It must be between 3 to 24 characters long and can contain only lowercase letters and numbers.
    </Note>

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Createstorageaccount.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=79001823ed4f17525c302705f0e81a58" alt="Create storage account" style={{border: "1px solid black"}} width="797" height="688" data-path="doc_snippets/ADLSGen2_Createstorageaccount.png" />

    * In **Advanced** tab, check **Enable hierarchical namespace** under **Data Lake Storage Gen2** section to create your ADLS Gen2 storage account.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_enablenamespace.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=c3322fb9056d87704090e27decb53a7d" alt="Enable namespace" style={{border: "1px solid black"}} width="642" height="745" data-path="doc_snippets/ADLSGen2_enablenamespace.png" />

    * Fill all text areas under each tab to configure and create your storage account.

    ### Finding the Account Key

    Follow below steps to find the account key for your storage account:

    1. Go to **Storage account > Access Key**.

    <img src="https://mintcdn.com/skypoint/867s-f5L6ea4TG6I/doc_snippets/ADLSGen2_Accesskeys.png?fit=max&auto=format&n=867s-f5L6ea4TG6I&q=85&s=2d8b9a9314807f72545216b9a83c3034" alt="Access keys" style={{border: "1px solid black"}} width="809" height="389" data-path="doc_snippets/ADLSGen2_Accesskeys.png" />

    2. Click **Show** for the **Key** text area.

    <Note>
      Copy this key and paste it into the Account Key text area of your Skypoint AI's Azure Data Lake Storage Gen2 configuration form.
    </Note>
  </Tab>
</Tabs>

## Next step

After completing the data import, start the [Master Data Management (MDM) - Stitch](stitch.md) process to develop a unified view of your customers.