Read a CSV Blob file in Azure

Introduction

Azure Blob storage is a service for storing large amounts of unstructured data. In this article we will look how we can read csv blob.

Step 1: Create a Source Blob Container in the Azure Portal

Open the Azure portal(https://portal.azure.com), and then choose Storage Account say Containers

Create a source container say sourcecontainer

It will appear as

Step 2: Upload a file in the Source Container

Create an csv file (say test.csv) whose content is as under

Now let us upload the file in sourcecontainer.

Step 3: Read CSV blob file programatically

Fire up a console application and add the below Nuget packages

Install-Package WindowsAzure.Storage

Install-Package Microsoft.WindowsAzure.ConfigurationManager

Next write the below function

/// <summary>
/// GetCSVBlobData
/// Gets the CSV file Blob data and returns a string
/// </summary>
/// <param name="filename"></param>
/// <param name="connectionString"></param>
/// <param name="containerName"></param>
/// <returns></returns>
private static string GetCSVBlobData(string filename, string connectionString, string containerName)
{
    // Retrieve storage account from connection string.
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);

    // Create the blob client.
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

    // Retrieve reference to a previously created container.
    CloudBlobContainer container = blobClient.GetContainerReference(containerName);

    // Retrieve reference to a blob named "test.csv"
    CloudBlockBlob blockBlobReference = container.GetBlockBlobReference(filename);

    string text;
    using (var memoryStream = new MemoryStream())
    {
        //downloads blob's content to a stream
        blockBlobReference.DownloadToStream(memoryStream);

        //puts the byte arrays to a string
        text = System.Text.Encoding.UTF8.GetString(memoryStream.ToArray());
    }

    return text;
}

The above function gets the blob's reference and using the DownloadToStream function, it downloads the blob's content to a memory stream. This stream of data is then converted to string by the System.Text.Encoding.UTF8.GetString function.

Now from inside the main function, let us write the below code

static void Main(string[] args)
{           
   string connectionString = CloudConfigurationManager.GetSetting("StorageConnectionString"); //blob connection string
   string sourceContainerName = ConfigurationManager.AppSettings["sourcecontainerName"]; //source blob container name           
   string sourceBlobFileName = "test.csv"; //source blob name
   var csvData = GetCSVBlobData(sourceBlobFileName, connectionString, sourceContainerName);            
}

The program invokes the GetCSVBlobData function to read the csv blob content and returns a string. The Text Visualizer reveals.

The app.config file looks as under

  <appSettings>    
    <add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=testaccount;AccountKey=CRd****==;EndpointSuffix=core.windows.net" />   
    <add key="sourcecontainerName" value="sourcecontainer" />   
  </appSettings>

The complete program is as under

using Microsoft.Azure;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System.Configuration;
using System.IO;

namespace UploadFilesToBlob
{
    class Program
    {
        static void Main(string[] args)
        {           
            string connectionString = CloudConfigurationManager.GetSetting("StorageConnectionString"); //blob connection string
            string sourceContainerName = ConfigurationManager.AppSettings["sourcecontainerName"]; //source blob container name            
            string sourceBlobFileName = "test.csv"; //source blob name

            var csvData = GetCSVBlobData(sourceBlobFileName, connectionString, sourceContainerName);            
        }

        /// <summary>
        /// GetCSVBlobData
        /// Gets the CSV file Blob data and returns a string
        /// </summary>
        /// <param name="filename"></param>
        /// <param name="connectionString"></param>
        /// <param name="containerName"></param>
        /// <returns></returns>
        private static string GetCSVBlobData(string filename, string connectionString, string containerName)
        {
            // Retrieve storage account from connection string.
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);

            // Create the blob client.
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

            // Retrieve reference to a previously created container.
            CloudBlobContainer container = blobClient.GetContainerReference(containerName);

            // Retrieve reference to a blob named "test.csv"
            CloudBlockBlob blockBlobReference = container.GetBlockBlobReference(filename);

            string text;
            using (var memoryStream = new MemoryStream())
            {
                //downloads blob's content to a stream
                blockBlobReference.DownloadToStream(memoryStream);

                //puts the byte arrays to a string
                text = System.Text.Encoding.UTF8.GetString(memoryStream.ToArray());
            }

            return text;
        }             
    }
}

Conclusion

In this article we looked into how we can read a CSV blob file. Hope this will be helpful. Thanks for reading. Zipped file attached.