Logic APP – Azure Serverless SFTP

Recently, I was tasked with coming up solution for getting financial data from one of our line of business applications that is a Software as a service (SAAS) offering over to one of our business partners. TLDR we decided on a Logic App.😎

We did not want to go down the route of having the traditional virtual machine and all the traditional patching and security concerns that come with creating this kind of resource.

Initial Findings

Our platform of choice is Microsoft Azure so I started researching some of the technologies that were available to us, finally I limited it down to 4 options;

  • Azure Containers – utilising this prebuilt template
  • Azure Virtual Machines – As discussed, if possible we wanted to avoid this route but was our fallback option
  • Azure Data Factory – never used but some of the prebuilt solutions looked interesting – more information available here
  • Azure Logic Apps – used a lot of Microsoft Flow (power automate) so this one stood out and the early favourite

Requirements

After doing some initial research we had another discussion to find out the key requirements so that we could narrow down which route to take; below is the key requirements;

  1. SFTP Connectivity with SSH Keys
  2. Ability to Schedule Files transfer
  3. Lockdown via IP Ranges

Solution

Logic Apps meet all the criteria above and the team was familiar with most of the concepts inside the service, so we started planning out what solution would look like.

Also when looking at Logic App vs Virtual Machine and the backup infrastructure costs. Logic App wins hands down.

Overview Diagram

Overview diagram of Azure Logic App for SFTP

Azure Blob Container, is being used to store the data that comes out of the line of business application, SAS keys will be used.

How to Create Push Logic App

Prerequisites

  • Azure Subscription (works with Azure Credits)
  • SSH Key and details to connect to an SFTP Server
  • Azure Blob Storage, a great walkthrough here

Resource Group Creation

The first step we need to achieve was to create the resource group, every resource in Azure must be within a Resource Group.

To create the resource group into the Azure Portal search Resource Groups in the top bar

Then Click the ➕ Add button in the top left, this will take you through the resource group wizard

I prefer to login to Azure via Powershell and using the following commands to login and create a storage group;

Logic App Creation

Now that we have a Resource Group created we can add the relevant resources, press the ➕ whilst in the resources group and search for Logic App and add then create.

After selecting the resources you will be shown the basics creation wizard, fill out the Logic App name and location as needed.

Create Resource Group

Once the Logic App has been deployed, select the resource, in logic app designer and scroll down to the templates section and select the blank template. You will then be taken into the Logic App Designer.

Blank Logic App

All Logic Apps need a trigger, for our requirements the trigger will be a Schedule, most of the time this will be a one of the first in the designer – if not you can use the search 🔍

Schedule and List Blob Contents

Searching for Schedule Step

While testing I recommend you set the schedule to ever 2-5 minutes, or shorter if you don’t want to be waiting around. When you are ready to go live, you change the frequency and press the add new parameter to set up custom schedules.

Schedule Step

Moving on, we need to add another step, this step involves connecting to an Azure Blob (our storage of choice) but you can use OneDrive, SharePoint etc.

We add a step called list blob to do this you search for blob and Azure Blob Storage should be present, scroll down the list and you’ll find list blob.

Adding List Step

N.B. we are using the above action to cover if there are multiple files on the Azure Blob, this won’t be needed if your sure there will only ever be one file on your Azure blob.

Once you select list blob you will be asked to select a name for your connection and connect to the correct storage blob and container.

Connect to storage

I’ve now gone thorugh creating the reosurce and connecting up to the storage account and hopefully, you now have some thing that looks like the image below.

List Blob Content

Now that we have the connection to the blob and we will use a for each control loop and the SFTP Connector to push the files to the SFTP Server with an SSH Key.

For the sake of timesaving and the length of this blog post (a lot longer than I wanted 😎), Microsoft has an excellent article on creating an SSH key here.

For each Control Loop

The for each loop will allow us to loop over the content of the Azure Blob and send/create each file on the SFTP Server.

To add a for each loop, press the new step and search for control, you can then select for each and you will get an empty grey box where we can start adding the additional actions.

foreach control step

The next 3 steps will involve getting the blob content id from the list blobs and creating a file on the SFTP server and deleting the content off the Azure Blob once complete.

The deletion is optional you may prefer to just archive it which can be done we a copy/delete steps to a different container.

The images below outline the remaining steps of the for each loop, basically following the same process of adding actions and selecting the right dynamic content as seen below.

For each and Send SFTP

So that’s the Push logic app created, to test/debug the application, at top of the logic designer their is a run button which will go through all the steps and if successful you should get a ✔ next to each step, as seen below.

To create the pull Logic App I was able to save time by using the clone option, to find the clone option browse to your Logic App and on the top toolbar press Clone.

Clone Logic App

SFTP Connection Settings

As mentioned earlier in the article SSH key was one of the requirements, but not needed if you just want to use username and password, for completeness see the screenshot below of the SSH properties that you can define on SFTP Connector.

SSH Connection Settings for logic app

Conclusion

This was my first venture down using serverless and Logic Apps and applying some of the knowledge I picked up from recently passing my Azure 900 Exam, to solve a business issue and I must say I really enjoyed learning how to do this, and as an added bonus saved money and time involved in patching and maintaining additional servers.

It did take me longer to achieve this, than creating an Azure Virtual Machine, but now I have a repeatable template that I can reuse in the future when I get the opportunity to use this technology again to solve business issues.