Recently, I was tasked with coming up solution for getting financial data from one of our line of business applications that is a Software as a service (SAAS) offering over to one of our business partners. TLDR we decided on a Logic App.😎
We did not want to go down the route of having the traditional virtual machine and all the traditional patching and security concerns that come with creating this kind of resource.
Our platform of choice is Microsoft Azure so I started researching some of the technologies that were available to us, finally I limited it down to 4 options;
- Azure Containers – utilising this prebuilt template
- Azure Virtual Machines – As discussed, if possible we wanted to avoid this route but was our fallback option
- Azure Data Factory – never used but some of the prebuilt solutions looked interesting – more information available here
- Azure Logic Apps – used a lot of Microsoft Flow (power automate) so this one stood out and the early favourite
After doing some initial research we had another discussion to find out the key requirements so that we could narrow down which route to take; below is the key requirements;
- SFTP Connectivity with SSH Keys
- Ability to Schedule Files transfer
- Lockdown via IP Ranges
Logic Apps meet all the criteria above and the team was familiar with most of the concepts inside the service, so we started planning out what solution would look like.
Also when looking at Logic App vs Virtual Machine and the backup infrastructure costs. Logic App wins hands down.
Azure Blob Container, is being used to store the data that comes out of the line of business application, SAS keys will be used.
How to Create Push Logic App
- Azure Subscription (works with Azure Credits)
- SSH Key and details to connect to an SFTP Server
- Azure Blob Storage, a great walkthrough here
Resource Group Creation
The first step we need to achieve was to create the resource group, every resource in Azure must be within a Resource Group.
To create the resource group into the Azure Portal search Resource Groups in the top bar
Then Click the ➕ Add button in the top left, this will take you through the resource group wizard
I prefer to login to Azure via Powershell and using the following commands to login and create a storage group;
Connect-AzAccount<br><br>New-AzureRmResourceGroup -Name SFTP-RG -Location "UK South"
Logic App Creation
Now that we have a Resource Group created we can add the relevant resources, press the ➕ whilst in the resources group and search for Logic App and add then create.
After selecting the resources you will be shown the basics creation wizard, fill out the Logic App name and location as needed.
Once the Logic App has been deployed, select the resource, in logic app designer and scroll down to the templates section and select the blank template. You will then be taken into the Logic App Designer.
All Logic Apps need a trigger, for our requirements the trigger will be a Schedule, most of the time this will be a one of the first in the designer – if not you can use the search 🔍
Schedule and List Blob Contents
While testing I recommend you set the schedule to ever 2-5 minutes, or shorter if you don’t want to be waiting around. When you are ready to go live, you change the frequency and press the add new parameter to set up custom schedules.
Moving on, we need to add another step, this step involves connecting to an Azure Blob (our storage of choice) but you can use OneDrive, SharePoint etc.
We add a step called list blob to do this you search for blob and Azure Blob Storage should be present, scroll down the list and you’ll find list blob.
N.B. we are using the above action to cover if there are multiple files on the Azure Blob, this won’t be needed if your sure there will only ever be one file on your Azure blob.
Once you select list blob you will be asked to select a name for your connection and connect to the correct storage blob and container.
I’ve now gone thorugh creating the reosurce and connecting up to the storage account and hopefully, you now have some thing that looks like the image below.
Now that we have the connection to the blob and we will use a for each control loop and the SFTP Connector to push the files to the SFTP Server with an SSH Key.
For the sake of timesaving and the length of this blog post (a lot longer than I wanted 😎), Microsoft has an excellent article on creating an SSH key here.
For each Control Loop
The for each loop will allow us to loop over the content of the Azure Blob and send/create each file on the SFTP Server.
To add a for each loop, press the new step and search for control, you can then select for each and you will get an empty grey box where we can start adding the additional actions.
The next 3 steps will involve getting the blob content id from the list blobs and creating a file on the SFTP server and deleting the content off the Azure Blob once complete.
The deletion is optional you may prefer to just archive it which can be done we a copy/delete steps to a different container.
The images below outline the remaining steps of the for each loop, basically following the same process of adding actions and selecting the right dynamic content as seen below.
So that’s the Push logic app created, to test/debug the application, at top of the logic designer their is a run button which will go through all the steps and if successful you should get a ✔ next to each step, as seen below.
To create the pull Logic App I was able to save time by using the clone option, to find the clone option browse to your Logic App and on the top toolbar press Clone.
SFTP Connection Settings
As mentioned earlier in the article SSH key was one of the requirements, but not needed if you just want to use username and password, for completeness see the screenshot below of the SSH properties that you can define on SFTP Connector.
This was my first venture down using serverless and Logic Apps and applying some of the knowledge I picked up from recently passing my Azure 900 Exam, to solve a business issue and I must say I really enjoyed learning how to do this, and as an added bonus saved money and time involved in patching and maintaining additional servers.
It did take me longer to achieve this, than creating an Azure Virtual Machine, but now I have a repeatable template that I can reuse in the future when I get the opportunity to use this technology again to solve business issues.