Azure Data Factory Web API activity

Question

We are using Azure data factory to get weather data from one of the API. We have to implement multiple activities first we need a table which holds all the latitude and longitude data then we need to build a Azure pipeline to loop through the locations ( coordinates ) and call the API to get the weather information. Below are steps which I performed.

1. I am using look up activity to get the data from the table.
2. Then I am using Foreach activity to loop through the latitude and longitude.
3. I am using set variable activity where I am using one variable for latitude and one for longitude.
4. I am using web activity to pass the URL and GET response from the API.

I am able to achieve most of the output but the issue where I am stuck is the output URL is not fetching any data because the for some part of my URL the hyperlink which is blue color is removed and is not able to read. Below are the screen grabs .

1. Two activities Lookup and foreach with four variables declared

   

2. My source dataset is a SQL table having latitude and longitude which i want.

   

3. This activity shows two set variable and web and copy activity.

   

4. I have added Name and Value of variable which will be passed onto the web URL

   

5. Below wen activity where I passed the weather API url

   

6. Dynamic content where I am passing the URL, key and latitude, longitude variable, format, no of days.

   

7. Final output where I am facing issue if you see the the hyperlink is removed for the URL after latitude value i.e. 41.4 so after that nothing is being read and data is not coming in JSON. How to fix this and how we can pass to variables in a URL because in my case the latitude and longitude is separated by a comma as a separator, and if I try to add comma it is not reading the URL

   

This is the final URL which I am trying to achieve through Azure Data Factory.

http://api.worldweatheronline.com/premium/v1/weather.ashx?key=*****************&q=41.14,80.68&format=JSON&num_of_days=5

question from:https://stackoverflow.com/questions/65646141/azure-data-factory-web-api-activity

Answer 1

What is the output of the query . Select lat ,long from Weather_location ? I think the long value is -80 in there and so you are having the issue . This is what I tried .

Created two variable Long and lat and set the value which you shared . Used the below expression .

@concat('http://api.worldweatheronline.com/premium/v1/weather.ashx?key=XXXXXX&q=',variables('lat')
,',',variables('long'),'&format=JSON&num_of_days=5')

It just works fine .

{
    "data": {
        "request": [
            {
                "type": "LatLon",
                "query": "Lat 41.14 and Lon 80.68"
            }
        ],
        "current_condition": [
            {
                "observation_time": "11:3

.....

You are right when you said about the hyperlink , it does not show as complete ( see the screen shot below ) but it still works for me .

Input { "url": "http://api.worldweatheronline.com/premium/v1/weather.ashx?key=XXXXXX=41.14,80.68&format=JSON&num_of_days=5", "method": "GET", "headers": {} }

Edit on 1/15

As I undersand the intend is to use the API and copy the response JSON to a ADLE GEN 2 .

This is what I did when i tried . Used the a copy activity select SINK as REST , base URL in the linked service was http://api.worldweatheronline.com.

Create a new Variable : relativeurl :

@concat('/premium/v1/weather.ashx?key=yourkey&q=',variables('lat')
,',',variables('long'),'&format=JSON&num_of_days=5')

We will use this as a parameter in the SOURCE relative url .

GO to Copy Activity ( CA ) -> Source DS -> Open -> Parameter -> relativeurl

GO to Copy Activity ( CA ) -> Source -> you should see relativeurl ->@variables('relativeurl')

GO to Copy Activity ( CA ) -> Source DS -> Open ->Relative URL -@dataset().relativeurl

HTH