python - Read a single column of a CSV for API Call

Question

How can I do such a task in python? I got a CSV file with a bunch of parameters, let's say "col" and "para". So, I would like to loop all the parameters in "para" column as an array and get every response data back and write into CSV file?

col	para	return
a	e
b	f
c	g

question from:https://stackoverflow.com/questions/65933349/read-a-single-column-of-a-csv-for-api-call

Answer 1

What you need is the apply() function for DataFrame. This function can do the same operation for each element of one column.

How to use apply function in pandas

Here is the example for your case:

import pandas as pd
import requests

data = pd.read_csv("input_file.csv") 
url = "http://api.com/query?name={}"

data['return'] = data['para'].apply(lambda x:requests.get(url.format(x)).text)

data.to_csv("output_file.csv",index=False)

For more process, you can define a function to replace lambda function above:

import pandas as pd
import requests
import json

def my_process(para):
    url = "http://api.com/query?name={}".format(para)
    try:
        return_json = json.loads(requests.get(url).text)
        response = return_json['data'][0]['response'] # get the first response string
    except:
        response = ""  # set a value for the case that the API call has no correct results
    return response

data = pd.read_csv("input_file.csv") 
data['return'] = data['para'].apply(my_process)

data.to_csv("output_file.csv",index=False)

<ipython-input-29-77b593e201eb> in <module>
     11 
     12 data = pd.read_csv("input.csv")
---> 13 data['return'] = data['para'].apply(my_process)
     14 
     15 data.to_csv("output_file.csv",index=False)

~Anaconda3libsite-packagespandascoreseries.py in apply(self, func, convert_dtype, args, **kwds)
   3846             else:
   3847                 values = self.astype(object).values
-> 3848                 mapped = lib.map_infer(values, f, convert=convert_dtype)
   3849 
   3850         if len(mapped) and isinstance(mapped[0], Series):

pandas\_libslib.pyx in pandas._libs.lib.map_infer()

<ipython-input-29-77b593e201eb> in my_process(para)
      6     url = "http://api.com/query?name={}".format(para)
      7     return_json = json.loads(requests.get(url).text)
----> 8     response = return_json['data'][0]['response '] # get the first response string
      9     return response
     10