Extracting data from API

 

APIs

Instead of downloading World Bank data via a csv file, you're going to download the data using the World Bank APIs. The purpose of this exercise is to gain experience with another way of extracting data.

API is an acronym that stands for application programming interface. API’s provide a standardized way for two applications to talk to each other. In this case, the applications communicating with each other are the server application where World Bank stores data and your Jupyter notebook.

If you wanted to pull data directly from the World Bank’s server, you’d have to know what database system the World Bank was using. You’d also need permission to log in directly to the server, which would be a security risk for the World Bank. And if the World Bank ever migrated its data to a new system, you would have to rewrite all of your code again. The API allows you to execute code on the World Bank server without getting direct access.


Using APIs

In general, you access APIs via the web using a web address. Within the web address, you specify the data that you want. To know how to format the web address, you need to read an API's documentation. Some APIs also require that you send login credentials as part of your request. The World Bank APIs are public and do not require login credentials.

The Python requests library makes working with APIs relatively simple.


Example Indicators API

Run the code example below to request data from the World Bank Indicators API. According to the documntation, you format your request url like so:

http://api.worldbank.org/v2/countries/ + list of country abbreviations separated by ; + /indicators/ + indicator name + ? + options

where options can include

  • per_page - number of records to return per page
  • page - which page to return - eg if there are 5000 records and 100 records per page
  • date - filter by dates

  • format - json or xml

    and a few other options that you can read about here.

Comments

Post a Comment

Popular posts from this blog

Read and Navigate XML - Beautiful Soup

 ### How to read and navigate XML There is a Python library called BeautifulSoup, which makes reading in and parsing XML data easier. Here is the link to the documentation: [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/) The find() method will find the first place where an xml element occurs. For example using find('record') will return the first record in the xml file: ```xml <record>   <field name="Country or Area" key="ABW">Aruba</field>   <field name="Item" key="SP.POP.TOTL">Population, total</field>   <field name="Year">1960</field>   <field name="Value">54211</field> </record> ``` The find_all() method returns all of the matching tags. So find_all('record') would return all of the elements with the `<record>` tag. Run the code cells below to get a basic idea of how to navigate XML with BeautifulSoup. To naviga...
Read more

difference-between-stream-processing-and-message-processing

 difference-between-stream-processing-and-message-processing Message Processing implies operations on and/or using  individual  messages. Stream Processing encompasses operations on and/or using individual messages as well as operations on collection of messages as they flow into the system. For e.g., let's say transactions are coming in for a payment instrument - stream processing can be used to continuously compute hourly average spend. In this case - a sliding window can be imposed on the stream which picks up messages within the hour and computes average on the amount. Such figures can then be used as inputs to fraud detection systems Although Rabbit supports streaming, it was actually not built for it(see Rabbit´s web site) Rabbit is a Message broker and Kafka is a event streaming platform. Kafka can handle a huge number of 'messages' towards Rabbit. Kafka is a log while Rabbit is a queue which means that if once consumed, Rabbit´s messages are not there anymore in c...
Read more

WordNet in Python

Image
  WordNet   is an English dictionary which is a part of   Natural Language Tool Kit   (NLTK) for Python. This is an extensive library built to make   Natural Language Processing   (NLP) easy. Some basic functions will be discussed in this article. To start using WordNet, you have to import it first: from  nltk.corpus  import  wordnet Synsets and Lemmas In WordNet, similar words are grouped into a set known as a  Synset  (short for  Synonym-set ). Every Synset has a name, a part-of-speech, and a number. The words in a Synset are known as  Lemmas . Getting Synsets The function  wordnet.synsets('word')   returns an array  containing all the Synsets related to the word passed to it as the argument. Example: print ( wordnet.sysnets ( 'room' )) Output: [Synset(‘room.n.01’), Synset(‘room.n.02’), Synset(‘room.n.03’), Synset(‘room.n.04’), Synset(‘board.v.02’)] The method returned five Synsets; four have the name...
Read more