Posts

Showing posts from March, 2022

PATH variable

Image
  The  PATH variable  is a list of directories where each directory contains the executable file for a command. When a command is entered into the Windows command prompt, the prompt searches in the PATH variable for an executable file with the same name as the command; i​n the case that the required file is not found, it responds with an error message that states that​ the specified command was not recognized. One way to overcome this error is to write the complete directory of the executable file  instead  of only entering the command name. However, this is not a very user-friendly approach. Another possible, and easier, way to avoid this error is to add the executable file’s directory to the PATH variable. Oftentimes, this needs to be done when installing Python. One way to overcome this error is to write the complete directory of the executable file  instead  of only entering the command name. However, this is not a very user-friendly approach. Ano...

Faster look up in list or dictionaries

 https://towardsdatascience.com/faster-lookups-in-python-1d7503e9cd38

Extracting data from API

  APIs ¶ Instead of downloading World Bank data via a csv file, you're going to download the data using the World Bank APIs. The purpose of this exercise is to gain experience with another way of extracting data. API is an acronym that stands for application programming interface. API’s provide a standardized way for two applications to talk to each other. In this case, the applications communicating with each other are the server application where World Bank stores data and your Jupyter notebook. If you wanted to pull data directly from the World Bank’s server, you’d have to know what database system the World Bank was using. You’d also need permission to log in directly to the server, which would be a security risk for the World Bank. And if the World Bank ever migrated its data to a new system, you would have to rewrite all of your code again. The API allows you to execute code on the World Bank server without getting direct access. Using APIs In general, you access APIs via the...

What is database storage engine

 https://israelg99.github.io/2017-06-07-Introduction-to-Database-Storage-Engines/

Pandas to open a SQL database or to run a SQL query against a database

  There is more than one way to do this depending on the type of SQL database you are working with: the   sqlite3 library   or the   sqlalchemy library . In the same folder as this Jupyter notebook, there is a SQLite database file called "population_data.db". SQLite is a database engine meant for single applications. The entire database is contained in one file. You can read more about SQLite  here . Sqlite is a database storage engine, which can be better compared with things such as MySQL, PostgreSQL, Oracle, MSSQL, etc. It is used to store and retrieve structured data from files. SQLAlchemy is a Python library that provides an object relational mapper (ORM). It does what it suggests: it maps your databases (tables, etc.) to Python objects, so that you can more easily and natively interact with them. SQLAlchemy can be used with sqlite, MySQL, PostgreSQL, etc. So, an ORM provides a set of tools that let you interact with your database models consistently across...

Read and Navigate XML - Beautiful Soup

 ### How to read and navigate XML There is a Python library called BeautifulSoup, which makes reading in and parsing XML data easier. Here is the link to the documentation: [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/) The find() method will find the first place where an xml element occurs. For example using find('record') will return the first record in the xml file: ```xml <record>   <field name="Country or Area" key="ABW">Aruba</field>   <field name="Item" key="SP.POP.TOTL">Population, total</field>   <field name="Year">1960</field>   <field name="Value">54211</field> </record> ``` The find_all() method returns all of the matching tags. So find_all('record') would return all of the elements with the `<record>` tag. Run the code cells below to get a basic idea of how to navigate XML with BeautifulSoup. To naviga...

JSON File data Orientation Types

  If you open the link with the documentation, you'll see there is an   orient   option that can handle JSON formatted in different ways: 'split' : dict like {index -> [index], columns -> [columns], data -> [values]} 'records' : list like [{column -> value}, ... , {column -> value}] 'index' : dict like {index -> {column -> value}} 'columns' : dict like {column -> {index -> value}} 'values' : just the values array In this case, the JSON is formatted with a 'records' orientation, so you'll need to use that value in the read_json() method. You can tell that the format is 'records' by comparing the pattern in the documentation with the pattern in the JSON file.