Pandas to open a SQL database or to run a SQL query against a database

 There is more than one way to do this depending on the type of SQL database you are working with: the sqlite3 library or the sqlalchemy library.

In the same folder as this Jupyter notebook, there is a SQLite database file called "population_data.db". SQLite is a database engine meant for single applications. The entire database is contained in one file. You can read more about SQLite here.


Sqlite is a database storage engine, which can be better compared with things such as MySQL, PostgreSQL, Oracle, MSSQL, etc. It is used to store and retrieve structured data from files.

SQLAlchemy is a Python library that provides an object relational mapper (ORM). It does what it suggests: it maps your databases (tables, etc.) to Python objects, so that you can more easily and natively interact with them. SQLAlchemy can be used with sqlite, MySQL, PostgreSQL, etc.

So, an ORM provides a set of tools that let you interact with your database models consistently across database engines.


SQLAlchemy is a powerful database access tool kit for Python, with its object-relational mapper (ORM) being one of its most famous components, and the one discussed and used here.

When you’re working in an object-oriented language like Python, it’s often useful to think in terms of objects. It’s possible to map the results returned by SQL queries to objects, but doing so works against the grain of how the database works. Sticking with the scalar results provided by SQL works against the grain of how Python developers work. This problem is known as object-relational impedance mismatch.

The ORM provided by SQLAlchemy sits between the SQLite database and your Python program and transforms the data flow between the database engine and Python objects. SQLAlchemy allows you to think in terms of objects and still retain the powerful features of a database engine

These two things are not mutually exclusive, because they aren't the same type of thing.

SQLite3 is a database engine, in other words it's the actual implementation of a database. It provides an abstraction over the messy details of how to store and access raw data on your filesystem.

SQLAlchemy is an Object-Relational Mapper, a thing that lets you use object-oriented Python (in this case*) code to represent your data, and abstracts away the fiddly parts of translating from code to the SQL used to communicate with the database engine, and vice versa.

The two are on separate layers: at the bottom you have SQLite, then on top of that you have SQLAlchemy, then you have your regular application code.

* other ORMs include ActiveRecord for Ruby or Doctrine for PHP, for example

Comments

Post a Comment

Popular posts from this blog

Read and Navigate XML - Beautiful Soup

 ### How to read and navigate XML There is a Python library called BeautifulSoup, which makes reading in and parsing XML data easier. Here is the link to the documentation: [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/) The find() method will find the first place where an xml element occurs. For example using find('record') will return the first record in the xml file: ```xml <record>   <field name="Country or Area" key="ABW">Aruba</field>   <field name="Item" key="SP.POP.TOTL">Population, total</field>   <field name="Year">1960</field>   <field name="Value">54211</field> </record> ``` The find_all() method returns all of the matching tags. So find_all('record') would return all of the elements with the `<record>` tag. Run the code cells below to get a basic idea of how to navigate XML with BeautifulSoup. To naviga...
Read more

difference-between-stream-processing-and-message-processing

 difference-between-stream-processing-and-message-processing Message Processing implies operations on and/or using  individual  messages. Stream Processing encompasses operations on and/or using individual messages as well as operations on collection of messages as they flow into the system. For e.g., let's say transactions are coming in for a payment instrument - stream processing can be used to continuously compute hourly average spend. In this case - a sliding window can be imposed on the stream which picks up messages within the hour and computes average on the amount. Such figures can then be used as inputs to fraud detection systems Although Rabbit supports streaming, it was actually not built for it(see Rabbit´s web site) Rabbit is a Message broker and Kafka is a event streaming platform. Kafka can handle a huge number of 'messages' towards Rabbit. Kafka is a log while Rabbit is a queue which means that if once consumed, Rabbit´s messages are not there anymore in c...
Read more

WordNet in Python

Image
  WordNet   is an English dictionary which is a part of   Natural Language Tool Kit   (NLTK) for Python. This is an extensive library built to make   Natural Language Processing   (NLP) easy. Some basic functions will be discussed in this article. To start using WordNet, you have to import it first: from  nltk.corpus  import  wordnet Synsets and Lemmas In WordNet, similar words are grouped into a set known as a  Synset  (short for  Synonym-set ). Every Synset has a name, a part-of-speech, and a number. The words in a Synset are known as  Lemmas . Getting Synsets The function  wordnet.synsets('word')   returns an array  containing all the Synsets related to the word passed to it as the argument. Example: print ( wordnet.sysnets ( 'room' )) Output: [Synset(‘room.n.01’), Synset(‘room.n.02’), Synset(‘room.n.03’), Synset(‘room.n.04’), Synset(‘board.v.02’)] The method returned five Synsets; four have the name...
Read more