Welcome to Ed2Ti Blog.

College: Trebas Institute
Professor: Iyad Koteich

It's impressive how fast and easy you can start your won API solution using Flask and Python.

In this post, I want to share with you my experience using tools and concepts to create a simple API solution.

Tools:

• Phyton (https://www.python.org/)
• Flask (https://flask.palletsprojects.com/en/2.2.x/)
• SQLite (https://www.sqlite.org/)
• Postman (https://web.postman.co/)

In this restful API I'm using the verbs:
* POST - To insert information into a database
* GET - To recover information from a database.
* PUT (I'm not using) - To update information into a database.
* DELETE (I'm not using) - To delete information from a database.

The next objective is to create a full CRUD API.
All the codes are here on my Git repository: https://github.com/ed2ti/python_flask_db

An important point when you start your journey on the Microsoft Azure platform is to understand what services they dispose of to us and how can we choose these services to attend to our budget and business.

In fact, for AZ 900 test, we need to study these concepts:

Cloud Models
Cloud Benefits
Azure Cloud Services
Core Azure Archteture
Core Azure Workloads
Azure Solutions
Azure Management Tools
Security tools and features
Core Azure identity services
Azure Governance Methodologies
Privacy, compliance, and data protection standards
Planning and Cost Management
Azure SLAs and service lifecycles

Every day, I'll write about one or two concepts here.

Today I'll try to discuss in a few words some important concepts about DevOps. The first is that DevOps is a concept that results in a paradigm change.

Starting as DevOps we need to understand the difference between Greenfield and Brownfield. In a simple view, Greenfield is untouched land, and a Brownfield is the used ground for other purposes. Now you are confused, but don't worry.

Greenfield => New Brownfield => Legacy

But, how about using a bimodal model? For example, we can see in some organizations two or more kinds of systems. We can categorize them as Record and Engagement Systems.

The Record Systems are more stable and generally don't have frequent updates. Engagement Systems generally are created to support pieces of information that come from Record Systems. Then, an issue with this Engagement System could impact Record Systems.

First, we need to understand some initial concepts and a little history of developing and managing the environments.

In past, ware common we listen to the history of some projects that fail because of the integrations between the Developer and Operation teams.

Today I don't want to compare MariaDB(Mysql) with PostgreSQL. I just want to share with you some ways to fastly start these databases as a Service for free.

** Just remember that MariaDB is not the same as Mysql. MariaDB is a community fork from Mysql.

I am talking about a study or test proposal.

The ElephantSql, the name refers to an animal that represents PostgreSql, and you can do your register at https://www.elephantsql.com/

The Skysql, you will receive 500$ to spend with your MariaDB databases. To register an account you can just click on the link https://mariadb.com/products/skysql/get-started/ [Remember to stop the service after you use]

But you can ask me: what is your proposal with this share?

Between 2022-10-31 and 2022-11-04, professor Iyad Koteich asks me to present some examples of using ETL with Pentaho(PDI). We will do an ETL process with Python, and then, we will do the same with Pentaho(PDI).

The main purpose is that you can understand the ETL pipeline at Pentaho.

All the files and requirements will be here, and you can download them on GitHub (https://github.com/ed2ti/Pentaho_Data_Integration)

Thank you Professor Iyad Koteich for this new challenger.

Today I was reading and organizing some files from my computer and I found my first article from Trebas.

It was interesting because after reading again this file I found a lot of necessary modifications and corrections. Of course, I can't do that on the original document, but I'm planning to do another with a similar title.

If you can read It, I'll really appreciate your feedback by mail. Thank you!

Professor: Ghazal G.Fard
College: Trebas Institute

Link to download https://github.com/ed2ti/articles/blob/main/trebas_assignment_2_english.pdf

As requested by Professor Iyad Koteich, now the application saves the data on the SQLite database.

In the first post (here) we did a python program to join three spreadsheets in just one. This work involves knowledge of pandas to organize the data.

Now we create a database and a table to store all data. This version is so simple and does not check the user store option.

I'll create a new project on GitHub because I'll show how to merge these steps in one project. (god to understand git commands line)

GitHub: https://github.com/ed2ti/exercise04

[CODE]

# *************************** #
# College: Trebas Institute 
# Professor: Iyad Koteich
# Class: Edward
# Day: 2022-10-13
# *************************** #

#Importing Libres
## Pandas ##
import pandas as pd
from pandas import DataFrame 

## sqlite3 ##
import sqlite3
from sqlite3 import Error

###
### INFORMATIONS FROM Concordia ###
###

##Loading Data
concordia = pd.read_excel('Concordia.xlsx')

## Informing that this informations are form Concordia
concordia["College"] = 'Concordia'

#show Concordia Result
print(concordia.head())
print('n')

###
### INFORMATIONS FROM McGill ###
###

##Loading Data
mcgill    = pd.read_excel('McGill.xlsx')

## Informing that this informations are form McGill
mcgill["College"] = 'McGill'

## Organizing the DataFrame
mcgill.rename(columns={'id': 'Student_Code'}, inplace = True)
mcgill.rename(columns={'name': 'Full_Name'}, inplace = True)
mcgill.rename(columns={'Course': 'Program'}, inplace = True)
mcgill.rename(columns={'country': 'Nationality'}, inplace = True)
mcgill.drop(columns=['city'], inplace = True)

#show McGill Result
print(mcgill.head())
print('n')

###
### INFORMATIONS FROM Trebas ###
###

##Loading Data And Print The Colums
trebas    = pd.read_excel('TREBAS.xlsx')

## Informing that this informations are form TREBAS
trebas["College"] = 'TREBAS'

## Organizing the DataFrame
trebas.rename(columns={'Country': 'Nationality'}, inplace = True)
trebas.rename(columns={'Student_ID': 'Student_Code'}, inplace = True)
trebas["Full_Name"] = trebas["First_Name"] + " " + trebas["Last_Name"]
trebas.drop(columns=['City'], inplace = True)
trebas.drop(columns=['First_Name'], inplace = True)
trebas.drop(columns=['Last_Name'], inplace = True)

#Ajusting the sequence of the columns
trebas = trebas[["Student_Code","Full_Name","Nationality","Program", "College"]]

#show Trebas Result
print(trebas.head())
print('n')

final = pd.concat([trebas, concordia,mcgill], ignore_index=True, sort=False)
print(final)


# Exporting to excel (to_excel)
final.to_excel ('final.xlsx', index = True, header=True)


## 
## Try to conect database (memory)
## This database is on the Memory (no file) for a best performance.
## If you want, you can change :memory: for a name file.
##

#target = ':memory:';
target = 'ex-04.db';

conn = None;
try:
    conn    = sqlite3.connect(target)
    print("connect on Database sqlite3 "+sqlite3.version)
    cur     = conn.cursor()
except Error as e:
    print(e)

# Create a final table (IF NOT EXISTS)#
## id = AUTOINCREMENT ##
cur.execute("CREATE TABLE IF NOT EXISTS final(id INTEGER PRIMARY KEY AUTOINCREMENT, Student_Code TEXT, Full_Name TEXT, Nationality TEXT, Program TEXT, College TEXT)")

## Trucate the final table ##
cur.execute("DELETE FROM final")

#
data = final.values

#print(len(data))

for i in range(len(data)):
    #datal = Data Line 
    datal = data[i]
    # writing on Database
    cur.execute("INSERT INTO final(Student_Code, Full_Name, Nationality, Program, College) VALUES (?,?,?,?,?)", datal)
    conn.commit()
conn.close()

A new challenge from Professor Iyad Koteich with Python and Pandas

The objective consists of organizing and joining (merge) three spreadsheets from Colleges in Montreal. All three spreadsheets are different and some work is necessary to normalize them.

1. Define the base columns. (Best option is the Concordia spreadsheet.
2. Insert College column.
3. Rename the names of the DataFrame columns (if necessary)
4. Delete some DataFrame columns (if necessary)
5. Organize DataFrame columns from trebas
6. Save the fina DataFrame on an xlsx file.

After that, just put your hands on it.

GitHub -> https://github.com/ed2ti/exercise03

[The Python CoDe]

# *************************** #
# College: Trebas Institute 
# Professor: Iyad Koteich
# Class: Edward
# Day: 2022-10-05
# *************************** #

#Importing Libres
import pandas as pd
from pandas import DataFrame 

###
### INFORMATIONS FROM Concordia ###
###

##Loading Data
concordia = pd.read_excel('Concordia.xlsx')

## Informing that this informations are form Concordia
concordia["College"] = 'Concordia'

#show Concordia Result
print(concordia.head())
print('n')

###
### INFORMATIONS FROM McGill ###
###

##Loading Data
mcgill    = pd.read_excel('McGill.xlsx')

## Informing that this informations are form McGill
mcgill["College"] = 'McGill'

## Organizing the DataFrame
mcgill.rename(columns={'id': 'Student_Code'}, inplace = True)
mcgill.rename(columns={'name': 'Full_Name'}, inplace = True)
mcgill.rename(columns={'Course': 'Program'}, inplace = True)
mcgill.rename(columns={'country': 'Nationality'}, inplace = True)
mcgill.drop(columns=['city'], inplace = True)

#show McGill Result
print(mcgill.head())
print('n')

###
### INFORMATIONS FROM Trebas ###
###

##Loading Data And Print The Colums
trebas    = pd.read_excel('TREBAS.xlsx')

## Informing that this informations are form TREBAS
trebas["College"] = 'TREBAS'

## Organizing the DataFrame
trebas.rename(columns={'Country': 'Nationality'}, inplace = True)
trebas.rename(columns={'Student_ID': 'Student_Code'}, inplace = True)
trebas["Full_Name"] = trebas["First_Name"] + " " + trebas["Last_Name"]
trebas.drop(columns=['City'], inplace = True)
trebas.drop(columns=['First_Name'], inplace = True)
trebas.drop(columns=['Last_Name'], inplace = True)

#Ajusting the sequence of the columns
trebas = trebas[["Student_Code","Full_Name","Nationality","Program", "College"]]

#show Trebas Result
print(trebas.head())
print('n')

final = pd.concat([trebas, concordia,mcgill], ignore_index=True, sort=False)
print(final)


# Exporting to excel (to_excel)
final.to_excel ('final.xlsx', index = True, header=True)

Today we did a program in Python just to convert celsius to Fahrenheit

# ********* #
# College : Trebas Institute 
# Professor: Iyad Koteich
# Class : Edward
# Day: 03/10/2022
# ********* #

def convert(gc):
    gf = (gc*9/5)+32
    return gf

def good(gf):
    if (gf>60 and gf<90):
        print(f"{gf} fahrenheit is a good Wealth")
    else:
        print(f"{gf} fahrenheit is not Good Wealth")

def trebas():
    print("College : Trebas Institute") 
    print("Professor: Iyad Koteich")
    print("Class : Edward")
    print("Day: 03/10/2022")
    print("")

trebas()

optoin = '' 
while optoin != '0':
    option = int(input("Value in celcus: "))
    if option != 0:
        gf = convert (option)
        good(gf)
    else:
        print("Exiting")
        break