Welcome to Ed2Ti Blog.

My journey on IT is here.

Trebas College

Here i try to describe my journey as a student at Trebas Institute.

A new challenge from Professor Iyad Koteich with Python and Pandas

The objective consists of organizing and joining (merge) three spreadsheets from Colleges in Montreal. All three spreadsheets are different and some work is necessary to normalize them.

  1. Define the base columns. (Best option is the Concordia spreadsheet.
  2. Insert College column.
  3. Rename the names of the DataFrame columns (if necessary)
  4. Delete some DataFrame columns (if necessary)
  5. Organize DataFrame columns from trebas
  6. Save the fina DataFrame on an xlsx file.

After that, just put your hands on it.

GitHub -> https://github.com/ed2ti/exercise03

[The Python CoDe]

# *************************** #
# College: Trebas Institute 
# Professor: Iyad Koteich
# Class: Edward
# Day: 2022-10-05
# *************************** #

#Importing Libres
import pandas as pd
from pandas import DataFrame 

###
### INFORMATIONS FROM Concordia ###
###

##Loading Data
concordia = pd.read_excel('Concordia.xlsx')

## Informing that this informations are form Concordia
concordia["College"] = 'Concordia'

#show Concordia Result
print(concordia.head())
print('n')

###
### INFORMATIONS FROM McGill ###
###

##Loading Data
mcgill    = pd.read_excel('McGill.xlsx')

## Informing that this informations are form McGill
mcgill["College"] = 'McGill'

## Organizing the DataFrame
mcgill.rename(columns={'id': 'Student_Code'}, inplace = True)
mcgill.rename(columns={'name': 'Full_Name'}, inplace = True)
mcgill.rename(columns={'Course': 'Program'}, inplace = True)
mcgill.rename(columns={'country': 'Nationality'}, inplace = True)
mcgill.drop(columns=['city'], inplace = True)

#show McGill Result
print(mcgill.head())
print('n')

###
### INFORMATIONS FROM Trebas ###
###

##Loading Data And Print The Colums
trebas    = pd.read_excel('TREBAS.xlsx')

## Informing that this informations are form TREBAS
trebas["College"] = 'TREBAS'

## Organizing the DataFrame
trebas.rename(columns={'Country': 'Nationality'}, inplace = True)
trebas.rename(columns={'Student_ID': 'Student_Code'}, inplace = True)
trebas["Full_Name"] = trebas["First_Name"] + " " + trebas["Last_Name"]
trebas.drop(columns=['City'], inplace = True)
trebas.drop(columns=['First_Name'], inplace = True)
trebas.drop(columns=['Last_Name'], inplace = True)

#Ajusting the sequence of the columns
trebas = trebas[["Student_Code","Full_Name","Nationality","Program", "College"]]

#show Trebas Result
print(trebas.head())
print('n')

final = pd.concat([trebas, concordia,mcgill], ignore_index=True, sort=False)
print(final)


# Exporting to excel (to_excel)
final.to_excel ('final.xlsx', index = True, header=True)

The principles of Data Warehouse.

- Posted in Trebas College by

This week, Mr. Iyad Koteich, my professor at Trebas Institute, gave me an opportunity to share my knowledge about Data Warehouses.

It was perfect because I could remember some concepts and share them with my classmates. Of course, to talk about business intelligence, data warehouse, and SGBD we need to

Initial Concepts.

Model ER - Or Entity–relationship model. Before we start to develop a solution, is important to have the base requirements of the system to model the first structure of the Database. There is a pattern to do that and is important to be attended to because any user from the team can read the model. If not, all the knowledge about the project will be with a manager or a little part of the team.

SGBG - This is an acronym for "Database management system." Sometimes is common confusion between the concept of Database and SGBD. To be simple, just understand that the Database is a part of the SGBD. For example, the SGBD can control access to databases and tables in these databases.

Transaction System.

OLTP

Analytics Concepts

OLAP Fact Table Dimensions Cube

Last week, my professor at Trebas Institute, Mr. Iyad Koteich started a class talking about ER Model. I confess that I really like to design database models(ER Models).

My classmate Ravi Rawat took a picture from a dashboard with a relationship model designer by Professor Iyad Koteich. This picture was the "kick-off" for this project.

First I need to explain the difference between ER Model and R Model.

R Model: Describe a vision of the problem to be resolved that any person can see and understand.

ER Model: Materializing the relational model in an Entity on a database requires some organizations.

I will use "WE" every time because Professor Iyad Koteich is my mentor and some classmates can contribute with us.

We need to understand how this project will be constructed:

  1. Create the full ER Model -> MVP (2022-07-29)
  2. Develop a PHP program to maintain the data (pĺayer, teams, clubs, games, etc) -> MPV (2022-08-31)
  3. Develop a python program, to run in the background, analyzing the teams and the games. (2022-12-31)

The Codes of this project are hosted on GitHub: https://github.com/ed2ti/soccer-game-ia

You can directly download the ER-Model here

Study objectives:

  1. Understand ER Model
  2. Construct a Database (MariaDB or Postgres SQL)
  3. FrontEnd Develop with PHP
  4. BackEnd Develop with PYTON (sklearn, pandas, numpy)

All players were created by a website: https://www.mockaroo.com/. You can directly download it here.

Professor: Iyad Koteich

Objective: Using a MariaDB database, create a simple PHP program to:

  1. Create a new account
  2. Login user

I'm using in this example some technologies like HTML5, CSS3, PHP, and MySQL(MariaDB)

Table: users

DROP TABLE IF EXISTS `users`;
CREATE TABLE `users` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `firstName` varchar(120) NOT NULL,
  `lastName` varchar(120) NOT NULL,
  `password` varchar(120) NOT NULL,
  `email` varchar(120) NOT NULL,
  `phone` varchar(120) NOT NULL,
  `last` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `email` (`email`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

All files from this project you can find on GitHub: GitHub-Link

Professor: Iyad Koteich

On 2022-07-04, I present at class a LAB with AWS using two services:

  1. EC2(Elastic Compute Cloud) Instances
  2. RDS(Relational Database Service) - Database (MariaDB)

The purpose is to compare the configuration and utilization of MariaDB as an IAAS or PAAS.

MariaDB - IAAS(infrastructure-as-a-service)

  1. Create an EC2 Instance with an ubuntu server
  2. Instal MariaDB - apt install mariadb-server
  3. Configure a Security group
  4. Change bind-address to 0.0.0.0
  5. Create a user and give access to him.

Create User:

  1. CREATE USER 'admin'@'%' IDENTIFIED BY 'password';
  2. GRANT ALL PRIVILEGES ON . TO 'admin'@'%' WITH GRANT OPTION;
  3. FLUSH PRIVILEGES;

Change Bind Address: /etc/mysql/mariadb.conf.d/ and change bind-address

MariaDB - PAAS(platform-as-a-service)

  • Create a RDS Instance item
  • Configure a Security group

Relational Database: MariaDB is Relational Database and what is it?

  1. relationship -> one-to-one [indicate]
  2. relationship -> one-to-many [orders]
  3. [son] relationship -> many-to-many [rent cars]

Data Definition Language.

  • CREATE DATABASE trebasdb;
  • ** USE trebasdb; **
  • CREATE TABLE users( id int(4) AUTO_INCREMENT, name varchar(30) NOT NULL, email varchar(50), PRIMARY KEY (id) );

Data Manipulation Language.

  • SELECT * FROM
  • DELETE FROM
  • INSERT INTO

Design patterns.

Professor: Iyad Koteich

You are organizing a party for your group of students.

  1. Check how many students are interested
  2. You are limited to accept 20 students only

Make a Python program that will choose 20 randomly out of All interested students

import pandas as pd
import numpy as np
df = pd.read_csv("class.csv")
df.head()
data = df.values
array = np.random.choice(16, 10)
array = np.array(array)
for i in array:
    print(data[i][3])

I did an adaptation to attend calss.csv file.

You can find all code at https://github.com/ed2ti/exercise01

At Trebas (Computer Hardware & Software Environment), the challenger proposed by Ghazal G.Fard starts with a WebServer in GoLang.

Before commenting on the code, we need to understand two differences between python and GoLang.

1) To "ignore" a line in Python we use # and in Golang we use // Some other languages like C, use #, and others, like PHP, use // or /* */.

2) In python I don't need to define the main() function.

We need to keep in mind that we need to develop clean code. For example, is a good practice to use MVC (Model, View, Controller) techniques.

On the code, we need to import three packages: "fmt", "log", "net/http"

The fmt package we use to give a response. Can be on screen, or in this case, a web response. For the log package, we are just writing a log. The net/http we use to start a webserver.

There are a lot of concepts that we can present in the future.

//# *************************** #
//# College: Trebas Institute 
//# Professor: Ghazal G.Fard
//# Class: Edward
//# Day: 2022-06-21
//# *************************** #

package main
import (
    "fmt"
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request){
        fmt.Fprintf(w, "Hello!")
    })


    fmt.Printf("Starting server at port 8080n")
    if err := http.ListenAndServe(":8080", nil); err != nil {
        log.Fatal(err)
    }
}

In this example, we do not use a concept of routes. Next week I'll start a new project to demonstrate rote on the GoLang web server.

Thank you for your time, and I hope that you can get an answer to your question.