Python : Matplotlib

Matplotlib is a Python library used to create charts and graphs.

Installation
$ sudo python3.6 -m pip install pandas
Simple Line plot
import codecademylib  
from matplotlib import pyplot as plt

days = [0, 1, 2, 3, 4,5,6]  
money_spent = [10, 12, 12, 10, 14,22,24]  
plt.plot(days, money_spent)  
plt.show()  

multiple line plots displayed on the same set of axes.

import codecademylib  
from matplotlib import pyplot as plt

time = [0, 1, 2, 3, 4]  
revenue = [200, 400, 650, 800, 850]  
costs = [150, 500, 550, 550, 560]

plt.plot(time, revenue)  
plt.plot(time, costs)  
plt.show()  

Plot options

import codecademylib  
from matplotlib import pyplot as plt

time = [0, 1, 2, 3, 4]  
revenue = [200, 400, 650, 800, 850]  
costs = [150, 500, 550, 550, 560]

plt.plot(time,revenue,color='purple',linestyle='--')  
# marker='o' is circle; s is squar , * is star
plt.plot(time,costs,color='#82edc9',marker='s')  
plt.show()  
special

zoom in and zoom out

# first wto are x coordinates and last two and y coordinates
plt.axis([0, 12, 2900, 3100])  
#Labelling the axis
plt.xlabel('Time of day')  
plt.ylabel('Happiness Rating (out of 10)')  
plt.title('My Self-Reported Happiness While Awake')  
Creating subplots

he command plt.subplot(2, 3, 4) would create “Subplot 4” from the figure above.

Any plt.plot that comes after plt.subplot will create a line plot in the specified subplot

import codecademylib  
from matplotlib import pyplot as plt

x = range(7)  
straight_line = [0, 1, 2, 3, 4, 5, 6]  
parabola = [0, 1, 4, 9, 16, 25, 36]  
cubic = [0, 1, 8, 27, 64, 125, 216]

# Subplot 1
plt.subplot(2, 1, 1)  
plt.plot(x, straight_line)

# Subplot 2
plt.subplot(2, 2, 3)  
plt.plot(x, parabola)

# Subplot 3
plt.subplot(2, 2, 4)  
plt.plot(x, cubic)

plt.subplots_adjust(wspace=0.35, bottom=0.2)

plt.show()  

Adding legends

plt.legend(['Hyrule', 'Kakariko','Gerudo Valley'],loc=8)  
specific ticks
ax.set_xticks([1, 2, 4])  
ax.set_yticks([0.1, 0.6, 0.8])  
ax.set_yticklabels(['10%', '60%', '80%'])  

Working example

from matplotlib import pyplot as plt

month_names = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep","Oct", "Nov", "Dec"]

months = range(12)  
conversion = [0.05, 0.08, 0.18, 0.28, 0.4, 0.66, 0.74, 0.78, 0.8, 0.81, 0.85, 0.85]

plt.xlabel("Months")  
plt.ylabel("Conversion")

plt.plot(months, conversion)

# Your work here
ax = plt.subplot()  
ax.set_xticks(months)  
ax.set_xticklabels(month_names)

ax.set_yticks([0.10, 0.25, 0.5, 0.75])  
ax.set_yticklabels(['10%', '25%', '50%','75%'])

plt.show()  
Axis lables
plt.xlabel('Time of day')  
plt.ylabel('Happiness Rating (out of 10)')  
plt.title('My Self-Reported Happiness While Awake')  
plt.show()  
Showing legends
plt.legend(['parabola', 'cubic'], loc=6)  
plt.show()  
saving fig
plt.figure(figsize=(7, 3))  
plt.plot(years, power_generated)  
plt.savefig('power_generated.png')  
Different type of graphs
bargraph
from matplotlib import pyplot as plt

drinks = ["cappuccino", "latte", "chai", "americano", "mocha", "espresso"]  
sales =  [91, 76, 56, 66, 52, 27]

plt.bar(range(len(drinks)), sales)

#create your ax object here
ax = plt.subplot()  
ax.set_xticks([0, 1, 2, 3, 4, 5])  
ax.set_xticklabels(["cappuccino", "latte", "chai", "americano", "mocha", "espresso"],  
rotation=90)  
plt.show()  

adjecent bargraph

from matplotlib import pyplot as plt

drinks = ["cappuccino", "latte", "chai", "americano", "mocha", "espresso"]  
sales1 = [91, 76, 56, 66, 52, 27]  
sales2 = [65, 82, 36, 68, 38, 40]

#Paste the x_values code here
n = 1  # This is our first dataset (out of 2)  
t = 2 # Number of dataset  
d = 6 # Number of sets of bars  
w = 0.8 # Width of each bar  
store1_x = [t*element + w*n for element  
             in range(d)]

plt.bar(store1_x, sales1)  
#Paste the x_values code here
n = 2  # This is our second dataset (out of 2)  
t = 2 # Number of dataset  
d = 6 # Number of sets of bars  
w = 0.8 # Width of each bar  
store2_x = [t*element + w*n for element  
             in range(d)]

plt.bar(store2_x, sales2)

plt.show()  
Creating range
from matplotlib import pyplot as plt

months = range(12)  
month_names = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]  
revenue = [16000, 14000, 17500, 19500, 21500, 21500, 22000, 23000, 20000, 19500, 18000, 16500]

#your work here
ax = plt.subplot()  
ax.set_xticks(months)  
ax.set_xticklabels(month_names)

y_upper = [1.1*i for i in revenue]  
y_lower = [0.9*i for i in revenue]  
plt.fill_between(range(len(months)), y_lower, y_upper, alpha=0.2) 

plt.plot(range(len(months)),revenue)  
plt.show()  
Pie charts
from matplotlib import pyplot as plt

payment_method_names = ["Card Swipe", "Cash", "Apple Pay", "Other"]  
payment_method_freqs = [270, 77, 32, 11]

plt.pie(payment_method_freqs, autopct="%0.1f%%")  
plt.axis('equal')  
plt.legend(payment_method_names)

plt.show()  
Histogram
a = normal(loc=64, scale=2, size=10000)  
b = normal(loc=70, scale=2, size=100000)

plt.hist(a, range=(55, 75), bins=20, alpha=0.5, normed=True)  
plt.hist(b, range=(55, 75), bins=20, alpha=0.5, normed=True)  
plt.show()  

 

Please refer to matplotlib tutorial for further information.

Python : Working with Pandas

Pandas is a Python module for working with tabular data (i.e., data in a table with rows and columns). Tabular data has a lot of the same functionality as SQL or Excel, but Pandas adds the power of Python.

  • A DataFrame is an object that stores data as rows and columns. You can think of a DataFrame as a spreadsheet or as a SQL table
  • You can manually create a DataFrame or fill it with data from a CSV, an Excel spreadsheet, or a SQL query.
  • DataFrames have rows and columns. Each column has a name, which is a string.
  • Each row has an index, which is an integer.
  • DataFrames can contain many different data types: strings, ints, floats, tuples, etc.

Please refer to how to install pandas to install it for python3.6

Pandas have following three data structures

  1. Series
  2. DataFrames
  3. Panels

we mostly will work with DataFrames. series can be considered as subset of DataFrames.

Creating Series and DataFrames

my_list=[1,3,5,6,8,11,18]
s1 = pd.Series(my_list)
s2 = pd.Series([1,3,5,6,8,11,18])
s3 = pd.Series(np.arange(100))

s4=pd.DataFrame(my_list)
df2 = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 4)),columns=['a', 'b', 'c', 'd'])

>>> type(my_list)
<class 'list'>
>>> type(s1)
<class 'pandas.core.series.Series'>
>>> type(s4)
<class 'pandas.core.frame.DataFrame'>
# Creating DataFrame using dict
sales = [{'account': 'Jones LLC', 'Jun': 150, 'Jul': 200, 'Aug': 140},
{'account': 'Alpha Co', 'Jun': 200, 'Jul': 210, 'Aug': 215},
{'account': 'Blue Inc', 'Jun': 50, 'Jul': 90, 'Aug': 95 }]
df = pd.DataFrame(sales)
#
sales = {'account': ['Jones LLC', 'Alpha Co', 'Blue Inc'],
'Jun': [111, 222, 55],
'Jul': [222, 210, 90],
'Aug': [140, 215, 95]}
df = pd.DataFrame.from_dict(sales)

>>> df.dtypes
Feb int64
Jan int64
Mar int64
account object
dtype: object

 

df2 = pd.DataFrame(data,columns=['Store ID','Location','Number of Employees'])  
print df2  
Reading from and writing into CSV files
df = pd.read_csv('my-csv-file.csv')
df.to_csv('new-csv-file.csv')
#displaying top 5 rows
df.head()
#displaying top 10 rows
df.head(10)
#statistical information about df
df.info()

df columns can be accessed by two ways

df['age']  
df.age  

type of this data

print(type(df['age'])
print(type(df.age)
<class 'pandas.core.series.Series'>

Accessing rows

print(df.loc[2]) #Python is zero indexed  
df.loc[3:7]  
df.loc[:4]  
df.loc[2:]  

Creating sub dataframe

# create using specific columns
df2 =df[['clinic_north','clinic_south']]  
# create using certian conditon
df2=df[df.month == 'january']  
df2=df[df.age > 27]  
df2=df[df.city != 'Mumbai']  
# using multyiple conditions / In Python, | means "or" and & means "and".
df2 = df[(df.age < 30) | (df.name == 'Martha Jones')] # (curly brackets are must)  
df2 = df[df.month.isin(['January','February','March'])]  
# Reset index to fetch correct row number
df2.reset_index(drop=True)  
Modifying DataFrames
#adding a column
df['height']=[1,2,3,4,5,5]  
#adding same value column
df['In Stock?'] = True  
df['In Stock?'] = 'Yes'  
#New column derived from other columns
df['Revenue'] = df.Price - df['Cost to Manufacture']  
#changing cap
from strings import lower  
df['Lowercase Name'] = df['Name'].apply(lower)  

Using lambda function to modify

get_last_name = lambda x: x.split(' ')[-1]  
df['last_name'] = df.name.apply(get_last_name)  

Columns can be renamed using

df.columns = ['ID', 'Title','Category','Year Released','Rating']  
#selective renaming
df.rename(columns={'name': 'First Name','age': 'Age'},inplace=True)  

Using rename with only the columns keyword will create a new DataFrame, leaving your original DataFrame unchanged. That’s why we also passed in the keyword argument inplace=True. Using inplace=True lets us edit the original DataFrame.

wokring with rows amd columns

import pandas as pd

orders = pd.read_csv('shoefly.csv')

print orders.head(5)

orders['shoe_source'] = orders.shoe_material.apply(lambda x:   
                            'animal' if x == 'leather'else 'vegan')

orders['salutation'] = orders.apply(lambda row:   
                                    'Dear Mr. ' + row['last_name']
                                    if row['gender'] == 'male'
                                    else 'Dear Ms. ' + row['last_name'],
                                    axis=1)

Aggregations

  • mean Average of all values in column
  • std Standard deviation
  • median Median
  • max Maximum value in column
  • min Minimum value in column
  • count Number of values in column
  • nunique Number of unique values in column
  • unique List of unique values in column
num_colors =orders.price.max()  
num_colors =orders.shoe_color.nunique()  
#groupby
df.groupby('column1').column2.measurement()  
#example
grades = df.groupby('student').grade.mean()  
pricey_shoes =orders.groupby('shoe_type').price.max()  
#groupby with muiple columns
shoe_counts = orders.groupby(['shoe_type','shoe_color']).id.count().reset_index()  

Sometimes, the operation that you want to perform is more complicated than mean or count. In those cases, you can use the apply method and lambda functions, just like we did for individual column operations. Note that the input to our lambda function will always be a list of values.

cheap_shoes = orders.groupby('shoe_color').price.apply(lambda x: np.percentile(x,25)).reset_index()  
print(cheap_shoes)

Pivots using dataframe

import pandas as pd

user_visits = pd.read_csv('page_visits.csv')  
print(user_visits.head())

click_source=user_visits.groupby('utm_source').id.count().reset_index()  
print(click_source)

click_source_by_month=user_visits.groupby(['utm_source','month']).id.count().reset_index()  
print(click_source_by_month)

click_source_by_month_pivot = click_source_by_month.pivot(index='utm_source',columns='month',values='id').reset_index()

print(click_source_by_month_pivot)  
Merge two dataframes
#this will happen if both dataframes have one column common
sales_vs_targets = pd.merge(sales,targets)  
#another way to merge
new_df = orders.merge(customers)  
#multile merge
new_df = orders.merge(customers).merge(products)  
#merge of columns name do not match
orders_products = pd.merge(orders,products.rename(columns={'id': 'product_id'}))  
#another way to merge if columns name do not match
orders_products = pd.merge(orders,products, left_on='product_id', right_on='id', suffixes=['_orders','_products'])  
#outermerge
pd.merge(company_a, company_b, how='outer')  
#left
store_a_b_left = pd.merge(store_a, store_b, how='left')  
#right
store_a_b_right = pd.merge(store_a, store_b, how='right')  
#concatenation
pd.concat([df1, df2])  
Query DataFrame
#simple condition
crushing_it = sales_vs_targets[sales_vs_targets.revenue > sales_vs_targets.target]  
#multiple conditions
results= all_data[(all_data.revenue > all_data.target) &(all_data.women>all_data.men)]  
print(results)  

Python : NumPy Basics

NumPy, which stands for Numerical Python.

NumPy has many uses including:

  • Efficiently working with many numbers at once
  • Generating random numbers
  • Performing many different numerical functions (i.e., calculating sin, cos, tan, mean, median, etc.)
Importing numpy

This is an universal way of importing NumPy and using np

import numpy as np  
NumPy Arrays

A NumPy array is a special type of list.
Each item can be of any type (strings, numbers, or even other arrays). You can even have different types of items in the same array.
Its best suited for numbers as it gives extra power for mathematical operations

import numpy as np

test_1 = np.array([92, 94, 88, 91, 87])  
print(test_1)  
print(type(test_1))

my_list = [1, 2, 3, 4, 5, 6]  
my_array = np.array(my_list)

print(my_list)  
print(type(my_list))  
print(my_array)  
print(type(my_array))  

Output is as bellow

[92 94 88 91 87]
<type 'numpy.ndarray'>  
[1, 2, 3, 4, 5, 6]
<type 'list'>  
[1 2 3 4 5 6]
<type 'numpy.ndarray'>  
reading from file
csv_array = np.genfromtxt('sample.csv', delimiter=',')  

NumPy arrays are more efficient than lists. One reason is that they allow you to do element-wise operations.

# With a list
l = [1, 2, 3, 4, 5]  
l_plus_3 = []  
for i in range(len(l)):  
    l_plus_3.append(l[i] + 3)
# With an array
a = np.array(l)  
a_plus_3 = a + 3  
>>> np.sqrt(a)
array([ 1, 1.41421356, 1.73205081, 2, 2.23606798, 2.44948974])  
Statistics with NumPy

Lets is consider following data set

import numpy as np

water_height = np.array([4.01, 4.03, 4.27, 4.29, 4.19,  
                         4.15, 4.16, 4.23, 4.29, 4.19,
                         4.00, 4.22, 4.25, 4.19, 4.10,
                         4.14, 4.03, 4.23, 4.08, 14.20,
                         14.03, 11.20, 8.19, 6.18, 4.04,
                         4.08, 4.11, 4.23, 3.99, 4.23])
# Calculate mean
np.mean(water_height)  
# Sort np array
np.sort(water_height)  
# Find median
np.median(water_height)  
# Find percentile value
np.percentile(water_height, 75)  
# Find standard Deviation
np.std(water_height)  
# Percentage of values greater than or equal to 4
np.mean(water_height >= 4)  

This works well for single dimensional array. Lets look at it how it looks for two dimensional array

ring_toss = np.array([[1, 0, 0],  
                          [0, 0, 1], 
                          [1, 0, 1]])
np.mean(ring_toss)  
0.44444444444444442  
# To find the means of each interior array, we specify axis 1 (the "rows"):
np.mean(ring_toss, axis=1)  
# To find the means of each index position, we specify axis 0 (the "columns"):
np.mean(ring_toss, axis=0)  

Python : Loops and iteration

In python loops or iteration can be achieved using following methods

  • while statement
  • for statement
While statement
  • Loop will run till condition is satisfied, once condition is satisfied, it will not execute the loop.
  • Values are checked before execution of block of code
count = 3

print('Starting while loop') 
while count < 5: 
      print('count is : ', count)
      count =count + 1

output

$ python3.6 loop02-while.py 
Starting while loop 
count is : 3 
count is : 4

For statement

following are different combination of for statement

cities = ['Pune','Mumbai','Hyderabad','Delhi','Bangalore']

for city in cities: 
    print('Current city is : ', city)

fname = 'Jasaon'

for char in fname: 
    print('char is : ', char)

for number in range(0, 10): 
    print('Current number is : ', number)

for x in range(2, 20, 3):
    print('Output is : ',x)

And output is

$ python3.6 loop02-for.py 
Current city is : Pune 
Current city is : Mumbai 
Current city is : Hyderabad 
Current city is : Delhi 
Current city is : Bangalore 
char is : J 
char is : a 
char is : s 
char is : a 
char is : o 
char is : n 
Current number is : 0 
Current number is : 1 
Current number is : 2 
Current number is : 3 
Current number is : 4 
Current number is : 5 
Current number is : 6 
Current number is : 7 
Current number is : 8 
Current number is : 9 
output is : 2
output is : 5
output is : 8
output is : 11
output is : 14
output is : 17
Control statements

pass

The pass statement in Python is used when a statement is required syntactically but you do not want any command or code to execute

continue

Causes the loop to skip the remainder of its body and immediately retest its condition prior to reiterating.

break

Terminates the loop statement and transfers execution to the statement immediately following the loop.

Sample program
rollNumbers = [0,1,2,3,4]


print('For loop with pass statemend started')  
for x in rollNumbers:  
    print('Current Number is :', x)
    if x == 2:
        pass
    print('This is inside loop for pass', x)
print('For loop with pass statemend ended')        

print('For loop with continue statemend started')  
for x in rollNumbers:  
    print('Current Number is :', x)
    if x == 2:
        continue
    print('This is inside loop for continue', x)
print('For loop with continue statemend ended')        

print('For loop with break statemend started')  
for x in rollNumbers:  
    print('Current Number is :', x)
    if x == 2:
        break
    print('This is inside loop for break', x)
print('For loop with break statemend ended')  

output is as below

$ python3.6 loop02-pass-continue-break.py 
For loop with pass statemend started  
Current Number is : 0  
This is inside loop for pass 0  
Current Number is : 1  
This is inside loop for pass 1  
Current Number is : 2  
This is inside loop for pass 2  
Current Number is : 3  
This is inside loop for pass 3  
Current Number is : 4  
This is inside loop for pass 4  
For loop with pass statemend ended  
For loop with continue statemend started  
Current Number is : 0  
This is inside loop for continue 0  
Current Number is : 1  
This is inside loop for continue 1  
Current Number is : 2  
Current Number is : 3  
This is inside loop for continue 3  
Current Number is : 4  
This is inside loop for continue 4  
For loop with continue statemend ended  
For loop with break statemend started  
Current Number is : 0  
This is inside loop for break 0  
Current Number is : 1  
This is inside loop for break 1  
Current Number is : 2  
For loop with break statemend ended  

Please note:
pass : nothing changes, pass statement is nothing but filler used for syntactical requirement

continue: This skips further processing in loop for current iteration only and it jumps to next iteration. Please note, second print statement is not printed for number 2

break: This terminates current iteration as well as whole loop and goes to next statement in python program

Python: Conditional statements

Python support singular IF,  If-ELSE, IF-ELIF-ELIF-ELSE statements. Sample formats are as below.

if b > a:
    print("b is greater than a")

if b > a:
    print("b is greater than a")
else:
    print("A is greater than b")

if A == 10:
    print("A is 10")
elif A == 20:
    print("A is 20")
else:
    print("A is neither 10 nor 20")

Conditional statements can be used as nested statements.

Boolean Expressions

Boolean expression is an expression that is either True or False. Python supports usual Boolean expressions. examples as below

Test Symbol Example
Equality == a == b
Inequality != a != b
Less than < a < b
Less than or equal <= a <= b
Greater than > a >b
Greater than or equa >= a >= b

Logical Operatiors

Python supports usual logical operators. examples as below. Symantic of these operator is similar to their English language meaning.

 

Operator Symbol Example
and and (a and b)
or or (a or b)

Try and Except

Since python is runtime programming language, many times errors are detected while running the job and whole program might fail due to any error. In such cases, you can use try..except.

Try .. except statement is like insurance in python

 

input_age =input(" Enter your age : ")

# days=int(input_age)*365

try:
days= int(input_age)*365
print("days inside try and except :" , days)
except Exception as e:
print(" Error :", e)

Python : Functions

Function is named sequence of statements that performs a required activity/computation. Best way to describe functions is that these are tools used to compartmentalise your code.

User defined functions

  • A function is a block of organised, reusable code that is used to perform a single, related action.
  • Function needs to be defined before its called.
  • It runs when it is called.
  • A function can return data as a result.

Function naming convention is same as that of variable naming conventions

  • It can have alphabets, numbers and underscore
  • It can not have special character except underscore
  • It can not start wit number, it can start only with alphabets or underscore

Simple function definition

def first_function():
    print("Inside first_function")

first_function()

Output

$ python3.6 funtions00.py 
Inside first_function

Empty parenthesis indicates that function does not take any input parameters.

Passing arguments

print("*** Start of the program ***")

def first_function(first_name,last_name,age):
    """
    simple functions with parameters
    """
    print('first_name : ', first_name)
    print('last_name  : ', last_name)
    print('age        : ', age)
    return

first_name = 'Jason'
last_name = 'Bourne'
age1 = 27
first_function(first_name,last_name,age1)

print("*** Second type of function ***")

def second_function(first_name,last_name,age):
    print('first_name : ', first_name)
    print('last_name  : ', last_name)
    print('age        : ', age)
    return

f_name = 'Jason'
l_name = 'Bourne'
second_function(f_name,l_name,27)

print("*** Third type of function ***")

def third_function(first_name,last_name,age=29):
    print('first_name : ', first_name)
    print('last_name  : ', last_name)
    print('age        : ', age)
    return

f_name = 'Jason'
l_name = 'Bourne'
third_function(f_name,l_name)

print("*** Fourth type of function ***")

def fourth_function(first_name,last_name='two',age=29):
    print('first_name : ', first_name)
    print('last_name  : ', last_name)
    print('age        : ', age)
    return

f_name = 'Jason'
l_name = 'Bourne'
age = 28
fourth_function(f_name,)

print("*** End of the program ***")
  • function arguments are positional in nature. i.e. you need to pass variables or values in the same sequence.
  • Optional parameters should be kept at the end and these needs to be handled using default values in function header.
  • We can use keyword parameters to have flexibility of using any sequence but in such cases, we need to define default value and also argument are named argument.

Output of above program is as below

$ python3.6 funtions03-passing\ arguments.py
*** Start of the program ***
first_name : Jason
last_name : Bourne
age : 27
*** Second type of function ***
first_name : Jason
last_name : Bourne
age : 27
*** Third type of function ***
first_name : Jason
last_name : Bourne
age : 29
*** Fourth type of function ***
first_name : Jason
last_name : two
age : 29
*** End of the program ***

 

Statements inside a function will not be executed unless function is called by name

Important note for passing arguments to function

  • list is always passed by reference
  • other variables are passed by value
  • Variables defined in function are local to that function

Types of function

Details seen above are user defined functions. In general functions can be of following types

  1. Built in functions
  2. Type conversion functions
  3. User defined functions

Let us look at other category of functions

Built-in functions

Python provides large number of built-in functions which we can use without defining them. These functions address common problems faced by most of the developers.

Here is simple example. It has two functions, len(), it returns left of the argument that is passed and second is print(). Yes, print is a functions, in fact this is a major change between python 2 and 3.

print("*** Start of the program ***")

length = len("techtrekking")
print("length : ", length)

print("*** End of the program ***")

Output

$ python3.6 funtions01.py
*** Start of the program ***
length : 12
*** End of the program ***

Few examples of python built-in functions

Function Purpose
abs() returns absolute value
max() returns maximum value from array passed
type() Returns the data type of the variable

Type Conversion Functions

These functions can convert data type of variable however not all data types can be changed. e.g. You can convert int to float and vice versa without any issue however if you try to convert character data type having actual characters it will throw some error however this function can be used to convert data type of variable having numeric value but character data type.

>>> int(2.1)
2
>>> int(aa)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'aa' is not defined
>>> int('10')
10
>>> 

Return Values and return Statements

When python function is called, it will do some calculations and return the result of the calculations using return statement. However sometimes you might call a function only to display something and you dont need anything return, in such cases, you can skip teh return statement.

If you don’t write the return statement a value “None” will be returned. In Python there is a value called None, which represents the absence of a value. None is the only value of the NoneType data type.  This is similar to undefined in JavaScript

 

lambda

A function is an object that is able to accept some sort of input, possibly modify it, and return some sort of output.
In Python, a lambda function is a one-line shorthand for function.
A lambda function can take any number of arguments, but can only have one expression.

>>> mylambda =lambda a: a+10
>>> print(mylambda(11))
21
>>>mylambda = lambda x, y : x * y
>>>print(mylambda(5, 6))
30
>>>mylambda = lambda x, y, z : x + y + x
>>>print(mylambda(5, 6, 2))

Due to characteristics of lambda, it can not have default values and it can not handle complex logic

Python : Data Types

Python has following datatypes

  • Numbers
    • int
    • float
    • complex
  • String
  • Collections
    • List
    • Tuple
    • Dictionary
Int
  • These are signed Integers.
  • They are positive or negative whole numbers with no decimal point.
  • Integers in Python 3 are of unlimited size.
  • Python 2 has two integer types – int and long. There is no ‘long integer’ in Python 3 anymore.
a = 22    # int
b = 3.14  # float
C = 1j   # complex
Float
  • Float represent real numbers and are written with a decimal point dividing the integer and the fractional parts.
  • Floats can be in scientific notation, with E or e indicating the power of 10
d =1.10 
e =1.0 
f =-35.59 
g = 2.5e2 = 2.5 x 100 = 250
Complex
  • Complex numbers are in the form a + bj, where a and b are floats and j (or J) represents the square root of -1 (This is an imaginary number).
  • The real part of the number is a, and the imaginary part is b.
  • Complex numbers are rarely used in Python programming.
p = 3+5j
q = 5j
r = -5j
Strings
  • Strings are the most widely used data types in Python.
  • Strings are created by enclosing characters in quotes either ‘single’ or “double”.
  • Python treats single quotes the same as double quotes.
var1 = 'hello' 
var2 = 'this is a String created @ 11:30 AM' 
#String operations
>>> 'hello'+'Jason'
'helloJason' 
>>> 'hello'*5
'hellohellohellohellohello' 
>>> 'hello'[1]
'e' 
>>> a[1:]
'ello' 
>>> a='hello'
>>> a[1]
'e' 
>>> 'e' in a
True 
>>> 'E' in a
False

Triple quotes can be used for providing multi line string character input

Python collections

Collection data can be stored in 4 different data types in python. Let us look at these one by one

lists
  • A list is a collection which is ordered and changeable.
  • Lists are written as comma-separated values (items) between square brackets.
  • Items in a list need not be of the same type
  • A list is a collection which is ordered and changeable.

Following are some of the examples of Lits (define, access, append, merge.

list1 = [1,2,3,4,5,6,7,8] 
list2 = ['one','two','three'] 
list3 = [1, 'two',3,'four'] 
#accessing list
>>> list1[0]
1 
>>> list1[-2]
7 
>>> len(list1)
9 
>>> 3 in list1
True 
>>> 1 in list1
False 
#updating list
list1[0] = 11 
print(list1[0]) 
11 
#slicing the list
>>> list1 = [0,1,2,3,4,5,6,7,8]
>>> list1[1:3]
[1, 2]
>>> list1[:3]
[0, 1, 2]
>>> list1[1:]
[1, 2, 3, 4, 5, 6, 7, 8]
# deleting list element
>>> list1
[0, 0, 1, 2, 3, 4, 5, 6, 7, 8]
>>> del list1[2]
>>> list1
[0, 0, 2, 3, 4, 5, 6, 7, 8]
# Methods for list
>>> list1
[0, 0, 2, 3, 4, 5, 6, 7, 8]
>>> list1.append(99)
>>> list1
[0, 0, 2, 3, 4, 5, 6, 7, 8, 99]
>>> list1.reverse()
>>> list1
[99, 8, 7, 6, 5, 4, 3, 2, 0, 0]
>>> list1.sort()
>>> list1
[0, 0, 2, 3, 4, 5, 6, 7, 8, 99]
# destructive read from the list
>>> list1.pop(1)
0 
>>> list1
[0, 2, 3, 4, 5, 6, 7, 8, 99]
>>> list1.pop()
99 
>>> list1
[0, 2, 3, 4, 5, 6, 7, 8]

Important List methods

Method Use
append() Adds element at the end
clear() removes all elements from list
count() Returns number of elements with specified value
pop() Removes element at specified position
reverse() Reverses the list
sort() Sorts list
Tuples
  • A tuple is a collection which is ordered and unchangeable.
  • The difference between the tuples and the lists is that the tuples cannot be changed unlike lists.
  • Tuples use parentheses, whereas lists use square brackets
thistuple = ("pune", "mumbai", "satara")
thistuple = tuple(("mango", "apple", "kivi")) # double round-brackets

 

Set
  • A set is a collection which is unordered and unindexed.
thisset = {"apple", "banana", "cherry"}
thisset = set(("apple", "banana", "cherry"))
Dictionary
  • A dictionary is a collection which is unordered, changeable and indexed.
  • Dictionary items are enclosed in curly braces.
  • Each items are separated by commas
  • Each key is separated from its value by a colon (:)
  • The values of a dictionary can be of any type, but the keys must be of an immutable data type such as strings, numbers, or tuple.
#Creating dictionary
firstDict = {'fname':'Jason','sname':'Bourne'} 
#Accessing dictionary
dict['fname'] 
#Updating dictionary element
dict['fname'] = 'Agent 007'; 
#deleting certain element
del firstDict['Name']
#Another way of creating 
dictthisdict = dict(apple="red", banana="yellow", cherry="pink")
#remove element
del(thisdict["banana"])
print(len(thisdict))

 

how Install pandas with Python3.6.4 on ubuntu 16.04

When you have multiple python versions installed on your ubuntu, installing modles becomes kind of complex. Also, you need to install python modules separately for each version. Meaning, pandas installed with python 2.7 will not be available if you want to run script in python 3.6.

Lets check this. I had installed pandas with python2.7, I wanted to test if pandas installed for python2.7.x will work for python3.6.x as well ?


$ python3.6
Python 3.6.4 (default, Jan 13 2018, 12:02:51)  
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.  
>>> import pandas as pd
Traceback (most recent call last):  
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'  
>>> 

As you can see, pandas installed with python2.7.x does not work for python3.6.4.

I thought lets try to reinstall pandas but it said, pandas is already installed


$ pip install pandas
Requirement already satisfied: pandas in /usr/local/lib/python2.7/dist-packages  
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python2.7/dist-packages (from pandas)  
Requirement already satisfied: python-dateutil in /usr/local/lib/python2.7/dist-packages (from pandas)  
Requirement already satisfied: numpy>=1.7.0 in /usr/local/lib/python2.7/dist-packages (from pandas)  
Requirement already satisfied: six>=1.5 in /usr/local/lib/python2.7/dist-packages (from python-dateutil->pandas)  

my pip version is


$ pip -V
pip 9.0.1 from /home/conquistador/.local/lib/python2.7/site-packages (python 2.7)  

Ok so pip installed on my machine is of python2.7.x

Packages for Python 2 and Python 3 are installed separately, and installing in one version won’t make a package available to the other.

Finally i found a way to install pandas with python3.6.4


$ sudo python3.6 -m pip install pandas
The directory '/home/conquistador/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.  
The directory '/home/conquistador/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.  
Collecting pandas  
  Downloading pandas-0.22.0-cp36-cp36m-manylinux1_x86_64.whl (26.2MB)
    100% |████████████████████████████████| 26.3MB 44kB/s 
Collecting numpy>=1.9.0 (from pandas)  
  Downloading numpy-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (17.2MB)
    100% |████████████████████████████████| 17.2MB 58kB/s 
Collecting python-dateutil>=2 (from pandas)  
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
    100% |████████████████████████████████| 194kB 193kB/s 
Collecting pytz>=2011k (from pandas)  
  Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
    100% |████████████████████████████████| 512kB 217kB/s 
Collecting six>=1.5 (from python-dateutil>=2->pandas)  
  Downloading six-1.11.0-py2.py3-none-any.whl
Installing collected packages: numpy, six, python-dateutil, pytz, pandas  
Successfully installed numpy-1.14.0 pandas-0.22.0 python-dateutil-2.6.1 pytz-2017.3 six-1.11.0  

and here is the confirmation that python is installed.


$ python3.6
Python 3.6.4 (default, Jan 13 2018, 12:02:51)  
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.  
>>> import pandas as pd
>>> 

Python : Variables, Operators, Expressions and Statements

Variables

As the name implies, a variable is something which can change. A variable is a way of referring to a memory location used by a computer program. A variable is a symbolic name for this physical location. This memory location contains values, like numbers, text or more complicated types.

Please note important point for Python. It does not have a command for declaring a variable. A variable is created the moment you first assign a value to it.

first_Name = 'your name'
y ='techTrekk'
password1 ='password string'
n00b = 'ac'
un_der_scores = 'sample text'
Variables Naming convention
  • Variables names must start with a letter or an underscore but not with number
  • A variable name cannot start with a number
  • A variable name can contain alpha-numeric characters and underscores (A-z, 0-9, and _ )
  • Names are case sensitive.
    casesensitive, CASESENSITIVE, case_sensitive and Case_Sensitive are each a different variable.
Variable naming

See Python PEP 8.

Function names should be lowercase, with words separated by underscores as necessary to improve readability. mixedCase is allowed only in contexts where that’s already the prevailing style

I personally Like camelCase/mixedCase used in JavaScript and hence I prefer to use the same in Python but you can take a call by yourself but make sure you stick with one style to maintain consistency.

Variable name chosen should be “mnemonic”, meaning, by variable name itself, reader should be able to understand what this variable is for. e.g. It is not convenient to store first name in first_name instead of abx123 or abc

Operators

Operators are special symbols that represent computations like additions and subtraction. The value these operators are applied to are called as operands.

Activity Symbol
Addition +
Subtraction
Multiplication *
Division /
Quotient //
Reminder %

Python follows PEDMAS rule

Expressions

An express is combination of values, variable and operators

>>> 1+4
5  

Statements

A statement is used to form the sequence of a program (e.g. if-then , while-do statements).

A statement can be simple or complex and can contain 0 or more expression

Comments

As program becomes bigger, it becomes difficult to track which part is doing what. For this reason, it is a good idea to add notes to the program. Notes can be added as a comments. Comment does not get execute or compile in program.  Comments in python can be added two ways.

Hash Sign (# )

Anything mention after hash symbol(#) is considered as comment. This is limited to the same line where this symbol is added

MULTI LINE comments

Multi line comments can be added by triple double inverted commas. To close the comment, triple inverted comma’s needs to be added again. This method is primarily used for function clarification.

How to install python 2.7 or 3.5 or 3.6 on Ubuntu

Ubuntu 18.04 as well as Ubuntu 17.10 come with Python 3.6 pre-installed, which is not the case for older Ubuntu versions. In this article, we will explain how to install latest Python 3.6 in Ubuntu 14.04, 16.04, 16.10 and 17.04.

Ubuntu 14.04 & 16.04

Ubuntu 14.04 and 16.04 ship in with Python 2.7 and Python 3.5. To install latest Python 3.6 version, we need to use PPA “deadsnakes”.

$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt update
$ sudo apt install python3.6

Ubuntu 16.10 and 17.04

For Ubuntu 16.10 and 17.04, Python 3.6 package is in the Universe repository and can be installed very easily using following commands.

$ sudo apt update
$ sudo apt install python3.6

Check current versions.

If you want to check current versions, try following commands

$ python -V
Python 2.7.12
$ python3 -V
Python 3.5.2
$ python3.6 -V
Python 3.6.4