Python Archives - Technology Trekking

15/05/201923/05/2019

Windows 10 | ‘pip’ is not recognized as an internal or external command

After installing python on windows 10, I wanted to install few more packages such as numpy, scipy, pandas, however I got following error


C:\>pip install pandas
'pip' is not recognized as an internal or external command,
operable program or batch file.

pip script is present in following folder


C:\Users\USERNAME\AppData\Local\Programs\Python\Python36\Scripts
OR
C:\Python\Python36\Scripts

To make pip work, we need to add this to path variable.


C:\>path %path%;C:\Users\USERNAME\AppData\Local\Programs\Python\Python36\Scripts

outcome after adding pip to path


C:\>pip install pandas
Collecting pandas
Downloading https://files.pythonhosted.org/packages/d0/4e/9db3468e504ac9aeadb37eb32bcf0a74d063d24ad1471104bd8a7ba20c97/pandas-0.24.2-cp36-cp36m-win_amd64.whl (8.8MB)
100% |████████████████████████████████| 8.8MB 36kB/s
Collecting python-dateutil>=2.5.0 (from pandas)
Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
100% |████████████████████████████████| 235kB 67kB/s
Collecting pytz>=2011k (from pandas)
Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
100% |████████████████████████████████| 512kB 64kB/s
Collecting numpy>=1.12.0 (from pandas)
Downloading https://files.pythonhosted.org/packages/2e/11/f006363050b24fb19a235e5efd219e7ac549398d531110d80b8f2ba3a909/numpy-1.16.3-cp36-cp36m-win_amd64.whl (11.9MB)
100% |████████████████████████████████| 11.9MB 39kB/s
Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas)
Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, pytz, numpy, pandas
Successfully installed numpy-1.16.3 pandas-0.24.2 python-dateutil-2.8.0 pytz-2019.1 six-1.12.0
You are using pip version 9.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

15/05/2019

Windows 10 | ‘python’ is not recognized as an internal or external command

After installing python on Windows 10, I got following error.

C:\>python
'python' is not recognized as an internal or external command,
operable program or batch file.

To my surprise py works fine but python does not work.

C:\>py
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

To make python works, we need to set up path. For Windows 10, python will be at following folder

C:\Users\USERNAME\AppData\Local\Programs\Python\Python36
OR
C:\Python\Python36

Use following command to set up path


C:\>path %path%;C:\Users\USERNAME\AppData\Local\Programs\Python\Python36

Once this is done, you can check the path variable using below command


C:\>echo %path%

Now let us try using python

C:\>python
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

30/03/201930/03/2019

How to find all files in a directory with specific extension using Python

Whenever we need to check if file with specific extension exists in a directory or get list of files with specific extension from a directory, it can be done using python. Here is simple program for doing the same.

There are three ways to do this using python

os.walk
os.listdir
glob.glob

All three methods are demonstrated in below program

Output of the program

*** Program Started ***
List of Files using os.walk: ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
List of Files using listdir: ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
List of Files using glob : ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
*** Program Completed ***

30/03/2019

How to count number of files in directory using Python

Counting number of files using python in specific folder can be done using multiple ways. Some of these are mentioned below

Here is the output

*** Program Started ***
Number of Files using os.walk : 7
Number of Files using listdir method#1 : 8
Number of Files using listdir method#2 : 7
Number of Files using listdir method#3 : 7
Number of Files using glob : 7
*** Program Completed ***

As you can see Number of Files using listdir method#1 are different, this is because, its counting a folder as a file.

13/02/2019

How to Check if a File Exists using Python

Recently I was working on file generation using python, before generating any file, I had to check if file exists or not, to check if file exists or not, I had to find a way to do in in program. As usual, this task is very easy using python.

Checking if file exists or not can be done in multiple ways using python, here is one using “os” module

os.path module has functions such as isfile, isdir and exists which helps us check if file or directory exists or not.

Here is the output

$ python3.6 file_exists_01.py 
file_exists : True
file_exists : False
dir_exists : False
dir_exists : True
dir_exists : False
exists : True
exists : True

If you use isfile() on directory, outcome will be False, you need to use isfile or isdir as per requirement. Alternatively, you can use exists function as well, this returns True if input file or directory path is valid. Please refer to os.path documentation for further details.

Here is another way to check if file exists or not using pathlib module.

Output is

$ python3.6 file_exists_02.py 
var : False
var : True
var : True
var : False
var : True
var : True

Both the modules have similar features, you can choose whichever is convenient to you.

11/02/201911/02/2019

How to read image using Pillow, Python and get image attributes

Pillow is the friendly PIL fork. PIL is the Python Imaging Library. This is the first article in series of image processing articles using python.

In this article we will see how to read file using pillow and get basic attributes.

Here is simplest program to read image file using pillow and get basic attributes

Output of this program

*** Program Started ***
im object: <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=1920x1285 at 0x7F7FB79F6518>
format : JPEG
size : (1920, 1285)
mode : RGB
filename : /home/conquistador/code/github/python-01-utilities/image/input/01_read_image.jpg
width : 1920
height : 1285
info : {'jfif': 257, 'jfif_version': (1, 1), 'jfif_unit': 0, 'jfif_density': (1, 1), 'progressive': 1, 'progression': 1}
*** Program Ended ***

11/02/2019

How to check file size in Python

Many times while doing file processing python, we need to know file size in bytes/KBs/MBs. You can get file size using multiple methods, following are two simple methods to get file size using os module.

I have run this file for two inputs,one is image and another is empty file.

Here is the output of the program

$ python3.6 file_size.py 
*** Program Started ***
Input file is not empty
File size (in Bytes) : 147162
File size (in Bytes) : 147162
Input file is empty
File size (in Bytes) : 0
File size (in Bytes) : 0
*** Program Ended ***

Actual file size

I have added a check to see if file size is empty, you might need to check file size before doing any processing on files.

02/02/2019

Generating data for Linear Regression using NumPy

We have already seen how to generate random numbers in previous article, here we will have a look at how to generate data in specific format for linear regression.

To test data for linear regression, we will need a data which has somewhat linear relationship and one set of random data. Please find below code to generate data having some linear relation and random data using Python and NumPy. I have provided graphs which will help you understand the data created by using these programs.

Data with Linear Trend for Linear Regression

Data without any Trend for Linear Regression

You can use this as an input data while training your model.

06/12/201811/12/2018

Python NumPy Tutorial : Getting started with NumPy

NumPy is BSD licensed fundamental package for scientific computing with Python. Most important feature is a powerful N-dimensional array object and sophisticated (broadcasting) functions. It also has useful linear algebra, Fourier transform, and random number capabilities

NumPy can also be used as an efficient multi-dimensional container of generic data. One of the most important feature is Arbitrary data-types.

Importing numpy and creating, accessing and modifying array

>>> import numpy as np
>>> a=np.array([1,2,3,4,5,6])
>>> a
array([1, 2, 3, 4, 5, 6])
>>> type(a)
<class 'numpy.ndarray'>
>>> a[1]
2
>>> a[1]=9
>>> a
array([1, 9, 3, 4, 5, 6])
>>> b = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
>>> b
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])

Numpy Properties

>>> a.shape
(6,)
>>> b.shape
(3, 4)
>>> a.size
6
>>> b.size
12
>>> a.data
<memory at 0x7faf92bb0dc8>
>>> b.data
<memory at 0x7faf92bbba68>
>>> a.dtype
dtype('int64')
>>> b.dtype
dtype('int64')

Mathematical operations on numpy numpyarray

>>> a
array([1, 2, 3, 4, 5, 6])
>>> b
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
>>> c
array([ 3, 6, 9, 12, 15, 18])
>>> a+c
array([ 4, 8, 12, 16, 20, 24])
>>> a-c
array([ -2, -4, -6, -8, -10, -12])
>>> a*c
array([ 3, 12, 27, 48, 75, 108])
>>> a/c
array([0.33333333, 0.33333333, 0.33333333, 0.33333333, 0.33333333,
0.33333333])
>>> np.sqrt(a)
array([1. , 1.41421356, 1.73205081, 2. , 2.23606798,
2.44948974])
>>> np.sum(b,axis=0)
array([15, 18, 21, 24])
>>> np.sum(b,axis=1)
array([10, 26, 42])
>>> b
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])
>>> b.T
array([[ 1,  5,  9],
       [ 2,  6, 10],
       [ 3,  7, 11],
       [ 4,  8, 12]])

Working on the arrays

>>> b
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
>>> b.reshape(4,3)
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]])
>>> b
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
>>> b.resize(4,3)
>>> b
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]])
>>> b[0:2,1:2] #[row range,column range]
array([[2],
       [5]])