Windows 10 | ‘pip’ is not recognized as an internal or external command

After installing python on windows 10, I wanted to install few more packages such as numpy, scipy, pandas, however I got following error


C:\>pip install pandas
'pip' is not recognized as an internal or external command,
operable program or batch file.

pip script is present in following folder


C:\Users\USERNAME\AppData\Local\Programs\Python\Python36\Scripts
OR
C:\Python\Python36\Scripts

To make pip work, we need to add this to path variable.


C:\>path %path%;C:\Users\USERNAME\AppData\Local\Programs\Python\Python36\Scripts

outcome after adding pip to path


C:\>pip install pandas
Collecting pandas
Downloading https://files.pythonhosted.org/packages/d0/4e/9db3468e504ac9aeadb37eb32bcf0a74d063d24ad1471104bd8a7ba20c97/pandas-0.24.2-cp36-cp36m-win_amd64.whl (8.8MB)
100% |████████████████████████████████| 8.8MB 36kB/s
Collecting python-dateutil>=2.5.0 (from pandas)
Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
100% |████████████████████████████████| 235kB 67kB/s
Collecting pytz>=2011k (from pandas)
Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
100% |████████████████████████████████| 512kB 64kB/s
Collecting numpy>=1.12.0 (from pandas)
Downloading https://files.pythonhosted.org/packages/2e/11/f006363050b24fb19a235e5efd219e7ac549398d531110d80b8f2ba3a909/numpy-1.16.3-cp36-cp36m-win_amd64.whl (11.9MB)
100% |████████████████████████████████| 11.9MB 39kB/s
Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas)
Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, pytz, numpy, pandas
Successfully installed numpy-1.16.3 pandas-0.24.2 python-dateutil-2.8.0 pytz-2019.1 six-1.12.0
You are using pip version 9.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Windows 10 | ‘python’ is not recognized as an internal or external command

After installing python on Windows 10, I got following error.

C:\>python
'python' is not recognized as an internal or external command,
operable program or batch file.

To my surprise py works fine but python does not work.

C:\>py
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 

To make python works, we need to set up path. For Windows 10, python will be at following folder

C:\Users\USERNAME\AppData\Local\Programs\Python\Python36
OR
C:\Python\Python36

Use following command to set up path


C:\>path %path%;C:\Users\USERNAME\AppData\Local\Programs\Python\Python36

Once this is done, you can check the path variable using below command


C:\>echo %path%

Now let us try using python

C:\>python
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

How to find all files in a directory with specific extension using Python

Whenever we need to check if file with specific extension exists in a directory or get list of files with specific extension from a directory, it can be done using python. Here is simple program for doing the same.

There are three ways to do this using python

  1. os.walk
  2. os.listdir
  3. glob.glob

All three methods are demonstrated in below program

Output of the program

*** Program Started ***
List of Files using os.walk: ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
List of Files using listdir: ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
List of Files using glob : ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
*** Program Completed ***

 

How to count number of files in directory using Python

Counting number of files using python in specific folder can be done using multiple ways.  Some of these are mentioned below

Here is the output

*** Program Started ***
Number of Files using os.walk : 7
Number of Files using listdir method#1 : 8
Number of Files using listdir method#2 : 7
Number of Files using listdir method#3 : 7
Number of Files using glob : 7
*** Program Completed ***

As you can see Number of Files using listdir method#1 are different, this is because, its counting a folder as a file.

 

How to write a list to a file and read a list from file using Python

Many a times we need to store list in a file for later usage, in such cases, its better to store list in a file and read this file into list whenever we need to use. Python has very easy methods for achieving this.

 

Output of the program

*** Program Started ***
lines : ['one', 'two', 'three', 'four', '']
type of lines : <class 'list'>
line : one
line : two
line : three
line : four
line :
*** Program Completed ***

file generated is as below

Here is another version of same program

output and file generated is same except file opening closing styles are different.

Getting started with Time Series Forecasting with Prophet

Facebook prophet is simplest way to get started with time series analysis with python. Please refer to this post to know how to install prophet on Ubuntu.

We will have a look at official example with one complete program.

Here is the output on terminal

$ python3.6 01_fbprophet_getting_started.py 
*** Program Started ***
ds y
0 2007-12-10 9.590761
1 2007-12-11 8.519590
2 2007-12-12 8.183677
3 2007-12-13 8.072467
4 2007-12-14 7.893572
INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Initial log joint probability = -19.4685
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
99 7975.37 0.00149529 224.247 1 1 128 
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
186 7992.27 5.72063e-05 157.088 5.678e-07 0.001 261 LS failed, Hessian reset 
199 7993.26 0.000312701 314.644 0.1004 1 277 
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
299 7997.05 0.0015387 170.701 1 1 408 
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
321 7998.61 0.00020668 308.573 1.22e-06 0.001 478 LS failed, Hessian reset 
369 8000.52 2.98767e-05 97.9518 2.746e-07 0.001 566 LS failed, Hessian reset 
399 8000.98 0.000153501 134.602 0.7945 0.7945 601 
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
457 8001.99 0.000276407 292.083 2.159e-06 0.001 718 LS failed, Hessian reset 
499 8002.58 0.000699641 197.602 1 1 770 
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
550 8003.07 5.79234e-05 181.032 3.403e-07 0.001 874 LS failed, Hessian reset 
599 8003.43 0.000218596 78.2273 0.7213 0.7213 928 
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
695 8004.08 3.66526e-05 116.76 2.994e-07 0.001 1095 LS failed, Hessian reset 
699 8004.11 0.000537041 111.615 1 1 1099 
Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes 
788 8004.7 3.21305e-06 76.4964 4.987e-08 0.001 1259 LS failed, Hessian reset 
797 8004.7 6.1457e-07 61.1166 0.6741 0.6741 1270 
Optimization terminated normally: 
Convergence detected: relative gradient magnitude is below tolerance
type of m <class 'fbprophet.forecaster.Prophet'>
type of future <class 'pandas.core.frame.DataFrame'>
type of forecast <class 'pandas.core.frame.DataFrame'>
*** Program Completed ***

 

 

Let us try to run the same by using column name as date and value instead of ds and y. When I tried this, I got following error.

 

$ python3.6 01_fbprophet_getting_started.py 
*** Program Started ***
time value
0 2007-12-10 9.590761
1 2007-12-11 8.519590
2 2007-12-12 8.183677
3 2007-12-13 8.072467
4 2007-12-14 7.893572
Traceback (most recent call last):
File "01_fbprophet_getting_started.py", line 19, in <module>
m.fit(df)
File "/usr/local/lib/python3.6/site-packages/fbprophet/forecaster.py", line 1016, in fit
"Dataframe must have columns 'ds' and 'y' with the dates and "
ValueError: Dataframe must have columns 'ds' and 'y' with the dates and values respectively.

Daily Seasonality

You might have observed following message

INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.

To get rid of this message, add daily_seasonality=True prophet object. It will look like below.

m = Prophet(daily_seasonality=True)

 

How to Install facebook prophet library Ubuntu

Prophet is forecasting librabry developed by facebook, it has been open sourced by facebook. Following is the simple command to install it.

 

$ sudo python3.6 -m pip install fbprophet
Collecting fbprophet
Downloading https://files.pythonhosted.org/packages/9b/a1/fef4ce00acbc28e75c0d33f60c9777527c4295656903b00ac4c9525cef7f/fbprophet-0.4.post2.tar.gz (45kB)
100% |████████████████████████████████| 51kB 354kB/s 
Collecting Cython>=0.22 (from fbprophet)
Downloading https://files.pythonhosted.org/packages/e1/fd/711507fa396064bf716493861d6955af45369d2c470548e34af20b79d4d4/Cython-0.29.6-cp36-cp36m-manylinux1_x86_64.whl (2.1MB)
100% |████████████████████████████████| 2.1MB 316kB/s 
Collecting pystan>=2.14 (from fbprophet)
Downloading https://files.pythonhosted.org/packages/17/77/dd86797a7e7fccca117233c6d50cc171e0c2b2f5a0cd2a8d9753ee09b7be/pystan-2.18.1.0-cp36-cp36m-manylinux1_x86_64.whl (50.0MB)
100% |████████████████████████████████| 50.0MB 312kB/s 
Requirement already satisfied: numpy>=1.10.0 in /usr/local/lib/python3.6/site-packages (from fbprophet) (1.14.0)
Requirement already satisfied: pandas>=0.20.1 in /usr/local/lib/python3.6/site-packages (from fbprophet) (0.22.0)
Requirement already satisfied: matplotlib>=2.0.0 in /usr/local/lib/python3.6/site-packages (from fbprophet) (2.1.2)
Collecting lunardate>=0.1.5 (from fbprophet)
Downloading https://files.pythonhosted.org/packages/4e/7e/377a3cbba646ec0cf79433ef858881d809a3b87eb887b0901cb83c66a758/lunardate-0.2.0-py3-none-any.whl
Collecting convertdate>=2.1.2 (from fbprophet)
Downloading https://files.pythonhosted.org/packages/74/83/d0fa07078f4d4ae473a89d7d521aafc66d82641ea0af0ef04a47052e8f17/convertdate-2.1.3-py2.py3-none-any.whl
Collecting holidays>=0.9.5 (from fbprophet)
Downloading https://files.pythonhosted.org/packages/16/09/c882bee98acfa310933b654697405260ec7657c78430a14e785ef0f1314b/holidays-0.9.10.tar.gz (73kB)
100% |████████████████████████████████| 81kB 370kB/s 
Collecting setuptools-git>=1.2 (from fbprophet)
Downloading https://files.pythonhosted.org/packages/05/97/dd99fa9c0d9627a7b3c103a00f1566d8193aca8d473884ed258cca82b06f/setuptools_git-1.2-py2.py3-none-any.whl
Requirement already satisfied: python-dateutil>=2 in /usr/local/lib/python3.6/site-packages (from pandas>=0.20.1->fbprophet) (2.6.1)
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/site-packages (from pandas>=0.20.1->fbprophet) (2017.3)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/site-packages (from matplotlib>=2.0.0->fbprophet) (0.10.0)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.6/site-packages (from matplotlib>=2.0.0->fbprophet) (1.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/site-packages (from matplotlib>=2.0.0->fbprophet) (2.2.0)
Collecting ephem<3.8,>=3.7.5.3 (from convertdate>=2.1.2->fbprophet)
Downloading https://files.pythonhosted.org/packages/c3/2c/9e1a815add6c222a0d4bf7c644e095471a934a39bc90c201f9550a8f7f14/ephem-3.7.6.0.tar.gz (739kB)
100% |████████████████████████████████| 747kB 246kB/s 
Installing collected packages: Cython, pystan, lunardate, ephem, convertdate, holidays, setuptools-git, fbprophet
Running setup.py install for ephem ... done
Running setup.py install for holidays ... done
Running setup.py install for fbprophet ... done
Successfully installed Cython-0.29.6 convertdate-2.1.3 ephem-3.7.6.0 fbprophet-0.4.post2 holidays-0.9.10 lunardate-0.2.0 pystan-2.18.1.0 setuptools-git-1.2
You are using pip version 10.0.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

 

It takes some time considering number of dependencies. You can verify installation by login to python3.6 console, importing module and checking it.

$ python3.6
Python 3.6.4 (default, Jan 13 2018, 12:02:51) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fbprophet
>>> fbprophet.__version__
'0.4'
>>>

Please note that while using facebook prophet, correct way to import library is as below

from fbprophet import Prophet

How to Handle Broken Images

If your website is displaying many images and some of their are from different sources, eventually you will run into a situation when some of the images no longer exists. They are either move to different path or got deleted altogether.

This is what you will start seeing something like below in your website.

This is very annoying to users and it needs to be fixed on topmost priority.

Here are some of the ways to fix broken images.

Remove img tag itself.

This is the simplest and most effective way of doing it.

<img src="Error.src" onerror="this.style.display='none'"/>

Another way of doing same thing

<img src="Error.src" onerror="this.parentNode.removeChild(this);">

This will remove img tag itself and user will never know if any image was to be displayed here, Only issue being, if you have referred image in your content, user might get confused.

Display Backup Image

Implementing this is also very easy but for various reasons I dont link this option.

<img src="Error.src" onerror="this.src='http://example.com/existent-image.jpg'"/>

This option remove awkwardness looking broken image picture, however it will be replace it will backup image. Humanly it’s not possible to set up back up image for each image so mostly, there will be one backup image for all images. This reduces the awkwardness of broken image but it replaces it with backup image. This highlights the broken image issue and if image is referred in content, it does not help user with anything.

How to Create Custom Query for WordPress

WordPress has a default query which fetches posts and display it in descending manner. If you are working on any WordPress customisation, you would come across a requirement to fetch WordPress posts in customised manner such as Fetching last 10 posts, Fetching post having specific category or tag.

Fortunately, we dont have to write MySQL level queries to get required data (and that what makes WordPress special), we can call a WP_Query function by passing required fields as input, we can get required data. This logic has two parts.

  1. Query : The query will select WordPress posts from the MySQL database based on our instructions.
  2. Loop : It will output the post information on the page. Each instance of output array will have attributes such as post thumbnail, title, date, and author etc.

Here is simple structure to help you understand custom WP_Query

// The Query
$the_query = new WP_Query( $args );

// The Loop
if ( $the_query->have_posts() ) {
    echo '<ul>';
    while ( $the_query->have_posts() ) {
        $the_query->the_post();
        echo '<li>' . get_the_title() . '</li>';
    }
    echo '</ul>';
    /* Restore original Post Data */
    wp_reset_postdata();
} else {
    // no posts found
}

 

You can customized the query as per your requirement.

<?php

// Creating array
$args = array( 
'posts_per_page' => 15, 
'orderby' => 'date', 
'order' ='ASC'
);
// Initiate the custom query
$custom_query = new WP_Query( $args );

?>

By Default, it will search for posts, if you want to search for specific custom post type, you can simply add a parameter in array

<?php

// Creating array
$args = array( 
'post_type' => 'vehicle',
'posts_per_page' => 15, 
'orderby' => 'date', 
'order' ='ASC'
);
// Initiate the custom query
$custom_query = new WP_Query( $args );

?>

How to Check if a File Exists using Python

Recently I was working on file generation using python, before generating any file, I had to check if file exists or not, to check if file exists or not, I had to find a way to do in in program. As usual, this task is very easy using python.

Checking if file exists or not can be done in multiple ways using python, here is one using “os” module

os.path module has functions such as isfile, isdir and exists which helps us check if file or directory exists or not.

Here is the output

$ python3.6 file_exists_01.py 
file_exists : True
file_exists : False
dir_exists : False
dir_exists : True
dir_exists : False
exists : True
exists : True

If you use isfile() on directory, outcome will be False, you need to use isfile or isdir as per requirement. Alternatively, you can use exists function as well, this returns True if input file or directory path is valid. Please refer to os.path documentation for further details.

Here is another way to check if file exists or not using pathlib module.

Output is

$ python3.6 file_exists_02.py 
var : False
var : True
var : True
var : False
var : True
var : True

Both the modules have similar features, you can choose whichever is convenient to you.