file processing Archives - Technology Trekking

30/03/201930/03/2019

How to find all files in a directory with specific extension using Python

Whenever we need to check if file with specific extension exists in a directory or get list of files with specific extension from a directory, it can be done using python. Here is simple program for doing the same.

There are three ways to do this using python

os.walk
os.listdir
glob.glob

All three methods are demonstrated in below program

################################################################################################

# name: file_with_specific_extension_in_directory.py

# desc: 

# date: 2019-03-30

# Author: conquistadorjd

################################################################################################

import os

import glob

print("*** Program Started ***")

dir_name = 'C:/sample/'

matches = []

for root, dirnames, filenames in os.walk(dir_name):

	for filename in filenames:

		# if filename.endswith(('.txt', '.MOV', '.avi', '.mpg')):  ## Checking for multiple extenstions

		if filename.endswith('.txt'):

			matches.append(filename)

print("List of Files using os.walk:", matches)

matches = []

filenames = os.listdir(dir_name)

for filename in filenames:

	# if filename.endswith(('.txt', '.MOV', '.avi', '.mpg')):     ## Checking for multiple extenstions

	if filename.endswith('.txt'):

		matches.append(filename)

print("List of Files using listdir:", matches)

# filenames =  glob.glob(dir_name+'*.txt')		## This will return file name with complete path name as well

matches = [os.path.basename(x) for x in glob.glob(dir_name+'*.txt')] ## This will return only file name

print("List of Files using glob   :", matches)

print("*** Program Completed ***")

view raw
file_with_specific_extension_in_directory.py
hosted with ❤ by GitHub

Output of the program

*** Program Started ***
List of Files using os.walk: ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
List of Files using listdir: ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
List of Files using glob : ['file3.txt', 'list_to_file_in_directory.txt', 'test.txt']
*** Program Completed ***

30/03/2019

How to count number of files in directory using Python

Counting number of files using python in specific folder can be done using multiple ways. Some of these are mentioned below

################################################################################################

# name: file_count_in_directory.py

# desc: 

# date: 2019-03-30

# Author: conquistadorjd

################################################################################################

import os

import glob

print("*** Program Started ***")

dir_name = 'C:/sample/'

file_count = sum([len(files) for r, d, files in os.walk(dir_name)])

print('Number of Files using os.walk          :', file_count)

file_count = os.listdir(dir_name)

print("Number of Files using listdir method#1 :", len(file_count))

file_count =  len([name for name in os.listdir(dir_name) if os.path.isfile(os.path.join(dir_name, name))])

print("Number of Files using listdir method#2 :", file_count)

file_count =  [name for name in os.listdir(dir_name) if os.path.isfile(os.path.join(dir_name, name))]

print("Number of Files using listdir method#3 :", len(file_count))

file_count =  glob.glob(dir_name+'*.*')

print("Number of Files using glob             :", len(file_count))

print("*** Program Completed ***")

view raw
file_count_in_directory.py
hosted with ❤ by GitHub

Here is the output

*** Program Started ***
Number of Files using os.walk : 7
Number of Files using listdir method#1 : 8
Number of Files using listdir method#2 : 7
Number of Files using listdir method#3 : 7
Number of Files using glob : 7
*** Program Completed ***

As you can see Number of Files using listdir method#1 are different, this is because, its counting a folder as a file.

11/02/2019

How to check file size in Python

Many times while doing file processing python, we need to know file size in bytes/KBs/MBs. You can get file size using multiple methods, following are two simple methods to get file size using os module.

I have run this file for two inputs,one is image and another is empty file.

################################################################################################

# name: file_size.py

# desc: Getting gile size using python

# date: 2019-02-10

# Author: conquistadorjd

################################################################################################

import io, os

print('*** Program Started ***')

image_path_input = '/home/conquistador/code/github/python-01-utilities/file/input/'

image_name_input =  'file_size-01.jpg' #'file_size-02'

if os.stat(image_path_input + image_name_input).st_size ==0 :

	print('Input file is empty')

else :

	print('Input file is not empty')

print('File size (in Bytes) : ',os.stat(image_path_input + image_name_input).st_size)

print('File size (in Bytes) : ',os.path.getsize (image_path_input+image_name_input))

image_name_input =  'file_size-02'

if os.stat(image_path_input + image_name_input).st_size ==0 :

	print('Input file is empty')

else :

	print('Input file is not empty')

print('File size (in Bytes) : ',os.stat(image_path_input + image_name_input).st_size)

print('File size (in Bytes) : ',os.path.getsize (image_path_input+image_name_input))

print('*** Program Ended ***')

view raw
file_size.py
hosted with ❤ by GitHub

Here is the output of the program

$ python3.6 file_size.py 
*** Program Started ***
Input file is not empty
File size (in Bytes) : 147162
File size (in Bytes) : 147162
Input file is empty
File size (in Bytes) : 0
File size (in Bytes) : 0
*** Program Ended ***

Actual file size

I have added a check to see if file size is empty, you might need to check file size before doing any processing on files.