Python Implementation of Data Preprocessing to Fill Missing Values Example

1、Given a dataset noise-data-1.txt, this dataset contains a large number of missing values (spaces, incomplete values, etc.). Use 'global constants', 'mean' or 'median' to fill in missing values.

noise-data-1.txt：

5.1 3.5 1.4 0.2
4.9 3 1.4 0.2
4.7 3.2 1.3 0.2
4.6 3.1 1.5 0.2
5 3.6 1.4 0.2
5.4 3.9 1.7 0.4
4.6 3.4 1.4 0.3
5 3.4 1.5 0.2
4.4 2.9 1.4 0.2
4.9 -3.1 1.5 0.1
5.4 3.7 1.5 0.2
4.8 3.4 1.6 0.2
4.8 3 -1.4 0.1
4.3 3 1.1 0.1
5.8 4 1.2 0.2
5.7 4.4 1.5 0.4
5.4 3.9 1.3 0.4
5.1 3.5 1.4 0.3
5.7 3.8 1.7 0.3
5.1 3.8 -1.5 0.3
5.4 3.4 1.7 0.2
5.1 3.7 1.5 0.4
4.6 3.6 1 0.2
5.1 3.3 1.7 0.5
4.8 3.4 1.9 0.2

Solution思路：First, read in the data, process the data, remove blank lines, use the 'mean' to fill in missing values, this problem is implemented in Python, the code is as follows:

import numpy as np
data = []
my_list = []
con=0
noise_data = open('noise-data-1.txt') 
clean_data = open("clean_data3.txt", 'w')
for line in noise_data.readlines():
 if len(line) == 0:
 break
 if line.count('\n') == len(line):
 continue 
 dataline =line.strip().split('\t')
 my_list.append(dataline)
 con+=1
for i in range(0,con):
 for j in range(0,len(my_list[i])):
 if my_list[i][j].count('.')==0:
  miss_row=[]
  for a in range(0,len(my_list[i])):
  if float(my_list[i][a])<0:
   miss_row.append(-float(my_list[i][a]) 
  miss_row.append(float(my_list[i][a])) 
  my_average=round(np.average(miss_row),1)
  my_list[i][j]=my_average
 else:
  if float(my_list[i][j])<0:
   my_list[i][j]=-float(my_list[i][j]) 
  my_list[i][j]=float(my_list[i][j]) 
print my_list
def file_write(filename, data_list):
 file1=open(filename,'w')
 for i in data_list:
 for j in i:
  if type(j)!=str:
  j=str(j)
  file1.write(j)
  file1.write(' ')
 file1.write('\n')
 file1.close()
 return file1
filename='clean_data.txt'
file_write(filename, my_list)

The running results are as follows:

The above example of data preprocessing implementation in Python for filling missing values is all the content that the editor shares with everyone. I hope it can be a reference for everyone, and I also hope that everyone will support and cheer for the tutorial.

Statement: The content of this article is from the Internet, and the copyright belongs to the original author. The content is contributed and uploaded by Internet users spontaneously. This website does not own the copyright, has not been manually edited, and does not assume relevant legal liabilities. If you find any content suspected of copyright infringement, please send an email to: notice#oldtoolbag.com (When reporting via email, please replace # with @) to report violations, and provide relevant evidence. Once verified, this site will immediately delete the infringing content.