Multiprocessing in Python

The multiprocessing package supports the generation of processes. It is a function that loads and executes new subprocesses. In order for the child process to terminate or continue executing concurrent computations, the current process must use an API similar to the thread module to wait.

Introduction

When using Multiprocessing, the first thing to do is to createprocessobject. Then, it calls astart()method.

Example Code

from multiprocessing import Process
　　　def display():
　　　　　　print('Hi !! I am Python')
　　　　　　if __name__ == '__main__':
　　　　　　p = Process(target=display)
　　　　　　p.start()
　　　　　　p.join()

In this example, we first import the Process class, and then use thedisplay()function to start the Process object.

Then, usestart()Processing starts, then use thisjoin()Processing completed.

We can also use the args keyword to pass parameters to the function.

Example

from multiprocessing import Process
　　　def display(my_name):
　　　print('Hi !!!')　+　" "　+　my_name)
　　　if __name__ == '__main__':
　　　　　　p = Process(target=display, args=('Python',))
　　　　　　p.start()
　　　　　　p.join()

In this example, we create a process that calculates the cube of a number and prints all the results to the console.

Example Code

from multiprocessing import Process
　　　def cube(x):
　　　　　　for x in my_numbers:
　　　　　　　　　print('%s cube is %s' % (x, x**3))
　　　　　　if __name__ == '__main__':
　　　　　　　　　my_numbers = [3,　4,　5,　6,　7,　8]
　　　　　　　　　p = Process(target=cube, args=('x',))
　　　　　　　　　p.start()
p.join
print　("Done")

Output Result

Done
3　cube is　27
4　cube is　64
5　cube is　125
6　cube is　216
7　cube is　343
8　cube is　512

We can also create multiple processes at once.

In this example, we first create a process, that is, process1process calculates the cube of a single number, while the second process process2Checking whether the number is even or odd.

Example

from multiprocessing import Process
def cube(x):
　　　for x in my_numbers:
　　　print('%s cube is %s' % (x, x**3))
def evenno(x):
　　　for x in my_numbers:
　　　if x %　2　==　0:
　　　print('%s is an even number' % (x))
　　　if __name__ == '__main__':
　　　　　　my_numbers = [3,　4,　5,　6,　7,　8]
　　　　　　my_process1　=　Process(target=cube,　args=('x',))
　　　　　　my_process2　=　Process(target=evenno,　args=('x',))
　　　　　　my_process1.start()
　　　　　　my_process2.start()
　　　　　　my_process1.join()
　　　my_process2.join()
print　("Done")

Output Result

3　cube is　27
4　cube is　64
5　cube is　125
6　cube is　216
7　cube is　343
8　cube is　512
4　is an even number
6　is an even number
8　is an even number
Done

Communication between processes

Multiprocessing supports pipes and queues, which are two communication channels between processes.

pipe

In multiprocessing, when we need to communicate between processes, in this case we usepipe.

Example

from　multiprocessing　import　Process,　Pipe
　　　def　myfunction(conn):
　　　　　　conn.send(['hi!!　I　am　Python'])
　　　　　　conn.close()
　　　　　　if __name__ == '__main__':
　　　　　　　　　parent_conn,　child_conn　=　Pipe()　　　　　　　　　p　=　Process(target=myfunction,　args=(child_conn,))
　　　　　　　　　p.start()
　　　　　　print　(parent_conn.recv()　)
p.join()

Output Result

['hi　!!!　I　am　Python']

The pipe returns two connection objects, representing both ends of the pipe. Each connection object has two methodssend()，一种是recv()method, another is method.

In this example, first we create a process that prints the message "hi !! I am Python", and then share data.

s column

When we pass data between processes, then we can use the Queue object.

Example

import　multiprocessing
　　　def　evenno(numbers,　q):
　　　　　　for　n　in　numbers:
　　　　　　if　n　%　2　==　0:
　　　　　　q.put(n)
　　　　　　if　__name__　==　"__main__":
　　　　　　　　　q　=　multiprocessing.Queue()
　　　　　　　　　p　=　multiprocessing.Process(target=evenno,　args=(range(10),　q))
　　　　　　　　　p.start()
　　　　　　　　　p.join()
　　　while　q:
print(q.get())

Output Result

0
2
4
6
8

In this example, first create a function to check if the weather is even. If the number is even, insert it at the end of the queue. Then, we create a queue object and a process object, and then start the process.

Finally, check if the queue is empty.

When we print numbers, first we print the value in front of the queue, then print the next one, and so on.

Locks

In this case, if we want to execute only one process at a time, we use Locks. This means that time will prevent other processes from executing similar code. After the process is completed, the lock will be released.

Example of using Locks method

Example

from multiprocessing import Process, Lock
def dispmay_name(l, i):
l.acquire()
print('Hi', i)
　　　l.release()
if __name__ == '__main__':
　　　my_lock = Lock() my_name = ['Aadrika', 'Adwaita', 'Sakya', 'Sanj']
for name in my_name:
Process(target=dispmay_name, args=(my_lock, name)).start()

Output Result

Hi Aadrika
Hi Adwaita
Hi Sakya
Hi Sanj

In the log

The multiprocessing module also provides a logging module to ensure that if the logging package does not use the locking feature, messages between processes will be mixed together during execution.

Example

import multiprocessing, logging
logger = multiprocessing.log_to_stderr()
logger.setLevel(logging.INFO)
logger.warning('Error has occurred')

In this example, first import the logging and multiprocessing modules, then use the multiprocessing.log_to_stderr() method. Then call get_logger() and add it to sys.stderr, and finally, we set the logger level and pass the message.