You're not a Senior Engineer if you don't know this | [Thread,Process,Multithread,Multiprocess,subprocess]

1.Thread:

3 min readJust now

A thread is like a worker inside a program. It can run tasks independently but shares the same resources (memory) with other threads in the program.
Imagine a single kitchen where multiple cooks (threads) are preparing different dishes at the same time. They share the same pantry and utensils(program).
Real-world Use Case: Downloading multiple files from the internet simultaneously.

Examples:

import threading
import time


def task(name):
    print(f"Hey I'm Thread {name}: Starting task")
    time.sleep(2)  # Simulates work
    print(f"Hey again I'm Thread {name}: this time Finished a task")

thread1 = threading.Thread(target=task, args=("A",))
thread2 = threading.Thread(target=task, args=("B",))

thread1.start()
thread2.start()

thread1.join()
thread2.join()
print(f'now all process has terminated')

2. Process

A process is a program(the kitchen) in execution. Each process has its own memory space and resources.

If threads are cooks in one kitchen(program), processes are like separate restaurants, each with its own kitchen and supplies.

Real-world Use Case: Running independent programs, like opening a web browser while running a game.

Example:

from multiprocess import Process
import os

def print_process_name(name):
  print(f'Hey Im the process {name} assigned Id ({os.getpid()})')

process1 = Process(targer=print_process_name,args("first one"))
process2 = Process(targer=print_process_name,args("Second one"))

process1.start()
process2.start()

process1.join()
process2.join()

print(f'All process have finished')

3. Multiprocessing

Using multiple processes to do tasks at the same time. Each process runs independently.

Like having multiple restaurants, each cooking its own menu at the same time.

Real-world Use Case: Processing large datasets by dividing them into chunks and using multiple processors.

from multiprocessing import Pool

def square(num):
  return num*num

with Pool(4) as pool:
  result = pool.map([1,2,3,4,5])
  print(f' pooling result {result}')

4. Multithreading

Using multiple threads in a single process to do tasks at the same time.

Several cooks (threads) working in the same kitchen (process) but focusing on different dishes.

Real-world Use Case: Handling user interactions in a graphical user interface (GUI).

Example:

import threading

def print_numbers():
    for i in range(5):
        print(i)


threads = [threading.Thread(target=print_numbers) for _ in range(3)]
for thread in threads:
    thread.start()

for thread in threads:
    thread.join()

5. Thread Pool

A group of pre-created threads that can be reused for different tasks.

Like having a fixed number of cooks who are assigned tasks dynamically as orders come in.

Real-world Use Case: Handling multiple simultaneous API requests.

from concurrent.futures import ThreadPoolExecutor

def fetch_data(url):
    print(f"Fetching data from {url}")
    return f"Data from {url}"

with ThreadPoolExecutor(max_workers=3) as executor:
    urls = ["https://example.com", "https://google.com", "https://github.com"]
    results = list(executor.map(fetch_data, urls))
    print(results)

6. Pages and Virtual Memory

Virtual memory allows your computer to use part of the storage (disk) as if it were RAM. Memory is divided into “pages,” which can be swapped between RAM and disk.
Think of RAM as a desk and the disk as a filing cabinet. If your desk is full, you temporarily store some documents in the cabinet (swap to disk).

Real-world Use Case: Running applications that require more memory than is physically available.

7. In-Memory

Data is stored directly in RAM rather than on disk for faster access.
Using a notebook on your desk (RAM) versus going to the filing cabinet (disk) to retrieve information.

8. Other Key Concepts

I/O Bound vs CPU Bound

I/O Bound: Tasks that wait for input/output, like file reading or network requests.
CPU Bound: Tasks that use the processor heavily, like calculations or image processing.

Real-world Use Case:

I/O Bound: Downloading files.
CPU Bound: Video encoding.

Asynchronous vs Synchronous

Synchronous: Tasks are done one at a time.
Asynchronous: Tasks can overlap, so one can start before another finishes.