Senior Software Engineer Edition
Last Updated: November 2025 | Python 3.10+ Focus
- https://realpython.com/python-magic-methods/
- https://www.geeksforgeeks.org/python/dunder-magic-methods-python/
Table of Contents
- Introduction
- Object Lifecycle Magic Methods
- Comparison Magic Methods
- Arithmetic Magic Methods
- Unary Operators and Functions
- Type Conversion Magic Methods
- Container Magic Methods
- Attribute Access Magic Methods
- Callable Objects
- Context Managers
- Descriptor Protocol
- Copying and Pickling
- String Representation
- Advanced Magic Methods
- Performance Optimization
- Type Hints & Protocols
- Metaclass Patterns
- Async Magic Methods
- Memory Management & Slots
- Pattern Matching (Python 3.10+)
- Production Best Practices
Introduction
Magic methods (also called dunder methods, from “double underscore”) are special methods in Python that allow you to define how objects of your class behave with built-in operations. They are always surrounded by double underscores (e.g., __init__, __str__).
What Are Magic Methods?
Magic methods are special predefined methods that Python calls automatically when you use certain syntax. Instead of calling these methods directly, Python invokes them behind the scenes.
Example – How Magic Methods Work:
# When you write this:
result = 5 + 3
# Python actually does this behind the scenes:
result = (5).__add__(3)
# When you write this:
my_list = [1, 2, 3]
length = len(my_list)
# Python actually does this:
length = my_list.__len__()Why Use Magic Methods?
- Intuitive Syntax: Make your objects work with Python’s built-in operators (+, -, *, [], etc.)
- Pythonic Code: Write code that feels natural and follows Python conventions
- Integration: Seamlessly integrate custom objects with Python’s standard library
- Operator Overloading: Define custom behavior for operators
- Protocol Implementation: Enable duck typing and structural subtyping
- Performance: Optimize object behavior at the C level
Real-World Use Cases
Without Magic Methods (Verbose):
class BankAccount:
def __init__(self, balance):
self.balance = balance
def add_funds(self, amount):
return BankAccount(self.balance + amount)
def is_greater_than(self, other):
return self.balance > other.balance
# Usage - awkward and not Pythonic
account1 = BankAccount(100)
account2 = BankAccount(50)
new_account = account1.add_funds(25) # Verbose
if account1.is_greater_than(account2): # Not intuitive
print("Account 1 has more money")With Magic Methods (Pythonic):
class BankAccount:
def __init__(self, balance):
self.balance = balance
def __add__(self, amount):
return BankAccount(self.balance + amount)
def __gt__(self, other):
return self.balance > other.balance
def __repr__(self):
return f"BankAccount(${self.balance})"
# Usage - clean and intuitive!
account1 = BankAccount(100)
account2 = BankAccount(50)
new_account = account1 + 25 # Natural syntax!
if account1 > account2: # Reads like English!
print("Account 1 has more money")
print(new_account) # Automatic nice representationLearning Path: Beginner → Expert
🟢 Beginner Level (Start Here – Week 1-2):
Essential Magic Methods:
__init__– Initialize objects (constructor)__str__– User-friendly string representation__repr__– Developer-friendly representation__len__– Length of containers__getitem__– Enable indexingobj[0]
First Project: Build a simple TodoList class
class TodoList:
def __init__(self):
self.tasks = []
def __len__(self): return len(self.tasks)
def __getitem__(self, i): return self.tasks[i]
def __repr__(self): return f"TodoList({len(self)} tasks)"🟡 Intermediate Level (Week 3-6):
Comparison & Arithmetic:
__eq__,__lt__,__gt__– Comparisons for sorting__add__,__sub__,__mul__– Arithmetic operations__hash__– Make objects hashable for sets/dicts
Container Protocol:
__setitem__– Enableobj[key] = value__delitem__– Enabledel obj[key]__contains__– Enableitem in obj__iter__,__next__– Make objects iterable
Resource Management:
__enter__,__exit__– Context managers (withstatement)
Second Project: Build a Money class with arithmetic and comparison
class Money:
def __init__(self, amount): self.amount = amount
def __add__(self, other): return Money(self.amount + other.amount)
def __lt__(self, other): return self.amount < other.amount
def __repr__(self): return f"${self.amount:.2f}"🟠 Advanced Level (Week 7-10):
Attribute Magic:
__getattr__,__setattr__– Control attribute access__getattribute__– Intercept all attribute access__get__,__set__,__delete__– Descriptors
Callable & Functional:
__call__– Make objects callable like functions
Advanced Patterns:
__new__– Control object creation- Custom iterators with
__iter__and__next__ - Property descriptors
Third Project: Build a Config class with dot notation access
class Config:
def __getattr__(self, key): return self.__dict__.get(key)
def __setattr__(self, key, val): self.__dict__[key] = val🔴 Expert Level (Week 11+):
Metaclass Magic:
__init_subclass__– Customize subclass creation__class_getitem__– Generic type syntax- Metaclass methods
Async Magic:
__aenter__,__aexit__– Async context managers__aiter__,__anext__– Async iterators
Performance & Memory:
__slots__– Memory optimization__sizeof__– Memory introspection- Custom pickling with
__getstate__,__setstate__
Pattern Matching (Python 3.10+):
__match_args__– Structural pattern matching
Expert Project: Build a production-ready ORM or caching system
Common Use Cases by Domain
🌐 Web Development:
__init__,__repr__– Model classes__enter__,__exit__– Database connections__getitem__,__setitem__– Session/config access__call__– Middleware, decorators
📊 Data Science:
__getitem__,__len__– Dataset classes__iter__– Batch generators__add__,__mul__– Matrix/vector operations__repr__– Debug data structures
🎮 Game Development:
__init__– Game entity setup__add__,__sub__– Vector math__eq__– Collision detection__lt__– Priority queues (AI, pathfinding)
🔧 System Programming:
__enter__,__exit__– Resource management__del__– Cleanup (use sparingly)__getattr__– Dynamic configuration
🧪 Testing:
__enter__,__exit__– Test fixtures__call__– Mock objects__eq__– Assertion helpers
Decision Tree: Which Magic Methods Do I Need?
Use this flowchart to decide what to implement:
START: Creating a new class
│
├─ Does it hold data? → YES
│ ├─ Implement __init__ (initialize data)
│ ├─ Implement __repr__ (for debugging)
│ └─ Need user-friendly output? → Implement __str__
│
├─ Need to compare instances? → YES
│ ├─ Implement __eq__ (equality)
│ ├─ Need ordering? → Implement __lt__ (and @total_ordering)
│ └─ Use in set/dict? → Implement __hash__ (with __eq__)
│
├─ Does it act like a number? → YES
│ ├─ Implement __add__, __sub__, __mul__ (arithmetic)
│ ├─ Support +=, -=? → Implement __iadd__, __isub__
│ └─ Need abs(), round()? → Implement __abs__, __round__
│
├─ Does it act like a list/sequence? → YES
│ ├─ Implement __len__ (length)
│ ├─ Implement __getitem__ (indexing & iteration)
│ ├─ Modifiable? → Implement __setitem__, __delitem__
│ └─ Support 'in'? → Implement __contains__
│
├─ Does it act like a dict/mapping? → YES
│ ├─ Implement __getitem__, __setitem__, __delitem__
│ ├─ Implement __len__, __contains__
│ └─ Implement __iter__ (for keys)
│
├─ Does it manage resources (files, connections)? → YES
│ └─ Implement __enter__, __exit__ (context manager)
│
├─ Should it be callable like a function? → YES
│ └─ Implement __call__
│
└─ Advanced needs?
├─ Control attribute access → __getattr__, __setattr__
├─ Create immutable type → __new__ + no __init__
├─ Singleton pattern → __new__
└─ Async operations → __aenter__, __aexit__, __aiter__, __anext__Quick Examples:
Data Class (Simple) → __init__, __repr__
class Point:
def __init__(self, x, y): self.x, self.y = x, y
def __repr__(self): return f"Point({self.x}, {self.y})"Comparable Data → Add __eq__, __lt__, __hash__
from functools import total_ordering
@total_ordering
class Point:
def __init__(self, x, y): self.x, self.y = x, y
def __eq__(self, other): return (self.x, self.y) == (other.x, other.y)
def __lt__(self, other): return (self.x, self.y) < (other.x, other.y)
def __hash__(self): return hash((self.x, self.y))Arithmetic Type → Add __add__, __sub__, etc.
class Vector:
def __init__(self, x, y): self.x, self.y = x, y
def __add__(self, other): return Vector(self.x + other.x, self.y + other.y)
def __repr__(self): return f"Vector({self.x}, {self.y})"Container → Add __len__, __getitem__, __iter__
class Playlist:
def __init__(self): self.songs = []
def __len__(self): return len(self.songs)
def __getitem__(self, i): return self.songs[i]
def __iter__(self): return iter(self.songs)Resource Manager → Add __enter__, __exit__
class DatabaseConnection:
def __enter__(self): self.connect(); return self
def __exit__(self, *args): self.close()🎯 Quick Tips for Success
- Start Simple: Don’t implement all magic methods at once
- Test Thoroughly: Magic methods are called implicitly – test edge cases
- Return NotImplemented: For unsupported operations, don’t raise TypeError
- Document Behavior: Use docstrings to explain what operations are supported
- Follow Conventions:
__repr__should be unambiguous,__str__user-friendly - Pair Methods:
__eq__with__hash__,__enter__with__exit__ - Use dataclass: For simple data containers, use
@dataclassdecorator - Profile Performance: Magic methods are called frequently
Common Mistakes to Avoid:
- ❌ Forgetting to return
selfin__iadd__,__isub__ - ❌ Modifying mutable default arguments in
__init__ - ❌ Infinite recursion in
__setattr__(usesuper()or__dict__) - ❌ Implementing
__eq__without__hash__(makes objects unhashable) - ❌ Not handling type mismatches (return
NotImplemented)
Testing:
__enter__,__exit__– Test fixtures__call__– Mock objects__eq__– Assertion helpers
🟡 Intermediate Level:
__eq__,__lt__,__gt__– Comparisons__add__,__sub__,__mul__– Arithmetic__enter__,__exit__– Context managers__call__– Callable objects__iter__,__next__– Iterators
🔴 Advanced Level:
__new__– Object creation control__getattr__,__setattr__– Attribute access__get__,__set__– Descriptors__init_subclass__– Subclass hooks- Metaclass magic methods
Senior Engineer Considerations
Type Safety: Always use type hints with magic methods for better IDE support and static analysis:
from typing import Self
from collections.abc import Iterator
class Vector:
def __init__(self, x: float, y: float) -> None:
self.x = x
self.y = y
def __add__(self, other: Self) -> Self:
return self.__class__(self.x + other.x, self.y + other.y)Performance Impact: Magic methods are called frequently. Profile before optimizing:
import timeit
# Test performance of magic methods
setup = "class Point: ..."
code = "p1 + p2"
time = timeit.timeit(code, setup=setup, number=1000000)Protocol-Oriented Design: Use typing.Protocol for structural subtyping:
from typing import Protocol
class Addable(Protocol):
def __add__(self, other: 'Addable') -> 'Addable': ...
def combine(a: Addable, b: Addable) -> Addable:
return a + b # Type-safe without inheritanceObject Lifecycle Magic Methods
Understanding Object Lifecycle
Every Python object goes through a lifecycle:
- Creation (
__new__) – Allocate memory, create instance - Initialization (
__init__) – Set up initial state - Usage – Object is used in your program
- Destruction (
__del__) – Clean up before removal
__new__(cls, *args, **kwargs)
What It Does: Controls the creation of a new instance before __init__ runs.
When to Use:
- Creating immutable objects (like
int,str,tuple) - Implementing singleton patterns
- Object pooling for performance
- Subclassing immutable built-in types
🟢 Beginner Use Case – Understanding the Basics:
class Demo:
def __new__(cls):
print("1. __new__ is called - creating instance")
instance = super().__new__(cls)
return instance
def __init__(self):
print("2. __init__ is called - initializing instance")
obj = Demo()
# Output:
# 1. __new__ is called - creating instance
# 2. __init__ is called - initializing instance🟡 Intermediate Use Case – Singleton Pattern:
class DatabaseConnection:
"""Only one database connection allowed in the entire application."""
_instance = None
def __new__(cls, connection_string):
if cls._instance is None:
print(f"Creating new database connection to {connection_string}")
cls._instance = super().__new__(cls)
else:
print("Reusing existing database connection")
return cls._instance
def __init__(self, connection_string):
self.connection_string = connection_string
# Real-world usage
db1 = DatabaseConnection("localhost:5432") # Creates new connection
db2 = DatabaseConnection("remotehost:5432") # Reuses same connection
print(db1 is db2) # True - same object!🔴 Advanced Use Case – Custom Immutable Type:
class PositiveInteger(int):
"""An integer that can only be positive - immutable type."""
def __new__(cls, value):
if value < 0:
raise ValueError(f"Value must be positive, got {value}")
# For immutable types, __new__ does all the work
return super().__new__(cls, value)
# No __init__ needed - value is set in __new__
# Usage
num = PositiveInteger(10)
print(num + 5) # 15 - works like a regular int
# PositiveInteger(-5) # ValueError: Value must be positive
# Why this matters: You can't modify it after creation
# This is how int, str, tuple work internally!🔴 Advanced – Thread-Safe Singleton Pattern (Production-Ready):
import threading
from typing import Optional, Any
class Singleton:
_instance: Optional['Singleton'] = None
_lock: threading.Lock = threading.Lock()
_initialized: bool = False
def __new__(cls, *args: Any, **kwargs: Any) -> 'Singleton':
if cls._instance is None:
with cls._lock:
# Double-checked locking pattern
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance
def __init__(self, value: int) -> None:
# Prevent re-initialization
if not self._initialized:
with self._lock:
if not self._initialized:
self.value = value
self._initialized = True
# Test
s1 = Singleton(10)
s2 = Singleton(20)
print(s1 is s2) # True - same instance
print(s1.value) # 10 - not re-initializedImmutable Objects with __new__ (like int, str):
from typing import Any
class PositiveInt(int):
"""An integer that must be positive."""
def __new__(cls, value: int) -> 'PositiveInt':
if value <= 0:
raise ValueError(f"Value must be positive, got {value}")
instance = super().__new__(cls, value)
return instance
# No __init__ needed - immutable types use __new__
num = PositiveInt(5)
print(num + 3) # 8
# PositiveInt(-5) # Raises ValueErrorObject Pooling Pattern (Flyweight):
from typing import Dict, Any
import weakref
class Flyweight:
"""Memory-efficient object pooling for immutable objects."""
_pool: Dict[tuple, 'Flyweight'] = weakref.WeakValueDictionary()
def __new__(cls, *args: Any) -> 'Flyweight':
key = args
if key not in cls._pool:
instance = super().__new__(cls)
cls._pool[key] = instance
return cls._pool[key]
def __init__(self, x: int, y: int) -> None:
# Only initialize if not already done
if not hasattr(self, 'x'):
self.x = x
self.y = y
# Reuses instances for same arguments
p1 = Flyweight(1, 2)
p2 = Flyweight(1, 2)
print(p1 is p2) # True - same object__init__(self, *args, **kwargs)
What It Does: Initializes a newly created instance with specific values and state.
When to Use: Almost every class you create! This is your constructor.
Key Point: __init__ does NOT create the object – it just sets it up after creation.
🟢 Beginner Use Case – Basic Setup:
class Person:
"""Simple person with name and age."""
def __init__(self, name, age):
# This runs AFTER the object is created
self.name = name # Setting instance attributes
self.age = age
print(f"New person created: {name}")
# Usage
person = Person("Alice", 30)
print(person.name) # Alice
print(person.age) # 30
# Real-world: Setting up any object's initial state
class ShoppingCart:
def __init__(self):
self.items = [] # Start with empty cart
self.total = 0.0🟡 Intermediate Use Case – Validation & Defaults:
class EmailAccount:
"""Email account with validation."""
def __init__(self, username, domain="gmail.com", storage_gb=15):
# Validate inputs
if not username or " " in username:
raise ValueError("Username cannot be empty or contain spaces")
# Set attributes
self.username = username
self.domain = domain
self.email = f"{username}@{domain}"
self.storage_gb = storage_gb
self.emails = []
def send_welcome(self):
print(f"Welcome email sent to {self.email}")
# Usage
account1 = EmailAccount("john.doe") # Uses defaults
account2 = EmailAccount("jane", "company.com", storage_gb=50)
# EmailAccount("") # ValueError - validation works!🔴 Advanced – Type-Safe Initialization with Validation:
from typing import Optional
import re
class Person:
"""Person class with validated initialization."""
def __init__(self, name: str, age: int, email: Optional[str] = None) -> None:
# Validation before assignment
if not isinstance(name, str) or not name.strip():
raise TypeError("Name must be a non-empty string")
if not isinstance(age, int) or age < 0:
raise ValueError("Age must be a non-negative integer")
if email and not re.match(r'^[\w.-]+@[\w.-]+\.\w+$', email):
raise ValueError(f"Invalid email format: {email}")
self._name = name
self._age = age
self._email = email
@property
def name(self) -> str:
return self._name
@property
def age(self) -> int:
return self._age
person = Person("Alice", 30, "alice@example.com")Alternative: Use Dataclasses for Less Boilerplate (Recommended):
from dataclasses import dataclass, field
from typing import Optional, ClassVar
@dataclass(slots=True, frozen=False) # slots for memory efficiency
class Person:
"""Modern Person implementation with dataclass."""
name: str
age: int
email: Optional[str] = None
_id_counter: ClassVar[int] = 0
id: int = field(init=False)
def __post_init__(self) -> None:
"""Called after __init__ for validation."""
if self.age < 0:
raise ValueError("Age must be non-negative")
Person._id_counter += 1
object.__setattr__(self, 'id', Person._id_counter)
person = Person("Alice", 30)
print(person.id) # Auto-assigned ID__del__(self)
Called when an object is about to be destroyed. Use with caution as timing is unpredictable.
class FileHandler:
def __init__(self, filename):
self.filename = filename
self.file = open(filename, 'w')
def __del__(self):
if hasattr(self, 'file') and not self.file.closed:
self.file.close()
print(f"File {self.filename} closed")Comparison Magic Methods
What They Do: Define how objects are compared using comparison operators (==, <, >, etc.)
Why Important: Enable sorting, searching, and logical comparisons of custom objects.
Real-World Use Cases:
- Sorting lists of custom objects
- Comparing business entities (prices, dates, priorities)
- Using objects as dictionary keys or in sets
- Implementing business logic (eligibility checks, ranking)
__eq__(self, other) – Equality (==)
What It Does: Determines if two objects are considered “equal”.
When to Use:
- Comparing objects for equality
- Using objects in sets or as dictionary keys (requires
__hash__too) - Finding duplicates
- Testing object state
🟢 Beginner Use Case – Simple Equality:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __eq__(self, other):
# Check if comparing with same type
if not isinstance(other, Point):
return NotImplemented # Let Python try other.__eq__(self)
return self.x == other.x and self.y == other.y
# Usage
p1 = Point(1, 2)
p2 = Point(1, 2)
p3 = Point(2, 3)
print(p1 == p2) # True - same coordinates
print(p1 == p3) # False - different coordinates
print(p1 == "string") # False - different types
# Why this matters: Without __eq__, Python compares object IDs
class PointWithoutEq:
def __init__(self, x, y):
self.x = x
self.y = y
p4 = PointWithoutEq(1, 2)
p5 = PointWithoutEq(1, 2)
print(p4 == p5) # False! Different objects in memory🟡 Intermediate Use Case – Business Logic:
class Product:
"""Product equality based on SKU only."""
def __init__(self, sku, name, price):
self.sku = sku
self.name = name
self.price = price
def __eq__(self, other):
if not isinstance(other, Product):
return NotImplemented
# Business rule: Products are equal if same SKU
return self.sku == other.sku
def __repr__(self):
return f"Product({self.sku}, {self.name}, ${self.price})"
# Usage
laptop1 = Product("LAP001", "Gaming Laptop", 1200)
laptop2 = Product("LAP001", "Gaming Laptop Pro", 1500) # Different name/price
mouse = Product("MOU001", "Wireless Mouse", 25)
print(laptop1 == laptop2) # True! Same SKU
print(laptop1 == mouse) # False - Different SKU
# Find duplicates in inventory
inventory = [laptop1, mouse, laptop2]
unique = []
for item in inventory:
if item not in unique: # Uses __eq__
unique.append(item)
print(f"Unique products: {len(unique)}") # 2 (laptop1 and laptop2 are same)🔴 Advanced Use Case – With Hashing for Sets/Dicts:
🔴 Advanced Use Case – With Hashing for Sets/Dicts:
from dataclasses import dataclass
@dataclass(frozen=True) # Makes it immutable and adds __hash__
class User:
"""Immutable user that can be used in sets and as dict keys."""
user_id: int
username: str
def __eq__(self, other):
if not isinstance(other, User):
return NotImplemented
return self.user_id == other.user_id
# frozen=True auto-generates __hash__, but here's manual version:
def __hash__(self):
# CRITICAL: If two objects are equal, they MUST have same hash
return hash(self.user_id)
# Usage - Set removes duplicates using __eq__ and __hash__
users = {
User(1, "alice"),
User(2, "bob"),
User(1, "alice_updated"), # Same ID - duplicate!
}
print(len(users)) # 2 - duplicate removed
# Dict keys
user_data = {
User(1, "alice"): {"role": "admin"},
User(2, "bob"): {"role": "user"},
}
print(user_data[User(1, "alice")]) # Works! Finds by equalityCommon Pattern – Comparing Different Types:
class Temperature:
def __init__(self, celsius):
self.celsius = celsius
def __eq__(self, other):
# Support comparing with numbers
if isinstance(other, Temperature):
return self.celsius == other.celsius
elif isinstance(other, (int, float)):
return self.celsius == other
return NotImplemented
temp = Temperature(25)
print(temp == 25) # True
print(temp == Temperature(25)) # True__lt__(self, other) – Less Than (<)
What It Does: Determines if one object is “less than” another.
When to Use:
- Sorting lists of custom objects
- Priority queues / heaps
- Ordering/ranking algorithms
- Comparison-based business logic
🟢 Beginner Use Case – Simple Sorting:
__ne__(self, other) – Not Equal (!=)
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __ne__(self, other):
return not self.__eq__(other)__lt__(self, other) – Less Than (<)
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
def __lt__(self, other):
if not isinstance(other, Person):
return NotImplemented
return self.age < other.age
people = [Person("Alice", 30), Person("Bob", 25), Person("Charlie", 35)]
sorted_people = sorted(people)
for p in sorted_people:
print(f"{p.name}: {p.age}")
# Output: Bob: 25, Alice: 30, Charlie: 35__le__(self, other) – Less Than or Equal (<=)
def __le__(self, other):
if not isinstance(other, Person):
return NotImplemented
return self.age <= other.age__gt__(self, other) – Greater Than (>)
def __gt__(self, other):
if not isinstance(other, Person):
return NotImplemented
return self.age > other.age__ge__(self, other) – Greater Than or Equal (>=)
def __ge__(self, other):
if not isinstance(other, Person):
return NotImplemented
return self.age >= other.ageUsing @functools.total_ordering
Instead of defining all comparison methods, define __eq__ and one other, then use the decorator:
from functools import total_ordering
@total_ordering
class Student:
def __init__(self, name, grade):
self.name = name
self.grade = grade
def __eq__(self, other):
if not isinstance(other, Student):
return NotImplemented
return self.grade == other.grade
def __lt__(self, other):
if not isinstance(other, Student):
return NotImplemented
return self.grade < other.grade
s1 = Student("Alice", 85)
s2 = Student("Bob", 90)
print(s1 < s2) # True
print(s1 <= s2) # True (auto-generated)
print(s1 > s2) # False (auto-generated)
print(s1 >= s2) # False (auto-generated)Arithmetic Magic Methods
Binary Arithmetic Operators
__add__(self, other) – Addition (+)
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
if not isinstance(other, Vector):
return NotImplemented
return Vector(self.x + other.x, self.y + other.y)
def __repr__(self):
return f"Vector({self.x}, {self.y})"
v1 = Vector(1, 2)
v2 = Vector(3, 4)
v3 = v1 + v2
print(v3) # Vector(4, 6)__sub__(self, other) – Subtraction (-)
def __sub__(self, other):
if not isinstance(other, Vector):
return NotImplemented
return Vector(self.x - other.x, self.y - other.y)__mul__(self, other) – Multiplication (*)
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __mul__(self, scalar):
if isinstance(scalar, (int, float)):
return Vector(self.x * scalar, self.y * scalar)
return NotImplemented
def __repr__(self):
return f"Vector({self.x}, {self.y})"
v = Vector(2, 3)
print(v * 3) # Vector(6, 9)__truediv__(self, other) – True Division (/)
def __truediv__(self, scalar):
if isinstance(scalar, (int, float)):
return Vector(self.x / scalar, self.y / scalar)
return NotImplemented__floordiv__(self, other) – Floor Division (//)
def __floordiv__(self, scalar):
if isinstance(scalar, (int, float)):
return Vector(self.x // scalar, self.y // scalar)
return NotImplemented__mod__(self, other) – Modulo (%)
class Number:
def __init__(self, value):
self.value = value
def __mod__(self, other):
if isinstance(other, Number):
return Number(self.value % other.value)
elif isinstance(other, (int, float)):
return Number(self.value % other)
return NotImplemented__pow__(self, other) – Exponentiation (**)
def __pow__(self, exponent):
if isinstance(exponent, (int, float)):
return Number(self.value ** exponent)
return NotImplemented__matmul__(self, other) – Matrix Multiplication (@)
class Matrix:
def __init__(self, data):
self.data = data
def __matmul__(self, other):
# Simplified 2x2 matrix multiplication
if not isinstance(other, Matrix):
return NotImplemented
# Implementation would do actual matrix multiplication
return Matrix([[0, 0], [0, 0]]) # PlaceholderReflected Arithmetic Operators
These are called when the left operand doesn’t support the operation.
__radd__(self, other) – Reflected Addition
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
if isinstance(other, Vector):
return Vector(self.x + other.x, self.y + other.y)
return NotImplemented
def __radd__(self, other):
# Called when other + self is evaluated and other doesn't have __add__
return self.__add__(other)Similar reflected methods exist:
__rsub__(self, other)– Reflected subtraction__rmul__(self, other)– Reflected multiplication__rtruediv__(self, other)– Reflected true division__rfloordiv__(self, other)– Reflected floor division__rmod__(self, other)– Reflected modulo__rpow__(self, other)– Reflected exponentiation__rmatmul__(self, other)– Reflected matrix multiplication
Augmented Assignment
__iadd__(self, other) – In-place Addition (+=)
class MutableVector:
def __init__(self, x, y):
self.x = x
self.y = y
def __iadd__(self, other):
if isinstance(other, MutableVector):
self.x += other.x
self.y += other.y
return self # Must return self
return NotImplemented
def __repr__(self):
return f"MutableVector({self.x}, {self.y})"
v1 = MutableVector(1, 2)
v2 = MutableVector(3, 4)
v1 += v2
print(v1) # MutableVector(4, 6)Similar augmented assignment methods:
__isub__(self, other)– In-place subtraction (-=)__imul__(self, other)– In-place multiplication (*=)__itruediv__(self, other)– In-place true division (/=)__ifloordiv__(self, other)– In-place floor division (//=)__imod__(self, other)– In-place modulo (%=)__ipow__(self, other)– In-place exponentiation (**=)__imatmul__(self, other)– In-place matrix multiplication (@=)
Unary Operators and Functions
__neg__(self) – Unary Negation (-)
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __neg__(self):
return Vector(-self.x, -self.y)
def __repr__(self):
return f"Vector({self.x}, {self.y})"
v = Vector(3, 4)
print(-v) # Vector(-3, -4)__pos__(self) – Unary Plus (+)
def __pos__(self):
return Vector(+self.x, +self.y)__abs__(self) – Absolute Value
import math
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __abs__(self):
return math.sqrt(self.x**2 + self.y**2)
v = Vector(3, 4)
print(abs(v)) # 5.0__invert__(self) – Bitwise NOT (~)
class BinaryNumber:
def __init__(self, value):
self.value = value
def __invert__(self):
return BinaryNumber(~self.value)
def __repr__(self):
return f"BinaryNumber({self.value})"
b = BinaryNumber(5)
print(~b) # BinaryNumber(-6)__round__(self, n=0) – Rounding
class Decimal:
def __init__(self, value):
self.value = value
def __round__(self, n=0):
return Decimal(round(self.value, n))
d = Decimal(3.14159)
print(round(d, 2)) # Decimal(3.14)__floor__(self) – Floor Function
import math
class Decimal:
def __init__(self, value):
self.value = value
def __floor__(self):
return Decimal(math.floor(self.value))__ceil__(self) – Ceiling Function
def __ceil__(self):
return Decimal(math.ceil(self.value))__trunc__(self) – Truncation
def __trunc__(self):
return Decimal(math.trunc(self.value))Type Conversion Magic Methods
__int__(self) – Convert to Integer
class Fraction:
def __init__(self, numerator, denominator):
self.numerator = numerator
self.denominator = denominator
def __int__(self):
return self.numerator // self.denominator
f = Fraction(7, 2)
print(int(f)) # 3__float__(self) – Convert to Float
def __float__(self):
return self.numerator / self.denominator
f = Fraction(7, 2)
print(float(f)) # 3.5__complex__(self) – Convert to Complex
class ComplexNumber:
def __init__(self, real, imag):
self.real = real
self.imag = imag
def __complex__(self):
return complex(self.real, self.imag)__bool__(self) – Convert to Boolean
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __bool__(self):
return self.x != 0 or self.y != 0
v1 = Vector(0, 0)
v2 = Vector(1, 2)
print(bool(v1)) # False
print(bool(v2)) # True
if v2:
print("Vector is non-zero")__bytes__(self) – Convert to Bytes
class Data:
def __init__(self, value):
self.value = value
def __bytes__(self):
return str(self.value).encode('utf-8')
d = Data("Hello")
print(bytes(d)) # b'Hello'__format__(self, format_spec) – Custom Formatting
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __format__(self, format_spec):
if format_spec == 'polar':
import math
r = math.sqrt(self.x**2 + self.y**2)
theta = math.atan2(self.y, self.x)
return f"(r={r:.2f}, θ={theta:.2f})"
elif format_spec == 'cartesian' or format_spec == '':
return f"({self.x}, {self.y})"
else:
raise ValueError(f"Unknown format: {format_spec}")
p = Point(3, 4)
print(f"{p}") # (3, 4)
print(f"{p:cartesian}") # (3, 4)
print(f"{p:polar}") # (r=5.00, θ=0.93)__hash__(self) – Make Object Hashable
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __eq__(self, other):
if not isinstance(other, Point):
return NotImplemented
return self.x == other.x and self.y == other.y
def __hash__(self):
return hash((self.x, self.y))
# Now can be used in sets and as dict keys
points = {Point(1, 2), Point(3, 4), Point(1, 2)}
print(len(points)) # 2 (duplicates removed)
point_data = {Point(1, 2): "Origin-ish", Point(3, 4): "Away"}__index__(self) – Convert to Index
class Index:
def __init__(self, value):
self.value = value
def __index__(self):
return self.value
idx = Index(2)
my_list = [10, 20, 30, 40]
print(my_list[idx]) # 30Container Magic Methods
What They Do: Make your custom objects behave like built-in containers (lists, dicts, sets).
Why Important: Enable natural Python syntax (len(), [], in, for loops) with custom data structures.
Real-World Use Cases:
- Custom collections (circular buffers, trees, graphs)
- Database query results
- Configuration objects
- API response wrappers
- Custom caching structures
__len__(self) – Length
What It Does: Returns the “length” of your object (called by len()).
When to Use: Any object that has a concept of “size” or “count”.
🟢 Beginner Use Case – Simple Collection:
class Playlist:
def __init__(self):
self.songs = []
def add_song(self, song):
self.songs.append(song)
def __len__(self):
return len(self.songs)
playlist = Playlist()
playlist.add_song("Song 1")
playlist.add_song("Song 2")
print(len(playlist)) # 2
# Why this matters:
if len(playlist) > 0: # Natural Python code!
print("Playlist has songs")
# Enables truthiness checks too:
if playlist: # Python calls __len__ if no __bool__
print("Playlist not empty")🟡 Intermediate Use Case – Business Logic:
class ShoppingCart:
def __init__(self):
self.items = {} # {product_id: quantity}
def add_item(self, product_id, quantity=1):
self.items[product_id] = self.items.get(product_id, 0) + quantity
def __len__(self):
# Return total number of items (not unique products)
return sum(self.items.values())
cart = ShoppingCart()
cart.add_item("LAPTOP", 2)
cart.add_item("MOUSE", 3)
print(len(cart)) # 5 items total
# Business logic
if len(cart) > 10:
print("Bulk order discount applies!")🔴 Advanced Use Case – Lazy Evaluation:
class DatabaseResultSet:
"""Query results that aren't loaded until accessed."""
def __init__(self, query):
self.query = query
self._count = None # Cached count
self._results = None
def __len__(self):
# Only count if we haven't already
if self._count is None:
# Efficient COUNT query instead of loading all data
self._count = self._execute_count()
return self._count
def _execute_count(self):
# Simulated database count query
print("Executing COUNT query...")
return 100 # Pretend we counted 100 rows
results = DatabaseResultSet("SELECT * FROM users")
print(len(results)) # Executing COUNT query... 100
print(len(results)) # 100 (cached, no query)__getitem__(self, key) – Indexing and Slicing
What It Does: Enables bracket notation obj[key] for accessing elements.
When to Use:
- Custom sequences (lists, arrays)
- Custom mappings (dicts)
- Matrix/multi-dimensional data
- Anything you want to index
🟢 Beginner Use Case – List-Like Access:
class Playlist:
def __init__(self):
self.songs = []
def add_song(self, song):
self.songs.append(song)
def __getitem__(self, index):
return self.songs[index]
def __len__(self):
return len(self.songs)
playlist = Playlist()
playlist.add_song("Song 1")
playlist.add_song("Song 2")
playlist.add_song("Song 3")
print(playlist[0]) # Song 1
print(playlist[-1]) # Song 3
print(playlist[0:2]) # ['Song 1', 'Song 2']
# Also enables iteration
for song in playlist:
print(song)__setitem__(self, key, value) – Item Assignment
class Playlist:
def __init__(self):
self.songs = []
def __getitem__(self, index):
return self.songs[index]
def __setitem__(self, index, value):
self.songs[index] = value
def __len__(self):
return len(self.songs)
playlist = Playlist()
playlist.songs = ["Song 1", "Song 2", "Song 3"]
playlist[1] = "New Song 2"
print(playlist[1]) # New Song 2__delitem__(self, key) – Item Deletion
def __delitem__(self, index):
del self.songs[index]
playlist = Playlist()
playlist.songs = ["Song 1", "Song 2", "Song 3"]
del playlist[1]
print(len(playlist.songs)) # 2__contains__(self, item) – Membership Test (in)
class Playlist:
def __init__(self):
self.songs = []
def add_song(self, song):
self.songs.append(song)
def __contains__(self, song):
return song in self.songs
playlist = Playlist()
playlist.add_song("Song 1")
playlist.add_song("Song 2")
print("Song 1" in playlist) # True
print("Song 3" in playlist) # False__iter__(self) – Iterator
class Countdown:
def __init__(self, start):
self.start = start
def __iter__(self):
self.current = self.start
return self
def __next__(self):
if self.current <= 0:
raise StopIteration
self.current -= 1
return self.current + 1
for num in Countdown(5):
print(num) # 5, 4, 3, 2, 1__reversed__(self) – Reverse Iteration
class Playlist:
def __init__(self, songs):
self.songs = songs
def __reversed__(self):
return reversed(self.songs)
def __iter__(self):
return iter(self.songs)
playlist = Playlist(["Song 1", "Song 2", "Song 3"])
for song in reversed(playlist):
print(song) # Song 3, Song 2, Song 1__missing__(self, key) – Handling Missing Keys (for dict subclasses)
class DefaultDict(dict):
def __init__(self, default_value):
super().__init__()
self.default_value = default_value
def __missing__(self, key):
# Called when key is not found
self[key] = self.default_value
return self.default_value
dd = DefaultDict(0)
dd['a'] = 5
print(dd['a']) # 5
print(dd['b']) # 0 (auto-created)
print(dd) # {'a': 5, 'b': 0}Attribute Access Magic Methods
__getattr__(self, name) – Attribute Access
Called when an attribute is not found through normal lookup.
class DynamicAttributes:
def __init__(self):
self.data = {'name': 'Alice', 'age': 30}
def __getattr__(self, name):
# Only called if attribute doesn't exist normally
if name in self.data:
return self.data[name]
raise AttributeError(f"'{type(self).__name__}' has no attribute '{name}'")
obj = DynamicAttributes()
print(obj.name) # Alice
print(obj.age) # 30
# print(obj.missing) # Raises AttributeError__setattr__(self, name, value) – Attribute Assignment
Called on every attribute assignment.
class ValidatedAttributes:
def __setattr__(self, name, value):
if name == 'age' and (not isinstance(value, int) or value < 0):
raise ValueError("Age must be a non-negative integer")
# Use super() or __dict__ to avoid infinite recursion
super().__setattr__(name, value)
obj = ValidatedAttributes()
obj.name = "Alice"
obj.age = 30
# obj.age = -5 # Raises ValueError
# obj.age = "thirty" # Raises ValueError__delattr__(self, name) – Attribute Deletion
class ProtectedAttributes:
def __init__(self):
self.public = "Can delete"
self._protected = "Cannot delete"
def __delattr__(self, name):
if name.startswith('_'):
raise AttributeError(f"Cannot delete protected attribute '{name}'")
super().__delattr__(name)
obj = ProtectedAttributes()
del obj.public # OK
# del obj._protected # Raises AttributeError__getattribute__(self, name) – Unconditional Attribute Access
Called for every attribute access. Use with caution to avoid infinite recursion.
class LoggedAccess:
def __init__(self):
super().__setattr__('_log', [])
super().__setattr__('value', 42)
def __getattribute__(self, name):
# Avoid logging access to _log itself
if name != '_log':
log = super().__getattribute__('_log')
log.append(f"Accessed: {name}")
return super().__getattribute__(name)
obj = LoggedAccess()
print(obj.value) # 42
print(obj._log) # ['Accessed: value']__dir__(self) – Directory Listing
class CustomDir:
def __init__(self):
self.x = 1
self.y = 2
self._hidden = 3
def __dir__(self):
# Customize what dir() shows
return ['x', 'y', 'custom_method']
obj = CustomDir()
print(dir(obj)) # Includes 'x', 'y', 'custom_method'Callable Objects
__call__(self, *args, **kwargs) – Make Object Callable
class Multiplier:
def __init__(self, factor):
self.factor = factor
def __call__(self, value):
return value * self.factor
double = Multiplier(2)
triple = Multiplier(3)
print(double(5)) # 10
print(triple(5)) # 15
# Can be used as a function
numbers = [1, 2, 3, 4]
doubled = list(map(double, numbers))
print(doubled) # [2, 4, 6, 8]Function Decorators Using __call__
class CountCalls:
def __init__(self, func):
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
print(f"Call {self.count} of {self.func.__name__}")
return self.func(*args, **kwargs)
@CountCalls
def greet(name):
return f"Hello, {name}!"
print(greet("Alice")) # Call 1 of greet
# Hello, Alice!
print(greet("Bob")) # Call 2 of greet
# Hello, Bob!Stateful Callable Objects
class RunningAverage:
def __init__(self):
self.total = 0
self.count = 0
def __call__(self, value):
self.total += value
self.count += 1
return self.total / self.count
avg = RunningAverage()
print(avg(10)) # 10.0
print(avg(20)) # 15.0
print(avg(30)) # 20.0Context Managers
What They Do: Enable the with statement for automatic setup and cleanup of resources.
Why Critical: Guarantee cleanup even if errors occur – prevents resource leaks.
The Pattern:
with SomeObject() as obj:
# __enter__ called here
# use obj
pass
# __exit__ ALWAYS called here, even if exception occurredReal-World Use Cases:
- File handling (auto-close files)
- Database connections/transactions
- Lock acquisition (threading)
- Network connections
- Temporary state changes
- Performance timing
- Logging context
__enter__(self) and __exit__(self, exc_type, exc_val, exc_tb)
What They Do:
__enter__: Called when enteringwithblock – setup code__exit__: Called when leavingwithblock – cleanup code (ALWAYS runs)
🟢 Beginner Use Case – Safe File Handling:
class FileManager:
"""Ensures files are always closed, even on errors."""
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
self.file = None
def __enter__(self):
print(f"Opening {self.filename}")
self.file = open(self.filename, self.mode)
return self.file # This is what 'as f' receives
def __exit__(self, exc_type, exc_val, exc_tb):
print(f"Closing {self.filename}")
if self.file:
self.file.close()
# Return False = propagate exceptions, True = suppress
return False
# Usage - file ALWAYS closes
with FileManager('test.txt', 'w') as f:
f.write("Hello, World!")
# Even if error occurs here, file still closes!
# File closed automatically
# Without context manager (BAD):
f = open('test.txt', 'w')
f.write("Hello")
# If error occurs, file never closes! Resource leak!🟡 Intermediate Use Case – Database Transactions:
class DatabaseTransaction:
"""Auto-commit on success, auto-rollback on failure."""
def __init__(self, connection):
self.connection = connection
self.transaction_started = False
def __enter__(self):
print("Starting transaction...")
self.connection.begin_transaction()
self.transaction_started = True
return self.connection
def __exit__(self, exc_type, exc_val, exc_tb):
if not self.transaction_started:
return False
if exc_type is None:
# No exception - success!
print("Committing transaction...")
self.connection.commit()
else:
# Exception occurred - rollback!
print(f"Error occurred: {exc_val}")
print("Rolling back transaction...")
self.connection.rollback()
return False # Let exception propagate
# Usage
# with DatabaseTransaction(db) as conn:
# conn.execute("INSERT INTO users ...")
# conn.execute("UPDATE accounts ...")
# # If ANY error, both rollback automatically!
# # Transaction committed automatically if no error🔴 Advanced Use Case – Temporary State Changes:
import sys
from io import StringIO
class CaptureOutput:
"""Temporarily capture stdout for testing."""
def __enter__(self):
self.original_stdout = sys.stdout
self.captured = StringIO()
sys.stdout = self.captured
return self.captured
def __exit__(self, exc_type, exc_val, exc_tb):
# ALWAYS restore original stdout
sys.stdout = self.original_stdout
return False
# Usage - testing print statements
with CaptureOutput() as output:
print("This won't show on screen")
print("This is captured")
captured_text = output.getvalue()
print("Captured:", captured_text) # Now visible!
# Real-world: Testing, logging redirection, temp config changes🔴 Advanced – Performance Timer:
import time
class Timer:
def __enter__(self):
self.start = time.time()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.end = time.time()
self.elapsed = self.end - self.start
print(f"Elapsed time: {self.elapsed:.4f} seconds")
return False
with Timer():
# Code to time
sum(range(1000000))
# Elapsed time: 0.0234 secondsSuppressing Exceptions
class SuppressException:
def __init__(self, *exceptions):
self.exceptions = exceptions
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
# Return True to suppress the exception
return exc_type is not None and issubclass(exc_type, self.exceptions)
with SuppressException(FileNotFoundError, PermissionError):
with open('nonexistent.txt', 'r') as f:
data = f.read()
print("Continued execution") # This runs even if file doesn't existDescriptor Protocol
Descriptors control attribute access on other objects. They implement __get__, __set__, and/or __delete__.
__get__(self, instance, owner) – Attribute Retrieval
class Descriptor:
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
if instance is None:
return self
return instance.__dict__.get(self.name, None)
def __set__(self, instance, value):
instance.__dict__[self.name] = value
class MyClass:
attr = Descriptor('attr')
obj = MyClass()
obj.attr = 42
print(obj.attr) # 42Validated Descriptor
class TypedProperty:
def __init__(self, name, expected_type):
self.name = name
self.expected_type = expected_type
def __get__(self, instance, owner):
if instance is None:
return self
return instance.__dict__.get(self.name)
def __set__(self, instance, value):
if not isinstance(value, self.expected_type):
raise TypeError(
f"{self.name} must be {self.expected_type.__name__}, "
f"got {type(value).__name__}"
)
instance.__dict__[self.name] = value
def __delete__(self, instance):
del instance.__dict__[self.name]
class Person:
name = TypedProperty('name', str)
age = TypedProperty('age', int)
def __init__(self, name, age):
self.name = name
self.age = age
p = Person("Alice", 30)
print(p.name, p.age) # Alice 30
# p.age = "thirty" # Raises TypeErrorRead-Only Property Descriptor
class ReadOnly:
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
if instance is None:
return self
return instance.__dict__.get(self.name)
def __set__(self, instance, value):
if self.name in instance.__dict__:
raise AttributeError(f"Cannot modify read-only attribute '{self.name}'")
instance.__dict__[self.name] = value
class Config:
api_key = ReadOnly('api_key')
def __init__(self, api_key):
self.api_key = api_key
config = Config("secret123")
print(config.api_key) # secret123
# config.api_key = "new" # Raises AttributeErrorLazy Property Descriptor
class LazyProperty:
def __init__(self, func):
self.func = func
self.name = func.__name__
def __get__(self, instance, owner):
if instance is None:
return self
# Compute value and cache it
value = self.func(instance)
setattr(instance, self.name, value)
return value
class DataProcessor:
def __init__(self, data):
self.data = data
@LazyProperty
def processed_data(self):
print("Processing data...")
return [x * 2 for x in self.data]
dp = DataProcessor([1, 2, 3, 4])
print(dp.processed_data) # Processing data... [2, 4, 6, 8]
print(dp.processed_data) # [2, 4, 6, 8] (no reprocessing)Copying and Pickling
__copy__(self) – Shallow Copy
import copy
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __copy__(self):
print("Creating shallow copy")
return Point(self.x, self.y)
def __repr__(self):
return f"Point({self.x}, {self.y})"
p1 = Point(1, 2)
p2 = copy.copy(p1)
print(p2) # Point(1, 2)__deepcopy__(self, memo) – Deep Copy
class Node:
def __init__(self, value, children=None):
self.value = value
self.children = children or []
def __deepcopy__(self, memo):
print(f"Deep copying node {self.value}")
# Create new instance
new_node = Node(self.value)
# Deep copy children
new_node.children = copy.deepcopy(self.children, memo)
return new_node
def __repr__(self):
return f"Node({self.value})"
root = Node(1, [Node(2), Node(3)])
root_copy = copy.deepcopy(root)__getstate__(self) and __setstate__(self, state) – Pickling
import pickle
class Connection:
def __init__(self, host, port):
self.host = host
self.port = port
self.socket = None # Not picklable
def __getstate__(self):
# Return state to pickle (exclude socket)
state = self.__dict__.copy()
state['socket'] = None
return state
def __setstate__(self, state):
# Restore state from pickle
self.__dict__.update(state)
# Reconnect if needed
self.reconnect()
def reconnect(self):
print(f"Reconnecting to {self.host}:{self.port}")
conn = Connection("localhost", 8080)
data = pickle.dumps(conn)
conn2 = pickle.loads(data) # Reconnecting to localhost:8080__reduce__(self) – Advanced Pickling
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __reduce__(self):
# Return (callable, args) to recreate object
return (Point, (self.x, self.y))
p1 = Point(3, 4)
data = pickle.dumps(p1)
p2 = pickle.loads(data)
print(p2.x, p2.y) # 3 4String Representation
What They Do: Control how objects are converted to strings for display and debugging.
Key Difference:
__repr__: For developers (debugging, logging) – should be unambiguous__str__: For end users (display, output) – should be readable
Rule of Thumb: Always implement __repr__. Implement __str__ only if needed for user-facing output.
__repr__(self) – Official String Representation
What It Does: Returns the “official” string representation – ideally code that recreates the object.
When to Use: ALWAYS! This is called by repr(), in debugger, and when object is in a container.
🟢 Beginner Use Case – Debug-Friendly Output:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __repr__(self):
# Should look like valid Python code
return f"Point({self.x}, {self.y})"
p = Point(3, 4)
print(repr(p)) # Point(3, 4)
print(p) # Point(3, 4) - falls back to __repr__
# Why this matters:
points = [Point(1, 2), Point(3, 4)]
print(points) # [Point(1, 2), Point(3, 4)] - easy to read!
# Without __repr__:
class BadPoint:
def __init__(self, x, y):
self.x = x
self.y = y
bad_points = [BadPoint(1, 2), BadPoint(3, 4)]
print(bad_points) # [<__main__.BadPoint object at 0x...>] - useless!🟡 Intermediate Use Case – Logging and Debugging:
import logging
class DatabaseQuery:
def __init__(self, table, conditions):
self.table = table
self.conditions = conditions
def __repr__(self):
# Detailed representation for debugging
return f"DatabaseQuery(table={self.table!r}, conditions={self.conditions!r})"
def execute(self):
logging.info(f"Executing: {self!r}") # Uses __repr__
# Execute query...
query = DatabaseQuery("users", {"age > 18": True})
query.execute()
# Log: Executing: DatabaseQuery(table='users', conditions={'age > 18': True})🔴 Advanced Use Case – Recreatable Representation:
from datetime import datetime
class Event:
def __init__(self, name, timestamp=None):
self.name = name
self.timestamp = timestamp or datetime.now()
def __repr__(self):
# Return code that can recreate the object
return (f"Event(name={self.name!r}, "
f"timestamp=datetime.fromisoformat({self.timestamp.isoformat()!r}))")
event = Event("User Login")
print(repr(event))
# Event(name='User Login', timestamp=datetime.fromisoformat('2025-11-18T10:30:00'))
# Can actually recreate it:
event_copy = eval(repr(event)) # Works!__str__(self) – Informal String Representation
What It Does: Returns a user-friendly string representation.
When to Use: When you need different output for end users vs developers.
🟢 Beginner Use Case – User-Friendly Display:
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
def __repr__(self):
# For developers - detailed and unambiguous
return f"Person(name={self.name!r}, age={self.age!r})"
def __str__(self):
# For users - readable and simple
return f"{self.name}, {self.age} years old"
p = Person("Alice", 30)
print(str(p)) # Alice, 30 years old (uses __str__)
print(repr(p)) # Person(name='Alice', age=30) (uses __repr__)
print(p) # Alice, 30 years old (print uses __str__ if available)
# In a list, uses __repr__:
people = [p]
print(people) # [Person(name='Alice', age=30)]🟡 Intermediate Use Case – Custom Formatting:
class Money:
def __init__(self, amount, currency="USD"):
self.amount = amount
self.currency = currency
def __repr__(self):
return f"Money({self.amount}, {self.currency!r})"
def __str__(self):
# User-friendly format with currency symbol
symbols = {"USD": "$", "EUR": "€", "GBP": "£"}
symbol = symbols.get(self.currency, self.currency)
return f"{symbol}{self.amount:.2f}"
price = Money(1234.5, "USD")
print(f"Total: {price}") # Total: $1234.50 (uses __str__)
print(f"Debug: {price!r}") # Debug: Money(1234.5, 'USD') (uses __repr__)
# In logs
import logging
logging.info(f"Price set to {price}") # Price set to $1234.50🔴 Advanced Use Case – Context-Aware Formatting:
from datetime import datetime
class SmartTimestamp:
def __init__(self, dt):
self.dt = dt
def __repr__(self):
return f"SmartTimestamp({self.dt.isoformat()!r})"
def __str__(self):
# User-friendly relative time
now = datetime.now()
diff = now - self.dt
if diff.days == 0:
if diff.seconds < 60:
return "just now"
elif diff.seconds < 3600:
return f"{diff.seconds // 60} minutes ago"
else:
return f"{diff.seconds // 3600} hours ago"
elif diff.days == 1:
return "yesterday"
elif diff.days < 7:
return f"{diff.days} days ago"
else:
return self.dt.strftime("%B %d, %Y")
ts = SmartTimestamp(datetime(2025, 11, 17, 10, 30))
print(ts) # "yesterday" or similar
print(repr(ts)) # SmartTimestamp('2025-11-17T10:30:00')__bytes__(self) – Bytes Representation
class Data:
def __init__(self, content):
self.content = content
def __bytes__(self):
return self.content.encode('utf-8')
def __str__(self):
return self.content
d = Data("Hello")
print(bytes(d)) # b'Hello'Advanced Magic Methods
Bitwise Operators
__and__(self, other) – Bitwise AND (&)
class BitSet:
def __init__(self, value):
self.value = value
def __and__(self, other):
if isinstance(other, BitSet):
return BitSet(self.value & other.value)
return NotImplemented
def __repr__(self):
return f"BitSet({bin(self.value)})"
b1 = BitSet(0b1100)
b2 = BitSet(0b1010)
print(b1 & b2) # BitSet(0b1000)__or__(self, other) – Bitwise OR (|)
def __or__(self, other):
if isinstance(other, BitSet):
return BitSet(self.value | other.value)
return NotImplemented__xor__(self, other) – Bitwise XOR (^)
def __xor__(self, other):
if isinstance(other, BitSet):
return BitSet(self.value ^ other.value)
return NotImplemented__lshift__(self, other) – Left Shift (<<)
def __lshift__(self, n):
return BitSet(self.value << n)__rshift__(self, other) – Right Shift (>>)
def __rshift__(self, n):
return BitSet(self.value >> n)Reflected and In-place Bitwise Operators
Similar patterns exist for:
__rand__,__ror__,__rxor__,__rlshift__,__rrshift__(reflected)__iand__,__ior__,__ixor__,__ilshift__,__irshift__(in-place)
__sizeof__(self) – Memory Size
import sys
class CustomList:
def __init__(self, items):
self.items = items
def __sizeof__(self):
# Return size in bytes
return sys.getsizeof(self.items) + sys.getsizeof(self.__dict__)
cl = CustomList([1, 2, 3, 4, 5])
print(sys.getsizeof(cl)) # Uses __sizeof____init_subclass__(cls, **kwargs) – Subclass Initialization
Called when a class is subclassed.
class PluginBase:
plugins = []
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls.plugins.append(cls)
print(f"Registered plugin: {cls.__name__}")
class Plugin1(PluginBase):
pass # Registered plugin: Plugin1
class Plugin2(PluginBase):
pass # Registered plugin: Plugin2
print(PluginBase.plugins) # [<class 'Plugin1'>, <class 'Plugin2'>]__class_getitem__(cls, item) – Generic Class Syntax
Enables Class[type] syntax (like list[int] in type hints).
class GenericList:
def __class_getitem__(cls, item):
print(f"Creating generic list of {item}")
return cls
# Works with type hints
MyList = GenericList[int] # Creating generic list of <class 'int'>__prepare__(metacls, name, bases, **kwargs) – Metaclass Namespace
Returns the namespace dict for a new class.
class OrderedMeta(type):
@classmethod
def __prepare__(metacls, name, bases, **kwargs):
# Return OrderedDict to preserve definition order (not needed in Python 3.7+)
return {}
def __new__(metacls, name, bases, namespace, **kwargs):
result = super().__new__(metacls, name, bases, dict(namespace))
result.members = list(namespace.keys())
return result
class MyClass(metaclass=OrderedMeta):
a = 1
b = 2
c = 3
print(MyClass.members) # ['__module__', '__qualname__', 'a', 'b', 'c']__instancecheck__(self, instance) – Custom isinstance()
class AcceptAnything(type):
def __instancecheck__(cls, instance):
return True
class MyClass(metaclass=AcceptAnything):
pass
print(isinstance(42, MyClass)) # True
print(isinstance("hello", MyClass)) # True__subclasscheck__(self, subclass) – Custom issubclass()
class MyMeta(type):
def __subclasscheck__(cls, subclass):
# Custom logic for subclass checking
return hasattr(subclass, 'special_method')
class MyBase(metaclass=MyMeta):
pass
class MySubclass:
def special_method(self):
pass
print(issubclass(MySubclass, MyBase)) # True__set_name__(self, owner, name) – Descriptor Name Assignment
Called when a descriptor is assigned to a class attribute.
class NamedDescriptor:
def __set_name__(self, owner, name):
self.name = name
self.private_name = '_' + name
def __get__(self, instance, owner):
if instance is None:
return self
return getattr(instance, self.private_name, None)
def __set__(self, instance, value):
setattr(instance, self.private_name, value)
class MyClass:
attr = NamedDescriptor()
obj = MyClass()
obj.attr = 42
print(obj.attr) # 42
print(obj._attr) # 42Practical Examples
This section shows real-world implementations progressing from beginner to expert level.
🟢 Example 1: Task Manager (Beginner)
Use Case: Simple task list with basic operations.
Magic Methods Used: __init__, __len__, __getitem__, __str__, __repr__
class TaskManager:
"""Manage a list of tasks."""
def __init__(self):
self.tasks = []
def add_task(self, task):
self.tasks.append(task)
print(f"Added: {task}")
def __len__(self):
"""Enable len(task_manager)."""
return len(self.tasks)
def __getitem__(self, index):
"""Enable task_manager[0], slicing, and iteration."""
return self.tasks[index]
def __str__(self):
"""User-friendly output."""
if not self.tasks:
return "No tasks"
return "\n".join(f"{i+1}. {task}" for i, task in enumerate(self.tasks))
def __repr__(self):
"""Developer-friendly output."""
return f"TaskManager({len(self.tasks)} tasks)"
# Usage
tm = TaskManager()
tm.add_task("Buy groceries")
tm.add_task("Write code")
tm.add_task("Exercise")
print(f"Total tasks: {len(tm)}") # 3
print(f"First task: {tm[0]}") # Buy groceries
print(tm) # Formatted list
print(repr(tm)) # TaskManager(3 tasks)
# Bonus: Can iterate thanks to __getitem__
for task in tm:
print(f"- {task}")🟡 Example 2: Money Class (Intermediate)
Use Case: Handle money with currency, prevent errors, support arithmetic.
Magic Methods Used: __init__, __add__, __sub__, __mul__, __eq__, __lt__, __str__, __repr__, __format__
class Money:
"""Type-safe money handling with currency."""
def __init__(self, amount, currency="USD"):
if not isinstance(amount, (int, float)):
raise TypeError("Amount must be numeric")
if amount < 0:
raise ValueError("Amount cannot be negative")
self.amount = round(amount, 2) # Prevent floating point issues
self.currency = currency.upper()
def __add__(self, other):
"""Add money only if same currency."""
if isinstance(other, Money):
if self.currency != other.currency:
raise ValueError(f"Cannot add {self.currency} and {other.currency}")
return Money(self.amount + other.amount, self.currency)
elif isinstance(other, (int, float)):
return Money(self.amount + other, self.currency)
return NotImplemented
def __sub__(self, other):
"""Subtract money."""
if isinstance(other, Money):
if self.currency != other.currency:
raise ValueError(f"Cannot subtract {self.currency} and {other.currency}")
return Money(self.amount - other.amount, self.currency)
return NotImplemented
def __mul__(self, multiplier):
"""Multiply by number (for calculations)."""
if isinstance(multiplier, (int, float)):
return Money(self.amount * multiplier, self.currency)
return NotImplemented
def __eq__(self, other):
"""Equal if same amount and currency."""
if isinstance(other, Money):
return self.amount == other.amount and self.currency == other.currency
return False
def __lt__(self, other):
"""Compare amounts (same currency only)."""
if isinstance(other, Money):
if self.currency != other.currency:
raise ValueError("Cannot compare different currencies")
return self.amount < other.amount
return NotImplemented
def __format__(self, format_spec):
"""Support f-string formatting."""
symbols = {"USD": "$", "EUR": "€", "GBP": "£", "JPY": "¥"}
symbol = symbols.get(self.currency, self.currency + " ")
return f"{symbol}{self.amount:.2f}"
def __str__(self):
return format(self)
def __repr__(self):
return f"Money({self.amount}, {self.currency!r})"
# Usage
price = Money(29.99, "USD")
tax = Money(2.50, "USD")
total = price + tax
print(f"Total: {total}") # Total: $32.49
print(f"With discount: {total * 0.9}") # With discount: $29.24
# Comparison
budget = Money(50, "USD")
if total < budget:
print("Within budget!")
# Prevents errors
# eur_price = Money(10, "EUR")
# mixed = price + eur_price # ValueError: Cannot add USD and EUR🔴 Example 3: Smart Cache (Advanced)
Use Case: LRU Cache with size limit, statistics, and context manager support.
Magic Methods Used: __getitem__, __setitem__, __delitem__, __contains__, __len__, __enter__, __exit__, __repr__
from collections import OrderedDict
from typing import Any, Optional
import time
class LRUCache:
"""Least Recently Used cache with auto-cleanup."""
def __init__(self, max_size=100, ttl=None):
self.max_size = max_size
self.ttl = ttl # Time to live in seconds
self._cache = OrderedDict()
self._access_times = {}
self.hits = 0
self.misses = 0
def __getitem__(self, key):
"""Get item from cache."""
if key not in self._cache:
self.misses += 1
raise KeyError(key)
# Check if expired
if self.ttl and time.time() - self._access_times[key] > self.ttl:
del self[key]
self.misses += 1
raise KeyError(f"{key} (expired)")
# Move to end (mark as recently used)
self._cache.move_to_end(key)
self._access_times[key] = time.time()
self.hits += 1
return self._cache[key]
def __setitem__(self, key, value):
"""Add/update item in cache."""
if key in self._cache:
self._cache.move_to_end(key)
self._cache[key] = value
self._access_times[key] = time.time()
# Evict oldest if over capacity
if len(self._cache) > self.max_size:
oldest_key = next(iter(self._cache))
del self[oldest_key]
def __delitem__(self, key):
"""Remove item from cache."""
del self._cache[key]
del self._access_times[key]
def __contains__(self, key):
"""Check if key exists (and not expired)."""
if key not in self._cache:
return False
if self.ttl and time.time() - self._access_times[key] > self.ttl:
del self[key]
return False
return True
def __len__(self):
"""Number of items in cache."""
return len(self._cache)
def __enter__(self):
"""Context manager support."""
return self
def __exit__(self, exc_type, exc_val, exc_tb):
"""Clean up on context exit."""
self.clear()
return False
def clear(self):
"""Clear all cache."""
self._cache.clear()
self._access_times.clear()
def get_stats(self):
"""Return cache statistics."""
total = self.hits + self.misses
hit_rate = (self.hits / total * 100) if total > 0 else 0
return {
"hits": self.hits,
"misses": self.misses,
"hit_rate": f"{hit_rate:.1f}%",
"size": len(self),
"max_size": self.max_size
}
def __repr__(self):
stats = self.get_stats()
return f"LRUCache(size={stats['size']}/{stats['max_size']}, hit_rate={stats['hit_rate']})"
# Usage
cache = LRUCache(max_size=3, ttl=60)
# Set items
cache["user:1"] = {"name": "Alice", "age": 30}
cache["user:2"] = {"name": "Bob", "age": 25}
cache["user:3"] = {"name": "Charlie", "age": 35}
# Get items
print(cache["user:1"]) # Hit
print(cache["user:1"]) # Hit again
# Check existence
if "user:2" in cache:
print("Found user 2")
# Add one more - evicts least recently used
cache["user:4"] = {"name": "Dave", "age": 40}
# print(cache["user:2"]) # KeyError - was evicted
# Statistics
print(cache.get_stats())
# {'hits': 3, 'misses': 0, 'hit_rate': '100.0%', 'size': 3, 'max_size': 3}
print(repr(cache))
# LRUCache(size=3/3, hit_rate=100.0%)
# Context manager usage
with LRUCache(max_size=5) as temp_cache:
temp_cache["temp"] = "data"
# Use cache...
# Auto-cleaned up🔴 Example 4: SQL Query Builder (Expert)
Use Case: Fluent API for building SQL queries safely.
Magic Methods Used: __init__, __str__, __repr__, method chaining pattern
class QueryBuilder:
"""Chainable SQL query builder with parameter safety."""
def __init__(self, table=None):
self._table = table
self._select = []
self._where = []
self._params = []
self._order_by = []
self._limit = None
self._joins = []
def select(self, *columns):
"""Select specific columns."""
self._select.extend(columns)
return self # Enable chaining
def where(self, condition, *params):
"""Add WHERE clause with parameterized query."""
self._where.append(condition)
self._params.extend(params)
return self
def join(self, table, condition):
"""Add JOIN clause."""
self._joins.append(f"JOIN {table} ON {condition}")
return self
def order_by(self, column, direction="ASC"):
"""Add ORDER BY clause."""
self._order_by.append(f"{column} {direction}")
return self
def limit(self, count):
"""Add LIMIT clause."""
self._limit = count
return self
def __str__(self):
"""Generate SQL query string."""
if not self._table:
raise ValueError("Table not specified")
# SELECT clause
columns = ", ".join(self._select) if self._select else "*"
query = f"SELECT {columns} FROM {self._table}"
# JOIN clauses
if self._joins:
query += " " + " ".join(self._joins)
# WHERE clause
if self._where:
query += " WHERE " + " AND ".join(self._where)
# ORDER BY clause
if self._order_by:
query += " ORDER BY " + ", ".join(self._order_by)
# LIMIT clause
if self._limit:
query += f" LIMIT {self._limit}"
return query
def __repr__(self):
"""Debug representation."""
return f"QueryBuilder(table={self._table!r}, params={self._params})"
def get_params(self):
"""Get query parameters for safe execution."""
return tuple(self._params)
def execute(self, connection):
"""Execute query safely with parameters."""
query = str(self)
params = self.get_params()
print(f"Executing: {query}")
print(f"Params: {params}")
# return connection.execute(query, params)
# Usage - Fluent, readable, SQL-injection safe
query = (QueryBuilder("users")
.select("id", "name", "email")
.where("age > ?", 18)
.where("country = ?", "USA")
.order_by("name", "ASC")
.limit(10))
print(query)
# SELECT id, name, email FROM users WHERE age > ? AND country = ? ORDER BY name ASC LIMIT 10
print(query.get_params())
# (18, 'USA')
# Complex query with joins
complex_query = (QueryBuilder("orders")
.select("orders.id", "customers.name", "orders.total")
.join("customers", "orders.customer_id = customers.id")
.where("orders.status = ?", "pending")
.where("orders.total > ?", 100)
.order_by("orders.created_at", "DESC"))
print(complex_query)
# SELECT orders.id, customers.name, orders.total FROM orders
# JOIN customers ON orders.customer_id = customers.id
# WHERE orders.status = ? AND orders.total = ?
# ORDER BY orders.created_at DESCThese examples show progression from simple to complex, demonstrating how magic methods create clean, intuitive APIs!
import math
class Complex:
def __init__(self, real, imag):
self.real = real
self.imag = imag
def __add__(self, other):
if isinstance(other, Complex):
return Complex(self.real + other.real, self.imag + other.imag)
elif isinstance(other, (int, float)):
return Complex(self.real + other, self.imag)
return NotImplemented
def __sub__(self, other):
if isinstance(other, Complex):
return Complex(self.real - other.real, self.imag - other.imag)
elif isinstance(other, (int, float)):
return Complex(self.real - other, self.imag)
return NotImplemented
def __mul__(self, other):
if isinstance(other, Complex):
real = self.real * other.real - self.imag * other.imag
imag = self.real * other.imag + self.imag * other.real
return Complex(real, imag)
elif isinstance(other, (int, float)):
return Complex(self.real * other, self.imag * other)
return NotImplemented
def __abs__(self):
return math.sqrt(self.real**2 + self.imag**2)
def __eq__(self, other):
if isinstance(other, Complex):
return self.real == other.real and self.imag == other.imag
return NotImplemented
def __repr__(self):
sign = '+' if self.imag >= 0 else '-'
return f"{self.real}{sign}{abs(self.imag)}j"
def __str__(self):
return repr(self)
# Usage
c1 = Complex(3, 4)
c2 = Complex(1, 2)
print(c1 + c2) # 4+6j
print(c1 * c2) # -5+10j
print(abs(c1)) # 5.0Example 2: Custom Dictionary with Default Values
class DefaultDict(dict):
def __init__(self, default_factory, *args, **kwargs):
super().__init__(*args, **kwargs)
self.default_factory = default_factory
def __missing__(self, key):
if self.default_factory is None:
raise KeyError(key)
self[key] = value = self.default_factory()
return value
def __repr__(self):
return f"DefaultDict({self.default_factory}, {dict.__repr__(self)})"
# Usage
dd = DefaultDict(list)
dd['fruits'].append('apple')
dd['fruits'].append('banana')
dd['vegetables'].append('carrot')
print(dd) # DefaultDict(<class 'list'>, {'fruits': ['apple', 'banana'], 'vegetables': ['carrot']})Example 3: Chainable Query Builder
class Query:
def __init__(self, items=None):
self.items = items or []
def filter(self, predicate):
return Query([item for item in self.items if predicate(item)])
def map(self, func):
return Query([func(item) for item in self.items])
def __iter__(self):
return iter(self.items)
def __len__(self):
return len(self.items)
def __getitem__(self, index):
return self.items[index]
def __repr__(self):
return f"Query({self.items})"
# Usage
data = Query([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
result = data.filter(lambda x: x % 2 == 0).map(lambda x: x ** 2)
print(list(result)) # [4, 16, 36, 64, 100]Example 4: Fluent Interface Builder
class EmailBuilder:
def __init__(self):
self._to = None
self._subject = None
self._body = None
def to(self, recipient):
self._to = recipient
return self
def subject(self, subject):
self._subject = subject
return self
def body(self, body):
self._body = body
return self
def __str__(self):
return f"To: {self._to}\nSubject: {self._subject}\n\n{self._body}"
def __repr__(self):
return f"EmailBuilder(to={self._to!r}, subject={self._subject!r})"
# Usage
email = (EmailBuilder()
.to("user@example.com")
.subject("Hello")
.body("This is a test email"))
print(email)Example 5: Resource Pool with Context Manager
from queue import Queue
import time
class ConnectionPool:
def __init__(self, size):
self.size = size
self.pool = Queue(maxsize=size)
for i in range(size):
self.pool.put(f"Connection-{i}")
def acquire(self):
print("Acquiring connection...")
return self.pool.get()
def release(self, connection):
print(f"Releasing {connection}")
self.pool.put(connection)
def __enter__(self):
self.connection = self.acquire()
return self.connection
def __exit__(self, exc_type, exc_val, exc_tb):
self.release(self.connection)
return False
# Usage
pool = ConnectionPool(3)
with pool as conn:
print(f"Using {conn}")
time.sleep(0.1)
# Connection automatically releasedBest Practices
1. Always Return NotImplemented for Unsupported Operations
def __add__(self, other):
if isinstance(other, MyClass):
# Implementation
pass
return NotImplemented # Not TypeError or NotImplementedError2. Implement Both __eq__ and __hash__ for Hashable Objects
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __eq__(self, other):
if not isinstance(other, Point):
return NotImplemented
return self.x == other.x and self.y == other.y
def __hash__(self):
return hash((self.x, self.y))3. Make __repr__ Unambiguous and Useful
# Good
def __repr__(self):
return f"Point(x={self.x}, y={self.y})"
# Better (can recreate object)
def __repr__(self):
return f"Point({self.x}, {self.y})"4. Use __str__ for User-Friendly Output
def __repr__(self):
return f"Person(name={self.name!r}, age={self.age})"
def __str__(self):
return f"{self.name} ({self.age} years old)"5. Be Careful with __getattribute__
Avoid infinite recursion by using super().__getattribute__():
def __getattribute__(self, name):
# Don't do: return self.something (infinite recursion)
# Do:
return super().__getattribute__(name)6. Context Managers Should Always Clean Up
def __exit__(self, exc_type, exc_val, exc_tb):
# Always clean up, even if exception occurred
self.cleanup()
return False # Don't suppress exceptions unless intended7. Make Augmented Assignment Return self
def __iadd__(self, other):
# Modify in place
self.value += other.value
return self # Must return self8. Use Type Checking
def __add__(self, other):
if not isinstance(other, MyClass):
return NotImplemented
# Implementation9. Document Magic Method Behavior
class Vector:
"""A 2D vector class.
Supports addition, subtraction, multiplication by scalar,
and comparison operations.
"""
def __add__(self, other):
"""Add two vectors component-wise."""
# Implementation10. Consider Using @functools.total_ordering
from functools import total_ordering
@total_ordering
class Version:
def __eq__(self, other):
# Implementation
def __lt__(self, other):
# Implementation
# Other comparison methods auto-generatedCommon Pitfalls
1. Modifying Mutable Default Arguments
# Bad
def __init__(self, items=[]):
self.items = items # All instances share same list!
# Good
def __init__(self, items=None):
self.items = items if items is not None else []2. Forgetting to Return self in Augmented Assignment
# Bad
def __iadd__(self, other):
self.value += other.value
# Forgot to return self!
# Good
def __iadd__(self, other):
self.value += other.value
return self3. Raising Wrong Exception Types
# Bad
def __add__(self, other):
if not isinstance(other, MyClass):
raise TypeError("Cannot add")
# Good
def __add__(self, other):
if not isinstance(other, MyClass):
return NotImplemented4. Making Objects Unhashable Unintentionally
class Point:
def __eq__(self, other):
return self.x == other.x and self.y == other.y
# Forgot __hash__! Now Point is unhashable
# Python sets __hash__ = None when __eq__ is defined5. Infinite Recursion in __setattr__
# Bad
def __setattr__(self, name, value):
self.name = value # Infinite recursion!
# Good
def __setattr__(self, name, value):
super().__setattr__(name, value)
# Or: self.__dict__[name] = valuePerformance Optimization
__slots__ – Memory Optimization
Reduce memory footprint by 40-50% and speed up attribute access.
from typing import ClassVar
import sys
class WithoutSlots:
def __init__(self, x: int, y: int) -> None:
self.x = x
self.y = y
class WithSlots:
__slots__ = ('x', 'y') # Define allowed attributes
def __init__(self, x: int, y: int) -> None:
self.x = x
self.y = y
# Memory comparison
w1 = WithoutSlots(1, 2)
w2 = WithSlots(1, 2)
print(sys.getsizeof(w1.__dict__)) # ~240 bytes
print(sys.getsizeof(w2)) # ~64 bytes
# Performance: attribute access is 10-15% faster with slotsSlots Best Practices:
from typing import ClassVar, Any
class OptimizedClass:
"""Production-ready slots usage."""
__slots__ = ('_x', '_y', '__weakref__') # Include __weakref__ for weak references
_cache: ClassVar[dict] = {} # Class variables need ClassVar
def __init__(self, x: int, y: int) -> None:
self._x = x
self._y = y
def __getstate__(self) -> dict[str, Any]:
"""Needed for pickling with slots."""
return {slot: getattr(self, slot) for slot in self.__slots__ if hasattr(self, slot)}
def __setstate__(self, state: dict[str, Any]) -> None:
"""Needed for unpickling with slots."""
for slot, value in state.items():
setattr(self, slot, value)Lazy Evaluation Pattern
from typing import Optional, Callable, Any
from functools import wraps
class LazyAttribute:
"""Descriptor for lazy evaluation with caching."""
def __init__(self, func: Callable) -> None:
self.func = func
self.name = func.__name__
wraps(func)(self)
def __get__(self, instance: Any, owner: type) -> Any:
if instance is None:
return self
# Compute and cache
value = self.func(instance)
setattr(instance, self.name, value) # Replace descriptor with value
return value
class DataAnalyzer:
def __init__(self, data: list[int]) -> None:
self.data = data
@LazyAttribute
def expensive_calculation(self) -> float:
"""Only computed when first accessed."""
print("Computing...")
return sum(x**2 for x in self.data) / len(self.data)
analyzer = DataAnalyzer([1, 2, 3, 4, 5])
print(analyzer.expensive_calculation) # Computing... 11.0
print(analyzer.expensive_calculation) # 11.0 (no recomputation)Type Hints & Protocols
Structural Subtyping with Protocols
from typing import Protocol, TypeVar, runtime_checkable
from collections.abc import Iterator
T = TypeVar('T', bound='Comparable')
@runtime_checkable
class Comparable(Protocol):
"""Protocol for comparable objects."""
def __lt__(self, other: 'Comparable') -> bool: ...
def __le__(self, other: 'Comparable') -> bool: ...
def __gt__(self, other: 'Comparable') -> bool: ...
def __ge__(self, other: 'Comparable') -> bool: ...
def find_max(items: list[Comparable]) -> Comparable:
"""Works with any comparable type without inheritance."""
return max(items)
# Works with built-in types
print(find_max([1, 5, 3])) # 5
print(find_max(['a', 'z', 'b'])) # 'z'
# Works with custom types
class Score:
def __init__(self, value: int) -> None:
self.value = value
def __lt__(self, other: 'Score') -> bool:
return self.value < other.value
def __le__(self, other: 'Score') -> bool:
return self.value <= other.value
def __gt__(self, other: 'Score') -> bool:
return self.value > other.value
def __ge__(self, other: 'Score') -> bool:
return self.value >= other.value
print(isinstance(Score(10), Comparable)) # True (runtime check)Generic Magic Methods
from typing import Generic, TypeVar, Callable
from collections.abc import Iterator
T = TypeVar('T')
U = TypeVar('U')
class Container(Generic[T]):
"""Generic container with type-safe magic methods."""
def __init__(self, items: list[T]) -> None:
self._items = items
def __getitem__(self, index: int) -> T:
return self._items[index]
def __iter__(self) -> Iterator[T]:
return iter(self._items)
def __len__(self) -> int:
return len(self._items)
def map(self, func: Callable[[T], U]) -> 'Container[U]':
"""Type-safe map operation."""
return Container([func(item) for item in self._items])
# Type checker knows the types
int_container: Container[int] = Container([1, 2, 3])
str_container: Container[str] = int_container.map(str)
print(str_container._items) # ['1', '2', '3']Async Magic Methods
Async Context Managers
import asyncio
from typing import Optional, Any
class AsyncFileManager:
"""Async context manager for file operations."""
def __init__(self, filename: str, mode: str = 'r') -> None:
self.filename = filename
self.mode = mode
self.file: Optional[Any] = None
async def __aenter__(self) -> Any:
"""Async version of __enter__."""
print(f"Opening {self.filename}")
# Simulating async file open
await asyncio.sleep(0.1)
self.file = open(self.filename, self.mode)
return self.file
async def __aexit__(
self,
exc_type: Optional[type],
exc_val: Optional[Exception],
exc_tb: Optional[Any]
) -> bool:
"""Async version of __exit__."""
if self.file:
self.file.close()
print(f"Closed {self.filename}")
return False
# Usage
async def process_file():
async with AsyncFileManager('data.txt', 'w') as f:
f.write("Hello, async world!")
# asyncio.run(process_file())Async Iterators
import asyncio
from typing import AsyncIterator
class AsyncRange:
"""Async iterator example."""
def __init__(self, start: int, end: int, delay: float = 0.1) -> None:
self.start = start
self.end = end
self.delay = delay
self.current = start
def __aiter__(self) -> 'AsyncRange':
"""Return async iterator."""
return self
async def __anext__(self) -> int:
"""Async version of __next__."""
if self.current >= self.end:
raise StopAsyncIteration
await asyncio.sleep(self.delay)
value = self.current
self.current += 1
return value
# Usage
async def main():
async for num in AsyncRange(0, 5):
print(num)
# asyncio.run(main())Pattern Matching (Python 3.10+)
__match_args__ for Structural Pattern Matching
from dataclasses import dataclass
from typing import Union
@dataclass
class Point:
__match_args__ = ('x', 'y') # Enable positional matching
x: float
y: float
@dataclass
class Circle:
__match_args__ = ('center', 'radius')
center: Point
radius: float
@dataclass
class Rectangle:
__match_args__ = ('top_left', 'width', 'height')
top_left: Point
width: float
height: float
Shape = Union[Circle, Rectangle, Point]
def describe_shape(shape: Shape) -> str:
"""Use pattern matching with custom classes."""
match shape:
case Point(0, 0):
return "Origin point"
case Point(x, 0):
return f"Point on X-axis at {x}"
case Point(0, y):
return f"Point on Y-axis at {y}"
case Point(x, y):
return f"Point at ({x}, {y})"
case Circle(Point(0, 0), r):
return f"Circle at origin with radius {r}"
case Circle(center, radius):
return f"Circle at {center} with radius {radius}"
case Rectangle(Point(x, y), w, h):
return f"Rectangle at ({x}, {y}) with size {w}x{h}"
case _:
return "Unknown shape"
# Test pattern matching
print(describe_shape(Point(0, 0))) # Origin point
print(describe_shape(Circle(Point(0, 0), 5))) # Circle at origin with radius 5Production Best Practices
1. Defensive Magic Method Implementation
from typing import Any
import logging
logger = logging.getLogger(__name__)
class RobustClass:
"""Production-ready magic method implementation."""
def __init__(self, value: int) -> None:
self._value = value
def __add__(self, other: Any) -> 'RobustClass':
"""Defensive addition with logging."""
if not isinstance(other, (RobustClass, int, float)):
logger.warning(f"Unsupported type for addition: {type(other)}")
return NotImplemented
try:
if isinstance(other, RobustClass):
result = self._value + other._value
else:
result = self._value + other
return RobustClass(result)
except (OverflowError, ValueError) as e:
logger.error(f"Addition failed: {e}")
raise
def __repr__(self) -> str:
"""Safe repr that never raises."""
try:
return f"RobustClass({self._value!r})"
except Exception as e:
logger.error(f"__repr__ failed: {e}")
return f"<RobustClass object at {hex(id(self))}>"
def __eq__(self, other: Any) -> bool:
"""Type-safe equality."""
if not isinstance(other, RobustClass):
return NotImplemented
return self._value == other._value
def __hash__(self) -> int:
"""Consistent hash with __eq__."""
return hash(self._value)2. Thread-Safe Magic Methods
import threading
from typing import Any
from contextlib import contextmanager
class ThreadSafeCounter:
"""Thread-safe counter with magic methods."""
def __init__(self, initial: int = 0) -> None:
self._value = initial
self._lock = threading.RLock() # Reentrant lock
@contextmanager
def _synchronized(self):
"""Context manager for thread-safe operations."""
self._lock.acquire()
try:
yield
finally:
self._lock.release()
def __iadd__(self, other: int) -> 'ThreadSafeCounter':
with self._synchronized():
self._value += other
return self
def __int__(self) -> int:
with self._synchronized():
return self._value
def __repr__(self) -> str:
with self._synchronized():
return f"ThreadSafeCounter({self._value})"
# Usage in multithreaded environment
counter = ThreadSafeCounter()
def increment():
for _ in range(1000):
counter += 1
threads = [threading.Thread(target=increment) for _ in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
print(int(counter)) # 10000 (always correct due to locking)3. Comprehensive Documentation Standards
from typing import Union
class Vector:
"""
Immutable 2D vector with comprehensive magic method implementation.
This class implements the full suite of arithmetic, comparison,
and container protocols for production use.
Attributes:
x: X-coordinate of the vector
y: Y-coordinate of the vector
Examples:
>>> v1 = Vector(1, 2)
>>> v2 = Vector(3, 4)
>>> v3 = v1 + v2
>>> print(v3)
Vector(4, 6)
>>> abs(v1)
2.23606797749979
Note:
All operations return new Vector instances (immutable).
"""
__slots__ = ('_x', '_y')
def __init__(self, x: float, y: float) -> None:
"""
Initialize a new Vector.
Args:
x: X-coordinate
y: Y-coordinate
Raises:
TypeError: If x or y are not numeric
"""
if not isinstance(x, (int, float)) or not isinstance(y, (int, float)):
raise TypeError(f"Coordinates must be numeric, got {type(x)}, {type(y)}")
object.__setattr__(self, '_x', float(x))
object.__setattr__(self, '_y', float(y))
def __add__(self, other: Union['Vector', tuple[float, float]]) -> 'Vector':
"""
Add two vectors or a vector and a tuple.
Args:
other: Vector or (x, y) tuple to add
Returns:
New Vector representing the sum
Raises:
TypeError: If other is not a Vector or tuple
Examples:
>>> Vector(1, 2) + Vector(3, 4)
Vector(4, 6)
"""
if isinstance(other, Vector):
return Vector(self._x + other._x, self._y + other._y)
elif isinstance(other, tuple) and len(other) == 2:
return Vector(self._x + other[0], self._y + other[1])
return NotImplemented
def __repr__(self) -> str:
"""Return unambiguous string representation."""
return f"Vector({self._x}, {self._y})"Summary
Summary
Magic methods are powerful tools that allow you to:
- Customize object behavior with Python’s built-in operations
- Create intuitive APIs that feel natural to Python developers
- Integrate seamlessly with Python’s standard library
- Implement design patterns elegantly
- Optimize performance with slots and lazy evaluation
- Enable type safety with protocols and generics
- Support async operations with async magic methods
- Implement pattern matching with
__match_args__
Key categories:
- Lifecycle:
__init__,__new__,__del__,__post_init__ - Representation:
__repr__,__str__,__format__,__bytes__ - Comparison:
__eq__,__lt__,__gt__,__le__,__ge__,__ne__ - Arithmetic:
__add__,__sub__,__mul__,__truediv__,__floordiv__,__mod__,__pow__ - Containers:
__len__,__getitem__,__setitem__,__contains__,__iter__,__reversed__ - Attributes:
__getattr__,__setattr__,__delattr__,__getattribute__,__dir__ - Callables:
__call__ - Context Managers:
__enter__,__exit__,__aenter__,__aexit__ - Descriptors:
__get__,__set__,__delete__,__set_name__ - Async:
__aiter__,__anext__,__aenter__,__aexit__ - Memory:
__slots__,__sizeof__ - Pattern Matching:
__match_args__
Senior Engineer Checklist:
- ✅ Return
NotImplementedfor unsupported operations - ✅ Implement
__hash__when implementing__eq__ - ✅ Use
super()to avoid infinite recursion - ✅ Add comprehensive type hints
- ✅ Document all magic methods with examples
- ✅ Test edge cases thoroughly (including thread safety)
- ✅ Profile performance for hot paths
- ✅ Use
__slots__for memory-intensive classes - ✅ Implement protocols for structural subtyping
- ✅ Consider immutability (frozen dataclasses)
- ✅ Use async variants for I/O-bound operations
- ✅ Add logging for debugging
- ✅ Implement defensive coding practices
- ✅ Follow PEP 8 and type checking (mypy/pyright)
Additional Resources:
Mastering magic methods will help you write more Pythonic, elegant, performant, and maintainable code at scale!
Quick Reference Card
Object Lifecycle
__new__(cls, ...) # Object creation (before __init__)
__init__(self, ...) # Object initialization
__del__(self) # Object destruction (finalizer)
__post_init__(self) # Dataclass post-initializationString Representation
__repr__(self) # Official representation (for developers)
__str__(self) # Informal representation (for users)
__format__(self, spec) # Custom formatting (f-strings)
__bytes__(self) # Bytes representationComparison Operators
__eq__(self, other) # ==
__ne__(self, other) # !=
__lt__(self, other) # <
__le__(self, other) # <=
__gt__(self, other) # >
__ge__(self, other) # >=
__hash__(self) # hash() - needed for sets/dictsArithmetic Operators
__add__(self, other) # +
__sub__(self, other) # -
__mul__(self, other) # *
__matmul__(self, other) # @ (matrix multiplication)
__truediv__(self, other) # /
__floordiv__(self, other)# //
__mod__(self, other) # %
__pow__(self, other) # **
__divmod__(self, other) # divmod()Reflected Operators (right-hand side)
__radd__, __rsub__, __rmul__, __rtruediv__, __rfloordiv__, __rmod__, __rpow__Augmented Assignment
__iadd__(self, other) # +=
__isub__(self, other) # -=
__imul__(self, other) # *=
__itruediv__(self, other)# /=
__ifloordiv__(self, other)# //=
__imod__(self, other) # %=
__ipow__(self, other) # **=Unary Operators
__neg__(self) # -x
__pos__(self) # +x
__abs__(self) # abs(x)
__invert__(self) # ~x
__round__(self, n) # round(x, n)
__floor__(self) # math.floor(x)
__ceil__(self) # math.ceil(x)
__trunc__(self) # math.trunc(x)Bitwise Operators
__and__(self, other) # &
__or__(self, other) # |
__xor__(self, other) # ^
__lshift__(self, other) # <<
__rshift__(self, other) # >>Type Conversion
__int__(self) # int(x)
__float__(self) # float(x)
__complex__(self) # complex(x)
__bool__(self) # bool(x), if x:
__index__(self) # operator.index(x)Container Emulation
__len__(self) # len(x)
__getitem__(self, key) # x[key]
__setitem__(self, key, value) # x[key] = value
__delitem__(self, key) # del x[key]
__contains__(self, item) # item in x
__iter__(self) # for i in x:
__reversed__(self) # reversed(x)
__missing__(self, key) # dict subclass, missing keyAttribute Access
__getattr__(self, name) # x.name (when not found normally)
__setattr__(self, name, value) # x.name = value
__delattr__(self, name) # del x.name
__getattribute__(self, name) # x.name (always called)
__dir__(self) # dir(x)Descriptors
__get__(self, instance, owner) # Descriptor getter
__set__(self, instance, value) # Descriptor setter
__delete__(self, instance) # Descriptor deleter
__set_name__(self, owner, name) # Descriptor name assignmentCallable Objects
__call__(self, ...) # x(args)Context Managers
__enter__(self) # with x:
__exit__(self, exc_type, exc_val, exc_tb) # End of with blockAsync Context Managers & Iterators
__aenter__(self) # async with x:
__aexit__(self, ...) # End of async with block
__aiter__(self) # async for i in x:
__anext__(self) # Next item in async iterationCopying & Serialization
__copy__(self) # copy.copy(x)
__deepcopy__(self, memo) # copy.deepcopy(x)
__getstate__(self) # pickle.dump(x)
__setstate__(self, state)# pickle.load()
__reduce__(self) # Advanced pickle control
__reduce_ex__(self, protocol) # Protocol-specific picklingMemory & Introspection
__sizeof__(self) # sys.getsizeof(x)
__slots__ # Class attribute for memory optimizationPattern Matching (3.10+)
__match_args__ # Class attribute for positional matchingMetaclass Methods
__init_subclass__(cls, **kwargs) # Called when subclassed
__class_getitem__(cls, item) # Class[Type] syntax
__prepare__(mcs, name, bases) # Metaclass namespace creation
__instancecheck__(cls, instance) # isinstance() behavior
__subclasscheck__(cls, subclass) # issubclass() behaviorCommon Patterns Quick Reference
Immutable Object:
@dataclass(frozen=True, slots=True)
class Point:
x: float
y: floatComparable Object:
from functools import total_ordering
@total_ordering
class Item:
def __eq__(self, other): ...
def __lt__(self, other): ...Hashable Object:
def __eq__(self, other): return self.x == other.x
def __hash__(self): return hash(self.x)Context Manager:
def __enter__(self): return resource
def __exit__(self, *args): cleanup(); return FalseIterator:
def __iter__(self): return self
def __next__(self):
if done: raise StopIteration
return next_itemCallable:
def __call__(self, *args, **kwargs): return resultEnd of Guide – Happy Pythoning! 🐍
Discover more from Altgr Blog
Subscribe to get the latest posts sent to your email.
