In Python, everything is an object. The object is kept alive by the interpreter as long as there is a reference variable pointing to it. As soon as the last reference to the object is removed, the object becomes eligible to be collected by the garbage collector. The garbage collector will collect it when it runs the collection cycle. The object in the meantime stays in the memory until collected by the garbage collector but inaccessible to the programmer. Python provides a module named weakref which lets us create weak references to the object. The weak references are not strong enough as normal reference variables and object with only weak references to them will be collected by the garbage collector whenever it sees right. The object will be accessible through weak reference until it’s collected by the garbage collector. This can be useful in situations like creating a cache, mappings of large objects whose loading and removal process is memory heavy. The weakref module let us create weak references to class instances, python functions, instance methods, sets, generators, sockets, arrays, etc. It does not let us create references to built-in types like list, dict, etc directly but we can subclass them, and then it'll let us create the weak references. The built-in types like tuple and int do not support weak references even with subclassing.
As a part of this tutorial, we'll explain with simple examples how we can use different methods of weakref modules with very simple and easy to use examples. It has methods to create weak references, proxy objects, dictionary with weakly referenced keys, dictionary with weakly referenced values, set with weakly referenced items, weakly references instance methods, etc. We'll now start explaining them with simple examples.
If you are interested in reading about how garbage collection works in Python then please feel free to check our small blog on it.
Our first example explains how we can create weak references to any object in the python ref() method of weakref module.
The code of our example starts with the creation of a simple employee class with attributes id, name, and age. We have then created 2 employee instance and printed their details. We have then created a weak reference to employee1 instance and printed it. We have then called weak reference and stored it in another reference variable named empl1. We have now two strong references (employee1, empl1) and one weak reference (e1_weak) to employee1 instance. We are then printing employee details using empl1 reference. Finally, we are deleting both employee1 and empl1 strong references and trying to access an object again through a weak reference.
We can notice from the output that when we try to print object details after both strong references are deleted and the object is garbage collected, it returns None.
Please make a note at the type of weakref object which is getting printed as initially, it shows the reference to employee instance and then it shows dead.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
print("Type of Employee (employee1) : {}".format(type(employee1)))
print("\nEmployee Details (Normal) : {}".format(employee1))
e1_weak = weakref.ref(employee1)
print("\nWeak Reference : {}".format(e1_weak))
empl1 = e1_weak()
print("\nEmployee Details (Instance Created through Weakref) : {}".format(empl1))
print("\nType of Employee (empl1) : {}".format(type(empl1)))
print("\nIs empl1 and employee1 same? : {}".format(empl1 is employee1))
del employee1, empl1
print("\nWeak Reference after deletion of both references : {}".format(e1_weak))
print("\nEmployee Details (Weakref) : {}".format(e1_weak()))
As a part of our second example, we are explaining how we can add callbacks when creating weak references using ref() method. Our code for this example builds on our last example.
This example has an employee class like the previous example. We'll be using this class for all of our examples. We are creating two employee instances. We are then creating three callback function all of which accepts weakly referenced object and prints simple statement about their execution. We are then creating three weak references to employee1 instance and giving different callbacks to each one (check sequence). We are then printing employee details through all three weak references. Finally, we are deleting strong reference to employee1 instance hence it becomes eligible for garbage collection. Our callbacks will be executed when the object is collected by the garbage collector.
Please make a note that callbacks will be executed in order from recent to old. We have changed the callback sequence to show that in action.
We can notice from the output that the second callback is executed first as it was last registered, then the third callback, and then the first callback.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
def callback1(weakrf):
print('\nFirst Callback Executed. Weakref : {}'.format(weakrf))
def callback2(weakrf):
print('\nSecond Callback Executed. Weakref : {}'.format(weakrf))
def callback3(weakrf):
print('\nThird Callback Executed. Weakref : {}'.format(weakrf))
e1_weak1 = weakref.ref(employee1, callback1)
e1_weak3 = weakref.ref(employee1, callback3)
e1_weak2 = weakref.ref(employee1, callback2)
print("Employee Details (Weakref-e1_weak1) : {}".format(e1_weak1()))
print("\nEmployee Details (Weakref-e1_weak2) : {}".format(e1_weak2()))
print("\nEmployee Details (Weakref-e1_weak3) : {}".format(e1_weak3()))
del employee1
The weakref let us create more than one reference to an object as we had discussed earlier. In this example, we'll explain two important methods that let us count and get all weak references to an object.
Our code for this example starts with the creation of two employee instances. We then create three weak references to employee1 instance. We are then checking the type of weak reference using weakref.ReferenceType constant available through weakref module. As of last, we are printing count of weak references and actual weak references to employee1 instance.
Please make a note that even though we have created three weak references the methods returns count as only one. This might be due to the scope of the code. It'll return more than one when weak references are created in different scope.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
e1_weak1 = weakref.ref(employee1)
e1_weak3 = weakref.ref(employee1)
e1_weak2 = weakref.ref(employee1)
print("WeakRef 1 : {}".format(e1_weak1))
print("WeakRef 2 : {}".format(e1_weak2))
print("WeakRef 3 : {}".format(e1_weak3))
print("\nIs e1_weak1 weak ref ? : {}".format(isinstance(e1_weak1, weakref.ReferenceType)))
print("\nNumber of Weak References to employee1 : {}".format(weakref.getweakrefcount(employee1)))
print("\nWeak References to Object : {}".format(weakref.getweakrefs(employee1)))
Our code for the fourth example is almost the same as our code for the first example with few minor changes. We are using proxy() method to create weak references.
The weakref.ProxyType let us check if the object is of proxy type but it won't work for the proxy to callable objects. The weakref.ProxyType will let us check if an object is of proxy type and will work for the proxy to callable objects as well.
Please make a note that we have covered call to proxy object at last in the try-except block because the proxy object will raise an error if the object referred to is garbage collected, unlike weak reference which returns None.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
print("Type of Employee (employee1) : {}".format(type(employee1)))
print("\nEmployee Details (Normal) : {}".format(employee1))
e1_weak = weakref.proxy(employee1)
print("\nType of Employee (e1_weak) : {}".format(type(e1_weak)))
print("\nIs e1_weak proxy ? : {}".format(isinstance(e1_weak, weakref.ProxyType)))
print("\nIs e1_weak proxy ? : {}".format(isinstance(e1_weak, weakref.ProxyTypes)))
print("\nEmployee Details (Weakref) : {}".format(e1_weak))
del employee1
try:
print("\nWeak Reference after deletion of both references : {}".format(e1_weak))
except Exception as e:
print("\nError : {} : Access to weak referenced object failed.".format(type(e).__name__))
Our code for our fifth example is exactly the same as our second example with the only change that we are using proxy() method instead of ref() method to create weak references.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
def callback1(weakrf):
print('\nFirst Callback Executed.')
def callback2(weakrf):
print('\nSecond Callback Executed.')
def callback3(weakrf):
print('\nThird Callback Executed.')
e1_weak1 = weakref.proxy(employee1, callback1)
e1_weak3 = weakref.proxy(employee1, callback3)
e1_weak2 = weakref.proxy(employee1, callback2)
print("Employee Details (Weakref-e1_weak1) : {}".format(e1_weak1))
print("\nEmployee Details (Weakref-e1_weak2) : {}".format(e1_weak2))
print("\nEmployee Details (Weakref-e1_weak3) : {}".format(e1_weak3))
del employee1
As a part of our sixth example, we are demonstrating how we can create a dictionary whose keys are weekly referenced using WeakKeyDictionary class. We can create an instance of WeakKeyDictionary class with an existing dictionary or without any argument. It provides API the same as a normal dictionary to add and remove mapping entries to it.
Our code first creates three employee instances. It then creates an instance of WeakKeyDictionary with a dictionary where the key is the employee instance and the value is the employee name. We are then printing dictionary, retrieving dictionary keys, and printing employee names. We are calling keyrefs() method to retrieve weak references. We are then deleting the reference to employee1 and printing dictionary keys again. We are then creating a new employee instance and adding it to a dictionary with weakly referenced keys. At last, we are printing keys of dictionary again.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
employee3 = Employee(789, "Russell", 40)
weak_key_dict = weakref.WeakKeyDictionary({employee1: employee1.name,
employee2: employee2.name,
employee3: employee3.name})
print("Weak Key Dictionary : {}\n".format(weak_key_dict.data))
print("Dictionary Keys : {}\n".format([key().name for key in weak_key_dict.keyrefs()]))
del employee1
print("Dictionary Keys : {}\n".format([key().name for key in weak_key_dict.keyrefs()]))
employee4 = Employee(135, "Peter", 32)
weak_key_dict.update({employee4: employee4.name})
print("Dictionary Keys : {}\n".format([key().name for key in weak_key_dict.keyrefs()]))
As a part of our seventh example, we are demonstrating how we can create a dictionary with weakly referenced values using WeakValueDictionary.
Our code for this example is exactly the same as our previous example but this time we are using employee name as key and employee instance as values. We are using valuerefs() method to retrieve weakly referenced values of the dictionary.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
employee3 = Employee(789, "Russell", 40)
weak_value_dict = weakref.WeakValueDictionary({employee1.name: employee1,
employee2.name: employee2,
employee3.name: employee3})
print("Weak Value Dictionary : {}\n".format(weak_value_dict.data))
print("Dictionary Values : {}\n".format([value().name for value in weak_value_dict.valuerefs()]))
del employee1
print("Dictionary Values : {}\n".format([value().name for value in weak_value_dict.valuerefs()]))
employee4 = Employee(135, "Peter", 32)
weak_value_dict.update({employee4.name: employee4})
print("Dictionary Values : {}\n".format([value().name for value in weak_value_dict.valuerefs()]))
Our eighth example explains how we can use WeakSet class to create a set where all set items is a weak reference.
Our code for this example creates three employee instances and creates a weak set of these three instances. It then prints individual values of the weak set. It then deletes strong reference to employee1 and prints weak set values. It then creates a new employee instance, adds it to the weak set, and prints weak set values again.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
employee3 = Employee(789, "Russell", 40)
weak_set = weakref.WeakSet([employee1, employee2, employee3])
print("Weak Set : {}\n".format(weak_set.data))
for value in weak_set:
print(value)
print("\nWeakset Values : {}\n".format([value.name for value in weak_set]))
del employee1
print("Weakset Values : {}\n".format([value.name for value in weak_set]))
employee4 = Employee(135, "Peter", 32)
weak_set.add(employee4)
print("Weakset Values : {}\n".format([value.name for value in weak_set]))
Our ninth example demonstrates how we can create a weak reference to the python function using ref() method.
We have created a simple function named addition which adds two values if they are float or int. We have then created a weak reference to addition function. We are then calling weak ref to execute the method. At last, we are deleting reference to addition method and then calling again weak reference.
Please make a note that calling weak reference will return a reference to function and we'll need to call it again with arguments.
import weakref
def addition(a, b):
if isinstance(a, (float, int)) and isinstance(b, (float, int)):
return a + b
else:
return None
weak_method = weakref.ref(addition)
print("Method Weak Ref : {}".format(weak_method))
print("\nAddition of {} & {} is {}".format(10,20, weak_method()(10,20) if weak_method() else None))
del addition
print("\nAddition of {} & {} is {}".format(20,20, weak_method()(10, 20) if weak_method() else None))
The code of our example is exactly the same as our ninth example with minor changes like usage of proxy() method to create a weak proxy and checking proxy object type using weakref.CallableProxyType and weakref.ProxyTypes constants. We have also covered code in the try-except block because proxy object fails if an object is collected by the garbage collector, unlike weak reference which returns None.
import weakref
def addition(a, b):
if isinstance(a, (float, int)) and isinstance(b, (float, int)):
return a + b
else:
return None
weak_method_proxy = weakref.proxy(addition)
print("Method Weak Ref : {}".format(weak_method_proxy))
print("\nIs weak_method_proxy proxy ? : {}".format(isinstance(weak_method_proxy, weakref.CallableProxyType)))
print("\nIs weak_method_proxy proxy ? : {}".format(isinstance(weak_method_proxy, weakref.ProxyTypes)))
try:
print("\nAddition of {} & {} is {}".format(10,20, weak_method_proxy(10,20)))
del addition
print("\nAddition of {} & {} is {}".format(20,20, weak_method_proxy(10, 20)))
except Exception as e:
print("\nError : {}".format(e))
Our eleventh example demonstrates how we can create a weak reference to instance methods using WeakMethod. The weak reference creation to instance methods using ref() method won't work hence we need to use WeakMethod class for it.
Our code creates 3 instances of employees and creates a weak reference to pretty_print() method of employee1 instance. We then print the type of weak reference created through WeakMethod. We then call the method through a weak reference. At last, we delete employee1 instance and try to check weak reference which will also return None if the instance is collected by the garbage collector.
import weakref
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
def pretty_print(self):
print("Employee Details (ID : {}, Name : {}, Age : {})".format(self.emp_id, self.name, self.age))
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
employee3 = Employee(789, "Russell", 40)
weak_method = weakref.WeakMethod(employee1.pretty_print)
print("Weak Ref to Method : {}".format(weak_method))
print("\nMethod Ref : {}\n".format(weak_method()))
weak_method()()
del employee1
print("\nWeak Ref to Method : {}".format(weak_method))
As a part of our twelfth and last example, we'll demonstrate usage of method finalize() which let us execute callback just before an object is collected by the garbage collector.
Our code first creates three employee instances. We have the created three callbacks that we want to execute at the time of garbage collection of objects. Two of the callbacks do not have any parameter and one does have. We are then creating three finalize instances by calling finalize() method giving it employee1 instance and different callbacks. We are passing the argument for free_up_resources callback as well. We are then calling detach() method of finalizing instance which indicates to detach that finalizer from instance hence it won't be executed during garbage collection. We are then printing values of alive and atexit attributes of the finalize instance which tells us whether the finalizer is alive and whether it'll be executed at the exit or not. We are then deleting a strong reference to employee1 instance. At last, we are printing the value of alive attribute of each finalize instance.
We can notice from the output that 2 callbacks get executed when the object is collected by the garbage collector. We can see that after we detached the third finalizer, atexit attribute returns False indicating that it won't be executed during collection.
import weakref
import datetime
class Employee:
def __init__(self, emp_id, name, age):
self.emp_id = emp_id
self.name = name
self.age = age
def __repr__(self):
return "{} : {} : {}".format(self.emp_id, self.name, self.age)
employee1 = Employee(123, "William", 30)
employee2 = Employee(456, "Dalton", 35)
employee3 = Employee(789, "Russell", 40)
def optional_finalizer():
print("This finalizer won't be executed when object will be deleted.")
def log_object_deletion():
print("Object deleted at : {}".format(datetime.datetime.now()))
def free_up_resources(*args):
print("\nReleasing all resources used by obejct before deleting it.")
for arg in args:
print("Releasing : {}".format(arg))
finalizer1 = weakref.finalize(employee1, log_object_deletion)
finalizer2 = weakref.finalize(employee1, free_up_resources, "db", "files")
finalizer3 = weakref.finalize(employee1, optional_finalizer)
finalizer3.detach()
print("Type of Finalize Object : {} for object : {}".format(finalizer1, employee1))
print("\nIs finalizer1 alive? : {}".format(finalizer1.alive))
print("Will finalizer1 be executed at exit? : {}".format(finalizer1.atexit))
print("\nIs finalizer2 alive? : {}".format(finalizer2.alive))
print("Will finalizer2 be executed at exit? : {}".format(finalizer2.atexit))
print("\nIs finalizer3 alive? : {}".format(finalizer3.alive))
print("Will finalizer3 be executed at exit? : {}".format(finalizer3.atexit))
del employee1
print("\nIs finalizer1 alive? : {}".format(finalizer1.alive))
print("Is finalizer1 alive? : {}".format(finalizer2.alive))
print("Is finalizer1 alive? : {}".format(finalizer3.alive))
This ends our small tutorial explaining the API of weakref module. Please feel free to let us know your views in the comments section.
If you are more comfortable learning through video tutorials then we would recommend that you subscribe to our YouTube channel.
When going through coding examples, it's quite common to have doubts and errors.
If you have doubts about some code examples or are stuck somewhere when trying our code, send us an email at coderzcolumn07@gmail.com. We'll help you or point you in the direction where you can find a solution to your problem.
You can even send us a mail if you are trying something new and need guidance regarding coding. We'll try to respond as soon as possible.
If you want to