Practical application scenarios of Python iterators and generators

Get more information

Personal website: Brother Tao talks about Python

In Python, iterators and generators are two key concepts that provide us with powerful tools for processing various data sequences.

Iterators and generators not only allow us to manipulate data more efficiently, but also greatly reduce memory usage, especially when processing large data sets.

Iterators

What is an iterator?

An iterator is a special object that can iterate over a sequence of data. It allows you to access the elements of a sequence one by one without loading the entire sequence into memory. Most data structures in Python can be used as iterable objects, such as lists, tuples, strings, etc.

Iterator protocol

Iterator objects must comply with the following two methods:

__iter__(): Returns the iterator itself.
__next__(): Returns the next element in the sequence. If there are no elements to iterate over, a StopIteration exception is raised.

Sample code demonstrating how to create a custom iterator:

class MyIterator:
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.current < self.end:
            self.current + = 1
            return self.current - 1
        raise StopIteration

# Use custom iterator
my_iterator = MyIterator(0, 3)
for item in my_iterator:
    print(item) # Output 0, 1, 2
</code><img class="look-more-preCode contentImg-no-view" src="//i2.wp.com/csdnimg.cn/release/blogv2/dist/pc/img/newCodeMoreBlack. png" alt="" title="">

Iterators and `for` loops

The for loop in Python is used to iterate over the elements in an iterable object. When we use a for loop, the __iter__() method of the iterable object will be automatically called, and the __next__() method will be used to traverse the elements. Until a StopIteration exception is raised.

numbers = [1, 2, 3, 4, 5]
for num in numbers:
    print(num) # Output 1, 2, 3, 4, 5

Iterable object

Iterable objects are objects that implement the __iter__() method and can be used as the basis for iterators. There are many built-in iterable objects in the Python standard library, such as range(), enumerate(), zip(), etc.

Generators

What is a generator?

Generators are a special type of iterator that allow you to generate values on demand rather than all at once.

This on-demand generation approach is useful, especially when working with large amounts of data, to reduce memory footprint and improve performance.

Generator function

A generator function is a function that contains a yield statement, not a return. When a function contains a yield statement, it becomes a generator function. Each time the generator’s __next__() method is called, the function will continue executing from the location of the last yield until the next yield is encountered. or the function ends.

Let’s look at an example showing how to create a generator function:

def countdown(n):
    while n > 0:
        yield n
        n -= 1

# Use generator function
for i in countdown(5):
    print(i) # Output 5, 4, 3, 2, 1

Generator expression

A generator expression is similar to a list comprehension, but it returns a generator object instead of generating all elements at once. Very useful when working with large amounts of data.

# Generator expression example
even_numbers = (x for x in range(10) if x % 2 == 0)
for num in even_numbers:
    print(num) # Output 0, 2, 4, 6, 8

Lazy evaluation of generators

Generators generate data in a lazy manner, calculating and returning data only when needed. This means that the generator does not generate all values at once, thus reducing the memory footprint.

Infinite sequence of generators

Thanks to lazy evaluation of generators, infinite sequences can be created without worrying about memory issues. For example, generate infinite Fibonacci numbers:

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# Generate infinite Fibonacci numbers
fib = fibonacci()
for _ in range(10):
    print(next(fib)) # Output 0, 1, 1, 2, 3, 5, 8, 13, 21, 34

The difference and connection between Python’s iterators and generators

Python’s Iterators and Generators are both concepts used to process data sequences, but there are some important differences and connections in working methods, uses, and implementations.

Difference:

1. Working method:

Iterator: An iterator is an object that implements the __iter__() and __next__() methods. By calling the __iter__() method, you can obtain the iterator itself, and then access the elements in the sequence one by one by repeatedly calling the __next__() method.
Generator: A generator is a special iterator created by including a yield statement in a function definition. The generator function will pause and retain the current state each time it calls yield. The next time it is called, it will continue execution from the last paused position.

2. Memory usage:

Iterators: Iterators need to hold the entire sequence in memory, which can take up a lot of memory if the sequence is large.
Generators: Generators generate data in a lazy evaluation manner, calculating and returning values only when needed, so they have a low memory footprint and are especially suitable for processing large data sets.

3. Implementation method:

Iterator: You can customize the iterator class and implement the __iter__() and __next__() methods to define the iteration behavior. In addition, Python provides many built-in iterable objects and iterators, such as lists, tuples, strings, etc.
Generators: Generators can be created through generator functions (containing yield statements) or generator expressions (similar to list comprehensions). Generator functions are a more flexible way to generate values dynamically.

Contact:

1. Both are used to process data sequences: Both iterators and generators are used to process data sequences, allowing access to elements one by one without having to load the entire sequence at once.

2. Both can be used in for loops: Iterators and generators can be used in for loops, which is a common use. The for loop will automatically call the iterator’s __next__() method to traverse the elements in the sequence.

3. Both iterators and generators can implement lazy evaluation: Both iterators and generators support lazy evaluation, which only calculates and returns values when needed, which helps save memory.

4. Infinite sequences can be created: Generators can be used to create infinite sequences, and iterators can also be used to process infinite sequences of data.

Example:

Sample code showing the differences and connections between iterators and generators:

# Iterator example
class MyIterator:
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.current < self.end:
            self.current + = 1
            return self.current - 1
        raise StopIteration

#Generator example
def countdown(n):
    while n > 0:
        yield n
        n -= 1

# Use iterator
my_iterator = MyIterator(0, 3)
for item in my_iterator:
    print(item) # Output 0, 1, 2

# Use generator
for i in countdown(5):
    print(i) # Output 5, 4, 3, 2, 1
</code><img class="look-more-preCode contentImg-no-view" src="//i2.wp.com/csdnimg.cn/release/blogv2/dist/pc/img/newCodeMoreBlack. png" alt="" title="">

In this example, you show how to use custom iterators and generator functions to process a sequence of data. Although different in implementation, both can access elements one by one and support lazy evaluation.

Summary

Iterators are the most basic iteration tools in Python, allowing us to access the elements of a data sequence one by one without loading the entire sequence into memory at once.

Generators take iteration to a whole new level. They generate data in a more flexible and efficient way, only calculating when needed, greatly improving performance.

In-depth study of the working principles, uses and examples of iterators and generators will help you fully understand these two important concepts and choose them reasonably to deal with various data processing tasks in actual programming.

Python learning route

Get more information

Personal website: Brother Tao talks about Python

If you want to get more and richer information, you can click on the business card below the article and reply [High-quality information] to get a comprehensive learning information package.

Click on the link card below the article and reply [Quality Information] to directly receive the information gift package.

Get more information

Iterators

What is an iterator?

Iterator protocol

Iterators and for loops

Iterable object

Generators

What is a generator?

Generator function

Generator expression

Lazy evaluation of generators

Infinite sequence of generators

The difference and connection between Python’s iterators and generators

Difference:

Contact:

Example:

Summary

Python learning route

Get more information

Iterators and `for` loops