“Tech’s first encounter” Linux driver blkdev

1. Motivation

In the world of Unix-like OS, I/O devices are treated as special files such as device files. For example, the same write() system call can be used to write data to a normal file or to a peripheral device such as a printer. According to the characteristics of device files, they can be divided into character devices and block devices.

Character devices generally do not support random access, such as mice and keyboards;
Block devices support random access, such as hard disks

Why should we write device drivers? Mainly to control the plugged-in external devices, this article mainly explains the block devices in detail. If you have any needs for character devices, please go to “Tech’s First Look” Linux driver blkdev

2. Solution

S1 – Block device driver framework

The kernel uses the block_device structure to represent block devices, which is defined in include/linux/fs.h. There are too many member variables inside the structure, so I decided not to explain them in detail until they are used. Regarding block_device, we only need to understand two points here.

(1) Register block device

This method is to register a new block device with the kernel and apply for a device number. The prototype is,

int register_blkdev(unsigned int major, const char* name)

Among them, major is the main device number, and name is the block device name. If major is 0, it means that the major device number is automatically assigned by the system; if major is between 1 ~ 255, it means a customized major device number. A return value of 0 indicates successful registration, otherwise it fails. The content of the primary and secondary device numbers is also explained in detail in “Tech’s First Look” Linux Driver blkdev

(2) Unregister block device

This method is to unregister the specified block device in the kernel. The prototype is,

void unregister_blkdev(unsigned int major, const char* name)

gendisk is the most important structure of the block device, which means a general disk. It is defined in include/linux/genhd.h. It can be understood that gendisk is the middleman between the block device node we created and the kernel. Similarly, we do not need to go too deep at the beginning. To fully understand the internal member variables, you only need to remember these methods for applying for and releasing gendisk.

(3) Apply for gendisk

Before using gendisk, you must apply first. The prototype is,

struct gendisk* alloc_disk(int minors)

Among them, minors is the number of minor device numbers, that is, the number of partitions corresponding to gendisk

(4) Delete gendisk

The prototype is,

void del_gendisk(struct gendisk* gdisk)

(5) Add gendisk to kernel

After applying to gendisk, initialize it and then add it to the kernel. The prototype is,

void add_disk(struct gendisk* gdisk)

(6) Set gendisk capacity

When initializing gendisk, you need to set its capacity. The prototype is,

void set_capacity(struct gendisk* gdisk, sector_t size)

The size is the disk capacity. Note that this refers to the number of sectors. A sector is usually 512 bytes.

(7)gendisk reference counting

Increase gendisk’s reference count,

struct kobject* get_disk(struct gendisk* gdisk)

Reduce gendisk’s reference count,

void put_disk(struct gendisk* disk)

When no one in the kernel refers to the gendisk anymore, the kernel can safely release this space.
block_device_operations is used to represent the operation set of block devices. It is defined in include/linux/blkdev.h. You don’t need to know too much about it. Just remember open() and release().

Regarding the block device I/O request process, the first thing I want to introduce is the concept of request queue request_queue. It is a queue that stores different I/O requests. We all know that I/O operations are much slower than CPU operations. In order to improve the system utilization and throughput, we generally wait for the I/O operation, and then write it to the disk all at once when it is established. This can reduce the disk seek time, queue and request, as well as all the The relationship between the magnetic block bio that needs to be operated is as shown below,

Each gendisk should have a queue of requests that can be passed through,

struct request_queue* blk_alloc_queue(gfp_t gfp_mask)

To apply, gfp_mask is generally GFP_KERNEL. and then pass,

void blk_queue_make_request(struct request_queue* que, make_request_fn* fn)

To bind the request function to the queue, it means that as long as it is a request in this queue, it will be processed according to the business logic of fn. Of course, you also need to implement specific fn,

void (make_request_fn) (struct request_queue* que, struct bio* bio)

Finally, I would like to explain the bio structure, which holds information such as the final data address to be read and written, and is defined in include/linux/blk_types.h

S2 – Define block device

Define some of your own block devices and corresponding operations. The struct gendisk included in struct myblkdev is very important. It can reflect that what we define is a block device. buf points to the simulated disk space. The macro DISKSIZE is the size of the disk, the default is 2 MB, the macro NDISKPART indicates that the disk has 3 sectors, and the size of each sector is macro SECTORSIZE 512,

#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/fs.h>
#include <linux/blkdev.h>
#include <linux/genhd.h>
#include <linux/slab.h>
#include <linux/string.h>

#define DISKSIZE (2*1024*1024) /* The size of the simulated disk is 2 MB */
#define DEVNAME "ramdisk"
#define NDISKPART 3 /* The simulated disk has 3 partitions */
#define SECTORSIZE 512 /* sector size */

/* Customized block device structure */
struct myblkdev {<!-- -->
int major;
unsigned char* buf; /* points to the space of the simulated disk */
struct gendisk* gdisk;
struct request_queue* que;
};

struct myblkdev mydev;

int
my_blkdev_open(struct block_device* dev, fmode_t mode)
{<!-- -->
printk(KERN_INFO "my_blkdev_open\
");
return 0;
}

void
my_blkdev_release(struct gendisk* gdisk, fmode_t mode)
{<!-- -->
printk(KERN_INFO "my_blkdev_release\
");
}

struct block_device_operations blkops = {<!-- -->
.owner = THIS_MODULE,
.open = my_blkdev_open,
.release = my_blkdev_release,
};

Then, our attention can turn to the driver registration process. Compared with the character device of blkdev in the “Tech’s first encounter” Linux driver, the initialization of the block device is a little more complicated, but the essence is the same, that is It is application-initialization-joining the kernel model. It is divided into five steps in total.

/* Initialization function */
static int __init
my_blkdev_init(void)
{<!-- -->
int ok = 0;
printk(KERN_INFO "my_blkdev_init\
");
\t
/* Register block device */
mydev.major = register_blkdev(0, DEVNAME);
if(mydev.major < 0) {<!-- -->
ok = -EINVAL;
goto over;
}
\t
/* Apply for memory space of the simulated disk */
mydev.buf = kzalloc(DISKSIZE, GFP_KERNEL);
if(mydev.buf == NULL) {<!-- -->
ok = -EINVAL;
goto alloc_buf_fail;
}
\t
/* allocate gendisk */
mydev.gdisk = alloc_disk(NDISKPART);
if(mydev.gdisk == NULL) {<!-- -->
ok = -EINVAL;
goto alloc_disk_fail;
}
\t
/* Allocate request queue */
mydev.que = blk_alloc_queue(GFP_KERNEL);
if(mydev.que == NULL) {<!-- -->
ok = -EINVAL;
goto alloc_que_fail;
}
blk_queue_make_request(mydev.que, my_blkdev_make_req_fn);
\t
/* Register gendisk */
mydev.gdisk->major = mydev.major;
mydev.gdisk->first_minor = 0;
mydev.gdisk->fops = & amp;blkops;
mydev.gdisk->queue = mydev.que;
mydev.gdisk->private_data = & amp;mydev;
\t
strcpy(mydev.gdisk->disk_name, DEVNAME); /* Name the gdisk of kobject, the core component of myblkdev */
set_capacity(mydev.gdisk, DISKSIZE/SECTORSIZE); /* Device capacity is in sector */
add_disk(mydev.gdisk);
\t
goto over;
\t
alloc_que_fail:
put_disk(mydev.gdisk);
alloc_disk_fail:
kfree(mydev.buf);
alloc_buf_fail:
unregister_blkdev(mydev.major, DEVNAME);
over:
return ok;
}

(1) Register the block device, apply for the main device number for the device and inform the kernel: the name of the block device; (2) Apply for the memory space of the simulated disk, apply for space through kzalloc() and hand it over to the block device management; (3) Apply for gendisk; (4) Allocate a request queue to the block device to store I/O requests; (5) Initialize gendisk and register it in the kernel model

These are roughly the processes. It should be noted that here I am using a request queue that does not use I/O scheduling. I would like to briefly talk about the difference between request queues with I/O scheduling and without. For some old block devices, such as mechanical hard disks, request queues with scheduling can use the characteristics of requests to reorder them and reduce the number of robotic arms. The number of moves can improve the performance of the disk; however, for current NAND flash or solid-state drives, sorting requests may not bring substantial performance improvement, but will increase additional overhead. Therefore, the specific choice of which request queue needs to be judged based on the actual situation.

I will explain the detailed usage of the request queue in the next section. We will continue to follow the process of registering the module and complete the driver program to the end, and explain the relevant steps of module cancellation.

/* Uninstall function */
static void __exit
my_blkdev_exit(void)
{<!-- -->
printk(KERN_INFO "my_blkdev_exit\
");
\t
/* Delete gendisk */
del_gendisk(mydev.gdisk);
/* Reduce gendisk’s reference count */
put_disk(mydev.gdisk);
\t
/* Clear the request queue */
blk_cleanup_queue(mydev.que);
\t
/* Unregister block device */
unregister_blkdev(mydev.major, DEVNAME);
\t
/* Release space */
kfree(mydev.buf);
}

Just like the registration process, whatever you apply for will be returned when you log out, including gendisk, request queue, etc.

S3 – Request queue without I/O scheduling

The request queue without I/O scheduling is bound to a request processing function, which I named my_blkdev_make_req_fn, which accepts the req_que request queue and bio I/O operation (page, length and offset) as parameters,

/* Manufacturing request function */
void
my_blkdev_make_req_fn(struct request_queue* req_que, struct bio* bio)
{<!-- -->
int offset;
struct bio_vec bvec;
struct bvec_iter iter;
unsigned long len = 0;
\t
/* Get the starting address of the disk to be operated on (in bytes) */
offset = (bio->bi_iter.bi_sector) << 9;
\t
bio_for_each_segment(bvec, bio, iter) {<!-- --> /* Process each segment in bio */
char* ptr = page_address(bvec.bv_page) + bvec.bv_offset;
len = bvec.bv_len;
\t\t
/* It is a read operation OR a write operation */
if(bio_data_dir(bio) == READ)
memcpy(ptr, mydev.buf + offset, len);
if(bio_data_dir(bio) == WRITE)
memcpy(mydev.buf + offset, ptr, len);
\t\t
offset + = len;
}
\t
bio_endio(bio);
}

The macro bio_for_each_segment traverses the unfinished data segments starting from the current offset in bio, while the macro bio_for_each_segment_all traverses all data segments in bio, regardless of whether they have been Finish. Perform corresponding read/write operations for each data segment within the scope of the macro. Finally, after processing all the data segments of bio, tell request_que that the current I/O task has been completed through bio_endio()

3. Result

In the /home/lighthouse/test-linuxdriver/blkdev directory, type the make command to compile the program.

lighthouse@VM-0-9-ubuntu:~/test-linuxdriver/blkdev$ make
make -C /lib/modules/5.4.0-126-generic/build M=/home/lighthouse/test-linuxdriver/blkdev modules
make[1]: Entering directory '/usr/src/linux-headers-5.4.0-126-generic'
  CC [M] /home/lighthouse/test-linuxdriver/blkdev/blkdev.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC [M] /home/lighthouse/test-linuxdriver/blkdev/blkdev.mod.o
  LD [M] /home/lighthouse/test-linuxdriver/blkdev/blkdev.ko
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-126-generic'

Type,

$ sudo insmod blkdev.ko

After mounting the driver, you can type in another terminal,

cat /proc/devices

Check the device number that the driver applied to the kernel. Because it is dynamically allocated, the device number may be different each time.

Block devices:
  2fd
  7 loop
  ...
  251 ramdisk
  ...

You can see that the driver’s major device number is 251, which means that the block device with a major device number of 251 will be managed by this driver in the future.

At this time, if you create a device node with a major device number of 251 through mknod, the driver can handle some regular operations, such as cat and echo, that is, reading OR writing to the device file.

$ sudo mknod /dev/myblkdev b 251 0

can pass through,

lighthouse@VM-0-9-ubuntu:~/test-linuxdriver/blkdev$ ls /dev/myblkdev -l
brwxrwxrwx 1 root root 251, 0 Sep 2 15:37 /dev/myblkdev

First look at the created block device node. We will see that the permissions of this block device are not very high. You can increase the permissions through chmod.

$ sudo chmod 777 /dev/myblkdev

Then you can do whatever you want, you can try to write a string of characters to /dev/myblkdev,

$ echo "hello myblkdev" > /dev/myblkdev