Introduction to Linux Interrupts and CPU SMP Affinity

by Sandeep Krishnan on January 27, 2014

Interrupts are signal that are sent across IRQ (Interrupt Request Line) by a hardware or software.

Interrupts allow devices like keyboard, serial cards and parallel ports to indicate that it needs CPU attention.

Once the CPU receives the Interrupt Request, CPU will temporarily stop execution of running program and invoke a special program called Interrupt Handler or ISR (Interrupt Service Routine).

The Interrupt Service or Interrupt Handler Routine can be found in Interrupt Vector table that is located at fixed address in the memory. After the interrupt is handled CPU resumes the interrupted program.

At boot time, system identifies all devices, and appropriate interrupt handlers are loaded into the interrupt table.

The following are two ways of requesting CPU attention:

  1. Interrupt based
  2. Polling based

All Linux based OS are interrupt driven.

When we press a key on keyboard, keyboards says to CPU that a key has been pressed. But CPU can be busy processing some stuff from RAM, System Clock, NIC card, may be video or PCI bus. In that case Keyboard places a voltage on IRQ line assigned to that hardware, here in this case [Keyboard]. This change in voltage serves as request from device saying that device has a request that needs processing.

/proc/interrupts File

On a Linux machine, the file /proc/interrupts contains information about the interrupts in use and how many times processor has been interrupted

# cat /proc/interrupts
           CPU0   CPU1  CPU2  CPU3
  0: 3710374484      0     0     0  IO-APIC-edge  timer
  1:         20      0     0     0  IO-APIC-edge  i8042
  6:          5      0     0     0  IO-APIC-edge  floppy
  7:          0      0     0     0  IO-APIC-edge  parport0
  8:          0      0     0     0  IO-APIC-edge  rtc
  9:          0      0     0     0  IO-APIC-level  acpi
 12:        240      0     0     0  IO-APIC-edge  i8042
 14:   11200026      0     0     0  IO-APIC-edge  ide0
 51:   61281329      0     0     0  IO-APIC-level  ioc0
 59:          1      0     0     0  IO-APIC-level  vmci
 67:   19386473      0     0     0  IO-APIC-level  eth0
 75:   94595340      0     0     0  IO-APIC-level  eth1
NMI:          0      0     0     0
LOC: 3737150067 3737142382 3737145101 3737144204
ERR:          0
MIS:          0

In the above file:

  • The first Column is the IRQ number.
  • The Second column says how many times the CPU core has been interrupted. In the above example timer is interrupt name [System clock] and 3710374484 is the number of times CPU0 has been interrupted. I8042 is Keyboard controller that controls PS/2 keyboards and mouse in Pc’s.
  • For interrupt like rtc [Real time clock] CPU has not being interrupted. RTC are present in electronic devices to keep track of time.
  • NMI and LOC are drivers used on system that are not accessible/configured by user.

IRQ number determines the priority of the interrupt that needs to be handled by the CPU.

A small IRQ number value means higher priority.

For example if CPU receives interrupt from Keyboard and system clock simultaneously. CPU will serve System Clock first since it has IRQ number 0.

  • IRQ 0 — system timer (cannot be changed);
  • IRQ 1 — keyboard controller (cannot be changed)
  • IRQ 3 — serial port controller for serial port 2 (shared with serial port 4, if present);
  • IRQ 4 — serial port controller for serial port 1 (shared with serial port 3, if present);
  • IRQ 5 — parallel port 2 and 3 or sound card;
  • IRQ 6 — floppy disk controller;
  • IRQ 7 — parallel port 1. It is used for printers or for any parallel port if a printer is not present.

For devices like joystick CPU doesn’t wait for the device to send interrupt. Since Joystick used for gaming and the movement of joystick will be fast it will be ideal to use polling and check whether device needs attention. The disadvantage behind this method is CPU can get into busy wait, checking the device many times.

On a related note, it is also essential to handle the signals properly in Linux.

Hardware Interrupts

All of the above discussed scenarios are example of Hardware interrupts.

Hardware interrupts are further classified into two major categories:

  1. Non-maskable interrupts [NMI]: As the name suggests these types of interrupts cannot be ignored or suppressed by the CPU. MNI’s are send over separate interrupt line and it’s generally used for critical hardware errors like memory error, Hardware traps indicating Fan failure, Temperature Sensor failure etc.
  2. Maskable interrupts: These interrupts can be ignored or delayed by CPU. The Interrupt Mask Register masks the interrupts being triggered on external pins of cache controller. Setting a bit by writing a 0, disables the interrupt triggering on the pin

Software Interrupts

These interrupts are generated when the CPU executes an instruction which can cause an exception condition in the CPU [ALU unit] itself.

For example, divide a number by zero which is not possible, it will lead to divide-by-zero exception, causing the computer to abandon the calculation or display an error message.

The file /proc/stat is also a file part of the /proc filesystem, which has information about system kernel statistics, also holds some interrupt information.

# cat /proc/stat
cpu  17028082 5536753 5081493 1735530500 42592308 90006 479750 0
cpu0 5769176 1170683 1495750 403368354 39406374 90006 284864 0
cpu1 3714389 1451937 1186134 444082258 1084780 0 64876 0
cpu2 3791544 1471013 1211868 443988514 1056981 0 64764 0
cpu3 3752971 1443119 1187740 444091373 1044172 0 65244 0
intr 417756956 --- Output Truncated

The line intr shows the count of the interrupt serviced since boot time. The first column is total of all interrupts serviced. Each subsequent column is the total for a particular interrupt.

SMP_AFFINITY

Symmetric multiprocessing is the processing of programs by multiple processors.

smp_affinity file holds interrupt affinity value for a IRQ number. The smp_affinity file associated with each IRQ number is stored in /proc/irq/IRQ_NUMBER/smp_affinity file. The value in the file is stored in hexadecimal bit-mask representing all CPU cores in the system. smp_affinity works for device that has IO-APIC enabled device drivers.

For example, smp_affinity entry for Ethernet driver is shown below:

grep eth0 /proc/interrupts
67: 23834931 0 0 0 IO-APIC-level eth0

IRQ number for eth0 is 67 and corresponding smp_affinity file is located at:

cat /proc/irq/67/smp_affinity
00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001

The decimal equivalent for value ‘000000001’ is ‘1’. ie All the interrupt related to Ethernet driver will be serviced by CPU0.

We can manually change the processor affinity by changing values in smp_affinity file for a particular controller or by using irqbalance.

IRQ Balance

Irqbalance is a Linux utility that distributes interrupts over the processor cores in your computer system which helps to improve performance.

Irqbalance goal is to find a balance between power saving and optimal performance.

If irqbalance is not installed on your system, install it using yum as shown below.

# rpm -qa | grep irqbalance
irqbalance-0.55-15.el5

# yum search irqbalance

# yum install irqbalance.x86_64

Start the irqbalance service:

service irqbalance start

The following is a sample output from a Linux machine where irqbalance is installed. We could see that interrupts are now being distributed between CPUs.

# cat /proc/interrupts
           CPU0     CPU1      CPU2       CPU3
  0:  950901695        0         0          0  IO-APIC-edge  timer
  1:         13        0         0          0  IO-APIC-edge  i8042
  6:         96    10989       470          0  IO-APIC-edge  floppy
  7:          0        0         0          0  IO-APIC-edge  parport0
  8:          1        0         0          0  IO-APIC-edge  rtc
  9:          0        0         0          0  IO-APIC-level  acpi
 12:        109     1787         0          0  IO-APIC-edge  i8042
 15:         99 84813914         0          0  IO-APIC-edge  ide1
 51:      17371        0  46689970          0  IO-APIC-level  ioc0
 67:       1741        0         0  225409160  PCI-MSI  eth0
 83:          0        0         0          0  PCI-MSI  vmci
NMI:          0        0         0          0
LOC:  950902917  950903742  950901202  950901400
ERR:          0
MIS:          0

Irqbalance is especially useful on systems with multi-core processors, as interrupts will typically only be serviced by the first core.


Linux Sysadmin Course Linux provides several powerful administrative tools and utilities which will help you to manage your systems effectively. If you don’t know what these tools are and how to use them, you could be spending lot of time trying to perform even the basic administrative tasks. The focus of this course is to help you understand system administration tools, which will help you to become an effective Linux system administrator.
Get the Linux Sysadmin Course Now!

If you enjoyed this article, you might also like..

  1. 50 Linux Sysadmin Tutorials
  2. 50 Most Frequently Used Linux Commands (With Examples)
  3. Top 25 Best Linux Performance Monitoring and Debugging Tools
  4. Mommy, I found it! – 15 Practical Linux Find Command Examples
  5. Linux 101 Hacks 2nd Edition eBook Linux 101 Hacks Book

Bash 101 Hacks Book Sed and Awk 101 Hacks Book Nagios Core 3 Book Vim 101 Hacks Book

{ 13 comments… read them below or add one }

1 sudi January 28, 2014 at 2:31 am

refreshed my knowledge of engineering..
another superb article.!!
please keep writing more on LINUX.
I m always happy to read articles of geekstuff.

2 prakash January 28, 2014 at 2:56 am

Great article…. It would be better if u provide code examples for H/W and S/W interrupts….Anyways great work….

3 Ezequiel January 28, 2014 at 7:17 am

Nice reading, very interesting. Keep it up!
Thanks.
Eze.

4 Sasikala January 28, 2014 at 8:05 am

Very nice article. Thanks for it.

5 Lalchand Mali January 28, 2014 at 9:03 am

great Article…….It provide good understanding on interrupts……
Thanks

6 Jalal Hajigholamali January 28, 2014 at 11:03 pm

Hi,

Nice and very useful article…

7 Hemant January 28, 2014 at 11:21 pm

Nice article.

8 sandeep krishnan January 29, 2014 at 12:45 am

Thanks guys for the appreciation

9 Bob January 29, 2014 at 12:57 pm

Great article.

10 Ratna January 30, 2014 at 12:09 am

gr8 article….helping to understand interrupts with linux case study. specially on multicore

11 Ratna January 30, 2014 at 12:13 am

as u explained binding affinity to interrupt request… can we bind affinity in a program( say c/c++) for executing function on multicore environment. plz explain

12 Rafa January 30, 2014 at 5:11 am

Great post.

I find useful “watch” command to read these files (interrupts or stat).

In example:

$ watch -cd cat /proc/interrupts

Regards

13 deva January 30, 2014 at 6:36 am

Awesome..good article

Leave a Comment

Previous post:

Next post: