Bus-Independent Device Accesses¶
Author: | Matthew Wilcox |
---|---|
Author: | Alan Cox |
Introduction¶
Linux provides an API which abstracts performing IO across all busses and devices, allowing device drivers to be written independently of bus type.
Memory Mapped IO¶
Getting Access to the Device¶
The most widely supported form of IO is memory mapped IO. That is, a part of the CPU’s address space is interpreted not as accesses to memory, but as accesses to a device. Some architectures define devices to be at a fixed address, but most have some method of discovering devices. The PCI bus walk is a good example of such a scheme. This document does not cover how to receive such an address, but assumes you are starting with one. Physical addresses are of type unsigned long.
This address should not be used directly. Instead, to get an address
suitable for passing to the accessor functions described below, you
should call ioremap()
. An address suitable for accessing
the device will be returned to you.
After you’ve finished using the device (say, in your module’s exit
routine), call iounmap()
in order to return the address
space to the kernel. Most architectures allocate new address space each
time you call ioremap()
, and they can run out unless you
call iounmap()
.
Accessing the device¶
The part of the interface most used by drivers is reading and writing memory-mapped registers on the device. Linux provides interfaces to read and write 8-bit, 16-bit, 32-bit and 64-bit quantities. Due to a historical accident, these are named byte, word, long and quad accesses. Both read and write accesses are supported; there is no prefetch support at this time.
The functions are named readb(), readw(), readl(), readq(), readb_relaxed(), readw_relaxed(), readl_relaxed(), readq_relaxed(), writeb(), writew(), writel() and writeq().
Some devices (such as framebuffers) would like to use larger transfers than
8 bytes at a time. For these devices, the memcpy_toio()
,
memcpy_fromio()
and memset_io()
functions are
provided. Do not use memset or memcpy on IO addresses; they are not
guaranteed to copy data in order.
The read and write functions are defined to be ordered. That is the compiler is not permitted to reorder the I/O sequence. When the ordering can be compiler optimised, you can use __readb() and friends to indicate the relaxed ordering. Use this with care.
While the basic functions are defined to be synchronous with respect to each other and ordered with respect to each other the busses the devices sit on may themselves have asynchronicity. In particular many authors are burned by the fact that PCI bus writes are posted asynchronously. A driver author must issue a read from the same device to ensure that writes have occurred in the specific cases the author cares. This kind of property cannot be hidden from driver writers in the API. In some cases, the read used to flush the device may be expected to fail (if the card is resetting, for example). In that case, the read should be done from config space, which is guaranteed to soft-fail if the card doesn’t respond.
The following is an example of flushing a write to a device when the driver would like to ensure the write’s effects are visible prior to continuing execution:
static inline void
qla1280_disable_intrs(struct scsi_qla_host *ha)
{
struct device_reg *reg;
reg = ha->iobase;
/* disable risc and host interrupts */
WRT_REG_WORD(®->ictrl, 0);
/*
* The following read will ensure that the above write
* has been received by the device before we return from this
* function.
*/
RD_REG_WORD(®->ictrl);
ha->flags.ints_enabled = 0;
}
PCI ordering rules also guarantee that PIO read responses arrive after any outstanding DMA writes from that bus, since for some devices the result of a readb() call may signal to the driver that a DMA transaction is complete. In many cases, however, the driver may want to indicate that the next readb() call has no relation to any previous DMA writes performed by the device. The driver can use readb_relaxed() for these cases, although only some platforms will honor the relaxed semantics. Using the relaxed read functions will provide significant performance benefits on platforms that support it. The qla2xxx driver provides examples of how to use readX_relaxed(). In many cases, a majority of the driver’s readX() calls can safely be converted to readX_relaxed() calls, since only a few will indicate or depend on DMA completion.
Port Space Accesses¶
Port Space Explained¶
Another form of IO commonly supported is Port Space. This is a range of addresses separate to the normal memory address space. Access to these addresses is generally not as fast as accesses to the memory mapped addresses, and it also has a potentially smaller address space.
Unlike memory mapped IO, no preparation is required to access port space.
Accessing Port Space¶
Accesses to this space are provided through a set of functions which
allow 8-bit, 16-bit and 32-bit accesses; also known as byte, word and
long. These functions are inb()
, inw()
,
inl()
, outb()
, outw()
and
outl()
.
Some variants are provided for these functions. Some devices require
that accesses to their ports are slowed down. This functionality is
provided by appending a _p
to the end of the function.
There are also equivalents to memcpy. The ins()
and
outs()
functions copy bytes, words or longs to the given
port.
Generalizing Access to System and I/O Memory¶
When accessing a memory region, depending on its location, users may have to
access it with I/O operations or memory load/store operations. For example,
copying to system memory could be done with memcpy()
, copying to I/O memory
would be done with memcpy_toio()
.
void *vaddr = ...; // pointer to system memory
memcpy(vaddr, src, len);
void *vaddr_iomem = ...; // pointer to I/O memory
memcpy_toio(vaddr, _iomem, src, len);
The user of such pointer may not have information about the mapping of that
region or may want to have a single code path to handle operations on that
buffer, regardless if it’s located in system or IO memory. The type
struct iosys_map
and its helpers abstract that so the
buffer can be passed around to other drivers or have separate duties inside
the same driver for allocation, read and write operations.
Open-coding access to struct iosys_map
is considered
bad style. Rather then accessing its fields directly, use one of the provided
helper functions, or implement your own. For example, instances of
struct iosys_map
can be initialized statically with
IOSYS_MAP_INIT_VADDR()
, or at runtime with iosys_map_set_vaddr()
. These
helpers will set an address in system memory.
struct iosys_map map = IOSYS_MAP_INIT_VADDR(0xdeadbeaf);
iosys_map_set_vaddr(&map, 0xdeadbeaf);
To set an address in I/O memory, use iosys_map_set_vaddr_iomem()
.
iosys_map_set_vaddr_iomem(&map, 0xdeadbeaf);
Instances of struct iosys_map do not have to be cleaned up, but
can be cleared to NULL with iosys_map_clear()
. Cleared mappings
always refer to system memory.
iosys_map_clear(&map);
Test if a mapping is valid with either iosys_map_is_set()
or
iosys_map_is_null()
.
if (iosys_map_is_set(&map) != iosys_map_is_null(&map))
// always true
Instances of struct iosys_map
can be compared for
equality with iosys_map_is_equal()
. Mappings that point to different memory
spaces, system or I/O, are never equal. That’s even true if both spaces are
located in the same address space, both mappings contain the same address
value, or both mappings refer to NULL.
struct iosys_map sys_map; // refers to system memory
struct iosys_map io_map; // refers to I/O memory
if (iosys_map_is_equal(&sys_map, &io_map))
// always false
A set up instance of struct iosys_map can be used to access or manipulate the
buffer memory. Depending on the location of the memory, the provided helpers
will pick the correct operations. Data can be copied into the memory with
iosys_map_memcpy_to()
. The address can be manipulated with iosys_map_incr()
.
const void *src = ...; // source buffer
size_t len = ...; // length of src
iosys_map_memcpy_to(&map, src, len);
iosys_map_incr(&map, len); // go to first byte after the memcpy
-
struct
iosys_map
¶ Pointer to IO/system memory
Definition
struct iosys_map {
union {
void __iomem *vaddr_iomem;
void *vaddr;
};
bool is_iomem;
};
Members
{unnamed_union}
- anonymous
vaddr_iomem
- The buffer’s address if in I/O memory
vaddr
- The buffer’s address if in system memory
is_iomem
- True if the buffer is located in I/O memory, or false otherwise.
-
IOSYS_MAP_INIT_VADDR
(vaddr_)¶ Initializes struct iosys_map to an address in system memory
Parameters
vaddr_
- A system-memory address
-
IOSYS_MAP_INIT_OFFSET
(map_, offset_)¶ Initializes struct iosys_map from another iosys_map
Parameters
map_
- The dma-buf mapping structure to copy from
offset_
- Offset to add to the other mapping
Description
Initializes a new iosys_map struct based on another passed as argument. It does a shallow copy of the struct so it’s possible to update the back storage without changing where the original map points to. It is the equivalent of doing:
iosys_map map = other_map;
iosys_map_incr(&map, &offset);
Example usage:
void foo(struct device *dev, struct iosys_map *base_map)
{
...
struct iosys_map map = IOSYS_MAP_INIT_OFFSET(base_map, FIELD_OFFSET);
...
}
The advantage of using the initializer over just increasing the offset with
iosys_map_incr()
like above is that the new map will always point to the
right place of the buffer during its scope. It reduces the risk of updating
the wrong part of the buffer and having no compiler warning about that. If
the assignment to IOSYS_MAP_INIT_OFFSET()
is forgotten, the compiler can warn
about the use of uninitialized variable.
-
void
iosys_map_set_vaddr
(struct iosys_map * map, void * vaddr)¶ Sets a iosys mapping structure to an address in system memory
Parameters
struct iosys_map * map
- The iosys_map structure
void * vaddr
- A system-memory address
Description
Sets the address and clears the I/O-memory flag.
-
void
iosys_map_set_vaddr_iomem
(struct iosys_map * map, void __iomem * vaddr_iomem)¶ Sets a iosys mapping structure to an address in I/O memory
Parameters
struct iosys_map * map
- The iosys_map structure
void __iomem * vaddr_iomem
- An I/O-memory address
Description
Sets the address and the I/O-memory flag.
-
bool
iosys_map_is_equal
(const struct iosys_map * lhs, const struct iosys_map * rhs)¶ Compares two iosys mapping structures for equality
Parameters
const struct iosys_map * lhs
- The iosys_map structure
const struct iosys_map * rhs
- A iosys_map structure to compare with
Description
Two iosys mapping structures are equal if they both refer to the same type of memory and to the same address within that memory.
Return
True is both structures are equal, or false otherwise.
Parameters
const struct iosys_map * map
- The iosys_map structure
Description
Depending on the state of struct iosys_map.is_iomem, tests if the mapping is NULL.
Return
True if the mapping is NULL, or false otherwise.
Parameters
const struct iosys_map * map
- The iosys_map structure
Description
Depending on the state of struct iosys_map.is_iomem, tests if the mapping has been set.
Return
True if the mapping is been set, or false otherwise.
Parameters
struct iosys_map * map
- The iosys_map structure
Description
Clears all fields to zero, including struct iosys_map.is_iomem, so mapping structures that were set to point to I/O memory are reset for system memory. Pointers are cleared to NULL. This is the default.
-
void
iosys_map_memcpy_to
(struct iosys_map * dst, size_t dst_offset, const void * src, size_t len)¶ Memcpy into offset of iosys_map
Parameters
struct iosys_map * dst
- The iosys_map structure
size_t dst_offset
- The offset from which to copy
const void * src
- The source buffer
size_t len
- The number of byte in src
Description
Copies data into a iosys_map with an offset. The source buffer is in system memory. Depending on the buffer’s location, the helper picks the correct method of accessing the memory.
-
void
iosys_map_memcpy_from
(void * dst, const struct iosys_map * src, size_t src_offset, size_t len)¶ Memcpy from iosys_map into system memory
Parameters
void * dst
- Destination in system memory
const struct iosys_map * src
- The iosys_map structure
size_t src_offset
- The offset from which to copy
size_t len
- The number of byte in src
Description
Copies data from a iosys_map with an offset. The dest buffer is in system memory. Depending on the mapping location, the helper picks the correct method of accessing the memory.
-
void
iosys_map_incr
(struct iosys_map * map, size_t incr)¶ Increments the address stored in a iosys mapping
Parameters
struct iosys_map * map
- The iosys_map structure
size_t incr
- The number of bytes to increment
Description
Increments the address stored in a iosys mapping. Depending on the buffer’s location, the correct value will be updated.
-
void
iosys_map_memset
(struct iosys_map * dst, size_t offset, int value, size_t len)¶ Memset iosys_map
Parameters
struct iosys_map * dst
- The iosys_map structure
size_t offset
- Offset from dst where to start setting value
int value
- The value to set
size_t len
- The number of bytes to set in dst
Description
Set value in iosys_map. Depending on the buffer’s location, the helper picks the correct method of accessing the memory.
-
iosys_map_rd
(map__, offset__, type__)¶ Read a C-type value from the iosys_map
Parameters
map__
- The iosys_map structure
offset__
- The offset from which to read
type__
- Type of the value being read
Description
Read a C type value from iosys_map, handling possible un-aligned accesses to the mapping.
Return
The value read from the mapping.
-
iosys_map_wr
(map__, offset__, type__, val__)¶ Write a C-type value to the iosys_map
Parameters
map__
- The iosys_map structure
offset__
- The offset from the mapping to write to
type__
- Type of the value being written
val__
- Value to write
Description
Write a C-type value to the iosys_map, handling possible un-aligned accesses to the mapping.
-
iosys_map_rd_field
(map__, struct_offset__, struct_type__, field__)¶ Read a member from a struct in the iosys_map
Parameters
map__
- The iosys_map structure
struct_offset__
- Offset from the beggining of the map, where the struct is located
struct_type__
- The struct describing the layout of the mapping
field__
- Member of the struct to read
Description
Read a value from iosys_map considering its layout is described by a C struct
starting at struct_offset__. The field offset and size is calculated and its
value read handling possible un-aligned memory accesses. For example: suppose
there is a struct foo defined as below and the value foo.field2.inner2
needs to be read from the iosys_map:
struct foo {
int field1;
struct {
int inner1;
int inner2;
} field2;
int field3;
} __packed;
This is the expected memory layout of a buffer using iosys_map_rd_field()
:
Address | Content |
---|---|
buffer + 0000 | start of mmapped buffer pointed by iosys_map |
… | … |
buffer + struct_offset__ |
start of struct foo |
… | … |
buffer + wwww | foo.field2.inner2 |
… | … |
buffer + yyyy | end of struct foo |
… | … |
buffer + zzzz | end of mmaped buffer |
Values automatically calculated by this macro or not needed are denoted by wwww, yyyy and zzzz. This is the code to read that value:
x = iosys_map_rd_field(&map, offset, struct foo, field2.inner2);
Return
The value read from the mapping.
-
iosys_map_wr_field
(map__, struct_offset__, struct_type__, field__, val__)¶ Write to a member of a struct in the iosys_map
Parameters
map__
- The iosys_map structure
struct_offset__
- Offset from the beggining of the map, where the struct is located
struct_type__
- The struct describing the layout of the mapping
field__
- Member of the struct to read
val__
- Value to write
Description
Write a value to the iosys_map considering its layout is described by a C struct
starting at struct_offset__. The field offset and size is calculated and the
val__ is written handling possible un-aligned memory accesses. Refer to
iosys_map_rd_field()
for expected usage and memory layout.
Public Functions Provided¶
-
phys_addr_t
virt_to_phys
(volatile void * address)¶ map virtual addresses to physical
Parameters
volatile void * address
- address to remap
Description
The returned physical address is the physical (CPU) mapping for the memory address given. It is only valid to use this function on addresses directly mapped or allocated via kmalloc.
This function does not give bus mappings for DMA transfers. In almost all conceivable cases a device driver should not be using this function
-
void *
phys_to_virt
(phys_addr_t address)¶ map physical address to virtual
Parameters
phys_addr_t address
- address to remap
Description
The returned virtual address is a current CPU mapping for the memory address given. It is only valid to use this function on addresses that have a kernel mapping
This function does not handle bus mappings for DMA transfers. In almost all conceivable cases a device driver should not be using this function
-
void __iomem *
ioremap
(resource_size_t offset, unsigned long size)¶ map bus memory into CPU space
Parameters
resource_size_t offset
- bus address of the memory
unsigned long size
- size of the resource to map
Description
ioremap performs a platform specific sequence of operations to make bus memory CPU accessible via the readb/readw/readl/writeb/ writew/writel functions and the other mmio helpers. The returned address is not guaranteed to be usable directly as a virtual address.
If the area you are trying to map is a PCI BAR you should have a
look at pci_iomap()
.
-
void
iosubmit_cmds512
(void __iomem * dst, const void * src, size_t count)¶ copy data to single MMIO location, in 512-bit units
Parameters
void __iomem * dst
- destination, in MMIO space (must be 512-bit aligned)
const void * src
- source
size_t count
- number of 512 bits quantities to submit
Description
Submit data from kernel space to MMIO space, in units of 512 bits at a time. Order of access is not guaranteed, nor is a memory barrier performed afterwards.
Warning: Do not use this helper unless your driver has checked that the CPU instruction is supported on the platform.
-
void __iomem *
pci_iomap_range
(struct pci_dev * dev, int bar, unsigned long offset, unsigned long maxlen)¶ create a virtual mapping cookie for a PCI BAR
Parameters
struct pci_dev * dev
- PCI device that owns the BAR
int bar
- BAR number
unsigned long offset
- map memory at the given offset in BAR
unsigned long maxlen
- max length of the memory to map
Description
Using this function you will get a __iomem address to your device BAR. You can access it using ioread*() and iowrite*(). These functions hide the details if this is a MMIO or PIO address space and will just do what you expect from them in the correct way.
maxlen specifies the maximum length to map. If you want to get access to
the complete BAR from offset to the end, pass 0
here.
-
void __iomem *
pci_iomap_wc_range
(struct pci_dev * dev, int bar, unsigned long offset, unsigned long maxlen)¶ create a virtual WC mapping cookie for a PCI BAR
Parameters
struct pci_dev * dev
- PCI device that owns the BAR
int bar
- BAR number
unsigned long offset
- map memory at the given offset in BAR
unsigned long maxlen
- max length of the memory to map
Description
Using this function you will get a __iomem address to your device BAR. You can access it using ioread*() and iowrite*(). These functions hide the details if this is a MMIO or PIO address space and will just do what you expect from them in the correct way. When possible write combining is used.
maxlen specifies the maximum length to map. If you want to get access to
the complete BAR from offset to the end, pass 0
here.
-
void __iomem *
pci_iomap
(struct pci_dev * dev, int bar, unsigned long maxlen)¶ create a virtual mapping cookie for a PCI BAR
Parameters
struct pci_dev * dev
- PCI device that owns the BAR
int bar
- BAR number
unsigned long maxlen
- length of the memory to map
Description
Using this function you will get a __iomem address to your device BAR. You can access it using ioread*() and iowrite*(). These functions hide the details if this is a MMIO or PIO address space and will just do what you expect from them in the correct way.
maxlen specifies the maximum length to map. If you want to get access to
the complete BAR without checking for its length first, pass 0
here.
-
void __iomem *
pci_iomap_wc
(struct pci_dev * dev, int bar, unsigned long maxlen)¶ create a virtual WC mapping cookie for a PCI BAR
Parameters
struct pci_dev * dev
- PCI device that owns the BAR
int bar
- BAR number
unsigned long maxlen
- length of the memory to map
Description
Using this function you will get a __iomem address to your device BAR. You can access it using ioread*() and iowrite*(). These functions hide the details if this is a MMIO or PIO address space and will just do what you expect from them in the correct way. When possible write combining is used.
maxlen specifies the maximum length to map. If you want to get access to
the complete BAR without checking for its length first, pass 0
here.