#16 32 bit vectored aio fails
Closed 2 years ago by bholman. Opened 2 years ago by bholman.

The problem: struct io_iocb_vector is not binary compatible with the linux kernel iocb members on 32b systems

For 64b x86 this isn't a problem, but on 32bit systems this causes vectored aio calls to fail, which leads to confusing failures modes.

It looks like this can be fixed with appropriate padding in struct io_iocb_vector.

Associated bug:
https://github.com/koverstreet/bcachefs-tools/issues/69#issuecomment-917821738


For additional context, here is an example failure on 32bit arm.

The following is the iocb data structure right before it gets passed into io_submit (which then fails with EINVAL). The values in this example were assigned to iocbp via the union member struct io_iocb_vector v.

(gdb) print *iocbp
$19 = {
  data = 0x2034c0, 
  __pad1 = 0, 
  key = 0, 
  aio_rw_flags = 0, 
  aio_lio_opcode = 7, 
  aio_reqprio = 0, 
  aio_fildes = 3, 
  u = {
    c = {
      buf = 0xbeffee00, 
      __pad1 = 1, 
      nbytes = 4096, 
      __pad2 = 0, 
      offset = 0, 
      __pad3 = 0, 
      flags = 0, 
      resfd = 0
    }, 
    v = {
      vec = 0xbeffee00, 
      nr = 1, 
      offset = 4096
    }, 
    poll = {
      events = -1090523648, 
      __pad1 = 1
    }, 
    saddr = {
      addr = 0xbeffee00, 
      len = 1
    }
  }
}

Note that in the union, the data in member struct io_iocb_vector v appears to have sensible values, however in the padded struct io_iocb_common c union member you can see that due to mis-alignment of the vector, the "number" actually lines up with __pad1, and the offset is assigned to the "nbytes" field. This behavior will not happen on 64 bit, because u.v.vec is 64b.

If the PR I included is applied, u.v.vec and u.v.nr are padded correctly, and this issue disappears for 32 bit (and since the PR uses the helper macros, it should not change the current padding for 64 bit).

Upon further investigation, it appears that the test harness doesn't actually exercise struct io_iocb_vector

In test 15.t (the only one I saw testing vectored aio), it uses the struct io_prep_p{read,write}v() helpers, which would succeed on 32bit since they use the correctly padded union member c, which has type: struct io_iocb_common.

When I get a chance, I'll see if I can add a test case to my PR that demonstrates this failure.

@jmoyer - Do you have anything to add regarding my analysis of this bug or regarding the associated PR?

Fixed in latest release

Metadata Update from @bholman:
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata