From: SPDK [mailto:firstname.lastname@example.org] On Behalf Of Nikos Dragazis
Sent: Thursday, November 8, 2018 1:49 AM
Subject: [SPDK] Questions about vhost memory registration
I would like to raise a couple of questions about vhost target.
My first question is:
During vhost-user negotiation, the master sends its memory regions to
the slave. Slave maps each region in its own address space. The mmap
addresses are page aligned (that is 4KB aligned) but not necessarily 2MB
aligned. When vhost registers the memory regions in
spdk_vhost_dev_mem_register(), it aligns the mmap addresses to 2MB
Yes, page aligned, but not PAGE_SIZE (4k) aligned. SPDK vhost require that initiator
pass memory backed by huge pages >= 2MB in size. On x86 MMU this imply
that page alignment is the same as page size which is >= 2MB (99% sure -
can someone confirm this to get this +1% ;) ).
The aligned addresses may not have a valid page table entry. So, in case
of uio, it is possible that during vtophys translation, the aligned
addresses are touched here:
and this could lead to a segfault. Is this a possible scenario?
My second question is:
The commit message here:
“We've had cases (especially with vhost) in the past where we have
a valid vaddr but the backing page was not assigned yet.”.
This refers to the vhost target, where shared memory is allocated by the
QEMU process and the SPDK process maps this memory.
Let’s consider this case. After mapping vhost-user memory regions, they
are registered to the vtophys map. In case vfio is disabled,
vtophys_get_paddr_pagemap() finds the corresponding physical addresses.
These addresses must refer to pinned memory because vfio is not there to
do the pinning. Therefore, VM’s memory has to be backed by hugepages.
Hugepages are allocated by the QEMU process, way before vhost memory
registration. After their allocation, hugepages will always have a
backing page because they never get swapped out. So, I do not see any
such case where backing page is not assigned yet and thus I do not see
any need to touch the mapped page.
This is my current understanding in brief and I'd welcome any feedback
you may have:
1. address alignment in spdk_vhost_dev_mem_register() is buggy because
the aligned address may not have a valid page table entry thus
triggering a segfault when being touched in
vtophys_get_paddr_pagemap() -> rte_atomic64_read().
2. touching the page in vtophys_get_paddr_pagemap() is unnecessary
because VM’s memory has to be backed by hugepages and hugepages are
not handled by demand paging strategy and they are never swapped out.
I am looking forward to your feedback.
Current start/end calculation in spdk_vhost_dev_mem_register() might be a actually
NOP for memory backed by hugepages.
I think that we can try to validate alignmet of the memory in
and fail if it is not 2MB aligned.
Have you hit any segfault there?
SPDK mailing list