Thanks Ben, Jim has already provided an explainer of the standard SPDK approach and some helpful pointers. Anyway, thanks for the extra info.


On Thu, Aug 2, 2018, 00:15 Walker, Benjamin <> wrote:
Hi Andrey,

Responding to your original note because I had some more fundamental questions about what you are
doing here.

On Tue, 2018-07-31 at 12:31 +0300, Andrey Kuzmin wrote:
> Hi guys,
> Working on a vbdev that maintains some global state and underlying data structures, I decided to
> explore an SPDKish alternative to the traditional synchronization, that of the single-threaded
> semantics where global data structures are being maintained by a dedicated thread, with requests
> being served by message passing to/from that thread.
> Coding such a thread against the spdk_allocate_thread() guidelines proved to be quite
> straightforward, and I've got it up and running. At the same time, I'm still unclear as to why I
> should be essentially duplicating the features of the pinned SPDK thread, with my gut feeling
> being that the very same single-threaded semantics could have been easily achieved by pinning my
> execution context to some randomly or round-robin chose pinned thread of the configured allowed
> cores mask.

I'm not sure I follow entirely, but it sounds like you spawned your own (POSIX) thread and
registered it using spdk_allocate_thread(). That's valid, but almost none of the SPDK code does
that. Instead, it typically assigns data to an existing thread (one of the ones pinned to a core)
and uses spdk_thread_send_msg() to pass messages to that thread to manipulate the data.

> If that gut feeling doesn't fool me, I will appreciate a pointer to any docs and/or examples
> showing how one can pin an arbitrary execution context inside a vbdev to an SPDK native thread
> (similar to app_start and friends, I believe), and do polling on that thread/ messaging to/from
> that thread similar to what a POSIX thread equipped with spdk thread envelope could do.

Elsewhere you responded to Jim that you are using an spdk_mempool to pool objects used for inter-
process communication and that sounded some alarm bells for me. I think if you assign your data to
one of the existing SPDK threads and use spdk_thread_send_msg(), you don't need to do any of that.
The message passing scheme is all already done for you. If you need to periodically execute code on
one of the existing threads, you can register a poller by calling spdk_poller_register(). You can
iterate over all of the threads using spdk_for_each_thread().

A strategy SPDK commonly employs is that the first thread to access some data "owns" it, and all
subsequent accesses are done via message passing. You may want to try that for your use case.

As far as documentation, there is always this:

There is active work going on in this area too. I put out this patch recently:

That's for discussion purposes only and it doesn't currently meet all the needs of everyone it's
intended for, but it should give some insight into the general direction the project is headed in.

SPDK mailing list