On Jul 20, 2018, at 5:17 PM, Walker, Benjamin <benjamin.walker@intel.com> wrote:

Hi Lance,

On Fri, 2018-07-20 at 16:34 -0500, Lance Hartmann ORACLE wrote:
Another especially notable take-away for me was the realization that the
environment library -- I'll refer to the default env for example here,
libspdk_env_dpdk.a -- consists only of the objects compiled from
SPDK_ROOT_DIR/lib/env_dpdk.   With the ability to specify an alternate
environment, semantically, I had mistakenly assumed that the env library would
contain not only the SPDK implementation of the environment API, but also
everything on which it depended; i.e. for the default environment, the DPDK
objects.   But, that doesn't appear to be so.   Instead, when performing final
linking of each SPDK executable, not only is the SPDK environment library
specified, but so are the necessary DPDK libs on which it depends.   Variables
pulled in from the SPDK's environment makefile, env.mk, are used to specify
those DPDK libs along with additional, special linking flags.

We need to update the doc's Porting Guide to reflect these details, and I'd be
happy to volunteer to work on that effort.

Please do!

However, before embarking on that task, I'd like to pose the question:   are
there reasons why we couldn't or shouldn't produce the environment library to
consist of both the SPDK env implementation layer and the objects on which it
depends?  It would seem that would make final linking of executables a little
easier and reduce the complexity/effort of someone wanting to develop and use
an alternate environment.

How do you envision this playing with environment libraries that need to link
against shared libraries, especially ones provided with the system itself? I'm
not an expert in all of the available linker options, so maybe there is some way
to embed enough information to properly link/load a shared library directly into
the static library.

This is an area in which I would greatly appreciate additional discussion, weighing the engineering cost/benefits of such flexibility. I, myself, could for example embrace the idea that for the sake of simplifying final linking that we propose the SPDK env implementation is built as a shared library which could be generated with a dependency on the DPDK (or its replaced equivalent) which also be shared lib(s).   Alternatively, I could also potentially be persuaded that we take an all static approach, though I hasten to add I can appreciate there may be stakeholders who strongly advocate/need shared libs.  I am aware of precedents where a package provided both static and shared versions of their libraries, and sometimes even in separate packages; i.e. a "static" package.   So, for greater flexibility, I could also envision a new SPDK configure option enabling the developer to specify static vs. dynamic, and perhaps with the options for the needed linking flags.  On the latter, I'm aware per configure that it appears the build of the SPDK can/will honor shell environment variables for CFLAGS, CXXFLAGS, LDFLAGS and DESTDIR, but I don't know how challenging the use of those could be to override (or concatenate) those already figured out in the *mk/Makefile files.

Out of curiosity, I retrieved the available DPDK packages from Ubuntu, and there are quite a few of them, including those with a static and dynamic dpdk library.   I haven't dug through all of the SPDK build options, but my initial pass would suggest it's currently built with only static env (and static DPDK) libs, at least for the default configuration.

Taking all of this a step further is the expectation that in the future we will provide two very different ways to build with the SPDK.   Today, we're checking out the SPDK repo and relying on the collection of makefiles therein to build the SPDK.   But, moving forward, we hope to produce an spdk-devel package(s) so that SPDK app developers could build against it/them.   Such a build environment there is outside our purview save for our need to provide the proper contents of those devel package(s).

In summary, I'm seeking, if possible, to simplify the build with respect to the env while keeping in mind the future with -devel package(s).   I'm very much hoping to get more input from others in the SPDK community on this topic, and am happy to volunteer on the implementation and documentation thereof once we can form a broader consensus.

thanks much,