Mark your calendar to catch the livestream of Storage Tech Field Day<https://TargetMailer.intel.com/TMService/Redir.aspx?ID=3397896> on September 21, at 9:30 am Pacific, Andrey Kudryavstev from Intel will talk about Intel(r) Optane(tm) SSDs, followed by Anu Rao and Jim Harris from Intel, who will present "Storage Performance Development Kit-Accelerating Non-Volatile Media Performance" at 10:30 am.
As I talked shortly in the community before, I'm working on linear bdev (aggregating mutilple bdevs) as part of my works.
The following is my present basic design of the linear bdev.
Linear bdev doesn't belong to raid but I want to add linear bdev as another level of raid bdev.
Raid bdev and IO splitting in bdev will be able to become the great foundation.
- Utilize current IO splitting as much as possible.
- In linear bdev, optimal_io_boundary is not constant and linear bdev can't use current implementation based on optimal_io_boundary.
- Add an array made of (start, length) pair to struct spdk_bdev instead of optimal_io_boundary.
- Abstract the following APIs:
- bool _spdk_bdev_io_should_split(struct spdk_bdev_io *bdev_io)
- uint32_t _to_next_boundary(uint64_t offset, uint32_t boundary)
- The (start, length) array and optimal_io_boundary are mutually exclusive.
- Utilize current implementation as much as possible.
- Abstract the following:
- check consistency among base bdevs, calculate total block counts, add split info to spdk_bdev
- raid_bdev_start_rw_request() and following functions
- Use -1 as the level of the linear bdev.
- Add an new parameter linear (bool) to construct_raid_bdev.
- raid_level and linear are mutually exclusive.
Your any feedback is very welcome.
Is anyone using the firmware update capability through SPDK to update Intel NVMe SSDs? I've been using the API to perform upgrades on specific firmware version combinations but I'm not sure it can be used as a general approach.
For background we were hoping to be able to offer a management interface feature which enabled programmatic firmware upgrades (through the SPDK API) without relying on specialized commandline interface sequences for each upgrade, but from some of the documentation regarding firmware updates for example the DC P4800X it seems like there might not be a general process for upgrading firmware on Intel NVMe SSDs and that specialized procedures are required for different firmware revision upgrade combinations.
There are obviously specialized cli tools for doing vendor specific upgrades but I was hoping this might be performed reliably through SPDK.
Maybe it just has to be offered as a raw feature which may cause unintended consequences and the validity of the specific upgrade and any required pre/post actions must be investigated first. Not a particularly compelling caveat for a management application but may be necessary.
Any thoughts would be much appreciated.
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
Intel and Packet are working together to bring free infrastructure for testing and benchmarking with Intel® Optane™ SSDs to the developer community. This community lab is a perfect fit for the SPDK community that cares about getting the most from their storage applications. Interested in testing your SPDK application on the world’s most responsive data center SSD? Head over to Accelerate with Optane<https://www.acceleratewithoptane.com/> for more information, and, once you’re ready, request access<https://github.com/AccelerateWithOptane/lab/issues>. The lab will provide you with a highly capable bare metal server powered by Intel Xeon® Scalable processors, with three 750GB Intel Optane™ SSD DC P4800X and one 4TB Intel SSD DC P4500 for comparison purposes.
Due to high demand, priority will be made for teams most interested in sharing their results and learning with the SPDK and broader open source communities that care about application performance. Take advantage of this resource to prepare for the next SPDK Developer Meetup October 16-17 and/or share your results on your own channels.
Throughout the course of the day today, I will be replacing fedora 26 with fedora 28 on most of the machines in the build pool. I intend to do this with very little disruption to build pool operation, but you may see the pool paused for 1-2 minutes at a time between builds a few times throughout the day.
I will be leaving at least 1 machine running fedora 26 in the pool for legacy purposes.
I am Mingzhe. Thanks for the reply. Somehow I have not received mail from mailing list, so I have to send the mail directly to you.
Regarding the question, 'by "host virtio device" you mean rocksdb application run on host or guest'?
We're trying to run rocksdb on top of SPDK against a virtio-blk device in the VM.
Pasting original email just in case:
I have a virtio device on my host, and I want to run rocksdb over SPDK on it
> (with http://spdk.io/doc/virtio.html probably). However, I cannot find
> detailed instruction on how to do it. Can I get some hints?
In CH test pool, we have 3 systems setup to run crypto testing in CI based on the flag SPDK_TEST_CRYPTO and on those systems a few things are needed:
* Nasm of 2.12.2 or higher
* Ipsec module installed as root
* Some misc DPDK things enabled
The third is part of the patch so nothing needs to be done there however the first two need to be done via running pkgdep on the test machines. If the packaged nasm isn't of high enough version the build log will let you know. If ipsec isn't installed it will also let you know (see below from latest crypto patch w/Jenkins running BlobFS_autotest
To enable crypto you must first go to the intel-ipsec-mb directory and
run 'make' then 'sudo make install' then re-run this script.
15:09:05 $ trap - ERR
15:09:05 $ print_backtrace
15:09:05 $ [[ ehxBE =~ e ]]
15:09:05 $ local shell_options=ehxBE
15:09:05 $ set +x
========== Backtrace start: ==========
Here is a list of the failed tests and what needs to be done to get them working with the Jenkins test pool, we don't need crypto running everywhere so if you can enable it just on a few machines (block dev tests) that would be great:
BlobFS_autotest: missing install of ipsec, run pkgdep.sh
CentOS test: missing nasm or old, maybe just don't run on this OS
FreeBSD: same as above
NVME autotest: LLooks to be configured correctly but crypto patch failed for what looks like unrelated reasons: https://ci.spdk.io/spdk-jenkins/results/autotest-per-patch/builds/10910/a...
Ubuntu16: old nasm, maybe don't enable crypto testing on this one
Ubuntu17: needs ipsec insetall (pkgdep.sh) and this is probably a good candidate for crypto testing