[SOLVED] PXE using CIFS having mixed results

[e_esprit]

First results, for USB vs NFS (baseline == USB, set1 == NFS) :
http://eesprit.free.fr/openelec.tv/iozone/usb_vs_nfs/

As you can see, NFS performs better, or as well as my local USB drive (an old 250GB disk) most of the time, with the exception of the f* function.
You can see that both are limited by RAM size used to buffer/cache (my zbox has 2GB, 512MB are reserved for GPU, and XBMC + openelec stuff is using a little less than 512MB, which let about 1GB for cache/buffers).

I didn't run iozone with the -c option to force NFS function to take in count the close() function.
Will do that later, I don't really understand the point of this option for now.

My conclusion : NFS is almost as performant as an old local drive (in my setup), the differences are not huge, so in terms of user-experience, it should not have any real impact.

iSCSI tests are running at the moment.

[KillerKelvUK]

e_esprit, thats some really interesting work, thank you! Will your Iozone results page stay live? Was thinking this could get a section in the Wiki to support user selection of connection protocol for PXE and others.

Will be interesting to see the iSCSI performance comparison to NFS, expecting big things but I must say that for ease of implementation, administrative access to the block for me iSCSI seems a cumbersome solution. However I do recognise its benefits for speed...if it was a set up and forgot type solution then I guess there is no question of choice.

Would the results your compiling be represetative for initial connection/boot timings? My thinking was around any differences in establishing the initial links and starting the boot process and whether there is any economy of combining NFS and ISCSI for OE? When I was doing testing to solve my problem in this thread I was combining NFS and CIFS protocols between boot= and disk= and for ISCSI I use NFS to boot the OE kernel and then used the ISCSI functionality within OE to setup the connection for disk. Would you foresee any benefits in this type of scenario?

[e_esprit]

Yes, my results will stay on my webpage.

But I'm not really sure that it will help a lot in selecting the good network backend storage for system and storage. I mean, I don't have the iscsi stats yet, but I'm pretty sure there will not be a huge difference between NFS and iSCSI (perhaps I'm wrong), and the point is more to tell "Hey guys, all those backends performs pretty well (as well as a local hard drive, not as well as a SSD, and probably far better than most USB pen drive), so just choose the one that you are the most at ease (not sure it sounds well in english new old :laugh: ) with.".
I mean, it's just for system and xbmc related data, there is not a lot of I/O involved here except on the boot/initialisation process. And for media sharing, neither iSCSI nor NBD should be used, NFS and Samba are far better options in my opinion.

In my case, I chose iSCSI because :
- everything (KERNEL + SYSTEM + storage) is using the same protocol (and no need for a TFTP server)
- I used netowrk block devices all day long in my job (mostly FC, but though, it the same logic behind) so I am really comfortable with that.

But for non-sysadmin people, it's probably easier to setup a NFS share... but still the problem of having the KERNEL + SYSTEM files on TFTP or iSCSI or HTTP, as I didn't find a boot ROM that allow to boot from an NFS share (at least iPXE does not).


The initial connection/boot timings are in my opinion more important in terms of user-experience, it will probably talk more to end-users that all the chart stuff I'm compiling now.

That will be another step, I'll bench boot time for the different protocols (except NBD, cause my NAS doesn't do NBD, and womparing with another hardware device would have no meaning).
I am also wondering how well booting the kernel on HTTP could be ?

It would be interesting to have your own stats to compare.
Something like, complete boot time (including DHCP request ?) until XBMC shows up, for :
- KERNEL over TFTP + System on NFS + Storage on NFS
- KERNEL over HTTP + System on NFS + Storage on NFS
- KERNEL over iSCSI + System on iSCSI + Storage on NFS
- KERNEL over TFTP + System on CIFS + Storage on CIFS
- KERNEL over HTTP + System on CIFS + Storage on CIFS
- KERNEL over iSCSI + System on iSCSI + Storage on CIFS
- KERNEL over iSCSI + System on iSCSI + Storage on iSCSI

[e_esprit]

Local USB drive vs iSCSI (baseline == usb, set1 == iSCSI) :
http://eesprit.free.fr/openelec.tv/iozone/usb_vs_iscsi/

And the awaited NFS vs iSCSI (baseline == NFS, set1 == iSCSI) :
http://eesprit.free.fr/openelec.tv/iozone/nfs_vs_iscsi/

As you can see on the last one, NFS and iSCSI are very close.
(If I had used the iozone -c option to take the close operation into account, iSCSI would outperformed NFS on write operations by a factor of ten, as I wrote yesterday. But that's not an important thing, because as said before, we are talking here about the system/storage partitions/shares, which are almost "reading" I/O, and when it comes to "writing" I/O, it only concerns (relatively) small sized files (my biggest sqlite XBMC database is the Music one with... 5.4M !), so it's not a big deal.

Would still be interesting to have this information, even if of no use for Openelec/XBMC, but I will let my NAS cool down a little for now :whistle:

EDIT: I'll also do benchmark with an USB pen drive, cause I think a lot of people use that for openelec. But actually I only have a 1GB one (so it would be hard to test 1GB files on it), I'll take one 4GB from work in order to test.

[KillerKelvUK]

All looking gr8 :laugh:

I'd love to help out/contribute with some testing and documenting although I may just need a little help/guidance. I'm out this evening for a friends wedding so will catch up back over the weekend and discuss next steps.

K

[ack006]

Thanks very much for the information, much appreciated :-)

[KillerKelvUK]

@e_esprit

So I had a play around using HTTP for booting the KERNEL and the speed seems to be pretty damn quick B). I've not done any proper timings yet as I wanted to check in before I start so here goes...

I have a QNap TS-259 Pro+ which is running DHCP, TFTP, Samba, NFS, ISCSI, Web Server and a few other services/apps e.g. Sabnzbd, Sickbeard etc.

I have a wired gigabit network and was thinking of keeping the client tests to using my Acer Aspire One 722 netbook. This has HDMI out capability so is reasonable for a HTPC test, only has a 100mb capable NIC though but not seeing that as much of an issue? I do also have a Shuttle XS35GT-V2 which is my main HTPC in the living room however the PXE boot on that takes an age due to the chipset, plus its not as convenient for me to test with.

I'll use the Fusion x86_64 Beta 2 v1.95.2 build for the testing.

I can test NFS, CIFS and ISCSI as well as using HTTP for the KERNEL...will consistently use iPXE throughout. So with these combination I get a total of 27 test scenarios...Boot-Timing-Tests.csv

What I wanted to mainly check are the actual test scenarios...would you rule out some as not being a candidate for any reason...For example, using ISCSI for KERNEL is it dumb to test the other combinations for the SYSTEM as if the ISCSI connection and mount are preset for the KERNEL is it reasonable to assume ISCSI is the quickest vs. using CIFS or NFS for the SYSTEM?

Hopefully my query above makes sense...let me know and in the mean time I'll set up my ISCSI LUN.

[e_esprit]

I think the seven cases that I listed should be sufficient.

Having KERNEL over HTTP or TFTP and using iSCSI as System doesn't make much sense.
And mixing NFS and CIFS for System and Storage neither.
Same thing for using iSCSI as Storage and something else for System, sounds a little illogical.
Whereas using iSCSI for KERNEL + System and something else (NFS/CIFS) for Storage does make sense.

[KillerKelvUK]

Depends on what the focus is with this...if its about how long it takes to get into xbmc then I suggest you'd get one combination of boot protocols that would be different to if the question was about the speed of use of xbmc with a large media database.

Having KERNEL over HTTP or TFTP and using iSCSI as System doesn't make much sense.

So we're using iPXE as the client bootloader, for those who don't have iPXE or who can't deploy it easily then delivering the KERNEL via a more avilable method that then enables ISCSI once loaded may be preferable? Plus I'd suggest KERNEL via HTTP or TFTP would be quicker than connecting to an ISCSI target and then reading it...back to the original question above.

And mixing NFS and CIFS for System and Storage neither.

My thinking here was that System just needs read access right? If so then choosing a protocol that delivers best read performance was advantagous vs. Storage needing read/write (more write) which may be better supported with something else.

Same thing for using iSCSI as Storage and something else for System, sounds a little illogical.

Agreed, I'll remove that test for sure.

What do you think, I'm open that I may be overthinking this?! :unsure: