Benchmarking Adobe HDS HTTP Video Streaming

  • Optimizing web servers for the machine
  • Different server machines
  • More batches of # concurrent requests, would give an interesting way to see (and graph), for each method how the 2 measured metrics change as a function of # concurrent requests — though the coarse approach here is still quite informative
  • Faster machine from which to test; my workstation couldn’t push nginx fast enough to break it
  1. I think the poor performance across the board re: txn/sec was due to the large number of concurrent requests (100, 500 and 1000 are quite high) and the size of each response (3.2mb).
  2. On the data used — for the without-HDS scenarios i.e. serving the fragment straight — I extracted that fragment via the f4f module first, and saved to the server’s html dir.
    For through-HDS scenarios, I requested e.g. Seg1-Frag1, which was the same fragment, but extracted on the fly by HDS. Granted the f4f segment was 15M and contained 5 fragments — something I should change to have a totally fair comparison.
    The 7000 copies are just duplicates of these files, in separate subdirectories; this is simply to simulate 7000 potential unique streams/content.
  3. 1. Tuning apache — yes this is needed for accurate Apache vs. Nginx comparisons, but mainly I’m after overhead of HDS module.
    Still, I did some adjustment of MaxClients, but found 256 to be around optimal.
  4. 2. I should have described — content was a 3.2mb fragment.
  5. 3. Initial runs were made from my local box over local gigabit connection to box physically connected to the same switch as me.
    But again, network issues shouldn’t matter as much when I’m really after relative comparison of HDS module vs without.
    Later tests (see below) have done locally though with similar trends.
  6. a. Makes sense, so you’d have more origins to spread the load
    b. By prepackaging the content, you mean pre-extracting fragments? I don’t think priming the edges is feasible for a large library (what I was aiming to simulate with 7000 copies)
    c. Yes, CDN architecture is another topic though — this is just about performance of an origin running the HDS module
  7. So, adjusting:
    - MaxClients to 256 (I tried this, 512, 1024 and 256 was the best for this machine)
    - reduced the number of concurrent requests to just 20
    - ran the test 10 times per batch (instead of once)
    - ran 5 batches, for each setup, averaging results
  8. Summary of results were:
    - In all cases 100% availability (as you would hope)
    - txn/sec for HDS Module, Straight Apache, and Straight Nginx were, in order: 14.6, 15.5, 18.7
  9. My conclusion is that the module still induces overhead (as would be expected) — though not huge (in this lightweight scenario ca. 5.8%, but at concurrency 100 ca. 17%), still considerable, something I would think an architect would count as a risk at scale.
    Losing the module and running straight Nginx (e.g. something you could do with your suggestion of pre-extraction, e.g. John Crosby from RealEyes’ extractor referred to — but not availabe — on http://www.thekuroko.com/) would seem to give improvement in vicinity of 28% (at low concurrency 20) to 41% (at concurrency 100).
  10. Again — and I guess the main outcome confirming my suspicions — dumping the HDS module for straight Nginx — to be conservative — would give expected speedup somewhere between 28% — 41%

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Nick Doyle

Nick Doyle

Computer Scientist. Agile Enthusiast. Past lives include Perl Hacker, Web Developer, DBA, Tech Lead, Motorcycle Instructor, Forensic Data Analyst, & Cloud Guy