Show Critical - desk issues on 7.9.6?

[drscoop]

Hi  forum folk,

I have been running the opening night of a show tonight and had several unexpected and significant issues with our FLX S24.  Last show on the desk was pretty solid.  Only changes here since last show were:

- Successful update to 7.9.6

- Modified Profile for Chavet FXPar 9 to split out inner LED, outer LEDs and strobe

- using 2 x USB desk lights on the desk USB ports.

Issues:

- seeing what appeared to be intermittent but fairly frequent rogue DMX on the lines, resulting in unexpected colour flashes and  intensity changes on our LED luminaires (Chauvet FXPar9s, 415s, 910s, ColorSource CYCs and Martin MH-5s) and the moving heads re-positioning, re-homing or otherwise moving at random

- odd desk activity with the playback LEDs occasionally randomly strobing from left to right on flash buttons 13-24, similar to the strobing, scrolling LEDs we'd expect to see on desk reboot

- Significant, intermittent lag on the master playback GO button, sometimes in the order of several seconds, or even not responding at all

- part-way through the show, a complete lock-up, all lights on zero intensity and no control or response to desk operation (faders, buttons, flash buttons or touch screen) whatsoever for 30-45 seconds.

After making do as best we could for a short while, we fortunately had a short section on just followspot (not run off the desk) where we power cycled the desk for a hard reboot and got the desk back and it ran fine after that, but the issues to that point did cause a significant distraction.  Possibly worth noting that we did have some minor quirks when programming, with some similar issues to a lesser extent.

I may just have done something stupid in the programming - the showfiles and the updated Chauvet profile are included here just in case.  We did remove the USB desk lights before the hard reset and didn't re-use them again, but could a simple USB accessory cause this much disruption on the DMX lines?

Wanted to raise this ASAP, both to see how we avert it for the show for the rest of the week, and also in case there's a 7.9.6 issue arising?  Can't see we have any crimped or compressed DMX cables,   and all lines are properly terminated...

I'm a bit perplexed.  All thoughts and suggestions welcome.

SHOWTIME_MASTER_FILE12.isf AJ_Chauvet FXpar 9 9Ch 3way.ift SHOWTIME_MASTER_FILE.isf

[drscoop]

Just a quick update - I quickly ran the same showfile on PhantOS last night, on 7.9.6 and linked to Capture to see what was going on and it seemed to behave itself, though I only ran through the cues quickly, rather than sitting there for hte full 2 hour show timings.  Could this be a dodgy firmware update or an emerging desk issue?  (Or dodgy USB peripherals - but would be surprised if they could cause significant interference?)

[Edward Z88]

Hello,

I would recommend disconnecting the LED desk lamps immediately.

It sounds like they are trying to draw too much current from the console, and is trying to pull the power rail voltage down. The USB ports on FLX S consoles are not designed for higher power peripherals. I’d suggest only a single low power desk lamp should be connected.

The full size FLX console has self healing USB ports, meaning if too much power is drawn, the USB ports are cut off until the console is power cycled, protecting the console. FLX S consoles do not contain this technology.

If you have any questions let us know.

Edward

[kgallen]

Maybe drscoop could tell us which desk lamps and how many.

Edward - what current output do the FLX S support? The minimum 100mA? which is still quite a lot and gives 500mW of output and I wouldn't expect an LED desk lamp to draw that much power.)

The Adam Hall SLED1 (quad COB) as an example claims to draw 50mA @ 5V, which should be supported easily by any USB output.

https://www.adamhall.com/shop/gb-en/stands-tripods/lamps/5491/sled-1-ultra-usb

[drscoop]

Hi all

These are the lamps in question:  https://www.amazon.co.uk/gp/product/B00BWTYMPM/ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1

 

I’ll see if there’s a current draw on the specs on the box later but will discontinue use.  I was not anticipating there would be a power draw issue but understand that the  FLX S has a few features removed to hit the price point and thus doesn’t have the self healing USB ports.  I will run the lamps off an external USB powerpack and see how we get on tonight.

 

thanks for the quick response on this.

[Edward Z88]

Hi both,

9 minutes ago, drscoop said:

These are the lamps in question:

Thanks for letting us know. I can't seem to find power information on these, but from the looks of them there are 8 LEDs on each lamp. One of the reviews says this gets quite hot, so sounds like they are being run pretty hard.

24 minutes ago, kgallen said:

Edward - what current output do the FLX S support? The minimum 100mA? which is still quite a lot and gives 500mW of output and I wouldn't expect an LED desk lamp to draw that much power.)

We do not have a quoted max USB current for FLX S consoles, however recommend using a powered USB hub for any peripherals other than usual keyboard, USB stick etc.

Edward

[drscoop]

Edward, is it worth a note to that effect in the manual?  I don’t think we’d been advised of that, but may have missed it?

[Edward Z88]

Hello,

Page 15 of the manual states:

"FLX S consoles do not include this feature (referring to self healing USB ports), and therefore it is not recommended to use the USB ports for devices such as lights or fans on FLX S consoles."

https://zero88.com/storage/downloads/7f906b85-80f1-4646-bd18-d2327d94fc60/FLX-Range-&-ZerOS-Accessories-User-Manual-v7.9.6.pdf

Page 82 of the manual states that USB ports are for keyboard & mouse, external touchscreen (S48), and external storage devices.

If you have any questions let me know. 

Edward

[Jon Hole]

56 minutes ago, drscoop said:

Edward, is it worth a note to that effect in the manual?  I don’t think we’d been advised of that, but may have missed it?

 

31 minutes ago, Edward- Z88 said:

Page 82 of the manual states that USB ports are for keyboard & mouse, external touchscreen (S48), and external storage devices.

 

On page 15 of the manual, it also states:

Quote

On FLX, the USB ports are “overload protected” in pairs. If a USB device tries drawing too much power, FLX will disable that pair or ports until the device is unplugged.

FLX S consoles do not include this feature, and therefore it is not recommended to use the USB ports for devices such as lights or fans on FLX S consoles.

As Edward says, we've updated the manual to put this in a nice big red box, on both Page 15 and Page 82. We haven't released this version of the manual yet, but it'll be included within the next public revision.

Before / after:

 

 

[drscoop]

Thanks Jon,  I think that covers it!!  

Fingers crossed for a smooth show tonight and I'll report back.

 

Whilst here, an unrelated question - the training sessions on our desk, both at TLG and at PLASA have been great and of huge value (it was a shame I had to duck out of the end of the last PLASA session and miss both the G&T and extra hints and tips!), but we are amassing lots of "it'd be great if we could just do..." or "Hmm....  Why did that happen like that...?" type questions.  I wondered if there are options for a more tailored and focussed intermediate-level desk training session and what might be available?

 

[Edward Z88]

15 minutes ago, drscoop said:

Whilst here, an unrelated question - the training sessions on our desk, both at TLG and at PLASA have been great and of huge value (it was a shame I had to duck out of the end of the last PLASA session and miss both the G&T and extra hints and tips!)

Glad you found them useful!

16 minutes ago, drscoop said:

we are amassing lots of "it'd be great if we could just do..." or "Hmm....  Why did that happen like that...?" type questions.  I wondered if there are options for a more tailored and focussed intermediate-level desk training session and what might be available?

I'd probably recommend booking onto a full day's FLX series training. You'll be able to ask any questions you might have, and the afternoon will cover the more complex parts of the console. We will soon be released our training dates and locations for 2020 on the website.

The alternative would be to go down the route of paid on-site training. If this is something that may be interest to you, please email training@zero88.com.

If you have any questions let us know.

Edward

[drscoop]

OK, so the saga continues....

No LED desk lamps plugged in tonight, but still some DMX issues on the lamps.  I updated a couple of cues as part of a slight tweak and as I was saving to USB I saw an odd message flash up.  Delving back into the system dialogue and I spotted the errors coming up on the screen, as shown in the files below...  These were streaming through in batches of a few at a time.  I have a video, but too big to upload... (I can drop you a Dropbox link if it helps, but doesn't really show any more than the image, just the frequency and timing of error messages...) This was after a clean reboot and just saving to USB stick.

 

[drscoop]

So, I removed the USB stick, and rebooted, and ran the show with no issue.  After the show, I checked back on the system and saw no Comms errors.  For completeness and curiosity, I tried adding back the USB desk lamps, one at a time, and both together, and again no issues (though I will heed advice and will not usually do this - I just wanted to see if I could replicate the issues...)   Plugged the USB stick into the top of the desk and immediately the desk locked up and was unresponsive for 5-10 seconds, then came back after a short flash of a few of the playback LEDs on Ch13-24.  A clean reboot of the desk and re-trying this exactly replicated the issue.

Strange thing was, if I clean reboot the desk and plug in the same USB stick into the rear USB port, saving and loading files to and from it, the desk runs perfectly stable and gives no comms issues or desk hesitancy.  It seems writing back to the USB stick on the front port only is instigating some form of comms issue that persists until the desk is power cycled and then is absent until the USB stick is used again.

Is there any hardware difference in the two ports, do we have a dodgy USB port, or is there some firmware glitch here, as this has only occurred since the flash to 7.9.6....?

Really interested in your thoughts on this.... 

I'll keep an eye on things over the final three show nights and continue to report anything unanticipated or unusual.

[Edward Z88]

Hello,

Sorry to see this.

We will take a look into this. Would I be able to give you a call to get more details on this?

If so please drop me an email with your number, and a convenient time to give you a call.

If you have any questions let me know.

Edward

[S24newbie]

Just to note that I experienced a similar issue when programming a show on a S24 running 7.9.6. (This is a hire desk so I have no previous experience with other zeros versions.) The whole desk became very slow and unresponsive on two occasions and I was only able to resolve the problem with a power cycle.

As it happens I also had a USB stick plugged into the front port at the time. I was also using an iPad as an external monitor via a WiFi router (which was also slow and unresponsive).

The issue hasn’t recurred again and I wasn’t able to reproduce it at the time but it did concern me that it could happen in a live show!

[Edward Z88]

Hello,

11 hours ago, S24newbie said:

Just to note that I experienced a similar issue when programming a show on a S24 running 7.9.6.

Thanks for the information. We are investigating what may have caused USB devices to slow the console down. 

If you have any questions let us know. 

Edward

[drscoop]

On 11/21/2019 at 6:36 PM, Edward- Z88 said:

 

We will take a look into this. Would I be able to give you a call to get more details on this?

If so please drop me an email with your number, and a convenient time to give you a call.

 

Will do - let me check what's going on at work next week as I know I'm out the office for a few days.