When does DEFINE_PROGRAM run

PhreaK

Tech note #993

Some interesting notes - most significantly DEFINE_PROGRAM will execute *every time* a global variable is written to.

vining

I like the reference to un- handled events. Like button events with out hold or release handlers. I believe this was brought up several years ago but any definitive answer. I guess I should go back through my old code and put in handlers even when I have no release or hold code to run.

the8thst

Interesting tech note. I will start reading them again if all of the tech notes are that well written and contain useful and lesser known information.

The only trigger that bugs me is the undefined event trigger. That means I will have to go back into my systems and add a lot of empty handlers (and a ton of channel events) to really take advantage of the info in the tech note.

Andrew G Welker

You shouldn't need to add the empty event handlers unless your system is having issues. It was designed to work this way, so it can handle it, unless you have so many unhandled events happening that the system is constantly running the DEFINE_PROGRAM section.

Auser

Tech Note 993 wrote:

That means I will have to go back into my systems and add a lot of empty handlers (and a ton of channel events) to really take advantage of the info in the tech note.

I doubt that this will gain you anything given that DEFINE_PROGRAM also runs when all event processing is completed.

vining

Auser wrote: »

I doubt that this will gain you anything given that DEFINE_PROGRAM also runs when all event processing is completed.

I agree it's not likely to make a noticeable difference in small systems but in a large system where events may get queued having to jump to and run DEF PROG becuase of unhandled events instead of going directly to the event queue and processing the next event pending doesn't seem like a good idea if it can be avoided. I'm not likely to go back to old code just to be anal but when I come across code with unhandled events or I'm writing new code I'll definitely consider this and write my code with this in mind, as long as I remember. It's so simple to avoid so why not.

jjames

Glad I started removing DEFINE_PROGRAM - don't have to think about what's being ran or when.

DHawthorne

jjames wrote: »

Glad I started removing DEFINE_PROGRAM - don't have to think about what's being ran or when.

It's also not really an issue if you keep it down to a minimum, which is best practice anyway.

jjames

DHawthorne wrote: »

It's also not really an issue if you keep it down to a minimum, which is best practice anyway.

I've been bitten several times by unnecessary actions in define_program that I just moved to an event only programming style to where things only happen when needed (i.e. channel channels, level updates, etc.) It makes for a very snappy system.

I do agree though that as long as you don't get too zealous in what you're doing and not repeating yourself a zillion times a second, it shouldn't too much of a problem.

ppdk

But how about third party .tko modules used in a program? If such a module uses the DEFINE_PROGRAM section there is nothing one can do to prevent this.

Kostas.

jjames

Very true. There are times that I have to use a module that I didn't write, but I typically do some pre-and-post install testing to make sure nothing has gone crazy, particularly the CPU.

You should see what it looks like with a Nuvo tuner module from AMX . . . every 5 seconds the CPU would spike to 55% - was interesting to look at it on a graph. I took that baby out and put my own module in . . . 8% all the time. Now granted, it was a Duet module - but still. The idea is the same, their define_program is an internal timeline that ran every five seconds to GET the data . . . so much for their argument that Duet is more CPU friendly than NetLinx.

mpullin

Good read!

The note recommends putting code in DEFINE_PROGRAM within a small wait, which is what I have been doing for quite a while now. After reading, I checked my code to make sure no code was outside of a wait in DEFINE_PROGRAM; in fact, my DEFINE_PROGRAM section is all waits:

wait 1{
     // do stuff every .1 sec
}
wait 10{
     // do stuff every second
}
wait 600{
     // do stuff every minute
}
// other waits for other things

The first fatal mistake that I ever made in a NetLinx program when I started doing this was having a SEND_COMMAND to a touchpanel to set button text in DEFINE_PROGRAM, just to make sure it was always up to date. Ah, those were the days.

ppdk

mpullin wrote: »

The first fatal mistake that I ever made in a NetLinx program when I started doing this was having a SEND_COMMAND to a touchpanel to set button text in DEFINE_PROGRAM, just to make sure it was always up to date.

You were NOT the only one ...

jjames wrote: »

I took that baby out and put my own module in . . . 8% all the time. Now granted, it was a Duet module - but still. The idea is the same, their define_program is an internal timeline that ran every five seconds to GET the data . . . so much for their argument that Duet is more CPU friendly than NetLinx.

It seems that they write the modules with the idea that it will run 1 module per controller. And how many times i found myself following your path.... writing my own modules for a device/protocol instead of using the provided ones....

Kostas

jjames

So, Matt mentioned a great tip if you're must have something in your d_f section: add a wait.

I'm working on an OLD where we're upgrading two old G3 panels to iPads,; so, I checked my CPU usage - almost 90% while running idle! So I removed the several for-loops in there, and used define_program as the loop:

nTP_LOOP ++
	
// Source feedback
[dv_TP[nTP_LOOP],nTP_G3_SRC_BTNS[ 1]]=(nPOP_FB[nTP_LOOP]== 1)
[dv_TP[nTP_LOOP],nTP_G3_SRC_BTNS[ 2]]=(nPOP_FB[nTP_LOOP]== 2)
[dv_TP[nTP_LOOP],nTP_G3_SRC_BTNS[ 3]]=(nPOP_FB[nTP_LOOP]== 3)
... etc.

Attached are my before and after results for CPU usage.

I should mention - I thought I got all the FOR loops - well there were two I missed, so there's a before, middle and after.

(The big hump in the beginning of the final one is part of the startup process - it wasn't finished - ouch, huh?

Summary
With FOR Loops ~90%
Two FOR Loops Remaining ~25%
Zero FOR Loops ~10%

PhreaK

If anyone from AMX is reading it would be great to see a follow up tech note on where the SNAPI Router and java's threading sits in all this as well. I assume the interaction with the virtual device is handled by the message dispatcher as with the rest of the NetLinx stuff but what happens if you have a thread that starts smashing the processor?

@jjames Those benchmarking graphs looks great - what are you using?

jjames

PhreaK wrote: »

If anyone from AMX is reading it would be great to see a follow up tech note on where the SNAPI Router and java's threading sits in all this as well. I assume the interaction with the virtual device is handled by the message dispatcher as with the rest of the NetLinx stuff but what happens if you have a thread that starts smashing the processor?

@jjames Those benchmarking graphs looks great - what are you using?

Just another one of my 'for-fun' projects; NetLinx pays the bills, R&D keeps me sane.

John Nagy

Realistically, it's probable that some of the wasteful loops in and high CPU usage when IDLE are happening because the processor is IDLE and sitting in DEFINE waiting for an event. They won't be taking cycles when events take the processing chain elsewhere. So some worry may be for naught, as you don't "save" anything by having an idle CPU spinning lower instead of higher. It's not an engine that will throw a rod or waste gas from overrevving at the stop signs with the clutch out.

Not to diminish the value of efficient program design, but let's not freak out and fearsomely go looking fix old stuff that isn't broken.

jjames

John Nagy wrote: »

Not to diminish the value of efficient program design, but let's not freak out and fearsomely go looking fix old stuff that isn't broken.

Agreed, I just happened to have found this one as I was working on it. I actually only looked at it and did this test because 1) this thread, 2) things seemed sluggish when testing (though I'm starting to think it's the latest TPC - everything was snappy, then I updated and it's slower than snot.)

For data purposes, I had:
DEFINE_PROGRAM
FOR loop a
{
FOR loop a1{ }
FOR loop a2{ }
}

FOR loop b
{
FOR loop b1{ }
FOR loop b2{ }
FOR loop b3{ }
}

Getting rid of the loops with nested loops helped with the CPU (obviously.) If the CPU was that high while idle, it's possible that during heavier events CPU could get even higher. What this means - not sure. I think I'm going to run some tests to try and get the CPU as high as possible, then throw in some intensive functions and call them to see what happens.

John Nagy

The current available TPC release is indeed very VERY slow compared to their prior release. I'm presently testing the 25th build post the current one, and the speed is again close to where it was at the peak (which was FAST, faster than an AMX panel), and the accuracy and robustness of the connection are improved significantly (dropped connections and missed commands were not uncommon under heavy load in the prior design, it was fast but a little too loose). A new replacement is due for release very soon, I think. They are still ironing out intercom issues in part irritated by ios5.

jjames

Thanks for the info. Didn't mean to thread drift, but this is good to know. I even went as far as reluctantly updating to iOS5 on the iPad.

mush

PhreaK wrote: »

DEFINE_PROGRAM will execute *every time* a global variable is written to.

The way I read that is it will only run if the global variable is contained within DEFINE_PROGRAM.
Otherwise a global won't trigger DEFINE_PROGRAM.
Anyone disagree?

ppdk

mush wrote: »

Anyone disagree?

Me not. That is why from now on (and maybe on old installations) i will try to avoid put a global variable in the
DEFINE_PROGRAM section. After all nowdays i am writting my feedback in functions running on events.

Kostas

felixmoldovan

mush wrote: »

The way I read that is it will only run if the global variable is contained within DEFINE_PROGRAM.
Otherwise a global won't trigger DEFINE_PROGRAM.
Anyone disagree?

Not disagreeing, but having a different understanding.
They use the explanation that many programmers are syncing button states based on variable values, and that would be a good reason to run DEFINE_PROGRAM "just in case" for ANY variable changed, whether its referenced or not in DEFINE_PROGRAM

Tech Note wrote:

...it makes sense to run DEFINE_PROGRAM if any change is detected in the states of any variable in the program. Normally, this is a very beneficial process.

vining

felixmoldovan wrote: »

Not disagreeing, but having a different understanding.
They use the explanation that many programmers are syncing button states based on variable values, and that would be a good reason to run DEFINE_PROGRAM "just in case" for ANY variable changed, whether its referenced or not in DEFINE_PROGRAM

The TN implies any global var change, anywhere will cause DP to run but it make slightly more sense if it indeed only applied to globals used in DP. Personally I think what's the point, DP is going to run during idle time anyway which is typically very many times a second, fast enough to handle FB properly.

felixmoldovan

"What makes sense"
"What actually is"
"What is explained to be like"
"What we understand out of it"
The very reason of having a lengthy dissertation based on something that's supposed to throw light on everything :-)

mush

ppdk wrote: »

...i will try...

Kostas

..there is no try..

PhreaK

mush wrote: »

The way I read that is it will only run if the global variable is contained within DEFINE_PROGRAM.
Otherwise a global won't trigger DEFINE_PROGRAM.
Anyone disagree?

I read it as any global. I tend not to use DP so if it's only referenced variables that'd be all good, however if it is in fact any global it would explain some odd behavior I've seen in systems that utilized some closed source modules which may have had expensive code in DP.

PHSJason

AMX Queue technote

Since reading TN870 relating to creating a queue and using to send strings to a device, I have been using the method as described in the tech note:

DEFINE_DEVICE
 
dvLutron                   = 5001:1:0     // Lutron Radio Ra
dvTP                         = 10001:1:0  // MVP-8400
 
DEFINE_VARIABLE
 
Integer BUSY
Char cLutronQue[255]

Define_Function SendLutronQue ()
{
    Local_Var Char cCmd[50]
    If(![dvLutron,BUSY] && Length_String(cLutronQue))
    {
        cCmd = Remove_String(cLutronQue,"13",1)
        Send_String dvLutron,"cCmd"
        Send_String 0,"'Sent Lutron Command: ',cCmd"
        On[dvLutron,BUSY]
        Wait 5
        Off[dvLutron,BUSY]
    }
}
Define_Function AddtoLutronQue (Char cCmd[50])
{
    cLutronQue = "cLutronQue,cCmd,13"
    Send_String 0,"'Added Lutron Command: ',cCmd"
}
DEFINE_EVENT
Button_Event[dvTP,1]
{
       Push:
       {
              AddtoLutronQue ('SDL,1,50')
              AddtoLutronQue ('SDL,2,75')
              AddtoLutronQue ('SDL,3,0')
              AddtoLutronQue ('SSL,4,68')
              AddtoLutronQue ('SDL,5,15') 
       }
}
DEFINE_PROGRAM
SendLutronQue ()

Since this method is calling a function that is writing a global variable, does that mean that it is force-looping itself? Not to mention calling DP on the button release and the channel events...

If I read this correctly, the first instance of 'AddtoLutronQue' causes DP to run by writing a global var, which calls to 'SendLutronQue ' which reads/modifies global var 'cLutronQue' which causes DP to run, then modifies global var 'Busy' which causes DP to run, then turns 'Busy' off which causes DP to run again....

How many times does this method of queueing cause DP to run per instance of adding to the queue?


Lately, I have been using a timeline to trigger the function call instead of placing it in the DP section.

mpullin

PHSJason wrote: »

DEFINE_PROGRAM
SendLutronQue ()[/code]

Add "wait 1 " before the SendLutronQue(). Solves problem with 7 keystrokes. No timelines necessary :-)

jjames

Or ditch the recurring all together and have it fire only when it's needed. I understand having something manage everything for you is simple, but now you're relying on something else's ability to, and getting whatever results you get. Manage it yourself.

Define_Function SendLutronQue ()
{
    Local_Var Char cCmd[50]
    If(![dvLutron,BUSY] && Length_String(cLutronQue))
    {
        cCmd = Remove_String(cLutronQue,"13",1)
        Send_String dvLutron,"cCmd"
        Send_String 0,"'Sent Lutron Command: ',cCmd"
        On[dvLutron,BUSY]
        Wait 5
        Off[dvLutron,BUSY]
    }
}
Define_Function AddtoLutronQue (Char cCmd[50])
{
    cLutronQue = "cLutronQue,cCmd,13"
    Send_String 0,"'Added Lutron Command: ',cCmd"
    SendLutronQue();
}

channel_event[dvLutron,BUSY]
{
	off:{SendLutronQue()}
}

Do it when needed . . .

HARMAN_icraigie

Forever I have had a "mainline" counter that increments a global integer variable +1 every pass through DEFINE_PROGRAM; reporting a 60second averaged value to the console and RMS through a virtual device level.

Back in the "old days" where all sorts of crap live in D_P this counter would read off some abysmally low numbers (under 10 wasn't unusual) and there was definitely performance issues with those systems. I have put this same counter into problematic systems where I have been contracted to "fix the system" and without fail will see very low reported values.

The current code model has minimal programming in D_P with all UI feedback triggered only when a valid effecting change has occurred (device events and UI device and room focus changes calling common functions), and the counter now reports significantly "improved" values (>1000 is the norm with 2500 and higher observed) on large residential systems (24+ rooms w/UI's) with great user perceived performance.

The caveat being cpu usage is consistently "maxed" out - 99.x% instantaneous readings, 100.0% 30 sec max values and ~10% 30sec averages.

After commenting out that "mainline" counter increment cpu usage consistently shows 1.8% 30 sec. average and 3.5% 30 sec. max.

Question is - does it matter? There is a valid use case for having the performance parameterized then reported and logged - but which performance parameter?