Who Tests Your Program??

Sensiva

Hello all,

When you write a NetLinx program, who tests it?? You, or you ask someone else do test on your behalf??

I tried many times to test my own program, and it yields to two things; First I never stop adding features and never finish the program, Second unconsciously I test it in the way I know it would work(from my programming perspective) not as any one who knows nothing about Netlinx would do.

So I am curious... How about YOU??

TonyAngelo

Sensiva wrote: »

Hello all,

When you write a NetLinx program, who tests it?? You, or you ask someone else do test on your behalf??

My boss and me just talked about this the other day. My concerns are basicly the same as yours, when I test the panel I tend to fall into a usage routine and may not discover some problems that exist. I always like to have someone else run through the panel a few times before I put it in the customers hands.

mpullin

Everyone in my office tests the program. Then we have the clients come into our office and test the program. Then once it's installed, everyone tests it again.

Self-testing is impossible, for the reasons listed above - it only catches the grossest possible blunders.

Sensiva

mpullin wrote: »

Then we have the clients come into our office and test the program.

I don't get it, you mean you have all the controlled equipments at your office?

mpullin

Sensiva wrote: »

I don't get it, you mean you have all the controlled equipments at your office?

Right, we have a staging area where we set up all the equipment and test everything, before it goes into the client's house. This is not always possible of course due to existing equipment and equipment that is just too big and heavy for our space, but we try to get everything working here first. That way when it goes into the client's house, the only problems that can arise are those with environmental causes.

jjames

Typically - my boss and I test our systems. My boss has a knack of using remotes unconventionally - on purpose, he had to because he was the only one testing it before I got there. I'm learning from him on beating up a system, but we don't have too many problems.

It really is key to use the system how it's not "supposed" to be used, because I can guarantee you - at some point it will be used that way. When testing, my boss and I do the oddest things - I can't quite explain it . . . it's strange. But . . . it definitely works. We also try to make the system "kid proof" even if there aren't any kids. Meaning, you just hit random buttons as fast as you can and see if the system follows. Stuff like that we do on a regular basis to make sure the system is good.

But - no matter how much testing you do - I can almost promise there will still be bugs. If you factor in how many buttons are on a touch panel, just think of all the combinations of sequences. The most "bullet proof" systems still have bugs - I promise that!

yuri

Sensiva wrote: »

I don't get it, you mean you have all the controlled equipments at your office?

we do too. In case we have new equipment, testing it out at our office is the best way to handle it.
This way, multiple people can test the system.

Thomas Hayes

I usually test all my own code and then pass it off for the rest of our techs to give it a whirl then I look at it again. Often I can pick up an error after I have a few days away from the code.

[Deleted User]

Code bugs and testing

As a rule, bugs always show up when you think you are completely done and you are showing someone the latest cool feature.

Just like a newspaper needs an editor, a code writer needs a separate button pusher.

TonyAngelo

B_Clements wrote: »

Just like a newspaper needs an editor, a code writer needs a separate button pusher.

So when can we expect a new SKU?

DHawthorne

I usually try to get my installers to see if they can break something for me, but I often have to test myself because I am in after they have moved on to the next job. So I generally leave an advisement with the customer along the lines of, "if you find after using it a while that you would like anything to operate differently, let me know and I'll adjust it if possible." Which is a veiled way of saying, "let me know if you find any bugs that I missed."

jweather

It's usually helpful to have someone test who isn't familiar with the particular project. It's even better if they don't have a technical background. Then have one of the installers or salespeople "train" them on the system (if applicable, for residential systems), or have them just try to figure it out on their own (as often happens with shared conference rooms). Either way, they're more likely to break things because they don't have any preconceived notions of how to operate the system. It also makes it easier to find usability problems -- if their first instincts about how to do something are wrong, it might indicate that the system would be more usable if it DID work that way.

On the flip side, you may find even more bugs from a very detail-oriented person... our organization has several people who are legendary for testing every last piece of the panel, often finding unusual situations that were not handled correctly. (but what if I'm listening to the rec room theater which is listening to the family room theater which is listening to the shared XM tuner? etc.)

JohnMichnr

I usually find bugs in teh middle of training the end users. There is something about a slow step through of the system while explaining it to other people that helps find the errors.

I always prefer to have somebody else walk through the panel and look at it. Hopefully pushing buttons that I don't

Sensiva

Thank you all

Thank you all for your contributions, but I still can't believe it, how could you bring the whole home, or the whole boardroom equipments into your lab and start doing your code?? its.... HUGE... real HUGE, lights, HVAC, all Audio video related device like players servers and distributors..... also Monitors and TVs

I always try to ask my boss if its possible to grab a device into office that I am not sure of its protocol to start findout what's the deal with it... but bringing the whole system??!? I always dreamed of that.

DHawthorne

Sensiva wrote: »

Thank you all for your contributions, but I still can't believe it, how could you bring the whole home, or the whole boardroom equipments into your lab and start doing your code?? its.... HUGE... real HUGE, lights, HVAC, all Audio video related device like players servers and distributors..... also Monitors and TVs

I always try to ask my boss if its possible to grab a device into office that I am not sure of its protocol to start findout what's the deal with it... but bringing the whole system??!? I always dreamed of that.

I have once, and only once, had a full system in my shop to tweak and test before it got installed. Usually, I get to nail down the operation of one or two devices in my shop, but most often, I just have my test MVP-8400 and NI-900. I almost never see the full system in operation until I am on site.

JohnMichnr

One of the contractors I work with still does things the old fasioned way - they eran it. no they build it all in the shop before it goes to site to allow testing of the systems. I can get to their shop, grab a projector or display hook it up ( they have a bunch of cables of every description - never the correct ones but still) they have speakers in the ceiling with jacks on the wall, I can set up an entire system to test if I need to.

Do I often do it? Nah... I am usually about a week behind so I miss the oportunity to get there, unless it is a really different application or new equipment that I have to play with.

[Deleted User]

The company I use to work for did this for every job. It really is the only way to go. It is so much faster and easier and you get rid of all the bugs before the client ever sees the system. Where I work now, we do everything onsite and I hate it when a client walks up to a unfinished touch panel, hits a buttons and notices nothing happened or the wrong thing happened. It seems to plant a seed of doubt in their head even though the system isn't finished. When the system is up and running that little bit of doubt means that if they do something stupid they will immediately think the system is broken or still unfinished rather than they did something wrong.
Paul

yuri

a_riot wrote: »

The company I use to work for did this for every job. It really is the only way to go. It is so much faster and easier and you get rid of all the bugs before the client ever sees the system. Where I work now, we do everything onsite and I hate it when a client walks up to a unfinished touch panel, hits a buttons and notices nothing happened or the wrong thing happened. It seems to plant a seed of doubt in their head even though the system isn't finished. When the system is up and running that little bit of doubt means that if they do something stupid they will immediately think the system is broken or still unfinished rather than they did something wrong.
Paul

so true

ericmedley

Sensiva wrote: »

Thank you all for your contributions, but I still can't believe it, how could you bring the whole home, or the whole boardroom equipments into your lab and start doing your code?? its.... HUGE... real HUGE, lights, HVAC, all Audio video related device like players servers and distributors..... also Monitors and TVs

I always try to ask my boss if its possible to grab a device into office that I am not sure of its protocol to start findout what's the deal with it... but bringing the whole system??!? I always dreamed of that.

We burn all our equipment in. We set up the system in our burn-in area. We build the racks here. We divide the wiring into three catagories, inter-rack, intra-rack (from one rack to another) and house wiring. Our average install time on $400K plus systems is about 1-2 weeks.

Lighitng and security are not totally done here. We typically set up the masters here and make sure we can communicate with them and put up mock sensors for the alarm systems.

Our clients will often come 'visit' the systems while in burn-in. They like to see the work in progress.

Program trouble-shooting is typically done my our designers and techs. They seem to know just how to break things.

An impatient designer is your worst enemy and best friend.

BCalderwood

I agree that having a burn-in is the only way to go. Having a full system to test before the installation is pretty much not going to happen or if it does, rarely unless you're a big company with warehouse space to spare. As many times as I code and try to make something cookie cutter, there are always some things that are changed or a client wants something different than the last one. A lot of people don't really see that because of the advent of tech, things change constantly. There is always a guy who wants the newest component or a manufacturer who thinks that the RS-232 protocol that they've been using and has worked great for the last year or two, all of a sudden has to completely change on the new model because they've added an HDMI output. I test at home and try to get as much done before the first download on-site, but there's always something and that something will always be the first thing to pop-up, not when you are testing, but when someone else is pushing the buttons. I hate to say it, and the dealers hardly understand this, but that new DVD player or new switcher is not as easy as they believe it to be because all dealers want a smooth system integration and have convinced themselves it will be before the programmer even gets word of what's going in there. I'll stop before this turns into a rant.... or am I too late?