CaptureIR: how to use data from digitrace

[johnsfine]

I'll add some notes on these points by the time of official release.

Please don't take any of my comments as complaints. I appreciate the fact that you made this available now rather than waiting until you had time to polish it.

In this model, the capturing part has to wait until it sees many enough sample (raw on/off cycles)

"enough" is a strange concept in this context. There should be some maximum, but reaching that maximum shouldn't be common.

I plan to review DecodeIr.dll to make sure it doesn't place any relevent limit on maximum number of bursts. I'd prefer it to be ABLE to handle monster raw captures of macros, rather than force the low level to slice.

or timeout occurs (I set it to 3 sec for now).

I think you mean total rather than gap. I don't think any total for duration is reasonable. It's both way too long and way to short. If you need an end of capture point, it should be a maximum gap. Probably 0.25 seconds is the right maximum gap to declare an absolute boundary in the capture.

It's probablly better setting the timeout for off time that is long enough to catch button release but not too long.

I hope you mean what I tried to say just above.

Another thing is the way I'm setting the thread priorities. I gave 7 (on the scale of 1 - 10 with the default 5) to the capturing thread and 3 to the thread that monitors the capturing thread.

I didn't realize it was threaded. When I thought about building something like this myself I hoped to be able to use threading to get instant results, but I wasn't sure I could.

If appropriate I'll build another entry point to DecodeIr.dll to make it more friendly to a threaded design.

Optimally decode attempts should begin within a tenth of a second of the start of signal. I was worried that might not be practical (might not be possible to get decent time slicing from Windows to give the capture side low enough latency to continue capturing while giving the decode side some time). But if the time slicing works the rest isn't too hard. You could kludge with what's already in the context array. But it would be better to add something more focused on threading to the dll interface.

Assuming decoding can start without wrecking the ongoing capture, my idea is to have the top level logic grap an incomplete capture and display all the decodes from that, then look at what the capture side has added in the meantime. Assuming it is enough, it would once again display all decodes, but limited to those that extend into the new part of the capture.

If it can all run continuously, you also would want a maximum for the raw source of a decode and a maximum gap. Either of those allow you to throw away the (already decoded) beginning of the raw data buffer.

I haven't looked into DecodeIR much enough to understand what the "context" array is exactly for, but my guess is that I can call it repeatedly with the same contents of the array and it'll decode each detected subsequence continuously?

Right. I half realized you weren't doing that but failed to appreciate all the consequences.

When you have a new set of raw data you must clear the context array. Then you call DecodeIr multiple times. Before each such call you must reset the output fields to their no-decode values (-1 numeric, null first char of string), but you don't reclear context. You pass each such call the context left by the previous. The loop ends when there is no decode.

No, it doesn't detect repeating patterns, at least yet. Perhaps it makes sense to have a checkbox for specifying whether you want to keep the duplicates or not.

I lept to a stupid conclusion based on misunderstanding part of the behavior.

Please don't add any pre-decode reduction.

Post decode reduction would help a lot: Once there is either the multiple calls described above OR the sliding window threading described above, many signals would show up 10 times per second. It would help a lot to compare each decode against the previous and increase a "frame count" column when they match rather than add a new output row.

- GNU Make. There are various kinds of ports that work on the
Windows platforms. Perhaps most of them work just fine.
I used the standalone Mingw version (as opposed to cygwin mingw.)

I've had nothing but trouble in all my previous attempts to use anything from Mingw.

I've got Cygwin on this computer and a very old install of DJGPP on another. Both have the GNU make and compiler etc. But both have their own view (different from each other and from Mingw) of how to pretend that Windows specific system services fit a GNU pretense of portability.

So I either have to try to kludge code written for Mingw to work in one of the others, or try to sort out the environment variable mess of having both installed at once. I'm sure either task is easy for someone who knows their way around GNU. But I've done almost everything in Visual Studio.

- GNU C compiler (gcc) if you want to rebuild the JNI (Java Native
Interface) program files written in C. You can download this from
the Mingw site mentioned above. Cygwin version may work, but not
tested. It should not be hard to make it work with MSVC, however,
again, it is untested and you may need to make some minor changes.

At first glance it looks like you included something in your dll to interface between Java code and DecodeIr.dll

If that's correct AND the same could have been done in msvc AND the Java code could talk to two dlls (one for capture, one for decode), I think it would make more sense for me to add a Java callable entry point directly inside DecodeIr.dll (other people want that for other similar projects).

I'll try to figure out how from looking at your code, but any hints or URLs of tutorials or other help would be appreciated.

- The following source files are included in porttalk22.zip
mentioned earlier in this document:

PortTalk_IOCTL.h
pt_ioctl.c

My previous attempts to work with those failed in compiler errors. The include files that came with my seriously obsolete msvc simply don't provide the definitions for symbols assumed by the source code in PortTalk.

[mtakahar]

I'll add some notes on these points by the time of official release.

Please don't take any of my comments as complaints. I appreciate the fact that you made this available now rather than waiting until you had time to polish it.

None taken. I don't expect this initial prototype I spent just a day or two would satisfy you.

In this model, the capturing part has to wait until it sees many enough sample (raw on/off cycles)

"enough" is a strange concept in this context. There should be some maximum, but reaching that maximum shouldn't be common.

I failed to mention that was also a pre-set value. It's 4096 raw cycles, and it comes back from capturing whichever the max (cycles or timeout) it reaches first. Longer the gaps and/or lower the carrier frequency, longer it takes to fill up the buffer.

or timeout occurs (I set it to 3 sec for now).

I think you mean total rather than gap.

Yes, the timeout is for the total at the moment.

I don't think any total for duration is reasonable. It's both way too long and way to short. If you need an end of capture point, it should be a maximum gap. Probably 0.25 seconds is the right maximum gap to declare an absolute boundary in the capture.

I realized that (using total duration timeout being a bad idea) from a few experiments I did in last few days. Ok, I'll use 0.25 sec as the default for the end-of-sequence.

I didn't realize it was threaded. When I thought about building something like this myself I hoped to be able to use threading to get instant results, but I wasn't sure I could.

If appropriate I'll build another entry point to DecodeIr.dll to make it more friendly to a threaded design.

I've been thinking of prototyping a pipe-lined version and see if it can sustain continuous decoding, which is kinda cool also from the design point of view. It might risk degrading the signal quality on slower machines, but I might still try.

Post decode reduction would help a lot: Once there is either the multiple calls described above OR the sliding window threading described above, many signals would show up 10 times per second. It would help a lot to compare each decode against the previous and increase a "frame count" column when they match rather than add a new output row.

Colapsing the duplicates into the frame count column sounds a good idea.

At first glance it looks like you included something in your dll to interface between Java code and DecodeIr.dll

If that's correct AND the same could have been done in msvc AND the Java code could talk to two dlls (one for capture, one for decode), I think it would make more sense for me to add a Java callable entry point directly inside DecodeIr.dll (other people want that for other similar projects). I'll try to figure out how from looking at your code, but any hints or URLs of tutorials or other help would be appreciated.

That would be nice if you can add the JNI entry points to DecodeIR.dll itself. I was experimenting a few different ways of returning the values, and I haven't cleaned them up yet, so please try not getting confused by my kludges.

This page has pointers to the spec and tutorials for JNI:
http://java.sun.com/j2se/1.5.0/docs/guide/jni/

My previous attempts to work with those failed in compiler errors. The include files that came with my seriously obsolete msvc simply don't provide the definitions for symbols assumed by the source code in PortTalk.

MingW used to be so crappy and unstable, but I think recent versions work very well. Maybe it's time for you to give it another try.

Hal

[ryanmc]

o man this thing is sweet nice job

[mtakahar]

The prototype 2 is here.

I've made a lot of changes, some based on feedback from John, some cleanup to the DecodeIR caller JNI and several others, including fixes to the Makefile, multi-threading related screen update problems, etc.

Known issues:

• I haven't changed carrier frequency calculation yet. This means it is still done against the whole captured sequence, and it will be wrong if there are multiple different kinds of signals with different frequencies. DecodeIR still works fine in many cases, but I noticed it doesn't do a good job for Pioneer without a correct frequency.
• I increased the sample size (in raw cycles) to 16384, but it might not be big enough depending on the macro length and kinds of signals in it.
• I think most people would try to keep holding the button until they see an indication of end-of-capturing of some sort from the program, but it'll wait until it fills up the buffer or you release the button whichever happens first. Typically you'll need to hold the button for a few seconds.

The things haven't done but I'm planning to do or try:

1. streaming
2. tool tips (aka balloon help)
3. Setup dialog
4. IRDC support

Hal

[johnsfine]

I wish I'd saved the first one before downloading and trying the second.

I didn't test the first enough when I had it to be certain, but I think the second is much worse.

Anyway the second works too rarely to be any use. Maybe my computer is too slow. It's a 2.8 Ghz Pentium 4, so if that's too slow you need quite a good computer to run this.

On occasion this version captures a signal well enough to decode, but mostly it delays or misses transitions enough to make the signal undecodable.

While it's running other tasks get close to zero CPU time (but that's OK).

I might have messed something up in soldering JP1 components onto the same DB25 as the IR sensor. But I think that would either kill it or have no effect. I don't see how it would just degrade performance.

Do you have any suggestions on how to make sure these are latency problems vs. something elese, and how to fix them if they're latency problems?

[johnsfine]

Oops!

I should be more careful to test only one variable at a time.

The key difference seems to be remote. I get all those missed transitions with the 15-2104 we normally use (for controlling the TV). CaptureIr works quite well with the 8910 Rob gave me a while back. I forget what remote I used to test CapturIr the first time, but it wasn't the 15-2104. Now I even tried swapping batteries between the 15-2104 and the 8910. CaptureIr still likes the 8910 and not the 15-2104.

The new CaptureIr seems to do a decent job of finding multiple frames in a fast macro. They decode wrong because the only fast macros I have set up are in the 15-2104. But I assume the 8910 will remain accurate once I figure out how to do fast macros.

Maybe I'll try removing that red filter in a 15-2104. I've seen some IR capture devices in the past that do much better with remotes that expose bare IR LEDs than remotes that have the red filter.

[mtakahar]

John,

There is a block of lines in CaptureIR.java that you can play with if you want to. I haven't fully implemented the setup dialog box yet partly because I haven't decided what are the options to be exposed and also I don't like the layout I initially came up with.
For example, value of the following variable decides what priority the capturing thread should run at on the scale of 1 to 10 with the default of 7. (the system default is 5)

Code: Select all

private int thread_priority = DEFAULT_THREAD_PRIORITY;

Lower the value you get better GUI response, higher the value, less chances of missing rising/falling edges but the GUI response will suck.

Actually, you have a machine that runs twice the speed of mine (1.3Ghz Celeron w/760Mhz RAM.) I've tried it with my 2104 after reading your post, but it seems to work okay both in unextended state and a fast macro with the extender.


By the way, I am playing with multi-signal/frequency stuff. In my current local copy, CaptureIR calls DecodeIR with frequency of the first part of the remaining sequence, then it peeks at the context array to get the next burst starting position and keeps calling DecodeIR until it stops decoding anything.

I noticed a few things that are not necessarily issues with DecodeIR, but I wanted to discuss with you how we handle them:

1. This is somewhat obvious from the interface design of DecodeIR. If there are multiple signals using protocols with different frequencies, it sometimes ignores a whole set of signal in the entire sequence.
2. Similarly it sometimes splits the entire sequence at incorrect places. More specifically, it sometimes treats the lead-in of the second (or any non-first signal) as the lead-out of the first signal.
3. Seems to work pretty well (based on experiments with my limited sets of specimens) if I split the whole burst sequence into groups where each of them consists of signals of the same frequency, and call DecodeIR with one group at a time.

Implementing it as described in the item #3 seems to be a reasonable choice, though, there are a few more options:

• Pass down an array/arrays of frequencies/burst position ranges
• Let DecodeIR figure out the frequencies and let it do demodulation as well

Demodulation on the DecodeIR side is irrelevant to IRDC (Tommy's previous design) if I add support for it.

Passing down the frequency data to DecodeIR is easy for me, too, but not sure if there's anything we gain by doing this with additional modifications to DecodeIR. (it's a different story if you know cases you want to handle pure raw signals.)


Another thing I noticed was that it seems to miss a pair of falling/rising edges sometimes. When it happens, it makes a few gaps look like very long on-periods. This happens more frequently while something else is running on the PC or there's some wireless data transferring taking place. (not surprisingly.) I tried putting a correction mechanism that address the cases it only missed on pair. This works sometimes, but it doesn't in other cases. I guess it can miss more than one pair. Still, I think this device has a big value because of its very simple design and easiness to build.

Hal

[johnsfine]

I wish I could tell whether any of the issues are latency or whether they're all poor performance in the IR detector hardware.

Also I wonder if my sloppy soldering is the problem in the IR detector or the printer port voltage, or what.

I won't have any time soon to make as big a change to DecodeIr as demodulating. In principle I don't have any objection to moving the demodulation and frequency detection into DecodeIr. But for any short term practical purpose I think your choice 3 is best.

There is at least one obscure protocol that encodes data via frequency. But DecodeIr is nowhere close to being able to decode that and that doesn't seem to bother anyone. For normal protocols, a true frequency shift would reliably indicate a boundary between steps for different device in a macro. That makes it helpful to cut the input to DecodeIr on such boundaries.

On the topic of frequency detection, I tried some RCA signals with CaptureIr. They are reported as 65Khz from both the 15-2104 and the 8910. I'm pretty sure they're really 57.? Khz. NEC signals give the frequency reliably, so why is it reported so high for RCA? How to you compute the frequency?

Also the capture of RCA signals seem to be much more reliable from both the 15-2104 and the 8910 than NEC or a few other protocols I tried. Is the higher frequency likely to be a factor in that?

Should I try adding a cap across the supply pins of the IR sensor? In other words, does it draw more current when receiving an IR pulse than otherwise and do the longer pulses of a lower frequency signal pull the voltage down?

[mtakahar]

On the topic of frequency detection, I tried some RCA signals with CaptureIr. They are reported as 65Khz from both the 15-2104 and the 8910. I'm pretty sure they're really 57.? Khz. NEC signals give the frequency reliably, so why is it reported so high for RCA? How to you compute the frequency?

I'm taking average of the most frequently seen cycle periods (distances between two rising edges) with a 16 usec allowance in the prototypes 1/2.

I changed that to use the average cycle periods within each "on" duration in my current version. I uploaded it as prototype 3 for your comparison. (I kept the prototype 2 this time.)

I've been seeing 57Khz for RCA most of the time and 65Khz sometimes. I think newer version is more reliable.

I also fixed a problem CaptureIR was keep calling DecodeIR when it gave up on the second or later signal chunk that would have caused a system freeze with a bad capture.

Also the capture of RCA signals seem to be much more reliable from both the 15-2104 and the 8910 than NEC or a few other protocols I tried. Is the higher frequency likely to be a factor in that?

To me, successful rate of RCA doesn't appear to be particularly higher than other protocols. Although I see Sony signals seem to be most reliably decoded.

Should I try adding a cap across the supply pins of the IR sensor? In other words, does it draw more current when receiving an IR pulse than otherwise and do the longer pulses of a lower frequency signal pull the voltage down?

I have a feeling the pins you picked may not be stably supplying much enough power. Is it possible you may have included some inverted pins (I think they are 1, 11, 14 and 17)? In Tommy's instructions, he says it's okay to pick one if you can find a pin with low internal resistance, but I didn't measure them and just went ahead and bundled 6 non-inverted pins together, which seem to be working fine so far.


Hal

[johnsfine]

I didn't want either the diode drop or the soldering difficulty to combine multiple supply pins. As I described in https://www.hifi-remote.com/forums/viewt ... 2855#p32855 I just selected pin 14 because it had the highest voltage under a 1K load (3.37 volts no load, 3.15 volts with a 1K load).

By "bundled" do you mean with diodes?

[mtakahar]

I didn't want either the diode drop or the soldering difficulty to combine multiple supply pins. As I described in https://www.hifi-remote.com/forums/viewt ... 2855#p32855 I just selected pin 14 because it had the highest voltage under a 1K load (3.37 volts no load, 3.15 volts with a 1K load).

Sorry, what I actually did was to pick the ones that are held high in a normal state, irrespective to presence of a negation logic.

I wonder if the printer driver touches pin 14 either spontaneously or in response to state transitions of other pins connected to the JP1 interface.
Or it may be just as what you suspect.

By "bundled" do you mean with diodes?

Yes.

Hal

[The Robman]

I wrote to Tommy to let him know about the progress you guys are making here. When he replied he attached updated versions of his docs that describe how to construct the probes. I haven't reviewed them yet, but I have loaded them up to the file section in case they are of use to anyone else.

The docs can be found here:
https://www.hifi-remote.com/forums/dload ... le_id=2044

I have also moved Hal's programs to the Programs folder to make them easier to find (the links haven't changed).

Here's Tommy's reply...

Rob,

Thank you for the update on Digitrace. That was very considerate of you, and I couldn't be more delighted that someone is getting some use out of it.
Perhaps I can return the favor. I always regretted that there were MANY errors in my original articles (the main article and the suppliment). Back in May I completely rewrote them to try and make them shorter, more accurate, and much easier to understand, but since there seemed to be no interest within the group I never did anything with the newer versions (attached). There are three articles. The basic article tells everything needed to use DigiTrace, without regard to a specific application. Then there are two separate articles on IR and RF probes. (There was a potential problem in the original RF probe design that has been corrected in the newer design by a change of chip, and you should try it, since you had unexplained problems.)

You can do the group a service by removing and throwing away my original articles and replacing them in some way with the attached, in case someone digging around or searching stumbles upon them. I realize that with what's going on between Hal and John they may have moved beyond the need for these new articles, but I don't like leaving the inaccurate stuff out there. It's your call.

Tommy

[johnsfine]

I brought the cable (combined JP1 cable and IR sensor) and 8910 to the office and tried it on my dual CPU 2.8Ghz Xeon. It was noticably less sluggish than on the single CPU Pentium 4.

Next I tried enabling hyperthreading (makes the OS think the dual CPU system has four CPUs). That is a big win for CaptureIR. Now there are no noticable performance issues. While the capture thread is running it basically gets one of the four CPUs, so it probably has zero latency problems. Menwhile most of the CPU power is available for other things so nothing else is sluggish. Even the buttons of CaptureIR have instant response (at home the Stop Capture button barely works at all).

Hyperthreading is usually a very mixed blessing. Many tasks go faster; Many others go slower. The product I work on runs much slower with Hyperthreading on, so even though many of the development tools I use while working on it run faster, I generally leave hyperthreading off so I can get through product test runs as fast as possible.

For playing with CaptureIr, hyperthreading is definitely worthwhile.

I also read the IR sensor documentation from Fairchild. It seems to say the model we use has open collector output. How is that reliable without a pull up resistor? It also seems to say supply voltage is 4 volts minimum. Hom much of my problems are from the 3 volt supply?

[johnsfine]

How does it decide where the end of a modulated burst is (beginning of the gap)?

It seems to have a very high cutoff.

I tried using a signal (pid 00 03 in RM) at 40Khz with some short gaps (around 300 uS).

It took me a while to figure out what the garbage CaptureIR reports means.

CaptureIr never sees the short gaps. It thinks the modulated pulse before and after the gap make up one long modulated pulse.

If you need a single cutoff value for an interval inside a modulated pulse vs a gap between, that cutoff should be much smaller than 300 uS. Maybe we want smarter than a simple cutoff, but a simple (smaller) cutoff would be better than what it's doing now.

I really need to figure out how to recompile this thing so I can tweak those things myself. But there is so much (mingw) stuff to install and so much risk of messing up my current build environment, I've been scared to.

[johnsfine]

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at CaptureIR$IRSignalTableModel.getColumnClass(CaptureIR.java:452)
at TableSorter.getColumnClass(TableSorter.java:266)
at javax.swing.JTable.getColumnClass(Unknown Source)
at javax.swing.JTable.getCellRenderer(Unknown Source)
at javax.swing.plaf.basic.BasicTableUI.paintCell(Unknown Source)
at javax.swing.plaf.basic.BasicTableUI.paintCells(Unknown Source)
at javax.swing.plaf.basic.BasicTableUI.paint(Unknown Source)
at javax.swing.plaf.ComponentUI.update(Unknown Source)
at javax.swing.JComponent.paintComponent(Unknown Source)
at javax.swing.JComponent.paint(Unknown Source)
at javax.swing.JComponent.paintWithOffscreenBuffer(Unknown Source)
at javax.swing.JComponent.paintDoubleBuffered(Unknown Source)
at javax.swing.JComponent._paintImmediately(Unknown Source)
at javax.swing.JComponent.paintImmediately(Unknown Source)
at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown Source)
at java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)

What is all that telling me?