CaptureIR: how to use data from digitrace

[mtakahar]

Maybe it takes a multi-cpu system to be this free of latency problems, and that 100 uS setting will cause spurious gaps on a less powerful system.

Sounds like that's what's happening...

That was just speculation on my part. I haven't tried the 100 uS setting on a single CPU system so I don't know it would have latency problems.

Now I got the proof here. (this is an Excel-generated html and kinda very large.) I'll look into Win32 and XP drivers specs and see if I can do something.

I uploaded the prototype 5:

• fixed the problem losing last few bursts
• fixed the problem losing the entire signal that couldn't be decoded
• stderr and stdout to files
• add command line options to print signals before/after demodulation to stdout (this is for diagnostics. I tried popping up a dialog box, but it was so annoying for this purpose.)
• change minGapLen to 100 (decreases success rate for some protocols on slower systems, but makes it easier to isolate other problems. I'll keep it this way in the meantime.)

The issue losing last few bursts was there mainly because handling the whole macro sequence was out of scope in my initial design, but I overlooked small/big things here and there that were still assuming that. It seems to be handling all the samples after fixing what John pointed out plus removing a few pieces of the code that were trimming off a sample or two.

The "-1" stuff got in there while I was playing with the time data and see how I could make the code look cleaner, that is, at one point in time, I was doing something like: time[i+1] - time, etc. This was also a part of the cause for last bursts issue above.

As for the final fake OFF, I'm plugging in 999999 for now.


Hal

[johnsfine]

Since the capture process is taking fine grain timestamps anyway, it can easily detect its own latency problems. We can then pass both edge times and latencies up to the next higher level. Then demodulation can be more accurate about both the frequency and the short gaps.

I have some good ideas on how to code all that effitiently, so if you want to focus on other issues, I can find time soon to try that.

Roughly the idea is:

newState = inputState()
prevStamp = currentStamp
currentStamp = time()
if ( newState != currentState )
{
currentState = newState
emitEdgeDelta( currentStamp - prevEdgeStamp )
emitLatencyDelta( currentStamp - prevStamp )
prevEdgeStamp = currentStamp
}

This generates a series of EdgeDeltas (duration between edges as measured) and LatencyDeltas (max error in postioning the current edge).

The true duration of a half cycle will be within the range of the stored duration plus or minus the sum of the two end latencies.

The true duration of a full cycle can ignore the middle latency and be the sum of two half cycle durations plus or minus the two end latencies.

When the minimum possible duration of a gap is 100 uS we can be confident that it's really a gap.

When the minimum duration of a cycle could be a modulated cycle, but the maximum is greater than 1.5 modulated cycles (at some working estimate of frequency) then we must assume it's part of a modulated burst, but it can't help us determine the frequency.

[johnsfine]

[quote="mtakahar"]
I uploaded the prototype 5

I tried that. When it gets a signal it randomly (about one time in five) crashes and creates this err.log file:
http://home.comcast.net/~john.fine/ir/h ... id3260.log

[mtakahar]

It looks like the garbage collector is calling the DecodeIRCaller.finalize method at a bad timing, but I can't reproduce it on my PC.
Could you try moving the following line all the way up to the beginning of the "run" method like this and see if it fixes or improves the situation?

Code: Select all

    public void run()
    {
      DecodeIRCaller decodeIR = new DecodeIRCaller();

      while (true)

Hal

Edit: I've made the change and replaced prottype 5 with prototype 6:
https://www.hifi-remote.com/forums/dload ... le_id=2035
I realized the err/out redirected files didn't work well depending on the way you invoke it. I fixed it in tihs prototype as well.

Edit2: I started raising the thread priority to the highest value during capture. Now even the mouse pointer stops moving while it's capturing signals. Instead, I stopped raising the priority on the Java side. This combination seems to work better both in terms of GUI response (assuming you don't try using mouse/keyboard at the same time as you press a button on the remote) and quality of the signals, however, improvements to the signal quality looks only marginal on a single processor/no HT system.

[johnsfine]

Now it dies every time, as soon as I press a button on the remote:
http://home.comcast.net/~john.fine/ir/hs_err_pid628.log
http://home.comcast.net/~john.fine/ir/h ... id3764.log

Since you seem to understand those err.log files, can you tell me why DecodeIr.dll isn't listed in the massive list of Dynamic libraries?

Do you have a guess how an invalid address got into the pc register?

The common way to get that type of failure in other environments is to have an array as a local variable in a C function, and have a bug write past the end of the array. Then on return from the incorrect function the pc register is trashed.

I haven't yet looked through your C code to see if anything might fit that description.

[mtakahar]

I previously thought it might be related the garbage collector, which calls the finalize method of DecodeIRCaller class, then the finalize method frees DecodeIR.DLL, and as you noticed, DecodeIR.DLL seems to be gone from the memory. I thouhgt it would go away if I stopped creating/throwing away the instance every time in the loop.

Anyway, I guess I was wrong, and I took a little closer look at the stack dump:

Code: Select all

Stack: [0x03170000,0x031b0000),  sp=0x031af8a8,  free space=254k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x8bbe7]
C  [CaptureIR.dll+0x1fc5]
j  DecodeIRCaller.decode([J[I[IILIRSignal;)Z+0
j  DecodeIRCaller.decode(LIRSignal;)Z+17
j  CaptureIR$monitorThread.run()V+296
...

"DecodeIRCaller.decode([J[I[IILIRSignal;)Z+0" means the DecodeIRCaller.decode method with arguments: [J=long[], , one more int[], I=int, LIRSignal=class IRSignal, which is the native method implemented in C.

objdump -D CaptureIR.dll shows:
(Note: This is a Unix-style disassembly. Order of operands are reversed from Intel/Microsoft style.)

Code: Select all

CaptureIR.dll:     file format pei-i386

Disassembly of section .text:

...

10001f7e <_Java_DecodeIRCaller_decode@28>:
10001f7e:       55                      push   %ebp
10001f7f:       89 e5                   mov    %esp,%ebp
10001f81:       81 ec 88 03 00 00       sub    $0x388,%esp
10001f87:       c7 85 b4 fc ff ff 00    movl   $0x0,0xfffffcb4(%ebp)
10001f8e:       00 00 00 
10001f91:       8b 45 08                mov    0x8(%ebp),%eax
10001f94:       8b 10                   mov    (%eax),%edx
10001f96:       8d 85 b8 fc ff ff       lea    0xfffffcb8(%ebp),%eax
10001f9c:       89 44 24 10             mov    %eax,0x10(%esp)
10001fa0:       c7 44 24 0c 01 00 00    movl   $0x1,0xc(%esp)
10001fa7:       00 
10001fa8:       c7 44 24 08 02 00 00    movl   $0x2,0x8(%esp)
10001faf:       00 
10001fb0:       8b 45 10                mov    0x10(%ebp),%eax
10001fb3:       89 44 24 04             mov    %eax,0x4(%esp)
10001fb7:       8b 45 08                mov    0x8(%ebp),%eax
10001fba:       89 04 24                mov    %eax,(%esp)
10001fbd:       8b 82 30 03 00 00       mov    0x330(%edx),%eax
10001fc3:       ff d0                   call   *%eax
10001fc5:       83 ec 14                sub    $0x14,%esp
10001fc8:       8b 45 08                mov    0x8(%ebp),%eax
10001fcb:       8b 10                   mov    (%eax),%edx
10001fcd:       8b 45 18                mov    0x18(%ebp),%eax
10001fd0:       89 44 24 04             mov    %eax,0x4(%esp)
10001fd4:       8b 45 08                mov    0x8(%ebp),%eax
10001fd7:       89 04 24                mov    %eax,(%esp)
10001fda:       8b 82 ac 02 00 00       mov    0x2ac(%edx),%eax
10001fe0:       ff d0                   call   *%eax
10001fe2:       83 ec 08                sub    $0x8,%esp
10001fe5:       89 45 e8                mov    %eax,0xffffffe8(%ebp)
10001fe8:       83 7d e8 00             cmpl   $0x0,0xffffffe8(%ebp)
10001fec:       75 0f                   jne    10001ffd <_Java_DecodeIRCaller_decode@28+0x7f>
10001fee:       c7 85 b0 fc ff ff 00    movl   $0x0,0xfffffcb0(%ebp)
10001ff5:       00 00 00 
10001ff8:       e9 6d 03 00 00          jmp    1000236a <_Java_DecodeIRCaller_decode@28+0x3ec>
...

"[CaptureIR.dll+0x1fc5]" is where it just came back from the call to GetLongArrayRegion, so I guess "[jvm.dll+0x8bbe7]" is somewhere in GetLongArrayRegion itself.

Edit: I previously thought the JVM would be calculating the caller's address, but I guess it's just printing conents of the stack that has the return address. I tried passing null as the first argument to the decode method from the Java code and got the same stack trace. Looks like the DecodeIRCaller instance was finalized (freed) for some reasons even though it is still in use.


I'll try to find something that might cause it. Whatever the case, maybe I should throw a nullpointerexception either here or on the Java side.

Hal

[mtakahar]

Looks like the DecodeIRCaller instance was finalized (freed) for some reasons even though it is still in use.

Ok, I found what was wrong. A field in the DecodeIRCaller class had the static keyword, which I believe I had to add, may be because I was using it from in a static context at some point, but it's no longer the case. I think it was freed when the garbage collector ran and cleaned up the first instance that was created to get the DecodeIR version number.

I hope this one works well on your system: prototype 7.

Hal

[johnsfine]

It's not crashing. But it doesn't work.

The raw signals look perfect. The first one after starting the .jar decodes right and occasionally after that another decodes right. But mostly they don't decode even though perfect raw data is displayed that should decode.

I expect some garbage is being left in the context array or some other part of the data associated with communicating with DecodeIr. I don't have time for careful testing and/or reading the code now. But if you don't find it first, I think I know enough to find and fix this one.

[mtakahar]

That was because I stopped constructing the DecodeIRCaller instance every time in the loop. I added explicit decoder context re-initialization for now (I might go back to the previous model of re-constructing the instance every time later if I can find cleaner way to deal with load/freedll), and posted Prototype 8.

Edit: I've just removed a few conditions in the frequency transition detection code that were intended to compensate fluctuations coming from latency because it may cause some confusions while we are focusing on quality of the captured signals.

Hal

[johnsfine]

We seem to be on a solid trend of getting worse.

What did you change in CaptureIR.dll?

I was keeping each version of CaptureIr.jar but I didn't keep the last few CaptureIR.dll.

New symptom: You get exactly one capture and decode, then it's DEAD (not just no decodes, not even captures). Stop capture and restart doesn't help. Close the program and restart doesn't help. Log out and log back in again doesn't help. Only shut the computer down and reboot helps. Then once more you get only one capture.

I did limited mix/match testing with the new CaptureIR.dll and previous few CaptureIr.jar, that all fails as above so the relevent change isn't in the .jar.

I still have the .jar and .dll from some older prototype (8/12 11:54) and those still work together as they did before, so the new symptom isn't a hardware or OS change.

Using those older files right after prototype 8 doesn't work (without a reboot) so whatever prototype 8 does to trash the environment (parallel port settings I assume) affects the previous version.

[johnsfine]

I just tried the old CaptureIr.dll with the prototype 8 .jar file and things look basically healthy.

The about box says Prototype 7, so I hope I'm not confused about what .jar I'm using (it says 9:14 AM in its file date, which is later that the prototype 7 jar I still have).

[mtakahar]

What did you change in CaptureIR.dll?

I started setting the byte mode bit in the ECR port (base + 0x402) because it stopped working after enabling the ECP mode for something else. I realized that I was incorrectly setting the previous ECR value to the control port (base + 1) after capturing was done. I didn't notice this screw up until now because it heppened to have been working on my PC. Sorry.

Try the Prototype 9.

The about box says Prototype 7, so I hope I'm not confused about what .jar I'm using (it says 9:14 AM in its file date, which is later that the prototype 7 jar I still have).

I often forget updating it. I'll patch it in the Makefile automatically from the next time.

Hal

[johnsfine]

I just uploaded a new Decode_IR_DLL.zip version 2.29

It tightens the rules for generic RC5 and for Async to greatly reduce the spurious decodes of both of those (hopefully without reducing correct decodes of them).

If you followed my suggestion and used a pid_0003 upgrade to do some moderate stress testing of CaptureIr, you'll want this new DecodeIr.

For perfect captures of pid_0003 the old one gave the correct pid_0003 decode but also gave rc5-1-16 and/or Async decodes. Now it gives just the pid_0003 decodes (in theory bad captures of pid_0003 should still often decode as Async, but I'm not getting any bad captures).

[johnsfine]

I think I just got a latency glitch (despite 4 apparent CPU's). But what is bothering me more is the behavior after the latency glich. Here is the beginning of the raw data:

6 -239 6 -133 6 -2262 6 -98 6 -2314 6 -81 6 -64 6 -99 6 -1158 7 -98 6 -99 6 -99 6 -1141 6 -99 6 -98 7 -116 6 -81 6 -1032 6 -12 6 -12 6 -12 7 -12 6 -12 6 -12 6 -47 6 -98 6 -116 7 -2082 6 -12 6 -12 6 -12 6 -12 6 -12 6 -12 6 -12 6 -6 6 -12 6 -12 7 -12 6 -12 6 -12 6 -12 6 -47 6 -12 6 -24 6 -12 6 -12 6 -12 6 -12 6 -12 6 -12 6 -12 6 -2030

Note the long series of "6 -12". Why does that not get demodulated? I assume it is somehow caused by the latency glitch before it, but I think we can make the system much less sensitive to occasional latency glitches. I may try coding some of that later. I assume I won't be able to test it on this computer because once I want a latency glitch I'm sure I won't be able to cause one (I've just had this one so far in all my testing).

Anyway, Prototype 9 seems to be robust relative to all the other issues just discussed.

[mtakahar]

Looks like it's failing to recognize the transistion from unmodulated to "6 -12 ..." and thinking this whole sequence is unmodulated.

I'll try this pattern once I get home tonight.

BTW, is this a part of a bigger signal stream? (I'm guessing so because I don't see -999999) If so, is the frequency splitting working ok to other parts in this capture?

Hal