How to decode a new protocol

[The Robman]

But OEM codes are a new one to me. I presume they are part of the Misc data that John said shows up even in some decoded protocols. I'll look more at protocols.ini, though I find that very tough going.

In the case of Panasonic, when the OEM codes (which are just additional bytes of fixed data) are set a certain way, the protocol decodes as Panasonic, otherwise it will most likely decode as one of the other Kaseikyo variants.

you need to be flexible enough that when the next device comes along and those bytes are not zeroes, you're not screwed.

But as I'm not precognitive that flexibility has to remain in the mind rather than the protocol decode! At present (in my DecodeIR addition) those zeroes and the checksum formula are part of the way that the protocol is distinguished from others. So anything with different values there will show up again as unknown.

And because it shows up as unknown, the user who encounters it will post it here and I'll have to go through the whole exercise of decoding it again, unless I just happen to remember it from this post, which is unlikely. Whereas, if the decode was more flexible, I wouldn't have to do all that extra work.

I understand that the current DecodeIR / IR.exe setup is limited to just 2 device codes, so I guess I'm asking for that to be more flexible.

[The Robman]

Generally speaking, whenever you see more than 8 bits of variable data, it's a good idea to look for a checksum...

Actually, it's not just for when there are more than 8 bits of variable data. I've recently decoded a couple of signals where there were only 8 variable bits, but I noticed that the last variable bit was always the complement of the previous bit, so in this case the OBC was only really 7 bits in length, followed by a 1-bit checksum.

In that case I would not suggest that you write assembler to handle it, instead you should handle it on the KM/RM side. But just knowing that it's there reduced the range of OBCs from 256 down to 128, which is good to know if you're intending to try all of them looking for discretes.

[mathdon]

And because it shows up as unknown, the user who encounters it will post it here and I'll have to go through the whole exercise of decoding it again, unless I just happen to remember it from this post, which is unlikely. Whereas, if the decode was more flexible, I wouldn't have to do all that extra work.

I understand that the current DecodeIR / IR.exe setup is limited to just 2 device codes, so I guess I'm asking for that to be more flexible.

This is where I need help, not in implementation but in what to implement. There is no other 40-bit "Gap" decode in DecodeIR, so I could make ANY 40-bit Gap code with a (16, -8) leadin show up as IODATA. Should I also check the 38k frequency and 550 base timing within 10%, say? I could make the two 7-bit zero bytes into OEM1 and OEM2 and make them display in the Misc field - or do so only if they are nonzero, which in present circumstances would be neater. But what about the checksum? I want to check its validity but I suspect the formula could be different with nonzero OEM1 and OEM2. My decode at present XORs three nibbles, the high and low nibbles of F and the check nibble C, and tests that the result is 1 (ie binary 1000 interpreted in lsb format). I could append this XOR result to the protocol name, so the present one would show up as IODATA-1. Then any 40-bit Gap protocol would decode as IODATA-x for some x, and if all the x's turn out to be different for someone's remote then we could rethink the checksum formula but at least would have a starting point.

Is this the sort of thing you mean? It might not be the best precognition but it would be a start.
__________________
Graham

[johnsfine]

This is where I need help, not in implementation but in what to implement.

That is a hard question in these situations. I'm always guessing and my guess isn't going to be much better than your guess.

There is no other 40-bit "Gap" decode in DecodeIR, so I could make ANY 40-bit Gap code with a (16, -8) leadin show up as IODATA.

The first value (the 16 in that leadin) is often very badly learned, so I avoid qualifying decodes on that value unless there is no other reasonably choice. The roughly -8 part of the leadin might be good to qualify on, but it wouldn't qualify much. I expect some other 40 bit protocol we've never seen would duplicate that as well.

I'm not thrilled with casting such a wide net, but I also don't have a better idea.

Should I also check the 38k frequency and 550 base timing within 10%, say?

Checking frequency at all or checking base timing anywhere near as tight as 10% tend to be counterproductive ideas. Lots of learning is bad. Some learning devices lose the frequency info entirely. Some even lose the pulse durations and have only gap durations (some substitute 0000 for all short pulse durations). DecodeIr gets the right decode from all those flawed captures as long as the protocol has enough internal structure to be unambiguous.

I could make the two 7-bit zero bytes into OEM1 and OEM2 and make them display in the Misc field - or do so only if they are nonzero,

I'm not exactly consistent, but I try to format OEM codes and such things that are thought to be less variable than device numbers into the protocol name itself, while formatting T bits and X codes and things that are more variable than device codes (especially things more variable than function codes) into the misc field. Doing so only if they are nonzero is a good idea.

But what about the checksum? I want to check its validity but I suspect the formula could be different with nonzero OEM1 and OEM2.

If we knew the checksum formula, that is the best way to qualify the signal (distinguish it from other 40 bit codes with leadin).

But I don't think we know the formula. I can't think of a way that mixing in the device code would make the checksum more plausible, but I suspect the device code is mixed in and we just can't see it, because we've only seen one device number. The same applies to the "OEM" codes or whatever those values actually are.

Is this the sort of thing you mean? It might not be the best precognition but it would be a start.

I suggest you "cast a wide net" and decode anything with 40 bit gap structure and a leadin. Load both the "OEM" values (only if nonzero) and the checksum result (I think only if non 1) into the protocol name.

When we find IODATA with a different device or OEM code we can refine that.

When we find a different 40 bit gap with leadin protocol, we can restrict IODATA.

In each case, we would have some wrong (but probably helpful) decodes when we first discover the cases we failed to exactly predict.

[mathdon]

I suggest you "cast a wide net" and decode anything with 40 bit gap structure and a leadin. Load both the "OEM" values (only if nonzero) and the checksum result (I think only if non 1) into the protocol name.

Thank you for your long and very enlightening post, John. I've done exactly what you suggest, except that I always have the checksum result in the name. So now I have

IODATAn {38k,550}<1,-1|1,-3>(16,-8,x:7,D:7,S:7,y:7,F:8,C:4,1,-60)+

where n can be from 0 to 15. C is the XOR of 3 nibbles, the two nibbles of F and n, taking n as a 4-bit lsb value. If either x or y is nonzero then they are appended to the name as IODATAn-x-y. The only change to the DecodeIR output that I posted is that the protocol name now shows as IODATA1.

I hope this covers all eventualities.

So the sledgehammer was useful in the short-term, but to truly reap the benefit I have to learn the art. And it definitely is an art.

It now seems we have another art to learn as well. I must get some fortune-teller's robes to add to my newly acquired crystal ball!
_________________
Graham

[johnsfine]

Since I keep finding less time to work on DecodeIr.dll than I intended, I'm glad someone else is picking up some of it.

Part of the DecodeIr development I do is testing against a very large collection of .ccf files that I have downloaded from RemoteCentral.

I have all the output from running DecodeCCF against all those .ccf files stored in a directory. When I have a new DecodeIr.dll, I run a script that runs DecodeCCF against all those CCF files storing the results in a different directory. Then I use Winmerge to compare the two result directories.

If I make a change to DecodeIr to add support for a new protocol, all the results for all those CCF files, that don't contain the new protocol, ought to be unchanged. So I manually decide whether the changes I do see are:

1) Good news: That previously unknown protocol was already present in some CCF files but not previously decoded, so now we have samples from a second source and maybe some alternate device codes.

2) Bad news: The new decoder is "loose" and decodes some of the random garbage learns scattered through those CCF files into false positives for the new protocol. Maybe I need to qualify the new decode logic more to reduce false positives.

3) Very bad news: A typo or other change in DecodeIr has broken the decoding for some other protocol that I didn't intend to change at all.

Before you make too many changes to DecodeIr.dll, one of us ought to run those tests. I don't know a practical way to give you the directory full of CCF files or an automated way for you to grab most of RemoteCentral's library yourself (any individual .ccf is easy to download, but I don't know how to script the process of downloading most of them). So it is probably easier if I do that testing.

[mathdon]

John, I would be absolutely delighted for you to test my augmented DecodeIR against your data. I take it all you need from me is DecodeIR.cpp and DecodeIR.h. Is that correct? I was wondering how you managed to make sure that a new addition hadn't broken anything old. I think my additions so far are for sufficiently distinctive protocols that I don't foresee them causing a problem, but one never knows.

If it's OK with you, I suggest waiting a few days for me to see if there is anything else I can easily put in. Otherwise you would have to re-run the tests after I've added a bit more. You will need to do that anyway when I reach a point that you and I agree is a good place to issue a revised version, but you won't want to do it too often.

Many thanks again.
________________
Graham

[The Robman]

I don't know a practical way to give you the directory full of CCF files

You could zip them all together and then load the resulting file to my server, then once Graham has downloaded it, I can delete it.

[mathdon]

This protocol was my first addition to DecodeIR. I've gone back to this protocol now to look at it from the executor side. I wrote earlier about the IRP

now I have

IODATAn {38k,550}<1,-1|1,-3>(16,-8,x:7,D:7,S:7,y:7,F:8,C:4,1,-60)+

where n can be from 0 to 15. C is the XOR of 3 nibbles, the two nibbles of F and n, taking n as a 4-bit lsb value. If either x or y is nonzero then they are appended to the name as IODATAn-x-y. The only change to the DecodeIR output that I posted is that the protocol name now shows as IODATA1.

I hope this covers all eventualities.

This form is a little more general than the executors that Rob and Vicky put together, in that they only cover IODATA1, ie the case n=1. For what it is worth, I've added a fifth fixed byte to cope with other n values, and turned it into protocols.ini form. It adds just 2 bytes to the length. I had no idea what to do about a PID, so I rather arbitrarily have picked 01F9, being near the end of the range but not quite at it, as I think 01FF and maybe a few down from there are used as "ad hoc" ones. Anyway, here is the protocols.ini entry:

Code: Select all

[IODATA]
PID=01 F9
DevParms=Device:7=81,Sub Device:7=75,n:4=1,x:7=0,y:7=0
DeviceTranslator=Translator(lsb,2,4) Translator(lsb,3,7,8) Translator(lsb,0,7,16) \
Translator(lsb,1,7,24) Translator(lsb,4,7,32) 
CmdParms=OBC:8=0
CmdTranslator=Translator(lsb,0,8)
FixedData=80 00 8A D2 00
Code.HCS08=20 18 23 47 51 8B 45 07 08 01 13 03 39 01 13 01 13 4E 20 11 C6 08 CA FF FF 04 B6 65 62 B8 65 B8 60 B7 66 3C AC CC FF 5F 

I haven't learned to handle S3C8 assembler yet, so this is only HCS08. If this extension is of interest, perhaps someone can translate the code into S3C8 and add it.
_________________
Graham

[The Robman]

Do you still have the PB file that you used to create this? If so, it would be helpful in creating an S3C8 version.

Just FYI, I always use $01FF as the pid for my ad hoc executors, but I haven't come up a good suggestion as to what to do with them when you want to add them to protocols.ini. I don't like the idea of us using up lots of PIDs with these homemade executors as those PIDs will undoubtedly get used by UEI eventually for something else.

[mathdon]

I didn't use PB, I just modified Vicky's HCS08 code directly. But here's a listing, which should serve the same purpose.

Code: Select all

FF00:	20 18		BRA	$FF1A
FF02:	23 47		DB 	$23, $47	; 37.736kHz 33.0% duty cycle 
FF04:	51   		DB 	$51  		; 5 fixed, 1 variable
FF05:	8B 45		DB 	$8B, $45	; pf0 = 0001011, pf1 = 1000101 
FF07:	07 08		DB 	$07, $08	; pd00/01, dev bits = 7, cmd bits = 8
FF09:	01 13		DW 	$0113		; pd02/03, 1-Burst ON = 550u
FF0B:	03 39		DW 	$0339		; pd04/05, 1-Burst OFF = 1650u
FF0D:	01 13		DW 	$0113		; pd06/07, 0-Burst ON = 550u
FF0F:	01 13		DW 	$0113		; pd08/09, 0-Burst OFF = 550u
FF11:	4E 20		DW 	$4E20		; pd0A/0B, Leadout OFF = 40m
FF13:	11 C6		DW 	$11C6		; pd0C/0D, Leadin ON = 9100u
FF15:	08 CA		DW 	$08CA		; pd0E/0F, Leadin OFF =4500u
FF17:	FF FF		DW 	$FFFF		; pd10/11, unused
FF19:	04   		DB 	$04  		; pd12, CmdBits2 = 4
FF1A:	B6 65		LDA	DCBUF+5
FF1C:	62   		NSA
FF1D:	B8 65		EOR	DCBUF+5
FF1F:	B8 60		EOR	DCBUF
FF21:	B7 66		STA	DCBUF+6
FF23:	3C AC		INC	dev_byte_ndx
FF25:	CC FF 5F	JMP	IR_Gen

I don't think dev_byte_ndx has a PB assembler direct equivalent. In location, it is CBYTES+2 but I doubt if that translates into other processors. It is an index that specifies where, in DCBUF, to start the fixed byte coding. The INC instruction sets it to 1, so only the 4 bytes DCBUF+1 to DCBUF+4 are encoded by IR_Gen instead of the full 5 fixed bytes. I hope that's enough for you to know what the S3C8 equivalent is.

I've no suggestion about PIDs. We could call "ours" all different variants of 01FF, but that would cause problems to users who wanted to use more than one. I have to leave the choice of PID to you.
_____________
Graham

[vickyg2003]

I didn't use PB, I just modified Vicky's HCS08 code directly. But here's a listing, which should serve the same purpose.

You are missing out here Graham. PB is a very powerful tool. Its very worth exploring.

[mathdon]

I've realised that from looking at it, Vicky. But I wasn't starting from scratch, and I also have an interest in knowing what all those bytes at the beginning (pf's and pd's) actually DO. PB sets them, sure, but it leaves the relationship between the info you provide and the data in the executor rather obscure. I'm still on a learning curve and want to understand what PB actually does. Or, just possibly, doesn't.
____________
Graham

[The Robman]

... and I also have an interest in knowing what all those bytes at the beginning (pf's and pd's) actually DO.

read this...
https://www.hifi-remote.com/forums/dload ... le_id=2306

[mr_d_p_gumby]

... and I also have an interest in knowing what all those bytes at the beginning (pf's and pd's) actually DO.

read this...
https://www.hifi-remote.com/forums/dload ... le_id=2306

Also, if you unhide the processor sheets in PB (HCS08, S3, etc.), you'll have a better idea of how the PFx and PDxx bytes are set up.

Note that there are some differences between the IR engines (and their respective data structures) in the different processors. Each newer generation has added new capabilities. From oldest to newest, the versions would be:
6805C9
P8/740
6805RC16/18
S3C8/S3C8+
HCS08

The S3F8 IR engine is identical to the S3C8+, and does not include the newer features that were added to the HCS08 engine.

PB only knows what we know about the IR engine. I would estimate our knowledge of it's operation to be about 20-25% of it's total capabilities.