[RESOLVED] LF T55XX remake.

iceman · 2015-01-30 17:28:41

@Marshmellow,

I thought we can continure here instead.

The T55X7 can simulation a bunch of different modulations and encodings. So I thought of using your demodulation functions.
The flow is suppose to be like.
try fsk ( yes / no )
try fsk2 ( yes / no )
try psk1 (yes / no )
try psk2 (yes / no )

When the first that works (ie yes) then it tries to decode the signal and then present the result.

Does your demodulation functions support the following?

Demodulate:
FSK1, FSK2, PSK1, PSK2, BIPHASE(?)

Decode:
Manchester | NZR

Last edited by iceman (2015-03-24 22:11:56)

marshmellow · 2015-01-30 18:12:28

so the ATA55x7s can emulate:

Modulation:
ASK
FSK1
FSK2
PSK1
PSK2
PSK3
NRZ/Direct

Encodings:
Manchester
Biphase

we currently have no function to detect the modulation used. and the demod functions used on the wrong modulated wave may return garbage binary.
we can discuss ways of detecting this later...

Each of those modulations and encodings can have various clocks and different settings that can change how to demod it. we have fairly accurate detect clock functions now (again psk needs work) so we should no longer need that information to demod (although an option to override the clock used to demod is sometimes useful so it is still an option in the demod functions)

you can attempt to demod a known modulation with the new code in 2 ways, if you need little control over the input and the output, and your raw wave is in the graphbuffer then you can call the cmddata.c Cmd functions just as if you were typing the command in the command prompt. (for fskrawdemod use CmdFSKrawdemod)

if however you want to manipulate the data before or after the demod or already have a uint_8 byteArray of samples not in graphbuffer then I'd call the functions you need from lfdemod.c (can use the cmddata.c functions as a guide).

modulation specific - lfdemod.c:
FSK1 and FSK2 can both be demodulated with fskdemod, you can auto detect the two if you use the detectFSKClk function to set the arguments for the fskdemod function. (see CmdFSKrawdemod for an example)
NOTE: FSK1a and FSK2a are the same as FSK1 and FSK2 respectively except they are inverted. (we cannot auto detect the invert.)

ASK with Manchester can be demoded most accurately with the askmandemod function (use DetectAskClock to set clock settings)
ASK with Biphase is a work in progress but works most of the time, use askrawdemod then BiphaseRawDecode. (no good way to detect biphase vs manchester in code yet)

PSK1 can be demoded with pskNRZrawDemod
PSK2 can be demoded with pskNRZrawDemod then psk1TOpsk2

PSK3 is a work in process No current solution.

NRZ is also a work in process but usually works with the PSK1 demod

Last edited by marshmellow (2015-01-31 01:30:05)

marshmellow · 2015-01-31 01:40:16

does anyone have any ideas on how to autodetect the modulation type?
FSK should be fairly easy with the detectFSKClk if it doesn't return 10/8 or 8/5 for field clocks then it isn't a known fsk. (shouldn't have too many false positives)

maybe we could adjust the error tolerance for askman (have it an argument) so we can reject it if there are any errors.

biphase.. ?

psk, maybe if we count the largest distance between phase shifts and if it is > the largest bit clock it is either PSK or NRZ... ??

Last edited by marshmellow (2015-01-31 01:41:25)

marshmellow · 2015-02-01 05:30:10

hold off on the PSK side of things i am close to a rework that i like.

iceman · 2015-02-01 13:36:30

Good question, how to detect a modulation type.?

When you look at a trace, it's kind of clear the difference but how to put that down to rules that is implementable in code...
I was thinking either apply some filters on the trace signal and determine some patterns to decide the modulation.
Or clock-detection since its different ways of calculating it for fsk/psk..

marshmellow · 2015-02-01 20:28:50

I think relying solely on clock detection would not be very reliable. FSK might be the exception, as you would rarely ever have a false positive using fsk field clocks. ... So unless someone can use a type of filter or pattern recognition, I'd lean towards try FSK, if FC's don't look right go to askman by default. Then take arguments for the rest (biphase, psk1, psk2, nrz)

asper · 2015-02-01 21:54:58

Just a question: looking at an enough long trace is it always possible to detect the modulation? If the answer is yes, why don't we analze a "graphic outupt" like a .png image obtained from the trace? We can "normalize" the data to a certain standard zoom/window level and then analyze it...

asper · 2015-02-01 22:01:17

Alternatively this can be useful: http://en.m.wikipedia.org/wiki/Detector_(radio)

iceman · 2015-02-02 11:31:53

hm, we can detect Manchester encoding, since the bytes is either 01 or 10.. any 11 or 00 is wrong.
the question is if manchester is always used. If we demodul fsk/psk and see if we get valid manchest from it,...

marshmellow · 2015-02-02 14:43:08

Yes, but we would need to edit askman to allow fewer errors. (Pass a new argument for it.).

iceman · 2015-02-02 15:45:56

in that case, have an option only to accept zero errors. Is that whay you meant?

marshmellow · 2015-02-02 15:47:48

Basically.

marshmellow · 2015-02-02 15:51:03

I've got psk1 and 2 done (not pushed yet) but I had to split out nrz (direct) and I'm trying to make nrz demod more robust now.

marshmellow · 2015-02-02 15:55:31

In doing the new psk, I made a new function to detect the psk carrier length. So we could check if that comes back with 2 or 4 then it is psk, if it comes back with 8 then it might be psk or ask RF/8 or FSK...

iceman · 2015-02-02 16:04:40

I agree with the idea of separating the demodulation from the decoding. Maybe getting a struct back out with clock, bits,bitslen, numoferrors

A "IsManchester" function is easier, loop the bits pair, make sure all of them are 01 | 10.
but what is the definition for nrz? http://en.wikipedia.org/wiki/Return-to-zero

marshmellow · 2015-02-02 16:18:58

Nrz is another modulation. It works similar to psk1 if you look at peaks, but doesn't have the carrier wave.

iceman · 2015-02-02 17:04:06

So, if all modulations work, loop and test if it is manchester?..

And if it is not manchester encoded? what to do then?

marshmellow · 2015-02-02 17:09:17

As far as the ata55x7 read block is concerned, Manchester is only used with ask. (Same with biphase)... I'll lay out what I think is possible to do without a new wave filter/pattern recognition algo.

marshmellow · 2015-02-03 00:56:15

The functions as they are will have many false positives. FSK is the only one that seems unlikely to think it has FSK data when it doesn't... I am seeing what I can do to reduce the false positives, but I'm not sure if I'll get it good enough. I think I should be able to get ask with Manchester encoding pretty accurately, but psk1 2 and 3 are the same wave so there is no way to detect the difference, and nrz is difficult to distinguish from psk. But then my math is weak in this area...

asper · 2015-02-03 10:47:17

Those are papers descrbing how to automatically detect a modulation signal; they seems to be based on statistical patterns. You can read them here.

marshmellow · 2015-02-04 05:56:16

sorry but that is all above me...

marshmellow · 2015-02-06 21:50:50

@iceman, best i can suggest right now is run fskraw, askman, psk1 if they return < 1 then it failed try the next, and run them in that order will yield best results with few false results. it should also get 90% of the tags out there.

for the rest (biphase, psk2, psk3, or nrz/direct) i would recommend an argument to specify which we expect it to be and try only that one (or leave the data in the graphbuffer to allow the user to try for themselves).

marshmellow · 2015-02-06 21:51:58

in my recent pull request i implemented an argument for lf search to search for unknown tags that shows how to do what i recommended above.

marshmellow · 2015-02-07 06:24:17

Other readers that handle the ata55x7s require you to set the an expected answer mode prior to reading. they don't attempt to auto detect.
I think we can do a little better than that, but I don't think it is worth building a lot of special filters for.

Last edited by marshmellow (2015-02-07 06:25:43)

iceman · 2015-02-09 10:07:20

I think what with your new cleaner api's, it will be enough. It sure will be much better than what we have at the moment.

iceman · 2015-02-20 18:26:32

Hey Marshmellow,
is there any chance we can add a print the bitstream optionally or like a "verbose" parameter? or g_debug =true ??

I get all these bitstreams printouts and clock/fc outputs when using your demods. Mostly the bitstream which is kind of too much. Or what do you suggest?

pm3 --> lf t55xx read 0
Args invert: 0 - Clock:128 - fchigh:64 - fclow: 5
FSK decoded bitstream:
1000000100000000
0000000000010000
0000010000000000
0000000000000000
0000000000000000
 Decoded     : 0x02000020  00000010000000000000000000100000
Args invert: 1 - Clock:128 - fchigh:64 - fclow: 5
FSK decoded bitstream:
0111111011111111
1111111111101111
1111101111111111
1111111111111111
1111111111111111
 Decoded     : 0xFDFFFFDF  11111101111111111111111111011111

Using Clock: 64 - Invert: 0 - Bits Found: 187
ASK/Manchester decoded bitstream:
0000000000001010
0100000000100000
0000000000001010
0100000000100000
0000000000001010
0100000000100000
0000000000001010
0100000000100000
0000000000001010
0100000000100000
0000000000001010
 Decoded     : 0x00148040  00000000000101001000000001000000
Using Clock: 64 - invert: 0 - Bits Found: 374
ASK demoded bitstream:
0101010101010101
0101010011001101
0011010101010101
0100110101010101
0101010101010101
0101010011001101
0011010101010101
0100110101010101
0101010101010101
0101010011001101
0011010101010101
0100110101010101
0101010101010101
0101010011001101
0011010101010101
0100110101010101
0101010101010101
0101010011001101
0011010101010101
0100110101010101
0101010101010101
0101010011001101
0011010101010101
 Decoded     : 0xAAAAA99A  10101010101010101010100110011010
Tried NRZ Demod using Clock: 32 - invert: 0 - Bits Found: 374
NRZ demoded bitstream:
1010101010101010
1010101100110010
1100101010101010
1011001010101010
1010101010101010
1010101100110010
1100101010101010
1011001010101010
1010101010101010
1010101100110010
1100101010101010
1011001010101010
1010101010101010
1010101100110010
1100101010101010
1011001010101010
1010101010101010
1010101100110010
1100101010101010
1011001010101010
1010101010101010
1010101100110010
1100101010101010
 Decoded     : 0x55555665  01010101010101010101011001100101
pm3 -->

marshmellow · 2015-02-21 19:52:04

I'm working on breaking out some functions that should help.

askman
askraw
fskraw
pskraw

Are completed in my fork. Still have a little to go before pushing it to the main.

Last edited by marshmellow (2015-02-21 22:06:54)

iceman · 2015-02-21 21:51:48

I saw it, thanks man!
just the NZR left

marshmellow · 2015-02-22 01:28:47

NRZ is now done in my fork/master

Last edited by marshmellow (2015-02-22 01:28:59)

iceman · 2015-02-22 17:58:40

Great!

pm3 ~/client$ proxmark3.exe com3
pm3 --> lf t55 read 0
0x02400102  00000010010000000000000100000010 [FSK]
0xFDBFFEFD  11111101101111111111111011111101 [FSK inv]
0x00148040  00000000000101001000000001000000 [ASK/MAN]
0xFFEB7FBF  11111111111010110111111110111111 [ASK/MAN Inv]
0x80000076  10000000000000000000000001110110 [NZR]
0x7FFFFF89  01111111111111111111111110001001 [NZR inv]
0x7FFFFF89  01111111111111111111111110001001 [PSK]
0x7FFFFF89  01111111111111111111111110001001 [PSK inv]
pm3 -->

marshmellow · 2015-02-22 20:59:40

Are you outputting data even if the demod function doesn't return a 1?

iceman · 2015-02-22 21:23:02

sample code, it should only output if it returns 1 (true). Or did I miss something?

if ( FSKrawDemod("", FALSE) ){
		printT55xx("FSK");
	}
	// FSK inverted
	if ( FSKrawDemod("1", FALSE)) { //invert?
		printT55xx("FSK inv");
	}

marshmellow · 2015-02-22 22:01:42

that should be correct.. for FSK. i looked again and if you are using PSKDemod then your test should be
if (PSKDemod("", FALSE)>=0) {

as pskdemod is built to pass back the number of errors, if there is an error it will return a -1. you could also do
if (PSKDemod("", FALSE)==0) {

if you don't want to allow any errors during demod. (i'd recommend this.)

iceman · 2015-02-22 23:00:09

Don't know about errors in that PSK case, I tried to specify it with the input

	// FSK
	if ( FSKrawDemod("", FALSE) ){
		printT55xx("FSK");
	}
	// FSK inverted
	if ( FSKrawDemod("1", FALSE)) {
		printT55xx("FSK inv");
	}

	// ASK/MAN (autoclock, normal, maxerrors 1)
	if ( ASKmanDemod("", FALSE, FALSE) ) {
		printT55xx("ASK/MAN");
	}

	// ASK/MAN (autoclock, inverted, maxerrors 1)
	if ( ASKmanDemod("0 1 1", FALSE, FALSE) ) {
		printT55xx("ASK/MAN Inv");
	}

	// NZR (autoclock, normal, maxerrors 1)
	if  ( NRZrawDemod("0 0 1", FALSE) ) {
		printT55xx("NZR");
	}
	// NZR (autoclock, inverted, maxerrors 1)
	if  ( NRZrawDemod("0 1 1", FALSE) ) {
		printT55xx("NZR inv");
	}
	
	// PSK (autoclock, normal, maxerrors 1)
	if (PSKDemod("0 0 1", FALSE)) {
		printT55xx("PSK");
	}
	// PSK (autoclock, inverted, maxerrors 1)
	if (PSKDemod("0 1 1", FALSE) == ) {
		printT55xx("PSK inv");
	}

marshmellow · 2015-02-22 23:25:02

Your current code will output psk demod only if it fails.

Change it to

	if (!PSKDemod("0 0 1", FALSE)) {
		printT55xx("PSK");
	}

marshmellow · 2015-02-22 23:26:55

Same for the invert psk code

iceman · 2015-02-22 23:36:45

With your suggest fix for PSK, the output is a bit less. The T55x5 is hard in that way since it can emulate so different modulations. It's a guessing game. I was thinking of either a config setting for t55xx and/or some "standard" config blocks to identify which modulation was correct...

pm3 --> lf t55 re 0
0x80040000  10000000000001000000000000000000 [FSK]
0x7FFBFFFF  01111111111110111111111111111111 [FSK inv]
0x00148040  00000000000101001000000001000000 [ASK/MAN]  --<< this is correct
0xFFEB7FBF  11111111111010110111111110111111 [ASK/MAN Inv]
0x80000076  10000000000000000000000001110110 [NZR]
0x7FFFFF89  01111111111111111111111110001001 [NZR inv]

marshmellow · 2015-02-22 23:59:57

We can do some more tests for FSK to remove false positives. I'll take a look.

Same with askman

marshmellow · 2015-02-23 02:31:41

for FSK, you can do the following before demoding:

    if (CmdDetectClockRate("f")){ //wave is almost certainly FSK
      //call FSK DEMOD
    } else {
      //Try other DEMODs
    }

marshmellow · 2015-02-23 02:39:20

also the askman test you showed above did not include the max error arguments "0 0 1".
(the inverted askman did)

marshmellow · 2015-02-23 03:05:22

in my fork i've added a test to PSKDemod to weed out invalid carrier clocks. should reduce false positives further. (but i'm not done with lf sim commands)

i think it would make sense for invert to be an option when sending the lf t55 read command.
we also may want to have a big endian option.

iceman · 2015-02-23 09:50:52

Inspired with Holimans LF config, I started to add the same functionality to t55xx, right this moment it's just keeping track of a selected modulation and to inverse the signal data.

I think a detection based on the "decoded" config block, and which modulation was used, will give a good indicator if the data is a good demodulation.

So many ideas, so much to do, so little time, however I like your new sim! Great work!

marshmellow · 2015-03-10 02:55:25

The changes I said were in my fork are now also in the master trunk.

iceman · 2015-03-10 09:59:50

I noticed it!

marshmellow · 2015-03-24 12:24:39

Now t55xx commands are in the main trunk. Thanks iceman.

iceman · 2015-03-24 12:43:50

Thanks to you, bro. Without your demods and remake of the LF/DATA commands this wouldn't been done.

app_o1 · 2015-03-31 12:12:45

I have successfully compiled latest github main trunk code and flashed my PM3. v2.0.0-8-g616970b

lf t5xx trace
lf t5xx read x
lf t5xx info
lf t5xx dump

Those commands do not always show the decoded bits.
But it does show the results in the graph buffer.
Is it normal? Do I have to decode it manually?

# LF antenna: 11.41 V @ 125.00 kHz

iceman · 2015-03-31 12:18:14

you missed a piece of information..

The new t55xx commands works on a configuration setting now.

lf t55xx detect

// if it could detect the signal it will autoselect it, 
// and you can now run the other new commands
// if it failed..
// then you will need to configure the t55xx manually by  "lf t55xx config"
// all new commands should have been implemented with the help parameter  h
//

Always start with a "lf t55xx detect".

lf t55xx detect
lf t55xx trace
lf t55xx info
lf t55xx read nn

Last edited by iceman (2015-03-31 12:19:33)

app_o1 · 2015-03-31 13:35:28

Thanks for the explanation.
This is sick! Awesome job guys!

iceman · 2015-03-31 13:37:45

Yeah, the t55xx commands are kind of good now. At least better than before. Both @marshmellow and I are quit satisified with how they turned out. Not perfect, but at least better.

Proxmark3 community

Announcement

#1 2015-01-30 17:28:41

[RESOLVED] LF T55XX remake.

#2 2015-01-30 18:12:28

Re: [RESOLVED] LF T55XX remake.

#3 2015-01-31 01:40:16

Re: [RESOLVED] LF T55XX remake.

#4 2015-02-01 05:30:10

Re: [RESOLVED] LF T55XX remake.

#5 2015-02-01 13:36:30

Re: [RESOLVED] LF T55XX remake.

#6 2015-02-01 20:28:50

Re: [RESOLVED] LF T55XX remake.

#7 2015-02-01 21:54:58

Re: [RESOLVED] LF T55XX remake.

#8 2015-02-01 22:01:17

Re: [RESOLVED] LF T55XX remake.

#9 2015-02-02 11:31:53

Re: [RESOLVED] LF T55XX remake.

#10 2015-02-02 14:43:08

Re: [RESOLVED] LF T55XX remake.

#11 2015-02-02 15:45:56

Re: [RESOLVED] LF T55XX remake.

#12 2015-02-02 15:47:48

Re: [RESOLVED] LF T55XX remake.

#13 2015-02-02 15:51:03

Re: [RESOLVED] LF T55XX remake.

#14 2015-02-02 15:55:31

Re: [RESOLVED] LF T55XX remake.

#15 2015-02-02 16:04:40

Re: [RESOLVED] LF T55XX remake.

#16 2015-02-02 16:18:58

Re: [RESOLVED] LF T55XX remake.

#17 2015-02-02 17:04:06

Re: [RESOLVED] LF T55XX remake.

#18 2015-02-02 17:09:17

Re: [RESOLVED] LF T55XX remake.

#19 2015-02-03 00:56:15

Re: [RESOLVED] LF T55XX remake.

#20 2015-02-03 10:47:17

Re: [RESOLVED] LF T55XX remake.

#21 2015-02-04 05:56:16

Re: [RESOLVED] LF T55XX remake.

#22 2015-02-06 21:50:50

Re: [RESOLVED] LF T55XX remake.

#23 2015-02-06 21:51:58

Re: [RESOLVED] LF T55XX remake.

#24 2015-02-07 06:24:17

Re: [RESOLVED] LF T55XX remake.

#25 2015-02-09 10:07:20

Re: [RESOLVED] LF T55XX remake.

#26 2015-02-20 18:26:32

Re: [RESOLVED] LF T55XX remake.

#27 2015-02-21 19:52:04

Re: [RESOLVED] LF T55XX remake.

#28 2015-02-21 21:51:48

Re: [RESOLVED] LF T55XX remake.

#29 2015-02-22 01:28:47

Re: [RESOLVED] LF T55XX remake.

#30 2015-02-22 17:58:40

Re: [RESOLVED] LF T55XX remake.

#31 2015-02-22 20:59:40

Re: [RESOLVED] LF T55XX remake.

#32 2015-02-22 21:23:02

Re: [RESOLVED] LF T55XX remake.

#33 2015-02-22 22:01:42

Re: [RESOLVED] LF T55XX remake.

#34 2015-02-22 23:00:09

Re: [RESOLVED] LF T55XX remake.

#35 2015-02-22 23:25:02

Re: [RESOLVED] LF T55XX remake.

#36 2015-02-22 23:26:55

Re: [RESOLVED] LF T55XX remake.

#37 2015-02-22 23:36:45

Re: [RESOLVED] LF T55XX remake.

#38 2015-02-22 23:59:57

Re: [RESOLVED] LF T55XX remake.

#39 2015-02-23 02:31:41

Re: [RESOLVED] LF T55XX remake.