Proxmark3 community

Enio

Contributor

Registered: 2013-09-24

Posts: 175

LF tag Manchester demodulation advice

Greetings.

[For anyone having troubles with errors on manchester decoding my patch later in this post might be helpful.]

I have troubles demodulating a tag, most likely a em4100 wave.

I usually did

lf read
data samples ...
data hpf
data threshold 0 (askdemod here just results in a 0 line)
data askdemod 0
data mandemod

I get a good looking Graph, but lots of errors on data mandemod and no results. I think its because of the shape of the wave, with threshold 0 i dont get distinctive points where the flanks start as its often a soft courve.

To overcome this i implemented a simple directional threshold function, which sets rising samples over a threshold to 1 and falling samples under a threshold to -1, the rest in between keep the value of the previous sample.

It works like a charm for me, as i can specify the middle of the steep flanks of the wave to correlate to resulting modulated flanks.
Patch:

Index: cmddata.c
===================================================================
--- cmddata.c	(Revision 848)
+++ cmddata.c	(Arbeitskopie)
@@ -17,6 +17,7 @@
 #include "ui.h"
 #include "graph.h"
 #include "cmdparser.h"
+#include "util.h"
 #include "cmdmain.h"
 #include "cmddata.h"
 
@@ -818,6 +819,41 @@
   return 0;
 }
 
+int CmdDirectionalThreshold(const char *Cmd)
+{
+	int8_t upThres = param_get8(Cmd, 0);
+	int8_t downThres = param_get8(Cmd, 1);
+  
+  printf("Applying Up Threshold: %d, Down Threshold: %d\n", upThres, downThres);
+  
+  int lastValue = GraphBuffer[0];
+  GraphBuffer[0] = 0; // Will be changed at the end, but init 0 as we adjust to last samples value if no threshold kicks in.
+  
+  for (int i = 1; i < GraphTraceLen; ++i) {
+    // Apply first threshold to samples heading up
+    if (GraphBuffer[i] >= upThres && GraphBuffer[i] > lastValue)
+    {
+      lastValue = GraphBuffer[i]; // Buffer last value as we overwrite it.
+      GraphBuffer[i] = 1;
+    }
+    // Apply second threshold to samples heading down
+    else if (GraphBuffer[i] <= downThres && GraphBuffer[i] < lastValue)
+    {
+      lastValue = GraphBuffer[i]; // Buffer last value as we overwrite it.
+      GraphBuffer[i] = -1;
+    }
+    else
+    {
+      lastValue = GraphBuffer[i]; // Buffer last value as we overwrite it.
+      GraphBuffer[i] = GraphBuffer[i-1];
+
+    }
+  }
+  GraphBuffer[0] = GraphBuffer[1]; // Aline with first edited sample.
+  RepaintGraphWindow();
+  return 0;
+}
+
 int CmdZerocrossings(const char *Cmd)
 {
   // Zero-crossings aren't meaningful unless the signal is zero-mean.
@@ -874,6 +910,7 @@
   {"scale",         CmdScale,           1, "<int> -- Set cursor display scale"},
   {"threshold",     CmdThreshold,       1, "<threshold> -- Maximize/minimize every value in the graph window depending on threshold"},
   {"zerocrossings", CmdZerocrossings,   1, "Count time between zero-crossings"},
+  {"dirthreshold",  CmdDirectionalThreshold,   1, "<thres up> <thres down> -- Max rising higher up-thres/ Min falling lower down-thres, keep rest as prev."},
   {NULL, NULL, 0, NULL}
 };
 
Index: cmddata.h
===================================================================
--- cmddata.h	(Revision 848)
+++ cmddata.h	(Arbeitskopie)
@@ -36,6 +36,7 @@
 int CmdSave(const char *Cmd);
 int CmdScale(const char *Cmd);
 int CmdThreshold(const char *Cmd);
+int CmdDirectionalThreshold(const char *Cmd);
 int CmdZerocrossings(const char *Cmd);
 
 #endif

As you can see there, i just do

data hpf
data dirthreshold <upThres> <downThres> (Choose good values, i chose those that are in the steep flanks..)
data askdemod 1
data mandemod

asper

Contributor

Registered: 2008-08-24

Posts: 1,409

Re: LF tag Manchester demodulation advice

I have trouble too using mandemod and also I found problems with autocorr command. I am not an LF expert and following this wiki tutorial I found out that autocorr plot a really unuseful graph (very small/weak waves after applying autocorr)... Enio can you please verify it and if possible update the wiki guide with your suggestions ? Thank you !

Last edited by asper (2014-03-29 20:13:34)

Enio

Contributor

Registered: 2013-09-24

Posts: 175

Re: LF tag Manchester demodulation advice

The graph you get off autocorr is only useful to find the length of unique data. Basically to find out how many samples are sent before they repeat.

Also, mandemod does weird things. It sometimes works for me on the original wave, but not on if i manually shape that same wave into a 0/1 square wave to make it easy to manually demod. In that state it throws errors, even with giving the right clock, while i easily can demodulate it just by lookin at it!

Its ony my plan to update the wiki and also to add the functionality of the directional threshold thing. Mandemod could also need a look at, to make it more robust (also on my todo list)..

Best!

holiman

Contributor

Registered: 2013-05-03

Posts: 566

Re: LF tag Manchester demodulation advice

Could you send/post some samples where the mandemod fails? I'm looking into it a bit...

Enio

Contributor

Registered: 2013-09-24

Posts: 175

Re: LF tag Manchester demodulation advice

Could you send/post some samples where the mandemod fails? I'm looking into it a bit...

When i looked more into it, it turned out that the issue is that not the whole samples were pure manchester, there were start and end modulations messing the mandemod function up (I read the configuration block of a t5577).

It would be great if a future mandemod could skip those "undecodable" parts and extract data where it can. It might not be trivial though.

holiman

Contributor

Registered: 2013-05-03

Posts: 566

Re: LF tag Manchester demodulation advice

It would be great if a future mandemod could skip those "undecodable" parts and extract data where it can. It might not be trivial though.

Well, if you send me some samples where that happens I'll see what I can do... 'data save mandemod_problematic', you could either email it to me (martin at swende dot se) or post it as attached file to an issue on github.

Even better would be to have a whole bunch, who knows, maybe we can add the first unit-test

holiman

Contributor

Registered: 2013-05-03

Posts: 566

Re: LF tag Manchester demodulation advice

I believe it's not very hard to detect start- and end modulations. Those are very easily identifiable, afaik

Enio

Contributor

Registered: 2013-09-24

Posts: 175

Re: LF tag Manchester demodulation advice

I believe it's not very hard to detect start- and end modulations. Those are very easily identifiable, afaik

True, if you know what kind of "markers" you are looking for - We might not know it, it would be cool to still have options to extract Manchester modulated parts of a data stream. We might need additional options to specify kind of the "exactness requirement" of the Manchester part - So we can tune it to be strict on the clock or more tolerant with deviations.

hamid

Contributor

Registered: 2014-04-15

Posts: 10

Re: LF tag Manchester demodulation advice

It would be great if a future mandemod could skip those "undecodable" parts and extract data where it can. It might not be trivial though.

Well, if you send me some samples where that happens I'll see what I can do... 'data save mandemod_problematic', you could either email it to me (martin at swende dot se) or post it as attached file to an issue on github.

Even better would be to have a whole bunch, who knows, maybe we can add the first unit-test

I had the same problem with decoding manchester , and the solution provided in this threat worked.
if you want more bogus samples, I`ve provided mine here:

http://www.proxmark.org/forum/viewtopic.php?id=2036

And thank you Enio for your patch! It rescued me from hours of head scratching, trying to figure out where the problem is.

Enio

Contributor

Registered: 2013-09-24

Posts: 175

Re: LF tag Manchester demodulation advice

And thank you Enio for your patch! It rescued me from hours of head scratching, trying to figure out where the problem is.

Welcome, glad it helped.

I might mention here that the version i put up here http://www.proxmark.org/forum/viewtopic … 473#p11473 includes the directional threshold function.

Best, Enio

hamid

Contributor

Registered: 2014-04-15

Posts: 10

Re: LF tag Manchester demodulation advice

Wondering why this dirthershold patch is not already commited to repo?

Btw, I checked your github and seems you`re doing something interesting there! Let me know if you need some testings for your generic snoop, specially in LF.

holiman

Contributor

Registered: 2013-05-03

Posts: 566

Re: LF tag Manchester demodulation advice

I agree, we should put this into the repo. I wanted to fix a more robust manchester decoder, but then other things got in the way.. I'll commit the patch as is.

holiman

Contributor

Registered: 2013-05-03

Posts: 566

Re: LF tag Manchester demodulation advice

Committed : https://github.com/Proxmark/proxmark3/commit/d51b2eda8f91b17dd02cdbd931b089fc8f8d61db

eskizle

Contributor

Registered: 2011-07-18

Posts: 26

Re: LF tag Manchester demodulation advice

Enio,

how do you identify the modulation on the waves

data askdemod 0

then

data mandemod

For me there was always a modulation (ask, fsk,psk) then an encoding (manchester or other).. But from the waves I have been told i have to "mandemod" directly... I'd like to understand why.

Regards

Enio

Contributor

Registered: 2013-09-24

Posts: 175

Re: LF tag Manchester demodulation advice

Cool, thank you holiman.

@eskizle: tbh, i think i just used it to cut the samples < 0 to 0.

Im not sure exactly.

marshmellow

Contributor

From: US

Registered: 2013-06-10

Posts: 2,302

Re: LF tag Manchester demodulation advice

For me there was always a modulation (ask, fsk,psk) then an encoding (manchester or other).. But from the waves I have been told i have to "mandemod" directly... I'd like to understand why.

Regards

The answer is that mandemod command has a built in check to see if it is dealing with demodulated 1s and 0s or a raw wave, and then ask demos it before decoding it. 

However it is not very accurate and fails on several types of tags.  I've built new commands which significantly improve the accuracy of demodulating ASK and decoding Manchester. 

These should be added to the master repository soon.

See this thread for details.
http://www.proxmark.org/forum/viewtopic.php?id=2216

marshmellow

Contributor

From: US

Registered: 2013-06-10

Posts: 2,302

Re: LF tag Manchester demodulation advice

...

Last edited by marshmellow (2015-01-03 05:03:37)

iceman

Administrator

Registered: 2013-04-25

Posts: 9,538

Website

Re: LF tag Manchester demodulation advice

Yeah, 
I would like to have seen a  decoupling between  demodulation and decoding in the source.  I think it would help understanding a new tag.

You have the:   Raw signal -> demodulated signal -> decoded signal.

We wouldn't have so many methods dedicated to one specific tag where it is intermixed and kind of hard to understand the different solutions. This would help out to make it easier to write new decode methods for tags
We would get better code reuse and a more rapid development.

marshmellow

Contributor

From: US

Registered: 2013-06-10

Posts: 2,302

Re: LF tag Manchester demodulation advice

We are a little better off now with the new commands, but I do see some benefit demodulating and decoding at the same time as it helps to identify the correct decode when you still have the full wave available.  ( note that I created an askrawdemod, but also a ask mandemod that is a little more accurate than askraw-manraw separately. As it can test different demod start positions for the best decoding result.