[closed] Proxmark3 device reconnecting from Offline mode

iceman · 2017-09-26 15:48:46

Let me show you some sweet feature I've been working on for some days.

The problem, once in standalone mode with a battery, there is no way today to reconnect to the pm3 client.
All data or tracelogs gathered with in standalone mode is gone.
This is based on the fact that all standalone modes has its own major loop. The device never leaves it.

Well, partly true, there is now the usb_poll detection but since the device is in standalone mode it stays there.

My idea is to start to overcome this flaw with proxmark3.

https://youtu.be/jeGsIi1UpIU

iceman · 2017-09-29 18:10:41

Yes!
I have successfully disconnected the usb cable on a battery driving PM3 in standalone mode AND reconnected the USB cable without restarting the client OR reseting the PM3 device

Now I need to verify that the collected data is still there... If that works, this will enable much funnier standalone modes!

Dmanufacturer · 2017-09-30 02:25:19

Good work, nicely done.

iceman · 2017-09-30 09:04:12

Sadly it is not very stable yet.

hexa3e8 · 2017-10-01 20:13:01

This would indeed be a very usefull feature to save the gathered info by reconnecting. makes everything much easier to carry along. Keep up the great work!

iceman · 2017-10-15 22:31:03

I've written a bit everywhere on this matter and the idea of not needing a inf-driver (unsigned) the last days.

A bit of a conondrum, when you start looking at it in depth. I try to summerize my problems.
The proxmark3 uses USB and it uses CDC ACM protocol over the usb channel in order to become a virtual serial port (COM). The serial port interface is where we normally talk about the pm3 client sending a usbcommand to the device and back.

The CDC ACM is what enables a operative system like windows to assign a virtual serial port over USB. Quite nifty I say. The benifits are good transfer speeds and interoperatbility between OS'es.

PM3 hardware:
The MCU, at91sam7s256,  has built-in support for usb,  and the spec's states 4 endpoints.   This is USB endpoints and its data frame size limits

AT91SAM7S256  USB Device Port
• Embedded 328-byte dual-port RAM for endpoints
• Four endpoints 

A) Endpoint 0: 8 bytes
B) Endpoint 1 and 2: 64 bytes ping-pong
C) Endpoint 3: 64 bytes
D) Ping-pong Mode (two memory banks) for bulk endpoints

The source code for the device to deal with USB and CDC_ACM is found here /common/usb_cdc.c https://github.com/Proxmark/proxmark3/b … /usb_cdc.c

[sidenote]
The comport / serial communication is dealt nowdays in a UART, which is located in /uart/uart_win32.c.
Its usually just reading / writing to the serial port handler.

Problems:
=========
To be able to install a proxmark3 you are needed to us an unsigned driver, since the proxmark3 project can't afford to pay for a signed one. To cut it short, its lots of money to get it. Is there any other way around it? Yes, there are. Microsoft decided since win8 to have an extra feature which is triggerd when enumerating a USB device the first time. If done according to their secret handshake the device will show up in the devicemanager as a WINUSB device without the need to install a driver. The keyword here is MS compatability id and MS OS Feature descriptors.

Now, we don't use WINUSB, we use CDC ACM. So, are we lost again? Not according to internet posts, there is a possibilty to use Microsoft OS feature descriptors to enable this USB enumeration and hook it up as a virtual comport and get a com assigned by OS which is exactly what we want to achieve.

So I have done the WINUSB, but my win10 doesn't want to make the device a virtual comport (CDC ACM). This is where I'm stuck.
As I mentioned first of this lengthy post, the current source seems to have some bugs.

The CDC ACM description requires the following setup:
1. control interface
1a. endpoint
2. data interface
2a. bulk out
2b. bulk in

The USB need a control interface / endpoint. Since this one is curcial to USB specs, the endpoint0 is designated to this.

Which leads my understanding to the following layout:

======
 device   | usb cdc                        |
 endpoint | endpoint                       | datasize
 0        |  ???                           |  8
 1        |  EP1_OUT bulk CDC data         |  64
 2        |  EP2_IN bulk  CDC data         |  64
 3        |  EP3_IN interupt CDC Control   |  64

The USB endpoint0 / control, is dealt with in function AT91F_CDC_Enumerate().
It reads the control endpoint and responses accordingly to USB specs.

however it also is configured to deal with CDC ACM commands.
But that should be read over the endpoint3 or?!? This is where I get really confused.
Since the CDC ACM setup is not working properly, how did it manage to work like it is today?
Or are we not reading CDC ACM commands because we are not dealing with them properly?

And since the USB endpoint0 is 8 bytes buffer while CDC ACM is 64bytes, the endpoint3 must be configured to 8bytes aswell?

piwi · 2017-10-16 01:14:09

Why do you want to use winusb.sys? Does it provide additional features compared to usbser.sys? Afaik since Windows 8.1 we don't need to install unsigned drivers because Windows detects PM3 as CDC Device and automatically uses usbser.inf and usbser.sys. https://msdn.microsoft.com/en-us/librar … s.85).aspx

iceman · 2017-10-16 08:45:44

Indeed, I don't want to use the winusb.sys. It is unclear in my previous posts but the tricking with feature descriptors and deviceinterfaceguid should make windows understand that we want usbser.sys instead. The reason is ms os feature descriptors works on OS as old as XP sp2.. It should work nice on Win7, which is what I have installed on some virtual machines. Within some few years when usage of win7 diminish this will not be a problem anymore.

However, I have always had the strange behavior when entering flash-mode, its like the device doesn't get identified correct, so my reason for going down this hole of usb, cdc acm, specifications and pm3 implementation more than just "driverless" installing.

This is my way of sharing what I stumble upon and try to figure out what is what.

iceman · 2017-10-16 08:54:14

About your link,

when I read it now, I understand exactly where and how to trigger that part. But I don't know what triggers some other parts.
And when I look at the current descriptors and the code in enumerate function it doesn't add up.
Windows doesn't do what I expect it to do.
like for instance, cdc acm v1.20 spec states it needs a IAD-descriptor in the configurationDescriptor, our PM3 code doesn't have that.
SO is our implementation faulty, we don't follow spec? or is things optional for different USB versions, (v10,v1.1, v2.0, v2.x, v3..) ?

Can we automatically configure windows to no suspend our usb device as mentioned in your link? that answer is yes, we don't need a driver for that, we can use ms os feature descriptors to add keys to that part in the registry. cool indeed.

It comes down to questions like, How do we get the PM3 to become more stable? For me there is a difference between normal operations and when entering bootrom-flashmode. why? it uses the same code. Could it be timings are wrong? Could it be wrong answer to usb packages? Could it be wrong descriptors usages?

If you have the answers to it, do please share and enlighten us.

iceman · 2017-10-16 18:04:17

Another strange thing I noticed today when sniffing the USB communication between host and device.
We have configured the BULK endpoints to have the size 64bytes (0x40) because of the MCU specification.
It means our forced limit on Usbpackages to 512 databytes is riddicul. We can set larger if needed. The USB hardware splits up the packages anyway. This is a good thing to know.

But what I found today is that BULK OUT transfers is of size 544. Ie, the exact size of UsbCommand struct. All well.
but BULK IN transfers is of size 4096. Which is not the expected size of UsbCommand struct...

For the fast person will, 544/64 == 8.5, meaning the usb-layer actually sends 9 usb-packages.
while 4096/64 == 64 packages. This means as it is today, the transfer rates from Device to Host is much slower than the other way around.

Isn't this strange? Why are we sending 4096 bytes UsbCommand struct?!?!?

iceman · 2017-10-16 18:28:16

Time to rephrase when I look more at the details.

Device to Host length request is 4096 (0x1000)
Actuall completed length is 544. (0x220) this is the expected one.

My false claim redacted, the PM3 is not slower in Device 2 Host comm.

So for BULK OUT both requestedlength and lengthtransfered is 544.
but BULK IN has 4096/544..

Where do we set that value...

piwi · 2017-10-17 07:09:14

iceman wrote:

Device to Host length request is 4096 (0x1000)

Where did you see/observe that?

iceman · 2017-10-17 08:07:50

If you want to look at a trace from Microsoft message analyzer v1.41, I can upload it.

http://www.icedev.se/proxmark3/traces/p … _trace.rar

iceman · 2017-10-17 09:12:34

A trace with PM3 offical v3.0.1

http://www.icedev.se/proxmark3/traces/o … 01_pm3.rar

iceman · 2017-10-17 09:20:50

You see both the 63 byte issue and the 4096 byte issue.

USB spec says something about bad packages when they are not in the expected size but acceptable as long as they are less than maxpackage.. 63 <64 so its acceptable but not optimal.

piwi · 2017-10-17 09:46:03

All transfers in both directions should be 544 bytes. Where should we expect 64byte transfers?

iceman · 2017-10-17 10:05:04

Since maxsize is 64bytes on MCU, all larger packages must be split up in order to transfer it.
Our usbcommand struct is 544bytes. Meaning usb transfers 9 packages to send it between device & host.

piwi · 2017-10-17 18:57:20

Found the reason for 63 Byte transfers. Fixed in PR #427 on official repository. And it results in increased speed!

iceman · 2017-10-17 19:06:16

Nice done!

I look at your PR, it still almost spot on to the one I'm fiddling with on my fork (not pushed), and I still get hangups in the client.
I'm getting frustrated now.
Did you find the reason for 4096 requests? I'm guessing you had a look at the traces..

piwi · 2017-10-18 19:50:39

I had a look at the trace today. Due to the fact that the trace is taken at the host, I can only guess that 4096 is the size of the OS input buffer, which it tries to fill...

iceman · 2017-10-18 21:30:25

Since you downloaded the message analyzer software, you can make your own trace...

steps involved:

Run the tools as admin,
new session
add providers -> add system providers
search for usbport (depends which kind u attach yr pm3 to. usb3 or usb2)
I select "microsoft-windows-usb-usbport" since I have USB2
start
group on usbdevice-column

iceman · 2017-10-23 22:23:41

Well, the aftermath from all this fiddeling around was some bug fixes, which all lead to better and more stable usb communications between device and client. Downside that during this time a lot of bricked proxmark3 devices happend, for that I'm sorry.

Proxmark3 community

Announcement

#1 2017-09-26 15:48:46

[closed] Proxmark3 device reconnecting from Offline mode

#2 2017-09-29 18:10:41

Re: [closed] Proxmark3 device reconnecting from Offline mode

#3 2017-09-30 02:25:19

Re: [closed] Proxmark3 device reconnecting from Offline mode

#4 2017-09-30 09:04:12

Re: [closed] Proxmark3 device reconnecting from Offline mode

#5 2017-10-01 20:13:01

Re: [closed] Proxmark3 device reconnecting from Offline mode

#6 2017-10-15 22:31:03

Re: [closed] Proxmark3 device reconnecting from Offline mode

#7 2017-10-16 01:14:09

Re: [closed] Proxmark3 device reconnecting from Offline mode

#8 2017-10-16 08:45:44

Re: [closed] Proxmark3 device reconnecting from Offline mode

#9 2017-10-16 08:54:14

Re: [closed] Proxmark3 device reconnecting from Offline mode

#10 2017-10-16 18:04:17

Re: [closed] Proxmark3 device reconnecting from Offline mode

#11 2017-10-16 18:28:16

Re: [closed] Proxmark3 device reconnecting from Offline mode

#12 2017-10-17 07:09:14

Re: [closed] Proxmark3 device reconnecting from Offline mode

#13 2017-10-17 08:07:50

Re: [closed] Proxmark3 device reconnecting from Offline mode

#14 2017-10-17 09:12:34

Re: [closed] Proxmark3 device reconnecting from Offline mode

#15 2017-10-17 09:20:50

Re: [closed] Proxmark3 device reconnecting from Offline mode

#16 2017-10-17 09:46:03

Re: [closed] Proxmark3 device reconnecting from Offline mode

#17 2017-10-17 10:05:04

Re: [closed] Proxmark3 device reconnecting from Offline mode

#18 2017-10-17 18:57:20

Re: [closed] Proxmark3 device reconnecting from Offline mode

#19 2017-10-17 19:06:16

Re: [closed] Proxmark3 device reconnecting from Offline mode

#20 2017-10-18 19:50:39

Re: [closed] Proxmark3 device reconnecting from Offline mode

#21 2017-10-18 21:30:25

Re: [closed] Proxmark3 device reconnecting from Offline mode

#22 2017-10-23 22:23:41

Re: [closed] Proxmark3 device reconnecting from Offline mode

Board footer