Move repo to Github

doegox · 2014-03-27 00:00:20

For Windows:
Please try the new approach of Martin which is now integrated in my addhelpm branch.
I did a pull request towards Proxmark master branch already, but it's just a request, feel free to ignore it and do a pull on your unstable branch instead

doegox · 2014-03-27 00:01:27

cf https://github.com/Proxmark/proxmark3/pull/9

0xFFFF · 2014-03-27 00:09:56

ikarus wrote:

0xFFFF wrote:
Will this make things difficult for Windows users?
I don't think so. Even simple windows editors like notepad++ are supporting linux EOLs.

Ok great. I wasn't sure.

holiman wrote:

I've messaged with Roel, and can happily report that he is on board with both the move to github and starting a mailinglist. He'd naturally also like to be administrator on both parts (but I don't know his github username). He volunteered to host the mailinglist, but is a bit busy this week, but maybe next.
And yes, he hopes to be more active again in the future, and enjoys to see the community getting stronger. I think it's picked up the pace the last year, and I think it'll be even stronger after these two moves!

Excellent!
I don't know why the administrator comment is there. Roel is more than welcome to have administrative control. This applies to any change we decide on as a community. The same also applies to contributors and moderators.

As far as the mailing list is concerned, I like the way google groups works (and strongly recommend its use). It is easy to manage and its mailing lists work brilliantly.
I have setup a test group - https://groups.google.com/forum/#!forum/proxmark
Did Roel suggest using anything for the mailing list? Or just offer the hosting?

holiman · 2014-03-27 08:53:44

Roel wrote:

I'll look into this mailing list if you want. However, I'm pretty busy this week, so if you want to get it
up before the weekend you may have to query the community.

So, he does not really care what we choose. Google groups is fine by me.

holiman · 2014-03-27 15:40:53

asper wrote:

I confirm there is an error using the -h parameter under Windows, here is a screenshot:
http://i.imgur.com/PauqZQ9.png
shell crashes here and a windows message appears to force close the program; additional error details here:
Problem signature:
  Problem Event Name:	APPCRASH
  Application Name:	proxmark3.exe
  Application Version:	0.0.0.0
  Application Timestamp:	5332b007
  Fault Module Name:	pthreadGC2.dll
  Fault Module Versione:	2.9.0.0
  Fault Module Timestamp:	4dd6e26b
  Exception Code:	c0000005
  Exception Offset:	00006627
  OS Version:	6.1.7601.2.1.0.256.1
  Locale ID:	1040
Additional Information 1:	0a9e
Additional Information 2:	0a9e372d3b4ad19135b953a78882e789
Additional Information 3:	0a9e
Additional Information 4:	0a9e372d3b4ad19135b953a78882e789
There seems to be a problem with pthreadGC2.dll and the new -h parameter; anyway proxmark3 client seems to work fine for the rest.
Similar error with WIndows XP:
http://i.imgur.com/RmYm1Eg.png

Could you try with the new version of the help-implementation?
It should work fine with both -m and -h .

I'm not a master at git (I am a lot more comfortable with hg/mercurial), but from what I understand, this is one way to check it out. If anyone has better suggestions on the best way to check out a specific branch, please correct me:

#Fetch the master
git clone https://github.com/doegox/proxmark3.git proxmark-testing
cd proxmark-testing

git checkout -b doegox-addhelpm master
git pull https://github.com/doegox/proxmark3.git addhelpm

#Now the source should be updated, compile and run...

Last edited by holiman (2014-03-27 15:41:17)

doegox · 2014-03-27 17:06:20

holiman wrote:

#Fetch the master
git clone https://github.com/doegox/proxmark3.git proxmark-testing
...

Your solution is very clean when you want to do it in a tmp dir and delete everything afterwards.
Another solution as we may often look into each other's repo and do some cherry-picking or merging between repos is to use your usual working copy and add ref to someone else's repo.
E.g. to get my branch that would be something like:

# be in your usual proxmark3 working copy:
cd proxmark3
git remote add doegox https://github.com/doegox/proxmark3.git
git fetch doegox
git checkout doegox/addhelpm

At this point your working copy contains the top of my branch, but you're in a "detached" state which means you're not anymore in one of your local branch, I'll come back to that.

git checkout master

and you're back in your working copy

And later if you want to check my code again, just do

git fetch doegox # to get latest changes
git checkout doegox/addhelpm

Or e.g. even without checkout you can do operations like

git diff doegox/addhelpm

or

git cherry-pick xxxxxx

to take one of my commits and apply it to your branch.

You can also create a local branch that mirrors my remote branch:

git fetch doegox # to get latest changes
git checkout -b addhelpm doegox/addhelpm

Now your local branch addhelpm contains the same commits as my branch.

Before changing your local working copy, always make sure you're in a clean state, no staged changes, no local changes.
If you've local changes you can always save them temporarily with "git stash" and "git stash pop". That's handy.

Personally I like a lot to keep a "gitk --all &" opened somewhere to have an helicopter view of the multiple local and remote branches.

I hope I didn't completely confused you :-)
Just to show you some ways of working with several remote repos at the same time.

Phil

ikarus · 2014-03-28 00:24:45

Whats up with the gh-pages branch?!
Did someone try to create pages like this?
For doing so, you have to create a new repository, not a new branch.
The gh-pages branch is 2 commits ahead and 693 commits behind master
and the files in there have "nothing" to do with the proxmar3 source code.

So if I'm not completely mistaken: please create a second repository for the website and delete the branch.

Edit: The unstable branch looks weird too...

Last edited by ikarus (2014-03-28 00:27:41)

0xFFFF · 2014-03-28 06:20:48

gh-pages was thrown in there pretty early. I added it for the sake of listing Tags / Stable releases.
It was a topic I forgot to bring up when we were discussing the move.
Destroying it is simple. Re-creating it as a separate repo is just as easy. Do we need it?

asper · 2014-03-28 07:30:28

Is there a way to upload files (pdf, rar) in a "download section"? If it is not important i will link to google code download page.

ikarus · 2014-03-28 09:22:22

@0xFFFF:
I'm not sure if I got this right: "for the sake of listing Tags / Stable releases."
Tags and release binaries are listed on git automatically if you do it the designated way.
Read this if you want to know how.
And if you still want to create an extra proxmark3 website based on github-pages
you have to do it this way. In any case, deleting the gh-pages branch should be the right thing.
(You can delete a remote branch using "git push origin --delete <branchName>")

EDIT:
And of course do not merge the gh-pages branch into master or unstable. Code for website has nothing to do with the
code of the Proxmark3 device. As a rule of thumb: for each independent "project" create a new repository
(e.g. Proxmark3, Website (github pages), Tool XY (e.g. source code of Aspers MF access condition calculator), etc.)

@asper:
github (and git repositories) is not designed for hosting binaries. If we need to host "extra" binaries
(not the releases, but additional information), github-pages is "right" way (althoug a github-pages website is
a git repository too ). The alternative would be to host a separate website (or use google code page for now).

Last edited by ikarus (2014-03-28 10:41:14)

asper · 2014-03-28 09:28:01

doegox wrote:

For Windows:
Please try the new approach of Martin which is now integrated in my addhelpm branch.
I did a pull request towards Proxmark master branch already, but it's just a request, feel free to ignore it and do a pull on your unstable branch instead

Now it works both with -h and -m (tested on XP and 7) ! Thank you guys.

@ikarus:
thank you, I will use links to google code download section (for now).

EDIT:
I am recreating some new wiki pages but I am not able to put them under the "Wiki Index" on the right so they are only visible from the "pages" tab... can someone tell me how to do that ?

Last edited by asper (2014-03-28 10:11:52)

holiman · 2014-03-28 13:05:58

asper wrote:

doegox wrote:
For Windows:
Please try the new approach of Martin which is now integrated in my addhelpm branch.
I did a pull request towards Proxmark master branch already, but it's just a request, feel free to ignore it and do a pull on your unstable branch instead
Now it works both with -h and -m (tested on XP and 7) ! Thank you guys.

Branch is now merged into main

asper · 2014-03-29 12:17:56

Thanks holiman.

As I told you I am reworking the new GitHub wiki (in spare time) and now the "only" missing pages are "Usage" ones; I tried to update the text with all the recent changes/tips with pictures but I could miss something... can someone have a look to see if there are mistakes or other wrong things in the new-made pages ?

Last edited by asper (2014-03-29 12:20:23)

vivat · 2014-03-30 13:40:32

How is version.c will be generated for now? What happens if you launch hw version?

holiman · 2014-03-31 17:06:17

@vivat: Two good questions!

1:

proxmark3> hw version
#db# Prox/RFID mark3 RFID instrument                 
#db# bootrom: svn 839-unclean 2014-01-22 08:01:15                 
#db# os: svn 0 2014-03-31 15:55:09                 
#db# FPGA image built on 2014/03/24 at 21:54:44                 
uC: AT91SAM7S256 Rev B          
Embedded Processor: ARM7TDMI          
Nonvolatile Program Memory Size: 256K bytes          
Second Nonvolatile Program Memory Size: None          
Internal SRAM Size: 64K bytes          
Architecture Identifier: AT91SAM7Sxx Series          
Nonvolatile Program Memory Type: Embedded Flash Memory          
proxmark3>

So, we need to fix that. It's in tools/mkversion.pl, should be pretty simple fix.

On another note: I don't agree with starting versioning from 0.0. Versioning should be at least 0.8.52, (since we branched off around revision 852).
On a third note: versioning happens with git, so unless someone tagged a version on git, we still don't have any official github releases (sorry asper ) . And I don't think we should produce the first tagged version until the mkversion.pl script works as it should, by checking local git-repo instead of subversion and doing something clever to display what code it is.

It would be good to include latest version tag (if present), current commit hash, branch and if it is clean or unclean.

asper · 2014-03-31 18:19:58

i will produce my next compiled version after holiman's suggestex code fix. I agree with starting from 0.8.52!

doegox · 2014-03-31 23:42:34

See "git describe", useful once you have set the first tags, e.g. for libnfc we get:

git describe
libnfc-1.7.1-10-gbd92f74

Where
* libnfc-1.7.1 is the latest tag found in the history of current branch
* 10 means we're ahead of that tag by 10 commits
* gbd92f74 is the short ref to the latest commit

vivat · 2014-04-01 06:46:33

And what will be with svn states (clean, unclean etc)? Git should also have this features.

ikarus · 2014-04-01 07:51:11

vivat wrote:

And what will be with svn states (clean, unclean etc)? Git should also have this features.

Sorry, this might be a really silly question but what is a svn state (e.g. unclean)?

holiman · 2014-04-01 07:54:28

Didn't get that exact syntax working (without options to describe), but this could do:

#git describe --all --dirty
heads/ioprox_fixes
#echo "x" >> HACKING.txt 
#git describe --all --dirty
heads/ioprox_fixes-dirty

Last edited by holiman (2014-04-01 07:55:16)

ikarus · 2014-04-01 08:02:17

holiman wrote:

Versioning should be at least 0.8.52, (since we branched off around revision 852).

I don't want to start at 0.0.1 ether, but starting a 0.8.52 seems wrong too. We agreed on using Semantic Versioning
and therefore the numbers have a meaning:

Given a version number MAJOR.MINOR.PATCH, increment the:
1. MAJOR version when you make incompatible API changes,
2. MINOR version when you add functionality in a backwards-compatible manner, and
3. PATCH version when you make backwards-compatible bug fixes.

0.8.52 has nothing to do with Semantic Versioning, is just an adoption of the svn revision number.

Maybe we should start at 1.0.0 (this is common for the first release) but I'm open to other suggestions.

ikarus · 2014-04-01 08:08:18

holiman wrote:

Didn't get that exact syntax working (without options to describe), but this could do:
#git describe --all --dirty
heads/ioprox_fixes
#echo "x" >> HACKING.txt 
#git describe --all --dirty
heads/ioprox_fixes-dirty

Ahhh. A clean git is a git repository with no changes and an unclean (dirty) repo is a git repository with changes that are not committed, right?

EDIT:
I don't see why we should include this flag (dirty). Git declares repos as dirty if they have any modified files or untracked files.
If I develop my own proxmark features I will get the latest version from the master branch, create my own branch and work there.
I will make commits and update the .gitignore. So at least for my development process I will never get a dirty branch.
(And I think this process is very common.) In svn clean/unclean is useful because you can't create local, private branches. But on
git this is possible. In your local branch you do commits like any other developer therefore the branch should never be dirty.

But anyways, adding "--dirty" to "git describe" wont harm us , it will just append "-dirty".

[change something and do not commit it]
# git describe --dirty
libnfc-1.7.1-10-gbd92f74-dirty

Last edited by ikarus (2014-04-01 08:25:31)

holiman · 2014-04-01 08:33:17

Git declares repos as dirty if they have any modified files or untracked files.

Hm, not quite, untracked files do not show up:

#git describe --all --dirty
heads/ioprox_fixes
#touch test.txt
#git describe --all --dirty
heads/ioprox_fixes

I think we should include it. Normally, things won't be dirty, but if someone modified something it'll show up. If they modified, committed (locally), it will should also show up according to doegox (unless I'm misinterpreting that), but on my test-branch I don't have any versions yet, so I guess that's why I don't see that below:

#git describe --all --dirty
heads/ioprox_fixes
#git add test.txt
#git commit -m "foo"
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 test.txt
#git describe --all --dirty
heads/ioprox_fixes

More testing, with versioning:

#git tag -a v0.8.8 -m "Testversion"
#git describe --all
v0.8.8
#echo "x" > test.txt 
#git add test.txt && git commit -m "foobar"
 1 file changed, 1 insertion(+)
#git describe --all --dirty
heads/ioprox_fixes
#git describe --dirty
v0.8.8-1-g9969116
#git describe
v0.8.8-1-g9969116
#echo "x" >> HACKING.txt 
#git describe
v0.8.8-1-g9969116
#git describe --dirty
v0.8.8-1-g9969116-dirty

So, my testing shows that 'git describe --dirty' works ok if we're on main branch and have a version tag somewhere behind us. However, it does not show the branch name we are on. But perhaps it works well enough for our purposes?

ikarus · 2014-04-01 08:41:17

@holiman:
"git describe --dirty" looks good!
You can get the branch name with "git symbolic-ref HEAD"

# git symbolic-ref HEAD
refs/heads/master

EDIT:
Better use "git rev-parse --abbrev-ref HEAD"

# git rev-parse --abbrev-ref HEAD
master

Last edited by ikarus (2014-04-01 08:45:00)

holiman · 2014-04-01 09:08:17

#git describe --dirty && git symbolic-ref HEAD
v0.8.8-1-g9969116-dirty
refs/heads/ioprox_fixes

vs

#git describe --dirty && git rev-parse --abbrev-ref HEAD
v0.8.8-1-g9969116-dirty
ioprox_fixes

Sure, let's use the latter.

holiman · 2014-04-01 19:54:36

Ok, I versioned, pushed the changes to mkversion, and versioned again (tought the mkversion change would come before the tag-commit. I'm used to mercurial, all tags are commits to a local file, but git seems to treat tags differently).

How to versionize:

git tag -a v1.0.1 -m 'Version 1.0.1' && git push --tags

asper · 2014-04-01 21:48:09

Some suggestions about compiling under Windows ? (I would like that version is present inside firmware .elfs).

holiman · 2014-04-02 07:57:00

asper wrote:

Some suggestions about compiling under Windows ? (I would like that version is present inside firmware .elfs).

As usual. Just compile.

Perhaps my post earlier was a bit too hasty and compressed. Another try:
I have now fixed the mkversion.pl-script, so that it uses the git-versioning information instead of svn-version. It would be great if someone were to test that on different platforms (windows and mac). Either compile or just run 'perl tools/mkversion.pl' and see if it looks correct .

I also added a version tag (or rather, two) to the github codebase. Earlier, we had "implicit" versioning with svn revision numbers, which we don't anymore - now a dev needs to explicitly tag a version. My command was just to document how to do that. We should port the HACKING.txt to a github wiki page instead, and include some basic git-handling (e.g. versioning).

Also, after I was finished, it dawned on me that I'd probably caused a bug: https://github.com/Proxmark/proxmark3/issues/10 . Haven't had time to look at it yet.

gaucho · 2014-04-25 17:23:38

i found now this post. assumed that github is better than googlecode (i have no experience), i think that the googlecode should be dismissed as soon as possible and the link on the proxmark.org homepage should point to the github folder.

iZsh · 2014-06-16 04:26:28

hey, can someone add me back to the new repository (i had commit access to the svn repo).
I am about to commit a bunch of code for the fpga (LF code changes + general DSP/filter code) and the LF arm/client part. I guess i could fork and do pull requests but that would be slightly annoying since I'll split the commits into multiple ones.

my github: https://github.com/iZsh

Last edited by iZsh (2014-06-16 04:26:56)

holiman · 2014-06-16 20:28:00

I'm all for it, however, I don't think I have that power.

As for pull requests, you can still split it up into multiple commits, then when you do the final pull request, it includes all those little commits, just as if they'd been made on the master branch.

iZsh · 2014-06-16 21:16:01

yeah but it makes stuff more annoying for me

0xFFFF · 2014-06-16 22:42:54

iZsh, has been added as a member.

holiman · 2014-06-23 14:35:44

Hey iZsh, have you seen this? https://github.com/Proxmark/proxmark3/issues/16

iZsh · 2014-06-26 11:06:37

heh thx, i hadn't seen it. I replied.
Both the bootrom and the osimage need to be updated since the entry point address has changed

Last edited by iZsh (2014-06-26 11:08:07)

Proxmark3 community

Announcement

#51 2014-03-27 00:00:20

Re: Move repo to Github

#52 2014-03-27 00:01:27

Re: Move repo to Github

#53 2014-03-27 00:09:56

Re: Move repo to Github

#54 2014-03-27 08:53:44

Re: Move repo to Github

#55 2014-03-27 15:40:53

Re: Move repo to Github

#56 2014-03-27 17:06:20

Re: Move repo to Github

#57 2014-03-28 00:24:45

Re: Move repo to Github

#58 2014-03-28 06:20:48

Re: Move repo to Github

#59 2014-03-28 07:30:28

Re: Move repo to Github

#60 2014-03-28 09:22:22

Re: Move repo to Github

#61 2014-03-28 09:28:01

Re: Move repo to Github

#62 2014-03-28 13:05:58

Re: Move repo to Github

#63 2014-03-29 12:17:56

Re: Move repo to Github

#64 2014-03-30 13:40:32

Re: Move repo to Github

#65 2014-03-31 17:06:17

Re: Move repo to Github

#66 2014-03-31 18:19:58

Re: Move repo to Github

#67 2014-03-31 23:42:34

Re: Move repo to Github

#68 2014-04-01 06:46:33

Re: Move repo to Github

#69 2014-04-01 07:51:11

Re: Move repo to Github

#70 2014-04-01 07:54:28

Re: Move repo to Github

#71 2014-04-01 08:02:17

Re: Move repo to Github

#72 2014-04-01 08:08:18

Re: Move repo to Github

#73 2014-04-01 08:33:17

Re: Move repo to Github

#74 2014-04-01 08:41:17

Re: Move repo to Github

#75 2014-04-01 09:08:17

Re: Move repo to Github

#76 2014-04-01 19:54:36

Re: Move repo to Github

#77 2014-04-01 21:48:09

Re: Move repo to Github

#78 2014-04-02 07:57:00

Re: Move repo to Github

#79 2014-04-25 17:23:38

Re: Move repo to Github

#80 2014-06-16 04:26:28

Re: Move repo to Github

#81 2014-06-16 20:28:00

Re: Move repo to Github

#82 2014-06-16 21:16:01

Re: Move repo to Github

#83 2014-06-16 22:42:54

Re: Move repo to Github

#84 2014-06-23 14:35:44

Re: Move repo to Github

#85 2014-06-26 11:06:37

Re: Move repo to Github

Board footer