Add `ignore` support to `ipfs add`

[lgarron]

I do not ever plan to add .DS_Store files into an object, but it's annoying to delete them manually (and Finder.app always puts them back if you're viewing the folder).

If would be nice to have one of the following:

• A --ignore flag for ipfs add.
• Support for .ipfsignore files (either within the node, or globally).

[whyrusleeping]

Ive been wanting this myself too, if i try to do ipfs add -r / i dont want my /dev or /tmp directories added. And i probably also dont want ~/.go-ipfs in that too... might get weird.

[btc]

+1 for .ipfsignore

[gatesvp]

I love this idea. For anyone who's a git user, there's a certain intuition to having an .ipfsignore flag.

That stated, I think this may actually be a pair of features.

1. Ignore any hidden files by default (files/folder beginning with .).
2. Add the ability to respect an .ipfsignore file. Something like the git one (http://git-scm.com/docs/gitignore).

Part 1 actually seems like a rational default in most OSes. We'll need to add a -f for explicit inclusion of these files.

Part 2 would need some consensus on where to store the .ipfsignore file. In git, this makes sense to store at the root of a repo. Given that I can run ipfs add basically anywhere on my computer, it's not clear if I want to look at some base file or check the values in the folder I'm currently adding.

[whyrusleeping]

re Part 2, i think both looking in $IPFS_PATH/.ipfsignore and the current directory would be a good way to do it

[jbenet]

@gatesvp agreed on the two parts. for part 1, i think maybe --hidden could include all of them. so:

> ls
.h1
.h2
f1
f2
> ipfs add -r .
added <> f1 
added <> f2 
> ipfs add .h1
error: .h1 is a hidden file. 
> ipfs add --hidden .h1
added <> .h1
> ipfs add --hidden -r .
added <> .h1
added <> .h2
added <> f1
added <> f2

for part 2, i think we should follow git's idea: recursively walk up and apply all .ipfsignores. (at least i think it does that? pls correct me). and +1 on using a default one in $IPFS_PATH/.ipfsignore.

[gatesvp]

So I am part way through this, and have just realized that Windows has a different notion of a hidden file and folder. "Hidden" is actually an attribute of the file rather than just a . at the beginning of the name.

It looks like git in Windows actually makes the .git file hidden.

My current plan is to make Windows support both the attribute and the dot prefix by default. Using the . as the first character in Windows is really rare and it hopefully makes things portable.

Thoughts? Concerns?

[whyrusleeping]

that sounds pretty good to me

[gatesvp]

Alright, quick update after a fun / busy weekend

Good news:

• I have a basic working version, it's here (https://github.com/gatesvp/go-ipfs/tree/iss653)
• The .gitignore file is a little complex, but someone has implemented it under the MIT license (https://github.com/sabhiram/go-git-ignore)
• There is something kind of cool about watching this correctly ignore hidden files by default :)

Bad news:

• The imported go-git-ignore is a little flaky because it doesn't handle encodings. Obviously an issue on a windows machine.
• I'm not 100% clear on $IPFS_PATH/.ignore. Do we use this somewhere else (flatfs)? Or is this just an invention for this run?
• My code currently includes a straight pointer to go-git-ignore, I know that we normally import this as a go-dep but I'm not clear on the methodology.

[gatesvp]

Arg.. still one bug at the end of this gist: https://gist.github.com/gatesvp/d9e26a391064ee50ab7b
But it's too late in the day for me to fix this.

In any case, I still appreciate inputs on the rest of the stuff :)

[whyrusleeping]

dont worry about vendoring, i can manage that for you. (or, when i fail horribly at figuring out godeps, jbenet can do it for you)

[jbenet]

@gatesvp great to hear!

I'm not 100% clear on $IPFS_PATH/.ignore. Do we use this somewhere else (flatfs)? Or is this just an invention for this run?

The idea is that we'd have one central location (similar to ~/.gitconfig) to add another .ipfsignore file to ignore things you always want ignored (e.g. if i always want to ignore .DS_Store on osx, even if i ask for --hidden files.)

I think it should be $IPFS_PATH/.ipfsignore (not .../.ignore) so the filename is consistent everywhere.

My code currently includes a straight pointer to go-git-ignore, I know that we normally import this as a go-dep but I'm not clear on the methodology.

try:

> make vendor

it may complain about not having things, you may have to godep restore them.

(or, when i fail horribly at figuring out godeps, jbenet can do it for you)

Yeah i can fix it.

(LMVTFY :) )

(vendoring came up: 2/100)