How to reliably(!) exclude files from a list in tar?

This is bothering me for years now, my backup script always takes everything with it, taking forever to finish.

I initially used the –exclude option, but this is rather restrictive, cluttered the script and still had the excluded directories.
Then i discovered -X/–exclude-from but same result here, weird globbing and still fails.
So i hacked a negative list via fd’s –ignore-file and tar -T/–files-from together. But tar still includes files and directories not on the fucking files.tmp.

So i’m not sure if it is a bug in Arch’s GNU tar or if it’s maybe a parameter in the wrong position, tar can be bitchy there. This is my current code


<span style="color:#323232;"># tar -cf - -X "$XDG_CONFIG_HOME"/backup/ignore "$INPUT" -P
</span><span style="color:#323232;">
</span><span style="color:#323232;">fd . -Hi --ignore-file "$XDG_CONFIG_HOME"/backup/ignore "${INPUT}" > "$_tmpfile"
</span><span style="color:#323232;">tar -cf - --verbatim-files-from --files-from="$_tmpfile" -P 
</span><span style="color:#323232;">	|pv -tapes "$_fssize" 
</span><span style="color:#323232;">	|compress >"${OUTPUT}.$_ext"
</span>

INPUT is $HOME in this case.

And if anyone has a solution that works on busybox tar as well…

MonkderDritte,

Thanks!

I save it for now, until i work on it again. Possibly the wildcards thing. And that tar includes files of folders given too, from someone else (how to work with that).

kbal,
@kbal@fedia.io avatar

I don't know what fd does, but at a guess maybe what you're missing is that tar includes all the files in directories you give it? So if you exclude 'foo/bar' but include 'foo' then foo/bar will be in your tar file.

What I do is basically tar cf ls ~ | grep -v $files_to_exclude but if you want to exclude something that isn't a top-level directory you'd need to get slightly more fancy.

kbal, (edited )
@kbal@fedia.io avatar

...more fancy such as using tar -X, which works for me. I'd never actually tried it before. The 'weird globbing' it uses is regular expressions, which are worth learning how to use. Run grep "$expression" $_tmpfile where $expression is a line from your exclude file to see which files it's going to match and exclude.

lurch,

–exclude works reliable for me. can you give us an example of an --exclude and the file name that tar outputs when adding it?

TCB13,
@TCB13@lemmy.world avatar

How to reliably(!) exclude files from a list in tar?

You delete the entire file because the format and tools are hard to deal with for basic operations like this one. :)

Just kidding… but we know there’s some truth to it.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • linux@lemmy.ml
  • fightinggames
  • All magazines
    •