the fear of missing out a better compression
the fear of missing out a better compression
the fear of missing out a better compression
Zip makes different tradeoffs. Its compression is basically the same as gz, but you wouldn't know it from the file sizes.
Tar archives everything together, then compresses. The advantage is that there are more patterns available across all the files, so it can be compressed a lot more.
Zip compresses individual files, then archives. The individual files aren't going to be compressed as much because they aren't handling patterns between files. The advantages are that an error early in the file won't propagate to all the other files after it, and you can read a file in the middle without decompressing everything before it.
Yeah that's a rather important point that's conveniently left out too often. I routinely extract individual files out of large archives. Pretty easy and quick with zip, painfully slow and inefficient with (most) tarballs.
Can you evaluate the directory tree of a tar without decompressing? Not sure if gzip/bzip2 preserve that.
Obligatory shilling for unar, I love that little fucker so much
unar <yourfile>
unzip
doesn'tWhat weird Japanese zip files are you handling?
Voicebanks for Utau (free (as in beer, iirc) clone of Vocaloid) are primarily distributed as SHIFT-JIS encoded zips. For example, try downloading Yufu Sekka's voicebank: http://sekkayufu.web.fc2.com/ . If I try to unzip
the "full set" zip, it produces a folder called ะ ัะโ ะะะtะขPะฃโะโฃะtะะะZะbะgะi111025ะj. But unar detects the encoding and properly extracts it as ้ชๆญใฆใๅ็ฌ้ณใใซใปใใ๏ผ111025). I'm sure there's some flag you can pass to unzip
to specify the encoding, but I like having unar
handle it for me automatically.
The same thing with zip, just use "unzip
<file>
" right?I'm an atool
s kinda person
Looks cool, I'll check it out
Where .7z at
.7z gang, represent
When I was on windows I just used 7zip for everything. Multi core decompress is so much better than Microsoft's slow single core nonsense from the 90s.
Small dick package kings/queens rise up.
Yeah, 7z is the clear winner.
It's much slower to decompress than DEFLATE ZIP though
cool people use zst
You can't decrease something by more than 100% without going negative. I'm assuming this doesn't actually decompress files before you tell it to.
Does this actually decompress in 1/13th the time?
Yeah, Facebook!
Sucks but yes that tool is damn awesome.
Meta also works with CentOS Stream at their Hyperscale variant.
Makes sense. There are actual programmers working at facebook. Programmers want good tools and functionality. They also just want to make good/cool/fun products. I mean, check out this interview with a programmer from pornhub. The poor dude still has to use jquery, but is passionate to make the best product they can, like everone in programming.
Right, I usually do that or lz4.
.tar.xz
When I'm feeling cool and downloading a *.tar*
file, I'll wget
to stdout, and tar from stdin. Archive gets extracted on the fly.
I have (successfully!) written an .iso
to CD this way, too (pipe wget to cdrecord). Fun stuff.
.tar.gz
, or .tgz
if I'm in a hurry
...or shipping to MSDOS
Where's .7z people?
Can someone explain why MacOS always seems to create _MACOSX folders in zips that we Linux/Windows users always delete anyway?
Window adds desktop.ini randomly too
Linux adds .demon_portal files all over my computer too.
Yeah, those tend to be pre-folder settings for the File Explorer.
Like View options, thumbnails and such.
It's been a while for me, but I think there was something specially for thumbnails too. You might find one if you go into the folder options and set a folder to optimized for pictures/videos and add some to it.
Huh, never noticed that. Probably always thought that was just part of the program/files needed.
this is a complete uneducated guess from a relatively tech-illiterate guy, but could it contain mac-specific information about weird non-essential stuff like folder backgrounds and item placement on the no-grid view?
They're Metadata specific for Macs.
If you download a third party compression tool they'll probably have an option somewhere to exclude these from the zips but the default tool doesn't Afaik.
"Resource forks" IIRC, old stuff. Same for the .DS_Store file.
For just $12.99 you can disable this https://apps.apple.com/us/app/blueharvest/id739483376
HFS+ has a different features set than NTFS or ext4, Apple elect to store metadata that way.
I would imagine modern FS like ZFS or btrfs could benefit from doing something similar but nobody has chosen to implement something like that in that way.
MacOS has two files per file, so the extras need to be stored somewhere.
Because Apple always gotta fuck with and "innovate" perfectly working shit
Windows's built-in tool can make zips without fucking with shit AND the resulting zip works just fine across systems.
Mac though...Mac produced zips always ALWAYS give me issues when trying to unzip on a non-mac (ESPECIALLY Linux)
.fitgirlrepack
I use .tar.gz in personal backups because it's built in, and because its the command I could get custom subdirectory exclusion to work on.
I'm the weird one in the room. I've been using 7z for the last 10-15 years and now .tar.zst
, after finding out that ZStandard achieves higher compression than 7-Zip, even with 7-Zip in "best" mode, LZMA version 1, huge dictionary sizes and whatnot.
zstd --ultra -M99000 -22 files.tar -o files.tar.zst
You can actually use Zstandard as your codec for 7z to get the benefits of better compression and a modern archive format! Downside is it's not a default codec so when someone else tries to open it they may be confused by it not working.
.tar.7z gang (probably not a good idea)
.tar.xz is. Or .tar.lzip.
.tar.7z.zip.bz2 gang unite
Mfโers act like they forgot about zstandatd
Me removing the plastic case of a 2.5' sata ssd to make it physically smaller
That's a big drive.
7z gang joined the chat.....
I mean xz/7z has kind of been the way for at least a decade now
.tar.gz
Can we please just never use proprietary rar ever. We have 7z, tar.gz, and the classic zip
all the cool kids use .cab
I use the command line every day, but can't be bothered with all the compression options of tar and company.
zip -r thing.zip things/
and unzip thing.zip
are temptingly more straightforward.
Need more compression? zip -r -9 thing.zip things/
. Need a faster option? Use a smaller digit.
"yes i would love to tar -xvjpf
my files"
-- statement dreamed up by the utterly insane
Present, I'm the tar cvJf
insane
Zip is fine (I prefer 7z), until you want to preserve attributes like ownership and read/write/execute rights.
Some zip programs support saving unix attributes, other - do not. So when you download a zip file from the internet - it's always a gamble.
Tar + gzip/bz2/xz is more Linux-friendly in that regard.
Also, zip compresses each file separately and then collects all of them in one archive.
Tar collects all the files first, then you compress the tarball into an archive, which is more efficient and produces smaller size.
The problem with that is that it will not preserve flags and access rights.
how about .tar.zip
or .tar.rar
?
Zip, RAR, 7z etc. store and compress the files. Tarballs work differently, tar stores the files and the second program compresses the tar as a continuous stream.
They originated in different mediums, programs like zip were born to deal with folder structures, tar was created to deal with linear tape archives (hence the name).
TIL
I prefer .tar.rar.tar.
.tar.zip.tar.xz
all the way
Kind of redundant. Both .zip
and .rar
store an index of files within the archive and are a bit 'inside-out' when it comes what we get from tar.gz
.
That is, ZIP is pretty close to what you'd get if you first gzipped all your files and then put them into a .tar
.
RAR does a little more (if I remember correctly), such as generating a dictionary of common redundancies between files and then uses that knowledge to compress the files individually, but better. Something akin to a .tar
file is still the result though.
That's fine. I'm use to being unrepresented in the arj, lha, and uc2 crew
If you download and extract the tarball as two separate steps instead of piping curl
directly into tar xz
(for gzip) / tar xj
(for bz2) / tar xJ
(for xz), are you even a Linux user?
I download and then tar. Curl pipes are scary
They really, really arenโt. Letโs take a look at this command together:
curl -L [some url goes here] | tar -xz
Sorry the formatting's a bit messy, Lemmy's not having a good day today
This command will start to extract the tar file while it is being downloaded, saving both time (since you donโt have to wait for the entire file to finish downloading before you start the extraction) and disk space (since you donโt have to store the .tar file on disk, even temporarily).
Letโs break down what these scary-looking command line flags do. They arenโt so scary once you get used to them, though. Weโre not scared of the command line. What are we, Windows users?
You may have noticed also that in the first command I showed, I didnโt put a - in front of the arguments to tar. This is because the tar command is so old that it takes its arguments BSD style, and will interpret its first argument as a set of flags regardless of whether thereโs a dash in front of them or not. tar -xz and tar xz are exactly equivalent. tar does not care.
the problem is if the connection gets interrupted your progress is gone. you download to a file first and it gets interrupted, you just resume the download later
.tar.xz
โ ๏ธ
one of the cool things about tar is that it's hard link aware
tar cvjf compressed-shit.tar.bz2 /path/to/uncompressed/shit/
Only way to fly.
I stopped doing that because I found it painfully slow. And it was quicker to gzip and upload than to bzip2 and upload.
Of course, my hardware wasn't quite as good back then. I also learned to stop adding 'v' flag because the bottleneck was actually stdout! (At least when extracting).
for the last 14 years of my career, I was using stupidly overpowered Oracle exadata systems exclusively, so "slow" meant 3 seconds instead of 1.
Now that I'm retired, I pretty much never need to compress anything.
Iโm curious about the contents in your compressed shit.
Good ol -xvjpf
I'm from tar.gz.gpg
gang to keep away nasty storage providers
.txz
, I'm too lazy to type the full name
This guy tar balls
What's wrong with .7z?
Nothing, but I've read people who act as if tar files are some sort of alien artifact ready to rip their faces off.
.tar.zst forever
Bzip2 compression is often surprisingly good with text files, especially log files. It seems to "see" redundancies there - and logs often have a lot of it - far better than gzip and sometimes even lzma.
Anyway, if I saw a bunch of tar.bz2
files, that's what I'd expect to find in them.
i just use either zip or tar+gzip. personally, i avoid rar as it's a proprietary format.
I unironically used xz for a long time. It was just eazy and all around very good compress. A close second is 7zip because I used it on windows for years.
.zlib
so silly loll
Iโm in this picture and I resent it.
Make the stage 4 tarball
I still wonder what that's like. Somebody must still occasionally get a notification that SOMEWHERE somebody paid for their WinRAR license and is like "WOAH WE GOT ANOTHER ONE!"
Never looked back since 7z though. :D
Compatibility aside, I'd say that .tar.pxz
aka .tpxz
is probably my vote.
LZMA is probably what I'd want to use. xz
and 7zip
use that. It's a bit slow to compress, but it has good compression ratios, and it's faster to decompress than bzip2.
pixz
permits for parallel LZMA compression/decompression. On present-day processors with a lot of cores, that's desirable.
It also can use .tar as its container format, which is desirable; that's everywhere.
The major drawback to .tar is that it doesn't support indexed access, so extracting a single file isn't fast, but .tar.pxz does.
What the fuck is that file extension?
Im a pirate, I am rar like, roawr, lol
Oblig. XKCD:
tar -h
Edit: wtf... It's actually
tar -?
. I'm so disappointedboom
tar eXtactZheVeckingFile
Me trying to decompress a .tar file
You don't need the v, it just means verbose and lists the extracted files.
tar -xzf
(read with German accent:) extract the files
Ixtrekt ze feils
German here and no shit - that is how I remember that since the first time someone made that comment
Thatโs so good I wish I needed to memorize the command
tar -uhhhmmmfuckfuckfuck
The Fish shell shows me just the past command with tar
So I don't need to remember strange flagsI use zsh and love the fish autocomplete so I use this:
https://github.com/zsh-users/zsh-autosuggestions
Also have
fzf
forctrl + r
to fuzzy find previous commands.I believe it comes with oh-my-zsh, just has to be enabled in plugins and itjustworksโข
man tar
you never said I can't run a command before it.
without looking, whatโs the flag to push over ssh with compression
scp
That's yet another great joke that GNU ruined.