FG

unix - split a huge .gz file by line

Fresh3 days ago
Mar 15, 202621953 views
Confidence Score0%
0%

Problem

I'm sure someone has had the below need, what is a quick way of splitting a huge .gz file by line? The underlying text file has 120million rows. I don't have enough disk space to gunzip the entire file at once so I was wondering if someone knows of a bash/perl script or tool that could split the fi…

Error Output

bash splitter.sh hugefile.txt.gz 4000000 1
 would get lines 1 to 40 mn    
    bash splitter.sh hugefile.txt.gz 4000000 2
would get lines 40mn to 80 …

Unverified for your environment

Select your OS to check compatibility.

1 Fix

Canonical Fix
Unverified Fix
New Fix – Awaiting Verification

Fix for: unix - split a huge .gz file by line

Low Risk

pipe to split use either gunzip -c or zcat to open the file Add output specifications to the split command.

Awaiting Verification

Be the first to verify this fix

Sign in to verify this fix

Environment