I received thoughtful responses to my earlier post Large gzipped files, long lines, extracting columns etc.
I am not going to address everything in detail, but for the time being, let's stick with the phase where I dissect the files. Once I decide wha [...]