SparkFS meets multi-core?

I was wondering if anyone have thoughts on passing SparkFS work off to
other cores (one of the ideas suggested by ROOL),

From what I read, the Zip algorithm cannot be done in parallel, but maybe some
of the other algorithms can be? Or is there some other way to do the work in
the background?

I don’t know of any parallel zip programs, but gzip has pigz on Linux which I use to speed up compression (and decompression) when backing up raspberry Pi disc images.

SparkFS could certainly make use of multi-core as it has to build the zip file in RAM (or very much more slowly to disc if it is too big) before writing the modified file back to where it came from, and it is optimised to only write the central directory a few seconds after activity has finished to avoid repeating the operation multiple as times as files being are added by filer action. Offloading the creation and compression tasks to a different core(s), would allow other processes to continue until the file is ready to be written out when activity has stopped.

However, I suspect it a large amount of work, and the speed up would only be noticed when dealing with huge archives. Making sure the RAM buffer is configured large enough is far more important.

“The app’s finished writing the ZIP, I can turn off now…”

We get that now, if you write files to a zip and immediately move it or email it, it will be corrupted as the central directory hasn’t been written. That is repairable though. The solution is far SparkFS to hold up the shutdown until it has flushed it’s buffers.

The solution is far SparkFS to hold up the shutdown until it has flushed it’s buffers.

Close to 30 years of dealing with the average user (PC’s) indicates that there is a near certainty that numerous users will simply hit the power button.
If the one on the actual computer doesn’t do the job, they will use the one on the power socket.

Can’t do anything about stupid. Every general purpose OS out there has write back disk caching, so pulling the plug can lose data. Plus it’s not like you can’t break Filecore that way even without caching.

there is a near certainty that numerous users will simply hit the power button…Can’t do anything about stupid…

And even unstupid can’t do anything about power cuts. Uninterruptible Power Supplies? Ask my friends at CUP…mains power, while not 100% reliable, has proved a bit less unreliable than even rather expensive UPSs, which have twice managed to bugger things up when there wasn’t even a mains failure.

Google: parallel deflation algorithm zip

ArjavNaik/-Parallel-Lossless-Compression-for-GPUs:
Parallel decompression of gzip-compressed files and …
Pigz parallel implementation of gzip for modern
multi-processor, multi-core machines

Talk about Zip compression, but Zip uses various algorithms, the most common is inflate/deflate. Roughly speaking it uses chunks of data so you could do more than one chunk at a time. Might not compress as well as going through the entire file, you’re probably going to create chunks that are non-optimal.