php - Using ZipStream in Symfony: streamed zip download will not decompress using Archive Utility on Mac OSX

Question

I have a symfony 2.8 application and on the user clicking "download" button, I use the keys of several large (images, video) files on s3 to stream this to the browser as a zip file using ZipStream (https://github.com/maennchen/ZipStream-PHP).

The streaming of files and download as a zip (stored, not compressed) is successful in the browser & the zip lands in Downloads. However, when attempting to open the zip in Mac OSX El Capitan using Archive Utility (native archive software on OSX), it errors out. The Error:

Unable to expand "test.zip" into "Downloads". (Error 2 - No such file or directory.)

I have seen older, identical issues on SO and attempted those fixes, specifically this post: https://stackoverflow.com/a/5574305/136151 and followed up on the Issues & PRs in ZipStream that relates and the upstream fixes in Guzzle etc.

Problem is, the above fixes were back in 2011 and things move on in that time. So applying the same fixes I am not getting a working result.

Specific fixes I have tried is: 1. Setting "version to extract" to 0x000A as suggested. Also as '20' as recommended in another post. I've set the same for "version made by". 2. I tried to force the compression method to 'deflate' instead of 'stored' to see if I got a working result. A stored resulted is all I need and suitable for a zip used as container file for images & video.

I am able to extract the zip using a third party archive app called The Unarchiver. However, users won't know & can't be expected to install an alternative archive app to suit my web app. Thats not an effective solution.

Does anyone have knowledge or experience of solving this issue and can help me out with how to resolve it?

NB: A streamed zip to browser is the required solution. Downloading assets from s3 to the server to create a zip and then stream the resulting zip to the browser is not a solution given the amount of time and overhead of such an approach.

Added info if required:

Code & Setup: - Files are stored on s3. - Web app is symfony 2.8 on PHP7.0 runing on ec2 with Ubuntu. - Using aws-sdk-php, create an s3client with valid credentials and I register StreamWrapper (s3client->registerStreamWrapper()) on the s3client. This is to fetch files from s3 via fopen to stream to ZipStream library:

$this->s3Client = $s3Client;  
$this->s3Client->registerStreamWrapper();

// Initialize the ZipStream object and pass in the file name which
//  will be what is sent in the content-disposition header.
// This is the name of the file which will be sent to the client.
// Define suitable options for ZipStream Archive.
$opt = array(
        'comment' => 'test zip file.',
        'content_type' => 'application/octet-stream'
      );
$zip = new ZipStreamipStream($filename, $opt);

$keys = array(
      "zipsimpletestfolder/file1.txt"
  );

foreach ($keys as $key) {
    // Get the file name in S3 key so we can save it to the zip file 
    // using the same name.
    $fileName = basename($key);

    $bucket = 'mg-test';
    $s3path = "s3://" . $bucket . "/" . $key;

    if ($streamRead = fopen($s3path, 'r')) {
      $zip->addFileFromStream($fileName, $streamRead);        
    } else {
      die('Could not open stream for reading');
    }
}

$zip->finish();

Zip Output Results:

• Extraction on mac via Archive Utility fails with error 2

• Extraction on mac with The unarchiver works.

• Extraction on windows with 7-zip works.

• Extraction on windows with WinRar fails - says zip is corrupted.

Response Headers:

Edit I'm open to using another method of streaming files to browser as a zip that can be opened on Mac, Windows, Linux natively without using ZipStream if suggested. Just not to creating a zip file server side to subsequently stream thereafter.

Answer 1

The issue with the zip downloaded is that it contained html from the response in the Symfony controller that was calling ZipStream->addFileFromStream(). Basically, when ZipStream was streaming data to create zip download in client browser, a controller action response was also sent and, best guess, the two were getting mixed up on client's browser. Opening the zip file in hex editor and seeing the html in there it was obviously the issue.

To get this working in Symfony 2.8 with ZipStream, I just used Symfony's StreamedResponse in the controller action and used ZipStream in the streamedResponse's function. To stream S3 files I just passed in an array of s3keys and the s3client into that function. So:

use SymfonyComponentHttpFoundationStreamedResponse;
use AwsS3S3Client;    
use ZipStream;

//...

/**
 * @Route("/zipstream", name="zipstream")
 */
public function zipStreamAction()
{
    //test file on s3
    $s3keys = array(
      "ziptestfolder/file1.txt"
    );

    $s3Client = $this->get('app.amazon.s3'); //s3client service
    $s3Client->registerStreamWrapper(); //required

    $response = new StreamedResponse(function() use($s3keys, $s3Client) 
    {

        // Define suitable options for ZipStream Archive.
        $opt = array(
                'comment' => 'test zip file.',
                'content_type' => 'application/octet-stream'
              );
        //initialise zipstream with output zip filename and options.
        $zip = new ZipStreamipStream('test.zip', $opt);

        //loop keys useful for multiple files
        foreach ($s3keys as $key) {
            // Get the file name in S3 key so we can save it to the zip 
            //file using the same name.
            $fileName = basename($key);

            //concatenate s3path.
            $bucket = 'bucketname';
            $s3path = "s3://" . $bucket . "/" . $key;        

            //addFileFromStream
            if ($streamRead = fopen($s3path, 'r')) {
              $zip->addFileFromStream($fileName, $streamRead);        
            } else {
              die('Could not open stream for reading');
            }
        }

        $zip->finish();

    });

    return $response;
}

Solved. Maybe this helps someone else use ZipStream in Symfony.