encryption - PHP: Warning mcrypt_generic_init(): Iv size is incorrect; supplied length: 12, needed: 8

Question

Basic Facts:

$algorithm  = MCRYPT_BLOWFISH;
$mode       = MCRYPT_MODE_CBC;
$randSource = MCRYPT_DEV_URANDOM;

Note This is not a strict coding question.

Context:

CentOS 7, Apache 2.4.12, & PHP 5.6.20.

I am making an HTML email with a "verify your email address" link that allows a registration process to complete. Everything on my virtual private server is UTF-8, and all form and query string input is processed with multi-byte (mb) funcions.

Background

As an experiment (I know about the age and state of the mcrypt library), I am attempting to decrypt Blowfish encrypted query string parameters. Assume that on the way up, the encryption sequence works perfectly and I am receiving email with the link.

On the way down, the hmac_hash() signing (SHA-512, just for this experiment) is working and I am able to separate each independent message (32 characters) from its hash checksum (128 characters). Base64 decoding of the separated message portion is working. For each parameter, I am left with the composite cipher text, where composite cipher text equals the IV + base cipher text. Assume I use a version of substr() to obtain the IV and the base cipher text independently (which is par for the course).

Problem

PHP: Warning  mcrypt_generic_init(): Iv size is incorrect; supplied length: 12, needed: 8

Assume I have combed the PHP manual and Stackoverflow. Assume I have looked at other questions similar, but not exactly like this one. Assume I have searched the Internet to no avail. Assume I have enough experience to setup mb_string properly. Assume that I will take care of mcrypt padding when I get past this current problem.

Could multi-byte issues be interfering with decryption?

Could base64 encoding the IV + base cipher text corrupt the IV?

Could base64 padding be an issue?

Should I be specifying a more specific MCRYPT_BLOWFISH_*?

Why does the blowfish IV size report 8 bytes, but rarely produces an 8 byte IV?

Which substr() should I use, substr() or mb_substr(), for a setup that leans towards making everything UTF-8 and processes all other input as multi-byte UTF-8. I know that is an odd question, but all of the PHP Manual mycrypt decryption sequence examples use substr(), and none use mb_substr(). Everything on my site works with mb_functions when possible, and I would not mind using substr() if it solved my problem, but it does not solve it. When I use mb_substr(), I get the following warning.

PHP: Warning  mcrypt_generic_init(): Iv size is incorrect; supplied length: 11, needed: 8

Does anyone have any experience with this exact issue? Constructive answers will be rewarded!

Latest

Above is an example Blowfish hash that I am trying to reconstruct from an array, received via a SHA512 HMACed, symmetricly Blowfish encrypted (CBC), url safe Base64 encoded, urlencoded, query string (phew!).

Below, is what the strings for the query string (having chopped up the blowfish hash above) look like after encrypting, signing, and base64 encoding, but before being urlencoded. Each one is 128 characters long (each string gets longer as you do more stuff).

Above is the Base64 decoded and Blowfish decrypted array derived from the query string (Obviously, there are security steps in between this result, but I am just trying to show the latest state of things.) Something is not right. Encryption appears to work without any errors. Decryption does not produce any errors either. The plain text is just wrong. If I join/implode these elements, they will not be like the Blowfish hash above.

Answer 1

I would guess that the issue will hide somewhere with the UTF-8 encoding, as you use it in incorrect contexts. It could also be that your framework does some magic for all use-cases. This could be too much and generally end up in security hole or just bugs like that, as you don't do what really needs to be done when it really needs to be done.

Strings in PHP are just collections of bytes. You can store text there, in encoding of your choosing, or you could just store binary data there, like images. PHP knows neither what kind of data is in what string nor what encoding is used there. This is up to developer to track this information.

When working with encryption, you get binary data when generating random strings or encrypting some payloads. It's saved in strings, but it does not have UTF-8 encoding, as it's just bytes. I wouldn't even say that it's encoding is ISO-8859-1, as this would mean that byte 77 (0x4D) stands for letter "M". But for real, it's just numbers - 77 does not stand for any letter at all.

One more thing to add - for ASCII symbols (Latin letters, digits etc. - 0-127 byte values) it takes one byte to represent that symbol in UTF-8 encoding (same as in ISO-8859). So as far as you pass base64_encoded data, you shouldn't worry too much about it. mb_substr will also work in the same way as substr. But! for the binary data, you cannot use mb_* functions, as it works with characters. For example, if encrypted data is two bytes 0xC5 0xA1, it's only single symbol in UTF-8. Encryption works with bytes (up until the final result, which could be anything - even binary files), not characters.

As you've not provided any code, I've put some for you - I hope it will help with your issue (if it's still relevant at all).

To show passing parameters in URL, there are two files: encrypt.php and decrypt.php. Save to a directory, run php -S localhost:8000 in it and go to http://localhost:8000/encrypt.php

encrypt.php:

<?php
// mcrypt_enc_get_key_size($td) gives 56, so it's longest that this key can be
$key = 'LedsoilgarvEwAbDavVenpirabUfjaiktavKekjeajUmshamEsyenvoa';
$data = 'This is very important data, with some ? UTF-8 ?? symbols';

$td = mcrypt_module_open(MCRYPT_BLOWFISH, '', MCRYPT_MODE_CBC, '');

// create random IV - it's just random 8 bytes. You should use random_bytes() instead if available
$ivSize = mcrypt_enc_get_iv_size($td);
$iv = mcrypt_create_iv($ivSize, MCRYPT_DEV_URANDOM);

mcrypt_generic_init($td, $key, $iv);

$encrypted = mcrypt_generic($td, $data);

mcrypt_generic_deinit($td);
mcrypt_module_close($td);

// payload that you want to send - binary. It's neither UTF-8 nor ISO-8859-1 - it's just bytes
$payload = $iv . $encrypted;

// base64 to pass safely
$base64EncodedPayload = base64_encode($payload);
// URL encode for URL. No need to do both URL-safe base64 *and* base64 + urlencode
$link = 'http://localhost:8000/decrypt.php?encryptedBase64=' . urlencode($base64EncodedPayload);

// in fact, just for the reference, you don't even need base64_encode - urlencode also works at byte level
// base64_encode takes about 1.33 more space, but urlencode takes 3 times more than original for non-safe symbols, so base_64 will probably be shorter
$link2 = 'http://localhost:8000/decrypt.php?encrypted=' . urlencode($payload);

?>
<!doctype html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
        <pre><?php
            var_dump('Data:', $data);
            var_dump('Data size in bytes:', strlen($data));
            var_dump('Data size in characters - smaller, as 3 of the characters take 2 bytes:', mb_strlen($data, 'UTF-8'));
            var_dump('Encrypted data size in bytes - same as original:', strlen($encrypted));
            var_dump('Encrypted data size in characters - will be pseudo-random each time:', mb_strlen($encrypted, 'UTF-8'));

            var_dump('IV base64 encoded:', base64_encode($iv));
            var_dump('Encrypted string base64 encoded:', base64_encode($encrypted));
        ?></pre>
        <!-- Link will not contain any special characters, so htmlentities should not make any difference -->
        <!-- In any case, I would still recommend to use right encoding at the right context to avoid any issues if something changes -->
        <a href="<?php echo htmlentities($link, ENT_QUOTES, 'UTF-8');?>">Link to decrypt</a><br/>
        <a href="<?php echo htmlentities($link2, ENT_QUOTES, 'UTF-8');?>">Link to decrypt2</a>
    </body>
</html>

decrypt.php:

<?php
$key = 'LedsoilgarvEwAbDavVenpirabUfjaiktavKekjeajUmshamEsyenvoa';

if (isset($_GET['encryptedBase64'])) {
    // just get base64_encoded symbols (will be ASCII - same in UTF-8 or other encodings)
    $base64EncodedPayload = $_GET['encryptedBase64'];
    $payload = base64_decode($base64EncodedPayload);
} else {
    // just get binary string from URL
    $payload = $_GET['encrypted'];
}

$td = mcrypt_module_open(MCRYPT_BLOWFISH, '', MCRYPT_MODE_CBC, '');

$ivSize = mcrypt_enc_get_iv_size($td);

$iv = substr($payload, 0, $ivSize);
$encrypted = substr($payload, $ivSize);

mcrypt_generic_init($td, $key, $iv);

/* Decrypt encrypted string */
$decrypted = mdecrypt_generic($td, $encrypted);

/* Terminate decryption handle and close module */
mcrypt_generic_deinit($td);
mcrypt_module_close($td);

?>
<!doctype html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
        <pre><?php
            var_dump('IV base64 encoded:', base64_encode($iv));
            var_dump('Encrypted string base64 encoded:', base64_encode($encrypted));
            var_dump('Result:', $decrypted);
        ?></pre>
    </body>
</html>