Stefan Arentz

This is rather serious. Consider this:

% dd if=/dev/zero of=data bs=1 count=2679
% sha256sum data
a25f4ccc56ddf88a4fb3e11baec5838c5181a496f376cdd569f8fc782f8fdcdf  data

A file of 2679 zeros and its SHA-256 hash. Nothing special.

Well, not exactly .. now look at the equivalent Python code:

$ python
Python 2.4.3 (#2, Oct  6 2006, 07:49:22)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Crypto.Hash import SHA256
>>> data = open("data", "rb").read()
>>> print SHA256.new(data).hexdigest()
14a59554dba8be8ac9aa03eea67a026b3773eb674d22719123be41ca774319a3

Oops!

So it seems the Python Crypto module generates wrong SHA-256 hashes for files that have a size of (N*8)-1. But only for files of reasonable size.

We discovered this the hard way because we cache images based on their SHA256 hash. All cache entries with wrong hashes had this (N*8)-1 file size.

I’m not sure if this is just limited to the Ubuntu Dapper Drake / X64 install that we use. For the record, this is the package I’m talking about:

$ aptitude show python2.4-crypto
Package: python2.4-crypto
State: installed
Automatically installed: yes
Version: 2.0.1+dfsg1-1ubuntu1
Priority: optional
Section: python
Maintainer: Andreas Rottmann 
Uncompressed Size: 557k
Depends: libc6 (>= 2.3.4-1), libgmp3c2, python2.4
Description: cryptographic algorithms and protocols for Python
 A collection of cryptographic algorithms and protocols, implemented for use from Python.
  Among the contents of the package: 

    * Hash functions: MD2, MD4.
    * Block encryption algorithms: AES, ARC2, Blowfish, CAST, DES, Triple-DES.
    * Stream encryption algorithms: ARC4, simple XOR.
    * Public-key algorithms: RSA, DSA, ElGamal, qNEW.
    * Protocols: All-or-nothing transforms, chaffing/winnowing.
    * Miscellaneous: RFC1751 module for converting 128-key keys into a
      set of English words, primality testing.

This sure taught us a lesson. Never trust (crypto) code until you have unit tested it yourself.

I’m trying to figure out where to report this.