Commit graph

586 commits

Author SHA1 Message Date
Herbert Xu
113adefc73 crypto: shash - Make descsize a run-time attribute
This patch changes descsize to a run-time attribute so that
implementations can change it in their init functions.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 12:58:00 +08:00
Herbert Xu
57cfe44bcc crypto: shash - Move null setkey check to registration time
This patch moves the run-time null setkey check to shash_prepare_alg
just like we did for finup/digest.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-12 10:46:02 +08:00
Herbert Xu
9b2fda7b94 crypto: sha256_generic - Add export/import support
This patch adds export/import support to sha256_generic.  The exported
type is defined by struct sha256_state, which is basically the entire
descriptor state of sha256_generic.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-11 18:23:34 +08:00
Herbert Xu
3d4d277cf8 crypto: sha256_generic - Use 64-bit counter like sha1
This patch replaces the two 32-bit counter code in sha256_generic
with the simpler 64-bit counter code from sha1.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-11 18:23:33 +08:00
Herbert Xu
e2a7ce4e18 crypto: sha1_generic - Add export/import support
This patch adds export/import support to sha1_generic.  The exported
type is defined by struct sha1_state, which is basically the entire
descriptor state of sha1_generic.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-11 18:23:33 +08:00
Herbert Xu
8267adab94 crypto: shash - Move finup/digest null checks to registration time
This patch moves the run-time null finup/digest checks to the
shash_prepare_alg function which is run at registration time.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-11 18:23:33 +08:00
Herbert Xu
99d27e1c59 crypto: shash - Export/import hash state only
This patch replaces the full descriptor export with an export of
the partial hash state.  This allows the use of a consistent export
format across all implementations of a given algorithm.

This is useful because a number of cases require the use of the
partial hash state, e.g., PadLock can use the SHA1 hash state
to get around the fact that it can only hash contiguous data
chunks.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-11 18:23:32 +08:00
Herbert Xu
7ede5a5ba5 crypto: api - Fix crypto_drop_spawn crash on blank spawns
This patch allows crypto_drop_spawn to be called on spawns that
have not been initialised or have failed initialisation.  This
fixes potential crashes during initialisation without adding
special case code.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-09 11:34:06 +08:00
Herbert Xu
deee2289b9 crypto: shash - Propagate reinit return value
This patch fixes crypto_shash_import to propagate the value returned
by reinit.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 23:39:01 +08:00
Herbert Xu
f88ad8de28 crypto: shash - Use finup in default digest
This patch simplifies the default digest function by using finup.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 23:32:08 +08:00
Herbert Xu
619a6ebd25 crypto: shash - Add shash_register_instance
This patch adds shash_register_instance so that shash instances
can be registered without bypassing the shash checks applied to
normal algorithms.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 19:00:38 +08:00
Herbert Xu
7d6f56400a crypto: shash - Add shash_attr_alg2 helper
This patch adds the helper shash_attr_alg2 which locates a shash
algorithm based on the information in the given attribute.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 18:58:30 +08:00
Herbert Xu
d06854f024 crypto: api - Add crypto_attr_alg2 helper
This patch adds the helper crypto_attr_alg2 which is similar to
crypto_attr_alg but takes an extra frontend argument.  This is
intended to be used by new style algorithm types such as shash.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 18:58:29 +08:00
Herbert Xu
942969992d crypto: shash - Add spawn support
This patch adds the functions needed to create and use shash
spawns, i.e., to use shash algorithms in a template.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 18:58:29 +08:00
Herbert Xu
97eedce1a6 crypto: api - Add new style spawn support
This patch modifies the spawn infrastructure to support new style
algorithms like shash.  In particular, this means storing the
frontend type in the spawn and using crypto_create_tfm to allocate
the tfm.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 18:57:10 +08:00
Herbert Xu
2e4fddd8e4 crypto: shash - Add shash_instance
This patch adds shash_instance and the associated alloc/free
functions.  This is meant to be an instance that with a shash
algorithm under it.  Note that the instance itself doesn't have
to be shash.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-08 18:57:10 +08:00
Herbert Xu
70ec7bb91a crypto: api - Add crypto_alloc_instance2
This patch adds a new argument to crypto_alloc_instance which
sets aside some space before the instance for use by algorithms
such as shash that place type-specific data before crypto_alg.

For compatibility the function has been renamed so that existing
users aren't affected.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-07 18:55:28 +08:00
Herbert Xu
f2ac72e826 crypto: api - Add new template create function
This patch introduces the template->create function intended
to replace the existing alloc function.  The intention is for
create to handle the registration directly, whereas currently
the caller of alloc has to handle the registration.

This allows type-specific code to be run prior to registration.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-07 12:30:33 +08:00
Sebastian Andrzej Siewior
fd09d7facb crypto: ansi_prng - alloc cipher just in init
As reported by Eric Sesterhenn the re-allocation of the cipher in reset leads
to:
|BUG: sleeping function called from invalid context at kernel/rwsem.c:21
|in_atomic(): 1, irqs_disabled(): 0, pid: 4926, name: modprobe
|INFO: lockdep is turned off.
|Pid: 4926, comm: modprobe Tainted: G   M 2.6.31-rc1-22297-g5298976 #24
|Call Trace:
| [<c011dd93>] __might_sleep+0xf9/0x101
| [<c0777aa0>] down_read+0x16/0x68
| [<c048bf04>] crypto_alg_lookup+0x16/0x34
| [<c048bf52>] crypto_larval_lookup+0x30/0xf9
| [<c048c038>] crypto_alg_mod_lookup+0x1d/0x62
| [<c048c13e>] crypto_alloc_base+0x1e/0x64
| [<c04bf991>] reset_prng_context+0xab/0x13f
| [<c04e5cfc>] ? __spin_lock_init+0x27/0x51
| [<c04bfce1>] cprng_init+0x2a/0x42
| [<c048bb4c>] __crypto_alloc_tfm+0xfa/0x128
| [<c048c153>] crypto_alloc_base+0x33/0x64
| [<c04933c9>] alg_test_cprng+0x30/0x1f4
| [<c0493329>] alg_test+0x12f/0x19f
| [<c0177f1f>] ? __alloc_pages_nodemask+0x14d/0x481
| [<d09219e2>] do_test+0xf9d/0x163f [tcrypt]
| [<d0920de6>] do_test+0x3a1/0x163f [tcrypt]
| [<d0926035>] tcrypt_mod_init+0x35/0x7c [tcrypt]
| [<c010113c>] _stext+0x54/0x12c
| [<d0926000>] ? tcrypt_mod_init+0x0/0x7c [tcrypt]
| [<c01398a3>] ? up_read+0x16/0x2b
| [<c0139fc4>] ? __blocking_notifier_call_chain+0x40/0x4c
| [<c014ee8d>] sys_init_module+0xa9/0x1bf
| [<c010292b>] sysenter_do_call+0x12/0x32

because a spin lock is held and crypto_alloc_base() may sleep.
There is no reason to re-allocate the cipher, the state is resetted in
->setkey(). This patches makes the cipher allocation a one time thing and
moves it to init.

Reported-by: Eric Sesterhenn <eric.sesterhenn@lsexperts.de>
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-03 12:10:47 +08:00
Sebastian Andrzej Siewior
ed94070058 crypto: ansi_prng - Use just a BH lock
The current code uses a mix of sping_lock() & spin_lock_irqsave(). This can
lead to deadlock with the correct timming & cprng_get_random() + cprng_reset()
sequence.
I've converted them to bottom half locks since all three user grab just a BH
lock so this runs probably in softirq :)

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-03 12:09:41 +08:00
Herbert Xu
a68f6610d4 crypto: testmgr - Allow implementation-specific tests
This patch adds the support for testing specific implementations.
This should only be used in very specific situations.  Right now
this means specific implementations of random number generators.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-02 16:32:12 +08:00
Dan Williams
daf4219dbc dmaengine: move HIGHMEM64G restriction to ASYNC_TX_DMA
On HIGHMEM64G systems dma_addr_t is known to be larger than (void *)
which precludes async_xor from performing dma address conversions by
reusing the input parameter address list.  However, other parts of the
dmaengine infrastructure do not suffer this constraint, so the
HIGHMEM64G restriction can be down-levelled.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-07-01 16:12:53 -07:00
Herbert Xu
0b67fb65d1 crypto: skcipher - Change default sync geniv on SMP to eseqiv
As it stands we use chainiv for sync algorithms and eseqiv for
async algorithms.  However, when there is more than one CPU
chainiv forces all processing to be serialised which is usually
not what you want.  Also, the added overhead of eseqiv isn't that
great.

Therefore this patch changes the default sync geniv on SMP machines
to eseqiv.  For the odd situation where the overhead is unacceptable
then chainiv is still available as an option.

Note that on UP machines chainiv is still preferred over eseqiv
for sync algorithms.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-25 18:43:48 +08:00
Herbert Xu
435578aeaa crypto: skcipher - Fix request for sync algorithms
When a sync givcipher algorithm is requested, if an async version
of the same algorithm already exists, then we will loop forever
without ever constructing the sync version based on a blkcipher.

This is because we did not include the requested type/mask when
getting a larval for the geniv algorithm that is to be constructed.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-25 14:46:31 +08:00
Herbert Xu
259c5e05c1 crypto: testmgr - Remove hash size check
Until hash test vectors grow longer than 256 bytes, the only
purpose of the check is to generate a gcc warning.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-24 13:48:13 +08:00
Neil Horman
215ccd6f55 crypto: fips - Select CPRNG
The ANSI CPRNG has no dependence on FIPS support.  FIPS support however,
requires the use of the CPRNG.  Adjust that depedency relationship in Kconfig.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-21 21:38:03 +08:00
Herbert Xu
ea40065769 crypto: tcrypt - Fix module return code when testing by name
We should return 0/-ENOENT instead of 1/0 when testing by name.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-19 20:37:00 +08:00
Herbert Xu
27300176d7 crypto: ansi_cprng - Do not select FIPS
The RNG should work with FIPS disabled.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-19 20:32:58 +08:00
Steffen Klassert
a873a5f1c4 crypto: tcrypt - Test algorithms by name
This adds the 'alg' module parameter to be able to test an
algorithm by name. If the algorithm type is not ad-hoc
clear for a algorithm (e.g. pcrypt, cryptd) it is possilbe
to set the algorithm type with the 'type' module parameter.

Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-19 19:46:53 +08:00
Neil Horman
5b739ef8a4 random: Add optional continuous repetition test to entropy store based rngs
FIPS-140 requires that all random number generators implement continuous self
tests in which each extracted block of data is compared against the last block
for repetition.  The ansi_cprng implements such a test, but it would be nice if
the hw rng's did the same thing.  Obviously its not something thats always
needed, but it seems like it would be a nice feature to have on occasion. I've
written the below patch which allows individual entropy stores to be flagged as
desiring a continuous test to be run on them as is extracted.  By default this
option is off, but is enabled in the event that fips mode is selected during
bootup.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Matt Mackall <mpm@selenic.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-18 19:50:21 +08:00
Vegard Nossum
722f2a6c87 Merge commit 'linus/master' into HEAD
Conflicts:
	MAINTAINERS

Signed-off-by: Vegard Nossum <vegard.nossum@gmail.com>
2009-06-15 15:50:49 +02:00
Vegard Nossum
33f65df7ed crypto: don't track xor test pages with kmemcheck
The xor tests are run on uninitialized data, because it is doesn't
really matter what the underlying data is. Annotate this false-
positive warning.

Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Vegard Nossum <vegard.nossum@gmail.com>
2009-06-15 12:40:10 +02:00
Alex Riesen
aa07a6990f crypto: api - Use formatting of module name
Besdies, for the old code, gcc-4.3.3 produced this warning:
  "format not a string literal and no format arguments"

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:13:14 +10:00
Herbert Xu
a0cfae59f8 crypto: testmgr - Allow hash test vectors longer than a page
As it stands we will each test hash vector both linearly and as
a scatter list if applicable.  This means that we cannot have
vectors longer than a page, even with scatter lists.

This patch fixes this by skipping test vectors with np != 0 when
testing linearly.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:05:02 +10:00
Herbert Xu
fd57f22a09 crypto: testmgr - Check all test vector lengths
As we cannot guarantee the availability of contiguous pages at
run-time, all test vectors must either fit within a page, or use
scatter lists.  In some cases vectors were not checked as to
whether they fit inside a page.  This patch adds all the missing
checks.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:05:00 +10:00
Jarod Wilson
4e033a6bc7 crypto: tcrypt - Do not exit on success in fips mode
At present, the tcrypt module always exits with an -EAGAIN upon
successfully completing all the tests its been asked to run. In fips
mode, integrity checking is done by running all self-tests from the
initrd, and its much simpler to check the ret from modprobe for
success than to scrape dmesg and/or /proc/crypto. Simply stay
loaded, giving modprobe a retval of 0, if self-tests all pass and
we're in fips mode.

A side-effect of tracking success/failure for fips mode is that in
non-fips mode, self-test failures will return the actual failure
return codes, rather than always returning -EAGAIN, which seems more
correct anyway.

The tcrypt_test() portion of the patch is dependent on my earlier
pair of patches that skip non-fips algs in fips mode, at least to
achieve the fully intended behavior.

Nb: testing this patch against the cryptodev tree revealed a test
failure for sha384, which I have yet to look into...

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:57 +10:00
Geert Uytterhoeven
3ce858cb04 crypto: compress - Return produced bytes in crypto_{,de}compress_{update,final}
If crypto_{,de}compress_{update,final}() succeed, return the actual number of
bytes produced instead of zero, so their users don't have to calculate that
theirselves.

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:56 +10:00
Jarod Wilson
a3bef3a31a crypto: testmgr - Skip algs not flagged fips_allowed in fips mode
Because all fips-allowed algorithms must be self-tested before they
can be used, they will all have entries in testmgr.c's alg_test_descs[].
Skip self-tests for any algs not flagged as fips_approved and return
-EINVAL when in fips mode.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:49 +10:00
Jarod Wilson
a1915d51e8 crypto: testmgr - Mark algs allowed in fips mode
Set the fips_allowed flag in testmgr.c's alg_test_descs[] for algs
that are allowed to be used when in fips mode.

One caveat: des isn't actually allowed anymore, but des (and thus also
ecb(des)) has to be permitted, because disallowing them results in
des3_ede being unable to properly register (see des module init func).

Also, crc32 isn't technically on the fips approved list, but I think
it gets used in various places that necessitate it being allowed.

This list is based on
http://csrc.nist.gov/groups/STM/cavp/index.html

Important note: allowed/approved here does NOT mean "validated", just
that its an alg that *could* be validated.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:48 +10:00
Jarod Wilson
f7cb80f2b9 crypto: testmgr - Add ctr(aes) test vectors
Now with multi-block test vectors, all from SP800-38A, Appendix F.5.
Also added ctr(aes) to case 10 in tcrypt.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:46 +10:00
Herbert Xu
f8b0d4d09d crypto: testmgr - Dynamically allocate xbuf and axbuf
We currently allocate temporary memory that is used for testing
statically.  This renders the testing engine non-reentrant. As
algorithms may nest, i.e., one may construct another in order to
carry out a part of its operation, this is unacceptable.  For
example, it has been reported that an AEAD implementation allocates
a cipher in its setkey function, which causes it to fail during
testing as the temporary memory is overwritten.

This patch replaces the static memory with dynamically allocated
buffers.  We need a maximum of 16 pages so this slightly increases
the chances of an algorithm failing due to memory shortage.
However, as testing usually occurs at registration, this shouldn't
be a big problem.

Reported-by: Shasi Pulijala <spulijala@amcc.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:45 +10:00
Jarod Wilson
29ecd4ab3d crypto: testmgr - Print self-test pass notices in fips mode
According to our FIPS CAVS testing lab guru, when we're in fips mode,
we must print out notices of successful self-test completion for
every alg to be compliant.

New and improved v2, without strncmp crap. Doesn't need to touch a flag
though, due to not moving the notest label around anymore.

Applies atop '[PATCH v2] crypto: catch base cipher self-test failures
in fips mode'.

Personally, I wouldn't mind seeing this info printed out regardless of
whether or not we're in fips mode, I think its useful info, but will
stick with only in fips mode for now.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:43 +10:00
Jarod Wilson
941fb3287c crypto: testmgr - Catch base cipher self-test failures in fips mode
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:42 +10:00
Jarod Wilson
e08ca2da39 crypto: testmgr - Add ansi_cprng test vectors
Add ANSI X9.31 Continuous Pseudo-Random Number Generator (AES mode),
aka 'ansi_cprng' test vectors, taken from Appendix B.2.9 and B.2.10
of the NIST RNGVS document, found here:
    http://csrc.nist.gov/groups/STM/cavp/documents/rng/RNGVS.pdf

Successfully tested against both the cryptodev-2.6 tree and a Red
Hat Enterprise Linux 5.4 kernel, via 'modprobe tcrypt mode=150'.

The selection of 150 was semi-arbitrary, didn't seem like it should
go any place in particular, so I started a new range for rng tests.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:40 +10:00
Jarod Wilson
7647d6ce20 crypto: testmgr - Add infrastructure for ansi_cprng self-tests
Add some necessary infrastructure to make it possible to run
self-tests for ansi_cprng. The bits are likely very specific
to the ANSI X9.31 CPRNG in AES mode, and thus perhaps should
be named more specifically if/when we grow additional CPRNG
support...

Successfully tested against the cryptodev-2.6 tree and a
Red Hat Enterprise Linux 5.x kernel with the follow-on
patch that adds the actual test vectors.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:38 +10:00
Jarod Wilson
5d667322a2 crypto: testmgr - Add self-tests for rfc4309(ccm(aes))
Add an array of encryption and decryption + verification self-tests
for rfc4309(ccm(aes)).

Test vectors all come from sample FIPS CAVS files provided to
Red Hat by a testing lab. Unfortunately, all the published sample
vectors in RFC 3610 and NIST Special Publication 800-38C contain nonce
lengths that the kernel's rfc4309 implementation doesn't support, so
while using some public domain vectors would have been preferred, its
not possible at this time.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:36 +10:00
Jarod Wilson
e44a1b44c3 crypto: testmgr - Handle AEAD test vectors expected to fail verification
Add infrastructure to tcrypt/testmgr to support handling ccm decryption
test vectors that are expected to fail verification.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:35 +10:00
Geert Uytterhoeven
2f6ceb7933 crypto: pcomp - pcompress.c should include crypto/internal/compress.h
make C=1:
| crypto/pcompress.c:77:5: warning: symbol 'crypto_register_pcomp' was not declared. Should it be static?
| crypto/pcompress.c:89:5: warning: symbol 'crypto_unregister_pcomp' was not declared. Should it be static?

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:20 +10:00
Geert Uytterhoeven
c79cf91006 crypto: testmgr - Kill test_comp() sparse warnings
make C=1:
| crypto/testmgr.c:846:45: warning: incorrect type in argument 5 (different signedness)
| crypto/testmgr.c:846:45:    expected unsigned int *dlen
| crypto/testmgr.c:846:45:    got int *<noident>
| crypto/testmgr.c:878:47: warning: incorrect type in argument 5 (different signedness)
| crypto/testmgr.c:878:47:    expected unsigned int *dlen
| crypto/testmgr.c:878:47:    got int *<noident>

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:18 +10:00
Huang Ying
2cf4ac8beb crypto: aes-ni - Add support for more modes
Because kernel_fpu_begin() and kernel_fpu_end() operations are too
slow, the performance gain of general mode implementation + aes-aesni
is almost all compensated.

The AES-NI support for more modes are implemented as follow:

- Add a new AES algorithm implementation named __aes-aesni without
  kernel_fpu_begin/end()

- Use fpu(<mode>(AES)) to provide kenrel_fpu_begin/end() invoking

- Add <mode>(AES) ablkcipher, which uses cryptd(fpu(<mode>(AES))) to
  defer cryption to cryptd context in soft_irq context.

Now the ctr, lrw, pcbc and xts support are added.

Performance testing based on dm-crypt shows that cryption time can be
reduced to 50% of general mode implementation + aes-aesni implementation.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:16 +10:00
Huang Ying
150c7e8552 crypto: fpu - Add template for blkcipher touching FPU
Blkcipher touching FPU need to be enclosed by kernel_fpu_begin() and
kernel_fpu_end(). If they are invoked in cipher algorithm
implementation, they will be invoked for each block, so that
performance will be hurt, because they are "slow" operations. This
patch implements "fpu" template, which makes these operations to be
invoked for each request.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:15 +10:00
Huang Ying
505fd21d61 crypto: cryptd - Use nivcipher in cryptd_alloc_ablkcipher
Use crypto_alloc_base() instead of crypto_alloc_ablkcipher() to
allocate underlying tfm in cryptd_alloc_ablkcipher. Because
crypto_alloc_ablkcipher() prefer GENIV encapsulated crypto instead of
raw one, while cryptd_alloc_ablkcipher needed the raw one.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:13 +10:00
Johannes Weiner
811d8f0626 crypto: api - Use kzfree
Use kzfree() instead of memset() + kfree().

Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:11 +10:00
Frank Seidel
376bacb0a2 crypto: tcrypt - Reduce stack size
Applying kernel janitors todos (printk calls need KERN_*
constants on linebeginnings, reduce stack footprint where
possible) to tcrypts test_hash_speed (where stacks
memory footprint was very high (on i386 1184 bytes to
160 now).

Signed-off-by: Frank Seidel <frank@f-seidel.de>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-06-02 14:04:09 +10:00
Herbert Xu
d315a0e09f crypto: hash - Fix handling of sg entry that crosses page boundary
A quirk that we've always supported is having an sg entry that's
bigger than a page, or more generally an sg entry that crosses
page boundaries.  Even though it would be better to explicitly have
to sg entries for this, we need to support it for the existing users,
in particular, IPsec.

The new ahash sg walking code did try to handle this, but there was
a bug where we didn't increment the page so kept on walking on the
first page over an dover again.

This patch fixes it.

Tested-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-05-31 23:09:22 +10:00
Linus Torvalds
cd208bcc7c Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
  crypto: padlock - Revert aes-all alias to aes
  crypto: api - Fix algorithm module auto-loading
  crypto: eseqiv - Fix IV generation for sync algorithms
  crypto: ixp4xx - check firmware for crypto support
2009-05-17 15:48:05 -07:00
Herbert Xu
37fc334cc8 crypto: api - Fix algorithm module auto-loading
The commit a760a6656e (crypto:
api - Fix module load deadlock with fallback algorithms) broke
the auto-loading of algorithms that require fallbacks.  The
problem is that the fallback mask check is missing an and which
cauess bits that should be considered to interfere with the
result.

Reported-by: Chuck Ebbert <cebbert@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-04-21 13:27:16 +08:00
Steffen Klassert
abe5fa7899 crypto: eseqiv - Fix IV generation for sync algorithms
If crypto_ablkcipher_encrypt() returns synchronous,
eseqiv_complete2() is called even if req->giv is already the
pointer to the generated IV. The generated IV is overwritten
with some random data in this case. This patch fixes this by
calling eseqiv_complete2() just if the generated IV has to be
copied to req->giv.

Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-04-15 20:45:03 +08:00
Linus Torvalds
133e2a3164 Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx:
  dma: Add SoF and EoF debugging to ipu_idmac.c, minor cleanup
  dw_dmac: add cyclic API to DW DMA driver
  dmaengine: Add privatecnt to revert DMA_PRIVATE property
  dmatest: add dma interrupts and callbacks
  dmatest: add xor test
  dmaengine: allow dma support for async_tx to be toggled
  async_tx: provide __async_inline for HAS_DMA=n archs
  dmaengine: kill some unused headers
  dmaengine: initialize tx_list in dma_async_tx_descriptor_init
  dma: i.MX31 IPU DMA robustness improvements
  dma: improve section assignment in i.MX31 IPU DMA driver
  dma: ipu_idmac driver cosmetic clean-up
  dmaengine: fail device registration if channel registration fails
2009-04-03 12:13:45 -07:00
Linus Torvalds
c54c4dec61 Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
  crypto: ixp4xx - Fix handling of chained sg buffers
  crypto: shash - Fix unaligned calculation with short length
  hwrng: timeriomem - Use phys address rather than virt
2009-04-03 09:45:53 -07:00
Linus Torvalds
223cdea4c4 Merge branch 'for-linus' of git://neil.brown.name/md
* 'for-linus' of git://neil.brown.name/md: (53 commits)
  md/raid5 revise rules for when to update metadata during reshape
  md/raid5: minor code cleanups in make_request.
  md: remove CONFIG_MD_RAID_RESHAPE config option.
  md/raid5: be more careful about write ordering when reshaping.
  md: don't display meaningless values in sysfs files resync_start and sync_speed
  md/raid5: allow layout and chunksize to be changed on active array.
  md/raid5: reshape using largest of old and new chunk size
  md/raid5: prepare for allowing reshape to change layout
  md/raid5: prepare for allowing reshape to change chunksize.
  md/raid5: clearly differentiate 'before' and 'after' stripes during reshape.
  Documentation/md.txt update
  md: allow number of drives in raid5 to be reduced
  md/raid5: change reshape-progress measurement to cope with reshaping backwards.
  md: add explicit method to signal the end of a reshape.
  md/raid5: enhance raid5_size to work correctly with negative delta_disks
  md/raid5: drop qd_idx from r6_state
  md/raid6: move raid6 data processing to raid6_pq.ko
  md: raid5 run(): Fix max_degraded for raid level 4.
  md: 'array_size' sysfs attribute
  md: centralize ->array_sectors modifications
  ...
2009-04-03 09:08:19 -07:00
NeilBrown
bff61975b3 md: move lots of #include lines out of .h files and into .c
This makes the includes more explicit, and is preparation for moving
md_k.h to drivers/md/md.h

Remove include/raid/md.h as its only remaining use was to #include
other files.

Signed-off-by: NeilBrown <neilb@suse.de>
2009-03-31 14:33:13 +11:00
Yehuda Sadeh
f4f689933c crypto: shash - Fix unaligned calculation with short length
When the total length is shorter than the calculated number of unaligned bytes, the call to shash->update breaks. For example, calling crc32c on unaligned buffer with length of 1 can result in a system crash.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-03-27 13:03:51 +08:00
Linus Torvalds
562f477a54 Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (29 commits)
  crypto: sha512-s390 - Add missing block size
  hwrng: timeriomem - Breaks an allyesconfig build on s390:
  nlattr: Fix build error with NET off
  crypto: testmgr - add zlib test
  crypto: zlib - New zlib crypto module, using pcomp
  crypto: testmgr - Add support for the pcomp interface
  crypto: compress - Add pcomp interface
  netlink: Move netlink attribute parsing support to lib
  crypto: Fix dead links
  hwrng: timeriomem - New driver
  crypto: chainiv - Use kcrypto_wq instead of keventd_wq
  crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq
  crypto: api - Use dedicated workqueue for crypto subsystem
  crypto: testmgr - Test skciphers with no IVs
  crypto: aead - Avoid infinite loop when nivaead fails selftest
  crypto: skcipher - Avoid infinite loop when cipher fails selftest
  crypto: api - Fix crypto_alloc_tfm/create_create_tfm return convention
  crypto: api - crypto_alg_mod_lookup either tested or untested
  crypto: amcc - Add crypt4xx driver
  crypto: ansi_cprng - Add maintainer
  ...
2009-03-26 11:04:34 -07:00
Dan Williams
729b5d1b8e dmaengine: allow dma support for async_tx to be toggled
Provide a config option for blocking the allocation of dma channels to
the async_tx api.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-25 09:13:25 -07:00
Dan Williams
06164f3194 async_tx: provide __async_inline for HAS_DMA=n archs
To allow an async_tx routine to be compiled away on HAS_DMA=n arch it
needs to be declared __always_inline otherwise the compiler may emit
code and cause a link error.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-25 09:13:25 -07:00
Geert Uytterhoeven
0c01aed50d crypto: testmgr - add zlib test
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-03-04 15:42:15 +08:00
Geert Uytterhoeven
bf68e65ec9 crypto: zlib - New zlib crypto module, using pcomp
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Cc: James Morris <jmorris@namei.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-03-04 15:16:19 +08:00
Geert Uytterhoeven
8064efb874 crypto: testmgr - Add support for the pcomp interface
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-03-04 15:16:18 +08:00
Geert Uytterhoeven
a1d2f09544 crypto: compress - Add pcomp interface
The current "comp" crypto interface supports one-shot (de)compression only,
i.e. the whole data buffer to be (de)compressed must be passed at once, and
the whole (de)compressed data buffer will be received at once.
In several use-cases (e.g. compressed file systems that store files in big
compressed blocks), this workflow is not suitable.
Furthermore, the "comp" type doesn't provide for the configuration of
(de)compression parameters, and always allocates workspace memory for both
compression and decompression, which may waste memory.

To solve this, add a "pcomp" partial (de)compression interface that provides
the following operations:
  - crypto_compress_{init,update,final}() for compression,
  - crypto_decompress_{init,update,final}() for decompression,
  - crypto_{,de}compress_setup(), to configure (de)compression parameters
    (incl. allocating workspace memory).

The (de)compression methods take a struct comp_request, which was mimicked
after the z_stream object in zlib, and contains buffer pointer and length
pairs for input and output.

The setup methods take an opaque parameter pointer and length pair. Parameters
are supposed to be encoded using netlink attributes, whose meanings depend on
the actual (name of the) (de)compression algorithm.

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-03-04 15:05:33 +08:00
Adrian-Ken Rueegsegger
8c882f6413 crypto: Fix dead links
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-03-04 14:43:52 +08:00
Herbert Xu
a760a6656e crypto: api - Fix module load deadlock with fallback algorithms
With the mandatory algorithm testing at registration, we have
now created a deadlock with algorithms requiring fallbacks.
This can happen if the module containing the algorithm requiring
fallback is loaded first, without the fallback module being loaded
first.  The system will then try to test the new algorithm, find
that it needs to load a fallback, and then try to load that.

As both algorithms share the same module alias, it can attempt
to load the original algorithm again and block indefinitely.

As algorithms requiring fallbacks are a special case, we can fix
this by giving them a different module alias than the rest.  Then
it's just a matter of using the right aliases according to what
algorithms we're trying to find.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-26 14:06:31 +08:00
Lee Nipper
bb402f16ec crypto: ahash - Fix digest size in /proc/crypto
crypto_ahash_show changed to use cra_ahash for digestsize reference.

Signed-off-by: Lee Nipper <lee.nipper@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:46:26 +08:00
Huang Ying
0a2e821d62 crypto: chainiv - Use kcrypto_wq instead of keventd_wq
keventd_wq has potential starvation problem, so use dedicated
kcrypto_wq instead.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:44:02 +08:00
Huang Ying
254eff7714 crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq
Original cryptd thread implementation has scalability issue, this
patch solve the issue with a per-CPU thread implementation.

struct cryptd_queue is defined to be a per-CPU queue, which holds one
struct cryptd_cpu_queue for each CPU. In struct cryptd_cpu_queue, a
struct crypto_queue holds all requests for the CPU, a struct
work_struct is used to run all requests for the CPU.

Testing based on dm-crypt on an Intel Core 2 E6400 (two cores) machine
shows 19.2% performance gain. The testing script is as follow:

-------------------- script begin ---------------------------
#!/bin/sh

dmc_create()
{
        # Create a crypt device using dmsetup
        dmsetup create $2 --table "0 `blockdev --getsize $1` crypt cbc(aes-asm)?cryptd?plain:plain babebabebabebabebabebabebabebabe 0 $1 0"
}

dmsetup remove crypt0
dmsetup remove crypt1

dd if=/dev/zero of=/dev/ram0 bs=1M count=4 >& /dev/null
dd if=/dev/zero of=/dev/ram1 bs=1M count=4 >& /dev/null

dmc_create /dev/ram0 crypt0
dmc_create /dev/ram1 crypt1

cat >tr.sh <<EOF
#!/bin/sh

for n in \$(seq 10); do
        dd if=/dev/dm-0 of=/dev/null >& /dev/null &
        dd if=/dev/dm-1 of=/dev/null >& /dev/null &
done
wait
EOF

for n in $(seq 10); do
        /usr/bin/time sh tr.sh
done
rm tr.sh
-------------------- script end   ---------------------------

The separator of dm-crypt parameter is changed from "-" to "?", because
"-" is used in some cipher driver name too, and cryptds need to specify
cipher driver name instead of cipher name.

The test result on an Intel Core2 E6400 (two cores) is as follow:

without patch:
-----------------wo begin --------------------------
0.04user 0.38system 0:00.39elapsed 107%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6566minor)pagefaults 0swaps
0.07user 0.35system 0:00.35elapsed 121%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6567minor)pagefaults 0swaps
0.06user 0.34system 0:00.30elapsed 135%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6562minor)pagefaults 0swaps
0.05user 0.37system 0:00.36elapsed 119%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6607minor)pagefaults 0swaps
0.06user 0.36system 0:00.35elapsed 120%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6562minor)pagefaults 0swaps
0.05user 0.37system 0:00.31elapsed 136%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6594minor)pagefaults 0swaps
0.04user 0.34system 0:00.30elapsed 126%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6597minor)pagefaults 0swaps
0.06user 0.32system 0:00.31elapsed 125%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6571minor)pagefaults 0swaps
0.06user 0.34system 0:00.31elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6581minor)pagefaults 0swaps
0.05user 0.38system 0:00.31elapsed 138%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6600minor)pagefaults 0swaps
-----------------wo end   --------------------------


with patch:
------------------w begin --------------------------
0.02user 0.31system 0:00.24elapsed 141%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6554minor)pagefaults 0swaps
0.05user 0.34system 0:00.31elapsed 127%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6606minor)pagefaults 0swaps
0.07user 0.33system 0:00.26elapsed 155%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6559minor)pagefaults 0swaps
0.07user 0.32system 0:00.26elapsed 151%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6562minor)pagefaults 0swaps
0.05user 0.34system 0:00.26elapsed 150%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6603minor)pagefaults 0swaps
0.03user 0.36system 0:00.31elapsed 124%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6562minor)pagefaults 0swaps
0.04user 0.35system 0:00.26elapsed 147%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6586minor)pagefaults 0swaps
0.03user 0.37system 0:00.27elapsed 146%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6562minor)pagefaults 0swaps
0.04user 0.36system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6594minor)pagefaults 0swaps
0.04user 0.35system 0:00.26elapsed 154%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6557minor)pagefaults 0swaps
------------------w end   --------------------------

The middle value of elapsed time is:
wo cryptwq: 0.31
w  cryptwq: 0.26

The performance gain is about (0.31-0.26)/0.26 = 0.192.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:42:19 +08:00
Huang Ying
25c38d3fb9 crypto: api - Use dedicated workqueue for crypto subsystem
Use dedicated workqueue for crypto subsystem

A dedicated workqueue named kcrypto_wq is created to be used by crypto
subsystem. The system shared keventd_wq is not suitable for
encryption/decryption, because of potential starvation problem.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-19 14:33:40 +08:00
Herbert Xu
6fe4a28d88 crypto: testmgr - Test skciphers with no IVs
As it is an skcipher with no IV escapes testing altogether because
we only test givcipher objects.  This patch fixes the bypass logic
to test these algorithms.

Conversely, we're currently testing nivaead algorithms with IVs,
which would have deadlocked had it not been for the fact that no
nivaead algorithms have any test vectors.  This patch also fixes
that case.

Both fixes are ugly as hell, but this ugliness should hopefully
disappear once we move them into the per-type code (i.e., the
AEAD test would live in aead.c and the skcipher stuff in ablkcipher.c).

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 21:41:29 +08:00
Herbert Xu
5852ae4242 crypto: aead - Avoid infinite loop when nivaead fails selftest
When an aead constructed through crypto_nivaead_default fails
its selftest, we'll loop forever trying to construct new aead
objects but failing because it already exists.

The crux of the issue is that once an aead fails the selftest,
we'll ignore it on the next run through crypto_aead_lookup and
attempt to construct a new aead.

We should instead return an error to the caller if we find an
an that has failed the test.

This bug hasn't manifested itself yet because we don't have any
test vectors for the existing nivaead algorithms.  They're tested
through the underlying algorithms only.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 21:21:24 +08:00
Herbert Xu
b170a137f4 crypto: skcipher - Avoid infinite loop when cipher fails selftest
When an skcipher constructed through crypto_givcipher_default fails
its selftest, we'll loop forever trying to construct new skcipher
objects but failing because it already exists.

The crux of the issue is that once a givcipher fails the selftest,
we'll ignore it on the next run through crypto_skcipher_lookup and
attempt to construct a new givcipher.

We should instead return an error to the caller if we find a
givcipher that has failed the test.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 21:20:06 +08:00
Herbert Xu
3f683d6175 crypto: api - Fix crypto_alloc_tfm/create_create_tfm return convention
This is based on a report and patch by Geert Uytterhoeven.

The functions crypto_alloc_tfm and create_create_tfm return a
pointer that needs to be adjusted by the caller when successful
and otherwise an error value.  This means that the caller has
to check for the error and only perform the adjustment if the
pointer returned is valid.

Since all callers want to make the adjustment and we know how
to adjust it ourselves, it's much easier to just return adjusted
pointer directly.

The only caveat is that we have to return a void * instead of
struct crypto_tfm *.  However, this isn't that bad because both
of these functions are for internal use only (by types code like
shash.c, not even algorithms code).

This patch also moves crypto_alloc_tfm into crypto/internal.h
(crypto_create_tfm is already there) to reflect this.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 16:56:59 +08:00
Herbert Xu
ff753308d2 crypto: api - crypto_alg_mod_lookup either tested or untested
As it stands crypto_alg_mod_lookup will search either tested or
untested algorithms, but never both at the same time.  However,
we need exactly that when constructing givcipher and aead so
this patch adds support for that by setting the tested bit in
type but clearing it in mask.  This combination is currently
unused.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 16:49:43 +08:00
Neil Horman
c5b1e545a5 crypto: ansi_cprng - Panic on CPRNG test failure when in FIPS mode
FIPS 140-2 specifies that all access to various cryptographic modules be
prevented in the event that any of the provided self tests fail on the various
implemented algorithms.  We already panic when any of the test in testmgr.c
fail when we are operating in fips mode.  The continuous test in the cprng here
was missed when that was implmented.  This code simply checks for the
fips_enabled flag if the test fails, and warns us via syslog or panics the box
accordingly.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 16:48:07 +08:00
Neil Horman
d7992f42c6 crypto: ansi_cprng - Force reset on allocation
Pseudo RNGs provide predictable outputs based on input parateters {key, V, DT},
the idea behind them is that only the user should know what the inputs are.
While its nice to have default known values for testing purposes, it seems
dangerous to allow the use of those default values without some sort of safety
measure in place, lest an attacker easily guess the output of the cprng.  This
patch forces the NEED_RESET flag on when allocating a cprng context, so that any
user is forced to reseed it before use.  The defaults can still be used for
testing, but this will prevent their inadvertent use, and be more secure.
   
Signed-off-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 16:48:06 +08:00
Huang Ying
54b6a1bd53 crypto: aes-ni - Add support to Intel AES-NI instructions for x86_64 platform
Intel AES-NI is a new set of Single Instruction Multiple Data (SIMD)
instructions that are going to be introduced in the next generation of
Intel processor, as of 2009. These instructions enable fast and secure
data encryption and decryption, using the Advanced Encryption Standard
(AES), defined by FIPS Publication number 197.  The architecture
introduces six instructions that offer full hardware support for
AES. Four of them support high performance data encryption and
decryption, and the other two instructions support the AES key
expansion procedure.

The white paper can be downloaded from:

http://softwarecommunity.intel.com/isn/downloads/intelavx/AES-Instructions-Set_WP.pdf

AES may be used in soft_irq context, but MMX/SSE context can not be
touched safely in soft_irq context. So in_interrupt() is checked, if
in IRQ or soft_irq context, the general x86_64 implementation are used
instead.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 16:48:06 +08:00
Huang Ying
1cac2cbc76 crypto: cryptd - Add support to access underlying blkcipher
cryptd_alloc_ablkcipher() will allocate a cryptd-ed ablkcipher for
specified algorithm name. The new allocated one is guaranteed to be
cryptd-ed ablkcipher, so the blkcipher underlying can be gotten via
cryptd_ablkcipher_child().

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 16:48:05 +08:00
Herbert Xu
1693531e9e crypto: shash - Remove superfluous check in init_tfm
We're currently checking the frontend type in init_tfm.  This is
completely pointless because the fact that we're called at all
means that the frontend is ours so the type must match as well.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-18 16:48:05 +08:00
Herbert Xu
8eb2dfac41 crypto: lrw - Fix big endian support
It turns out that LRW has never worked properly on big endian.
This was never discussed because nobody actually used it that
way.  In fact, it was only discovered when Geert Uytterhoeven
loaded it through tcrypt which failed the test on it.

The fix is straightforward, on big endian the to find the nth
bit we should be grouping them by words instead of bytes.  So
setbit128_bbe should xor with 128 - BITS_PER_LONG instead of
128 - BITS_PER_BYTE == 0x78.

Tested-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-17 20:00:11 +08:00
Herbert Xu
4f3e797ad0 crypto: scatterwalk - Avoid flush_dcache_page on slab pages
It's illegal to call flush_dcache_page on slab pages on a number
of architectures.  So this patch avoids doing so if PageSlab is
true.

In future we can move the flush_dcache_page call to those page
cache users that actually need it.

Reported-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-09 14:30:25 +11:00
Herbert Xu
7b2cd92adc crypto: api - Fix zeroing on free
Geert Uytterhoeven pointed out that we're not zeroing all the
memory when freeing a transform.  This patch fixes it by calling
ksize to ensure that we zero everything in sight.

Reported-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-05 16:48:53 +11:00
Adrian-Ken Rueegsegger
4abfd73e34 crypto: shash - Fix module refcount
Module reference counting for shash is incorrect: when
a new shash transformation is created the refcount is not
increased as it should.

Signed-off-by: Adrian-Ken Rueegsegger <rueegsegger@swiss-it.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-05 16:19:31 +11:00
Herbert Xu
b8e15992b4 crypto: api - Fix algorithm test race that broke aead initialisation
When we complete a test we'll notify everyone waiting on it, drop
the mutex, and then remove the test larval (after reacquiring the
mutex).  If one of the notified parties tries to register another
algorithm with the same driver name prior to the removal of the
test larval, they will fail with EEXIST as only one algorithm of
a given name can be tested at any time.

This broke the initialisation of aead and givcipher algorithms as
they will register two algorithms with the same driver name, in
sequence.

This patch fixes the problem by marking the larval as dead before
we drop the mutex, and also ignoring all dead or dying algorithms
on the registration path.

Tested-by: Andreas Steffen <andreas.steffen@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-01-28 14:09:59 +11:00
Jarod Wilson
516280e735 crypto: ccm - Fix handling of null assoc data
Its a valid use case to have null associated data in a ccm vector, but
this case isn't being handled properly right now.

The following ccm decryption/verification test vector, using the
rfc4309 implementation regularly triggers a panic, as will any
other vector with null assoc data:

* key: ab2f8a74b71cd2b1ff802e487d82f8b9
* iv: c6fb7d800d13abd8a6b2d8
* Associated Data: [NULL]
* Tag Length: 8
* input: d5e8939fc7892e2b

The resulting panic looks like so:

Unable to handle kernel paging request at ffff810064ddaec0 RIP: 
 [<ffffffff8864c4d7>] :ccm:get_data_to_compute+0x1a6/0x1d6
PGD 8063 PUD 0 
Oops: 0002 [1] SMP 
last sysfs file: /module/libata/version
CPU 0
Modules linked in: crypto_tester_kmod(U) seqiv krng ansi_cprng chainiv rng ctr aes_generic aes_x86_64 ccm cryptomgr testmgr_cipher testmgr aead crypto_blkcipher crypto_a
lgapi des ipv6 xfrm_nalgo crypto_api autofs4 hidp l2cap bluetooth nfs lockd fscache nfs_acl sunrpc ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink xt_
tcpudp iptable_filter ip_tables x_tables dm_mirror dm_log dm_multipath scsi_dh dm_mod video hwmon backlight sbs i2c_ec button battery asus_acpi acpi_memhotplug ac lp sg 
snd_intel8x0 snd_ac97_codec ac97_bus snd_seq_dummy snd_seq_oss joydev snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss ide_cd snd_pcm floppy parport_p
c shpchp e752x_edac snd_timer e1000 i2c_i801 edac_mc snd soundcore snd_page_alloc i2c_core cdrom parport serio_raw pcspkr ata_piix libata sd_mod scsi_mod ext3 jbd uhci_h
cd ohci_hcd ehci_hcd
Pid: 12844, comm: crypto-tester Tainted: G      2.6.18-128.el5.fips1 #1
RIP: 0010:[<ffffffff8864c4d7>]  [<ffffffff8864c4d7>] :ccm:get_data_to_compute+0x1a6/0x1d6
RSP: 0018:ffff8100134434e8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8100104898b0 RCX: ffffffffab6aea10
RDX: 0000000000000010 RSI: ffff8100104898c0 RDI: ffff810064ddaec0
RBP: 0000000000000000 R08: ffff8100104898b0 R09: 0000000000000000
R10: ffff8100103bac84 R11: ffff8100104898b0 R12: ffff810010489858
R13: ffff8100104898b0 R14: ffff8100103bac00 R15: 0000000000000000
FS:  00002ab881adfd30(0000) GS:ffffffff803ac000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff810064ddaec0 CR3: 0000000012a88000 CR4: 00000000000006e0
Process crypto-tester (pid: 12844, threadinfo ffff810013442000, task ffff81003d165860)
Stack:  ffff8100103bac00 ffff8100104898e8 ffff8100134436f8 ffffffff00000000
 0000000000000000 ffff8100104898b0 0000000000000000 ffff810010489858
 0000000000000000 ffff8100103bac00 ffff8100134436f8 ffffffff8864c634
Call Trace:
 [<ffffffff8864c634>] :ccm:crypto_ccm_auth+0x12d/0x140
 [<ffffffff8864cf73>] :ccm:crypto_ccm_decrypt+0x161/0x23a
 [<ffffffff88633643>] :crypto_tester_kmod:cavs_test_rfc4309_ccm+0x4a5/0x559
[...]

The above is from a RHEL5-based kernel, but upstream is susceptible too.

The fix is trivial: in crypto/ccm.c:crypto_ccm_auth(), pctx->ilen contains
whatever was in memory when pctx was allocated if assoclen is 0. The tested
fix is to simply add an else clause setting pctx->ilen to 0 for the
assoclen == 0 case, so that get_data_to_compute() doesn't try doing
things its not supposed to.

Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-01-27 17:11:15 +11:00
Herbert Xu
bac1b5c469 crypto: blkcipher - Fix WARN_ON handling in walk_done
When we get left-over bits from a slow walk, it means that the
underlying cipher has gone troppo.  However, as we're handling
that case we should ensure that the caller terminates the walk.

This patch does this by setting walk->nbytes to zero.

Reported-by: Roel Kluin <roel.kluin@gmail.com>
Reported-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-01-27 17:11:13 +11:00
Herbert Xu
29b37f4212 crypto: authenc - Fix zero-length IV crash
As it is if an algorithm with a zero-length IV is used (e.g.,
NULL encryption) with authenc, authenc may generate an SG entry
of length zero, which will trigger a BUG check in the hash layer.

This patch fixes it by skipping the IV SG generation if the IV
size is zero.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-01-15 15:33:49 +11:00
Dan Williams
209b84a88f dmaengine: replace dma_async_client_register with dmaengine_get
Now that clients no longer need to be notified of channel arrival
dma_async_client_register can simply increment the dmaengine_ref_count.

Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-01-06 11:38:17 -07:00
Dan Williams
2ba05622b8 dmaengine: provide a common 'issue_pending_all' implementation
async_tx and net_dma each have open-coded versions of issue_pending_all,
so provide a common routine in dmaengine.

The implementation needs to walk the global device list, so implement
rcu to allow dma_issue_pending_all to run lockless.  Clients protect
themselves from channel removal events by holding a dmaengine reference.

Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-01-06 11:38:14 -07:00
Dan Williams
bec085134e dmaengine: centralize channel allocation, introduce dma_find_channel
Allowing multiple clients to each define their own channel allocation
scheme quickly leads to a pathological situation.  For memory-to-memory
offload all clients can share a central allocator.

This simply moves the existing async_tx allocator to dmaengine with
minimal fixups:
* async_tx.c:get_chan_ref_by_cap --> dmaengine.c:nth_chan
* async_tx.c:async_tx_rebalance --> dmaengine.c:dma_channel_rebalance
* split out common code from async_tx.c:__async_tx_find_channel -->
  dma_find_channel

Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-01-06 11:38:14 -07:00
Dan Williams
6f49a57aa5 dmaengine: up-level reference counting to the module level
Simply, if a client wants any dmaengine channel then prevent all dmaengine
modules from being removed.  Once the clients are done re-enable module
removal.

Why?, beyond reducing complication:
1/ Tracking reference counts per-transaction in an efficient manner, as
   is currently done, requires a complicated scheme to avoid cache-line
   bouncing effects.
2/ Per-transaction ref-counting gives the false impression that a
   dma-driver can be gracefully removed ahead of its user (net, md, or
   dma-slave)
3/ None of the in-tree dma-drivers talk to hot pluggable hardware, but
   if such an engine were built one day we still would not need to notify
   clients of remove events.  The driver can simply return NULL to a
   ->prep() request, something that is much easier for a client to handle.

Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-01-06 11:38:14 -07:00
Dan Williams
07f2211e4f dmaengine: remove dependency on async_tx
async_tx.ko is a consumer of dma channels.  A circular dependency arises
if modules in drivers/dma rely on common code in async_tx.ko.  It
prevents either module from being unloaded.

Move dma_wait_for_async_tx and async_tx_run_dependencies to dmaeninge.o
where they should have been from the beginning.

Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-01-05 18:10:19 -07:00
Herbert Xu
0ee4a96902 crypto: aes - Precompute tables
The tables used by the various AES algorithms are currently
computed at run-time.  This has created an init ordering problem
because some AES algorithms may be registered before the tables
have been initialised.

This patch gets around this whole thing by precomputing the tables.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-12-25 11:05:13 +11:00