Stefan Arentz

I found an interesting quote about SOAP API usage for Amazon’s Simple Storage Service:

We are continuing to support our existing SOAP APIs. That said, given that SOAP requests currently make up < 1% of our request volume in the US, we made the pragmatic decision to not extend the SOAP APIs as part of our EU launch. This allowed us to make our EU location available sooner than we could have otherwise.

It does not specifically say whether this is just for S3 but I think it is a good indication of REST popularity.

All my AWS code is also using the REST API’s. Much simpler and you don’t have to use bloated SOAP frameworks to get things done.

Spring 2.5 has some great improvements that make configuration much simpler. One that I very much like is the auto detection of components in the classpath. The Spring 2.5 documentation on this subject is worth reading. It also gives a nice and simple example.

For the 2.0 branch of the Polar Rose Web Service Foundation I have added an AnnotationWebServiceController that uses this new Spring 2.5 functionality.

This makes creating a web service very very simple. All you have to do to get going with a public web service is annotate your Actions with @WebServiceAction and tell Spring to scan for component.

This is an example action:

package com.foo.action.
@WebServiceAction(name = "Hello", version = "2008-04-01")
public class HelloAction implements WebServiceActionHandler
{
    public Object execute(WebServiceActionContext context, Parameters parameters)
        throws ActionHandlerException
    {
        return “Hello, ” + parameters.getName();
    }

    public static class Parameters
    {
        private String name;

        public String getName() {
            return name;
        }

        @WebServiceParameter
        public void setName(String name) {
            this.name = name;
        }
    }
}

The following Spring configuration is enough to setup the web service controller:

<beans ...>
  <context:component-scan base-package="com.foo.action"/>
  <bean id="annotationWebServiceController" class="com.polarrose.wsf.controller.AnnotationWebServiceController">
    <property name="version" value="2008-04-01"/>
  </bean>
</beans>

I think this is a really nice way to auto-detect and configure ‘top level’ components. It removes a lot of XML configuration. Which I don’t mind, but less is always better.

If you want to play with the framework, you can check it out from Google Code.

I’ve created a 2.0 branch in the polarrose-wsf project where I want to try out some new ideas.

Some things on the list:

• Merge the wsf and wsf-examples into one maven project as modules (already done)
• Merge the projects on Google code
• Add the wsf-client module to the project
• Depend on Spring 2.5 for auto detection of action handlers. Just like Spring 2.5 can do with @Controller annotated Spring MVC controllers.
• Remove as many dependencies as possible (commons-*)
• Remove the need for prototype scope beans by seperating handler and parameters (work in progress)
• Write proper JavaDoc for at least all top-level code

I’ve already made a rough implementation of the seperation of Action and Parameters task. The ActionHandler interface now looks like this:

public interface WebServiceActionHandler<Account extends WebServiceAccount, Parameters>
{
   Object execute(WebServiceActionContext<Account> context, Parameters parameters)
       throws ActionHandlerException;
}

The Parameters parameter is new. So that means you need to define a bean that contains the fields to which the request parameters are mapped. For example the AddNumbersAction action from the example project now becomes:

public class AddNumbersAction implements WebServiceActionHandler<DummyWebServiceAccount, AddNumbersAction.Parameters>
{
   public Object execute(WebServiceActionContext<DummyWebServiceAccount> context, Parameters parameters)
       throws ActionHandlerException
   {
       int total = 0;
       for (Integer number : parameters.getNumbers()) {
           total += number;
       }
       return total;
   }

   public static class Parameters
   {
       private List<Integer> numbers;

       public List<Integer> getNumbers() {
           return numbers;
       }

       @WebServiceParameter
       public void setNumbers(List<Integer> numbers) {
           this.numbers = numbers;
       }
   }
}

I like to use inner classes to keep the action and parameters together but if you don’t like that then you can also do this of course:

public static class AddNumbersParameters
{
   private List<Integer> numbers;

   public List<Integer> getNumbers() {
       return numbers;
   }

   @WebServiceParameter
   public void setNumbers(List<Integer> numbers) {
       this.numbers = numbers;
   }
}

public class AddNumbersAction implements WebServiceActionHandler<DummyWebServiceAccount, AddNumbersParameters>
{
   public Object execute(WebServiceActionContext<DummyWebServiceAccount> context, AddNumbersParameters parameters)
       throws ActionHandlerException
   {
       int total = 0;
       for (Integer number : parameters.getNumbers()) {
           total += number;
       }
       return total;
   }
}

Personally I think this is a little cleaner design than what I did for the 1.x version.

The speed also went up!

• 1.2-SNAPSHOT ~ 1200 requests/second
• 2.0-SNAPSHOT ~ 2100 requests/second

This is because the prototype beans are gone of course; they don’t have to be initialized every time. I can probably get the speed up more by caching some things that are looked up every time with introspection.

Spring 2.5 has a lot of interesting new features. I really hope to make WSF 2.0 much more Spring friendly and make it possible to get a web service up and running with as less code and configuration as possible.

This is rather serious. Consider this:

% dd if=/dev/zero of=data bs=1 count=2679
% sha256sum data
a25f4ccc56ddf88a4fb3e11baec5838c5181a496f376cdd569f8fc782f8fdcdf  data

A file of 2679 zeros and its SHA-256 hash. Nothing special.

Well, not exactly .. now look at the equivalent Python code:

$ python
Python 2.4.3 (#2, Oct  6 2006, 07:49:22)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Crypto.Hash import SHA256
>>> data = open("data", "rb").read()
>>> print SHA256.new(data).hexdigest()
14a59554dba8be8ac9aa03eea67a026b3773eb674d22719123be41ca774319a3

Oops!

So it seems the Python Crypto module generates wrong SHA-256 hashes for files that have a size of (N*8)-1. But only for files of reasonable size.

We discovered this the hard way because we cache images based on their SHA256 hash. All cache entries with wrong hashes had this (N*8)-1 file size.

I’m not sure if this is just limited to the Ubuntu Dapper Drake / X64 install that we use. For the record, this is the package I’m talking about:

$ aptitude show python2.4-crypto
Package: python2.4-crypto
State: installed
Automatically installed: yes
Version: 2.0.1+dfsg1-1ubuntu1
Priority: optional
Section: python
Maintainer: Andreas Rottmann 
Uncompressed Size: 557k
Depends: libc6 (>= 2.3.4-1), libgmp3c2, python2.4
Description: cryptographic algorithms and protocols for Python
 A collection of cryptographic algorithms and protocols, implemented for use from Python.
  Among the contents of the package: 

    * Hash functions: MD2, MD4.
    * Block encryption algorithms: AES, ARC2, Blowfish, CAST, DES, Triple-DES.
    * Stream encryption algorithms: ARC4, simple XOR.
    * Public-key algorithms: RSA, DSA, ElGamal, qNEW.
    * Protocols: All-or-nothing transforms, chaffing/winnowing.
    * Miscellaneous: RFC1751 module for converting 128-key keys into a
      set of English words, primality testing.

This sure taught us a lesson. Never trust (crypto) code until you have unit tested it yourself.

I’m trying to figure out where to report this.

Now that SimpleDB is out of it’s secret stealth beta I thought it was a good idea to make my Amazon Python modules available on Google Code.

I’ve written these for Polar Rose but as usual they are happy to release things like this under an open source license.

What the package contains is two things:

• Command line utilities to control SQS and SDB. I’ve found it extremely useful to be able to do things like creating domains and running queries from the command line during development or API exploration.
• Python modules to use SimpleDB, SQS and S3 in a Twisted environment. Twisted is a really nice Python framework to do asynchronous IO.

Good starting points on the wiki are:

• GettingStarted
• XmlRpcAndSimpleDatabaseExample
• SimpleDatabaseService
• SimpleDatabaseServiceCommandLineTools

The SimpleDB code is pretty stable and I encourage others to play with it. The SQS and S3 code needs some love. I have to check if it is still up to date with the current web service versions. I’ll do that soon and I will also try to make a really official release.

Enjoy!