Thursday, March 31, 2011

WCF Recommend approaches for serializing multiple objects

I am attempting to optimise around a possible bottleneck.

I have a server application that is serving objects from a database to applications remotely, who can work with 1 - n objects of 1 - n different types (where n can be a relatively high number) that all implement a common interface but may contain many unique properties on different types.

The client applications store the server objects in a local cache, until they are ready to persist them back, through the server, to the database.

This is being done currently in WCF with each class defining a DataContract.

Due to the possibly large amount of objects that may need to be passed back to the server (it changes depending on implementation), I would prefer not to do these all as individual calls any more, rather wrap all the objects in a single serialized (or better still compressed) stream and send them through as one connection to the server.

I can quite simply roll my own, but would prefer to use a recommended approach, and hoping someone may suggest one. If you can convince me, I am also willing to accept that my approach is probably not the best idea.

From stackoverflow
  • How high is "relatively high"?

    For example, one option that occurs is to use a wrapper object:

    [DataContract]
    public class Wrapper {
        [DataMember(Order = 1)]
        public List<Foo> Foos {get {...}}
    
        [DataMember(Order = 2)]
        public List<Bar> Bars {get {...}}
    
        [DataMember(Order = 3)]
        public List<Blop> Blops {get {...}}
    }
    

    Then you should be able to send a single message with any number of Foo, Bar and/or Blop records. My inclusion of the Order attribute was deliberate - if you want to reduce the size of the stream, you might consider protobuf-net - with the above layout, protobuf-net can hook into WCF simply by including [ProtoBehavior] on the method (in the operation-contract interface) that you want to attack (at both client and server). This switches the transfer to use google's "protocol buffers" binary format, using base-64 to encode. If you are using the basic-http binding, this can also use MTOM if enabled, so even the base-64 isn't an issue. Using this, you can get significant data transfer savings (~1/5th the space based on the numbers shown).

    (edit 1 - protobuf-net assumes that Foo, Bar and Blop also use the Order attribute)

    (edit 2 - note that you could always break up the request into a number of mid-size Wrapper messages, then call a method to apply all the changes once you have them at the server (presumably in a staging table in the database))

    johnc : Thanks for that Marc, some food for thought
    Marc Gravell : As always, other options are available and your mileage may vary ;-p
    Jonathan Parker : Yeah, programming is so deterministic isn't it guys? lol.

0 comments:

Post a Comment