WorkZone 2018

We are closing in on the next WorkZone release. In a week we will host the annual customer seminar. It is time for an update of what to expect for the OData part of the WorkZone product suite.

As we haven’t finalized the release yet, there could be changes in the final release compared to information given here.

  1. The QueryBuilder (delivered as part of our OData provider) has been thru a major UI refresh. We now include more user friendly labels for the system names making it easier to find what you are looking for in our rather big model. Information about data types and lengths are provided as placeholder texts on input fields. Results can be paged (we now have a Next button). The results are now placed in a more readable table structure with inlined expanded data. A new color scheme has been applied making it easier to distinguish between single (blue) and multi (yellow) navigation properties. Columns can be resized and reordered. Links are included on ID properties of selected entity types enabling you to directly open WorkZone Client and show the selected Case, Document or Contact. Document contents are likewise directly accessible from a link on the SuggestedFileName of the Document entity. Queries can be stored and reused in WorkZone Client,  WorkZone Explorer and WorkZone Mobile. Saved OData QueryBuilder queries can now be reopened and edited directly in QueryBuilder or from WorkZone Client. The reference model documentation available from within our QueryBuilder now opens faster, looks prettier and automatically navigates to the entity chosen in the QueryBuilder. Happy 2018 QueryBuild’ing.
  2. I have submitted a couple of changes to Microsoft’s OData client framework that they included in their WCF Data Service Client release 5.8.2 and onwards. The most interesting one is the ability to have the WCF Data Services (OData) Client framework to handle null propagation check automatically. By setting DataServiceContext.AutoNullPropagation = true, you can now avoid doing explicit null checks in your C# linq select expression. The other submitted changes relates to fixing various performance issues in the WCF DS Client. The updated Microsoft framework can also be used (with the same benefits) together with earlier versions of WorkZone.
  3. We now support CORS on our OData service (requires configuration of allowed origins). By configuring this you would be able to securely fetch WorkZone data from other origins – like a SharePoint Intranet Portal.
  4. SaveChanges: $batch support for independent updates (SaveChangesOptions.BatchWithIndependentUpdates). This will submit multiple creates, updates, deletes and gets in a single batch’ed request, but instructing the server to treat these as individual transactions.
  5. Support for OData date functions year(), month() and day() functions as part of a filter expression enabling you to do queries like $filter=month(Created) eq 12 for finding entries that were created in a December month regardless of the year.
  6. The Custom Property feature we included in the 2017 release has been simplified on the OData part. Custom Properties are now exposed as _Value and _Summary properties directly on the entity where they belong. You no longer have to expand your way to get to their values. This change is backwards compatible – if you have existing code that reference and manipulate them the “old” way – it will still work as always.
  7. All UserKey, Summary and _Summary properties can now be used in the $filter part of a query.
  8. The service is now more immune to model changes in save operations.
  9. We have removed around 2000 redundant properties from our OData metadata model. We had some redundancy in our properties (like – there were multiple navigation paths to get from a Record to its File or Summary properties that would never have any contents), these redundant and always null properties are now removed from the metadata model. The OData server backend still knows about them so any existing javascript and .net based client code should run just fine without needing to change anything. If you at some point decide to update an existing .net proxy/service reference you could experience compiler errors if your code is referencing one of the now removed redundant properties, you can either fix these errors in your code rewriting it to use the preferred property instead, our you can use our new /OData/2017/ endpoint that is identical to the standard endpoint except that all properties are still available in the metadata model. The good part of this change is that it now should be a lot easier to find what you are looking for in the model. Reverted. See 17.
  10. Performance of expand a collection/multi property has been optimized. Prior to WorkZone 2018 a separate query was generated for each main entry per collection/multi expand path. In WorkZone 2018 a single query is generated on the server selecting everything at once. Our test case shows a performance gain of more than a factor 60 in these cases (the actual number depends on the actual query and data present).
  11. Single character wildcard ‘?‘ can now be used as search criteria on non free text based properties.
  12. Support getting an entity prefilled with default values returned from server. You can trigger this in a query by doing $top=-1.
  13. Advanced: You can now disable 2 automatic performance optimizations from an OData client: Disable Oracle Hints and Disable Inline Select (of expanded collections/multis).
  14. A new custom header FixedPageSize can be set to true to prevent the server to increase the page size for each page request and $skiptoken.
  15. A new demo showcase application (unsupported beta client) is included called Flex. It is a simple html5 application showcasing how a full functional but very simple client build on OData could look. Find it on your workzone host at http(s)://[HOST]/OData/Flex/
  16. Last but not least – good news for our OData Service… WorkZone Client and WorkZone PDF have now adopted OData completely. They now runs on the best standard data http protocol – OData. Welcome to both of them.
  17. We include a new OData endpoint V3 (found at http(s)://[host]/OData/V3/). The QueryBuilder will default to this endpoint. The V3 endpoint removes around 2000 redundant properties from our OData metadata model. We had some redundancy in our properties (like – there were multiple navigation paths to get from a Record to its File and Summary properties that would never have any contents). These redundant or always null properties are now removed from the metadata model at the V3 endpoint. We recommend that all new code targets this new V3 endpoint. We encourage existing code be rewritten to target the new V3 endpoint. The good part of the V3 endpoint is that it should be a lot easier to find what you are looking for in our rather large model.

DataServiceContext – Roundup

We have a number of posts with tips and tricks for setting different properties and event handlers to accomplish different optimizations, logging requirements etc. Her you will find a small code snippet with all these combined during DataServiceContext construction. For this I utilize the fact that the DataServiceContext (or the ODataService as it is called in your WorkZone proxy) is a partial class.

// As this class is partial it must be placed in the same namespace as the rest of the specification in your proxy
public partial class ODataService
{
    public ODataService() : this((ICredentials)null)
    { }

    public ODataService(ICredentials credentials) : this(new Uri("http://host/OData/"))
    {
        if (credentials == null)
            credentials = CredentialCache.DefaultCredentials;
        Credentials = credentials;
        Format.UseJson();
    }

    public string UseLogApplication { get; set; }
    public string UseLogWindow { get; set; }
    public string UseLogIdentity { get; set; }
    public string OnBehalfOfUser { get; set; }
    public bool AcceptCompression { get; set; }

    partial void OnContextCreated()
    {
        IgnoreMissingProperties = true;
        AcceptCompression = true;
        UseLogApplication = "MyDemoProject";
        SendingRequest2 += (sender, args) =>
            {
                if (!args.IsBatchPart)
                {
                    ApplyRequestHeaders(((HttpWebRequestMessage)args.RequestMessage).HttpWebRequest);
                }
            };
    }

    public void ApplyRequestHeaders(HttpWebRequest request)
    {
        if (!string.IsNullOrEmpty(UseLogApplication))
            request.Headers["UseLog-Application"] = UseLogApplication;
        if (!string.IsNullOrEmpty(UseLogWindow))
            request.Headers["UseLog-Window"] = UseLogWindow;
        if (!string.IsNullOrEmpty(UseLogIdentity))
            request.Headers["UseLog-Identity"] = UseLogIdentity;
        if (!string.IsNullOrEmpty(OnBehalfOfUser))
            request.Headers["OnBehalfOf"] = string.Concat("User=", OnBehalfOfUser);
        if (AcceptCompression)
            request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
    }
}

Immunity to model changes – IgnoreMissingProperties

As our OData model is generated automatically runtime to include all the available entities and properties including all customizations and modules (products) you can run into issues where the model on the server get misaligned with the model in your client side proxy (generated from Add Service Reference). This can happen in the following situations…

  1. You generate the proxy against one WorkZone installation and run your code against another installation and the two installations are not identical (in terms of the underlying database).
  2. You upgrade from one version of WorkZone Content Server to a newer release and the new release has additions to the underlying database.
  3. You install a new WorkZone  module or product that includes changes to the model.

If one of the above cases is true you could run into a problem during querying because the server is sending some new property that does not exist in your proxy because the proxy was generated at a point where the property did not exist. This issue manifests itself as an InvalidOperationException during object materialization stating “The closed type [X] does not have a corresponding [Y] settable property”.

To remedy this you could simply perform an “Update Service Reference” and recompile your code, but you can also make your code more immune to this situation by simply telling the DataServiceContext to ignore such properties that are not present in the proxy, like this…

context.IgnoreMissingProperties = true;

We recommend that you always set the IgnoreMissingProperties to true. Apart from this we also highly recommend that you always include a projection for your results – this primarily for performance reasons, but it will also circumvent the materialization problem with server model changes.

Minimize payload by switching to JSON format

The OData V3 protocol is two faced. It has a XML/ATOM based representation as well as a JSON based representation. The JSON format is roughly 4 times lighter than the ATOM format. The default format used by WCF Data Service Client (for OData V3) is ATOM, we highly recommend that you switch to the JSON format. The format to use is specified on the DataServiceContext by simply calling…

context.Format.UseJson();

Use HTTP compression to minimize the payload

In cases where the network bandwidth is a bottleneck you can benefit from enabling dynamic compression on the OData service. Enabling compression on the service is setup in the IIS manager, you can follow the directions here: , or you can ask our service technicians to do it for you. Once the service is correctly set up for dynamic compression you need to request compressed content when you send requests to the OData service. This is simply accomplished by a single line in a SendingRequest2 event handler on the DataServiceContext like this…

context.SendingRequest2 += (sender, args) =>
    {
        if (!arg.IsBatchPart)
        {
            // Tell the OData server that you accept compressed content. You can do this even if the server has not been set up for dynamic compression, the server will just send uncompressed content.
            ((HttpWebRequestMessage)args.RequestMessage).HttpWebRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
        }
    };

Logging

The WorkZone system has build in logging of user access to the primary entities File, Record and Contact. The logging feature logs both the user and the entities that the user has accessed as well as some additional information regarding the origin of the access like the name of the integration system that issued the request. We kindly ask you to always send this extra information when you write code against our OData service.

The DataServiceContext will automatically fire the event SendingRequest2 whenever it issues a request to the server. You can write an event handler for this event to ensure that the extended logging information is always included in all requests.

The following code snippet show how this can be done.

context.SendingRequest2 += (sender, args) =>
    {
        if (!args.IsBatchPart) // SendingRequest2 will be called for each contained request in a Batch request. We only need to supply the headers on the top level
        {
            // Specifying the UseLog-Application MUST ALWAYS be performed
            args.RequestMessage.SetHeader("UseLog-Application", "NameOfIntegration/System");
            // If you are always connecting with a system user, you should use the UseLog-Identity to specify the identity of the user you are servicing
            args.RequestMessage.SetHeader("UseLog-Identity", servicingUser);
        }
    };

WorkZone Content Server 2014 Released

News in WorkZone Content Server OData

There are no breaking model changes in this release (compared to WZCS 2013 R2). There are however functional changes that could impact certain parts of existing code – read below.

  1. Navigation properties can now be used in the $filter part of a query/GET. It is now possible to do a query like ‘Records?$filter=File eq null’ to have returned all records where the File is hidden, or ‘Records?$filter=File ne null’ to only return Records where the File is available to the current user.
  2. OData navigation properties representing extension tables from the underlying datamodel (our data dictionary) will now have the value null in cases where there is no entry in the table in the data base. Prior to this release that was only true for extension tables that were also main tables in their own register but not for extension tables that were only accessible from within the containing register. This is now fixed so that the experience and behavior is the same regardless of the differences in the modelling. This change is a breaking change, affected scenario is updating of properties in extension tables that does not have an entry in the DB table (the navigation property value is now null) – now you have to construct the object before assigning property values.
  3. Filtering on properties in extension tables could give incorrect results if the supplied criteria is ‘eq null’. An example is ‘Records?$filter=File/FileType eq null’ that would only return Records that had a visible File and the File has a blank/empty FileType. This same query will now also return Records that does not have a visible File. This change is to be in compliance with the old search behavior in the rest of our system as well to have the same behavior as one would expect from EF.
  4. Fixed a bug where creating of entities that have update mapping to another register generated wrong SQL if the reference was only specified by supplying the join value instead of using the models reference property. An example of this is creation a new FileDate by filling the FileKey_Value with the ID of the File instead of assigning the File to the FileKey property or adding the FileDate to the Dates collection of the File.
  5. Fixed a bug where filtering on properties on Renditions from the Document entity would fail. Now you can do a query like ‘Documents?$filter=Renditions/any(r: r/Valid eq true and r/Extension eq ‘pdf’)’ to return Documents that have a valid rendition of type pdf.
  6. The DocumentRendition type now have two new navigation properties Document and Record so you can get and filter on the original Document and the Record.
  7. Fixed a bug where dates sent to the server were not correctly converted to the local server time on the server. This could lead to incorrect time offsets in both data and search criteria’s. If you have written compensating code for circumvent this issue, you might need to update your client code.
  8. Introducing a new query option adjustDatesFrom that enables the construction of queries where the server is instructed to treat filters on date properties as being relative. The classic example is the construction of a query that searches for cases created today. Normally when you resend that exact query tomorrow it will not find the cases created on that day (tomorrow). By specifying adjustDatesFrom with the original date and time where the query was created the server will be able to compensate and the resubmitted query will then automatically match cases created on “today” whenever it is executed.
  9. A simple html OData Query Builder is included as part of the OData installation. It allows for simple exploring and query building against the WorkZone system. OData Query Builder is intended for developers, advanced/super users as well as an aid for users that want to build advanced queries for something like Excel Power Pivot. The OData Query Builder can expose both the system names and friendly names for the properties.
  10. Custom endpoints can now override the default documentation labels for properties (found in Documentation/LongDescription in the $metadata document).
  11. Navigation properties in custom endpoint definitions are now also filtered on basis of the definition.
  12. The standard supplied web.config file are now preconfigured for both http and https access. You no longer need to update the web.config file to enable https.
  13. The maximum file upload size have been increased to 2GB.
  14. Fixed a bug where commas in the supplied file name (slug) during document upload would fail.
  15. Fixed a bug where queries using the $orderby clause could result in rows being skipped if the order by column was null.
  16. We now have several new custom endpoint options that can be used to shape the $metadata model to be compliant with the limited set that is supported by Microsoft LightSwitch. With these new options you can use LightSwitch as a RAD tool to create streamlined HTML5 applications for both mobile and desktop use – while writing very little code (if any).
  17. New feature – comparison between properties. You can now perform queries that compares the content of properties directly in the data base. For example: “Files?$filter=substringof(Officer_Value, Title)” will return cases where the case handlers initials is part of the case title. “Files?$filter=Closed eq Created” returns cases that where closed on the same date as it were created. “Files?$filter=Parties/any(p: p/NameKey_Value eq Officer/NameKey)” returns cases where the case handler is also a party of the case.
  18. The $metadata model now uses a new feature in WCF DS 5.6.2 that enable us to be more explicit in the declaring of relations and their multiplicity. Prior to this release all multiplicities were either “0..1” or “*” now it is also possible to have “1”. We are using this whenever the underlying model (the datadictionary) declares “1”. This change should have no impact of existing code. The generated proxy will be the same regardless this change. The change were necessary to have LightSwitch play nicely with our model.
  19. We have shortened the address of the OData service as well as removed the default odata.svc document from the address part. You can now find your OData service at http://host/OData/