Monday, January 10, 2011

ArcSDE–Adding domains from code

(The images for this post were taken from the ESRI site because I no longer use the ArcDesktop applications)

In our project we are using Domains to display text instead of the codes written in our tables.

Well first what are Domains?

I always found Domains and Subtypes very similar. They are both used to bring description to a field in a feature. The difference I guess is that Subtypes also separate the features to groups whereas Domains only give text value.

According to ESRI domains are used as validators for the data inserted using ArcMap/ArcCatalog. The trouble with that is that in all my systems so far we didn’t insert the feature data by those programs – we did it all by code! And at least in 9.3.1 there were no validations on the field…

 

(from ESRI site using domain coded values)

Whereas according to ESRI subtypes allow much more. They increase performance, they create all kinds of rules and they allow the applying of domains on the fields.

How do I create domains regularly (in ArcCatalog)?

In ArcCatalog right click on the Geodatabase –> Properties –> Domains tab:

 

(Taken from the ESRI site)

On the first empty line in the top table add your domain name and description.

On the next table enter the Domain Type: Coded Values, Range

If you choose Range you will have to enter a minimum and maximum values

If you choose Coded Values you will have to enter in the bottom table the codes of the domain and their description. We use Coded Values domains.

How do I associate a field with a domain(in ArcCatalog)?

In ArcCatalog right click on a table/layer –> Properties –> Subtypes:

(Taken from the ESRI site)

On the bottom table click on the field you want to add the domain to and select the domain from the combo box.

How do I create domains by C# code?

You can read more about it here.

  1. #region Create/Delete Domains
  2.  
  3. public void CreateDomain(string name, esriFieldType type, List<KeyValuePair<object ,string >> domains)
  4. {
  5.     ICodedValueDomain codedValueDomain = new CodedValueDomainClass();
  6.  
  7.     foreach (var pair in domains)
  8.     {
  9.         codedValueDomain.AddCode(pair.Key, pair.Value);
  10.     }
  11.  
  12.     // The code to set the common properties for the new coded value domain.
  13.     var domain = (IDomain)codedValueDomain;
  14.     domain.Name = name;
  15.     domain.FieldType = type;
  16.     domain.SplitPolicy = esriSplitPolicyType.esriSPTDuplicate;
  17.     domain.MergePolicy = esriMergePolicyType.esriMPTDefaultValue;
  18.  
  19.     ((IWorkspaceDomains) _workspace).AddDomain(domain);
  20. }
  21.  
  22. public void DeleteDomain(string name)
  23. {
  24.     ((IWorkspaceDomains)_workspace).DeleteDomain(name);
  25. }
  26.  
  27. //For tests
  28. internal IDomain GetDomain(string name)
  29. {
  30.     return ((IWorkspaceDomains)_workspace).DomainByName[name];
  31. }
  32.  
  33. #endregion

The code uses an inner variable _workspace which is the current IWorkspace.

Weird things (found while unit testing):

1. You can create a domain with a name of “” or null but they will have the same name (later in my code I added a validation for the name field so that it has to have a value).

2. You can create a domain without any coded values (empty list not null list!)

3. You cannot create two domains with the same name, an exception will be thrown: COMException: Domain name already in use.

4. You cannot delete a domain with a name of “” or null – an exception is thrown that is NOT COMException - System.ArgumentException: Invalid function arguments – from the DeleteDomain method(my DB is doomed…). I manually deleted it from the database it was the only line in GDB_ITEMS where Name,PhysicalName,Path and Url were empty string (and just because its good to know – the type for domains is ‘8C368B12-A12E-4C7E-9638-C9C64E69E98F’).

How do I use a domain in a feature field by C# code?

You can read more about it here.

  1. #region Assign Domains
  2.  
  3. public void AssignDomain(string layerName, string fieldName, string domainName)
  4. {
  5.     var featureClass = GetFeatureClass(layerName);
  6.     var domain = ((IWorkspaceDomains) _workspace).DomainByName[domainName];
  7.     var field = featureClass.GetField(fieldName);
  8.  
  9.     if (field.Type != domain.FieldType)
  10.         throw new ApplicationException(
  11.             "The field and the domain are not of the same type. Cannot assign domain to field.");
  12.  
  13.     var schemaLock = (ISchemaLock)featureClass;
  14.     try
  15.     {
  16.         schemaLock.ChangeSchemaLock(esriSchemaLock.esriExclusiveSchemaLock);
  17.         ((IClassSchemaEdit)featureClass).AlterDomain(fieldName, domain);
  18.  
  19.     }
  20.     finally
  21.     {
  22.         schemaLock.ChangeSchemaLock(esriSchemaLock.esriSharedSchemaLock);
  23.     }
  24. }
  25.  
  26. #endregion

Weird things (found while unit testing):

1. If you try to delete a domain that is in use it will throw a COMException: The domain is used as a default domain

2. To remove a domain simply do it with null -  AlterDomain(fieldName, null) . I added a method called RemoveDomain that does that.

In GDB_ITEMS a field in a feature class that uses domain looks like:

  1. <GPFieldInfoEx xsi:type="typens:GPFieldInfoEx">
  2.   <Name>Type</Name>
  3.   <AliasName>Type</AliasName>
  4.   <ModelName>Type</ModelName>
  5.   <DomainName>DefaultDomain</DomainName>
  6.   <FieldType>esriFieldTypeSmallInteger</FieldType>
  7.   <IsNullable>false</IsNullable>
  8. </GPFieldInfoEx>

And a field in a feature class that doesn’t use domain looks like:

  1. <GPFieldInfoEx xsi:type="typens:GPFieldInfoEx">
  2.   <Name>Type</Name>
  3.   <AliasName>Type</AliasName>
  4.   <ModelName>Type</ModelName>
  5.   <FieldType>esriFieldTypeSmallInteger</FieldType>
  6.   <IsNullable>false</IsNullable>
  7. </GPFieldInfoEx>

 

The end result was this:

  1. #region Create/Delete Domains
  2.  
  3. /// <summary>
  4. /// Create a domain in the DB
  5. /// </summary>
  6. /// <param name="name"></param>
  7. /// <param name="type"></param>
  8. /// <param name="domains"></param>
  9. public void CreateDomain(string name, esriFieldType type, List<KeyValuePair<object ,string >> domains)
  10. {
  11.     if (String.IsNullOrEmpty(name))
  12.         throw new ArgumentException("Parameter name cannot be null or empty.", "name");
  13.  
  14.     ICodedValueDomain codedValueDomain = new CodedValueDomainClass();
  15.  
  16.     foreach (var pair in domains)
  17.     {
  18.         codedValueDomain.AddCode(pair.Key, pair.Value);
  19.     }
  20.  
  21.     // The code to set the common properties for the new coded value domain.
  22.     var domain = (IDomain)codedValueDomain;
  23.     domain.Name = name;
  24.     domain.FieldType = type;
  25.     domain.SplitPolicy = esriSplitPolicyType.esriSPTDuplicate;
  26.     domain.MergePolicy = esriMergePolicyType.esriMPTDefaultValue;
  27.  
  28.     ((IWorkspaceDomains) _workspace).AddDomain(domain);
  29. }
  30.  
  31.  
  32. /// <summary>
  33. /// Delete a domain from the DB
  34. /// </summary>
  35. /// <param name="name"></param>
  36. public void DeleteDomain(string name)
  37. {
  38.     if (String.IsNullOrEmpty(name))
  39.         throw new ArgumentException("Parameter name cannot be null or empty.", "name");
  40.  
  41.     ((IWorkspaceDomains)_workspace).DeleteDomain(name);
  42. }
  43.  
  44. //For tests
  45. internal IDomain GetDomain(string name)
  46. {
  47.     return ((IWorkspaceDomains)_workspace).DomainByName[name];
  48. }
  49.  
  50. #endregion
  51.  
  52. #region Assign Domains
  53.  
  54. /// <summary>
  55. /// Assign the domain to the field in layerName
  56. /// </summary>
  57. /// <param name="layerName"></param>
  58. /// <param name="fieldName"></param>
  59. /// <param name="domainName"></param>
  60. public void AssignDomain(string layerName, string fieldName, string domainName)
  61. {
  62.     if (String.IsNullOrEmpty(domainName))
  63.         throw new ArgumentException("Parameter name cannot be null or empty.", domainName);
  64.  
  65.     var domain = ((IWorkspaceDomains)_workspace).DomainByName[domainName];
  66.     AssignDomain(layerName, fieldName, domain);
  67. }
  68.  
  69. /// <summary>
  70. /// Remove the domain usage from the field
  71. /// </summary>
  72. /// <param name="layerName"></param>
  73. /// <param name="fieldName"></param>
  74. public void RemoveDomain(string layerName, string fieldName)
  75. {
  76.     AssignDomain(layerName, fieldName, (IDomain)null);
  77. }
  78.  
  79. private void AssignDomain(string layerName, string fieldName, IDomain domain)
  80. {
  81.     if (String.IsNullOrEmpty(layerName) || String.IsNullOrEmpty(fieldName))
  82.         throw new ArgumentException("Parameter name cannot be null or empty.");
  83.  
  84.     var featureClass = GetFeatureClass(layerName);
  85.  
  86.     if(domain != null)
  87.     {
  88.         var field = featureClass.GetField(fieldName);
  89.  
  90.         if (field.Type != domain.FieldType)
  91.             throw new ApplicationException(
  92.                 "The field and the domain are not of the same type. Cannot assign domain to field.");
  93.     }
  94.  
  95.     var schemaLock = (ISchemaLock)featureClass;
  96.     try
  97.     {
  98.         schemaLock.ChangeSchemaLock(esriSchemaLock.esriExclusiveSchemaLock);
  99.         ((IClassSchemaEdit)featureClass).AlterDomain(fieldName, domain);
  100.  
  101.     }
  102.     finally
  103.     {
  104.         schemaLock.ChangeSchemaLock(esriSchemaLock.esriSharedSchemaLock);
  105.     }
  106. }
  107.  
  108. //For Tests
  109. internal IField GetLayerField(string layerName, string fieldName)
  110. {
  111.     return GetFeatureClass(layerName).GetField(fieldName);
  112. }
  113. #endregion

//TODO: post this only after writing about extension methods and WorkspaceUtils

//TODO: Refactor the code…

Resources:

ESRI: A quick tour of attribute domains

ESRI: A quick tour of subtypes

ESRI: About maintaining attribute integrity while editing (using domains)

ESRI: Exercise 3: Creating subtypes and attribute domains

ESRI: Creating and modifying domains

ESRI: Assigning domains to fields

 

Keywords: ESRI, ArcSDE, SDE, domain, subtype, ArcCatalog, ArcMap, ArcDesktop,IWorkspace, C#, Add, Delete, ArcObjects

 

Calling ArcGIS Server service model

Our application manages the process of installing a site that has CAD drawings that usually don't fit in the place that it is supposed to be. For the longest time someone in our organization used to go through the process of uploading the file to a file GeoDatabase => moving, rotating the lines till they fit in the map => dissolve the line to polyline => save the polyline in the DB. She doesn't want to do that anymore because of all the trouble in the location of the CAD.

For that reason we had to make the process a part of our application.

Uploading the CAD data to lines was relatively easy and is in the post - Working with CAD files in ArcObjects.

The next step was dissolving the polylines to one polyline, and that’s where the troubles started – there are no dissolve methods in ArcObjects (I even posted a question about that in their forums but got no answers).

So after some discussion in our team we decided to go with a Geoprocessor task. We added a model to the ArcGIS server that dissolves one table to another table.

The code is fairly simple (the base for it was taken from the interactive samples for ESRI's Silverlight API):

  1. private void LoadRawCadCompleted()
  2. {
  3.     var dissolveTask = new Geoprocessor(MapApplicationConfigWrapper.Instance.DissolveCadServiceUrl);
  4.     dissolveTask.CancelAsync();
  5.     dissolveTask.JobCompleted += DissolveTask_ExecuteCompleted;
  6.     dissolveTask.Failed += DissolveTask_Failed;
  7.     dissolveTask.SubmitJobAsync(new List<GPParameter>());
  8. }

 

But I got this error back:

{ESRI.ArcGIS.Client.Tasks.ServiceException: Execute operation is not allowed on this service.}

I have tried making the service synchronic – like this guy:

image

I even called did the clear cache thing – see "Clearing ArcGIS Server REST API Cache" Post.

This (of course) didn't work!

 

The problem was:

image

The model didn't run…

(It didn’t work because it used a connection file that was on the Model’s Designer computer but not on the ArcGis Server)

 

 

Well after the problem was solved (and I changed the Execution type back to asynchronous):

image

 

I used that code and got back:

-        jobInfoEventArgs    {ESRI.ArcGIS.Client.Tasks.JobInfoEventArgs}    ESRI.ArcGIS.Client.Tasks.JobInfoEventArgs
-        base    {ESRI.ArcGIS.Client.Tasks.JobInfoEventArgs}    ESRI.ArcGIS.Client.Tasks.TaskEventArgs {ESRI.ArcGIS.Client.Tasks.JobInfoEventArgs}
+        base    {ESRI.ArcGIS.Client.Tasks.JobInfoEventArgs}    System.EventArgs {ESRI.ArcGIS.Client.Tasks.JobInfoEventArgs}
        UserState    "j5fac49ec349241759b248eb20d9a716d"    object {string}
-        JobInfo    {ESRI.ArcGIS.Client.Tasks.JobInfo}    ESRI.ArcGIS.Client.Tasks.JobInfo
        JobId    "j5fac49ec349241759b248eb20d9a716d"    string
       JobStatus    esriJobSucceeded    ESRI.ArcGIS.Client.Tasks.esriJobStatus
-        Messages    Count = 3    System.Collections.Generic.List<ESRI.ArcGIS.Client.Tasks.GPMessage>
        Capacity    4    int
        Count    3    int
-        Static members       
+        Non-Public members       
+        Non-Public members   

               

Resources:

Interactive samples for ESRI's Silverlight API

ESRI forum - Error executing Geoprocessing Service in Silverlight

 

Keywords: ESRI, ArcGIS Server, ArcGIS Manager, Job, Dissolve, Silverlight

Silverlight ESRI App - IIS is dead–Error 5011

On the start of this week (2/1/20) our application in production started having troubles. And we didn’t know why but the IIS process from time to time just decided to quit.

In the EventViewer’s Application log we found:

An unhandled exception occurred and the process was terminated.

Application ID: /LM/W3SVC/1/ROOT/AppPool

Process ID: 1340

Exception: System.OutOfMemoryException

Message: Exception of type 'System.OutOfMemoryException' was thrown.

In the EventViewer’s System log we found:

A process serving application pool 'XAppPool' suffered a fatal communication error with the Windows Process Activation Service. The process id was '1976'. The data field contains the error number.

Something about the system message in TechNet.

Googled the system message and found on StackOverFlow, the advice there was to use IIS Debug Diagnostics tools but we have IIS 7 and this link doesn’t apply to it.

I and my team leader started studying how to read the trace and how to debug the failure.

We thought we found the solution because we found that an unknown someone changed a config value (at the start of the week) that is used in our Silverlight feature layer to check the layer for updates ( see post TODO) was far too low meaning that every second the server got 12*number of users requests that queried the layer status. And even though the server uses caching it was too much.

The WCF trace log has shown the Exception: System.ServiceModel.ProtocolException, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 The number of bytes available is inconsistent with the HTTP Content-Length header.  There may have been a network error or the client may be sending invalid requests.

Googled and found this post. Hope it is the Anti Virus but we shall see tomorrow!

Well No AntiVirus was installed, bummer!

 

Went over the ArcGIS Server logs and found these lines (in C:\Program Files (x86)\ArcGIS\server\user\log):

<Msg time='2011-01-07T22:27:00' type='ERROR' code='10837' target='BaseLayers.MapServer' methodName='MapServer.QueryFeatureData2' machine='liveil-gtm' process='4048' thread='3564'>GeoDatabase Error :FDO error: -2147024890 [ScratchRecordSet].  The table was not found. [GDB_ReleaseInfo].  The table was not found. [GDB_Release].  The table was not found. [GDB_ReleaseInfo].  The table was not found. [GDB_Release].  The table was not found. [GDB_DBTune].  The table was not found. [GDB_ReleaseInfo].  The table was not found. [GDB_Release].  The table was not found. [GDB_ReleaseInfo].  The table was not found. [GDB_Release].  The table was not found. [GDB_DBTune].</Msg>
<Msg time='2011-01-07T22:27:00' type='ERROR' code='100005' target='BaseLayers.MapServer' methodName='MapServer.QueryFeatureData2' machine='liveil-gtm' process='4048' thread='3564' elapsed='0.95282'>Method failed.HRESULT = 0x80004005 : Unspecified error  .</Msg>


And this Error in the EventLog:

[15:01:38.201 GisModule] System.Runtime.InteropServices.COMException (0x80041538): Underlying DBMS error[Microsoft OLE DB Provider for SQL Server: Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.][DATABASE.GIS.LAYER]
   at ESRI.ArcGIS.Geodatabase.IFeature.Store()
   at Namespace.OnPeriodicUpdateRequest(PeriodicUpdateRequest request) in c:\Folder\App Module GIS\Source\Namespace\Module.OnPeriodicUpdateRequest.cs:line 51


Another thing we found was that the AppPool for our ArcGis server thread was using a lot of CPU (by simply looking at the Task Manager). And that the ArcSoc processes were also very high in CPU usage - ~30%.


At the end my team leader found the problem one of the IFeature.Store failed but an empty row was added to the DB (and since there was no primary key set on the table the next time it happened it again added an empty row). That table grew to a very large number of rows but almost all of them were empty so that in every request to the Server for the Layer (from our Silverlight application) the AppPool for the ArcGis Server worked and it made the ArcSoc processes work as well KILLING the server.

 

//TODO: post about our implementation of ESRI Silverlight API’s FeatureLayer

//TODO: write about enabling WCF trace log

//TODO: study how to troubleshot this kind of errors and post it here

//TODO: Reformat this post – it’s just erratic

Resources:

TechNet Error Event Id 5011

Silverlight Forums: HttpRequestTimedOutWithoutDetail

Keywords: fatal communication error, Windows Process Activation Service, IIS, Error, Event Id 5011, WAS, ProtocolException, Anti Virus, IFeature, Store, ArcSoc, AppPool, CPU, COMException