Getting Started
This is the third post in a series from Locana on using and maintaining the Esri Utility Network for gas. Here, we’ll introduce the concept of Utility Network data errors and some methodologies to resolving them. This is not a comprehensive review of all the different Utility Network errors, but a focused look at some of the different error types that our team encountered during a recent migration to the Utility Network from Smallworld.
What Are Utility Network Data Errors?
Data errors are one of the benefits of the Utility Network and provide a means to identify, track, and resolve inconsistencies in your gas network data. Data error layers form a Quality Control framework that display exceptions to the rules defined in the Utility Network. Dirty areas display any edit that has not yet been evaluated against the UN rules, as well as those that have failed validation.
When Are Data Errors Identified?
Utility Network errors will be encountered at a couple of different stages of your Utility Network project.
When The Enable Network Topology Tool is Executed
The first way errors are created is when the Enable Network Topology tool is executed. This tool has an option to Only Generate errors. This step can be run as part of the network build process to test your network configuration ahead of fully enabling the network. This will indicate the number of errors the utility network will contain.
Through Everyday Editing
After the network topology is enabled, new data errors may be introduced into the network through everyday editing. Believe it or not, this is a good thing! Upon every edit to any feature that participates in the Utility Network, a dirty area is generated around the extent of the edit.
The dirty area remains until the Validate Topology tool is run to validate new edits against the UN rule base. If the edit is allowed, the dirty area is removed and you can go along your merry way. If the edit is not allowed, an error feature is created within the point, line, or polygon UN error layer. The error feature will remain until it is resolved and re-validated. The data error layers are displayed in the Utility Network layer group when the UN feature service is added into ArcGIS Pro.
Why Do Data Errors Need to Be Resolved?
Data errors should be resolved to realize the full analytical capabilities of the Utility Network, such as accurate connectivity and pressure traces. They should also be resolved to keep the UN running at optimal performance.
Large error counts should be expected following an initial migration from a legacy system into the Utility Network. These errors are due to two main factors:
- Past mapping procedures that do not meet the Utility Network’s exacting standards
- Differences in how the legacy system modeled gas networks. Due to these factors, the number and types of data errors will vary from dataset to dataset.
Performance Considerations
During a recent migration from Smallworld, it was discovered that there is an upper limit to the number of data errors that a Utility Network may contain and remain performant. Performance impacts were noted above 100,000 errors with this dataset.
Exceeding this threshold translated to longer network topology enabling and validation time. It’s recommended to resolve as many of the initial data errors as possible for the best user experience. Implementing mapping procedures that do not result in additional errors being introduced to the dataset is also useful.
How to Resolve Utility Network Errors
The Three Ways to Resolve Data Errors
- Automated fixes
- Adding a rule to the Utility Network
- Manual fixes
As each has their merit and place, the chosen methodology is determined by many factors. The examples in this article focus on automated approaches—let’s dive in!
Automated Fixes
The table below shows the resulting point errors following migration from Smallworld.
Table 1—resulting point errors following data migration to the Utility Network from Smallworld before fixes were applied.
This dataset had an overwhelming number of Stacked Point errors (error type 25) comprising 49% of the total errors in the dataset. This error describes geometrically coincident points being assigned the same X, Y, and Z coordinates.
In the dataset, the coincident geometry was intentional. This is because the mapping procedure in Smallworld was to retire outdated devices and junctions plus place the new fitting directly on top of the old one. Service points were not deleted when retired since they were attributed with customer information that the business wanted to retain. It was decided to separate each overlapping point by 1 foot in the vertical direction using FME.
Applying this approach, the features would remain overlapping from the viewer’s perspective on the map, but would no longer have identical coordinates. This solution resolved the point error. Here is the same area with the z-offset applied after running Validate Network Topology to remove the error features and dirty areas.
This approach resolved 74% of the overall point errors and paved the way to a successful Utility Network implementation. Please note that care should be taken when applying a programmatic methodology to resolve data errors to ensure that the approach has the intended result across the dataset. This is often easier said than done, but our data team at Locana has strong expertise in this area (contact us to learn more).
Invalid Connectivity
Now let’s look at the most prevalent line error in the dataset, Invalid connectivity – The edges are different subtypes and cannot connect (error type 10) comprising 18% of the total errors. This error describes pipes of a different asset group or asset type without a device or junction between them to “allow” the connection.
Table 2—resulting line errors following data migration to the Utility Network from Smallworld before fixes were applied.
A specific example of a location where this is occurring is captured in the screenshot below.
Here, a cast-iron service is connecting to a plastic service without a transition in-between. The team proposed creating “proxy” features that would resolve the network error quickly, providing the benefit of improved network performance without a compromise in data quality. As there is now a feature present to identify the pipe transitions, the business would still have the means to return to each location to confirm the data.”
It should be noted that currently, the Utility Network does not take into consideration the Life Cycle Status of the network features when evaluating against the rule base. This means that proposed and retired features will generate network errors, not only in-service or “gassed” features in the Utility Network world.
About Utility Network Rules
The Utility Network’s rule base is inclusive. This means that rules are explicitly stated to allow connections between features (this is a departure from the Geometric Network, whose rule base is exclusive). A rule may need to be added if the connection is one that we want to allow.
If the particular connection is not allowed according to the business rules, then the subtype of the device, junction, or line must be changed to a combination that’s allowed by an existing rule. You can view the Utility Network’s existing rules by viewing the layer properties (accessible by right-clicking on the Utility Network layer group).
Rules are added to the Utility Network with the Add Rule Tool. To do this, you must disable the network topology before running the tool. My next post “When to Disable & Enable the Utility Network” will cover this scenario in detail.
***
Be sure to take a look at the first two posts in this series to learn more about using and maintaining the Esri Utility Network for gas:
