Deploy faster, more efficiently using in-built SAP HANA platform capabilities
After spending seven years at SAP developing patterns for a more efficient way to integrate GIS and ERP systems, I’ve returned to Locana to get directly involved in implementations that use in-built HANA platform capabilities to accomplish that objective. Below, I’ve listed steps that eliminate technical debt and complexity found in typical integration patterns.
IT leaders, GIS managers, and others can use the HANA platform to simplify integrations, enhance performance, and access advanced analytical capabilities by taking these steps. Combining ArcGIS Enterprise and S/4HANA ERP on the HANA platform makes this possible.
Not only does the enterprise geodatabase perform analytic transactions more efficiently on the HANA platform, but leveraging the tools and functions within the HANA platform provides access to real-time analytical capabilities. In addition, you can use S/4HANA data when it lives on the same platform as S/4HANA. Specific advantages include:
- A simplified system that eliminates the complexities associated with data replication
- The ability to use SAP Core Data Services (CDS) views to retrieve combined SAP+GIS data in real-time since data is pulled on demand
- The ability to easily construct queries of SAP data from ArcGIS
- Improved data accuracy since data is being pulled directly from the source
- Streamlined workflows with reduced manual data entry and data transfers
- Better decision making with real-time and extensive access to SAP data
In typical integration patterns where data from SAP S/4HANA and ArcGIS Enterprise are mashed together, attributes are replicated from the S/4HANA system to a data warehouse. This data warehouse can be an ArcGIS publication geodatabase, or the replicated data can be placed into additional attribute tables in an ArcGIS enterprise geodatabase for asset geometries and related spatial metadata. These typical patterns use capabilities originally developed to deal with bottlenecks in standard disk-based RDBMS implementations, namely disk IO reduction.
This replication is accomplished using ETL, such as an off-the-shelf tool like SAFE FME or Mulesoft, or a custom ETL solution. But remember, ETL has some drawbacks:
- Increased data footprint for pre-aggregated and replicated data. Data is usually pre-aggregated, and this pre-aggregation is done because disk-based DBMSs cannot aggregate on-the-fly quickly; the results would take too long to calculate on-demand.
- Immediate data latency once data is replicated. Since ETL processes fail from time to time, data latency of the copied data increases until the ETL processes resume.
- Creation, testing, and maintenance of ETL scripts and processes.
- Increased operations and maintenance costs to maintain and enhance ETL scripts and related processes.
- Increased license cost for the off-the-shelf ETL tool. Custom ETL tools incur additional costs to architect, design, implement, test, and maintain the custom ETL tool.
- The length of time it takes to obtain change control approval before any ETL scripts and processes can be placed in production or any updates are made.
These drawbacks occur because ETL is a file-based interface between the two most important systems of record: the ERP system to manage assets and work and the enterprise GIS (EGIS) that manages asset geometry and related metadata. ETL is always schema on write – you can only access what has been replicated from the source S/4HANA system. If you want other attributes or to perform different aggregations, you must update, test, and deploy the ETL after you’ve obtained approval via change control.
The diagram below shows a typical pattern used in many enterprises.
Figure 1 – Typical file-based integration
Figure 1 shows the ETL processes operating at the application layer using the API calls for each system: SAP BAPIs for S/4HANA and ArcGIS feature services for ArcGIS Enterprise. S/4 PM in Figure 1 refers to SAP S/4HANA Enterprise Asset Management (EAM), also called Plant Maintenance (PM).
One of the most useful features that S/4HANA offers is business-level views called ABAP CDS views. These views are for 3rd party systems to consume data from S/4HANA. ArcGIS Enterprise can use them to retrieve data on-demand from the S/4HANA system if the enterprise geodatabase is on the HANA platform. Esri certified HANA (on-premises or “on-prem”) as an enterprise geodatabase in early 2018 and HANA Cloud in 2020. This means the system of record for your GIS data and geometries can reside in the HANA platform.
The HANA platform has capabilities that provide extremely fast on-the-fly aggregation – HANA can aggregate across 10s or 100s of millions of rows in a handful of seconds. This is because the DBMS in HANA is a modern in-memory columnar DBMS. Pre-aggregation is not necessary when data resides in the HANA platform.
Figure 2 shows a typical pattern that leverages the HANA platform for the ArcGIS enterprise geodatabase. Only the geodatabase that will be used with the S/4HANA data needs to be on the HANA platform.
Figure 2 – On-demand access to S/4 data
Most organizations have a process (called “synchronization”) to update ArcGIS feature classes for assets with the corresponding SAP Technical Object ID from the S/4HANA system. This ID can be for equipment, functional locations, work orders, and notifications. Without this ID in the associated ArcGIS feature classes, it would be impossible to mash up the GIS and ERP data regardless of the DBMS the enterprise geodatabase resides in.
Figure 2 shows the enterprise geodatabase on the HANA platform, in this case, in an on-prem HANA tenant. Built into the HANA platform (cloud and on-prem) is a Smart Data Access/Smart Data Integration (SDA/SDI) facility. This facility provides query federation capabilities and has many connectors to remote data sources, including Oracle, SQL Server, and cloud-based data stores like S3 buckets and the Azure Data Lake. One of those connectors enables Smart Data Integration to access SAP S/4HANA as an S/4HANA user. The SAP Data Provisioning Agent is supplied as a part of SDA/SDI and accesses S/4HANA as an S/4HANA user, which respects S/4HANA security. Only the HANA platform provides this ability.
The synchronization process in Figure 2 is used to create equipment and functional locations in S/4HANA EAM once the version of the asset in ArcGIS Enterprise is in the default version (in service). There are a handful of sync tools, including products and engineered solutions, that can be used. When the sync process calls the SAP BAPI to create the object in S/4HANA, the SAP Technical Object ID is returned. The synchronization process then updates the ArcGIS feature class using the ArcGIS feature service REST endpoint.
With the ability to connect to an S/4HANA system from a remote HANA system, the ABAP CDS views can be queried from ArcGIS Enterprise as if the views reside in the ArcGIS HANA instance. To the ArcGIS content creator, they can’t tell that SDA/SDI will send a query to S/4HANA on the fly. Querying on the fly provides schema-on-read, meaning the content creator modifies a SQL query to retrieve different or additional attributes from S/4HANA. This reduces or eliminates the need to replicate S/4HANA master and transactional data in ArcGIS Enterprise.
There are ABAP CDS views for work orders, notifications, equipment, and other S/4 Technical Objects. Because SAP supports ABAP CDS views, any queries written against them won’t break because SAP ensures upward compatibility of those CDS views.
In this example in Figure 3, S/4HANA master data (“nameplate data”) in EAM for a system valve is displayed on a web map in ArcGIS Portal.
Figure 3 – SAP Plant Maintenance master data displayed on a web map in Portal
All the visible attributes in the popup are retrieved on-demand from the equipment ABAP CDS view in S/4HANA. Since the ABAP CDS views are object-level business views, the column names, as shown, are in plain English, which allows an ArcGIS content creator to quickly determine what attributes they want to retrieve. The content creator doesn’t need to understand the underlying EAM schema, which is complex.
Using SAP HANA Studio or the web-based Cockpit, the ArcGIS admin or content creator can browse the available ABAP CDS views in S/4HANA and right-click or issue SQL to register the desired views in the ArcGIS HANA instance. The green box around the table names in Figure 4 highlights the registered ABAP CDS views.
Figure 4 – Registered ABAP CDS views in the ArcGIS HANA tenant
Once the desired ABAP CDS views are registered, as shown in Figure 4, a simple SQL query can be phrased using the SAP Technical Object key stored in the feature class, as shown in Figure 5.
Figure 5 – A SQL query that joins to the equipment ABAP CDS view
Having written the desired SQL query, you can execute it, as shown in Figure 6. The attributes from S/4HANA are retrieved on-demand. Once the SQL queries are written, they can be wrapped in a SQL view, a HANA calculation view, or as-is in an ArcGIS query layer using ArcGIS Pro.
This built-in capability in the SAP HANA platform eliminates the need to replicate from S/4HANA to the enterprise geodatabase.
Figure 6 – Query results from a join between the valve feature class and the equipment ABAP CDS view
Since there is no longer ETL used to obtain the master and transactional attributes from EAM; there is no longer any associated ETL process to fail or be maintained. Aside from doing away with the drawbacks listed above, there is substantial business value in using this built-in HANA platform capability. They include:
- Improved data accuracy
- Enhance your data processes and integrate systems to gain timely, accurate, and complete data.
- Streamlined workflows
- Improve operational efficiency and streamline workflows with reduced manual data entry and transfers.
- Enhanced reporting and analytics
- Identify areas for improvement and improve asset performance and maintenance activities, as well as more accurate reports and analytics.
- Better decision-making
- Gain real-time information on asset performance, maintenance activities, and strategic initiatives.
- Improved regulatory compliance of asset management/maintenance by reducing the risk of non-compliance penalty fees.
Reducing technical debt and complexity from the elimination of ETL will provide a lower cost of ownership for the geodatabase on the HANA platform versus other supported geodatabases. The business value is also only available when the ArcGIS enterprise geodatabase is on HANA.
The patterns just described are also available with HANA Cloud. Locana has and continues to deliver the business value made possible by placing the geodatabase on the HANA platform. That’s why I’ve outlined this specific series of steps – to offer a better way to implement your integrated platform that eliminates technical debt and complexity.
If you have questions or want to discuss the business value of putting geodatabases onto the HANA platform, please contact me at tom.turchioe@locana.co. For readers attending the SAP for Utilities Conference from October 9th to 11th, 2023, please come visit us at Booth 107.
***
Use this link to learn more about the benefits of streamlined SAP–ArcGIS integration, and how Locana can help.
