Data Modernisation Platform

An Australian state government has embarked on creation of a Spatial Digital Twin and a Next Generation Spatial Cadastre for their jurisdiction. This initiative requires modern 4D geospatial data for full functionality. Data provided by existing systems needs to be modernised before being input into the new systems. Our client (the government agency responsible for land information), as the lead agency for this initiative, was tasked with providing a secure and performant spatial platform for collaborative data modernisation activities. Akkodis, using their deep understanding of geospatial systems and cloud capabilities, responded in September 2025 by delivering a solution environment for the data modernisation activities.

The Scenario

The collaborative spatial platform supports multi-agency data storage, manipulation and analytic activities to prepare data for the initiative. It’s based on secure, highly available and scalable cloud services hosted by Amazon Web Services (AWS) in Australia. At a high level, the platform provides virtualised applications including ESRI ArcGIS and FME, streamed to users’ desktops via Amazon AppStream 2.0. It also provides users a shared Amazon FSx filesystem with data import and export capabilities.

The Solution

The solution required a secure and scalable spatial platform with capability to run many different Windows applications, tools and custom scripts without worrying about the management of underlying hardware infrastructure. Amazon AppStream 2.0 was selected as the platform for configuration, deployment and streaming of applications to end users. Various applications for spatial manipulation were provided, along with applications for supporting functionality such as file management.

Architecture Diagram

A custom AppStream image with all the required applications and their configuration has been configured. It also has the NFS Client installed so it can integrate with the FSx OpenZFS shared drive. On-demand scale-in and scale-out policies based on the CapacityUtilization metric are in place. Scheduled scaling policies have been added for cost optimisation outside of office hours, and to cope with the expected sudden increase in demand in the morning. At the end of each standard workday minimum capacity decreases to 0, allowing actual capacity to drop eventually to 0 in line with demand. An additional scale-out policy has been added to cope with the fleet when its actual capacity is 0, as happens outside of office hours. The CapacityUtilization scale-out policy will never trigger in this situation, even when a user attempts to start a session, as CapacityUtilization will be 0. The additional scale-out policy is based on the InsufficientCapacityError metric, increasing desired capacity when the metric value is >0 as happens when a user attempts to start a session and none are available. Scale-in is then prevented for a reasonable period allowing time for the user to retry starting a session. Managed image updates have been implemented using a Step Function that orchestrates updating of the fleet with a new custom image when updates become available in the AWS-managed base image.

State Machine

Amazon FSx for OpenZFS has been used to provide a shared filesystem for collaboration between users. OpenZFS is being used instead of Windows File Server due to its lower cost, better performance and expanded functionality. It enables access via an S3 Access Point, opening up more possibilities with data analytics and ML services, and providing additional data transfer methods between FSx and S3 buckets.

The client’s AWS tenancy employs a Landing Zone structure with AWS Organizations. There are core Management and Audit (Security) accounts along with shared services accounts – both production (for networking infrastructure and shared production workloads) and non-production (for CI/CD pipelines and shared workloads lower environments).

Organisation Structure

Workload accounts are divided into Dev, Test, UAT and Production environments, and also divided on another dimension of Business Domain. A Domain is an area of business capability with its own team and security boundary enforced by having its own set of accounts. The Data Modernisation Platform for example has its own account under the Workloads/Controlled/Production Organizational Unit (OU). Changes to accounts under the Workloads/Controlled OU are under formal change control with Change Records raised in Service Now and sign-off by multiple authorisation groups including a weekly Change Approval Board for production changes with outages. AWS Transit Gateway (TGW) is deployed in the shared services production account, providing workload access to shared network gateways and VPC Interface Endpoints. Domain network connectivity is selective, as managed by the TGW routing tables; for example domain VPCs can communicate with shared services VPCs but not with other domain VPCs. TGW routing tables are managed by AWS CloudFormation with as much configuration as possible divided across separate Dev, Test, UAT and Production stacks. This minimises risk of TGW routing changes breaking production functionality as many can be first tested against lower environments.

AppStream users log in with Single Sign-On (SSO) via SAML integration with on-premises Active Directory. The “Data Team” role assumed by these users is one of many SSO roles deployed across AWS accounts by CloudFormation Stack Sets. One Stack Set deploys global standard SSO roles across all accounts (such as those used by developers), and other domain-specific Stack Sets deploy domain-specific SSO Roles (such as the Data Team one) across the workload accounts of a domain. This Stack Set approach allows centralisation of SSO role configuration. For example the SSO roles are IP-restricted, authorising access only from within the client’s network; the whitelist of allowed IP ranges is defined in only one place. One of the applications available to AppStream users is the “AWS Credentials App”. This vends Data Team role temporary credentials for use in on-premises applications such as Cyberduck for exchanging data between desktop PCs and S3 buckets.

Additional security measures for this workload include the following. An incoming documents virus scanning service in the shared services account has been integrated with this workload via in Import Bucket S3 event definition that invokes the virus scanning front-end Lambda function when files are uploaded. The virus scanning service uses an autoscaled fleet of EC2 Spot instances to scan files with ClamAV, maintaining status in DynamoDB tables. The Data Modernisation Platform is, like other client AWS accounts, included in Organization-wide security configurations such as AWS Security Hub, AWS GuardDuty and AWS Backup. AWS Backup for example implements client’s compliance backups policy, ensuring that backups are taken and retained as required. A Stack Set deploys the compliance backup workload to all accounts in the Organization, including a Lambda function that processes backup failure events. These events are filtered, enriched and automatically raise incidents in Service Now as required. Remedial action is also triggered, for example if a scheduled annual backup fails then the most recent successful daily backup is promoted to be an annual one by automatically modifying its retention period.

The ArcGIS and FME commercial products in AppStream are supported by license servers in the shared services production account. Each of these is configured as an EC2 autoscale group with a maximum of one instance as high availability isn’t required but auto-recovery is needed when an instance fails its health check. For cost optimisation no load balancer is needed. The instances directly update records in Amazon Route 53 as they boot up to ensure traffic is directed to them. The production license servers are covered under the client’s Compute Savings Plans, and non-production license server autoscale groups are scaled to 0 at the end of each workday.

The shared services non-production account contains many CI/CD pipelines in AWS CodePipeline. These are used to deploy resources specifically for the Data Modernisation Platform as well as globally across all accounts. Stack Sets for example have pipelines that progressively target Dev, Test UAT and Production accounts in the Stack Set, with manual approval steps in between. Pipelines are created via CloudFormation from the client’s standard pipeline templates that often can be applied without change to new workload CloudFormation templates (including SAM) for Stack or Stack Set deployment. These pipelines are self-updating when changes are made to their pipeline templates.

The Outcome

The solution enables the Data Modernisation team to stream virtualised applications from a highly scalable and secure environment to their desktop over an internet browser or the AppStream 2.0 client. The solution has provided the team a capability to collaborate on multiple tasks, perform spatial analysis, and publish modernised spatial data. The cost-effective design of the Akkodis solution allows costs to be minimised while scaling to meet demand. Akkodis continues to maintain the Data Modernisation platform.