System Design Process 29th Edition (Spring 2011)
Spring 2011 System Design Process 29th Edition
The purpose of this document is to share a system design methodology that promotes successful deployment of geographic information system (GIS) technology. This system design methodology includes guidelines for identifying business requirements, making appropriate software selection, using properly configured data sources, and providing sufficient hardware to meet user productivity needs. This document is focused on system performance and scalability - building a GIS that will perform during peak operational loads. Much more can be said about business requirements analysis (GIS User Needs) and available software functionality - this is not the focus of this documentation. Dr Roger Tomlinson (Father of GIS) provides an excellent book that shares a proven framework for comprehensive GIS planning called Thinking about GIS. Understanding the information products you want out of the GIS and identifying the software candidates you might use to produce these information products is a prerequisite for completing your system architecture design.
- 1 What Is System Architecture Design?
- 2 Why we do planning
- 3 Why Is System Architecture Design Important?
- 4 What is the System Design Process?
- 5 What Does It Take to Support Successful GIS Operations?
- 6 Why use Capacity Planning tools?
- 7 Concluding remarks
- 8 CPT Video: System Design Process
- 9 Previous Editions
What Is System Architecture Design?
Early GIS customer implementation experiences highlighted the importance of understanding the relationship between user business workflow requirements and the supporting hardware and network infrastructure loads. A process was established to understand GIS business needs, identify common user workflows (application and data architecture patterns), and apply a system architecture design methodology to identify technology solutions that contribute to successful GIS operations.
Why we do planning
The primary reasons for planning include identifying business needs, defining project requirements, and reducing implementation risk. In practical terms, we need a plan if we hope to get something done. The plan provides a foundation for successful implementation. Figure 1-2 identifies some of the primary planning objectives.
Business requirements may include a variety of user workflows and Web services that would improve business operations – these workflows and services must be identified early during the planning process. The workflow and service definitions establish a foundation for the rest of the planning process.
The peak user workflow and service processing requirements must be identified and translated to appropriate system design loads – server processing times and network traffic. These processing loads provide a foundation for generating appropriate hardware specifications (performance and capacity requirements) and network infrastructure needs.
A project must be defined to procure and implement the required technology. The project will include a budget and schedule along with a variety of project performance milestones. Project approval is required before any major design and/or implementation efforts begin. Project approval is based on identified or perceived benefits that will accrue to the business resulting from the proposed level of time (schedule) and financial (budget) investment.
A good plan can be used to reduce system implementation risk. Delivering a system that will satisfy identified business requirements is fundamental to project success. Understanding key system performance parameters, identifying incremental system performance targets, and establishing a system performance validation plan can chart a solid framework for managing implementation risk.
Planning is an incremental process driven by rapidly changing technology. The most effective plans are designed to adapt to changing business needs and evolving technology opportunities. The established change control process must be agile and effective to ensure project tasks are managed to deliver within established schedule and budget constraints.
Why Is System Architecture Design Important?
Building a high performance GIS requires more than getting the hardware right. User workflows must be designed to optimize client productivity (simple maps) and efficiently manage heavier geoprocessing loads (service request queue). The geodatabase design and data source selection should be optimized to address system performance and scalability requirements. The selected production platform components (servers, workstations, storage) must have the capacity to handle peak user workflow processing loads within an acceptable service response time. The system architecture design must address performance needs and bandwidth constraints over distributed communication networks—technology and solution architecture must be selected to conserve shared infrastructure resources. System architecture design can provide a solid foundation for building a productive operational environment.
What is the System Design Process?
The traditional system design process includes a GIS needs assessment and a system architecture design. The system architecture design is based on user workflow requirements identified in the GIS needs assessment. Figure 1-5 provides an overview of the traditional system design process.
GIS Needs Assessment: The GIS needs assessment includes a review of user workflow requirements and identifies where GIS technology can improve the quality and productivity of the business process flow. This assessment identifies GIS application and data requirements and an implementation strategy for supporting GIS user needs. The user organization must be actively involved throughout the user needs assessment. A GIS professional consultant familiar with current GIS solutions and customer business practices can help facilitate this planning effort.
System Architecture Design: The system architecture design is based on user requirements identified by the GIS needs assessment. Customers must have a clear understanding of their GIS application and data requirements before they are ready to complete the system architecture design. System implementation planning should schedule hardware purchases "just in time" to support user deployment needs.
The system design begins with a technology exchange. The technology exchange provides a foundation for client support throughout the design process. Client participation is a key ingredient in the design process. The design process includes a review of the existing computer environment, GIS user requirements, and system design alternatives. The system design capacity planning tools translate projected peak user workflow requirements to specific platform specifications. An integrated implementation strategy is developed to support GIS deployment milestones.
The GIS software technology selection process should include a discussion of architecture options and available system deployment alternatives. The existing hardware environment, peak user workflow operations, and user location information required for a system design should be identified during the user needs workflow interviews. Software technology selection should consider hardware configuration options, required platform selection, peak system loads for each configuration option, and overall system costs. User productivity may be a primary consideration in making the right software technology choice. And finally the system implementation schedule must consider hardware and application delivery milestones. A primary goal for developing capacity planning tools is to automate the system architecture design analysis in such a way that GIS professional consultants can complete a user needs assessment and a system architecture design within an integrated business needs assessment.
What Does It Take to Support Successful GIS Operations?
There are several critical deployment stages that support a successful implementation. Understanding the importance of each stage and the key objectives for success leads to more effective enterprise implementations. Figure 1-7 shows a series of typical system deployment stages for building and maintaining successful enterprise GIS operations.
The system architecture design Capacity Planning Tool (CPT) was developed as a framework to promote successful GIS system design and implementation. CPT functions contribute throughout the implementation cycle.
Understanding GIS technology alternatives, quantifying GIS user requirements, and establishing an appropriate system architecture design deployment strategy are critical during requirements definition. Capacity planning during this phase can help establish appropriate software performance specifications. The CPT includes a Calculator tab for use in technology review and selection, a Workflow tab for establishing workflow performance targets, and a Design tab to identify user locations and network infrastructure limitations. Figure 1-8 provides an overview of the CPT modules provided for use during your business requirements analysis.
A review of available software technology patterns and software performance parameters can provide key information for technology selection. This is a planning stage where "getting it right" can save considerable effort and money throughout the implementation. User workflow analysis should generate reasonable workflow performance targets for use in the system architecture design.
Peak concurrent users and workflow productivity establish system throughput loads that drive the system architecture design. These loads include network traffic that must travel between the user display location and the central data center. Understanding where the users are located and how these locations are connected with the central data center can directly impact software technology selection.
System development and prototype testing build confidence for the follow-on deployment. This is where time and money are invested to build and test the selected environment. Initial prototype testing demonstrates functionality and reduces implementation risk. Preliminary software performance testing can validate initial capacity planning assumptions. Figure 1-9 shows the CPT modules that share vendor platform performance information and enable final hardware platform configuration and selection.
The system design provides specifications for vendor hardware selection. The CPT Hardware tab shares vendor hardware relative throughput benchmarks. The CPT Design analysis integrates user requirements with published vendor platform performance benchmark results to identify system capacity needs.
Platform performance will impact user productivity and peak system capacity. The CPT Design platform selection module can represent up to 10 separate platform tier, each with your specific physical or virtual server platform technology selection. Each platform tier will expand as required to represent peak capacity design requirements.
The CPT software configuration module is used to identify software installation and data source selection for business workflows supporting users located throughout your enterprise environment. Workflow software executables can be installed on your data center platform tier to represent your selected design configuration . The selected platform architecture and software installation will determine availability and scalability of the final system design.
The CPT Design tab includes a platform solution layout and a Workflow Performance Summary as shown in Figure 1-10.
The platform solution layout shows the hardware architecture, number of platform nodes, and the peak utilization on each platform tier. The user requirements analysis assigns the workflow processing loads to the installed platform tier and calculates the required platform nodes and resulting utilization. A variety of platform configurations can be represented for each enterprise design, representing your operational data center platform configuration.
User workflow productivity is directly related to display performance. Workflow performance depends on user location, network connectivity, software install, platform selection, and peak system loads. The CPT Design requirements module includes a Workflow Performance Summary reporting component service times, queue times, and average display response time for each user workflow.
User workflow performance at locations throughout the enterprise environment is an important consideration when selecting a final software technology solution. The selected design solution must be configured to satisfy user performance needs.
A successful initial system deployment can set the stage for success. This is where the solution comes together and final operational support needs are validated. This is an important time to demonstrate performance and capacity of the deployed system and validate that the selected hardware and infrastructure will support full production deployment. Several tools are provided on the CPT test tab to translate measured performance to software technology workflow performance targets (service times) used for capacity planning. Figure 1-11 highlights test measurement tools that can be used to validate final software workflow performance.
System processing loads are defined by workflow service times - these are the software component service times identified on the CPT Workflow tab. Workflow service times are used to translate user throughput requirements to platform processor utilization. You can also measure system throughput and platform processor utilization, and if you know the platform configuration (processor type and number of core) you can calculate the workflow service times.
System throughput and utilization measurements can be used to validate CPT workflow service times. Web service throughput measurements can be used as an input to calculate workflow service times. Peak concurrent power users can also be used as an input if measured transaction throughput measurements are not available. You can select your test platform configuration and identify your throughput and utilization measurements, and excel will generate the source workflow service times.
Display rendering time can be used to estimate Web service workflow service times. Map display complexity is a primary contributor to overall Web mapping service display performance. ArcGIS Desktop user can use the map publishing tools to measure display performance when authoring a Web service map document. ArcGIS Desktop Map document rendering time can be used by the CPT Test MXDPerfstat tool to estimate published Web service processing loads. Once the Web service is published, a perfheatmap tool can be used to measure Web service render times throughout the map service area. The CPT can use render times collected by the perfheatmap test tool to generate appropriate service time processing profiles for the capacity planning tool Web service workflow models.
Final system procurement and deployment demonstrate operational success. Capacity planning metrics can be used to monitor and maintain system performance objectives. Good planning, development, and testing will support a smooth deployment, productive operations, and satisfied users.
Establishing workflow performance target milestones and managing software performance to achieve performance goals throughout deployment is a positive recipe for success. Building systems without regard to performance and scalability can lead to disappointing results and costly recovery.
Why use Capacity Planning tools?
Capacity planning tools were developed to simplify the system design process and reduce implementation risk. GIS workflows generate network and server processing loads across the enterprise. GIS user productivity depends on the available capacity of these shared resources. There are many software technology patterns and performance parameters that impact workflow performance - it is not easy to put all these pieces together without modeling your system. Capacity planning tools allow users to focus on technology selection and information product design - workflow performance and the system architecture design solution is generated as a function of the technology selection.
CPT Platform Capacity Calculator
The Platform Capacity Calculator gives you a simple answer. It lets you select your favorite platform, identify your desired output (peak users or transactions per hour), and the resulting analysis is displayed on a chart showing the platform capacity for 5 different workflows. Result is provided as a range, showing both medium and light workflow complexity for each software deployment pattern.
CPT Platform Sizing Worksheet
There are some questions you will need to answer; identify your user workflow or Web service, peak system loads from your user needs, the type of data source, platform architecture, and selected hardware server environment. You can identify bandwidth connections for a couple key remote client sites, along with the total number of remote concurrent users connecting from those sites, and the tool will complete an optional network suitability assessment (if you like). Once you enter what you want to do with the hardware, the resulting platform solution is provided as a list and in a graphic hardware configuration display. The Workflow Performance Summary shows average display performance for local and remote users.
CPT Calculator tab:
The software technology and selected performance parameters generate GIS workflow performance targets (service times) than can be used for the system architecture design. The CPT Calculator generates workflow performance targets from technology baselines and performance adjustment factors derived from Esri performance validation test results. Workflow performance targets are updated with each Esri software release to incorporate the latest performance metrics.
The CPT Calculator also provides system architecture design and workflow performance information for the selected workflow. There is an option (cell C30) to select your workflow source (Calculator or Workflow tab). User requirements (peak users and productivity or peak service transaction rates) identify system processing loads, and platform architecture (tiers, high availability, SDE connection, data source) and platform technology selection (platform configuration, rollover) complete the system design inputs. Client network bandwidth connectivity inputs (rows 12-14) incorporate a preliminary communications assessment and generate local and remote user workflow display performance. The system design solution is displayed in the platform sizing section. The CPT Calculator was developed to provide a complete system design assessment for an informed software technology selection (single workflow) and generate workflow service times (Workflow tab) for use in building an enterprise solution (Design tab).
CPT Workflow tab:
A list of Standard Esri Workflows are maintained in the Workflow tab for use in selecting the most common workflow performance targets. The Standard Esri Workflows were generated from the CPT Calculator and the workflow name identifies the recipe (software technology and performance parameter selections).
Workflows not included in the Standard Esri Workflow list can be generated from the CPT Calculator tab, with the output workflow recipe and service times included on the CPT Workflow tab (Calculator Workflow section).
A composite workflow analysis section generates average service times for a combination of two or more workflows. Figure 1-15 provides an overview of the CPT Workflow tab, identifying the primary sources for selecting design workflow performance targets (Standard Esri Workflows, Calculator, Test, or Composite workflows).
Project workflows can be established at the top of the Workflow list by making a copy of Standard Esri Workflows or custom workflows from the list and including in your project workflows - this is what I refer to as your user workflow analysis. Once your user workflow analysis and software technology selection is complete, your selected workflow performance targets will be available at the top of the workflow selection list for use in your system architecture design.
CPT Design tab:
Additional Design tabs can be created by using the Excel copy sheet command. Each CPT Design tab provides a complete model of each project deployment milestone. Each Design milestone identifies peak user workflow requirements, network bandwidth and projected network utilization, platform selection and projected platform utilization, final hardware selection requirements, and expected workflow average display response times and productivity for each user site location.
CPT Test tab:
The CPT Test tab was developed for use in validating workflow performance compliance. The CPT Design tab uses workflow performance targets (software component service times) as input values to generate system level throughput and utilization outputs. The CPT Test tab does just the opposite, it takes live system level throughput and utilization performance measurements as input values to calculate workflow software component service times (workflow performance targets that would generate these results).
The CPT Test tab provides four different ways to translate live software technology performance measurements to CPT workflow service times. Measured throughput transaction rate (transactions per hour) is the most accurate. Throughput (TPH) entry will override the other three Test tab input options. Peak users is the second throughput input option. In this case, peak users is multiplied by user productivity (displays per minute/user) to estimate the throughput transaction rate. The third and fourth options use measured map display render time (ArcGIS Desktop author can use MXDPerfstat or the ArcGIS Desktop Map Publishing tool to measure MXD render time; an ArcGIS Desktop client can use the PerfHeatMap tool to generate a map of ArcGIS Server display rendering times throughout a defined service area). Both of these tools can be used to measure map display complexity during initial service design and implementation. The Test tool links map display complexity to the selected Web mapping software technology pattern to generate baseline workflow service times. Figure 1-17 provides an overview of the CPT Test tab.
The CPT Test tab can be used by GIS "Authors" who create map documents (MXD) for Web publishing. Map display rendering time can be measured during map publishing optimization or following initial server deployment. This is the first milestone in validating workflow performance. During initial deployment, measured throughput values (map display transactions per hour or peak concurrent users) can be used to validate workflow performance. During live operations, system performance measurements can be used to validate design compliance.
The Capacity Planning Tool provides a model that connects user workflow requirements with system performance and scalability. Building a GIS based on a system design model that can both establish and validate user workflow productivity requirements establishes a foundation for successful GIS deployment. Identifying proper system deployment performance milestones and validating workflow design compliance with established performance design goals will reduce implementation risk and ensure system deployment success. The Capacity Planning Tool is designed to help GIS and IT managers understand business needs and promote more productive GIS operations.
Proper GIS planning is the most important investment any organization can make in building a GIS. Understanding your GIS needs, selecting the right technology at the right time, and establishing documented implementation milestones to measure your progress can ensure your success. This document is focused on sharing how to build and maintain successful GIS operations. The Capacity Planning Tools provide a framework for collecting what you know about your business needs and your system environment. The Capacity Planning Tool models connect what you understand about GIS user requirements with the network and platform loads your IT support teams can measure in the data center.
The next Chapter will review the most common GIS technology patterns and share best practices in making the right technology selection.