System Design Process 29th Edition (Spring 2011)
Spring 2011 System Design Process 29th Edition
The purpose of this document is to share a system design methodology that promotes successful deployment of geographic information system (GIS) technology. This system design methodology includes guidelines for identifying business requirements, making appropriate software selection, using properly configured data sources, and providing sufficient hardware to meet user productivity needs. This document is focused on system performance and scalability - building a GIS that will perform during peak operational loads. Much more can be said about business requirements analysis (GIS User Needs) and available software functionality - this is not the focus of this documentation. Dr Roger Tomlinson (Father of GIS) provides an excellent book that shares a proven framework for comprehensive GIS planning called Thinking about GIS. Understanding the information products you want out of the GIS and identifying the software candidates you might use to produce these information products is a prerequisite for completing your system architecture design.
Contents
- 1 What Is System Architecture Design?
- 2 Why we do planning
- 3 Why Is System Architecture Design Important?
- 4 What is the System Design Process?
- 5 What Does It Take to Support Successful GIS Operations?
- 6 Why use Capacity Planning tools?
- 7 Concluding remarks
- 8 CPT Video: System Design Process
- 9 Previous Editions
What Is System Architecture Design?
System architecture design is a process developed by Esri to promote successful GIS enterprise operations. This process builds on your existing information technology (IT) infrastructure and provides specific recommendations for hardware and network solutions based on existing and projected business (user) needs. Application requirements, data resources, and people within an organization are all important in determining the optimum hardware solution as shown in figure 1-1.
Early GIS customer implementation experiences highlighted the importance of understanding the relationship between user business workflow requirements and the supporting hardware and network infrastructure loads. A process was established to understand GIS business needs, identify common user workflows (application and data architecture patterns), and apply a system architecture design methodology to identify technology solutions that contribute to successful GIS operations.
Why we do planning
The primary reasons for planning include identifying business needs, defining project requirements, and reducing implementation risk. In practical terms, we need a plan if we hope to get something done. The plan provides a foundation for successful implementation. Figure 1-2 identifies some of the primary planning objectives.
Business requirements may include a variety of user workflows and Web services that would improve business operations – these workflows and services must be identified early during the planning process. The workflow and service definitions establish a foundation for the rest of the planning process.
The peak user workflow and service processing requirements must be identified and translated to appropriate system design loads – server processing times and network traffic. These processing loads provide a foundation for generating appropriate hardware specifications (performance and capacity requirements) and network infrastructure needs.
A project must be defined to procure and implement the required technology. The project will include a budget and schedule along with a variety of project performance milestones. Project approval is required before any major design and/or implementation efforts begin. Project approval is based on identified or perceived benefits that will accrue to the business resulting from the proposed level of time (schedule) and financial (budget) investment.
A good plan can be used to reduce system implementation risk. Delivering a system that will satisfy identified business requirements is fundamental to project success. Understanding key system performance parameters, identifying incremental system performance targets, and establishing a system performance validation plan can chart a solid framework for managing implementation risk.
Planning is an incremental process driven by rapidly changing technology. The most effective plans are designed to adapt to changing business needs and evolving technology opportunities. The established change control process must be agile and effective to ensure project tasks are managed to deliver within established schedule and budget constraints.
Why Is System Architecture Design Important?
A distributed computer environment must be designed properly to support user performance (productivity) requirements. The weakest "link" in the system will limit performance. The system architecture design process derives specifications for a balanced hardware solution. Investment in hardware and network components based on a balanced system load model provides the highest possible system performance at the lowest overall cost as represented by the chain in figure 1-3.
Building a high performance GIS requires more than getting the hardware right. User workflows must be designed to optimize client productivity (simple maps) and efficiently manage heavier geoprocessing loads (service request queue). The geodatabase design and data source selection should be optimized to address system performance and scalability requirements. The selected production platform components (servers, workstations, storage) must have the capacity to handle peak user workflow processing loads within an acceptable service response time. The system architecture design must address performance needs and bandwidth constraints over distributed communication networks—technology and solution architecture must be selected to conserve shared infrastructure resources. System architecture design can provide a solid foundation for building a productive operational environment.
Getting it right from the start is best done by taking the time to understand the technology, quantify user requirements, select the right software technology, and deploy the right hardware. Not getting it right from the start will cost money to make it right later. The cost of change increases exponentially as implementation proceeds as shown in Figure 1-4.
What is the System Design Process?
The traditional system design process includes a GIS needs assessment and a system architecture design. The system architecture design is based on user workflow requirements identified in the GIS needs assessment. Figure 1-5 provides an overview of the traditional system design process.
GIS Needs Assessment: The GIS needs assessment includes a review of user workflow requirements and identifies where GIS technology can improve the quality and productivity of the business process flow. This assessment identifies GIS application and data requirements and an implementation strategy for supporting GIS user needs. The user organization must be actively involved throughout the user needs assessment. A GIS professional consultant familiar with current GIS solutions and customer business practices can help facilitate this planning effort.
System Architecture Design: The system architecture design is based on user requirements identified by the GIS needs assessment. Customers must have a clear understanding of their GIS application and data requirements before they are ready to complete the system architecture design. System implementation planning should schedule hardware purchases "just in time" to support user deployment needs.
The system design begins with a technology exchange. The technology exchange provides a foundation for client support throughout the design process. Client participation is a key ingredient in the design process. The design process includes a review of the existing computer environment, GIS user requirements, and system design alternatives. The system design capacity planning tools translate projected peak user workflow requirements to specific platform specifications. An integrated implementation strategy is developed to support GIS deployment milestones.
Traditionally, the user needs assessment and the system architecture design were two separate efforts. There are some key advantages in completing these efforts together. Figure 1-6 shows the project time line relationship between the traditional user needs assessment and the system architecture design process.
The GIS software technology selection process should include a discussion of architecture options and available system deployment alternatives. The existing hardware environment, peak user workflow operations, and user location information required for a system design should be identified during the user needs workflow interviews. Software technology selection should consider hardware configuration options, required platform selection, peak system loads for each configuration option, and overall system costs. User productivity may be a primary consideration in making the right software technology choice. And finally the system implementation schedule must consider hardware and application delivery milestones. A primary goal for developing capacity planning tools is to automate the system architecture design analysis in such a way that GIS professional consultants can complete a user needs assessment and a system architecture design within an integrated business needs assessment.
What Does It Take to Support Successful GIS Operations?
There are several critical deployment stages that support a successful implementation. Understanding the importance of each stage and the key objectives for success leads to more effective enterprise implementations. Figure 1-7 shows a series of typical system deployment stages for building and maintaining successful enterprise GIS operations.
The system architecture design Capacity Planning Tool (CPT) was developed as a framework to promote successful GIS system design and implementation. CPT functions contribute throughout the implementation cycle.
Requirements Stage:
Understanding GIS technology alternatives, quantifying GIS user requirements, and establishing an appropriate system architecture design deployment strategy are critical during requirements definition. Capacity planning during this phase can help establish appropriate software performance specifications. The CPT includes a Calculator tab for use in technology review and selection, a Workflow tab for establishing workflow performance targets, and a Design tab to identify user locations and network infrastructure limitations. Figure 1-8 provides an overview of the CPT modules provided for use during your business requirements analysis.
A review of available software technology patterns and software performance parameters can provide key information for technology selection. This is a planning stage where "getting it right" can save considerable effort and money throughout the implementation. User workflow analysis should generate reasonable workflow performance targets for use in the system architecture design.
Peak concurrent users and workflow productivity establish system throughput loads that drive the system architecture design. These loads include network traffic that must travel between the user display location and the central data center. Understanding where the users are located and how these locations are connected with the central data center can directly impact software technology selection.
Design Stage:
System development and prototype testing build confidence for the follow-on deployment. This is where time and money are invested to build and test the selected environment. Initial prototype testing demonstrates functionality and reduces implementation risk. Preliminary software performance testing can validate initial capacity planning assumptions. Figure 1-9 shows the CPT modules that share vendor platform performance information and enable final hardware platform configuration and selection.
The system design provides specifications for vendor hardware selection. The CPT Hardware tab shares vendor hardware relative throughput benchmarks. The CPT Design analysis integrates user requirements with published vendor platform performance benchmark results to identify system capacity needs.
Platform performance will impact user productivity and peak system capacity. The CPT Design platform selection module can represent up to 10 separate platform tier, each with your specific physical or virtual server platform technology selection. Each platform tier will expand as required to represent peak capacity design requirements.
The CPT software configuration module is used to identify software installation and data source selection for business workflows supporting users located throughout your enterprise environment. Workflow software executables can be installed on your data center platform tier to represent your selected design configuration . The selected platform architecture and software installation will determine availability and scalability of the final system design.
The CPT Design tab includes a platform solution layout and a Workflow Performance Summary as shown in Figure 1-10.
The platform solution layout shows the hardware architecture, number of platform nodes, and the peak utilization on each platform tier. The user requirements analysis assigns the workflow processing loads to the installed platform tier and calculates the required platform nodes and resulting utilization. A variety of platform configurations can be represented for each enterprise design, representing your operational data center platform configuration.
User workflow productivity is directly related to display performance. Workflow performance depends on user location, network connectivity, software install, platform selection, and peak system loads. The CPT Design requirements module includes a Workflow Performance Summary reporting component service times, queue times, and average display response time for each user workflow.
User workflow performance at locations throughout the enterprise environment is an important consideration when selecting a final software technology solution. The selected design solution must be configured to satisfy user performance needs.
Construction Stage:
A successful initial system deployment can set the stage for success. This is where the solution comes together and final operational support needs are validated. This is an important time to demonstrate performance and capacity of the deployed system and validate that the selected hardware and infrastructure will support full production deployment. Several tools are provided on the CPT test tab to translate measured performance to software technology workflow performance targets (service times) used for capacity planning. Figure 1-11 highlights test measurement tools that can be used to validate final software workflow performance.
System processing loads are defined by workflow service times - these are the software component service times identified on the CPT Workflow tab. Workflow service times are used to translate user throughput requirements to platform processor utilization. You can also measure system throughput and platform processor utilization, and if you know the platform configuration (processor type and number of core) you can calculate the workflow service times.
System throughput and utilization measurements can be used to validate CPT workflow service times. Web service throughput measurements can be used as an input to calculate workflow service times. Peak concurrent power users can also be used as an input if measured transaction throughput measurements are not available. You can select your test platform configuration and identify your throughput and utilization measurements, and excel will generate the source workflow service times.
Display rendering time can be used to estimate Web service workflow service times. Map display complexity is a primary contributor to overall Web mapping service display performance. ArcGIS Desktop user can use the map publishing tools to measure display performance when authoring a Web service map document. ArcGIS Desktop Map document rendering time can be used by the CPT Test MXDPerfstat tool to estimate published Web service processing loads. Once the Web service is published, a perfheatmap tool can be used to measure Web service render times throughout the map service area. The CPT can use render times collected by the perfheatmap test tool to generate appropriate service time processing profiles for the capacity planning tool Web service workflow models.
Implementation Stage:
Final system procurement and deployment demonstrate operational success. Capacity planning metrics can be used to monitor and maintain system performance objectives. Good planning, development, and testing will support a smooth deployment, productive operations, and satisfied users.
Establishing workflow performance target milestones and managing software performance to achieve performance goals throughout deployment is a positive recipe for success. Building systems without regard to performance and scalability can lead to disappointing results and costly recovery.
Why use Capacity Planning tools?
Capacity planning tools were developed to simplify the system design process and reduce implementation risk. GIS workflows generate network and server processing loads across the enterprise. GIS user productivity depends on the available capacity of these shared resources. There are many software technology patterns and performance parameters that impact workflow performance - it is not easy to put all these pieces together without modeling your system. Capacity planning tools allow users to focus on technology selection and information product design - workflow performance and the system architecture design solution is generated as a function of the technology selection.
CPT Platform Capacity Calculator
The Platform Capacity Calculator, located on the CPT Workflow tab, was developed to answer one of the most common questions asked by Esri customers – What throughput can I expect on my favorite hardware platform? This is not a simple question, because it depends on what you want the server platform to do. We talk about platform performance in Chapter 7, and share how performance and capacity has changed over the past 5 years. Figure 1-12 shows an overview of the CPT Platform Capacity Calculator. Platform performance has a direct impact on user productivity and system cost.
The Platform Capacity Calculator gives you a simple answer. It lets you select your favorite platform, identify your desired output (peak users or transactions per hour), and the resulting analysis is displayed on a chart showing the platform capacity for 5 different workflows. Result is provided as a range, showing both medium and light workflow complexity for each software deployment pattern.
CPT Platform Sizing Worksheet
For many years we have been asked for a simple step by step methodology that can be used for platform sizing. A simple process that you can use during planning to identify the required platform environment for a standard user workflow or Web service deployment. The Platform Sizing Worksheet, located on the CPT Sizing tab, was designed for this very purpose. Figure 1-13 shows an overview of the Platform Sizing Worksheet.
There are some questions you will need to answer; identify your user workflow or Web service, peak system loads from your user needs, the type of data source, platform architecture, and selected hardware server environment. You can identify bandwidth connections for a couple key remote client sites, along with the total number of remote concurrent users connecting from those sites, and the tool will complete an optional network suitability assessment (if you like). Once you enter what you want to do with the hardware, the resulting platform solution is provided as a list and in a graphic hardware configuration display. The Workflow Performance Summary shows average display performance for local and remote users.
CPT Calculator tab:
The Capacity Planning Calculator was developed to address user workflow analysis and software technology selection. Understanding user needs is the first step in building a GIS. Identifying the information product specifications and selecting the right software technology is an important part of the user needs analysis. The Calculator includes dropdown selection list inputs for the software technology pattern (software, map document, display complexity, and percent data cache) and key software performance parameters (display resolution, density, output format). The GIS Software Technology (Chapter 2) and Software Performance (chapter 3) will describe the technology patterns and software performance in more detail. Figure 1-14 provides an overview of the Capacity Planning Calculator tab.
The software technology and selected performance parameters generate GIS workflow performance targets (service times) than can be used for the system architecture design. The CPT Calculator generates workflow performance targets from technology baselines and performance adjustment factors derived from Esri performance validation test results. Workflow performance targets are updated with each Esri software release to incorporate the latest performance metrics.
The CPT Calculator also provides system architecture design and workflow performance information for the selected workflow. There is an option (cell C30) to select your workflow source (Calculator or Workflow tab). User requirements (peak users and productivity or peak service transaction rates) identify system processing loads, and platform architecture (tiers, high availability, SDE connection, data source) and platform technology selection (platform configuration, rollover) complete the system design inputs. Client network bandwidth connectivity inputs (rows 12-14) incorporate a preliminary communications assessment and generate local and remote user workflow display performance. The system design solution is displayed in the platform sizing section. The CPT Calculator was developed to provide a complete system design assessment for an informed software technology selection (single workflow) and generate workflow service times (Workflow tab) for use in building an enterprise solution (Design tab).
CPT Workflow tab:
The CPT Workflow tab provides a lookup table for workflow performance targets (component software service times) used in the CPT Design tab.
A list of Standard Esri Workflows are maintained in the Workflow tab for use in selecting the most common workflow performance targets. The Standard Esri Workflows were generated from the CPT Calculator and the workflow name identifies the recipe (software technology and performance parameter selections).
Workflows not included in the Standard Esri Workflow list can be generated from the CPT Calculator tab, with the output workflow recipe and service times included on the CPT Workflow tab (Calculator Workflow section).
A composite workflow analysis section generates average service times for a combination of two or more workflows. Figure 1-15 provides an overview of the CPT Workflow tab, identifying the primary sources for selecting design workflow performance targets (Standard Esri Workflows, Calculator, Test, or Composite workflows).
Project workflows can be established at the top of the Workflow list by making a copy of Standard Esri Workflows or custom workflows from the list and including in your project workflows - this is what I refer to as your user workflow analysis. Once your user workflow analysis and software technology selection is complete, your selected workflow performance targets will be available at the top of the workflow selection list for use in your system architecture design.
CPT Design tab:
The CPT Design tab was developed as a framework to document and execute an enterprise level system architecture design analysis. User workflow requirements identify user locations and peak workflow loads (concurrent users or service transaction rates) for each system implementation milestone (you can use the Excel copy sheet command to create multiple design sheets, one for each deployment milestone). Client display traffic loads are generated to complete an enterprise network suitability analysis. Software configuration (columns J through R - not shown) and platform selection complete inputs for your system architecture design. The CPT Design tab provides the platform solution (number of nodes and platform utilization) based on an integrated analysis of the user workflow requirements and platform selection. A Workflow Performance Summary provides average display response time for each user workflow. The CPT Design also includes functions to automatically adjust user workflow productivity to show expected system performance within selected network and platform constraints. Figure 1-16 provides an overview of the CPT Design tab.
Additional Design tabs can be created by using the Excel copy sheet command. Each CPT Design tab provides a complete model of each project deployment milestone. Each Design milestone identifies peak user workflow requirements, network bandwidth and projected network utilization, platform selection and projected platform utilization, final hardware selection requirements, and expected workflow average display response times and productivity for each user site location.
CPT Test tab:
The CPT Test tab was developed for use in validating workflow performance compliance. The CPT Design tab uses workflow performance targets (software component service times) as input values to generate system level throughput and utilization outputs. The CPT Test tab does just the opposite, it takes live system level throughput and utilization performance measurements as input values to calculate workflow software component service times (workflow performance targets that would generate these results).
The CPT Test tab provides four different ways to translate live software technology performance measurements to CPT workflow service times. Measured throughput transaction rate (transactions per hour) is the most accurate. Throughput (TPH) entry will override the other three Test tab input options. Peak users is the second throughput input option. In this case, peak users is multiplied by user productivity (displays per minute/user) to estimate the throughput transaction rate. The third and fourth options use measured map display render time (ArcGIS Desktop author can use MXDPerfstat or the ArcGIS Desktop Map Publishing tool to measure MXD render time; an ArcGIS Desktop client can use the PerfHeatMap tool to generate a map of ArcGIS Server display rendering times throughout a defined service area). Both of these tools can be used to measure map display complexity during initial service design and implementation. The Test tool links map display complexity to the selected Web mapping software technology pattern to generate baseline workflow service times. Figure 1-17 provides an overview of the CPT Test tab.
The CPT Test tab can be used by GIS "Authors" who create map documents (MXD) for Web publishing. Map display rendering time can be measured during map publishing optimization or following initial server deployment. This is the first milestone in validating workflow performance. During initial deployment, measured throughput values (map display transactions per hour or peak concurrent users) can be used to validate workflow performance. During live operations, system performance measurements can be used to validate design compliance.
The Capacity Planning Tool provides a model that connects user workflow requirements with system performance and scalability. Building a GIS based on a system design model that can both establish and validate user workflow productivity requirements establishes a foundation for successful GIS deployment. Identifying proper system deployment performance milestones and validating workflow design compliance with established performance design goals will reduce implementation risk and ensure system deployment success. The Capacity Planning Tool is designed to help GIS and IT managers understand business needs and promote more productive GIS operations.
Concluding remarks
Proper GIS planning is the most important investment any organization can make in building a GIS. Understanding your GIS needs, selecting the right technology at the right time, and establishing documented implementation milestones to measure your progress can ensure your success. This document is focused on sharing how to build and maintain successful GIS operations. The Capacity Planning Tools provide a framework for collecting what you know about your business needs and your system environment. The Capacity Planning Tool models connect what you understand about GIS user requirements with the network and platform loads your IT support teams can measure in the data center.
CPT Video: System Design Process
The next Chapter will review the most common GIS technology patterns and share best practices in making the right technology selection.
Previous Editions
System Design Process 28th Edition
System Design Process 27th Edition
Page Footer
Specific license terms for this content
System Design Strategies 26th edition - An Esri ® Technical Reference Document • 2009 (final PDF release)
