System Design Process 30th Edition (Fall 2011)
Fall 2011 System Design Process 30th Edition
The purpose of this document is to share a system design methodology that promotes successful deployment of geographic information system (GIS) technology. This system design methodology includes guidelines for identifying business requirements, making appropriate software selection, using properly configured data sources, and providing sufficient hardware to meet user productivity needs. This document focuses on system performance and scalability - building a GIS that will perform during peak operational loads.
Much more can be said about business requirements analysis (GIS User Needs) and available software functionality - this is not the focus of this documentation. Dr Roger Tomlinson (Father of GIS) provides an excellent book that shares a proven framework for comprehensive GIS planning called Thinking about GIS. Understanding the information products you want out of the GIS and identifying the software candidates you might use to produce these information products is a prerequisite for completing your system architecture design.
Contents
- 1 What Is System Architecture Design?
- 2 Why we do planning
- 3 Why Is System Architecture Design Important?
- 4 What is the System Design Process?
- 5 What Does It Take to Support Successful GIS Operations?
- 6 Product Architecture Selection
- 7 Hardware Platform Selection
- 8 Enterprise System Architecture Design
- 9 System Performance Validation
- 10 Additional CPT Design Tools
- 11 Concluding remarks
- 12 CPT Video: System Design Process
- 13 Previous Editions
What Is System Architecture Design?
System architecture design is a process developed by Esri to promote successful GIS enterprise operations. This process builds on your existing information technology (IT) infrastructure and provides specific recommendations for hardware and network solutions based on existing and projected business (user) needs. Application requirements, data resources, and people within an organization are all important in determining the optimum hardware solution as shown in figure 1-1.
Early GIS customer implementation experiences highlighted the importance of understanding the relationship between user business workflow requirements and the supporting hardware and network infrastructure loads. A process was established to understand GIS business needs, identify common user workflows (application and data architecture patterns), and apply a system architecture design methodology to identify technology solutions that contribute to successful GIS operations.
Why we do planning
The primary reasons for planning include identifying business needs, defining project requirements, and reducing implementation risk. In practical terms, we need a plan if we hope to get something done. The plan provides a foundation for successful implementation. Figure 1-2 identifies some of the primary planning objectives.
Business requirements may include a variety of user workflows and Web services that would improve business operations – these workflows and services must be identified early during the planning process. The workflow and service definitions establish a foundation for the rest of the planning.
The peak user workflow and service processing requirements must be identified and translated to appropriate system design loads – server processing times and network traffic. These processing loads provide a foundation for generating appropriate hardware specifications (performance and capacity requirements) and network infrastructure needs.
A project must be defined to procure and implement the required technology. The project will include a budget and schedule along with a variety of project performance milestones. Project approval is required before any major design and/or implementation efforts begin. Project approval is based on identified or perceived benefits that will accrue to the business resulting from the proposed level of time (schedule) and financial (budget) investment.
A good plan can be used to reduce system implementation risk. Delivering a system that will satisfy identified business requirements is fundamental to project success. Understanding key system performance parameters, identifying incremental system performance targets, and establishing a system performance validation plan can chart a solid framework for managing implementation risk.
Planning is an incremental process driven by rapidly changing technology. The most effective plans are designed to adapt to changing business needs and evolving technology opportunities. The established change control process must be agile and effective to ensure project tasks are managed to deliver within established schedule and budget constraints.
Why Is System Architecture Design Important?
A distributed computer environment must be designed properly to support user performance (productivity) requirements. The weakest "link" in the system will limit performance. The system architecture design process identifies specifications for a balanced hardware solution. Investment in hardware and network components based on a balanced system load model provides the highest possible system performance at the lowest overall cost as represented by the chain in figure 1-3.
Building a high performance GIS requires more than getting the hardware right. User workflows must be designed to optimize client productivity (simple maps) and efficiently manage heavier geoprocessing loads (service request queue). The geodatabase design and data source selection should be optimized to address system performance and scalability requirements. The selected production platform components (servers, workstations, storage) must have the capacity to handle peak user workflow processing loads within an acceptable service response time. The system architecture design must address performance needs and bandwidth constraints over distributed communication networks—technology and solution architecture must be selected to conserve shared infrastructure resources. System architecture design can provide a solid foundation for building a productive operational environment.
Getting it right from the start is best done by taking the time to understand the technology, quantify user requirements, select the right software technology, and deploy the right hardware. Not getting it right from the start will cost money to make it right later. The cost of change increases exponentially as implementation proceeds as shown in Figure 1-4.
What is the System Design Process?
The traditional system design process includes a GIS needs assessment and a system architecture design. The system architecture design is based on user workflow requirements identified in the GIS needs assessment. Figure 1-5 provides an overview of the traditional system design process.
GIS Needs Assessment: The GIS needs assessment includes a review of user workflow requirements and identifies where GIS technology can improve the quality and productivity of the business process flow. This assessment identifies GIS application and data requirements and an implementation strategy for supporting GIS user needs. The user organization must be actively involved throughout the user needs assessment. A GIS professional consultant familiar with current GIS solutions and customer business practices can help facilitate this planning effort.
System Architecture Design: The system architecture design is based on user requirements identified by the GIS needs assessment. Customers must have a clear understanding of their GIS application and data requirements before they are ready to complete the system architecture design. System implementation planning should schedule hardware purchases "just in time" to support user deployment needs.
The system design begins with a technology exchange. The technology exchange provides a foundation for client support throughout the design process. Client participation is a key ingredient in the design process. The design process includes a review of the existing computer environment, GIS user requirements, and system design alternatives. The system design capacity planning tools translate projected peak user workflow requirements to specific platform specifications. An integrated implementation strategy is developed to support GIS deployment milestones.
Traditionally, the user needs assessment and the system architecture design were two separate efforts. There are some key advantages in completing these efforts together. Figure 1-6 shows the project time line relationship between the traditional user’s needs assessment and the system architecture design process.
The GIS software technology selection process should include a discussion of architecture options and available system deployment alternatives. The existing hardware environment, peak user workflow operations, and user location information required for a system design should be identified during the user needs workflow interviews. Software technology selection should consider hardware configuration options, required platform selection, peak system loads for each configuration option, and overall system costs. User productivity may be a primary consideration in making the right software technology choice. And finally the system implementation schedule must consider hardware and application delivery milestones. A primary goal for developing capacity planning tools is to automate the system architecture design analysis in such a way that GIS professional consultants can complete a user needs assessment and a system architecture design within an integrated business needs assessment.
What Does It Take to Support Successful GIS Operations?
There are several critical deployment stages that support a successful implementation. Understanding the importance of each stage and the key objectives for success leads to more effective enterprise implementations. Figure 1-7 shows a series of typical system deployment stages for building and maintaining successful enterprise GIS operations.
The system architecture design Capacity Planning Tool (CPT) was developed as a framework to promote successful GIS system design and implementation. CPT functions contribute throughout the implementation cycle. The CPT tasks include a review of business needs, establish performance targets, identify user locations, review network suitability, select product architecture, and complete the system architecture design analysis. Additional tools are provided to validate system performance during the design, construction, and implementation phases. CPT models are very easy to adapt to changes in business requirements or system design configuration strategies, providing an adaptive model addressing a variety of incremental planning activities.
Best Practice: Build and maintain a system performance model that links user requirements with system design.
Review Business Needs
Understanding GIS technology alternatives, quantifying GIS user requirements, and establishing an appropriate system architecture design deployment strategy are critical during requirements definition. Capacity planning during this phase can help establish appropriate software performance specifications.
You must identify what you want to get out of the GIS before you are ready to complete the system design. There are several questions that must be answered. What are the information products (business needs) that enable effective business operations? What software technology patterns best support the identified business needs? What are the software processing loads that must be supported by available hardware platform and network infrastructure resources?
Figure 1-8 provides an introduction to the CPT Workflow and Calculator tabs. System Architecture Design begins with an understanding of user workflow requirements. The CPT is designed to support the workflow needs analysis. Standard Esri Workflows are identified on the CPT Workflow tab, providing the most common translation of workflow software technology patterns to software transaction service times (workflow performance targets).
Workflow service times provided on the Workflow tab are initially generated by the CPT Calculator. Software technology pattern and primary performance factors are identified to establish the selected workflow software deployment pattern. Excel generates the workflow name (recipe) and the associated workflow service times from a variety of lookup tables representing performance metrics gathered from Esri benchmark testing and a variety of software performance validation sources.
Common terminology used to describe components of the business workflow helps us understand system performance and scalability as we work through the System Architecture Design process. User is the person using the computer application to do work. Service time is the computer processing time required to complete an average application work transaction. Queue time identifies any delays caused by application executables (computer program instructions) waiting in line (behind competing executables) to be processed by the available (limited) hardware resources across a busy system. Response time is the total time from initial user request and final display delivery (sum of all service times and queue times across the system). These terms will be described in more detail in Chapter 9 Performance Fundamentals.
CPT Workflow tab
Figure 1-9 provides an overview of the CPT Workflow tab, identifying the primary sources for selecting your Design project workflow performance targets (Standard Esri Workflows, Calculator, Test, or Composite workflows). The CPT Workflow tab contains the lookup table for workflow performance targets (component software service times) used in the CPT Design tab.
A list of Standard Esri Workflows is maintained in the Workflow tab for use in selecting common workflow performance targets. The Standard Esri Workflows were generated from the CPT Calculator and the workflow name identifies the recipe (software technology and performance parameter selections).
Custom workflows configured on the CPT Calculator tab include an output to the CPT Workflow tab (Calculator Workflow section). The output includes the workflow recipe and service times generated by the Calculator workflow selection. Workflows generated from the CPT Calculator tab can be copied and entered into the Project Workflow section of the CPT Workflow tab.
The Project workflows section is established at the top of the workflow list by including a copy of Standard Esri Workflows or custom workflows in your project workflow list - this process is what I refer to as your user workflow analysis. Once your software technology selection is complete, your selected workflow performance targets will be available at the top of the workflow selection list for use in your system architecture design.
The composite workflow analysis section located at the bottom of the Workflow tab is used to calculate average service times for a use case that combines two or more component workflows. Results of the composite workflow analysis is included in the custom generate workflow section for use in building a single project workflow from multiple workflow sources – you can include a copy of the composite workflow analysis service times in your list of Project Workflows for use during your system design.
CPT Calculator tab
The CPT Calculator is a simple tool developed for use during business workflow analysis and software technology selection. Understanding user needs is the first step in building a GIS. Identifying the information product specifications and selecting the right software technology is an important part of the user needs analysis. The Calculator includes dropdown selection lists for the software technology pattern (software, map document, display complexity, and percent data cache) and key software performance parameters (display resolution, density, output format). The content on GIS Software Technology (Chapter 2) and Software Performance (chapter 3) will describe the available technology patterns and performance factors in more detail. Figure 1-10 provides an overview of the Capacity Planning Calculator tab.
The software technology and performance parameter selections generate GIS workflow performance targets (service times) that can be used for your system architecture design. The CPT Calculator generates workflow performance targets from software technology baselines and appropriate adjustment factors derived from Esri performance benchmarks. Workflow performance targets are updated with each Esri software release to incorporate the latest available performance metrics.
The CPT Calculator generates system architecture design and workflow performance information for the selected workflow. There is an option (cell C30) available to select your workflow source (Calculator or Workflow tab). User requirements (peak users and productivity or peak service transaction rates) identify system processing loads, and platform architecture (tiers, high availability, SDE connection, data source) and platform technology selection (platform configuration, rollover) complete the system design inputs. Client network bandwidth connectivity inputs (rows 12-14) provide a preliminary communications assessment and generate local and remote user relative workflow display performance expectations. The system design hardware solution is displayed in the platform solution section. The CPT Calculator was developed to provide a complete system design assessment for an informed software technology selection (single workflow) and generate workflow service times (Workflow tab) for use in building an enterprise solution (Design tab).
Identify Business Requirements
Once you have identified your project workflows, you are ready to complete your system design. The CPT Design tab (figure 1-11) includes a Requirements Analysis section for identifying user locations, project workflows used at each location, peak users and use productivity for each user workflow, peak service transaction rates for Web services, along with any concurrent batch processes you wish to include during peak processing loads.
The CPT Design requirements analysis section provides a way for business units to define their system design needs, using terms that represent their own user requirements terminology. Software performance targets from the selected project workflows will be used to translate these business requirements to network traffic loads and processing loads on the selected platform solution. Network and server platform capacity are presented in standard IT terminology (network traffic, bandwidth, server utilization, throughput, peak system capacity) Some key terminology used in translating business needs to system processing loads include user productivity, display cycle time, think time, and batch process. Productivity is a measure of user activity in doing work, Cycle time is the average time between user application information requests, Think time is the time available for user workflow input (computed and minimum think times), and Batch process is a workflow with zero minimum think time (no user input between service transactions). These terms will be described in more detail in Chapter 9 Performance Fundamentals.
Business Requirements Analysis
Figure 1-12 shows the CPT Design tab Requirements Analysis module. The grey rows represent the Data Center network connections (LAN, WAN, Internet) while the green rows represent remote site network connections. Workflows for WAN users are located under the Data Center LAN network. Remote user workflows are located below each remote site network, based on user work locations. Workflows are drop down selections from the CPT Workflow tab, importing workflow processing times from the CPT Workflow tab for use during the design analysis. Peak concurrent users and service transaction rates are identified in columns C and D for each workflow. Excel computes the total throughput for each workflow based on user productivity and identified peak service rates.
Peak concurrent users and workflow productivity establish system throughput loads that drive the system architecture design. These loads include network traffic that must travel between the user display location and the central data center. Understanding where the users are located and how these locations are connected with the central data center can directly impact software technology selection. Configuring the CPT Design tab to represent peak business requirements will be discussed in more detail in Chapter 5 Network Communications.
Network Suitability Analysis
The network suitability analysis is completed on the CPT Design tab. You will need information from the network administrator to identify remote site bandwidth connections and network latency (network latency between remote clients and the central data center can be measured using a PING or TRACERT command on the client computer. PING measures round trip time to the server, while TRACERT measures round trip time including intermediate router locations).
The terminology overview in Figure 1-13 shows a valid user workflow. A user workflow must include time between displays for user input (computed think time must be equal to or greater than minimum think time). As the system gets busy, the response time will grow (due to longer queue times) reducing the computed think time. If computed think time is less than the minimum think time, you have an invalid workflow and you need to reduce user productivity (slow down).
Network suitability is a critical design component for any distributed GIS environment (users working at external remote site locations located across a WAN or Internet connection). Network bandwidth constraints can reduce user productivity, and cost to resolve traffic bottlenecks can be cost prohibitive. It is important to confirm network infrastructure bandwidth is adequate to support peak user traffic loads before making a final technology decision.
Figure 1-14 provides an overview of the information needed to complete the network suitability analysis. Once the business requirements analysis is complete (user locations, peak workflow loads) available bandwidth for each network must be entered in column H. Excel completes the network suitability analysis, and identifies associated design conflicts. Analysis results provide information needed to complete the network design.
The diagram shows traffic exceeding bandwidth for the Data Center WAN, Remote site 1 WAN gateway, Data Center Internet, and Remote site 2 Internet gateway. Peak network traffic exceeds available bandwidth for each connection, showing RED cells indicating this is not a valid workflow. User productivity cells in column E are also RED, indicating user productivity must be reduced to support a valid workflow. Excel clearly identifies there is too much traffic and not enough bandwidth to support the desired workflow productivity. Network suitability analysis will be discussed in more detail in Chapter 5 Network Communications.
Product Architecture Selection
CPT Design tab Product architecture selection is shown in Figure 1-15. Product architecture is defined in the Software Configuration section (columns J through R). Client workflow software can be installed on any data center platform tier from a drop down menu selection. Software components for each user workflow are highlighted in the CPT for easy selection. Default install is provided on CPT row 6 for each software column.
Figure 1-16 shows how the product architecture is established on the CPT Design tab. The platform selection is normally configured first, and platform nicknames followed by a colon (:) can be identified for each tier just above the platform selection in column B. You can also set a rollover setting in column H, telling excel when to add new platforms.
Once the platform names are set, you can select the platform nicknames in the Software Configuration section. You have a choice of identifying a default configuration in row 6, or install each software component individually on each workflow row. This provides flexibility to install workflow software in any Data Center hardware configuration. Data source selection for each workflow is identified in column R. GIS Product architecture will be discussed in more detail in Chapter 6.
Hardware Platform Selection
Once the product architecture is established, you can select hardware for each platform tier in column B. Hardware performance metrics are pulled from the hardware tab shown in Figure 1-17. The hardware tab includes hardware platform processor, number of core, core per chip, processor MHz, and the SPEC relative performance benchmark measurements. This information is used by the CPT to complete the System Architecture Design. Platform Performance will be discussed in more detail in chapter 7.
Enterprise System Architecture Design
Figure 1-18 provides a summary of the analysis performed by the CPT Design tab. Once you configure the CPT Design tab to reflect your user requirements and identify the network connections, the CPT completes a network suitability analysis. You can then increase network bandwidth to accommodate peak traffic loads. Once you make your platform selections and configuration the workflow software (software install), the CPT will complete the platform sizing and show number of servers and peak utilization levels for each platform tier. Figure 1-19 provides an overview of the configured CPT Design tab. The Design tab provides a snapshot view of your user requirements and associated platform solution as a single integrated information product. User workflow requirements show the user location, the peak throughput for each workflow (peak users or transactions per hour), the network bandwidth connecting each user site with the central data center, results of the network suitability analysis showing workflow traffic, site traffic, and network utilization, and the relative display response time for each workflow at each user location (Workflow Performance Summary). The platform selection area shows your final physical or virtual server configurations along with a diagram of the final platform solution (number of platform per tier and peak server utilization).
Several Enterprise Design solutions can be saved using the Excel move or copy sheet commands, providing a final Design workbook with multiple snapshots detailing each step in your design process.
System Performance Validation
Figure 1-20 highlights the importance of performance validation as you begin to build and deploy your design. The CPT Test tab was designed to help with the validation process, and includes several tools that can be used to translate performance measurements to equivalent workflow service times. The Design tab model used workflow service times as an input to generate platform throughput and utilization loads for platform sizing. The Test tab using measured throughput and utilization values as input to calculate platform service times. You can compare measured service times with planned service times to see if you are delivering a system solution within your initial planning performance targets.
Measuring performance during the system build and deployment reduces implementation risk. System design is only the beginning of the process; then you need to build a system that performs within the design performance targets. If measured performance loads exceed planned budgets, adjustments can be made during the build process to deliver services within initial performance targets. Software Performance will be discussed in more detail in chapter 3, identifying key performance parameters and how to build high performance map services. These same recommendations can be applied to any user workflow. The key is to establish workflow performance targets and manage system build and delivery within planned performance specifications.
There are four Test tools shown in Figure 1-21 that you can use during system deployment. The first uses measured desktop map display rendering time to estimate deployed map service times (make sure map display complexity is within performance budgets during initial authoring of the map display). The second tool uses measured map service render time to estimate map service times (check of map service performance during initial deployment). Two other tools can be used in final test or system monitoring, which translate throughput and utilization measurements to equivalent software display service times you can compare with your design targets. Performance validation testing will be discussed in more detail when we review Performance Fundamentals in Chapter 9.
Figure 1-21 provides an overview of the CPT Test tab validation tools. Two tools are available for translating measured map rendering time to equivalent map service times (Desktop rendering time and Server rendering time). Two additional tools are available for translating system throughout and platform utilization to equivalent map service times (measured transaction throughput and platform utilization are used to identify software service times). One tool uses measured service throughput as a direct entry, while the other tool uses concurrent user loads times average user productivity to estimate throughput values. Vendor published relative platform performance benchmarks are used to translate test platform measurements to baseline platform service times used in the design.
Performance validation is an activity that normally occurs after enterprise planning, once you have an established your deployment budget and delivery schedule. There are several opportunities to measure performance during your system build – integrating performance measurements into your system delivery project schedule can reduce implementation risk. Performance anomalies identified and addressed early and often throughout deployment can ensure performance and scalability of the deployed system.
Additional CPT Design Tools
Figure 1-22 shows two additional system design tools that are included in the CPT. The first is a Platform Capacity Calculator and the second is a Platform Sizing Tab.
The Platform Capacity Calculator was developed to identify what customers can expect from a selected hardware platform configuration. Hardware platform capacity is calculated for five common standard Esri workflows. Tool provides capacity for both physical and virtual platform configurations, and generates results in either transactions per hour (TPH) or peak concurrent clients (users).
The Platform sizing tab was developed for technical marketing folks that wanted a step by step procedure for completing a single workflow sizing analysis. The platform sizing tab uses the CPT Calculator tab for the analysis, and provides a step by step procedure to complete the configuration process.
Platform Capacity Calculator
Figure 1-23 takes a closer looks at the Platform Capacity Calculator. This is a handy simple application for comparing different vendor server platforms, and evaluating whether those server platforms will support your peak throughput needs.
The platform configuration under consideration is entered in the top left of the calculator in cell A324. You can select whether you want the output in transactions per hour (TPH) or concurrent users (users) in cell C322. You can identify whether you want results based on a physical server configuration, or you would like results for virtual server configurations deployed on the selected physical server. The Platform Capacity Calculator generates an output range for five selected common Standard Esri Workflows, showing a medium capacity output in blue and light capacity output in red.
CPT Sizing tab
Figure 1-24 takes a closer look at the CPT Sizing tab. This tab provides a step by step process for completing a system architecture design for a single workflow. The CPT Calculator tab models are used to complete the analysis, and the inputs cells include step by step instructions along with hyperlinks to the System Design Strategies wiki site through the design process.
The design process includes the following steps:
1. Identify the user workflow (dropdown selection from the CPT Workflow tab)
2. Identify peak load (either in thousands of transactions per hour or peak users times user productivity).
3. Identify data source from a dropdown selection.
4. Select a product architecture (number tier, minimum or high available, and SDE connection)
5. Identify hardware selection (dropdown platform selection from hardware tab, rollover setting, and physical or virtual server configuration).
6. Evaluate remote user performance (identify peak users at two different remote sites).
7. Identify bandwidth and latency for each of the remote sites to complete the network evaluation. Any traffic constraints will be identified along with recommended bandwidth upgrades to support remote user performance needs.
8. Platform solution includes list of server platforms and a drawing of the final server configuration.
9. Workflow performance summary shows the relative display rendering performance between local and remote site user location based on the selected configuration.
Concluding remarks
Proper GIS planning is the most important investment any organization can make in building a GIS. Understanding your GIS needs, selecting the right technology at the right time, and establishing documented implementation milestones to measure your progress can ensure your success. This document is focused on sharing how to build and maintain successful GIS operations. The Capacity Planning Tools provide a framework for collecting what you know about your business needs and your system environment. The Capacity Planning Tool models connect what you understand about GIS user requirements with the network and platform loads your IT support teams can measure in the data center.
CPT Video: System Design Process
The next Chapter will review the most common GIS technology patterns and share best practices in making the right technology selection.
Previous Editions
System Design Process 29th Edition
System Design Process 28th Edition
System Design Process 27th Edition
Page Footer
Specific license terms for this content
System Design Strategies 26th edition - An Esri ® Technical Reference Document • 2009 (final PDF release)
