NG9-1-1 Considerations for PSAP Governing Authorities
NG9-1-1 is a crucial upgrade from the current 9-1-1 system, providing enhanced situational awareness and collaboration opportunities. The implementation process, timelines, and necessary actions for PSAPs and Governing Authorities are outlined, emphasizing the need for immediate planning, budgeting, and engagement with service providers and stakeholders to smoothly transition to NG9-1-1.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
BSAD 482: BUSINESS ANALYTICS The Data Warehouse - Architecture
Contents Define the concept of architecture Discuss the various components of data warehouse architecture Review different data architectures in a data warehouse Data warehouse Data Marts
Information Systems Architecture Information Systems Architecture definition is the process of making the key choices that are essential to the design of an information system (like a Data Warehouse). Business Strategy Architecture Removal of Choices Design Implementation Line of Code / Process Step Employee / Computer
Architecture or Design? The Customer Dimension will consist of the following 12 attributes We will use Microsoft SQL Server Integration Services (SSIS) as our ETL tool All data marts will employ star schema data models The lowest level of granularity in the sales fact table will be daily sales by store by product.
The Value of Architecture Communication: To business sponsors, and business users Between members of the project team Planning: Ensure that all important components of the system are accounted for Flexibility and Growth Thinking about overall architecture will reduce risk associated with the success of the data warehouse Productivity and Reuse
Architectural Principles Mature IS organizations make architectural choices that are driven by well thought out Guiding Principles: Truths that guide decision making and hold true irrespective of changes in goals, strategy, and leadership. Examples: We will seek to buy solutions vs. build them where possible. We strive for simplicity in our Technology Infrastructure Footprint. Solutions must align/integrate with Corporate Security standards We will implement solutions that emphasize ease of use for stakeholders. We strive to implement solutions that can support needs across the enterprise and allow data sharing as needed, subject to appropriate governance
Architectural Views Strategy Business Processes: support the strategy, Operational organization Business M tier The Applications support the Business, implement the business functions in the IT systems Applications The Information is key for the organization: It is the fuel that drives the architecture Information & Data Our Focus . Infrastructure that supports the IS: -Technical components: servers, networks, etc. -Technology: platforms, etc. Network & Infrastructure
BI Application Architecture No data marts option Applications (Visualization) Data Sources Access Routine Business Reporting ERP ETL Process Data mart (Marketing) Select Legacy Metadata Data/text mining A P I / Middleware Extract Data mart (Engineering) Enterprise Data warehouse Transform POS OLAP, Dashboard, Web Integrate Data mart (Finance) Load Other OLTP/wEB Replication ODS Data mart (...) Custom built applications External data
Whats driving the explosive growth of data warehouses? Business need . Organizations need great information to: Sense what is happening (and predict what will happen) Respond effectively by making fast, smart decisions Technical enablers Computing Power . Cost of Computing .
Moores Law and its counterparts Drivers: Moore s Law: computer chip performance per dollar doubles every eighteen months to 2 years: CPU, RAM, Flash Data transmission rates: the number of bits that can be transmitted / second between computers: as of 2005: 9 Gbit/sec; 2008: 38 Gbit/second. Data storage (magnetic disk drives): 1 TB drive $100?
Whats Special About Data Warehouse Architecture? Transaction processing systems growth is (relatively) predictable Example: A company uses SAP for order processing They are opening a new retail store They predict (based on experience) 2000 transactions per week To process this volume, we need 3 workstations to capture the transactions Peak time each day is 11-2 when 50% of transactions occur
Whats Special About Data Warehouse Architecture? Success drives explosive growth More users More (complex) queries More data Performance is non- deterministic Unpredictable queries Unpredictable use patterns Data Warehouse Growth SAP Banner Time
Data Warehouse/BI Architecture Choices need to be made regarding: Choice of Approach/Tools for ETL, Quality Management, User Access Data architecture: the way data is organized and managed in the data warehouse Server Architecture/Processing Architecture . Degree of Parallelism needed Database Management Software Metadata management: approach for managing technical, operational and user metadata Network connectivity BI Tools (the tools that the users access) Many more Our focus today: Data Architecture But first a word on parallelism
14 Parallelism Essentially, parallel computing improves performance by breaking up the task and working on the pieces simultaneously. Performance is improved either through speeding the process (doing the same amount of work faster) or scaling up (doing more work in the same amount of time). In most cases, a data warehouse needs both types of performance gains over its life cycle.
15 Parallelism Hardware Vendors offer different architectures to achieve parallelism: Symmetric Multi-Processing (SMP) Single computer, 2:N processors sharing a common system bus, memory and disk array Massively Parallel Processing (MPP) A series of independent nodes interconnected by high speed switch Each node is a computer controlling its own processor(s), memory and disk Nodes work independently, activity coordinated by a master program Hybrid (common today) Applications: BI, Big Data Analytics, Watson
Build vs. Buy MANY options today for acquiring Information Technology: Build it your self Buy a package (SAP) Rent a package (SAAS) There are also options about where you run the technology Host it in your own data centre Have someone else host it in the Cloud With a data warehouse, we buy a set of tools, and typically have to build the DW AND the majority of our BI applications
A Reference DW Data Architecture No data marts option Applications (Visualization) Data Sources Access Routine Business Reporting ERP ETL Process Data mart (Marketing) Select Legacy Metadata Data/text mining A P I / Middleware Extract Data mart (Engineering) Build Enterprise Data warehouse Transform POS OLAP, Dashboard, Web Integrate Data mart (Finance) Today often in Cloud Load Other OLTP/wEB Replication ODS Data mart (...) Custom built applications External data ETL Tools DBMS BI Tools Buy Metadata Tools
BSAD 482: BUSINESS ANALYTICS The Data Warehouse - Architecture
DATA ARCHITECTURE Databases Data Warehouse Data Architecture Data Mart Data Architecture
What is a Database? At the core of a data warehouse is a huge database Databases are used to efficiently store and manage large amounts of data. Managed by a special class of software called database management software (DBMS)
Database Basics Database tables store information and resemble Excel worksheets. Tables consist of: fields (columns) records (rows) Tables are connected (or related) through keys. Tables normally store information about one specific type of data.
Data Architecture Components Enterprise data warehouse (EDW) A centralize data store for the enterprise. Goals: Flexibility, detail, history Feeds Data Marts Data Mart A subset of DW data that stores only data relevant to a specific user group, department, business area Dependent data mart A subset that is created directly from a data warehouse Independent data mart A data mart created directly from source systems
Data Architecture Components Operational data stores (ODS) A type of data store used to provide access to information required in real time or near real time Metadata Information about the data in a data warehouse. Metadata describes: Meaning Quality Lineage Access Methods (where, how, who)
Data Warehouse Data Architecture The purpose of the data warehouse is to be the primary repository for data that will feed the BI access layer (the data marts). It is designed to provide the flexibility to meet both current and future data needs. Generally, data is highly normalized as this provides the greatest flexibility. The design based on an enterprise data model NOT any specific application.
Data Warehouse Data Architecture (cont) Key characteristics: The data in the data warehouse should generally be stored at an atomic (transaction) detail in order to provide flexibility to meet future needs. The data warehouse is responsible for maintaining historical data required for decision support purposes. The data warehouse is designed for flexibility and load performance, but not for direct access by end-users. The exception to this rule is for specialized analytics such as exploratory data mining.
DATA MART DATA ARCHITECTURE Relational Star Schema Multi-dimensional
27 Data Mart Data Model: Star Schema Facts contains information about things that an organization wants to measure (aka measures ). A measure provides some indication of performance and on which calculations (e.g., sum, count, average, minimum, maximum) can be made Dimensions and Hierarchies A dimension is a structure that categorizes facts and measures in order to enable users to answer business questions. A hierarchy contains different levels for a dimension: Product Cat Product Sub-Cat Product VIDEO
28 Star Schema Example Dimension Tables ( Descriptive attributes of ordered tests ) Fact Table ( Instances of ordered tests )
29 Facts & Dimensions DIM_TEST TEST_ID MEDITECH_ID TEST_DESC TEST_DEPARTMENT PROFILE_TEST 424 HGB HEMOGLOBIN HAEMATOLOGY N FACT_ORDERED_TEST INDIVIDUAL_TEST_ID TEST_ID ORDER_PROVIDER _ID ORDERED_FROM_LOCATION _ID 405732 424 1024 83 Might be 1000s of fact rows referencing this specific type of test. All would have Test_ID = 424 Test_ID is KEY that a) uniquely identifies each test in DIM_TEST and b) allows us to link Fact_Ordered_Test with Dim_Test to retrieve descriptive info about tests
Simple Example An auto company wants to analyze its auto service business. Key information needed includes: Date of service (want to be able to categorize by week, month, year) Customer Vehicle (VIN, model, type, manufacturer) Location of Service (assume many locations, want to know garage, city, province) Service Technician Service Reason (assume standard categories) Tech time spent, in minutes Customer wait time, in minutes Total time vehicle in garage, in minutes Cost of Service Labour Cost of Service - Parts
Service Star Schema DIM_SERVICE_DATE DATE WEEK MONTH YEAR DIM_CUSTOMER FACT_SERVICE TECH_TIME_MIN WAIT_TIME_MIN TOTAL_TIME_MIN LABOUR_COST PARTS_COST DIM_VEHICLE VIN MODEL TYPE MANUFACTURER DIM_SERVICE_TECH DIM_LOCATION GARAGE CITY PROVINCE DIM_SERVICE_RSN
Scenario GASHA (the local health district) wants to build a data mart to analyze patient stays for surgery. The following is the key information needed: Patient information (name, address, age, gender, etc.) Total time patient waited for a bed in days Total time of stay in hospital in days Cost of Drugs Cost of Nursing Care Primary Physician and the specialty area of that physician Cost of surgery rental Fee paid to Physician Hospital where surgery took place Hospital Unit where patient bed located Condition requiring surgery (assume we have a standard list) and the category for the condition Start and End Date of Stay (want to analyze by day of the week, month, and year) Draw a simple star schema to represent this scenario What are the measures (all in one table) and what are the dimensions (separate tables)?
MDBMS A 3-dimensional OLAP cube with slicing operations Sales volumes of a specific Product on variable Time and Region In most cases we use relational databases to store data warehouse data Multidimensional database: A form of data mart implemented via a specialized multidimensional database software that pre- calculates / summarizes ALL measure/dimension intersections in advance (as part of ETL) Also called an OLAP database Time Product Geography Cells are filled with numbers representing sales volumes Sales volumes of a specific Region on variable Time and Products Pros: very, very fast answers Sales volumes of a specific Time on variable Region and Products Cons: lack of flexibility, limits on data volumes / level of detail VIDEO
Detailed data Relational table: Assume millions of rows
From a star schema perspective Dim_Model Dim_Color Type Model Color Fact_price Type Car Car Car SUV SUV Truck Truck Truck Model Fiesta Fusion Taurus Edge Explorer F150 F250 Ranger Color Blue Green Red Yellow Price_dollars
OLAP representation Dimension: Color Measure: Price (PRE- CALCUATED) Dimension: Model. Has a hierarchy Type Model
OPTIMAL ARCHITECTURE?
What is the optimal DW architecture? (a) Independent Data Marts Architecture ETL End user access and applications Source Systems Staging Area Independent data marts (atomic/summarized data) (b) Data Mart Bus Architecture with Linked Dimensional Datamarts ETL Dimensionalized data marts linked by conformed dimensions (atomic/summarized data) End user access and applications Source Systems Staging Area (c) Hub and Spoke Architecture (Corporate Information Factory) ETL End user access and applications Source Systems Staging Area Normalized relational warehouse (atomic data) Dependent data marts (summarized/some atomic data)
What is the optimal DW architecture? (d) Centralized Data Warehouse Architecture ETL Normalized relational warehouse (atomic/some summarized data) End user access and applications Source Systems Staging Area (e) Federated Architecture Data mapping / metadata End user access and applications Logical/physical integration of common data elements Existing data warehouses Data marts and legacy systems Each architecture has advantages and disadvantages!
The Great Data Architecture Debate Sources Enterprise data warehouse with dependent marts Pros: more scalable, more efficient in the long run Cons: early implementations time consuming and expensive, complexity Independent data marts Pros: simple, faster to implement Cons: not scalable, inefficient, difficult to maintain when large number. DW Sources
Factors that impact data architecture. Information Interdependence between Organizational Units: Higher the need to share information across organizational units, greater likelihood of enterprise DW Upper Management s Information Needs: Senior management needs information from lower organizational levels to do their job. Higher this need, greater likelihood of enterprise DW. Urgency of Need for a Data Warehouse: The more time pressures to implement, greater likelihood that there will NOT be enterprise DW.
Factors that impact data architecture. Nature of End-User Tasks: Some users perform non- routine tasks. Structured queries and reports are insufficient for their needs. Some users need access to enterprise wide data (not just a specialized mart) Constraints on Resources: Some data warehouse architectures require more resources (IT and business personnel, money) than others. Greater resource constraints, less likely enterprise DW built Strategic View of the Data Warehouse Prior to Implementation: The more strategic the approach, the greater the likelihood of enterprise DW.
Factors that impact data architecture. Compatibility with Existing Systems: Operational systems in place may dictate architecture. Example: SAP and SAP BW. Perceived Ability of the In-House IT Staff: Building a data warehouse can challenging, and some architectures are more difficult to build. Staff skills influences ambition of scope. Less skill, less likely to build enterprise DW Technical Factors: Number of users, volume of data, query performance requirements, stability of source systems all influence choices. Cultural/Political Factors: Influence of experts, culture (do we share data well?), influence of current technical team, etc.
