Data Analysis Essentials Across Sectors

Dr.SNS RAJALAKSHMI COLLEGE OF ARTS & SCIENCE, COIMBATORE. Autonomous 1 DATA ANALYSIS USING PYTHON DATA ANALYSIS USING PYTHON M.Rajeswari, Assistant Professor, Department of Computer Science with Data Analytics, Dr.SNS Rajalakshmi College of Arts & Science, Coimbatore. SNS Design Thinkers/M.Rajeswari/Data Analytics

DATA ANALYSIS 2 What is Data Analysis? Data analysis is an essential aspect of modern decision-making processes across various sectors, including business, healthcare, finance, and academia. As organizations generate massive amounts of data daily, understanding how to extract meaningful insights from this data becomes crucial. The process can be broken down into several steps, including: Data Collection: Gathering relevant data from various sources, which could be databases, surveys, sensors, or web scraping. Data Cleaning: Identifying and correcting inaccuracies or inconsistencies in the data to ensure its quality and reliability. Data Transformation: Modifying data into a suitable format for analysis, which may involve normalization, aggregation, or creating new variables. Data Analysis: Applying statistical methods and algorithms to explore the data, identify trends, and extract meaningful insights. Data Interpretation: Translating the findings into actionable recommendations or conclusions that inform decision-making. SNS Design Thinkers/M.Rajeswari/Data Analytics

DATA ANALYSIS 3 Knowlwdge Domain of Data Analyst domain knowledge gives data analysts the necessary context for turning data into actionable insights. It allows them to correctly interpret data, identify meaningful patterns, and provide recommendations addressing real-world business issues. Understanding the nature of the data Understanding the nature of data is important in data analytics because it helps determine which analytical methods, statistical techniques, and visualization approaches to use. The nature of data refers to its characteristics, types, and properties, including its structure, format, scale, and distribution. In order to understand the nature of data it is necessary to categorise them into various types. Different categorisations of data are possible. The first such categorisation may be on the basis of disciplines, e.g., Sciences, Social Sciences, etc. in which they are generated. SNS Design Thinkers/M.Rajeswari/Data Analytics

QUANTITATIVE AND QUALITATIVE DATA ANALYSIS 4 Qualitative and quantitative data The internet has enabled us to create large volumes of data at a staggering pace. Moreover, the way you analyze it depends on the type of data you are working with. We broadly classify data into two forms qualitative and quantitative. Quantitative data analysis Quantitative data analysis is a more traditional form of analysis. As mentioned earlier, this process crunches numbers to get results. Since one of the major functions of this process is to run algorithms on statistical data to obtain the outcome, the methods used in quantitative data analytics range from basic calculations like mean, median, and mode to more advanced deductions such as correlations and regressions. Some of the scopes of quantitative data analysis include: Project management, Marketing, Finance, Research and Development and Product planning. SNS Design Thinkers/M.Rajeswari/Data Analytics

QUANTITATIVE AND QUALITATIVE DATA ANALYSIS 5 Qualitative data analysis Qualitative data analysis is used when the data you are trying to process cannot be adjusted in rows and columns. It involves the identification, examination, and elucidation of themes and patterns in data (mostly textual) to bolster the decision-making process. Unlike quantitative analysis, qualitative data analysis is subjective. This method of analysis allows us to move beyond the quantitative traits of data and explore new avenues to make informed decisions. The following are some of the scopes of qualitative data analysis: Measuring customer satisfaction Monitoring competition Analyzing customer behavior Evaluating market trends SNS Design Thinkers/M.Rajeswari/Data Analytics

PYTHON AND DATA ANALYSIS 6 Introduction To Python Python is a widely used high-level, interpreted programming language. It was created by Guido van Rossum in 1991 and further developed by the Python Software Foundation. It was designed with an emphasis on code readability, and its syntax allows programmers to express their concepts in fewer lines of code. Python is a programming language that lets you work quickly and integrate systems more efficiently. Key Features of Python Python is Easy to Learn and Use: There is no prerequisite to start Python, since it is Ideal programming language for beginners. High Level Language: Python don t let you worry about low-level details, like memory management, hardware-level operations etc. Python is Interpreted: Code is executed line-by-line directly by interpreter, and no need for separate compilation. Which means You can run the same code across different platforms. You can make the changes in code without restarting the program. SNS Design Thinkers/M.Rajeswari/Data Analytics

THE PROGRAMMING LANGUAGE PYTHON 7 PYTHON Python is a high-level, general-purpose programming language that is used for a variety of tasks, including: Web development: Python is used to build websites and web applications Software development: Python is used for software development, including testing, prototyping, and build control Data science: Python is used for data analysis and machine learning System scripting: Python is used for system scripting Python is popular because it is easy to learn, efficient, and can run on many platforms. It is also beginner-friendly, with a syntax that is similar to English and handles much of the complexity for the user. Some benefits of Python include: Readability Python's design philosophy emphasizes code readability with the use of significant indentation. Reusable code Python has a large standard library that contains reusable codes for almost any task. SNS Design Thinkers/M.Rajeswari/Data Analytics

PYTHON 2 AND PYTHON 3 8 What is Python 2? Python 2.0 was introduced to the tech world in the year 2000. Created by the BeOpen Python Labs team, the purpose of the introduction of Python 2 was to make programming simple and easy to learn for the common masses. Python 2 was successful in implementing the technical details of the Python Enhancement Proposal (PEP). What is Python 3? Released in the year 2008, Python 3 was not just another version of Python 2 after debugging. The introduction of Python was mostly surrounded by the motive that redundancy writing repetitive code or writing the same piece of code again and again should be removed from coding. Python 3 is backwards incompatible and aims at eliminating the problems which new programmers face while learning a programming language. SNS Design Thinkers/M.Rajeswari/Data Analytics

PYTHON DISTRIBUTIONS 9 Python Distributions A Python distribution is a software bundle, which contains a Python interpreter and the Python standard library. Installer programs for common operating systems are a frequent mode of distribution. Many Python distributions also have package managers so that you can install or upgrade various Python packages. Some of the most popular distributions are listed below. Distributions which are marked as scientific are ones which come with IPython, numpy, pandas, and matplotlib, at a minimum. All of the distributions provide at least one integrated development environment (IDE) for free. A Python IDE provides a Python-aware code editor integrated with the ability to run code from that editor. SNS Design Thinkers/M.Rajeswari/Data Analytics

IPYTHON 10 IPython Standard distribution of Python comes with a REPL (Read-Evaluate-Print Loop) environment in the form of Python shell with >>> prompt. IPython (stands for Interactive Python) is an enhanced interactive environment for Python with many functionalities compared to the standard Python shell. Features of IPython IPython offers more features compared to the standard Python. They are as follows Offers a powerful interactive Python shell. Acts as a main kernel for Jupyter notebook and other front end tools of Project Jupyter. Possesses object introspection ability. Introspection is the ability to check properties of an object during runtime. Syntax highlighting. Stores the history of interactions. Tab completion of keywords, variables and function names. SNS Design Thinkers/M.Rajeswari/Data Analytics

Python IDEs 11 Python IDEs The term "IDE" refers for "Integrated Development Environment," which is a coding tool that aids in automating the editing, compiling, testing, and other steps of an SDLC while making it simple for developers to execute, write, and debug code. It is specifically made for software development and includes a number of tools that are used in the creation and testing of the software. There are some Python IDEs which are as follows: PyCharm, Spyder, PyDev, Atom, Wing, Jupyter Notebook, Thonny, Rodeo, Microsoft Visual Studio and Eric SNS Design Thinkers/M.Rajeswari/Data Analytics

The NumPy Library 12 What is NumPy? NumPy is a Python library used for working with arrays. It also has functions for working in domain of linear algebra, fourier transform, and matrices. NumPy was created in 2005 by Travis Oliphant. It is an open source project and you can use it freely. NumPy stands for Numerical Python. History of NumPy NumPy is a foundational Python library that provides array data structures and related fast numerical routines. When started, the library had little funding, and was written mainly by graduate students many of them without computer science education, and often without a blessing of their advisors. Nowadays, NumPy is relied upon by scientists, engineers, and many other professionals around the world. For example, the published scripts used in the analysis of gravitational waves import NumPy, and the M87 black hole imaging project directly cites NumPy. SNS Design Thinkers/M.Rajeswari/Data Analytics

NumPy Installation 13 Installing NumPy The only prerequisite for installing NumPy is Python itself. If you don t have Python yet and want the simplest way to get started, we recommend you use theAnaconda Distribution - it includes Python, NumPy, and many other commonly used packages for scientific computing and data science. NumPy can be installed with conda, with pip, with a package manager on macOS and Linux, or from source. For more detailed instructions, consult our Python and NumPy installation guide below. CONDA If you use conda, you can install NumPy from the defaults or conda-forge channels: PIP If you use pip, you can install NumPy with: pip install numpy Also when using pip, it s good practice to use a virtual environment - see Reproducible Installs below for why, and this guide for details on using virtual environments. SNS Design Thinkers/M.Rajeswari/Data Analytics

Ndarray : The Heart of the Library 14 Numpy | ndarray N-Dimensional array(ndarray) in Numpy Array in Numpy is a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers. In Numpy, number of dimensions of the array is called rank of the array.Atuple of integers giving the size of the array along each dimension is known as shape of the array. An array class in Numpy is called as ndarray. Elements in Numpy arrays are accessed by using square brackets and can be initialized by using nested Python Lists. Array Creation There are various ways to creaFor example, you can create an array from a regular Python list or tuple using the array function. The type of the resulting array is deduced from the type of the elements in the sequences. To create sequences of numbers, NumPy provides a function analogous to range that returns arrays instead of lists. arange: returns evenly spaced values within a given interval. step size is specified. linspace: returns evenly spaced values within a given interval. num no. of elements are returned.te arrays in NumPy. SNS Design Thinkers/M.Rajeswari/Data Analytics

Conditions and Boolean Arrays 15 Numpy Boolean Array The Numpy boolean array is a type of array (collection of values) that can be used to represent logical True or False values stored in an array data structure in the Python programming language. The use of a boolean array in conjunction with logic operators can be an effective way to reduce runtime computational requirements when a single logical value is needed from one or more complex variables. Boolean Arrays also find their usefulness in resultant arrays, on performing some operations. While there may at first seem to be little use for such a construct, it is particularly important to beginners, who will often find themselves using boolean variables and arrays before they are familiar with other complex Python data types with greater flexibility. Boolean Arrays in Python are implemented using the NumPy python library. Numpy contains a special data type called the numpy.BooleanArray(count, dtype=bool) . This results in an array of bools(as opposed to bit integers) where the values are either 0 or 1. SNS Design Thinkers/M.Rajeswari/Data Analytics