Pandas¶

Table of Contents¶

1. Introduction to Pandas¶

What is Pandas?¶

Pandas is a powerful, open-source data analysis and manipulation library for Python. It provides data structures and functions needed to work with structured data seamlessly.

Key Features¶

DataFrame: 2D labelled data structure (like a spreadsheet or SQL table)

Series: 1D labelled array

Data alignment: Automatic and explicit alignment

Handling missing data: NaN representation and handling

Reshaping and pivoting: Flexible reshaping operations

Grouping and aggregation: SQL-like operations

Time series functionality: Date range generation and frequency conversion

Integration: Works seamlessly with NumPy, Matplotlib, and other libraries

Why Use Pandas?¶

Data Manipulation: Easy filtering, sorting, grouping

Data Cleaning: Handle missing values, duplicates, type conversions

Data Analysis: Statistical operations, aggregations

Data I/O: Read/write various formats (CSV, Excel, SQL, JSON, etc.)

Performance: Optimized C/Python implementations

Intuitive: SQL-like and spreadsheet-like operations

2. Installation and Setup¶

Installing Pandas -¶

Importing Pandas¶

Common Display Options¶

2. Pandas Series¶

A Series is a one-dimensional labeled array capable of holding any data type.

Creating Series¶

Series Attributes¶

Series Operations¶

Series Indexing and Selection¶

4. Pandas Dataframe¶

A DataFrame is a 2D labeled data structure with columns of potentially different types.

Creating DataFrames¶

Dataframe Attributes¶

Adding and Removing Columns¶

Adding and Removing Rows¶

5. Reading and Writing Data¶

Reading CSV Files¶

Writing CSV Files¶

Reading Excel Files¶

Writing Excel Files¶

Reading JSON Files¶

Writing JSON Files¶

6. Data Selection and Indexing¶

Column Selection¶

Row Selection¶

loc - Label-based Indexing¶

iloc - Position-based Indexing¶

at and iat - Fast Scalar Access¶

Boolean Indexing¶

Setting Values¶

7. Data Inspection¶

Basic Information¶

Statistical Summary¶

Unique Values and Counts¶

Memory Usage¶

8. Data Cleaning¶

Handling Missing Data¶

Removing Duplicates¶

Data Type Conversion¶

Renaming Columns¶

Handling Outliers¶

9. Data Transformation¶

Applying Functions¶

Sorting¶

Ranking¶

10. Grouping and Aggregation¶

Basic Groupby¶

Aggregation Functions¶

Transform¶

Filter Groups¶

Apply Custom Functions¶

Pivot and Crosstab¶

11. Merging, Joining and Concatenating¶

Concatenating DataFrames¶

Merging Dataframes¶

Join Method¶

12. Pivot Tables and Reshaping¶

Pivot Tables¶

Melt (Unpivot)¶

Stack and Unstack¶

Transpose¶

13. String Operations¶

String Methods¶

Pattern Matching¶

String Concatenation¶

14. Handling Categorical Data¶

Creating Categorical Data¶

Categorical Operations¶

Benefits of Categorical Data¶

15. Visualization with Pandas¶

Basic Plots¶

Different Plot Types¶

Customization¶

Subplots¶

16. Performance Optimization¶

Efficient Data Types¶

Vectorization¶

Chunking Large Files¶

Efficient Merging¶

Query Optimization¶

Use eval for Complex Expressions¶

17. Best Practices¶

Code Organization¶

Naming Conventions¶

Data Validation¶

Error Handling¶

Documentation¶

18. Resources¶