Data Mining &

Database Applications
Overview
Explore complementary fields: pattern extraction and data storage.
Key roles span marketing to operations. Benefits include informed
decisions and process optimization. Impact crosses finance,
healthcare, retail, and government. Challenges arise with growing
data volumes. Synergy exists between mining algorithms and
database performance. Scalable infrastructure and governance are
vital. Integration with BI tools is common. Ethical data use gains
focus.
by Muziyonge Liberty
Introduction to Data Mining
Data mining extracts patterns from diverse sources like
logs, sensors, and social media.

Primary goals: classification, clustering, regression,

association, anomaly detection.

Structured and unstructured data require quality

cleaning and feature engineering.
CRISP-DM Process Breakdown
1 Business Understanding
Define KPIs and metrics to guide analysis.

2 Data Preparation
Clean, transform, and normalize data for modeling.

3 Modeling & Evaluation

Choose algorithms, tune parameters, assess accuracy and business fit.

4 Deployment & Feedback

Implement scoring and iterate with documentation and collaboration.
Core Data Mining Techniques
Classification Regression Clustering Anomaly
Detection
Decision trees, Linear, logistic, K-means, DBSCAN,
random forests, ridge, lasso, hierarchical, Isolation Forest,
SVM, neural support vector Gaussian mixtures one-class SVM,
networks regression statistical
thresholds
Decision Tree Deep Dive
Hierarchical binary splits classify data by maximizing
information gain.

Pre-pruning limits depth; post-pruning reduces errors.

Handles missing and categorical data efficiently.

Used in credit scoring, churn prediction, and medical

diagnosis.
Data Mining Case Studies
Retail Basket Analysis
Uncover cross-sell opportunities to boost sales.

Telecom Churn Model

Reduce customer attrition by 15% with predictive analytics.

Fraud Detection
Identify banking anomalies in real time.

Healthcare Prognosis
Stratify patient risk for better outcomes.
Database Systems Overview
Relational DBs ensure ACID compliance and normalization.

OLTP handles transactions; OLAP supports analytics.

NoSQL types include document, key-value, column-

family, and graph stores.

Scalability via sharding, replication, and clustering.

In-Database Analytics &
Architectures
Push Analytics to DB Engine
Use SQL extensions and UDFs for mining algorithms.

Open-Source & Commercial Tools

PostgreSQL MADlib, Oracle Data Mining, SQL Server ML Services.

Benefits
Reduce ETL, improve security, enable streaming analytics.

Hybrid Architectures
Combine SQL and NoSQL with microservices or monolithic apps.
Best Practices & Future Directions

Governance & Ethics Scalability & Automation

Lineage, stewardship, GDPR, 1 Cloud elasticity, AutoML, MLOps
HIPAA compliance, anonymization. 2 pipelines.

Emerging Trends
Explainability & Real-Time
Graph analytics, augmented 4 Transparent AI, edge analytics,
analytics, quantum computing, 3 stream processing.
federated learning.
Data Management Concepts and Terms
Data Storage Data Integrity
Where and how data is saved, including Ensuring accuracy and consistency of data over
databases and cloud systems. its lifecycle.

Data Governance Metadata

Policies and standards governing data access and Data about data, describing its source, format,
security. and usage.
Data Governance
Policy Framework Compliance Standards
Establishes rules for data quality, access, and usage. Ensures adherence to GDPR, HIPAA, and industry
regulations.

Data Stewardship Audit and Lineage

Accountability roles to maintain data integrity and Tracks data origin and transformations for
security. transparency.
Data Quality
Data quality ensures accuracy, completeness, and reliability for analysis.

Maintaining high data quality reduces errors and supports better decisions.

Accuracy Completeness
Measures how correct and error-free data is Ensures all required data is present with no
compared to real-world values. missing values.

Consistency Timeliness
Data should be uniform across different sources Data must be up-to-date for relevant and
and times. effective use.
Types of Data
• Structured Data: Organized in tables with rows and columns, easy to query.
• Unstructured Data: Includes text, images, videos, and social media content.
• Semi-Structured Data: Has some organization, like JSON or XML files.
• Transactional Data: Captures business transactions and events over time.
Master Data
Master data represents the critical business entities shared across an organization.

It ensures consistency, accuracy, and control for key data like customers and products.

Centralized Control Data Consistency Integration Ready

Single source of truth for Eliminates duplicates and Supports seamless integration
essential data shared across discrepancies to boost with applications and analytic
departments. decision quality. platforms.
Transactional Data
Definition Key Characteristics Business Impact

Transactional data captures • Time-stamped and sequential Supports operational systems,

detailed records of business events • Contains quantities, prices, analytics, and customer insights.
and exchanges. dates, and parties involved
Enables tracking of sales, orders,
• Often voluminous and payments, and interactions in real-
continuously generated time.
Key Terms in Data Management
Data Governance Data Quality
Framework that defines rules and accountability for Measures accuracy, completeness, consistency, and
data use and quality. timeliness of data.

Master Data Transactional Data

Core business entities ensuring consistent and Records of daily business operations supporting real-
integrated data across systems. time analytics.
Logical Database
A logical database defines how data is organized and accessed without concern for physical storage.

It structures data abstractly to support efficient querying and data manipulation.

1 Data Models 2 Schema 3 Query Logic

Defines relationships between Blueprint of database Supports data retrieval
data entities, such as structure including tables, through structured queries
relational or object-oriented fields, and constraints. without exposing physical
models. storage details.
Key
Characteristics
of a Logical
Database
A logical database defines data organization and access abstractly.

It focuses on efficient querying without concern for physical storage.

Physical Database
The physical database defines how data is stored on hardware.

It includes file structures, indexing, and storage optimization techniques.

Storage Structures Indexing Techniques Performance

Optimization
Data files, pages, and blocks Improve data retrieval speeds
organize data at the hardware by creating efficient access Includes caching, partitioning,
level. paths. and data compression
methods.
Key Differences: Logical vs. Physical Databa
Feature Logical Database Physical Database

Data Organization Abstract structure focusing on Concrete storage on hardware

data relationships including files and blocks

Focus Efficient querying and data Optimization of storage and

manipulation retrieval speed

Storage Concern No concern for physical storage Manages file structures, indexing,
details and partitions

Performance Techniques Relies on schema and query logic Uses caching, compression, and
indexing
Importance of
Both Logical and
Physical
Databases
Logical and physical databases complement each other to optimize
data management.

Together, they ensure efficient data organization, access, and

performance.
What is a Database Management System (D
Central Software Data Organization
Manages creation, retrieval, and updating of data Helps structure data and enforce rules for data
in databases. integrity.

User Interaction Supports Multiple Databases

Provides tools for users to query, modify, and Enables management of multiple databases
control data access. simultaneously in one system.
Functions of a DBMS
• Data Storage Management: Efficiently stores and retrieves data for user applications.
• Data Security: Controls access to protect data integrity and privacy.
• Data Manipulation: Supports querying, updating, and managing data.
• Backup and Recovery: Ensures data is safely backed up and recoverable after failures.
• Multi-User Access: Manages concurrent data access without conflicts.
Types of Database Management Systems (DBMS)
Type Description Example

Hierarchical DBMS Organizes data in a tree-like structure IBM Information Management System
with parent-child relationships. (IMS)

Network DBMS Data is represented as records Integrated Data Store (IDS)

connected by links, supporting many-
to-many relations.

Relational DBMS Stores data in tables with rows and Oracle Database, MySQL
columns, supports SQL queries.

Object-oriented DBMS Manages data as objects, integrating db4o, ObjectDB

database capabilities with object
programming.

Document-oriented DBMS Stores semi-structured data as MongoDB, CouchDB

documents, ideal for flexible schemas.
Components of a DBMS
• Database Engine: Core service for storing, processing, and managing data.
• Database Schema: Defines structure, tables, fields, and relationships.
• Query Processor: Interprets and executes database queries.
• Transaction Manager: Ensures data consistency and integrity during transactions.
• Storage Manager: Handles physical data storage and file management.
Advantages of Using a DBMS
• Improved Data Sharing: Facilitates concurrent access by multiple users securely.
• Enhanced Data Security: Provides robust access control and user authentication features.
• Data Consistency: Manages transactions to maintain accuracy and integrity.
• Backup and Recovery: Automates data backups and simplifies recovery after failures.
• Efficient Data Management: Streamlines data storage, retrieval, and updates for applications.
Disadvantages of DBMS
• Complexity: DBMS software can be complicated to install and manage effectively.
• Cost: Licensing, hardware, and maintenance expenses can be significant.
• Performance: Large-scale DBMS may experience slower responses under heavy loads.
• Security Risks: Centralized data increases impact in case of security breaches.
• High Resource Use: DBMS requires substantial computing power and storage capacity.
What is Data
Mining?
Data mining uncovers valuable patterns and insights from large
datasets.

It combines statistics, machine learning, and database technology.

This process helps organizations make informed decisions by

extracting meaningful information.
The Data Mining
Process (Part of KDD)
Data Mining is a crucial step within the Knowledge Discovery in Databases (KDD) framework.

It involves discovering meaningful patterns and insights from large datasets.

Data Selection
Choose relevant data from the database for mining.

Data Preprocessing
Clean and prepare data to improve mining quality.

Data Transformation
Convert data into suitable formats for mining tasks.

Data Mining
Apply algorithms to extract patterns and models.

Interpretation/Evaluation
Analyze results for usefulness and relevance.
Techniques Used in Data Mining
Technique Purpose Example

Classification Assigns items to predefined Email spam filtering

categories

Clustering Groups similar items without pre- Customer segmentation

labeled categories

Association Rule Mining Discovers relationships between Market basket analysis

variables

Regression Predicts continuous values based Sales forecasting

on input variables

Anomaly Detection Identifies rare or unusual data Fraud detection

patterns
Tools for Data Mining
Weka RapidMiner
Open-source tool offering a collection of machine Provides an easy visual interface to design and
learning algorithms for data mining tasks. execute data mining workflows.

R and Python SQL-based Tools

Popular programming languages with extensive Use in-database analytics to mine data directly
libraries for data analysis and mining. within database systems efficiently.
Applications of Data Mining
Marketing Finance
Segment customers to tailor campaigns and Detect fraud patterns and predict credit risk to
increase sales effectiveness. secure assets.

Healthcare Retail
Analyze patient data for early disease detection Discover product associations for optimized
and personalized treatment. inventory and promotions.
Benefits of Data Mining
Enhanced Decision Making Increased Efficiency
Uncover hidden patterns to support data-driven Automate data analysis to save time and reduce
decisions. errors.

Competitive Advantage Risk Management

Identify market trends and customer preferences Detect anomalies to prevent fraud and minimize
early. losses.
ZVAMAREWA TINOTENDA R2310773H
MUZIYONGE LIBERTY R2311313A
MARAMBA TANAKA R2311260G
ZAMBEZI PATIENCE R2311552F
MASHUNGU SHELTON R2312721E