MAHAKAL INSTITUTE OF TECHNOLOGY, UJJAIN

Approved By: All India Council of Technical Education (New

Delhi)

DEPARTMENT OF
COMPUTER SCIENCE AND ENGINEERING

LAB MANUAL

Name of Student : …………………..

Name of Lab : Lab Open Elective (Data Mining and

Warehousing)

Subject Code : CS- 705

Branch : Computer Science and Engineering

Year/Sem : IV/VII.

Affiliated to Rajiv Gandhi Prodyogiki Vishwavidyalaya, Bhopal (MP)

 INDEX
S. No. Name of Experiment Date Sign Remark

1. To list all the categorical(or nominal) attributes and the real

valued attributes using Weka mining tool.

2. To identify the rules with some of the important attributes by a)

manually and b) Using Weka .

3. To create a Decision tree by training data set using Weka mining

tool.

4. To find the percentage of examples that are classified correctly

by using the above created decision tree model? ie.. Testing on
the training set.
5 To “Is testing a good idea”.

6 To create a Decision tree by cross validation training data set

using Weka mining tool.

7 Delete one attribute from GUI Explorer and see the effect using
Weka mining tool.
 List of Experiments
Lab Elective V (Data Mining and Warehousing)
(CS- 703 B)

S. No. Name of Experiment

1. To list all the categorical(or nominal) attributes and the real valued attributes using
Weka mining tool.

2. To identify the rules with some of the important attributes by a) manually and b)
Using Weka .

3. To create a Decision tree by training data set using Weka mining tool.

4. To find the percentage of examples that are classified correctly by using the above
created decision tree model? ie.. Testing on the training set.

5 To “Is testing a good idea”.

6 To create a Decision tree by cross validation training data set using Weka mining
tool.

7 Delete one attribute from GUI Explorer and see the effect using Weka mining tool.
 EXPERIMENT-1

OBJECTIVE:
To list all the categorical(or nominal) attributes and the real valued attributes using Weka mining
tool.

PROCEDURE:

INPUT SET:

OUTPUT SET:
EXPECTED VIVA QUESTIONS:
1. What is data warehouse?
2. What is the benefits of data warehouse?
3. What is Fact?

NAME OF FACULTY:
SIGNATURE:
DATE:
 EXPERIMENT-2
OBJECTIVE:
To identify the rules with some of the important attributes by a) manually and b) Using Weka .

THEORY:
Association rule mining is defined as: Let be a set of n binary attributes called items. Let be a set of
transactions called the database. Each transaction in D has a unique transaction ID and contains a
subset of the items in I. A rule is defined as an implication of the form X=>Y where X,Y C I and X
Π Y=Φ . The sets of items (for short itemsets) X and Y are called antecedent (left hand side or
LHS) and consequent (righthandside or RHS) of the rule respectively.
To illustrate the concepts, we use a small example from the supermarket domain.
The set of items is I = {milk,bread,butter,beer} and a small database containing the items (1 codes
presence and 0 absence of an item in a transaction) is shown in the table to the right. An example
rule for the supermarket could be meaning that if milk and bread is bought, customers also buy
butter.
Note: this example is extremely small. In practical applications, a rule needs a support of several
hundred transactions before it can be considered statistically significant, and datasets often contain
thousands or millions of transactions.
To select interesting rules from the set of all possible rules, constraints on various measures of
significance and interest can be used. The best known constraints are minimum thresholds on
support and confidence. The support supp(X) of an itemset X is defined as the proportion of
transactions in the data set which contain the itemset. In the example database, the itemset
{milk,bread} has a support of 2 / 5 = 0.4 since it occurs in 40% of all transactions (2 out of 5
transactions).
The confidence of a rule is defined . For example, the rule has a confidence of 0.2 / 0.4 = 0.5 in the
database, which means that for 50% of the transactions containing milk and bread the rule is
correct. Confidence can be interpreted as an estimate of the probability P(Y | X), the probability of
finding the RHS of the rule in transactions under the condition that these transactions also contain
the LHS .

ALGORITHM:
Association rule mining is to find out association rules that satisfy the predefined minimum support
and confidence from a given database. The problem is usually decomposed into two subproblems.
One is to find those itemsets whose occurrences exceed a predefined threshold in the database;
those itemsets are called frequent or large itemsets. The second problem is to generate association
rules from those large itemsets with the constraints of minimal confidence.
Suppose one of the large itemsets is Lk, Lk = {I1, I2, … , Ik}, association rules with this itemsets
are generated in the following way: the first rule is {I1, I2, … , Ik1} and {Ik}, by checking the
confidence this rule can be determined as interesting or not. Then other rule are generated by
deleting the last items in the antecedent and inserting it to the consequent, further the confidences of
the new rules are checked to determine the interestingness of them. Those processes iterated until
the antecedent becomes empty. Since the second subproblem is quite straight forward, most of the
researches focus on the first subproblem. The Apriori algorithm finds the frequent sets L In
Database D.
· Find frequent set Lk − 1.
· Join Step.
o Ck is generated by joining Lk − 1with itself
· Prune Step.
o Any (k − 1) itemset that is not frequent cannot be a subset of
a frequent k itemset, hence should be removed.
Where · (Ck: Candidate itemset of size k)
· (Lk: frequent itemset of size k)

Apriori Pseudocode
Apriori (T,£)
L<{ Large 1itemsets that appear in more than transactions }
K<2
while L(k1)≠ Φ
C(k)<Generate( Lk − 1)
for transactions t € T
C(t)Subset(Ck,t)
for candidates c € C(t)
count[c]<count[ c]+1
L(k)<{ c € C(k)| count[c] ≥
£
K<K+ 1
return Ụ L(k) k

PROCEDURE:

INPUT SET:

OUTPUT SET:
EXPECTED VIVA QUESTIONS:
1. What is support and confidence?
2. What is association rule?
3. How apriori algorithm is used?

NAME OF FACULTY:
SIGNATURE:
DATE:
 EXPERIMENT-3
OBJECTIVE:
To create a Decision tree by training data set using Weka mining tool.

THEORY:
Classification is a data mining function that assigns items in a collection to target categories or
classes. The goal of classification is to accurately predict the target class for each case in the data.
For example, a classification model could be used to identify loan applicants as low, medium, or
high credit risks.
A classification task begins with a data set in which the class assignments are known. For example,
a classification model that predicts credit risk could be developed based on observed data for many
loan applicants over a period of time.
In addition to the historical credit rating, the data might track employment history, home ownership
or rental, years of residence, number and type of investments, and so on. Credit rating would be the
target, the other attributes would be the predictors, and the data for each customer would constitute
a case.
Classifications are discrete and do not imply order. Continuous, floating point values would
indicate a numerical, rather than a categorical, target. A predictive model with a numerical target
uses a regression algorithm, not a classification algorithm.
The simplest type of classification problem is binary classification. In binary classification, the
target attribute has only two possible values: for example, high credit rating or low credit rating.
Multiclass targets have more than two values: for example, low, medium, high, or unknown credit
rating.
In the model build (training) process, a classification algorithm finds relationships between the
values of the predictors and the values of the target. Different classification algorithms use different
techniques for finding relationships. These relationships are summarized in a model, which can then
be applied to a different data set in which the class assignments are unknown.
Classification models are tested by comparing the predicted values to known target values in a set
of test data. The historical data for a classification project is typically divided into two data sets: one
for building the model; the other for testing the model.
Scoring a classification model results in class assignments and probabilities for each case. For
example, a model that classifies customers as low, medium, or high value would also predict the
probability of each classification for each customer.
Classification has many applications in customer segmentation, business modeling, marketing,
credit analysis, and biomedical and drug response modeling.

PROCEDURE:

INPUT SET:

OUTPUT SET:
The decision tree constructed by using the implemented C4.5 algorithm

EXPECTED VIVA QUESTIONS:

1. What is decision tree?
2. What is classification?
3. How decision tree helps in data classification?

NAME OF FACULTY:
SIGNATURE:
DATE:
 EXPERIMENT-4

OBJECTIVE:
To find the percentage of examples that are classified correctly by using the above created decision
tree model? ie.. Testing on the training set.

THEORY:
Naive Bayes classifier assumes that the presence (or absence) of a particular feature of a class is
unrelated to the presence (or absence) of any other feature. For example, a fruit may be considered
to be an apple if it is red, round, and about 4" in diameter. Even though these features depend on the
existence of the other features, a naive Bayes classifier considers all of these properties to
independently contribute to the probability that this fruit is an apple.
An advantage of the naive Bayes classifier is that it requires a small amount of training data to
estimate the parameters (means and variances of the variables) necessary for classification. Because
independent variables are assumed, only the variances of the variables for each class need to be
determined and not the entirecovariance matrix The naive Bayes probabilistic model :
The probability model for a classifier is a conditional model P(C|F1..................Fn) over a dependent
class variable C with a small number of outcomes or classes, conditional on several feature
variables F1 through Fn. The problem is that if the number of features n is large or when a feature
can take on a large number of values, then basing such a model on probability tables is infeasible.
We therefore reformulate the model to make it more tractable.
Using Bayes' theorem, we write P(C|F1...............Fn)=[{p(C)p(F1..................Fn|C)}/p(F1,. Fn)]

In plain English the above equation can be written

as Posterior= [(prior *likehood)/evidence]
In practice we are only interested in the numerator of that fraction, since the denominator does not
depend on C and the values of the features Fi are given, so that the denominator is effectively
constant. The numerator is equivalent to the joint probability model p(C,F1. Fn) which can be
rewritten as follows, using repeated applications of the definition of conditional probability:
p(C,F1........Fn) =p(C) p(F1............Fn|C) =p(C)p(F1|C) p(F2........Fn|C,F1,F2)
=p(C)p(F1|C) p(F2|C,F1)p(F3........Fn|C,F1,F2)
= p(C)p(F1|C) p(F2|C,F1)p(F3.........Fn|C,F1,F2)......p(Fn|C,F1,F2,F3........Fn1)
Now the "naive" conditional independence assumptions come into play: assume that each feature Fi
is conditionally independent of every other feature Fj for j≠i .
This means that p(Fi|C,Fj)=p(Fi|C)
and so the joint model can be expressed as p(C,F1,.......Fn)=p(C)p(F1|C)p(F2|C)...........
=p(C)π p(Fi|C)
This means that under the above independence assumptions, the conditional distribution over the
class variable C can be expressed like this:
p(C|F1.........Fn)= p(C) πp(Fi|C)
Z
where Z is a scaling factor dependent only on F1. Fn, i.e., a constant if the values of the feature
variables are known.
Models of this form are much more manageable, since they factor into a so called class prior p(C)
and independent probability distributions p(Fi|C). If there are k classes and if a model for eachp(Fi|
C=c) can be expressed in terms of r parameters, then the corresponding naive Bayes model has (k −
1) + n r k parameters. In practice, often k = 2 (binary classification) and r = 1 (Bernoulli variables as
features) are common, and so the total number of parameters of the naive Bayes model is 2n + 1,
where n is the number of binary features used for prediction
P(h/D)= P(D/h) P(h) P(D)
• P(h) : Prior probability of hypothesis h
• P(D) : Prior probability of training data D
• P(h/D) : Probability of h given D
• P(D/h) : Probability of D given h
Naïve Bayes Classifier :
Derivation
• D : Set of tuples
– Each Tuple is an ‘n’ dimensional attribute vector
– X : (x1,x2,x3,…. xn)
• Let there me ‘m’ Classes : C1,C2,C3…Cm
• NB classifier predicts X belongs to Class Ci iff
– P (Ci/X) > P(Cj/X) for 1<= j <= m , j <> i
• Maximum Posteriori Hypothesis
– P(Ci/X) = P(X/Ci) P(Ci) / P(X)
– Maximize P(X/Ci) P(Ci) as P(X) is constant
Naïve Bayes Classifier : Derivation
• With many attributes, it is computationally expensive to evaluate P(X/Ci)
• Naïve Assumption of “class conditional independence”
• P(X/Ci) = n P( xk/ Ci)
k=1
• P(X/Ci) = P(x1/Ci) * P(x2/Ci) *…* P(xn/ Ci)

PROCEDURE:

INPUT SET:

OUTPUT SET:
=== Evaluation on training set ===
=== Summary ===

Correctly Classified Instances 554 92.3333 %

Incorrectly Classified Instances 46 7.6667 %
Kappa statistic 0.845
Mean absolute error 0.1389
Root mean squared error 0.2636
Relative absolute error 27.9979 %
Root relative squared error 52.9137 %
Total Number of Instances 600

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure ROC Area Class

0.894 0.052 0.935 0.894 0.914 0.936 YES
0.948 0.106 0.914 0.948 0.931 0.936 NO
Weighted Avg. 0.923 0.081 0.924 0.923 0.923 0.936

=== Confusion Matrix ===

 a b <-- classified as
245 29 | a = YES
17 309 | b = NO

EXPECTED VIVA QUESTIONS:

1. What is naïve bayes classifier?
2. What are the main advantages of bayes classifier?

NAME OF FACULTY:
SIGNATURE:
DATE:
 EXPERIMENT-5
OBJECTIVE:
To Is testing a good idea”.

PROCEDURE:
INPUT SET
OUTPUT SET:
This can be experienced by the different problem solutions while doing practice.

The important numbers to focus on here are the numbers next to the "Correctly Classified
Instances" (92.3 percent) and the "Incorrectly Classified Instances" (7.6 percent). Other important
numbers are in the "ROC Area" column, in the first row (the 0.936); Finally, in the "Confusion
Matrix," it shows the number of false positives and false negatives. The false positives are 29, and
the false negatives are 17 in this matrix. Based on our accuracy rate of 92.3 percent, we say that
upon initial analysis, this is a good model.One final step to validating our classification tree, which
is to run our test set through the model and ensure that accuracy of the model

Comparing the "Correctly Classified Instances" from this test set with the "Correctly Classified Instances"
from the training set, we see the accuracy of the model , which indicates that the model will not break down
with unknown data, or when future data is applied to it.

EXPECTED VIVA QUESTIONS:

1. What is classification accuracy?

NAME OF FACULTY:
SIGNATURE:
DATE:
 EXPERIMENT-6
OBJECTIVE:
To create a Decision tree by cross validation training data set using Weka mining tool.

THEORY:
Decision tree learning, used in data mining and machine learning, uses a decision tree as a
predictive model which maps observations about an item to conclusions about the item's target
value In these tree structures, leaves represent classifications and branches represent conjunctions of
features that lead to those classifications. In decision analysis, a decision tree can be used to
visually and explicitly represent decisions and decision making. In data mining, a decision tree
describes data but not decisions; rather the resulting classification tree can be an input for decision
making. This page deals with decision trees in data mining.
Decision tree learning is a common method used in data mining. The goal is to create a model that
predicts the value of a target variable based on several input variables. Each interior node
corresponds to one of the input variables; there are edges to children for each of the possible values
of that input variable. Each leaf represents a value of the target variable given the values of the
input variables represented by the path from the root to the leaf.
A tree can be "learned" by splitting the source set into subsets based on an attribute value test. This
process is repeated on each derived subset in a recursive manner called recursive partitioning. The
recursion is completed when the subset at a node all has the same value of the target variable, or
when splitting no longer adds value to the predictions.
In data mining, trees can be described also as the combination of mathematical and computational
techniques to aid the description, categorization and generalization of a given set of data.
Data comes in records of the form:
(x, y) = (x1, x2, x3..., xk, y)
The dependent variable, Y, is the target variable that we are trying to understand, classify or
generalize. The vector x is comprised of the input variables, x1, x2, x3 etc., that are used for that
task.
PROCEDURE:

INPUT SET:
OUTPUT SET:

=== Stratified cross-validation

===
=== Summary ===

Correctly Classified Instances 539 89.8333 %

Incorrectly Classified Instances 61 10.1667 %
Kappa statistic 0.7942
Mean absolute error 0.167
Root mean squared error 0.305
Relative absolute error 33.6511 %
Root relative squared error 61.2344 %
Total Number of Instances 600

=== Detailed Accuracy By Class ===

 TP Rate FP Rate Precision Recall F-Measure ROC Area Class
0.861 0.071 0.911 0.861 0.886 0.883 YES
0.929 0.139 0.889 0.929 0.909 0.883 NO
Weighted Avg. 0.898 0.108 0.899 0.898 0.898 0.883

=== Confusion Matrix ===

a b <-- classified as
236 38 | a = YES
23 303 | b = NO

EXPECTED VIVA QUESTIONS:

1. What is training data set?
2. What is testing data set?

NAME OF FACULTY:
SIGNATURE:
DATE:
 EXPERIMENT-7

OBJECTIVE:
Delete one attribute from GUI Explorer and see the effect using Weka mining tool.

PROCEDURE:
INPUT SET:
OUTPUT SET
EXPECTED VIVA QUESTIONS:
1. What is nominal and numeric attributes?
2. Which type of test are performed using weka tool?

NAME OF FACULTY:
SIGNATURE:
DATE: