Mining Fuzzy Periodic-Frequent Patterns in Fuzzy Temporal Databases

1. What is fuzzy periodic-frequent pattern mining?

The fuzzy frequent pattern model aims to discover frequently occurring patterns in a fuzzy transactional database. This model implicitly assumes that the temporal occurrence information of the transactions, if any, will not affect the interestingness of a pattern to the user. However, this is seldom not the case in many real-world applications. In many applications, the user may consider a pattern occurring at periodic intervals to be more interesting than a pattern occur at aperiodic (or irregular) intervals. With this motivation, the fuzzy frequent pattern model was generalized to discover fuzzy periodic-frequent patterns in a fuzzy temporal database.

The basic model of fuzzy periodic-frequent pattern aims to discover all patterns in a fuzzy temporal database that satisfy the user-specified minimum support (minSup) constraint and maximum periodicity (maxPer) constraints. The minSup controls the minimum number of transactions that a pattern must appear in a database. The maxPer controls the maximum time interval within which a pattern must reappear in the database.

Reference: R. U. Kiran et al., “Discovering Fuzzy Periodic-Frequent Patterns in Quantitative Temporal Databases,” 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 2020, pp. 1-8, doi: 10.1109/FUZZ48607.2020.9177579. Link

2. What is a fuzzy temporal database?

A fuzzy temporal database is a collection of transactions at a particular timestamp, where each transaction contains a timestamp, set of items, and its fuzzy values respectively.
A hypothetical fuzzy temporal database containing the items a, b, c, d, e, f, and g as shown below

TS	Transactions
1	(a.L,0.2) (b.M,0.3) (c.H,0.1) (g.M,0.1)
2	(b.M,0.3) (c.H,0.2) (d.L,0.3) (e.H,0.2)
3	(a.L,0.2) (b.M,0.1) (c.H,0.3) (d.L,0.4)
4	(a.L,0.3) (c.H,0.2) (d.L,0.1) (f.M,0.2)
5	(a.L,0.3) (b.M,0.1) (c.H,0.2) (d.L,0.1) (g.M,0.2)
6	(c.H,0.2) (d.L,0.2) (e.H,0.3) (f.M,0.1)
7	(a.L,0.2) (b.M,0.1) (c.H,0.1) (d.L,0.2)
8	(a.L,0.1) (e.H,0.2) (f.M,0.2)
9	(a.L,0.2) (b.M,0.2) (c.H,0.4) (d.L,0.2)
10	(b.M,0.3) (c.H,0.2) (d.L,0.2) (e.H,0.2)

Note: Duplicate items must not exist in a transaction.

3. What is acceptable format of a fuzzy temporal database in PAMI?

Each row in a fuzzy temporal database must contain timeStamp, list of fuzzy items, colon as a seperator, and their list of fuzzy values.

• Colon ‘:’ must be used as a seperator to distinguish fuzzy items and their fuzzy values. This seperator is fixed and cannot be over-written by the users.
• fuzzy items and fuzzy values have to be seperated from each other with a delimitar. The default delimitar is ‘tab space,’ however, users can over-ride this default seperator.
• There is no need for a seperator to distinct timestamp and fuzzy items. The first element in a transaction is considered as a timestamp by default.

A sample fuzzy temporal database file, say fuzzyTemporalDatabase.txt, is provided below:

1 a.L b.M c.H g.M:0.2 0.3 0.1 0.1
2 b.M c.H d.L e.H:0.13 0.2 0.3 0.2
3 a.L b.M c.H d.L:0.2 0.1 0.3 0.4
4 a.L c.H d.L f.M:0.3 0.2 0.1 0.2
5 a.L b.M c.H d.L g.M:0.3 0.1 0.2 0.1 0.2
6 c.H d.L e.H f.M:0.2 0.2 0.3 0.1
7 a.L b.M c.H d.L:0.2 0.1 0.1 0.2
8 a.L e.H f.M:0.1 0.2 0.2
9 a.L b.M c.H d.H:0.2 0.2 0.4 0.2
10 b.M c.H d.L e.H:0.3 0.2 0.2 0.2

For more information on how to create a fuzzy temporal database from a quantitative (or utility) temporal database, please refer to the manual utility2FuzzyDB.pdf

4. What is the need for understanding the statistics of a database?

The performance of a pattern mining algorithm primarily depends on the satistical nature of a database. Thus, it is important to know the following details of a database:

• Total number of transactions (Database size)
• Total number of unique items in database
• Minimum lenth of transaction that existed in database
• Average length of all transactions that exists in database
• Maximum length of transaction that existed in database
• Standard deviation of transaction length
• Variance in transaction length
• Sparsity of database

The below sample code prints the statistical details of a database.

import PAMI.extras.dbStats.FuzzyDatabase as stats

obj = stats.FuzzyDatabase('fuzzyTemporalDatabase.txt', ' ')
obj.run()
obj.printStats() 

5. What are the input parameters to be specified for a fuzzy periodic-frequent pattern mining algorithm?

• Fuzzy Temporal database
  Acceptable formats:

  • String : E.g., ‘fuzzyDatabase.txt’
  • URL : E.g., https://u-aizu.ac.jp/~udayrage/datasets/fuzzyDatabases/fuzzyTemporal_T10I4D100K.csv
  • DataFrame with the header titled ‘Transactions’ and ‘fuzzyValues’

• minSup
  specified in

  • count (beween 0 to length of a database) or
  • [0, 1]

• maxPer
  specified in

  • count (beween 0 to length of a database) or
  • [0, 1]

• seperator
  default seperator is ‘\t’ (tab space)

6. How to store the output of a fuzzy periodic-frequent patternn mining algorithm?

The patterns discovered by a fuzzy periodic frequent pattern mining algorithm can be saved into a file or a data frame.

7. How to run the fuzzy periodic-frequent pattern algorithm in a terminal?

• Download the PAMI source code from github.
• Unzip the PAMI source code folder.
• Enter into fuzzyPeriodicFrequentPattern folder.

foo@bar: cd PAMI/fuzzyPeriodicFrequentPattern/basic

• Execute the python program on the terminal using the following syntax:

foo@bar: python3 algorithmName.py inputFile outputFile minSup maxPer seperator

Example: python3 FPFPMiner.py inputFile.txt outputFile.txt 3 4 ' '

8. How to execute a Fuzzy Periodic Frequent mining algorithm in a Jupyter Notebook?

• Install the PAMI package from the PYPI repository by executing the following command: pip3 install PAMI
• Run the below sample code by making necessary changes

import PAMI.fuzzyPeriodicFrequentPattern.basic.FPFPMiner as alg 

iFile = 'sampleFuzzyTemporal.txt'      #specify the input temporal database 
minSup = 0.9                      #specify the minSup value
seperator = ' '                 #specify the seperator. Default seperator is tab space.
maxPer = 3                      #specify the maxPer value
oFile = 'FuzzyPeriodicPatterns.txt'   #specify the output file name


obj = alg.FPFPMiner(iFile, minSup, maxPer, seperator) 
obj.mine() 
obj.save(oFile)           #(to store the patterns in file) 
Df = obj.getPatternsAsDataFrame() #(to store the patterns in dataframe) 
obj.printResults()                  #(to print the no of patterns, runtime and memory consumption details)

Total number of Fuzzy Periodic-Frequent Patterns: 7
Total Memory in USS: 81637376
Total Memory in RSS 119406592
Total ExecutionTime in seconds: 0.0004661083221435547

The FuzzyPeriodicPatterns.txt file contains the following patterns (format: pattern:support:periodicity):!cat FuzzyPeriodicPatterns.txt

!cat 'FuzzyPeriodicPatterns.txt'
#format: fuzzyPeriodicPattern:support:periodicity

b.M:1.23:2 
b.M	c.H:0.9299999999999999:2 
d.L:1.4999999999999998:1 
d.L	c.H:1.2:3 
a.L:1.5:2 
a.L	c.H:1.0:2 
c.H:1.9000000000000001:2 

The dataframe contains the following information:

Df

	Patterns	Support	Periodicity
0	b.M	1.23	2
1	b.M c.H	0.93	2
2	d.L	1.50	1
3	d.L c.H	1.20	3
4	a.L	1.50	2
5	a.L c.H	1.00	2
6	c.H	1.90	2