PAMI.highUtilityPattern.basic package

Submodules

PAMI.highUtilityPattern.basic.EFIM module

class PAMI.highUtilityPattern.basic.EFIM.EFIM(iFile, minUtil, sep='\t')

Bases: _utilityPatterns

Description:

EFIM is one of the fastest algorithm to mine High Utility ItemSets from transactional databases.

Reference:

Zida, S., Fournier-Viger, P., Lin, J.CW. et al. EFIM: a fast and memory efficient algorithm for high-utility itemset mining. Knowl Inf Syst 51, 595–625 (2017). https://doi.org/10.1007/s10115-016-0986-0

Parameters:

• iFile – str : Name of the Input file to mine complete set of High Utility patterns

• oFile – str : Name of the output file to store complete set of High Utility patterns

• minUtil – int : The user given minUtil value.

• candidateCount – int Number of candidates specified by user

• maxMemory – int Maximum memory used by this program for running

• sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:

iFilefile

Name of the input file to mine complete set of high utility patterns

oFilefile

Name of the output file to store complete set of high utility patterns

memoryRSSfloat

To store the total amount of RSS memory consumed by the program

startTime:float

To record the start time of the mining process

endTime:float

To record the completion time of the mining process

minUtilint

The user given minUtil value

highUtilityitemSets: map

set of high utility itemSets

candidateCount: int

Number of candidates

utilityBinArrayLU: list

A map to hold the local utility values of the items in database

utilityBinArraySU: list

A map to hold the subtree utility values of the items is database

oldNamesToNewNames: list

A map which contains old names, new names of items as key value pairs

newNamesToOldNames: list

A map which contains new names, old names of items as key value pairs

maxMemory: float

Maximum memory used by this program for running

patternCount: int

Number of HUI’s

itemsToKeep: list

keep only the promising items ie items having local utility values greater than or equal to minUtil

itemsToExplore: list

list of items that have subtreeUtility value greater than or equal to minUtil

:Methods :

mine()

Mining process will start from here

getPatterns()

Complete set of patterns will be retrieved with this function

save(oFile)

Complete set of patterns will be loaded in to a output file

getPatternsAsDataFrame()

Complete set of patterns will be loaded in to a dataframe

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function

getRuntime()

Total amount of runtime taken by the mining process will be retrieved from this function

backTrackingEFIM(transactionsOfP, itemsToKeep, itemsToExplore, prefixLength)

A method to mine the HUIs Recursively

useUtilityBinArraysToCalculateUpperBounds(transactionsPe, j, itemsToKeep)

A method to calculate the sub-tree utility and local utility of all items that can extend itemSet P and e

output(tempPosition, utility)

A method to output a high-utility itemSet to file or memory depending on what the user chose

is_equal(transaction1, transaction2)

A method to Check if two transaction are identical

useUtilityBinArrayToCalculateSubtreeUtilityFirstTime(dataset)

A method to calculate the sub tree utility values for single items

sortDatabase(self, transactions)

A Method to sort transaction

sort_transaction(self, trans1, trans2)

A Method to sort transaction

useUtilityBinArrayToCalculateLocalUtilityFirstTime(self, dataset)

A method to calculate local utility values for single itemsets

Executing the code on terminal:

Format:

(.venv) $ python3 EFIM.py <inputFile> <outputFile> <minUtil> <sep>

Example Usage:

(.venv) $ python3 EFIM sampleTDB.txt output.txt 35

Note

maxMemory will be considered as Maximum memory used by this program for running

Sample run of importing the code:

from PAMI.highUtilityPattern.basic import EFIM as alg

obj=alg.EFIM("input.txt",35)

obj.mine()

Patterns = obj.getPatterns()

print("Total number of high utility Patterns:", len(Patterns))

obj.save("output")

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits:

The complete program was written by pradeep pallikila under the supervision of Professor Rage Uday Kiran.

getMemoryRSS() → float

Total amount of RSS memory consumed by the mining process will be retrieved from this function :return: returning RSS memory consumed by the mining process :rtype: float

getMemoryUSS() → float

Total amount of USS memory consumed by the mining process will be retrieved from this function :return: returning USS memory consumed by the mining process :rtype: float

getPatterns() → dict

Function to send the set of patterns after completion of the mining process :return: returning patterns :rtype: dict

getPatternsAsDataFrame() → _pd.DataFrame

Storing final patterns in a dataframe :return: returning patterns in a dataframe :rtype: pd.DataFrame

getRuntime() → float

Calculating the total amount of runtime taken by the mining process :return: returning total amount of runtime taken by the mining process :rtype: float

mine() → None

Start the EFIM algorithm. :return: None

printResults() → None

This function is used to print the results

save(outFile: str) → None

Complete set of frequent patterns will be loaded in to an output file :param outFile: name of the output file :type outFile: csv file :return: None

sort_transaction(trans1: _Transaction, trans2: _Transaction) → int

A Method to sort transaction :param trans1: the first transaction :type trans1: Trans :param trans2:the second transaction :type trans2: Trans :return: sorted transaction :rtype: int

startMine() → None

Start the EFIM algorithm. :return: None

PAMI.highUtilityPattern.basic.HMiner module

class PAMI.highUtilityPattern.basic.HMiner.HMiner(iFile1, minUtil, sep='\t')

Bases: _utilityPatterns

Description:

High Utility itemSet Mining (HMIER) is an importent algorithm to miner High utility items from the database.

Reference:

Parameters:

• iFile – str : Name of the Input file to mine complete set of High Utility patterns

• oFile – str : Name of the output file to store complete set of High Utility patterns

• minUtil – int : The user given minUtil value.

• minSup – int or float or str : The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float.

• maxPer – float : The user can specify maxPer in count or proportion of database size. If the program detects the data type of maxPer is integer, then it treats maxPer is expressed in count.

• sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:

iFilefile

Name of the input file to mine complete set of frequent patterns

oFilefile

Name of the output file to store complete set of frequent patterns

memoryRSSfloat

To store the total amount of RSS memory consumed by the program

startTime:float

To record the start time of the mining process

endTime:float

To record the completion time of the mining process

minUtilint

The user given minUtil

mapFMAP: list

EUCS map of the FHM algorithm

candidates: int

candidates genetated

huiCnt: int

huis created

neighbors: map

keep track of nighboues of elements

Methods:

mine()

Mining process will start from here

getPatterns()

Complete set of patterns will be retrieved with this function

save(oFile)

Complete set of frequent patterns will be loaded in to a output file

getPatternsAsDataFrame()

Complete set of frequent patterns will be loaded in to a dataframe

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function

getRuntime()

Total amount of runtime taken by the mining process will be retrieved from this function

Explore_SearchTree(prefix, uList, minUtil)

A method to find all high utility itemSets

UpdateCLosed(x, culs, st, excul, newT, ex, ey_ts, length)

A method to update closed values

saveitemSet(prefix, prefixLen, item, utility)

A method to save itemSets

updateElement(z, culs, st, excul, newT, ex, duppos, ey_ts)

A method to updates vales for duplicates

construcCUL(x, culs, st, minUtil, length, exnighbors)

A method to construct CUL’s database

Executing the code on terminal:

Format:

(.venv) $ python3 HMiner.py <inputFile> <outputFile> <minUtil>

Example Usage:

(.venv) $ python3 HMiner.py sampleTDB.txt output.txt 35

Note

minSup will be considered in percentage of database transactions

Sample run of importing the code:

from PAMI.highUtilityPattern.basic import HMiner as alg

obj = alg.HMiner("input.txt",35)

obj.mine()

Patterns = obj.getPatterns()

print("Total number of high utility Patterns:", len(Patterns))

obj.save("output")

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits:

The complete program was written by B.Sai Chitra under the supervision of Professor Rage Uday Kiran.

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function :return: returning RSS memory consumed by the mining process :rtype: float

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function :return: returning USS memory consumed by the mining process :rtype: float

getPatterns()

Function to send the set of frequent patterns after completion of the mining process :return: returning frequent patterns :rtype: dict

getPatternsAsDataFrame()

Storing final frequent patterns in a dataframe :return: returning frequent patterns in a dataframe :rtype: pd.DataFrame

getRuntime()

Calculating the total amount of runtime taken by the mining process :return: returning total amount of runtime taken by the mining process :rtype: float

mine()

Main program to start the operation

printResults()

This function is used to print the results

save(outFile)

Complete set of frequent patterns will be loaded in to an output file :param outFile: name of the output file :type outFile: csv file

startMine()

Main program to start the operation

PAMI.highUtilityPattern.basic.UPGrowth module

class PAMI.highUtilityPattern.basic.UPGrowth.UPGrowth(iFile: str, minUtil: int, sep: str = '\t')

Bases: _utilityPatterns

Description:

UP-Growth is two-phase algorithm to mine High Utility Itemsets from transactional databases.

Reference:

Vincent S. Tseng, Cheng-Wei Wu, Bai-En Shie, and Philip S. Yu. 2010. UP-Growth: an efficient algorithm for high utility itemset mining. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD ‘10). Association for Computing Machinery, New York, NY, USA, 253–262. DOI:https://doi.org/10.1145/1835804.1835839

Parameters:

• iFile – str : Name of the Input file to mine complete set of High Utility patterns

• oFile – str : Name of the output file to store complete set of High Utility patterns

• minUtil – int : The user given minUtil value.

• candidateCount – int Number of candidates specified by user

• maxMemory – int Maximum memory used by this program for running

• sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:

iFilefile

Name of the input file to mine complete set of frequent patterns

oFilefile

Name of the output file to store complete set of frequent patterns

memoryRSSfloat

To store the total amount of RSS memory consumed by the program

startTime:float

To record the start time of the mining process

endTime:float

To record the completion time of the mining process

minUtilint

The user given minUtil

NumberOfNodesint

Total number of nodes generated while building the tree

ParentNumberOfNodesint

Total number of nodes required to build the parent tree

MapItemToMinimumUtilitymap

A map to store the minimum utility of item in the database

phuislist

A list to store the phuis

MapItemToTwumap

A map to store the twu of each item in database

Methods:

mine()

Mining process will start from here

getPatterns()

Complete set of patterns will be retrieved with this function

createLocalTree(tree, item)

A Method to Construct conditional pattern base

UPGrowth( tree, alpha)

A Method to Mine UP Tree recursively

PrintStats()

A Method to print number of phuis

save(oFile)

Complete set of frequent patterns will be loaded in to an output file

getPatternsAsDataFrame()

Complete set of frequent patterns will be loaded in to a dataframe

getMemoryUSS()

Total amount of USS memory consumed by the mining process will be retrieved from this function

getMemoryRSS()

Total amount of RSS memory consumed by the mining process will be retrieved from this function

getRuntime()

Total amount of runtime taken by the mining process will be retrieved from this function

Executing the code on terminal:

Format:

(.venv) $ python3 UPGrowth <inputFile> <outputFile> <Neighbours> <minUtil> <sep>

Example Usage:

(.venv) $ python3 UPGrowth sampleTDB.txt output.txt sampleN.txt 35

Note

maxMemory will be considered as Maximum memory used by this program for running

Sample run of importing the code:

from PAMI.highUtilityPattern.basic import UPGrowth as alg

obj=alg.UPGrowth("input.txt",35)

obj.mine()

highUtilityPattern = obj.getPatterns()

print("Total number of Spatial Frequent Patterns:", len(highUtilityPattern))

obj.save("output")

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits:

The complete program was written by Pradeep pallikila under the supervision of Professor Rage Uday Kiran.

PrintStats() → None

A Method to print number of phuis :return: None

getMemoryRSS() → float

Total amount of RSS memory consumed by the mining process will be retrieved from this function :return: returning RSS memory consumed by the mining process :rtype: float

getMemoryUSS() → float

Total amount of USS memory consumed by the mining process will be retrieved from this function :return: returning USS memory consumed by the mining process :rtype: float

getPatterns() → dict

Function to send the set of frequent patterns after completion of the mining process :return: returning frequent patterns :rtype: dict

getPatternsAsDataFrame() → DataFrame

Storing final frequent patterns in a dataframe :return: returning frequent patterns in a dataframe :rtype: pd.DataFrame

getRuntime() → float

Calculating the total amount of runtime taken by the mining process return: returning total amount of runtime taken by the mining process :rtype: float

mine() → None

Mining process will start from here :return: None

printResults() → None

This function is used to print the results :return: None

save(outFile: str) → None

Complete set of frequent patterns will be loaded in to an output file :param outFile: name of the output file :type outFile: csv file :return: None

startMine() → None

Mining process will start from here :return: None

PAMI.highUtilityPattern.basic.abstract module

PAMI.highUtilityPattern.basic.efimParallel module

class PAMI.highUtilityPattern.basic.efimParallel.efimParallel(iFile, minUtil, sep='\t', threads=1)

Bases: _utilityPatterns

Description:

EFIM is one of the fastest algorithm to mine High Utility ItemSets from transactional databases.

Reference:

Zida, S., Fournier-Viger, P., Lin, J.CW. et al. EFIM: a fast and memory efficient algorithm for high-utility itemset mining. Knowl Inf Syst 51, 595–625 (2017). https://doi.org/10.1007/s10115-016-0986-0

Parameters:

• iFile – str : Name of the Input file to mine complete set of High Utility patterns

• oFile – str : Name of the output file to store complete set of High Utility patterns

• minUtil – int : The user given minUtil value.

• candidateCount – int Number of candidates specified by user

• maxMemory – int Maximum memory used by this program for running

• sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.

Attributes:

inputFile (str):

The input file path.

minUtil (int):

The minimum utility threshold.

sep (str):

The separator used in the input file.

threads (int):

The number of threads to use.

Patterns (dict):

A dictionary containing the discovered patterns.

rename (dict):

A dictionary containing the mapping between the item IDs and their names.

runtime (float):

The runtime of the algorithm in seconds.

memoryRSS (int):

The Resident Set Size (RSS) memory usage of the algorithm in bytes.

memoryUSS (int):

The Unique Set Size (USS) memory usage of the algorithm in bytes.

Methods:

read_file():

Read the input file and return the filtered transactions, primary items, and secondary items.

binarySearch(arr, item):

Perform a binary search on the given array to find the given item.

project(beta, file_data, secondary):

Project the given beta itemset on the given database.

search(collections):

Search for high utility itemsets in the given collections.

mine():

Start the EFIM algorithm.

savePatterns(outputFile):

Save the patterns discovered by the algorithm to an output file.

getPatterns():

Get the patterns discovered by the algorithm.

getRuntime():

Get the runtime of the algorithm.

getMemoryRSS():

Get the Resident Set Size (RSS) memory usage of the algorithm.

getMemoryUSS():

Get the Unique Set Size (USS) memory usage of the algorithm.

printResults():

Print the results of the algorithm.

Executing the code on terminal:

Format:

(.venv) $ python3 efimParallel.py <inputFile> <outputFile> <minUtil> <sep>

Example Usage:

(.venv) $ python3 efimParallel sampleTDB.txt output.txt 35

Note

maxMemory will be considered as Maximum memory used by this program for running

Importing this algorithm into a python program

from PAMI.highUtilityPattern.basic import efimParallel as alg

obj = alg.efimParallel("input.txt",35)

obj.mine()

Patterns = obj.getPatterns()

print("Total number of high utility Patterns:", len(Patterns))

obj.save("output")

memUSS = obj.getMemoryUSS()

print("Total Memory in USS:", memUSS)

memRSS = obj.getMemoryRSS()

print("Total Memory in RSS", memRSS)

run = obj.getRuntime()

print("Total ExecutionTime in seconds:", run)

Credits:

The complete program was written by Tarun Sreepada under the supervision of Professor Rage Uday Kiran.

getMemoryRSS()

Get the Resident Set Size (RSS) memory usage of the algorithm.

Returns:

int: The RSS memory usage in bytes.

getMemoryUSS()

Get the Unique Set Size (USS) memory usage of the algorithm.

Returns:

int: The USS memory usage in bytes.

getPatterns()

Get the patterns discovered by the algorithm.

Returns:

dict: A dictionary containing the discovered patterns.

getPatternsAsDataFrame()

Storing final patterns in a dataframe :return: returning patterns in a dataframe :rtype: pd.DataFrame

getRuntime()

Get the runtime of the algorithm.

Returns:

float: The runtime in seconds.

mine()

Start the EFIM algorithm.

printResults()

This function is used to print the results

save(outFile)

Complete set of frequent patterns will be loaded in to an output file :param outFile: name of the output file :type outFile: csv file

startMine()

Start the EFIM algorithm.

Module contents